Do NOT do this.
Amazon is very, very, very rigid and anal about having shipping and tracking times/numbers, etc. Amazon uses how quickly you get it shipped, how quickly it arrives to the customer, in your seller metrics. A hand delivered item will not show up as shipped or delivered and you'll get a big negative for selling metric in that area.
In your Amazon Seller Central page, click on the Performance tab at the top of the page. Some of the metrics logged: Defect and Cancellation Rate, On-Time Delivery Rate (the customer cannot put in that the item was delivered on their end....only a shipping number via UPS/Fedex/USPS will do), Late Shipment Rate, etc.
Do NOT take advice from anyone who does not sell on Amazon. Amazon takes their metrics very seriously and will delist your seller's account for any perceived lack of taking their policies seriously.
Go ask in the seller's forums about this......or email seller services at Amazon. You'll get told explicitly to ship it with tracking.
Besides, you got an allowance for shipping costs.
Tracking is not required for shipping with Amazon. It is however highly recommended and it is part of their seller metrics. You could if you wanted to mark the item as shipped and hand deliver it and Amazon would have no issue with that.
Heck you can even buy Media Mail with no delivery confirmation straight from Amazon.com when selling a media item (though for the 20 cents extra you'd be a little crazy not to get delivery confirmation)
The only metric that would hit is the "Packages with tracking info" which has a target of >98% (the "Delivered on Time" metric is only for packages with tracking info that gets updated, so if you ship without tracking it is not included in that particular metric).
And in regards to if a customer complains that they never received it, it wouldn't make a difference if there was tracking or not. Amazon does not consider tracking info the same as proof that an item was delivered. Only signature confirmation will do that. So a buyer can just say they never received an item, and even if UPS/Fedex/USPS say it is delivered, unless they have a signature, it never happened. (thankfully most people are honest most of the time)
All that to say, unless there is a massive savings (like if you are shipping a 50LB box), just pop it in the mail and do it the non-creepy way, rates will probably be dirt cheap within the same town.
Edit:
Here's a recent thread on Amazon's sellers forum about this issue. Amazon's official response in that thread is
Although it may seem to be easier or more convenient to simply deliver an item personally in a situation such as this, we would still encourage you to ship your item rather than delivering personally, as your buyer did pay shipping costs in this order. Additionally, for the security of both parties in a transaction, we ask you to ship your items rather than personally deliver them.
While we do not ask sellers to write to us every time they ship an item, on occasion we may ask sellers for proof of shipment. You will want to have shipment information available if this should occur at some point.