In that scenario the answer is yes. However, since he threatened you with Negative FB you can contact eBay and have it removed because that is a policy violation.
Feedback manipulation policy | eBay
Did you refund out of fear of the Negative Feedback? Just because something is late on delivery does not mean you have to refund a Buyer.
Regards,
Mr. Lincoln - Community Mentor
Message 3 of 3