I can think of two scenarios where this might occur, assuming you were not sitting on a female's nest. You may have been wearing red or bright pink clothing, and she/he may have wanted to get close to you to see if you were a tasty nectar source. Or if you have had feeders up in the past, she/he may have been "reminding" you to hang it up again!...some resident hummers can be quite insistent.

If neither of these two scenarios seem likely, then it was probably just having fun, or squabbling with another hummer behind you that you didn't see. But don't worry--you have absolutely nothing to fear from hummingbirds! They are as harmless a creature as can be. You may want to hang some feeders to keep them (and you!) happy.

Thanks for the follow-up! I was actually wearing a white tank top and black shorts, about as neutral as they come, haha:-)! I didn't see him today but I've been looking for him! I'm really not affraid of hummingbirds and I actually do like them quite a bit but those long, sharp looking little beaks can be quite intimidating! Just kidding! I've never hung feeders at our house before because we just moved here about this time last year but the previous owners might have done so, which would explain why they might be coming back to look for food. I will be hanging some feeders in the near future as soon as I can get to Lowes to purchase some! Thanks again!

my guess she has a nest near where you were sitting

I had one who would get right in my face, if I didn't have my window feeder up. He didn't like the other kind. He'd flap away at the window and then a foot in front of my eyes. I'd put it right up then. They communicate very well. Good chance she is nesting near by and is use to feeders in your yard. Probably was her turf years before yours, and checking you out thoroughly.

I've had hummingbirds come up to me to inspect my silver jewelry, and one even tapped at my teeth as I laughed at him. Coolaces may be right, there could well be a nest near your patio.
Lorna