Possible. Especially since they followed you that far.
Do you have protective gear? Have you thought of requeening the hive? That might make them more docile.

This is going to be one of those kind of questions where people not directly familiar with the situation can't really give an exacting answer. All we can do is, more-or-less point to some likely answers - some of which, maybe you know and some perhaps you don't (considering you're describing yourself as 'New-bee').
Worse case first (and unfortunately for you, maybe the more likely situation). It's entirely possible they're Africanized. If this is the case, the best solution will be to spray them with a soapy-water solution and start over with known genetics. You can do a Gardenweb search on my name for posts where I describe how to do this. I say this is a likely possibility because you're located in an AHB infested region and the quickness with which a feral swarm "found" your TBH. Initially, they weren't overly defensive but once the colony was established and the queen perhaps started laying, they became very defensive. This description all points to the likelihood of AHB.
Now a couple other possibilities; if they're not AHB, but just a hot hive due to a bad crossbred strain, then it may be worth your time to locate the queen, kill her and requeen (again, with a queen of known genetics).
And the last couple of possibilities. Handling and manipulating a TBH requires a little more "finesse" than a Langstroth-type hive. More care has to be taken so as to not crush worker bees between the top bars. If you're not careful, it's easy to trap and squash a few workers, releasing more alarm pheromone than you'd experience with a "normal hive". And last (that I can think of), weather conditions can play a significant role in how the bees react to being disturbed. Was the day very humid? They really don't like to be bothered on a high humidity day and they'll remind you of it pretty quickly. There's some point at which a hot temperature and high humidity just doesn't sit well with them so it's best to choose a different day to turn their world upside down by moving combs around and pulling combs out for inspection, etc.
If you have access to an experienced beekeeper in your area and he or she is willing to accompany you on a hive inspection (they may welcome an opportunity to see a TBH), then they should also be able to tell you fairly quickly if the hive is likely AHB or not. Good luck to you!

Did you smoke the bees before opening?