Those holes are part of the vacuum break and should not leak under pressure. If they do, the vacuum breaker is damaged, likely from not being drained and freezing, or hardness buildup. In either case, remove the vacuum breaker and replace it - it should unscrew with a little persuasion.

What purpose does a vacuum breaker serve?

Vacuum breakers prevent water from flowing back into the pipes.

http://structuretech1.com/why-do-i-need-a-vacuum-breaker/ 

Lazy G has the right info. Look for a ser screw that prevent's breaker accidently coming off when hose is removed. Lots of times the screw head is broken off and screw must be drilled out before the breaker can be "persuaded".

Everyone, thank you for your help. I'll have to look for that screw as I've tried to remove the breaker and it wouldn't even slightly budge. Would a vacuum breaker tend to be used in a northern climate where freezing pipes could occur in the winter. I live in Florida so wouldn't think that water flowing back into the pipe would be an issue, maybe I'm still not totally understanding the significance. We have two outside faucets and only one has the vacuum breaker.

Though a breaker "MIGHT HELP" prevent freezing if a hose is left on a frostproof,that's not it's intended purpose. If for instance a hose was running water into a livestock watering tank or flower bed when the city interupted service by turning water supply off to a whole neighborhood. There's several ways water flow could reverse and suck water out of the tank or garden farrow. The breaker will prevent siphoning. Most places require all outdoor fixtures,including irrigation systems to have anti-siphon devices.

When water is turned off for maintenance, in the house or elsewhere in the public water system, siphoning can create a negative pressure (vacuum). If a hose is connected to an outdoor faucet which is turned on, it can cause dirty water to be sucked into the plumbing, thus contaminating the entire system. A vacuum break causes air to be drawn in instead.

Thanks so much for the clarification randy427. So I probably should replace the vacuum breaker to play it safe. Now to get the darn thing off!

Where would I find a replacement?

Google "vacuum breaker".