You never want to seal any carpenter bee gaps until the late fall. Leave them all open until then. Otherwise if you seal them in they'll burrow another one to get out, especially if you didn't spray them with something that has an immediate knock down effect.

They first appear in April or May, whenever temps go above 70 degrees. This was a long and really cold Winter so I know I didn't see any until closer to June. They dig tunnels and lay 5-6 eggs in each one for the first few weeks. Then you'll see them occasionally throughout the rest of the summer in the early mornings or evenings when they are leaving early to get food and then bring it back at the end of the day to store in the tunnel for their larvae. Activity will tend to die down after a couple months when the original generation of adults dies off. Then in the fall the larvae will come out to feed one last time and then go back in the hole to hibernate through Winter.

You want to leave the holes un-sealed until then because you are going to treat them with a dust insecticide such as 7dust from Home Depot. Puff it into each hole you can find and then just leave it alone. This is a way more important step than attacking one bee at a time. The adults will pick it up and carry it inside with them and then the larvae will touch it and die when they come out in the fall. Patch the holes in September or October when you know for sure that everything in there is dead.

Next year get some traps and put those up, and have an exterminator put a 90 day residual strength repellent spray on all exposed exterior wood. Be prepared to still see a few of them burrow fresh holes, which you can then deal with using more of the dust. If you follow this routine they won't go away over night but activity should die down to little or nothing after 2-3 seasons.

If you are around an area where there is a heavy population of them then putting up traps and have your exterior wood treated by an exterminator should be the routine every year as soon as temps go above 70 degrees.