Hi Goldenpond,
What's the hole closed off with? Does it look like mud?

Mud wasps do things like that.

Not really like mud more like homemde paper made of organic matter

Paper wasp?

Could you post a picture of it? Is it just a sheet-like thing over the opening? Wasps can do alot of stuff like that. I'm thinking it has to be an insect, since those are probably the only small enough things to get in and out elsewhere in the house.

If you can, get a pair of binoculars and try to get a good description of the insects. How big is this birdhouse??

There's always an upside to everything...you have that many more pollinators flitting about your wonderful-sounding gardens!! If the house is up and out of the way of anybody unknowingly disturbing them, maybe you could let them "have" this one and hang others in different locations for the real birds. However....Africanized bees. There's a different situation.
We had problems with honeybees taking over our wood duck boxes. The combs would get so big they would start to protrude from the hole! Of course, this was happening at about the same time when the mites were killing off all the native bees, so we let those bees buzz and simply made new ones for the ducks and squirrels and owls to use. Never knew a duck would fight a squirrel.

Goldenpond you will want to get rid of the bees if you want the songbirds to nest. Song birds will actually abandon their babies if wasps or bees start bilding in the box. It is easy to keep this from happening by puting a thin film of vaseline or bar soap on the inside of the roof and about 1/4 inch down. Anyone offering nest boxes should do some homework on learning about proper mounting and safety for the birds. Sialis.org is a great site to learn these things.