Don't use treated wood for the reason you stated. It can kill the bees.

You can build you own house by drilling a block of wood. That challenge is to find a drill bit that makes a hole 6 inches deep. Try an online search with "build mason bee house site:edu" but without the quote marks.

Another problem w/ wood block housing is that the artificial conditions tend to encourage damaging populations of mites to increase. So, if you use drilled blocks, discard the old ones every 3 or 4 years and replace with new ones

Or purchase a ready made nest -- several different styles are available.

Only 6 inches I've already got those is the diameter also a factor I'm told that when the local horticulture society has bee houses for sale at the annual sale

Put homemade paper nesting tubes in a wooden box, making your own bee nest. Some people use quart and half-gallon milk cartons (cardboard, not plastic). Put in place in spring, facing south or east so they get the morning sun, which helps them get started in the morning. These bees start flying and mating when the outdoor temp reaches 54F, usually in March, but your area may vary. They have a very short life-span/breeding cycle, so you have to be ready. It all starts with the temperature.

Find or make a small wooden box that is 6 to 8" deep. Maybe 4 or 5" wide/square (exactness not required).

Buy a 5/16" wooden dowel from a hardware or craft store. Cut it about 10-12" long.

Use brown paper bags or brown wrapping paper on a roll. (They have this at the Dollar Tree, if you have one near.) Cut the brown paper into rectangles about 6" long and 8" wide. Roll the paper around the dowel and fasten with a glue stick (just down the edge, not all around). If you have some drinking straws lying around, cut them in 1-1/2" lengths, then slit them once lengthwide. Open them up and use them as temporary 'clamps' on the tubes until the glue dries.

When you've got a bunch of the tubes finished, drop them into your wooden box. Pack them in as snugly as you can. If your bee box is long enough, you can roll the tubes so they're 8" long. But you can't let them hang out in the rain, so if your box is shorter, roll them so they're 6" long. Less than 6" is not good, as most of the young hatchlings will be male.

The female bees need three things: the nesting spots (the bee tubes), a water source (a large rock sitting in a container of water so it forms an island that the bees can land on and drink from), and a source of natural clay to seal the ends of the tubes, like from a riverbed, or the kind that people buy to make kiln-fired pots -- terra cotta is good, and common in many areas. The clay must be kept moist and accessible to the bees w/o allowing them to drown in it.