Perhaps it is pollen.

But yes, a photo could help.

Pollen if dustlike. Hazelnuts and alders are all pollinating now, and maples soon.

Lichen is another possibility. Lichen grows on anything that holds still for a couple of years, and comes in many colors, including yellow.

Lichen clings to the bark, where pollen is a very fine dust.

Sounds like maybe it's lichen. Will it harm anything? Should I do something to get rid of it?

no, and no.

Lichens are very interesting and beautiful creatures. They are not plants, nor animals, nor fungi. They are a combination of a fungus with a photosynthesizing alga. They produce their own food, through the alga part, and pick up nutrients from the air and water and collected dirt of the surface they're living on, through the fungus part. They are not parasitic and do not have roots.

To me lichens are as beautiful as wildflowers, and a lot more common around here! I take pictures of different colors and patterns in lichens like other people take pictures of flowers.

I have a friend who collects lichens to use for making dye for the yarn she spins and knits into hats and stuff. And, another friend who surveys lichens to monitor air quality. They're very sensitive to air pollution. Couple days ago I was listening to an archaeologist talk about dating rock structures through the size of the lichens growing on the rocks. She compares the size of the lichen clusters on dated gravestones and USGS survey markers, to the size of the lichen clusters on the rock cairn she's dating.

Lichens are great!

The original post said:
"the same stuff is also on the exposed parts of our deck."

Lichen is possible there too. But pollen is more likely.

That said, please describe the yellow stuff.

Is it powder-like?
Is it solid pieces, perhaps with 3 dimensions?

As was said -- yes, a photo will help.

Gosh, thanks for all the help, but how do I put a photo in with my message? Sorry... :O)

You have to have a web space you can upload photos to, then you put the link to it in the Optional URL spot.

But, there's no reason for lichen to NOT grow on a deck, in the places that don't get foot traffic.

And, pollen would have washed away by now with the rain. Lichen stays put, once it's there.

Lichens are substantial and adherent, pollen grains are like dust.

Definitely sounds like Lichens. I guess they are doing no harm so I won't mess with them. Thanks for all the help, everyone. :o)