Also, I should mention, the red color you see in the background is from the red curtains in the bathroom, usually they are open. I'm also going to be using different furniture in this room, have a white crib, and changer, and I plan on making curtains, one I figure out the colors of everything else...

I don't have specific color recommendations for low VOC, but I think light grey would keep it airy and gender neutral. Super cute "theme".

Another decor idea - a fabric flag bunting in the bedding colors. Tons of folks on Etsy sell them.

Benjamin Moore Hawthorne yellow
Muralo chamonix
Muralo green glimmer

I would also agree with light grey. CIL Whiter Shade would look good in this room (I'm sure you could get it in a low box paint). I think an accent wall would make the space a bit more interesting. Yellow goes well with grey so I would do either the window wall or the fireplace wall in Benjamin Moore Conchord Ivory (it's a really pretty yellow).

The long wall on the right side of the photo is Whiter Shade and the yellow stenciled wall is Conchord Ivory

You could find either some darker yellow, darker grey or white curtains (white would look crisp and clean) for a more monochrome look or pick a colour from your sheets. A mint green or light turquoise would look pretty as well.

If you did the window wall as an accent you could do thick white and yellow stripes and then keep the curtains solid. If you go for solid colours then you could pick a pattern for the fabric or go solid

Here they are in a better format BM Conchord Ivory, CIL whiter shade

I would go for a soft yellow on the wall opposite to the windows, this for a sunny splash feeling. The window wall I would paint light grey, the others walls bright white. Above the fireplace I would put some cute wallpaper in the same color theme as your sheets, just to spice it up a little! Curtains solid or just white/grey stripes.
Good luck!

BM Green Tint or Gray Tint

You should probably do a pale yellow.