Glass and keeping the materials light and consistent throughout the bathroom would help a bit.

Paint/tile it black/midnight blue. Completely tile walls and floor- wall hung toilet and sink. Consider just having a floor drain with no shower enclosure at all, just a curtain or plain glass if you're concerned about the splash. Unless it's a large space, glass block will weigh down the room.

The room is about 11 ft by 7.5...

That's enough room- do you have enough ceiling height so that the block doesn't have to go to the ceiling? I saw a curved wall treatment in a showroom that looked kind of cool.

We were thinking of just painting the ceiling and ductwork a shade of white. We think it might look too enclosed if we tried to finish it off. The ceiling in most of the bathroom is about 7'2"
We thought that with the glass block it might help it look more open...?

I think the newer glass materials will help it feel even more open- it's just preferences I guess. If I was going for a retro look, I'd chose block, 6' height wall. For a more modern look, an obscured architectural glass could separate the shower area, or a combination of tiled wall and glass.
I'd probably draw a line at 6' for the whole room, and use interesting tiles to keep the eye focused lower. To me, a darker ceiling creates an illusion that the space is continuing, although in this room having the wall and ceiling the same color would probably work best.

But here's a bath I like that runs somewhat counter to what I'm saying but I think gives the spacious feeling you're looking for.

A friend who is a painter suggested that we should paint the ceiling a black luster color to hide everything. And says its easier to keep clean. We have travertine tile for floor and shower, so most things will be light colored. The vanity is medium wood colored with lighter top. Anyone have thoughts on whether using dark or light on the ceiling will make it seem higher?

A dark ceiling with a matte finish will seem to recede more than a lighter color or a glossier finish. Also, the larger the room appears in plan, the lower the ceiling is likely to feel, so, breaking the room into smaller spaces (i.e. not having a transparent shower wall) might actually help.