You shouldn't have any structural issues moving the drywall back to expose the whole closet opening. I caution you about having completely open closets though. You may think it will push you to stay organized but that's not always the case. Especially with a baby coming keeping the closet clean and organized just won't be a priority.

If storage is really an issue consider removing the closet and installing an IKEA pax system or something similar. Also take the time now to really purge old items you don't wear.

Ya I know organization is something we have to commit to but a big part of that is not currently having enough storage space. I notice that the areas behind the walls of our closet are basically unreachable. With these walls removed I would purchase something like the IKEA pax, or similar as this isn't currently an option for us.

While you could enlarge the opening, I can't imagine wanting to live in a closet without doors.

I am finishing up a gut remodel which included doing custom interiors of the closet. Since almost everything is still packed up except for those articles of clothing which I wear, much of my closet space is still open. I can't imagine wanting to look at the interior of my closet even with a minimal amount of stuff in it and I see what it is like when the doors are open versus when they are shut.

I also have a closet in the master bedroom which has some bit of storage running behind a wall and I don't find it to be an issue. I've always used it to store items that I don't need to access frequently - i.e. winter coats or evening gowns :-). In other words, it doesn't detract from my functionality.

Perhaps you would be better off with sliding doors rather than bifold doors. The doors in my master bedroom are sliding doors and give me better access than bifold doors I have on other closets. I replaced the original sliding doors with new mirrored doors that have invisible tracks and I really appreciate having full length mirrors as well as the visual spaciousness which mirrors provide.

Gonna bump this for some more tips and insight :)

Were you thinking of placing a PAX unit in the closet itself? Those things can be big so measure carefully whether you intend to place it in the bedroom or in the closet.

The success of your idea also depends on how you feel about all that "visual clutter" of clothing. Are you as neat at hanging up your clothes as the editors were for the above pictures? Notice also the restrained colour palette of those wardrobes. Let's face it, the clothing in these closets is artfully arranged. Would you be all right with it if it were less so? Perhaps you want to test drive your idea by removing the closet doors first. Hard to go back once you remove the drywall.

In my experience, it's not so much a lack of storage that's the issue as much as too much stuff. Also, if it helps any, I usually have to do a load of laundry every day or two to keep on top of the dirty clothes piling up. (I don't want more than one laundry bin in the bedroom.)