Will... I would guess that if this ever was a fire place it was a coal burning insert (very shallow). This would be typical of a home built in the 20's. Shallow bricked chase with a coal burning insert and flue pipe that exited via the chase to the roof. Usually the coal burner was cast iron, sometimes ceramic. Did you open up this chase or was it like this when you removed the plaster coat? There are several issue you will have to contend with. 1. Standard gas inserts require more depth than you currently have. You'll have to find one that is designed for a shallow depth and most likely one that will extend out into the room some. 2. You'll need to install a steel shelf angle to support the brick properly.......not all the hard for someone who has some experience..........the 2x4 ain't cutting it. 3. I suspect the circular opening you have at the top was an in line flue damper at one time . Of course the old flue will have to be replaced with a double walled code compliant flue.

Have you thought of just bricking up the opening with like salvaged brick and going with a free standing unit with the an exposed flue that utilized the damper opening.............this could be wood burning or gas (if you have a means of getting gas to it)...........so many options, but I'm thinking an insert it pretty much out of the question unless it's a specially designed one for very shall depths (i.e. something gas that mimics an old coal burner and which are usually very expensive).

As to the plaster.............hand chisel, hammer and lots of muscle work. A good mason can close up the opening for you and match (closely, maybe not perfect match) the brick and joint appearance. There certainly seems to be many possibilities here, but not so sure they are DIY unless you've done something like this before...........certainly not something in my opinion that you would want to learn on since if left exposed will be a very prominent feature..................would love to see photos of the finished project when completed.