The "mortar" looks like plaster of paris or Fixall.

*I don't know why pictures are sideways!

No advice sorry!!!

So mine must be much lighter in weight. It was used on an inside wall and both the “rock” and “mortar” is extremely light in weight

One of my rental houses has some 70s era fake brick. I've been told that it's made of fiberglass, but I don't know for sure.

It’s truly horrible, so it’s a great idea to get rid of it. I doubt that it contains asbestos, but you can take a small sample of the mortar to a certified asbestos testing lab to check it out.

Presuming you don’t need to get a remediation company involved, and can do it yourself, wear a good respirator and cordon the area off with visqueen so you don’t get a lot of dust around your house. A small sledge, a cold chisel, and a crow bar should make short work of it.

I would pull the trim off the end of the wall in order to be able to see a cross section. You might be able to remove it in a few large chunks which would help to keep the dust down and any potentially hazardous material release minimized.