I'd work with a landscape architect, especially since you might be doing structural items. Yes, we had a lot of concrete removed because like yours, it was badly cracked and raised. The cost of removal was very high, pouring new was even higher, which is why I wouldn't do this without an overall plan.

You have such a great unique space. Have you thought of having a centerpiece in your yard such as a fountain? We have a great gallery on our website with rock gardens (which is what inspires me with that photo of the stone pathway you and cement block you have). I have attached some pictures for inspiration, please do not hesitate to reach out for more ideas or pricing.