It's best if you show an overall picture of the area, including surroundings, so we can see details of things you might not explain accurately or in enough detail.

Soil can hold some degree of slope if it is protected by permanent plants, but where you're gardening and constantly disturbing the soil surface, it can only hold a much lesser degree of slope. So it depends on the details of what you're trying to do and the conditions of the site. Terracing -- leveling out the ground and retaining it with some sort of retaining wall -- is a time-tested method for lessening erosion. A primitive retaining wall can be built out of dry-stacked rocks, if they are large enough, and if you do it correctly.

I'll try to take a video or photos if I can. It's a bit difficult to explain, but there are interspersed terraces surrounding it already for decades. This plots slope ends at a cliff, which is covered in bamboo and vines, so it does not erode. But given the bed rock, shallow soil, and slope, I feel a lot of the moisture and extra added soil is disappearing down the hill. That's my idea to add a small retaining wall up a bit from the cliff edge to see if it helps. I don't want to level the entire plot, as it would be too much work. Just deepening the soil and moisture content next to the wall would be adequate to plant something like kiwis or fruit trees I hope. I thought vines would work well in a deepened pocket of soil, and the trellis could extend over the shallower parts which would not need to be improved. Citrus I've planted in that area is doing OK, but it needs to be frequently fertilized. Blueberries required frequent watering in the summer to survive.

Would piling a rock wall eventually fill up with dirt and act as a barrier, or with this just let water and dirt flow through?

It is possible to pile up rocks into a wall in order to retain soil.