Only about once a month/other month, hardly anything.

EDIT: I have added pictures below

Yar, if it is squishy to the touch and brown in color, you need to unpot it ASAP. I am suspecting the root system has been rotten. Under water or over water will kills the plant. Over water will kills it much faster then under water. It is possible that the plant beings in the rotten stage for too long and its already healing itself. Either use a knife or spoon and clean out the rotten part (don't use the same knife or spoon that you use on yourself). Most others will suggest cutting until you see white. I have used a spoon and scooped until I see a yellowish stick gel like layer. Then used hight presure water to spray the yellow substance out, let dry and replant. Much easier way, is to throw it out and buy new plant and enjoy. Don't feed the plant while it is weak. Unlike human, plant don't feed while they are weak. others might have better suggestions. If you can upload a photo it will greatly help us pin point the issue.

Every other month would be too extreme in my point. But there are many here with lots of experience and knowledge, hopefully they wI'll chime in soon.

Yar, if this is my plant, I will do the following: un-pot it, clean and cut off damaged roots, shorten the long branches( the biggest branch can stay, it seems healthy), the branch should be green on the outside and white from the inside. Any brown branches should be cut off, sprinkle with cinnamon or turmeric power if any roots have been cut, let dry 3-7 days, if there is no rotten root, soil can be reuse. Otherwise change new soil, after the cut has been dried, moisten the soil and replant, slowly put it into full sun (the more the better). Sunburn can happen if it stayed too long in the shade and suddenly go into full summer sun.

I just noticed that there is a spot on the back of the caudex. I can feel the thin and papery bark. It seems like there would be a hollow spot there if I cut the thin layer of bark. Should I slice it open to look or just leave it alone?

Just rip it off. Like I suspected, the plant was rotted and recovered itself. Luckyly, you underwater the plant and the rot has been dried out on it own. If the papery part touched the soil it might already developed new roots. If you don't might how the roots look like, you can leave those roots alone.

Yes, luckily the dormant season was still happening until recently. I do agree that I probably tried to wake it up (so to speak) by giving it a good soak last month and it wasn't hot enough

I dont know what the temperature and what the current weather where you are at. The rules for adenium are water it every two to three day or the top 2 inchese of soil is dry to the touch. Fertilize it one a week during active grow season. No leave, no new grown, no fertilizer. Place the plant at full sun if possible. It will not get sunburn. Wilted branches should be cut off. New grown will start in about a week. Keep us posted on how your plant is doing.