Humidity tries to move from high to low, and will travel slowly through building materials and quickly through air gaps. High humidity in your cellar will spread to floors above and can lead to mold and decay if high enough. I'd put in a dehumidifier. Try to run a hose to a drain, or get one with a built in pump. Buy a cheap temperature and humidity monitor to see what levels you have.