I think you can feed your worms once a week without any problems. Just try not to feed them too much at a time in case it heats up.

Another thing to think about is harvesting. You might have to have your face over the pile of castings for a number of hours at a time.

Good luck,
Marshall

Im wondering if wearing a dust mask would help so that youre not inhaling any spores?

with harvesting you can spend hours getting all the eggs out or just collect the castings and use them like I do, sorting through might cause you more problems than its worth with your allergy.