Toronto Zone 6

It would be much happier planted in the ground. If you intend to keep it in a container for any length of time, it will need a much bigger one than it currently has, plus you will need to move it somewhere protected for winter - the root system will freeze solid if left outdoors. And containerized plants need routine fertilization almost from day one - unless a nutrient enriched commercial potting mix, there are no nutrients available to support or nourish even proper leaf growth, let alone roots and any fruit production. You can use any water soluble or liquid fert (Dyna Gro or Miracle Gro) at half strength weekly.

GardenGal Thank you for the advice, I planted in the ground. Turns out the soil in the pot was quite moist so I think that was one of the problems as well. Hopefully, it bounces back. Thanks so much!