Peppers grow much slower than tomato plants. That's why one normally would start the seeds 2-4 weeks before starting tomato seeds. You got a late start for peppers. I've started seeds in various mediums; I've not noticed that peat pellets lead to slower growth than any other method. The casing should be removed the first time you pot the plants up. And you should pot the seedlings up at least 2, maybe three times before putting them out in their final pots. Letting them get rootbound can stunt growth.

I usually fertilize the seedlings every time I pot them up, then when I plant them out, then every month or so. You might want to look into a more well-balanced fertilizer. Fish emulsion is almost all N, which peppers don't want a lot of, and you may be lacking in micronutrients. Don't fertilize heavy - peps don't need as much as tomatoes, but try to be consistent about it and definitely try to use a more balanced fertilizer. (I use all purpose MG plant food, mixed at regular strength).

Your temps are fine for tomatoes, but peppers really like that heat. What have your night time lows been like? That can affect growth a lot. Also make sure you aren't watering too much because that can wash nutrients out faster and also cause problems. And full sun is an absolute must, especially in your latitude.

The good news is that once that cayenne takes off, it will go like gangbuster.

Oddly enough my peppers loved the peat pellets more than the coconut coir. I do always remove the casing when potting up. I've seen too many posts about restricted growth otherwise. Plus when i remove the casing, if more than one sprouted, sometimes i can split them into multiple plants.

I also planted tomatoes and peppers at the same time and the tomatoes are insanely huge. Peppers are just showing buds. I gave both the tomato and veggie fert.