Take the trim off and insulate the cavity between the door frame and the framing.

Use something like Spray foam sealant for window/door.

Also, check the weather-seal around the door and make sure the door is tight against it in the closed position. If the reveal between the door and the frame is too big, install a few shims on the sides and put the trim back.

Good luck

If the door frame is not square, that means the door surface does not mate to the weather stripping. That causes the problem you have, to fix it properly, the fasteners must be removed, the frame resquared and fasteners reinstalled. If screws were used to install, those can be reused. If nails were used, replace them with screws.

There is a difference process to using screws to install a door frame than the one used for nails. Screws can be over tightened, causing a short section of the frame to bow. Also, wood screws(the kind with no threads from the head to where the threads start) need to be used to minimize bowing. Shims are used, as with nails. I've found the shim pack needs to be a bit thicker when screws are used, to allow for the extra tightening. I use five shim packs/mounting points on vertical framing, and three on the top.

If you use the spray foam, make sure it is not the heavy duty expanding type, since that can push out the frame.

Mac's procedure for squaring frame is spot on but somtime's must be tweaked in a few years. For that reason I never foam the cavity. Pack it instead with bat insulation, caulk backer rods,pipe wrap or something similar.