Here's one-way Battic Door.

I just have a hatch, no stairs. Have to put a ladder up through to gain access.

How tightly sealed is the hatch and is the hatch cover insulated? Rigid insulated foam is reasonably cheap and light. You can glue it to the hatch cover. If the hatch cover has a poor seal, use products designed for weatherstripping doors.

For starters, there looks to be a sizable gap all around the hatch. Calculate it and it ends up being several square inches of hole in the ceiling. Because warm air rises, you have a chimney in the hallway.

Honestly I haven’t taken a look to see what’s there now, but what and how is common?

Maybe it is just me, but the first thing that I would do is take a peak at the other side.

Until you can tell us more, I'm fairly confident that the biggest reason why your hallway is cold is that big air leak in ceiling.

Took a look and there’s just sheet rock covering the hole. No insulation on the back side of the hatch cover at all.

The hatch lid doesn’t sit in there level. The drywall ceiling isn’t level. Where to begin?

Is it the ceiling or is it because the hatch is warped/ curled?

The pictures don't tell us enough.

Like, is the hatch lid only a chunk of drywall. Is the lid sitting on a lip of drywall?

Typically, this sort of access is framed--and looks like one of these two photos:

Is there a possibility of relocating attic access to an unconditioned space? Like an attached garage if the house is single level.

The access covers I’ve seen have all been trimmed around like the above pictures. Your cover then sits on the trim not the ceiling sheetrock. you could do this and replace your cover with a properly sized piece and attach rigid or batt insulation or go old school and just put batt up over lid so it falls on top as lid is lowered.

This would provide insulation but not a seal but if you put weatherstripping along topside of your trim your cover would rest on it and hopefully stop air leakage.

That’s my plan. I’ll need to check level on the ceiling

You are in Illinois?

For right now since it is very cold, what I would do is get some fiberglass bat insulation on the back side of that removable hatch. Cover it with some nylon net (from Jo-ann's, etc.) and staple the edges to the hatch cover. Replace it in the hole. Then, on the room side, cover the entire opening with either Visqueen, other plastic or Tyvek and tape the perimeter of the plastic to the ceiling. (Painter's tape?)

Quick but dirty. Research a better permanent solution to be tackled for when the weather is warmer.

Perhaps the hatch can be permanently covered with a drywall patch and attic insulation IF the attic access can be relocated. Like to a garage if this is a single level dwelling and garage access will allowed you to access the attic space over the house...

If we knew more about the entire structure, someone would be able to suggest other possibe remedies.

I’m in a split level/raised ranch so this is really the only location for the hatch.

You can google for prefabricated insulated attic hatches in that case. (Home Depot, Lowes....) My suggestion above was for a temporary solution for this winter with the idea to tackle a permanent one for when the weather is more moderate.

I’ll look into the prefab. The hatch does have a build up around it up in the attic itself, some wood surrounded by insulation.

Really? My close big box stores have plenty.

In that case, just get a small roll of insulation - bigger R value the better - and cut one or two pieces to a size that will overlap the edges by 1 -2 inches. You can ask your neighbors if they have any left over sitting around.

You can glue the insulation to that sheet rock (I am not sure if there is a lip that the sheet rock sits on - but if not, get some trim and attach around the hole to make one.) Put foam window or gap insulation on ithe lip for the piece of sheetrock to sit on and seal the gap.

Does your electric or gas utility offer a home energy audit? Mine does, for $20, and through that audit you can get services like adding insulation or air sealing with a rebate that pays most of the cost. Either of those jobs would fix that access.