At the least, low-expansion foam should be used all around the opening. Flashings and sill pans or equivalent would be advised too.

Yes

He is then applying caulk over the surface of the gap, leaving a significant air pocket between the frame and block wall. Should this gap should be filled with some sort of insulation?

Pictures?

Gaps do not have a surface, so what is he applying the caulk to?

I'd be concerned about what he is using for a sealant.

Tremco has a great low expansion foam. Then - consider some backer rod, and caulk as the final layers.

I got more info from the contractor. They are installing backer rod when the space is big enough, but nothing when it’s too skinny of a gap. They are caulking the interior and exterior of the window frame. Drywall will then cover the interior caulked areas. The window company told the contractor that they went above and beyond by also caulking the interior, as most people just run drywall up to the frame. The other problem is the windows are aluminum frame, which are empty, so I think this is more space for sound transmittance. The contractor is going to speak with the spray foam contractor about removing the interior caulking and filling the aluminum frame and wall gap with insulation. What do you guys think of this?

Thanks Janni. That sounds like what he is proposing. Of course it will be an added cost, but I think it’d be worth it.

I’m trying to load pics, but theyre going slow.

that gap is big enough you need spray foam. and I'd push for both sides caulked / backer rod'd.

Definitely backer rod and caulk both sides as these are sound attenuation windows. In fact I would probably push for a double exterior bead of caulk.