Weather proofing comes from the use of insualtion and no amount of plugs will have any real affect if you don’t insulate

I’m just looking to keep the water out. Not worried about regulating temperature.

Use some silicone caulk.

That would have been helpful and yes silicone .

https://www.fastenal.com/

There's a location almost everywhere...they will have good advice & what you need.