Are you starting at the floor of the first floor and drilling down? If not, how far above the flooring of the first floor does the bit stop?

I'm drilling from the top floor down, yes. The basement ceiling is only about 8 feet tall. I can only get my bit about 4' down

Exterior basement wall is likely concrete that supports the main floor possibly with some furring inside for the finished wall.

Ohh I hadn't considered that, meaning that walls are not actually aligned, the basement wall protrudes further out? So in that case my options would be pretty limited, short of drilling through the exterior wall then

So, by main floor, you mean the first floor. I assume that floor has baseboard, maybe 8" high at this outside wall. Are you starting right above the baseboard? You could try getting a long, 1/8" diameter bit and drilling through the floor into the basement ceiling and investigate where that comes out, assuming the basement walls and ceiling are unfinished.

Unfortunately (or fortunately) the basement is completely finished, but I do have potlights, so I could pop one out to check

Can you drill through the floor, pop one of your potlights out, and look for the drill? Or get a drill long enough to go through the ceiling drywall.