It can make a difference in which parts of the lamp or lamp base are "hot". Black to black will minimize the possibility of touching something hot. It makes no difference to the light which way it's wired, it's a safety thing.

Yes, to amply what rwiegand said, the circuit has a grounded conductor (often called the neutral, which should be the white wire) and an ungrounded conductor (the hot or black wire). In addition there may be a bare or green equipment ground. These days all the exposed metal should be connected to the equipment ground. However, there are a few things that can't, one being the theads of a screw in bulb and its socket. The grounded conductor is usually connected to this because it's possible you might touch this while changing the bulb, etc... You're more likely to have issues touching the hot (as there are lots of grounds around) than touching the grounded current carrying conductor.

But this light has NO socket. Just small LED squares that light up. So unless some other part of the transformer is hot I don't see the safety risk. I'm thinking about undoing the light and use a plug in circuit tester to connect to hot and neutral wires to see if white is really white. Thoughts?

Incandescent bulbs will still work with polarity reversed, but LED drivers will either not work or (worst case scenario) burn out with polarity reversed. I would check each location to ensure proper polarity.

On the plug in circuit tester which prong is hot and which neutral?

Hook it up correctly, no reason to switch black and whites on old fixtures. If the white is a 3 way traveler they should have marked it with black tape to show it is hot.

I think we dont have the whole story. The only way I could see it done like this is it is a three way circuit and the white is not marked.

Here are diagrams, figure out your scenario. If it doesnt make sense they probably did it wrong and the previous lights just worked even though it was reversed. The new LED lights wont work in reverse as mentioned.

There is no POLARITY in AC. The difference between these two wires is one is at the same potential as the house ground (or very close to it) and the other is 120V away from it (on average).

I believe all LEDs are low voltage devices. A LED bulb with a screw base which goes into a standard lamp socket has a LV power built into its base. If your fixture has the little square LEDs, then it must have a power supply somewhere else in the fixture. If not, then there needs to be an external LV power supply. Anyway, the wires connecting to the squares are after the power supply and grounding has no bearing.