Probably aphids. Yes, you should get rid of them. They are sucking on the plant's juices. For starters, wipe the plant down with a wet, soapy paper towel and kill as many as you can.

Home Depot and Lowes sell a great spray that gets rid of aphids and other pets. Its natural and after 2-3 treatments it has always wiped out any pest that was bothering my palms or other plants.

https://www.homedepot.com/p/Miracle-Gro-Nature-s-Care-24-oz-Insecticidal-Soap-0747210/204631793

For WHATEVER reason, some plants are much more sensitive to bug infestations than others. I have had more than one Roystonea been afflicted with mealy bugs while nearby palms were fine, and mine are outside for several months of the year! I think I have given up on them for the time being. Queen palms (which them somewhat resemble) have been much easier for me to overwinter in a variety of locations (house and garage).

A systemic kills them without bothering the palm and kills the aphids off for years.

Yes, use systemic house plant insect killer...works on any sucking insect.