That is a scale infestation, a harmful sucking insect. And lychees have their very own dedicated scale species, although they can be infested with other types as well.

There are many online publications addressing how to deal with scale insects but all will point out a need for correct timing as to the spray schedule. Horticultural oils are most often recommended and are an ecologically safe control but may require repeat applications to be fully effective. There are also systemic insecticides that can be used where timing is not so critical but they typically come with a heavier environmental impact as well.

Thanks gardengal48, this is very helpful!