It might be thrips. You could try the blue cup method like people do for roses and see if that helps. I grow this plant and hibiscus and the same thrips like both.

Your leaves do look healthy. Assuming its getting full sun, water and has excellent drainage it is probably thrips. The Sambac jasmines also like humidity so maybe mist them daily or whenever you water.

ETA..I did a little research on this and there is also a gall midge insect that affects buds before they swell as opposed to thrips that wait till the buds swell to attack. Im thinking yours must be the gall midge. The suggested tx was for bayer drench but I dont spray so I'm not up on how to use that. Another suggestion was to gather affected buds and dispose of them in trash bags so they don't spread. Research it and see if thats what you think it might be.

Sultry, you may be right. The tiniest of budd are also affected. I'll give everything a haircut and see if it helps.