There are some possible options: adjustable pendant lights can move up and down with pulleys while linear chandeliers (sometimes called kitchen island lights) tend to be longer and more rectangular, so they could suit the opened-up table as well as its closed arrangement.

We need pictures of the room to really offer suggestions

We are dreaming of the space yet. It is part of the renovation of our 116 year old home. We plan to use salvaged ceiling tins for the ceiling and original hardwood oak sanded and restained in special walnut for the floor. Our table and chairs is Amish built mission style. It can open to 21' when we need it.
Sorry I have no pictures as we are still in the planning stages. Thank you for your suggestions.

I would think the simplest thing would be an orderly array of recessed lights to cover the space evenly, so that you don't really need to adjust anything if you move the table. Table lamps as required elsewhere...

The one trick is that recessed lights + ceiling fans can be a bit of a stroboscopic nightmare, so you may want to swing wide in those areas...

How about a nice wood strip on the ceiling with a skinny ribbon track and heads that can be moved along the track when you need a longer table I would do 5 heads and just have them closer together when the table is smaller as for fans they really are a bad idea in a DR they cool the food off and create air movement when you don’t want it.