Yes, you can do it.

• After removing the fan you will have to remove knockouts from the electrical box and secure the new cables using cable clamps.
• Install 3" or 4" dimmable lights with recessed lenses (not wafers to help prevent glare). Choose 3000K or selectable color temp (examples: Juno CS WF DREG SM, Maxxima 4 in. Slim Round Recessed Anti-Glare).
• Replace the fan/light switch with dual LED-compatible dimmer switches (e.g., TOPGREENER Kalide).
• Test compatibility between the lights and dimmer. Confirm that the lights can be fully turned off.

Note: you mustn't hide the old fan box under drywall. Attach a cover to it.

Probably. There may be a 3 wire to the fan. So pick a switched hot wire for each pair of lights.

Are you saying, I may have an extra hot wire to cap off?

No, each switch will control one hot wire and will each control 2 lights, as I understand your post to say you want to have two differently switched banks of lights.

You probably have a fan/light wired like this. You'd connect the red wire to one pair of lights, the black wire to the second pair of lights, and the white neutral wire to both pairs of lights.

(But substitute a double-dimmer for the two switches in the diagram.)