An attic fan is not necessary if you have a ridge vent. Do you have soffit vents and if so are they unblocked and there an adequate number? If not then you need to address that problem.

When was the last time the oil burner was serviced? If you smell fuel oil in the boiler room then you have incomplete combustion. You need to have this addressed as soon as possible. I would also would investigate if the oil smell is being pulled from the chimney or a flue pipe running through the attic. This is a safety issue so you want to hire an expert to do an inspection.

The oil smell only happens when the fan is on I had it serviced and never used the fan since per furnace maintenance guy. The soffits are vented

There are two types of "attic" fans that people refer to:

1. Attic fan typically on a temp/humidity sensor, placed at the roof plane or attic gable wall. People add them because they feel their attic is too hot. Unfortunately and contrary to what non-pros testify on the internet, these fans can cause harm because they pull makeup air from anywhere that is easy: "short circuiting" the flow of attic air needed for moisture mitigation; through the ceiling plane bringing conditioned air which can cause condensation; exacerbating stack effect which can alter gas appliance's safe burning and atmospheric venting.
2. Whole House Fan placed in the ceiling, usually a hallway, to cool the interior of house down by opening the windows.

If it is #1, abandon or remove it and use proper passive attic ventilation of low-to-high venting.

If it is #2 open windows to provide makeup air when operating.

House fan over garages in attic with separate switch to control