You might get away with it, but you would be far better with a charger that is designed for the type of battery. Even if it does charge the battery, unless it has the right kind of charging controls it will shorten the life of the battery.

you really want a slower rate of charge if you don't want to cook the cells.

Chris hit it on the head. Heat is what kills batteries being recharged. What you are attempting to do is DANGEROUS. A car battery charger will sink a lot more current into that battery than it can handle. An Carbon-Zinc standard cell will cook badly. An alkaline cell may even burn. Don't even think of doing that to a NiCad/NiMH/LiOn battery unless you want to see something truly amazing (stand back and place the apparatus on a noncombustible surface).

If you want a rechargeable lantern either buy one that has a rechargeable pack or one that will take rechargeable cells. I've seen a gizmo that is the size of a 6V lantern battery that takes 4 D cells. Charge using a charger designed for the type of battery you are using.

You need to match the charger to the battery in type and charge rate.
You can't charge a nicad with a lead acid charger or NiMh charger etc.
Your battery will also have charge rates in amps and hours. These must be adheared to.