Passion vines are the host plant to the Gulf Fritillary butterfly caterpillar. It would be a shame to kill them. Most of us just plant some extra passies for them to eat. That being said I can't tell if your plant looks eaten or diseased. Hopefully one of the experts will answer.
Karyn

Definitely caterpillars. They won't harm the plant long-term, it will bounce back.

Instead of using harsh systemic insecticide that kills *everything*, if you must get rid of the caterpillars use bacillotoxin, often referred to as BT. It's a bacteria strain that only attacks the larval form of certain insects (ie caterpillars). It's sold at Lowe's and Home Depot and other places under the name "Thuricide." There are other brands, too. It's not expensive, works great and mild on the environment.

Oh...please try to 'live' with the minor damage...you will be rewarded with these gorgeous creatures! :)

And yes...I realize not everyone loves butterflies...that's OK, too...but, just in case you welcome them into your yard, this is a wonderful way to entice them to stay.