Only two photos makes it tough to see the flow of the yard, patio, driveway/parking. That said, were it me, if I had the funds, I would tear out that beat-up asphalt and lay down new asphalt or concrete. I would better delineate the planting areas but don't think I would use boxes. By periphery, do you mean the periphery of the patio? Driveway/parking? Unclear. I don't think I would put gravel over that asphalt, but where I live we never have freezing weather...don't know if you have to deal with issues like that that make it more difficult to keep a "crack-free" driveway.

Here are some more photos.

I meant the periphery of the asphalt area if we cover it with gravel. The problem with redoing the asphalt is that it will be very expensive-like at least $15,000- $20,000 because we also have to redo the rest of the driveway. That's why I was thinking gravel. We are located in the northeast (NJ).

Well done with the patio.

You might consider outlining your driveway with treated landscaping posts before adding your gravel. IF you can create a circle around a tree in the back to drive around rather than needing to add a space specifically for backing up to turn around, do. And don't forget to add at least a couple of parking places as you outline where you want the vehicles to be permitted to go -- doing that will help protect the rest of your yard.

FYI:

My son had a low place in the driveway that kept wanting to wash. He turned a half full five gallon bucket of roofing cement upside down in the beginning of the ditch/wash during the heat of July and it oozed out of the can and filled in the hole securing the couple of buckets of rocks and gravel that had been dumped there previously -- a "fix" that the rainwater had been just washing its way around. That roofing cement fix is still holding a decade later. It would be too expensive to do that to the entire driveway but if you have one or two low places where you need a DIY bonding agent, you might consider it.

"I meant the periphery of the asphalt area if we cover it with gravel." I can't picture what you're talking about ... how to cover just the edge of asphalt with gravel. You might sketch out where and how you're talking about on a plan. (Surely, there is a plan someone drew for the patio.) I can't picture gravel at the edge being pretty either.

We would put gravel over the entire asphalted area till the beginning of the patio that leads to the steps. From there down the driveway will be re-asphalted. I will try to post a sketch after work.

OK. That's not a good idea. Likely, it's creating a mess for the future.

Even if you can't do it now, it looks like you need a complete new drive/parking since the asphalt is quite deteriorated. If it were "new and shiny" it would not be bothersome to look at next to the patio. That would be my long range plan. Also, the step up from the drive to the pavers seems not near as good as if the surfaces were flush. In my long range plan, I'd plan to bring the driveway to an elevated level that meets the pavers, biting off however much installation I could chew at a time, if it needs to be phased in.

I agree it would be better if the driveway were flush with the pavers, but besides the cost I am concerned that my neighbors could experience water difficulties since their property is down slope. May I ask what is the concern with gravel? I see lots of articles about people using gravel as a low cost alternative. Are they downplaying the issues?

I see. I was thinking the asphalt would be a good base because weeds would be less of an issue. Thanks for the info.

Keep researching. It may be possible for a landscaper (the kind that do earth moving) to rough up and level the rubble of the existing asphalt to use as a bed for a new surface. That would save the cost of removal. You are fortunate that the driveway is lower, as this wouldn't be advisable if it meant building it up. Look into all possible surfaces, prioritizing those that are permeable so you will capture stormwater runoff. If you have lawn you are mowing anyway, you can make the driveway appear seamless with it by embedding permeable pavers. The right gravel (a crusher run) could work if you solve the underlayment. Take your time on this as it's worth getting right. A shiny new asphalt driveway would be my last resort, for aesthetic and ecological reasons.

Where gravel would cover asphalt, it is hart to image how the transition would be made. You can't feather the gravel to a thin layer.

As it stands there is X amount of rain that falls on the drive & walk, and from there it travels downhill to wherever it goes. Raising the drive to meet the walk wouldn't change the amount or its final destination.