I am assuming this is dry laid work with no mortar present. The patio should not have any lumps and depressions nor should any of the pavers be loose, some gaps not being completely filled with sand is acceptable although filled joints aids stability. Get him back to correct the work before you pay the final bill. If the difference from level is as much as you say then I would call the work shoddy by any standard.

Was the patio pavers, concrete bricks or similar, or flagstone. A concrete paver patio should be completely flat with no "trips". Even a flagstone patio should not have any "trips" or stones that sit higher than others. Walking would be a liability, and with a 3" difference you'd have to shove 10 matchbooks under your table leg to even it out.

Thanks for the replies. I think I will sit down with the contractor and find out what he can/ is willing do to fix the patio.

It should start completely flat. TIme will make it worse, not better.