Are you talking about an in ground yellow jacket nest or some type of wasp nest hanging on the eaves?

If it is a nest on the eaves that has no activity on/around it, you are probably safe to knock it down.

I don't know as much about in ground nests or if the buggers over winter in your area. Actual yellow jackets can be quite troublesome to get rid of in my climate. They can have big in ground nests with more than one entrance and are pretty easy to stumble upon by pets and people and get multiple stings for their carelessness.

the nest is up in the porch under the roof. So I can move it? I dont want to get bit!
thanks!

Are you sure these were Yellow Jackets?
Are you sure these were not the other, more beneficial, Paper Wasps?
Both around here are still out and about and active because the weather has not been cold enough, long enough yet to stop them.
If that nest was a Yellow Jacket Nest all that will be left in it are some queens that, next spring, will venture out and find other places to nest in. They will not nest up there next year. If that nest is accessible, and there are still live wasps around you can get some cheap hair spray and spray them to keep them from harming you while you clean out the nest.
Whether they are Yellow Jackets or just the Paper Wasps, once the nest has been cleaned out caulk around the are to seal any and all entry points, just be sure that all the nest and wasps are gone before you do.

The colony survives for just one season, then dies. So, if you're certain they're gone, you can safely remove the nest.

whats the difference between paper and yellow wasps? i have not seen a wasp in weeks, night or day so I think that they are gone. will try to move it soon thanks for all the help