I'd say you're in a far better position than us to judge whether it's worth climbing an extension ladder and spraying some Wasp & Hornet spray on the nest and possibly getting stung, loosing your balance, and falling off the ladder verses leaving them alone if they don't have a track record of stinging anyone yet. If you're asking a tangible question like, "will they become more aggressive and attack someone?"...I'm guessing they're paper wasps so this isn't very likely. If you decide to wait until Winter and knock the nest down then, I'd dare guess, you'll have the same problem next summer....seems like an ideal place to build a home (from a wasp's point of view).