I have no fence or shed, and am having the same problem with one in a row of four silver maples planted (I think) within five years of yours. It's strangely the only one that has bladder gall too. The only info I've been able to find is that die-back and failure to thrive happen some times for no identifiable reason. Perhaps someone more knowledgeable will chime in.

Do you see strips of dead bark leading up to the branches that are dying? The first thing that springs to mind would be the increased soil depth, but it seems like the root flare is still reasonably above the soil level. Compaction could also be the culprit. I've heard of people aerating soil by driving holes in it, but I've never seen it done. You might get up on a ladder and look for insect damage.
One other thing to look for is dark spots in the wood of the dead branches. If you find these spots, you should sterilize your pruners immediately. This could be a sign of verticillium wilt, a fungus that colonizes the interior of the tree, blocking water and nutrients from getting to the branches. If this is the case, the only thing I've heard of helping to slow the decline is regular waterings with compost tea. I don't mean to be alarmist. I've seen maples planted too deep that look like they could suffering from verticillium, but it is a possibility.

best of luck!

Here is a link that might be useful: Verticillium wilt

on second glance, the dead branches don't look black, so it's unlikely that it's verticillium, but it wouldn't hurt to check.

Hello, yes I was going to rent a soil aerator in the fall when the soil is moister. A lot of activity around the tree, including getting my truck stuck twice. I also bought some maple fertilizer that involves drilling down 6 inches to apply. I was going to do that in the spring when it rains more. However with all the rain lately I can probably aerate and fertilize now.

I was also thinking of cutting down the trunk that extends over the shed. Any thoughts on cutting a major trunk off a sliver maple?? Thanks.

I think the aeration and fertilization is a good idea. Regarding removing a substantial trunk of the tree, I would recommend getting in touch with a certified arborist.