Is it a bad idea? I don't know your particular situation well enough, but I know I wouldn't do it.

Hm. It's promising that you've seen a project of his go well. You don't know what you haven't seen though.
Ramifications? Many people are able to sort out issues maturely, when, not if, issues arise. Not everyone is though, and it is a risk.

Do you have an architect, and have you checked out their recommended builders, or asked if they know Mr Neighbour?

Neither would I.

Do your homework on him as you would with any other builder, and if all checks out fine, you could go with it.

Have you talked with previous clients? There's a lot of due diligence involved in selecting a builder.

We used a friend for our major renovation at our old house and it worked out beautifully. But it only did because it was a proper business arrangement with a clearly worded contract on pricing, responsibilities and expectations. It left nothing out and helped us all stay professional and on track. We expected nothing other than good paid for work and didn't expect any bonuses or cheap deals because he was a friend. We were extremely happy with the resutl.

We used someone who lives in the neighborhood as our GC for our last major remodel. We knew she had remodeled many, many houses in our neighborhood and thought she would understand our expectations of quality. I talked to several people she had done remodeling for and they all raved about her.

She was a total disaster! She had no control over her subs, who were also lousy. She would not show up but once a week or so and did not return phone calls. It was a total mess and the quality was awful.

After it was all over, I spoke again to some of the people who so highly recommended her. "Well, there WERE a FEW problems with the painters/flooring guys/carpenter, but we hated to say anything bad about her as we're all members of the Country Club"! Gee, thanks a lot!

The neighbors hired their neighbor on the other side. It has turned out poorly so far. I was, frankly, surprised they didnt know better.

"we hated to say anything bad about her as we're all members of the Country Club"

It has potential to be a good experience or a bad experience. It will help to have an architect involve either way. See if the neighbor on the other side of you is an architect.