have you already tried some banks and been denied, or asking in general? I'd start with local non large name banks, credit unions, etc.

Have you thought about using companies like Owner Builder Network? Or the other one called U Build It? I only know one person that used the first one and they were happy. Obviously, you have building/contacting experience, but they are able to get bank approval since they are more reliable in the eyes of the banks.. Or perhaps you could call your local offices of those two companies and see if they are willing to share a few banks they use?