Have you tried your local bank?
I was able to obtain my construction loan and Mortgage all at my local bank. Which made things easier to deal with. Surely New Jersey Has bankers who will help you without going to some people you don't know or work with.
Are you being your own contractor??