I'm surprised no one has mentioned the HABS/HAER collection of the Library of Congress. (Historic American Buildings Survey)
Accessible online, it is a fantastic and comprehensive resource including many early homes, many with detailed drawings - some right down to moldings and hardware.
<
http://lcweb2.loc.gov/ammem/collections/habs_haer/>
Pete