Clay and chips have poor resistance to shear - such walls are infill only, not meant to be load bearing. If you wish to stick frame a wall, use Larsen trusses, etc, that structure needs to be able to bear the loads all by itself.
Adding sand to the mix would give you greater shear resistance, and if you add enough, pretty soon you are building a cob wall, which is fine without a frame, but going 2 stories means a lot of material lower down - maybe 3' thick walls at the foundation. I've heard recently of people adding pumice to their cob mixes to improve it's lay-ability, and this apparently greatly increased the amount that can be placed with slough-off.
My $0.02