The north side would be effected the most I suspect. You may extend the climate by building a temporary green house of kind around the building. This would aid in heat during the day and protect slightly at night. You may want to vent it periodically as well. On the south side it will get very warm during the day. Maybe something to consider, even go out to have morning coffee in the warmth. I have often thought of building some such "green house" against my house. Plans keep changing and not everyone is in favor of such things. It would also protect the clay wall from the elements.