I can see it taking about 40 or 50 years before an area had recovered economically from the civil war enough to build new barns. By then, perhaps the rising tide of prosperity in the area floated everyone's boat enough to build a new barn.
It sounds like the grandchildren of the veterans were building these barns as they took over the farms. (at least that generation as opposed to ones who were children during or immediately after the war.)
In our area, times were tough until 1910-1920. And then the boll weevil hit in '26 or so and wiped it all out again to the point that my grandparents didn't even feel the crash on Wall street -- it was already bad.
What was the main cash crop in that area during that time?