Depends.
If they 'stacked' short posts on top of tall horizontal members (on one side?) then it could list to one side by quite a bit... I've seen this in a relatively new frame out of white oak - short post with horizontals stacked on top of it - and then short posts on top of that, etc.
A 12" tall oak timber, green, will shrink a ton, enough to push things all askew.
But, I'd have to see it, too.
3" out of plumb isn't much - depending on how tall the building is.
If I recall, WTC 1 and WTC could sway 3' on a windy day.