Going along with Dave,

Go across the grain at an angle, there is less risk of tearout in the grain.

I also think there should be some texture when hand-planing. If I want it smooth, there are more effective means.

The thing to watch out for with oak, remove a very thin shaving at a time and keep your blades SHARP. Oak can be tough, it might chatter or want to dig in with the grain, causing tearout.
But it will finish nice when you get it figured out. Hand planed oak can be one of the most attractive things