Not a student of physics, I see.
You’re conflating atmospheric pressure with combustion chamber pressure. If a balloon will scoot like a stuck pig when released, from simple air pressure flowing out the open end… what do you think a rocket will do… just lay there?
Common sense.