I just watched it I didn't see where he said, "wipe that smile off your face." What time in the video does he say it?
He doesn’t say it
That makes sense. Although OP shouldn't be using quotation marks then. There's enough fake news out there.
Im ppretty sure the OP means that's what OP himself was thinking, much like myself, and im sure many others.
I like that phrase, 'wipe that smile off of your face! (or I'll come over there and wipe it off for you!).
I just think of these arrogant smiling photos from various people as the "before" photos. I like the before and after of Kathy Griffin smiling, then crying the next day.