I beg to differ.
Notice how often it is said or thought with many different peoples:
"They look (fill in the blank)."
"Jewish" is ONE of the terms used to fill in the blank; though (they) do not all look alike, there is something about (them) that make (them) look "jewish".
As another example: Canines.
One would NEVER confuse a French Poodle with a German Shepherd; though they are both DOGS.