It is pretty simple. White people have always been difficult to enslave. Black and brown people have always been easily enslaved. When you want a world populated only with the elites and a slave class you need to get rid of whites to have any chance at it.