Much too restrictive maybe, as it depends on that restrain you mentioned (positive contribution to income or payroll taxes).
Some institution in the US must have a list of all legal citizens i.e. people eligible for a U.S. passport.
In times of digital databases, it is, as one quick from-the-hip example, no problem to have every citizen register in person or digitally in the county/state of their residence before an election.
These registrations can easily (and digitally) be compared to the list of U.S. citizens, and then the only thing you'd need is a list of those registrants at the respective polling place, where one's name would be crossed off, once you've voted.
Just as one quick example … which much smarter people could certainly perfectionize to make it as cheat-proof as possible, without introducing too much of control over citizens' current place of residence or too much costs.