The Fabian socialists worked to have "mandatory schooling" adopted in every state Constitution. In Massachusetts, when "public schooling" was first imposed, parents protested and the police were sent to forcibly remove children from their homes.
Of course, the children of the "elite" at the time were not subject to this.
To end public schooling, you will have to amend every state Constitution. I agree with you the solution is to refuse to hand over your children to them. They can't demand funding for empty seats.
If you insist on using the "public school system" because "you're already paying for it", you are part of the problem, not part of the solution.