Only issue is that would require force from a ballpoint pen. Optical scan ballots machines (at least the ones used in my voting district) require a bold black mark from a felt-tip pen.
But I like the idea of being able to check you vote using a serial number you take with you. Perhaps a tear-away tag would be better.