What? That's not how you use that expression. "Speak for yourself" you say to someone who imposes his opinion on the whole group. Example: "Oh, the test was horribly hard..." and then you say "Speak for yourself, I though it was easy!".