first off, RFK didn't say that about himself, he used it in his eulogy for his brother, and in the second place, he did NOT write it, he plagiarized it from the original author. IIRC it was George Bernard Shaw who originated that quote.
use whom when it's the object of the sentence (or clause). prepositions are not the determining factor.