Greylisting only delays mails. Proper spammers just use ISP relays and
then they will try forever. Or they will just nicely do the full SMTP thing for the first message and try again later, or stall sending to you as the 450 is recognized and spam run you again later. So many easy ways around it and it only causes annoyance.
I don't know anything about proper spammers. Greylisting has reduced the amount of incoming spam significantly, probably at 90-95%. Of course there are spambots which play around greylisting, but they aren't yet that widely used.
On top of that, I guess you have whitelisted large mail providers like gmail who try to send a single mail from several IP's, thus hitting your greylist over and over again, and then just giving up? :)
some greylisting packages have their whitelists already pre-filled in the distribution package, so it works quite well (we used http://policyd.sourceforge.net/ , and I have no complaints about this software, and neither do the users.