"For laymen's purposes it pretty much is, though. When was the last time anyone on Linux/OSX got some adware / popups?"
For OS X, one or two months ago. Do a web search, the times are changing for Mac security.
"I've also never heard of antivirus for Linux. Which doesn't mean there aren't viruses, it means it's not a concern on the most part."
No, it just means it's not your concern. When online crime has become a business, it makes sense to try to protect yourself. Major AV companies have a product for Linux.
They are also, for the most part, add-on (and readily removable) parts of the system. Usually some network service or web app vulnerability.
There have been a few kernel-level exploits, most of which are DoS vulnerabilities, though a few are privilege escalations (meaning: paths to root or full system ownership).
Still, as a whole, the modular architecture and high system transparency of Linux means that it's far easier to avoid, detect, and recover from attacks than Windows. Mac OS X is slightly less protected, but only somewhat.
Contrast this to the gaping security whole that remains the Windows shell, the tightly integrated default Web browser, the "document as application" model, various unsecured default services, very low system transparency (/proc, /sys, strace/ltrace/dtrace, netstat, etc., are wonderful), and, oh, say, the fucking impossibility of deleting open files, and you've got a massive security migraine.
Still.
And, yes, Virginia, there's antivirus for Linux. We run clamav on our servers to keep all those damned Windows viruses from proliferating by way of our services. But viruses as an attack vector for Linux itself? No.
It's great as an additional protection in your mail setup. My personal domains are few and email accounts not widely exposed. ClamAV caught 7 viruses in 2010, though I get about 20 spam emails per day. Since I never check the imap folders for spam, it might be that some of them were not caught by ClamAV, but by spam filters instead.
Anyway, for 7 virus emails per year I couldn't justify 100+Mb memory requirement on my 512 linode, so ClamAV no more.
Maybe you don't remember the root-kitted Redhat boxes in Korea which were, for some years, responsible for a surprising volume of spam.
Linux distributions then started shipping with un-needed services turned off, and increased broadband meant home machines were attractive targets for botnet malware.
In the past anti-virus on Linux tended to be used by people with a Linux mail server and MS Windows clients.
But, to bring it back to this particular discussion: It'd be fairly easy to wrap malware around a Mac OS X software. The user would need to click and give it permission to install. But how many Mac users run as admin and would just click the warning anyway?
rkhunter and chrootkit are two free malware scanners (also in ubuntu repos). There's also Avast for Linux. Avira have a free scanner (no GUI though) etc. If you actually look there are plenty of antivirus and antimalware tools.
I've also never heard of antivirus for Linux. Which doesn't mean there aren't viruses, it means it's not a concern on the most part.