I think it is absolutely hilarious (and infuriating) that in their opening paragraph they forgot to list spam blacklists, like themselves, as being perhaps the biggest hurdle for small companies and individuals running mail servers.
I ran a mail server for almost five years, and 9/10 of the issues I had to resolve were various anti-spam blacklists or individual hosts blocking our emails and then having customers complaining because we never emailed them.
There was never any reason given for the blocks, and our logs didn't show any outgoing spam. However all of these spam blacklists would accept a "donation" if you didn't want to wait the 24-72 hours for them to unblock you. Essentially it is a shakedown.
I've just given up self-hosting and I recommend others do the same. The issue isn't the spam, it is how much time you'll waste fighting the anti-spam blacklists and large email hosts. A job better left to e.g. SendGrid or SES.
The subject is self-hosting a _small_ email server. When you get into 10K/day range, you are bound to have false positive spam reports - both automatic and from pissed users - and this in turn can lead to RBL listings.
I've been running a mail server since 2004 and not once had I have any issues with RBLs. It's a low-volume server though, perhaps several thousand emails per month.
It's not even a matter of raw volume. The system I'm looking after sends just short of a thousand external emails a day but none of those are marketing or email blasts; it's all people mailing other people and reports being sent to specific addresses that have requested them.
The occasional marketing type bulk email is always sent through an external company. This means I never have to worry about getting blacklisted by those messages.
Note that the "addresses that requested them" out has a number of problems. I've worked for any number of shops that had quite putatively legitimate notification emails that they sent out regularly. Which at one point some person had double-opt-in confirmed....
... but for which there was clearly no valid reason for the mails being sent any more. As in, "the domain expired five years ago and is now a block hole", or "AT&T retired that domain a decade ago and migrated all its users elsewhere", or "that former email provider has had its domain parked on some Russian server for years".
If you're not periodically revalidating your email lists, checking for retired domains, and/or processing bounces properly, you are sending spam, I can guarantee it. Quite possibly not spam that anyone particularly cares about, but it's creating traffic for somebody's email system (starting with your own).
I have this problem at one company at the moment. We use gmail with spf and dkim, and have a low mail count and no mass marketing that I'm aware of. We're on two blacklists. One gives zero reason for being on it, and has a 'contact us for reevaluation' form which I've submitted. No feedback, no change, no idea what to do next.
The other blacklist suggests that it's our commercial site that's hosted with a provider who uses linode, and the linode CIDR block our company website is on has been 'banned with no recourse to remove' due to the huge amounts of spam it has sent. For that list, it apparently doesn't matter that our mail comes from google, not linode. When I get some free time, I'll try to move that website.
I've had some success contacting postmasters of smaller orginizations and explaining to them why blacklist $x really wasn't a good one and offering some alternatives, but this method is a bit tedious and prone to failure
I just set up Postfix to use an SMTP relay and then send them through a provider specializes in transactional emails (even though the emails I send through them are all hand written).
RFC-Clueless ('abuse' and 'postmaster'). Reading over their site http://rfc-clueless.org/ again I see that there is the information to the policies for listing, but I just didn't happen to follow the correct series of links to get there.
It's really not very clear with that site, and ironic that a list apparently named after people who aren't compliant with docs has such poor presentation of information. The documentation is clear... eventually, if you pick the right path. I'll take this on the chin as my misreading, but unclear documentation isn't helpful (particularly if you're juggling a few other fires at the same time...)
Edit 2: I forgot I removed the line where I mentioned that I thought postmaster was baked in :)
Edit 3: It looks like the domain I'm using with gmail has a screwed up abuse and postmaster. It's a domain alias, one of two. I made new mail groups for abuse/postmaster (the goog still monitors them, re: the link above) and noticed that only one domain alias was covered by them, when all other mail groups listed both. Sending test mails confirmed that setup. After a lengthy goog support chat, it seems that the way to fix it may be to remove and re-add the faulty domain alias.
I both love and hate these long chains of cause-and-effect that we get in IT...
Yeah :/. I have tons of operational knowledge of email and have been running my own email setup since 1998. My email servers were ever only used for email I typed manually by hand (due to being dumb I never even used shortcuts to auto type repetitive responses to customer messages). Then, one day a couple years ago, I realize no one @yahoo.com is getting my email anymore. Talking to Yahoo, I get barely any response and certainly no explanation. I can easily believe that my entire IP block was cut for being a hosting provider.
I now pay some company to deliver my personal mail, which is just dumb, and clearly isn't caused by them being better about spam (as I sent no spam): it is caused by a militarized anti-spam-crazed group of people who have managed to build a system that essentially requires an oligopoly for email to function. A lot of these people don't even consider it a problem to use "report as spam" as a ways to punish companies for things that aren't even sending spam (which is something I only bring up as it demonstrates the "militant" comment).
The reality is that an open internet where people can talk to each other without going through third parties requires a world in which you, heaven forbid, receive some spam. It is actually a similar construction as the one for why we should have network neutrality: large internet carriers in the world of email should, if you really believe in open access, not be allowed to say "these people can't send mail because they are too small and not paying someone". We all need to accept at least a little spam to guarantee everyone can send mail.
If this means that you have to get better at dealing with spam, as some of the heavy-handed techniques people have put in place for dealing with spam are no longer possible, well: so be it; and if that means you need to turn off the notifications for new email on your phone to keep from getting angry when you receive some spam (which tends to come up as a reason to hate spam), I hope people realize that they caused their own reason to be upset, and that getting notifications for anything is probably unhealthy, at least for you, anyway.
Sadly, the large companies who have slowly come to "own email" have no incentive to break this cycle, whether in action or in words, as the current state of affairs is what is giving them their power: whether they build solutions that can solve spam in a way that is both fair and federated, or they just try to educate users about how to better deal with spam and decrease the stress in their lives (including telling people to stop letting email interrupt them), it will just harm them. This doesn't sound like something the private sector can solve.
> A lot of these people don't even consider it a problem to use "report as spam" as a ways to punish companies for things that aren't even sending spam
I get a fair many spams that have "unsubscribe" links, or even that look like newsletters (for companies or organizations of varying legitimacy). I'm always pleased when they come through one of the major mailing list providers that provides a separate "report abuse" link, with which I can report that no, I don't just want to unsubscribe, I want to report that I never subscribed in the first place so that the list itself gets terminated.
If it's possible for someone to be subscribed to your "newsletter" without having explicitly consented to doing so (and in the process proving ownership of the subscribing email address), you are sending spam.
> If it's possible for someone to be subscribed to your "newsletter" without having explicitly consented to doing so (and in the process proving ownership of the subscribing email address), you are sending spam.
A problem with this attitude is that most people have no idea whether or not they consented. Perhaps they consented in 2003 but don't remember. Perhaps they consented yesterday but want to revoke consent because they're too busy right now to read your newsletter. There's a reason why Gmail silently converts your spam report into an unsubscribe if the correct headers are set. Too many PEBCAK false positives.
No, the problem is figuring out whether or not to punish the sender.
If it's obviously spam, the sender's IP needs to be blacklisted, his hosting account terminated, etc. in order to protect other people.
If the recipient just changed his mind about whether he wants to receive a newsletter that he explicitly signed up for less than a week ago, there is no need to punish the sender. The sender just needs to be notified that the recipient unsubscribed.
I don't know about you, but if all the social networks, instant messengers, and "we're gonna replace email" startups in the world went dark for 24 hours, I probably wouldn't even notice. They're all but useless to me, after all. But if my email went down for 24 hours, I'd definitely make a big fuss about it. Ditto if someone makes the wrong decision about which senders to block.
If the sender is sending mail that causes significant numbers of people to flag as spam, even if those people signed up and confirmed just a week ago, they need some reminder that their behaviour is not acceptable.
Very many senders stretch the boundaries of what they send.
I disagree with your description of what is / isn't spam. Any automated message-generation system should be exceptionally open in sorting out how to detect lack of interest in continued communications. Recipients simply cannot and will not sort through anything more complex than "delete" or "this is spam".
Many major email providers do provide some form of full-loop feedback to bulk senders, and aren't unreasonable in their policies. The expectation that all email senders be perfect isn't reasonable (and generally isn't applied), but reasonable diligent effort really must be followed, and far too often isn't.
The model I've strongly endorsed for a long time is to simply apply a mail acceptance criteria that's scaled to the level of non-problematic mail originating from a sender. If most of the mail meets acceptability tests, most of it is accepted. If most of the mail doesn't meet acceptability tests, then most of it's rejected. If the problem's modest, the rejections are temporary (e.g., re-try in a bit). If it's severe, the rejections might be made permanent.
Poorly-behaved senders (those trying to re-transmit the same message before a reasonable retry interval transpires) are penalized harshly (all connection attempts are refused for some period).
RFC 2821 suggests an initial retry delay of 30 minutes, and up to 4-5 days for delivery attempts.
A typical exim4 retry configuration "specifies retries every 15 minutes for 2 hours, then increasing retry intervals, starting at 1 hour and increasing each time by a factor of 1.5, up to 16 hours, then retries every 6 hours until 4 days have passed since the first failed delivery."
Arranging with specific high-interest peers for expedited delivery, especially from individually trusted senders, might also be a useful thing.
Note that email is not guaranteed to be reliable or instantaneous. And as I've commented elsewhere on this thread, the problem now is that both you as an individual server admin have to deal with everyone else on the Net. And they've got to deal with you....
It's the sheer drudgery, more than anything else, of that, which doesn't scale. What's forgotten of the "golden age" of the Internet (1980 - 1992 or so) is that there were, comparatively, very few hosts. Dozens initially, a few tens of thousands toward the end of the period. Individual accountability largely worked. And while some individuals had direct connections, it was largely universities, a few employers, and government agencies who played the role of ISP / mail service provider. The system wasn't quite so chaotic as is commonly thought.
I recently saw a talk where someone held up a physical copy of the directory to the Internet circa 1985 or so. And it actually had everyone listed not once but 2-3 times -- by name, by organization, and by email address, or something like that. A not-very-substantial document, really.
Today's equivalent would have many millions of entries, I suspect.
Maybe it'd be useful to have a standard for mailing list confirmations that would let the mail client keep track of what you actually sign up for and/or confirm as opt-in.
As a user I also appreciate the ability to report abuse to someone I trust to deal with it.
But as someone operating various mailing lists for customers, the vast majority of the time when we get abuse reports from customers that "never subscribed", we have detailed history records showing that they did actually specifically take action to subscribe and confirm. This is consistent with pretty much every other thing - for every action in the systems, there are always customers that insists they have done no such thing.
Until we show them the audit trail.
Users are notoriously bad at remembering what they did. Which is not so odd given that most of the time the action is an inconsequential spur of the moment thing that they do or don't do as a matter of reflex and never think about again. But it makes "Report as spam" horribly abused.
Maybe the world you describe could have been possible, but so long as it is possible to send email by a bot we need bits to filter it.
And yes I Mark things as spam that may not meet the legal definition of spam, because companies tend to start sending you all sorts of crap once you sign up for their service.
In my ideal world you wouldn't have to pay a third party to send the email you would have to pay me for me to receive it.
It's never "some" spam. On my own mail domain, I get slightly more spam than wanted mail after spam filtering. If I didn't filter it would be tens or hundreds of spam items to actual mail.
The notifications question: well, in some ways people have moved to IM because it offers lower latency than email, generally better antispam, and usable notifications. Of course this is because all the IM solutions are centralized and can ban abusers more effectively.
I've only ever encountered one truly "militant" person against spam, and he refuses to accept mail from gmail because of NSA surveillance. Everyone else wants to avoid rejecting legitimate mail as much as possible while keeping email usable at all.
Lack of feasible antispam mechanisms is one of the things responsible for the decline of USENET.
I've thought about this. All the spammer has to do is 'buffer overflow' the capacity of the heuristics algorithm. If google's anti spam can only worry about X number of unique spam message signatures write a script the generates x + 1 unique human readable spam messages. Not easy but probably possible, what's a good guess for X?
I've also had similar issues in which it appears that email from me was being blocked purely because I was within a particular IP range, not because my system was actually sending any spam.
I've been running my own mail server since 2001 (and other servers before then). The only time when I had a problem with blacklists was when I used a colocation/server provider that didn't give much care to RBLs blacklisting his entire subnets (so, check your provider's reputation before signing up!).
Except for this problem, blacklists have never been a problem for me. I guess it's different if you send newsletters or other mass-marketing mail.
Same scenario here. Blacklists have never been a problem for me, but they have been incredibly useful for keeping the spam out. Email wouldn't work without them.
I've been running my mail server since 1997. I've got once blacklisted due to someone spamming URL to my web site around. It seems that your email can get blacklisted, even if you don't send any email.
I've run one since 2000 and the same, I've only had one problem and that was because I (stupidly) ran a TOR node on the server to play with and got myself blacklisted for a while.
I have a number of Customers who have been self-hosting email since the late 1990's. The only blacklist that I've regularly had any problems with has been one-- ahem-- "fishy" company who, at least in the past, seemed to run their blacklist like a protection racket.
Yeah, you would have to be stupid to self-host email these days. If you're an individual person, it's way too time-consuming to set everything up properly. If you're a company, there is a very high risk that you'll get put on some anti-spam blacklist or other for doing one of 10,000 harmless things (or even not doing anything wrong at all, but just sharing a block of IPs with a bad guy). And then your mails will get silently dropped, and you might never know that it happened. You just... won't get replies from customers or business partners that you were expecting.
The anti-spam zealots ruined email for everyone, and they don't even have the honesty to admit what they did. "Hilarious and infuriating" is right.
And everything works just peachy. Until it doesn't.
The problems are that:
1) the determination of "works well" or "doesn't" is almost entirely out of your hands. A DNSBL or major email service provider decides they don't like the tilt of your kilt and it's titsup.com for you. Been there, done that, shredded the t-shirt.
2) How much of a problem this becomes is a matter of who's inconvenienced, how much, and what alternative contact channels they've got. Generally, though, it's something of a PITA.
Things can work well for a long time. Or not. You're just never quite sure when they'll blow up.
A lifetime of maintenance. Not in terms of hours, but it's something that requires monitoring. Something that will fail at inconvenient times. Something that requires security updates.
And if that's sendmail smtpd, it's not "a few minutes" to configure if you're unfamiliar with it.
It's opensmtpd and my seven-line config took literally minutes to initially write back when I first did it years ago, with no prior experience.
It's required basically no maintenance at all from me. As for monitoring, I check my mail daily anyway and I will notice it if it stops flowing.
A few weeks ago I actually moved and took down my server. I took an old netbook and made it a new, temporary server. Again that took me literally minutes.
Thankfully smtp is a robust protocol so if something fails, you normally have a couple days to fix it before mail starts dropping.
But what do you do when you're using third-party email service and they decide to kick you out all of a sudden? Happened to me with Google. Happened to many other people as well, also with other providers.
I ran my own debian-exim for about a decade until I got tired of the care and feeding. opensmtpd does look suitably low-maintenance and I might pick it if I ever wanted to do this again, but it was released after I gave up being an email admin. I still have the mail domain, but having the same email address for sixteen years means ending up on a lot of spam lists.
You run your own server so you can easily set it up to accept any email matching some simple rules. This means you can give a different email address to every online service you use. If one of them starts getting spam, blackhole it.
I have been running my own email server this way for about 10 years and I have never needed any kind of antispam measure except a simple .procmailrc.
It's both. The spam war causes a lot of collateral damage. For a decade I ran a mail server for a small client. Yes, they ran a ~15k membership newsletter, yes some cranky people would flag as spam since they were too lazy to click UNSUBSCRIBE, but that's small business for you.
Self hosting was once fun; now I hate it. Now I would only ever recommend getting a business gmail setup and using something like Mail Chimp. Not because I like either option, but fighting with blacklist operators and arrogant mail admins jaded me and now I refuse to support email servers for people.
Don't get me started on the web hosting side of this equation.
The raw mail flow may be useless without filtering, but blocking legit mail from a tainted IP addresses (tough to avoid anymore for the small business) is simply counter-productive. Our job as mail admins is to send and receive mail. We should let the receiving mail server tag the mail and let the mail clients filter.
We should never, never, never reject a message during the SMTP transaction. Just never.
There should be a new Fry meme: Shut up and accept my SMTP connection!
Allowing clients to filter sounds great until you look at the volumes of spam that major (and even minor) email service providers handle. SMTP-time rejection is the only feasible way to handle it.
What you're seeing (and a conversation I've had numerous times, particularly with some individuals who seem to think it's all some Vast Conspiracy Against Personal Communications) is simply the challenges of dealing with email on a scale where you've got more peers than most people have email contacts -- where a peer is another peering email system.
It turns into a reputation management system. And it's a lot easier to deal with those reputations when you've got a handful of major public providers (Google, Microsoft, Yahoo, Aol, Inbox, etc.), and a few thousand major corporations. Once you get outside of the Fortune 500, even corporate emails get hard to deal with. At Krell Power Systems, we had a customer who required us to provide our email server IPs (already listed in both SPF and DKIM, natch) before they'd allow mail from us. The fact that they run a few machines powered by U-235 and are concerned about SCADA threats might have something to do with that level of paranoia.
But allowing all spam through, storing it, and relying on users / client software to filter it is expansive and quite error prone. The good thing about SMTP-time rejection is that it's unambiguous: any well-formed server will recognize that it's failed delivery, and in most cases the message is immediately bounced back to the sender. Accepting email and later trying to determine whether or not it's legitimate risks spoofing, Joe-Jobs, and silently-lost messages. That's actually far worse.
Much as I wish everyone could simply run their own servers, with systems as they stand now, it's just not possible.
The real problem for a small company mail server isn't setting up a respectably secure operation: set up SPF and DKIM and you're good to go. The problem is maintaining deliverability for anything more than very casual usage of mail into a very complicated ecosystem. At any point in time a random switch can flip on some heuristic somewhere and suddenly a large mail provider will reject you, or you're on a blacklist that provides no explanation or ability to remove your mails, and that isn't isolated: it will spread throughout the ecosystem, and leave a stain that can linger for a long time.
I've run a few-thousand person newsletter mail list for more than ten years, and fairly recently switched to SES because it's simply not possible to guarantee deliverability even at that modest scale running it yourself from your own servers. If you're not hooked into the ecosystem then you have to fight ongoing battles that only a professional in the mail deliverability business has time and connections to understand and effectively win.
Even then, you have to be very careful: don't mention sums of money (e.g. censor all discussion of research funding), don't trigger dumb spamassassin rules (e.g. sentences with lots of long words in a scientific paper), don't talk about known trigger words (e.g. no scientific articles on the role of growth hormone in mouse aging) and so forth.
It is crazy. The absolutely rational response to all of this is to outsource mail delivery.
I set up my own Postfix server about a year ago, because I wanted to send newsletters to my users and have an "official" mail box for support requests and such. I used a guide[1] to secure it against spammers. I don't use spam assassin. In the year that followed I received exactly zero spam messages. There have been plenty of attempts to send spam using my server, but the postfix filters intercepted all of them.
To make sure that my messages are not treated as spam by the major email providers, I checked that my VPS IP is not in any of the spam blacklists and configured SPF and DKIM records. Gmail spam-flagged the messages for a while, but after a couple of months it learned they're not spam. Surprisingly, other email providers (yahoo, ms, aol) never batted an eye.
Fwiw I've had better luck without SPF records than with them (or at least without hard-fail SPF records). As far as I know, the lack of SPF never resulted in lack of delivery (and I have no delivery problems to the big providers). But my previous setup with a hardfail SPF would cause my mail to be rejected from various people's forwarding setups, especially when forwarding to corporate or university email addresses. The problematic scenario is when they have their own domain hosted somewhere with mail forwarding (e.g. via Dreamhost), that sends it onwards to their real email account hosted by an institution. The final-destination email server then sees the forwarding server as the source of the message, and bounces it for failing SPF.
Openspf.org has a page about that [1] which recommends email providers allow their users to configure authorized forwarding sources as exempt from SPF checks on their incoming mail, and default to not rejecting based on SPF when users haven't configured the preference. But in my experience few institutions follow this recommendation.
Once I ditched the SPF records, I've had no deliverability problems running my own mailserver.
I got flagged as spam pretty reliably by GMail, but I eventually discovered it was because my mail server was delivering mail to GMail over IPv6, and I had never set up proper records for my IPv6 addresses.
GMail was the only one I had that problem with because few mailservers are using IPv6.
Hmm does the 'mx' flag in SPF include the IPv6 addresses too?
The RFC seems to suggest it does but the website clearly says "All the A records for all the MX records":
http://www.openspf.org/SPF_Record_Syntax#mx
Gmail seems to accept mail from an IPv6 mx when the SPF record is "All the A records for all the MX records" (the headers says spf passed), but I'm not sure how others interpret it.
Is it better to explicitly list the IPv6 addresses in the SPF record?
I read him/her as referring to the mail server lacking an IPv6 reverse DNS address. This is a fairly common oversight, since IPv6 connectivity is getting more common, but many providers' control panels allow only easy self-setting of the IPv4 RDNS. Many mailservers are set up to reject mail from senders who lack an RDNS (and/or have one that doesn't forward-resolve back to the original IP), so in that situation, if you connect over IPv6 you may get rejected. The solution is to either disable IPv6 (less preferred option), or set a reverse DNS name for your IPv6 address (more preferred).
I have a work-in-progress guide for folks wanting to host their own email servers which provides a step-by-step process for taking a Debian (or Ubuntu) server and building out a secure install of Postfix and Dovecot to provide TLS 1.2/PFS enabled SMTP, IMAP, and webmail services along with competent spam filtering with low false-positive rate and all the required DNS and other settings to ensure your mail is accepted by GMail and the like.
I've never really posted it up, but I feel like if you've read this article and are wanting more, this might help you. It's work-in-progress because I've been sidetracked from finishing the Client piece and also creating a Dockerfile that generates a group of self-contained images configured as recommended, but all the steps for the server are complete and tested.
I am the author of http://gogs.info/books/debian-mail/chunked/ and I am working on something similar (I have switched to Dovecot and wrote a nice GUI for managing users and domain) but in the end it will be a set of Ansible scripts.
The things I did different:
* Amavisd-new instead of dspam and OpenDKIM
* SQLGrey instead of Postgrey
* unbound instead of BIND as a caching DNS server
I have been thinking on adding PolicyD for rate limiting accounts but then I get a lot of overlap with other services. How are you satisfied with postfwd?
Nice, I hadn't seen that book before but am glad to see there's some more detailed documentation out there on this subject. Setting up an email server can be pretty confusing even for someone who's experienced just because there's so many moving parts and its very finicky.
I haven't really worked with Amavisd, SQLGrey, or PolicyD before. I've now switched to using unbound instead of BIND myself, but this hasn't yet been reflected in my instructions since I kind of stalled on updating them.
I like postfwd quite a bit for how I'm using it, which is to do hybrid greylisting based on DNSBL weighting. It allows me to reduce latency for inbound messages that are free and clear rather than greylisting everything. I don't really do any rate limiting or anything like that with it though, so I doubt I'm using enough of it to really form a clear opinion.
I haven't tried it, honestly, but I'd imagine it'd work fine. The SQL usage there isn't very complex, so I'm pretty sure the syntax would even be exactly the same between the two for the few things I'm using it for. There is some difference in the connection strings though. It looks like Gentoo has some good docs on this in their Wiki at https://wiki.gentoo.org/wiki/Complete_Virtual_Mail_Server/Po...
Good luck, let me know how it goes, and I'm happy to accept a pull request to add additional instructions for Postgres if you feel up to adding them.
I spent a few weeks this past fall trying to configure spamassassin to be useful, but I was getting up to 30 pieces of spam a day and it was getting obnoxious. I configured my setup to use greylisting and it wound up being significantly more effective. I get maybe one piece of junk a week.
I'd like to do that, but greylisting is typically done at the SMTP level -- before the messages are anywhere close to SpamAssassin. Can I ask how you're accomplishing this?
You can run SpamAssassin at SMTP time in the DATA phase. At least, you can with both Exim and Postfix. I assume other MTA's support that too. You can run Greylisting at the same stage after SpamAssassin has been run and before the message has been accepted.
I'm currently using Fastmail and might be too lazy/busy to switch as I'm currently a trial user. The Android + Web app is better then K9-Mail and Roundcube IMHO.
If I did switch back to running my own show, I'd do Digital Ocean + Mailinabox via Docker.
Mailinabox aims to do all the things right out of the box:
We had a bit of a laugh about that, because we don't really have any secrets. The fact is, it's just a ton of hard work - new spammer tricks, new interoperability woes (ask us about SSL negotiation with other people's MXes some time, but make sure you provide the alcohol), and just trying to get the best possible performance out of the hardware and the software so we can handle both the person with 500,000 emails in their Inbox who never moves anything and the person with 10,000 folders each with 5 emails in it, and everything in between - and provide a fast loading experience for every single user when they hit the site, even though there isn't space in memory to hold that many mailboxes cached.
Great. The thing I like is that you get maximum flexibility, and good web interface. For example: I run a mail server which handles mail for some subdomains, and the root domain mail I route through them. Every time I give an email out to a company, I name it company-name@one-of-many-short-fastmail-domains.com. I never have to worry about spam from those email address, as I can filter and cycle those at will.
I do something very similar, but instead of using the company name (or something close to it) I use a long random string instead. It mainly prevents someone from 'company hopping' (i.e. changing 'facebook' to 'google'). The chance of that happening might be small, but I figure I may as well as it isn't really any more effort
And I secured it down appropriately whilst also using multitude of external scanners to verify what I thought I did. It was somewhat problematic, mainly with the AUTH between dovecot and postfix.
And then I do a RBL search. Some idiot spammer did the dirty deed back in September. 3 RBLs, all lesser known ones. So, I responded as they requested, with a justification of why I should be removed. I did that today around noon.
I'm now on none of them. So the whole "shakedown" issue, at least for me, seems not to be the case.
We[1] have a multi-domain mail server container with imap, pop, GUI admin, webmail, and spam lists that is basically ready to go, based on LXC.
You can launch the container in seconds and should be the fastest and perhaps easiest way to have a fully functioning mail server. There is a starter guide [2] and even a [3] video to get you going.
However a mail server unlike most other apps is not only complex to install - which we try to address - but also complex to run.
For most if not all users mail just can't fail and this needs a fair bit of knowledge of dns, smtp, spam prevention, security. I think in all but the most committed cases it can prove a bit overwhelming for the average user.
Like others have commented, I've been running a small email service for several years. The key point here is "small" meaning serving a few users for mainly personal mail. It's a very low volume affair, no "mass mailings", newletters or anything of that kind.
I'm using Exim only because it was the MTA I knew how to configure. It's a simple setup, allows no relays, uses TLS when possible, a few bad actors are RCPT blacklisted, that's about it. I use a webmail program I wrote to retrieve and send mail. It's all quite basic.
Have had very little trouble with spam or or being blocked, which may be a benefit of keeping such a low profile. All in all, I've enjoyed the freedom and privacy of running my own email server. I suspect the secret of success is keeping it small, personal and inconspicuous.
I run a small server with a few dozen users, the only problem I've had is verizon.net bounces all messages claiming spam from my IP, and steadfastly refuses to whitelist me despite numerous attempts. Always just a form-letter response, despite having checked all the boxes (SPF, DKIM, rDNS). Luckily I'm happy to live without being able to email people on @verizon.net.
I'm just not willing to store all my personal and business communication with a 3rd party. It goes against the entire purpose of the standard, and I figure the day it becomes impossible to run my own server is the day I no longer use email.
Guide for small server operators is to put a bullet in the server. Buy O365/Google/etc.
I ran a smallish email system for a year and a largish email operation for several years. It's a lot of work to do things right and the benefits are really small. The security related issues presented are very niche -- if you are concerned about data leakage for confidential discussions leaving your datacenter perimeter, you probably need to look at alternate solutions anyway.
Really? Isn't data you give to a 3rd party their data, so they can give it away without penalty? With colo you can have monitoring and pick a friendly jurisdiction. (Market opportunity for a full featured Office 365 competitor that operates entirely in say, Switzerland, Iceland (?) or maybe Russia for a certain demographic.
Absolutely not. If that were true, the 4chan guy would have been in jail for possession of illegal material posted there.
In the US, the Electronic Communications Privacy Act and Stored Communications Act provide protections here... In the event of a civil or criminal litigation, a subpoena delivered to Microsoft will be redirected to the customer or customer's counsel to respond to or quash. If the customer is non-responsive, Microsoft may be compelled to produce data.
There are situations where a criminal investigation will result in a search warrant directed at an individual or organization. Search warrants are a different ballgame... The standards are higher and the cops show up and take everything. Keep in mind in that situation, the cloud has no impact in terms of data privacy -- the police will literally knock down your door or pry the colo cage open to seize what they after.
If the nature of your business requires additional layers of secrecy or offshore servers in Iceland, you should not be using email.
I ran a mail server for almost five years, and 9/10 of the issues I had to resolve were various anti-spam blacklists or individual hosts blocking our emails and then having customers complaining because we never emailed them.
There was never any reason given for the blocks, and our logs didn't show any outgoing spam. However all of these spam blacklists would accept a "donation" if you didn't want to wait the 24-72 hours for them to unblock you. Essentially it is a shakedown.
I've just given up self-hosting and I recommend others do the same. The issue isn't the spam, it is how much time you'll waste fighting the anti-spam blacklists and large email hosts. A job better left to e.g. SendGrid or SES.
PS - We sent tens of thousands of emails a day.