This story doesn't mention that Spamhaus is protected by CloudFlare and we took a beating from this attack. At some point I'm hoping the full technical story about how the attack morphed from our infrastructure to Internet infrastructure can be told.
At some point I'm hoping the full technical story about how the attack morphed from our infrastructure to Internet infrastructure can be told.
See poorly configured DNS servers and ISP's failing to configure their networks properly - so traffic with a source address which is not part of your allocated IP block is not allowed to leave your network. It is not that hard!
The Internet Infrastructure is working as designed.
If you run a DNS server - it is your responsibility to maintain and protect it so that it cannot be used to attack others, and by doing that you are helping the 'Internet infrastructure' remain intact as designed. By not doing this you are helping the 'attackers'
BCP 38, RFC 2827, is designed to limit the impact of distributed
denial of service attacks, by denying traffic with spoofed addresses
access to the network,
> See poorly configured DNS servers and ISP's failing to configure their networks properly - so traffic with a source address which is not part of your allocated IP block is not allowed to leave your network. It is not that hard!
It may not be that hard to set things up this way, but very few ISPs configure their network with this restriction.
"But we're up - they haven't been able to knock us down. Our engineers are doing an immense job in keeping it up - this sort of attack would take down pretty much anything else." --Steve Linford from the BBC article.
If Spamhaus' Linford is quoted accurately, he's kind of full of it. The NYT article gives more detail about CloudFlare's involvement.
CloudFlare protects the website, which has no bearing on Spamhaus' actual service which are run on their DNSBL servers in various networks around the world.
Can you also describe exactly what the connection between CyberBunker and the attack is. Is there any indication that the hosting company is actually involved? It seems dubious but of course there are defunct hosting companies that have done such things (Russian Business Network comes to mind). However, this host does not seem shady in comparison to RBN.
It has an actual location. The name of the owner is known. It has evidentially been involved in legal disputes so it is on record with the government.
Much more likely is someone using the hosting system for something nefarious is retaliating against spamhaus. I don't think the hosting company should go down for that.
Thanks. It was totally unclear from the original article that it was in anyway actually a bad actor. It seems like these types of hosts are double edged swords.
It's not censorship; it's simple property rights and basic freedom. I don't consent to someone else using bandwidth that I have paid for, space on my hard disk, or my attention and time, for advertising. Hosting is cheap enough; they can get their own damn website and opt-in mailing list. And spamhaus subscriptions are completely voluntary and optional.
That being said, the problem with many BL's is that they are run by incompetents or extremists. They usually either end up blocking things that are not spam by accident (see lists of supposedly dynamic IPs), or block whole subnets (sometimes entire ISPs) to try and "teach them a lesson." or blackmail them into fixing the problem.
> That being said, the problem with many BL's is that they are run by incompetents or extremists.
Unfortunately, that includes Spamhaus.
It's a bit sad to see how many companies will blindly support such entities because they've "heard" that they somehow help fight spam. As someone who's had issues with them because of their badly configured hosts and shady practices (e.g. using domains previously used by mail providers as "spam honeypots", meaning anyone who emails someone with an old address can be banned [all content mailed there is considered spam, regardless of what it actually is]), I am disappointed (yes, looking at you cloudflare).
AIUI, as mhurron says, what Spamhaus does is publish a list.
The nominal purpose of that list is to identify spammers, so that people who wish to filter out spam can be assisted by that. People do, in fact, use that list, to filter email. The email recipient wants to be protected from spam, so the recipient's ISP attempts to perform that service, and Spamhaus contributes an opinion that the ISP takes seriously.
So, in practice, if Spamhaus adds you to their list, many many users will stop seeing email from you. Spamhaus has a great deal of power to mostly-silence domains.
I have no reason to believe that Spamhaus uses their power for anything other than good. But it's not quite as simple as "do they censor? no".
I think, basically, Spamhaus should be thought of as the Internet's equivalent to a Credit Ratings Agency (S&P, Moody's, Fitch, et al.)
In CRAs' own opinions, they are practicing "free speech" and giving what amounts to "numeric editorials on the quality of companies." In critics' opinions, large corporations and sometimes governments are relying on these "editorials", so CRAs' abilities to say whatever-the-heck-they-want should be regulated.
From what I understand, Spamhaus basically provides lists that identify known spammers, or known spam hosts. These lists are used for things like filtering out spam emails. So Spamhaus is basically saying that Cyberbunker is a host to spammers, and therefore email coming from the Cyberbunker's IP addresses should be treated as spam.
Some of my sites have had short periods of slow response times for the past few days but I assumed it's the crappy host their on. One of my clients on CF hasn't had any issues.
I and some friends noticed 4chan being inaccessible (DNS failure) for parts of yesterday evening. Don't know who else is a CloudFlare client. (I'm in the UK like the sibling comment).
"Before the break of dawn on a morning in April, a full SWAT team was sent to execute a search warrant on CyberBunker's property."
"It must not have occurred to the officers that the blast doors were designed to withstand a 20 megaton nuclear explosion from close range. When the SWAT team realized that the door was not being opened for them, they throw flashbangs and take other actions to draw attention."
---
And from the NYT article:
“Dutch authorities and the police have made several attempts to enter the bunker by force,” the site said. “None of these attempts were successful.”
I looked up a video with an actual Dutch SWAT team and their uniform looks different. The one in the picture is pretty derpy, and they're carrying what looks like tiny medieval shields. I'm not convinced.
Light could be coming from the top (around midday) and be blocked in arbitrary parts by the trees, so one part of the place gets light from the right, while another gets it from the left.
I think they shopped the photo to illustrate their story -- they’re not claiming it’s genuine I imagine (after all, apparently it took place while they were asleep).
To HN user "stiller" who commented on my comment: it looks like you were hellbanned so none of your comments show up. You should probably get that fixed.
Nothing Sven Kamphuis says should be taken seriously. He's a raving lunatic and a fantasist with delusions of grandeur.
Everything he does however should be taken very seriously. He has the technical chops (old school hacker) and the means and criminal connections to back it up.
The comments on the NYT article you linked are interesting. The sheer amount of venom being directed at Cyberbunker. It's as if people cannot imagine that multiple parties might somehow be involved within that organization (hosting company).
Also, not everybody seems to love Spamhaus, which is largely overlooked in the articles.
Missing in this is who is attacking Spamhaus and why -- what is motivating the attackers to risk eventual detection or capture. Also, it seems implied but not stated, are all of the attacks originating from leased servers in Spamhaus???
Is this the HN equivalent of "fake!" comments in YouTube videos?
You can read that they were "operating from a Cold War era government command bunker that was purpose-built by the military to house sensitive electronic gear".
This makes the story about the SWAT team very believable.
When I read the story I also thought it was most likely BS.
They claim that the SWAT team just gave up and went back to their police station and then denied they ever went there. If a SWAT team were executing a search warrant or seeking to arrest people, they would not just decide to drop the case because they couldn't get in.
They also claim to have extensive video of this event but decided not to release any of it because the police gave them 8,000 Euros to repair their fence. That video would be worth orders of magnitude more for PR purposes if they released it.
According to a convenient nuclear effects calculator I found[1], 20MT at 5km and an optimal burst height will generate over 40psi of overpressure. The fireball itself will reach about 4km in radius. According to another site[2], 20psi is enough to severely damage or destroy heavily built concrete buildings, and the maximum wind speed at 40psi will be greater than the speed of sound. For comparison, 3psi is enough to destroy a normal house.
Indeed; if you really want to survive such effects, you need to do what they did in Cheyenne Mountain: build a tunnel through a mountain and install your blast doors etc. perpendicular to it some distance from the entrances; Wikipedia says it'll handle 600 psi. One of the reasons we deemphasized it and went to Boeing E-4s (747 command posts) in the early '70s was that we judged the Soviets had a good chance of landing serious warheads at both ends of the tunnel at the same time, which would exceed its design limits.
Tom Clancey described these missiles as having the mission to: "turn Cheyenne Mountain into Cheyenne Lake."
The novel "Arc Light" also has a description (probably not that accurate) of a limited strike by Russia against the US that includes destruction of the bunkers at Cheyenne Mountain, Raven Rock Mountain and Omaha:
Probably not that accurate indeed: several Amazon reviewers noted his descriptions of "vicious" recoil from M16 rifles (!!!). If he got something so widely, if not infamously, well known so wrong, why would we expect anything else to be correct?
On the other hand, an early strike at both ends of the tunnel is very different from eroding it over some period of time (has to be some separation to avoid fratricide); if the construction was done well enough, the latter might have provided more time for NORAD to transmit additional data about the attack in progress.
I don't think that's caused by Spamhaus. Possibly fallout of the attack? I'm having no problems connecting, although the page doesn't want to finish loading.
Spamhaus identifies spammers. They don't take down websites or block connections.
Its most certainly fake.
That is dutch riot police. We don't have SWAT teams but "arrest teams".
They definitely don't wear white helmets. They definitely don't just trow around flash bangs.
And the kicker, knowing dutch police bureaucracy, they did not get a refund this quick.
You can't "turn off" that bunker, it has multiple diesel generators and lots of fuel reserves. It also has clean water reserves and air filtering capabilities, so you can't gas the people inside.
Physically cutting its uplink lines would be IMHO the most efficient way to neutralize that datacenter.
There is, of course, the other end of the cable connecting to the rest of the world... no need to actually cut the cable. You could do it with software alone really.
Or just getting all their peering exchanges to null route packets in and out. Advantage being that it's cheaper, more effective and less lawsuit prone.
I don't have much time to write this comment. But before I head to school I'd like to posit that these sorts of attacks are largely our fault.
When I say "our", I mean the loose knit group of sysadmins, self proclaimed "computer people", hackers, phreakers, security experts, and government officials trying to quell the increasing lurch of botnets and malware that has gone on since the Eternal September.
Botnets get big because users don't know any better, users don't know better partly out of laziness, partly because they feel they can't know any better. I don't know of a single site I can point to and say "If you really give a shit about not getting your credit card data stolen, go here." Instead as far as I can tell the majority of users in this demographic have their needs "met" by fraudsters selling bogus antivirus packages and weird proprietary utilities.
If you want a computing environment that can survive open, it needs users who can use open.
I agree, the problem however is that threats are constantly evolving and getting more complicated. Even most IT people don't understand the threats properly (I know I struggle).
It used to be that could just tell people to install a security suite on their computer and they'd be mostly OK. I don't think that's really true any longer.
You could also partly lay the blame at Microsoft's door in getting users to start connecting to internet with an OS designed without any reasonable security (Windows 95).
Now that we have operating systems with better security it's hard to change people's usage patterns to take advantage of that.
If she has Microsoft Security Essentials installed, doesn't run unknown software, and doesn't give out her password, what more could my mother, as a layman, reasonably be expected to do? I understand there are all sorts of complicated steps she could take if she had good intuition about sniffing out bad guys, but she doesn't. Isn't the problem with the crappy software, not her?
The fundamental problem is that bad guys can come up with new attack methods faster than we can educate or produce reliable user friendly software to counter their methods.
Even a sophisticated user is just as vulnerable in many cases. If I give personal information to a third party site that I presume to be trustworthy (say a government site) there's no way I can know if someone is going to find SQLi vulnerabilities in that site next week and exfiltrate all of that data.
New attack methods are not the problem. If there is new technology there will always be a new attack method. Right now the existing attack methods are the problem. Specifically, that technology is being developed using the same lack of basic security standards and thus the same old attacks keep working.
SQLi should not be a thing. At all. It's the most trivial fucking thing in the world to validate data before you use it in an SQL query, and people get it wrong, every single day. Security isn't hard, it's just tedious.
Speaking of such, I just recently accidentally happened across an SQLi vulnerability on a government site containing confidential information of tens of millions of people.
Reasonably? That's pretty much good. Add what another comment said about not giving out personal information, but having a good AV like MSSE, using automated Windows Updates, use an updated browser (even if it's just the latest version of IE), don't run untrusted software, and having strong passwords (using something like LastPass to remember them all but still retain convenience) is basically all you can reasonably ask an average user to do. And for the most part, that's good enough.
Sure, there's evolving threats, there's drive-bys that will slip around all of this, there's ways attackers could still get through. But as scary as it all is, anything beyond these steps gets into the territory of major inconvenience. The problem with that is, the more intrusive and inconvenient the security becomes, the less likely people are going to be to actually use and remember their security practices. If mom can't repeat it at her book club, it's not going to be effective. And to be honest, these types of attacks that bypass these restrictions are exceedingly rare when it comes to mom and grandma. The biggest threat there is phishing and malware. Corporate security has professionals enforcing a policy that meets the business's own requirements.
So to answer your question, yes, that's all you can reasonably do. In most cases, you'll be pretty well protected with just that, and those steps aren't too complicated to follow or remember.
Oh, encryption. Why didn't anyone else think of that!
Have you noticed the spate of attacks against SSL lately? BEAST, CRIME, Lucky 13, RC4 in general? https://en.wikipedia.org/wiki/BEAST_%28computer_security%29#... Not profitable for some things, maybe, but definitely worth mounting such an attack for banking info.
Reading on through the article, they continue about Spamhaus. What's that got to do with slowing down the internet? And "But we're up - they haven't been able to knock us down." is factually incorrect, Spamhaus did go down. They're winning in the end, but they did go down.
> He added: "These attacks are peaking at 300 gb/s (gigabits per second).
Source? 300gbps would definitely be visible, and I think I remember hearing about something between 60 and 100gbps.
> Spamhaus is able to cope, the group says, as it has highly distributed infrastructure in a number of countries
AKA cloudflare
> We can't be brought down
We've seen that. Am I missing information or is this a lie?
Source? 300gbps would definitely be visible, and I think I remember hearing about something between 60 and 100gbps.
From Cloudflares response [1 - nice graph in blog] ~75Gbps extra traffic was hitting part of their network.
Obliviously there would have been much more traffic floating around and getting dropped by ISPs that have correctly configured their outgoing traffic filters.
Many [not all] ISPs that were affected only have themselves to blame, the 'Internet' didn't slow down - the part that they are responsible for did - and it was their fault....
Wikipedia seems to suggest that Spamhaus blocked a large chunck of Cyberbunkers IP allocation, when the problem originated with a subset. I suppose given the conditions, Wikipedia is perhaps not to be trusted, but it does make me think less of Spamhaus.
In October, 2011, Spamhaus identified CyberBunker as providing hosting for spammers and contacted their upstream provider, A2B, demanding service be cancelled. A2B initially refused, blocking only a single IP address linked to spamming. Spamhaus retaliated by blacklisting all of A2B address space. A2B capitulated, dropping CyberBunker, but then filed complaints with the Dutch police against Spamhaus for extortion.
Spamhous has a history of blocking much more than the offending IP's, sometimes whole ISPs.
It effectively makes the issue a high priority one for the ISP, some call it blackmail.
You are almost certainly mistaken. The statement was followed immediately by the claim that Spamhaus has the largest network of DNS servers in the world.
"a Dutch web host which states it will host anything with the exception of child pornography or terrorism-related material"
Since it's a Dutch company I highly doubt they host anything illegal (as the article implies). The same rules apply to them as they do to other hosting companies in The Netherlands (and EU).
"These attacks are peaking at 300 gb/s (gigabits per second).
Is that around like 3000 compromised computers? Maybe 2-5 botnets worth? I might be a bit off on the prices here, but that sound like maybe ~$1k/day on the market? would be nice to get a price tag on the "'biggest attack in history'".
Nope, this could easily be done with far less. This is an amplification attack.
Due to the design of DNS and UDP it allows you to send a simple/small request to a poorly configured DNS server [one that open resolves for anybody - there are a lot out there] and pretend you are doing it from your targets IP address.
UDP is a fire and forget protocol, you send it a source address and it will reply to that address. With DNS recursion you can easily send a request which will reply to your target. The amount of data returned from these DNS servers and sent to your victim can often be a 50x larger than your initial request. The more open resolvers you find, the more damage you can do, without needing much more upload bandwidth from your host [relative]
eg:
You request from your host:
dig ANY isc.org @x.x.x.x +edns=0 == 64bytes
Response to your victim:
; <<>> DiG 9.7.3 <<>> ANY isc.org @x.x.x.x
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5147
;; flags: qr rd ra; QUERY: 1, ANSWER: 27, AUTHORITY: 4, ADDITIONAL: 5
;; QUESTION SECTION:
;isc.org. IN ANY
;; ANSWER SECTION:
isc.org. 4084 IN SOA ns-int.isc.org. hostmaster.isc.org. 2012102700 7200 3600 24796800 3600
isc.org. 4084 IN A 149.20.64.42
isc.org. 4084 IN MX 10 mx.pao1.isc.org.
isc.org. 4084 IN MX 10 mx.ams1.isc.org.
isc.org. 4084 IN TXT "v=spf1 a mx ip4:204.152.184.0/21 ip4:149.20.0.0/16 ip6:2001:04F8::0/32 ip6:2001:500:60::65/128 ~all"
isc.org. 4084 IN TXT "$Id: isc.org,v 1.1724 2012-10-23 00:36:09 bind Exp $"
isc.org. 4084 IN AAAA 2001:4f8:0:2::d
isc.org. 4084 IN NAPTR 20 0 "S" "SIP+D2U" "" _sip._udp.isc.org.
isc.org. 484 IN NSEC _kerberos.isc.org. A NS SOA MX TXT AAAA NAPTR RRSIG NSEC DNSKEY SPF
isc.org. 4084 IN DNSKEY 256 3 5 BQEAAAAB2F1v2HWzCCE9vNsKfk0K8vd4EBwizNT9KO6WYXj0oxEL4eOJ
<snip>
;; MSG SIZE rcvd: 3223 [bytes]
You start sending 100s of these request a second, the reply data builds up.
This was mainly a DNS amplification attack, so each node doesn't need to put out the total traffic. I'm not sure what the highest achievable return is, but doubling the traffic this way is pretty straightforward.
From what I understood, these attacks used DNS amplification. I am no expert on botnets either, but here is the basic idea: they basically send a small request to a DNS server with the source spoofed. The server sends a much larger response to the spoofed source, which in this case is Spamhaus. This happens on those DNS servers that don't check whether the request originated from inside their own network.
So the botnets involved don't have to send 300 Gbps of traffic to Spamhaus. The DNS servers being much more powerful will take care of that. I have no idea about the going rate for a botnet, though.
I think the onus for this one is on ISPs who don't properly filter outgoing traffic. It's pretty simple, really, you have a block of IP addresses you allocate to your customers, any outgoing traffic with a source outside of this block should be dropped. A simple iptables rule on the router handling that block would suffice.
There is no legitimate use case for sending traffic with a spoofed source IP. I'm simply amazed that ISPs who should have the technical knowhow still haven't eradicated all kinds of network attacks that rely on spoofed source addresses(of which DNS amplification is only one).
This isn't just any bunker. It is a 5 story cold war bunker, with blast doors designed to take a hit from a freaking 20 megaton bomb at 5 km.
The US may have a bunker buster that can open that thing, but it would kill everybody inside, and possibly around it. I am not quite sure that the US bunker busters of that size are non-nuclear either.
Would the US FBI really drop a nuke to serve a warrent? That seems, excessive and counter productive (as you would destroy whatever it was you wanted).
Bunker busters are not nuclear. They use turrets from tanks made of depleted uranium which is unbelievably strong - dropped from altitude to piece just about any underground dwelling and deliver a non-nuclear bomb.
I have no idea how I know or remember this, I must have read it somewhere but I am very anti-war, so why I know it is strange to me.
They don't really need to get into the bunker. I'm assuming said bunker has limited supplies of food, and that it is reliant on the outside world for water, electricity, and network connectivity.
Just yank the network and turn off the water, it'll either be pointless to stay inside, or untenable. Either way gets the door open.
"In this case, Spamhaus's Domain Name System (DNS) servers were targeted"
I'm not sure to understand why this should slow down the whole internet. It seems to be only for email filtering, not for the web, and only those ISP that use their service should be impacted, and only when their DNS cache is not triggered. Am I missing something ?
I got there from the comments on the CloudFlare blog posts, where a user named "STOPhaus" posted taunting CloudFlare, with a link to the stophaus.com website. Apparently it's a meeting place for anti-Spamhaus sentiment and includes such classy stuff as personal information on Spamhaus employees.
Wild stuff, and thanks to CloudFlare for their writeup.
I wonder if they realize how unreadable those posts are - they look like mindless ramblings. It took me a few minutes to realize it was an interview/chat.
I realize it's a news headline but I feel obligated to point out that this doesn't rank as the 'biggest attack in history' except maybe for Spamhaus. It doesn't appear to come close to what the SQL Slammer, Blaster, or Sasser worms did. Slammer in particular increased latency globally and the impact was easily visible to anyone browsing the web.
To this day - with v4 exhausted and despite numerous delisting attempts - I have a /21 listed in Sorbs because it happened to be in the past part of an ISP's /18 dynamic range for customers.
They deserve all that's coming to them and more.
Too bad other's get affected in the process.
> RBLs are a bad idea, they often end up in abuse.
Indeed, all they should be used for is perhaps 0.5 points worth of spamassassin weight. Sysadmins who use spamhaus as a blacklist are just as incompetent as those who bounce virus e-mails to the address in "From:" ...
When it comes to DoS attacks, bandwidth is a much less meaningful metric than packets/s. 300Gbps could be anywhere from 200,000,000 PPS to 4,687,500,000 PPS. High bandwidth attacks just cause congestion issues, while high packets/s actually take networks and servers down.
It's great that all of our criminal justice resources in the US are dedicated to stopping real crimes like TOU violations of academic journals instead of things like investigating and stopping industrial espionage, sabotage, etc.
This discussion is quite intense. Frankly it's scary that a single attack manages to slow down popular sites like Netflix. What consequences for future attacks will this attack have?
No because it was a DNS amplification attack so it was a bunch of DNS requests with spoofed source IPs. Best way to reduce their impact in the future would be to threaten to null route ISPs that don't do anything to stop spoofed packets.
According to the article(s) the DDoS is so big that it is not just clogging up spamhaus' links as it is intended to but also the backbone leading up to said links. That would affect everyone and not just spamhaus.
Also, the DNS servers are being used to perform the attack via DNS amplification, the slowdown is not caused by clogged DNS servers.
I don't have the exact quote but one of the articles likens the situation to having a motorway with on-ramps and off-ramps to individual networks/hosts. The usual DDoS seeks to clog the on-ramp or off-ramp the target uses by sending too many cars their way. However, this attack is so big that it's clogging up the motorway itself not just on/off-ramps.
for the biggest attack in the history, its kinda looking boring from here (India). :/
1 billion+ people won't hear of it much.
I would be interested to know, what did Spamhaus paid google to use its resources, and would such a type of cooperation on global scale means the end of DDoS in the long term?
The main problem is that some people decide what's good and what's not online and paint with the broadest brush possible. Spamhaus, sadly I say, is used by a lot of providers as gospel and a lot of innocent sites are hurt.
If you ever tried to handle any mail server at all, you would recognize that you have choice in using spamhaus (or any other DNSBL).
These people have put in place a high quality method to discriminate spammers. I've been around since their beginnings, and their list has been incredibly successful (very high quality) for me, compared to njabl and other "dynamic" lists based on honeypots, or backlash entirely (say hi to spamcop).
You would also recognize that you can just as well tag the message with "likely spammminess" for use along the chain, and people would still complain that your "legitimate" message was tagged as spam by SOMEBODY, while you wouldn't complain if it was tagged as spam by a learning algorithm.
In short, people would complain anyway, except that spamhaus is doing real damage to the spammers (as in "the mail really didn't go through") and reducing their revenue, and thus forcing them to come out which such measures. Not that they will accomplish anything anyway. Spamhaus helped stop a lot of known/professional spammers, and I applaud them for that.
I do not have any choice at all about what spam filters the recipients of my email may be using. I have never had this problem personally, but there are many, many accounts on webhostingtalk.com of IP ranges being banned Spamhaus without any evidence of spam; of IP addresses that remained banned after ownership changes hands and other problems. There are always two sides in every story. On balance I think Spamhaus is doing a very good and necessary thing. I don't know about this particular case, but I've read accounts of what sound like very reasonable grievances.
I've been designing honeypot/traps as triggers for mail filtering infrastructures for years, and this is very hard to automate process. It started like something that you could watch from time to time in the late '90, maybe slab a DNSBL or two, but right now has become a bloody nightmare. I remember when at some point almost everybody started to "reinvent" greylisting out of convergence, even before it was called as such. Reading nanae (via NNTP) was always a good read.
You constantly have to check if there is a chance that spammers noticed your honeypots so that they can avoid them or use them against you as well (the bigger you get the more sophisticated these attackers get too), you have to use tagged email addresses that can be linked back to the offenders. Methods to probe address ranges multiple times before validating them, and ways to automate the unlisting as well. False positives are basically unavoidable at some point, also because spammers themselves like to rotate their addresses based on their previous owners or known datacenters that are "too big to be blocked" wholesale for this exact reason. If they had a chance to know one of your trigger addresses, a common practice is to generate spam from a "safe" range into the trigger address, in an attempt to generate a false positive and thus, of course, backlash. It's sickening.
Exchanging digests of message contents among multiple server cooperatively became a good indicator of spammyness (vipul's razor), though you would catch bulk emails in the process, and spammers quickly adapted to random email contents so that the method became quickly ineffective.
The real problem here is that these assholes don't care as long as they can deliver the message, that's the only metric they have and care for. Maybe you don't care for it, because you can then use filtering later, but that's a huge volume of trash that needs to be shoveled around. I actually witnessed many cases in organizations bigger than a hundred eployees where several servers were used 24/7 just to churn messages through "dspam" or similar filters before delivering to the final mailbox. This is a huge cost in terms of measurable power wasted for a couple of assholes.
I had a similar problem and never found out exactly why it happened.
The hypothesis I came to was that we weren't using SPF records on the domain associated with our IP address for a long time.
Some spammers were taking advantage of this by sending emails from different IP ranges with the From: header spoofed to be from our domain.
So Spamhaus blocked our IP address on the grounds that spam filters would also be able to confidently block anything appearing to originate from a domain name that resolved to our IP address.
It's extremely unlikely that spoofed headers or the lack of an SPF record would get you listed on an RBL, especially Spamhaus. I can't guess what happened in your case, but somehow your IP address obtained a bad reputation or was unlucky enough to be in a tainted block. FWIW, the very first thing I do after getting an IP allocation is run an RBL check on it and demand a replacement if it's listed anywhere.
Yes, it was a fairly weak hypothesis.
OTOH we got on several RBLs a number of times and managed to get taken off them.
Once I added an SPF record it hasn't been a problem since.
Didn't use SPF to begin with because there was a large number of hosts legitimately sending mail for the domain and it was a pain to get all of the IP numbers for various crazy reasons.
You seem to be taking the line of the attackers' spokesman, who accused, rather hysterically, Spamhaus of deciding what goes on the internet. Of course, all Spamhaus does is supply a list of hosts who are sending email spam, and other things like lists of dynamic IPs. Sounds like this hosting outfit was making money hosting spammers and their business is threatened by legitimate countermeasures.
I wanted to believe him. But before I could reply to his mail, I got first-hand evidence that the SBL has in fact gone bad.
As of this writing, any filter relying on the SBL is now marking email with the url "paulgraham.com" as spam. Why? Because the guys at the SBL want to pressure Yahoo, where paulgraham.com is hosted, to delete the site of a company they believe is spamming.
Furthermore, there's a lot of FUD in this thread about Spamhaus listing people who don't emit spam. IF this is true, then Spamhaus would have an unacceptably high false positive rate, and we would be able to observe this. In reality, Spamhaus has the lowest FP rate in the industry. Occam's Razor suggests those who claim to have been wrongly blocked are mistaken about the reason for their listings (if they ever existed in the first place).
You are incorrect. (Well, you're correct that Spamhaus doesn't filter content -- but they don't filter anything, they publish lists that various filtering software uses.)
I hear the SBL can also block domains, how? What is "URIBL_SBL"?
Yes, the SBL can also be used as a URI Blocklist and is particularly effective in this role. In tests, over 60% of spam was found to contain URIs (links to web sites) whose webserver IPs were listed on the SBL. SpamAssassin, for example, includes a feature called URIBL_SBL for this purpose. The technique involves resolving the URI's domain to and IP address and checking that against the SBL zone.
And of course they also have the DBL (Domain Block List), though I don't know if that existed back when PG ran into problems.
Do you have a link to the false positive rankings? I'm curious as to how that is measured.
Good point; I think both of our statements are true due to ambiguous wording upstream. I also took it literally, "any filter relying on the SBL" -- I use the SBL (via ZEN) but don't use SpamAssassin. And so my mail servers wouldn't block any domain that resolves to an IP address in the SBL, as described in the link you provided.
As for DNSBL false positive rates, I haven't seen statistics in a few years, and by now they wouldn't be worth much. The only ones I saw were from 2005 or 2007. This one (linked to from the below article) from 2011 doesn't even test Spamhaus:
This is just my personal experience saying (in 2013) that Spamhaus has the lowest FP rate, which isn't scientific. I'm kind of surprised there haven't been more FP comparison reports of major DNSBLs in recent years. If anyone has a link I'd love to see it.
"want to pressure" - I applaud PG for still staying so civil after this stunt by Spamhaus (and it was not the only one, e.g. they also blocked nic.at in 2007) - I am more direct: Spamhaus blackmails others to get what they want. People who blackmail others should be send into jail, even if their actions are useful. The end doesn't justify the means.
You seem to be taking the line of the attackers' spokesman, who accused, rather hysterically
Ummm, my ISP IPs hav been blocked several times for absolutely no fault of mine. I have a shared IP for browsing and turns out that cloudfare has blocked them. I also had issues with my sites, the IPs signed to me were blacklisted.
I understand that no one is forcing usage of spamhaus db but it seems unfair and white-listing is near impossible.
Could you elaborate on what happened in your case that you'd be so vehemently opposed to spamhaus(to the point of being willing to commit crime(s) to hurt them)?
I'm truly curious on why the reaction to spamhaus being DDoS is so polarised.
A long time ago in a galaxy far away I was a sysadmin at a local university.
Trying to keep mail-servers running and keeping up with the different spam clearing houses different policies that kept changing without notice was a lot of work back then.
Once you got black-listed getting removed wasn't always an easy process no matter how quickly you tried fix whatever caused it. Methods of communicating were not always available and when it were, responses were not always helpful or even very polite.
I haven't managed mail servers for over ten years, and really hope that the conditions for being included on a blacklist and process for getting removed is more transparent by now.
Given the amount of trust that most people running mail servers are putting into the different blacklists organizations like spamhaus get a lot of power over the internet.
Judging from my experience with spam clearing houses it looks like that power have once more corrupted when I read the news stories about cyberbunker.
We need places like cyberbunker to keep the internet free and open. When all the pr0n, w4r3z and 1337 stuff have been cleaned out from the internet the infrastructure to stop anything at will on the internet will be in place and functional.
I wonder what would be the next thing to be removed from the internet?
Once you got black-listed getting removed wasn't always an easy process no matter how quickly you tried fix whatever caused it.
I took a job in the year 2000, at a company with 3000 email users, listed by Spamhaus. First thing I did was close the open relay they were running. The listing was promptly removed, and the mail queue was back to normal within only a few days. I'm skeptical of your claim. I've never seen a confirmed case of Spamhaus aggression, but I've seen a lot that were disproven, and even more that sound like they were written by miscreants. Like the kind who would advocate DDoS attacks cough.
Open relays were at the time manageable, even the ones that suddenly appeared when someone installed an old OS-version, as were the process for getting removed from the blacklists due to open relays.
Once you had one a computer lab workstation hacked and used for spamming - not so easy to get whiteliested anymore.
The university had a class B-network, trying to get the staffs subnet whitelisted while keeping the computer-labs blacklisted was apparently not possible according to the spam clearing houses. Blocking port 25 for outgoing traffic not possible to check from the outside and didn't help.
I can understand that organizations like spamhaus are overworked don't have the resources to handle every non-standard case on the internet as quickly as the blocked ip-range would like, but the replies we got were truly unhelpful.
The fact that someone bothered to register the domain stophaus.com seems to indicate that my experiences isn't uniqueue.
I can't speak for np422, but a few years ago Spamhaus blocked one of my IPs and insisted I had sent messages to one of their "spam trap" addresses. I think they were wrong or (less likely) someone was maliciously messing with us.
But since the spam trap addresses are secret, it was an impossible charge to defend against or investigate. Not fun.
Every so often, spamhaus drops my mailserver/forwarder onto their policy blacklist. My mailserver forwards my mail on to where I actually pick it up, which unfortunately, uses the PBL. When that happens, I first have to notice (normally due to a lack of spam in my inbox), I have to jump through their hoops again.
Yeah, I could fix that, either by hosting my full stack of email, or not doing it at all. Either way, it's a pain.
Also, http://openresolverproject.org
PS Technical details: http://blog.cloudflare.com/the-ddos-that-almost-broke-the-in...