Hacker News new | past | comments | ask | show | jobs | submit login
TP240PhoneHome Reflection/Amplification DDoS Attack Vector (akamai.com)
186 points by leohonexus on March 9, 2022 | hide | past | favorite | 84 comments



I'm really concerned that DDOS attacks are going to lead to the death of the open Internet and its balkanization and isolation behind walled gardens. If you look at where Cloudflare and some of the big clouds are going with their private networks, private backplanes, and "secure your traffic by putting it all over our network" zero trust plans it seems to be going that way.

If open peering and the open Internet are to survive I think serious work needs to be done to fight DDOS attacks. It needs to be an effort analogous to the "war on spam" in the late 1990s / early 2000s. Unfortunately that war was sort of lost; e-mail is in practice barely an open protocol anymore and almost all e-mail is handled by a few giant companies that can leverage big data to filter spam. If you try to DIY a mail server you'll be simultaneously hit by spam and have to constantly fight mistaken filtration by larger e-mail providers who tend to distrust small mail servers by default.

If the open Internet succumbs to DDOS "spam," we will lose something really huge and important. It would be the ultimate casualty of what so far has been almost a law (with very few exceptions): all open systems are destroyed by abuse if they become sufficiently popular.

We also can't just leave it to the free market because the only solution the market will likely come up with is walled gardens. It's the easiest to engineer solution and the easiest to monetize.


> If you look at where Cloudflare and some of the big clouds are going with their private networks, private backplanes, and "secure your traffic by putting it all over our network" zero trust plans it seems to be going that way.

All the networks of the Internet are already private, just like the networks of AOL and CompuServe were private back in the day: your ISP's network is private, YouTube's network is private, AWS' network is private. It's just that those private networks agree to talk to each other.

Otherwise your ISP would have to re-create YouTube and Reddit/forums and eBay/marketplace and…, and YouTube would have to buildout (inter)national network to connect their video services to people's homes.

Just like AOL and CompuServe had to build out information services and a connectivity infrastructure back in the day.

Now each of the previously walled gardens (messaging, forums, marketplaces, connectivity, etc) is done by its own entity, each taking a slice of the monetary pie for the service(s) they provide.

The Internet is a 'network of networks', but it is also an agreement: an agreement for everyone to talk to everyone else.


I think that's kind of semantic. The agreement is what I'm talking about. It makes the Internet open. I can just send you a packet. That's what's in danger here.


> If you try to DIY a mail server you'll be simultaneously hit by spam and have to constantly fight mistaken filtration by larger e-mail providers who tend to distrust small mail servers by default.

I have managed my own e-mail server for around 20 years.

Filtering spam has never been a problem.

On the other hand your second problem has indeed existed, i.e. with various large e-mail providers which either blocked completely my e-mail messages without signalling any error, or they delayed for 1 day or 2 my messages, or they required many resendings of a message until really passing it to the destination.

Fortunately such cases seem to have become much more seldom during the last couple of years.


I've seen hideously inconvenient email pauses between Office365 and a massive NGO, so it's not just little mail servers.


It's interesting that you say that, because we've already sort of balkanized around ISPs. However, CDNs and DDOS protection popped up around services that ISPs couldn't provide. Maybe the dream is for ISPs to provide these services as well, making it more tenable for regular users to self-host.


220 billion percent! And other scary numbers!

Coordinated disclosure: https://blog.cloudflare.com/cve-2022-26143/

Info for Cloudflare customers: https://blog.cloudflare.com/cve-2022-26143-amplification-att...


On the bright side, we're lucky they did not use a 64-bit int.


Now that's a ping of death!



We're approaching the limits here, I think.


Why would there be a theoretical limit?


Limit would end up being when you send 1 byte of traffic to a box and that box amplifies it to whatever its own max outbound bandwidth rate is.

This seems like it would exceed that in many cases, since 1 byte in => 4.2 gigabytes out. Which is roughly 33.6 gbps. Not sure many of these vulnerable boxes actually have that amount of outbound bandwidth to utilize.

(Please feel free to correct my quick math if I messed it up)


Why do you want to send everything in one second?


This is a good point, but then you need more boxes to perform the DDOS as the reason they are effective is overwhelming the packets per second or bandwidth per second of the receiving networks. So it definitely does allow for a sustained attack by a single box with limited outbound bandwidth, but that blunts the usual reasoning for why the amplification is so dangerous.

Another interesting impact of this is that the higher the amplification, the more likely it is noticeable by the server that is being abused. I mean if you clog the outbound network for a company they will notice and try to resolve immediately. Versus some milder amplification where it can go under the radar, or at least the business impact urgency radar of a company much longer.


because that value is a physical limit


How so? If I find a vector that triggers the remote system to `cat /dev/random | netcat $target` then there's no limit for how much traffic my refelection generates, no?


I assume by limit OP means the remote system's bandwidth.

at 4 billion to 1, there's in practice very little difference between CVE-2022-26143 and what you describe. Both will be capped at the same number by the bandwidth available to the offending system.


look at the binary, it's an overflow value. Like it didn't fit the spreadsheet.


At least it was a 32-bit integer, not 64


Seems like a potential mitigation would be to send the affected devices a small stream of packets that tell them to generate traffic for e.g. an invalid IP, local IP, or their own public IP.

Once that hits, the device would then be sending the traffic harmlessly to /dev/null for the next 14 hours and be unavailable for attacks.

Not sure about the legal and ethical implications of that.


Tracking down these systems is easy, so these issues can normally be solved pretty easily.

Thats because typically any amplification vector doesn't allow the source IP of the amplifier to be spoofed. So as soon as a DDoS attack begins, a sample of the packets can be taken to get a list of the amplifiers used. Those can then be tracked down and patched to no longer act as amplifiers.


It could be easily solved by the operator, but that doesn't mean it's easy for the victims to get the operators to fix their stuff. These amplifiers are already run by people who ignored the software manufacturer's directions. What are the odds they will actually install the new version that's harder to abuse?


Usually[0] contacting the operator's ISP and informing them of the situation will get said ISP to contact said operator. All that outbound traffic does represent a cost to the ISP, after all. A call from your ISP usually gets a bit more respect than a call from some random person.

[0]- In the US; I don't know about anywhere else


In the past what usually happens is the ISP disconnects you until you prove you've fixed whatever it was (sometimes they're nice and block just part of the connection, or give you a warning).

Surprisingly enough, the ISP often has no real way of contacting anyone; the easiest is to cut the connection and wait for a complaint.


Yep. Sad but true. Nobody bothers to keep their contact info up to date with their ISP it seems. Non-critical stuff sometimes can be mailed to a customer's service address, but often disconnecting someone is all an ISP can do to make them aware they have a problem.


It really depends on the ISP. After spending some time trying to get phishing sources taken down and not getting anywhere, I wouldn't be hopeful about DDoS (reflection) sources being taken down either. When I was running servers that were getting DDoSed frequently (but thankfully for short intervals and not with tons of bandwidth), trying to get chargen servers or wordpress servers fixed didn't even seem like an option. Just make sure my servers wouldn't fall over, or at least would fall over gracefully.


Your comment underestimates the task of remediation. Sure, we can very easily get a list of DDoS source IP addresses. Any decent network operator can get a list of flows matching some DDoS criteria and generate a report of IP addresses.

In the case of this TP240 attack, you're talking about ~2600 independent businesses across the world. Assuming you are able to determine the actual source of the traffic and work with a vendor to patch it, you're still tasked with somehow getting 2600 businesses to patch their systems or modify firewall rules.

In the case of the memcached amplification attack, Cloudflare saw upwards of 5800 source IPs in the attacks, and Shodan reported nearly 88000 IPs responding on port 11211 [1]. Tracking down the owners of 88k installations across public clouds, businesses, probably some residential networks, is a monumental task. There's nothing easy about it.

[1] https://blog.cloudflare.com/memcrashed-major-amplification-a...


> you're talking about ~2600 independent businesses across the world. Assuming you are able to determine the actual source of the traffic and work with a vendor to patch it, you're still tasked with somehow getting 2600 businesses to patch their systems or modify firewall rules.

You can be sure that by only null-routing their entire C-class, adjacent customers will loudly complain to the operator who will quickly identify the source and disconnect it. The best way to deploy fixes on the net has always been to first disconnect them. This way you don't have to convince anyone, it's done the other way around. Typically the CEO will instantly throw all the phones to the trash to get the net opened again.


In general that's not really an option.

Unless you have coordination with the network operators on which those amplifiers are sitting, your null-routing of the amplifier in your own network isn't going to stop it from attacking other targets. If the amplifier is something like a DNS server, then your collateral damage isn't just "adjacent customers", it's potentially thousands of other users and resolvers on your own network. If those amplifiers are on a cloud service provider like AWS, you're going to potentially inflict even more pain onto your own paying customers who will no longer be able to communicate with AWS. You will essentially perform the DoS they were aiming for.


Let's get started! [1][2] You wanna take the odd numbered IP's and I take the even? Just kidding I am way too lazy for this

[1] - https://www.shodan.io/search?query=mitel

[2] - https://www.shodan.io/search?query=mivoice


We need proper liability laws for malicious traffic.

You are liable unless you can pass off that liability to someone else. So the ISP would be liable by default, and would have an incentive to filter their customers, or require them to abide by certain rules, pass some audits, provide proof of insurance or post a large deposit.

You could have insurers who in exchange of automated security scans will insure you, solving the problem for end-users at a reasonable cost.

This will actually encourage internet users (both consumers and businesses) to take security more seriously.


Litigation seems too heavy handed for these kinds of attacks.

A major issue here is how your smart toaster or MiVoice box can be spamming the internet and there's no real way to realize it for most people.

Since you pitched a controversial solution, let me make one that's probably even more controversial: maybe bandwidth is too cheap. Maybe the problem would fix itself without legal hell if your C&C'd smart toaster / VoIP box had an impact on your ISP bill instead of being folded into your unlimited bandwidth billing.


unless the amplifier mechanism is widespread.


Concerning this particular vector:

> Approximately 2,600 of these systems have been incorrectly provisioned so that an unauthenticated system test facility has been inadvertently exposed to the public internet


Does anyone have any data on what networks or organizations were on the receiving side of these attacks?


Is it just me, or does it seem crazy that we all just accept that private businesses are obligated to protect themselves from state-sponsored hacking? Imagine if Wal-Mart had to fund a private air force and patrol over their stores in order to combat foreign bombers coming in and everyone was like, "Yeah, that's just how it goes."

Isn't a primary responsibility of government to protect its citizens and businesses from other states' militaries?


In principle, that's what the NSA would be doing. When DES was developed and standardized in 1976, the NSA had input in selecting some of the constants that were chosen for it [0]. It wasn't until the late 80s when independent development of differential cryptanalysis [1] came out, and people realized that the DES constants were deliberately chosen to be resistant to this attack.

The NSA has since turned away from this responsibility, and has done the exact opposite. When Dual_EC_DRBG was developed [2], there was a similar choice of constants, with the final values having been chosen by the NSA. In this case, rather than protecting against a attack method known only by the NSA, the constants were chosen to allow an attack method known only by the NSA.

[0] https://en.wikipedia.org/wiki/Data_Encryption_Standard

[1] https://en.wikipedia.org/wiki/Differential_cryptanalysis

[2] https://en.wikipedia.org/wiki/Dual_EC_DRBG


Or maybe the choice of Dual EC DRBG constants are intended to protect against a new cryptanalysis technique known only to the NSA


internal documents leaked by Snowden and reported by the NYT confirmed the intent of the program

https://en.m.wikipedia.org/wiki/Bullrun_(decryption_program)

stop simping for the nsa


I highly doubt it. Dual EC DRBG basically works by encrypting your seed value with a NSA provided public key. It’s kinda amazing how blatant the back door is.


Sorry but Walmart has cameras, guards, and most importantly locked windows and doors.

Just because nobody has figured out (or bothered to invest into) building the equivalent of basic security doesn't mean it's the state's responsibility.

It is the government's responsibility to make sure companies take their responsibilities of protecting their customers' data, and the internet more broadly from the impact of the company's decisions.


I think the question is about foreign government operations. If North Korean agents threw up some graffiti on a Wal-Mart and stole some soda, the private security would not be expected to handle the situation on their own. Even if the stakes seem low, that's an international incident.


I think a somewhat comparable scenario could be: it's reasonable to expect that Walmart should defend against most counterfeit currency on their own. But should they be expected to defend against counterfeit currency made with state-level resources, such as supernotes with the same paper, ink, printing process and security features, where there's no guarantee that any reasonable detection method will work? This is, interestingly, something that has been linked to North Korea as well (https://en.wikipedia.org/wiki/Superdollar).


That's... a very weird, reaching argument to make. And also not an international incident, since it's just some graffiti, not espionage or assassination or whatever. I'm not sure what point you're trying to make here.


It doesn't seem that far reaching. There's a difference between "foreign citizen action" and "foreign government action". If another government comes to your territory, to break your laws and deprive one of your businesses of their property or rights, that's a big deal. But because it happens online, it's given a pass and pushed on to private individuals to deal with.


The original argument is that it's weird private businesses have to protect themselves against state actors such as foreign governments. The equivalent would be if Walmart was expected to protect itself while a foreign governments special forces raided their stores.

Of course I'm not sure that's how it's playing out anyway, as I'm certain that the relevant three letter agencies are interested in foreign state actors digital incursions, it's just a very delicate situation and not as simple or clear cut as the Walmart example.


Honestly, ignoring the state actor part of this, even if a bunch of local kids run up and graffiti the outside of a Walmart, I don't think we tend to regard it as a fundamental failure of Walmart's duty to secure their business, or a failure of their architects and security staff to do basic diligence or follow best practices to allow it to happen.

It's just a criminal act, of which Walmart are the victim, and it's the state's job to find and prosecute and deter that kind of thing from happening again.


I am pretty sure that it does not matter who stole the soda - North Koreans or locals. Either way it is up to store security to catch them and hand over to police. Police may then hand NKs over to someone else, but this doesn't change what store security must do.


> this doesn't change what store security must do.

There is no _must_ here. The police _must_ deter and punish crime. A private entity _may_ hire security if they find the police to be ineffective at stopping certain crimes. If walmart was robed while the security guard was off duty, it is still the police's job to investigate and arrest the criminal.


You're missing one thing though: North Korea doesn't rob Walmarts. Expecting private entities to be able to stand up to the kind of attack a hostile nation state can muster is unrealistic, and, quite frankly, probably a drag on the economy.

Besides, what ever happened to "provid[ing] for the common defense?"


How can Walmart defend itself from foreign government's attack, with cameras, guards, and most importantly locked windows and doors?

Defending internet infrastructure from foreign government's attack is not "basic security".


The correct equivalency would be the roads leading to the Walmart. If a Walmart were blocked by people pointlessly driving on the road to make the Walmart effectively unreachable, police would intervene and clear the road of the noise.


There is no such thing as a "correct equivalency".


The US government does do quite a bit to protect their citizens from electronic attacks. There are organizations like CISA and NIST that do a lot of work to help prevent attacks, and the FBI and DOJ do a lot of investigate and enforcement work after the fact.

For example:

https://www.fbi.gov/investigate/cyber/partnerships

https://www.cisa.gov/stopransomware/ransomware-guide


Well stated.

We also have a tendency to conflate the requirements on software systems with respect to security threats as being somewhat similar to the requirements on other kinds of engineering with respect to safety and environmental threats, and I think that does a disservice to the vastly different scope of responsibility involved.

When I see people arguing that software engineers need to treat security as seriously as, say civil engineers treat structural stability when designing a bridge, or mechanical engineers treat vehicle crash safety, I agree to an extent, but I also think it’s worth considering:

Most bridges are not designed to actually survive being deliberately attacked with the kinds of weapons nation states can bring to bear on them. When militaries get involved, bridges tend to fail.

Likewise, civilian car safety testing does not make cars that are able to survive attacks that nation state actors can carry out with things like tanks, mines, or drones.

We need to be realistic in our expectations for what level of military threat civilian systems can reasonably be expected to deal with unaided.


While your logic is solid and I do think this would be ideal I struggle to see how this would work.

Dropping bombs on a walmart store is clearly unwelcome, sending traffic to walmart's website? Much less clear. You can guess based on the traffic pattern but the only way to really know is to ask walmart if this is welcome traffic (not just a burst because some new product came out). Especially since many cases are DoS with encrypted TLS traffic that looks much like any other traffic to an outside observer.

However much of the protection is threat of retaliation ("if you drop bombs on us we will flatten your country"). So maybe that is the solution here, the government should treat these attacks as real threats and punish those responsible.


Well, a government could start by mandating that internet peers authenticate their packages, and cutting the access of bad actors.

People can't do that, and it's a very basic defense.


"Mandate private companies protect their customers" sounds very different than "the government should protect everyone" even if the result is similar.


Start by mandating BCP38 (RFC2827).


What stops that? (Both its widespread implementation and making it mandatory)


You mean what that protects against?

It provides the first part of my post, authenticating the packages.

The second part is cutting out misbehaving connections. On this case on the article, it would be trivial, and governments should be on the ISP shoulders making them make call everywhere and cutting some of their clients. But there are many attacks where the ISPs don't have enough information to act if they implement something like BCP38.


What I find much more crazy is how this is made out as "state-sponsored hacking", even tho the article doesn't mention with a single sentence who or what the attackers are.

In that context instantly jumping to "state-sponsored!" strikes me not only as a needless, but particularly dangerous escalation.

It's like people forget that "cyber" is most of all asymmetrical and attribution is usually more of a guessing game than an exact science.

Yet nearly every larger hack is very quickly labeled as some kind of "state sponsored offense!" to serve foreign policy narratives, and most of all; Excuse the incompetence that often enabled such attacks in the very first place.


It is just you. In the physical world a military can observe an attack, can announce that it is not cool, and can drive a tank through most intruders.

Now ask yourself this question, would you like to give your military the full access to your infrastructure together with command and control capabilities to do with your devices and the software on them as it pleases according to the situation? If you actually think that in fact you are not okay with 24/7 monitoring and management from a centralized government institution, you should own up to your desires and get your defense together.

Of course, this is a simplistic and extreme scenario. Much of the missed part is about availability and basic institutional capability for military cyber operations, but the fundamental question is: when one demands something from the government, what exactly they wish to give up as a consequence of the proposed solution.


Yes, it's walmart responsibility to protect their customers. It's their responsibility that their supply chain is not hacked to say distribute poison, it's their responsibility that the cameras they use in store are theirs and only they have access, it's their responsibility that the card I use in their terminal is safe. The example you gave won't be hurting the people, otherwise yes if they want to gain trust in dangerous land they have to ensure safety of people.


23 people were killed and 23 more injured in a Walmart in El Paso in 2019 in a mass shooting. Is it your position that Walmart has sole responsibility for failing to prevent those deaths?


I feel like it more broadly boils down to, if you put it on the internet you are responsible for securing it.

If you can't or don't want to secure it, don't put it online.


Many private businesses already are expected to protect themselves from state (and similar capability) physical interference and attacks, especially if they are in the supply chain of critical infrastructure. It's one of the things you have to do effectively to earn profits in that sector.


We rolled over to state-sponsored election meddling. There’s no way we’re going to care about this.


Indeed. We definitely need laws to hold companies accountable for their IT-related activity.

For one, we need to hold commercial vendors accountable - that means especially to refuse to provide security updates for the reasonably expected life time of a piece of software or hardware.

But especially, we need the companies using IT systems to be held accountable. The magic word is "defense in depth" - the scenario of the post we're talking about is a piece of equipment that was not supposed to be reachable from the Internet and despite that knowledge it was made accessible to the Internet. Seriously, anyone caught exposing dangerous stuff to Shodan should be fined to hell and back. Or to continue using your military comparison: most governments have laws that call for harsh punishment for "aid to the enemy" or similar. Time to update the law to the new digital world.


A failure to defend yourself is not aiding the enemy. That is insane.

The government provides for the common security. That's one of it's most fundamental jobs.

Imagine if your house was destroyed by a Russian drone and you were thrown in jail for not having enough "defense in depth" against drone strikes.


> A failure to defend yourself is not aiding the enemy. That is insane.

Let's say you are the leader of a border post, and you leave your post unmanned allowing the enemy in - of course you will be held accountable.

Exposing stuff to the Internet despite the manufacturer warning against it is at least grossly negligent and should be punished.

We are at war with Russia and China on a nation-state level and on top of that we also have cybercrime gangs.


> Let's say you are the leader of a border post, and you leave your post unmanned allowing the enemy in - of course you will be held accountable.

Yes! Because if you are a member of the state operated defense force, then defense is your responsibility. The state is responsible for defense.

If on the other hand, you are a civilian who just happens to own property near a border, you have absolutely zero obligation to defend the border yourself. The same is true for businesses near a border.

> We are at war with Russia and China on a nation-state level and on top of that we also have cybercrime gangs.

Man, if only society had a way to form some sort of governance body which could provide defense against other nations and provide some sort of justice system to protect against and punish crimes. Oh well, I guess its every man for themselves ¯\_(ツ)_/¯


> Let's say you are the leader of a border post

Let's not say that. Suggesting that civilians have defence duties on par with members of the military is ridiculous.


"We are at war with Russia and China on a nation-state level"

Not aware of any country in the world that is currently in a declared state of war with Russia and China.


The insistence of people on a formal declaration of war is one of the reasons why the situation has escalated so far.

Just how much evidence do you need to realize that the actions of both Russia and China have been - for years now - to undermine Western societies and the global set of rules?


Likewise, if I sell... garden fencing, and the military decides to buy it as just another customer, how liable should I be if it's easily bypassed?


Depends. Do you want the government to control the internet?


I guess it depends on the analogy, and whether physical location means anything for the internet. Piracy of commercial ships, for example.


That is also very much a problem where states are expected to intervene.


It seems similar to me. They sometimes intervene, but often don't.


maybe one day when the people in charge are at least somewhat technically litterate


so, china's internet? because that's what you're actually asking when you ask a policed internet.


It very much is the responsibility of the NSA, but they mostly fail at their primary job, and conduct illegal activities such as mass surveillance instead.

Another example of a government agency that fails at their job: the FDA. The FDA is supposed to protect consumers from harmful foods and medications, but the fact that you can walk into any store and grab a can of food or bottle of vitamins/supplements contaminated with heavy metals is a huge red flag [0][1][2][3][4][5]. The FDA does 0 product screening whatsoever. If the FDA actually did their job, healthcare revenues would be at an all time low in America. It pays off big time to have a diseased population.

[0] https://www.consumerlab.com/answers/supplements-and-foods-th...

[1] https://www.consumerlab.com/answers/do-zinc-supplements-cont...

[2] https://www.consumerlab.com/news/metals-in-seaweed-snacks/12...

[3] https://www.consumerlab.com/recalls/11882/herbal-supplements...

[4] https://www.consumerlab.com/news/contamination-in-greens-who...

[5] https://www.consumerlab.com/news/caution-with-spirulina-supp...




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: