These aren't issued patents, they're published applications for patents.
When you read these applications, look first at the claims. If you believe any of those shouldn't be granted by the PTO in an issued patent, read the application in detail to learn more about the claim(s) you don't like. Why?
Because if you're right (example: claim shouldn't issue because of citeable prior art), you (anyone) can make a Preissuance Submission to the PTO saying why the claim(s) shouldn't be allowed. Here's a real brief view of how P.S works [1].
Really, if you as a person with domain knowledge see a problem with an application, it serves all of us for you to challenge it.
I've done this once in an energy storage context. The target application never issued as a patent (and rightfully so).
>Really, if you as a person with domain knowledge see a problem with an application, it serves all of us for you to challenge it.
If the PTO wants people to help them with review like this then the triple damages for having read a patent needs to be dropped (by congress). As it stands, corporate policy just about everywhere is that engineers shouldn't read patents.
The law around willful infringement (read: triple damages) changed recently.[1] It is perceived to be much more difficult to find willfulness now, and is generally determined by a jury. It is much more in line now with the common understanding that most people have, not some asinine bright-line test based on if you read a patent.
tl;dr: just reading a patent probably isn’t going to land you in triple damages by itself.
It is the idea that you shouldn't look for prior IP because knowledge of that prior IP makes you liable for triple damages because it's "willful infringement" rather than ignorance or accident. I have heard it before, I think it's ill-advised but I guess if you're 3M you have the legal team to deal with the fallout.
Regarding these particular patent applications (yes, even the broad claims and not just the titles and abstracts), I am baffled how willful infringement couldn’t apply.
I mean, you could just look at the papers the Google researchers themselves published, and just consider the sections on prior work and the bibliography.
Claims, even about a specific software system embodying an RNN device for a specific audio sample generation program, would obviously include research effort by Google staff to understand prior IP, much of which their neural network software mechanisms are inherently and directly based on in a way that most experts would view as clearly dependent on prior art.
What frustrates me is seeing so many people in the thread trying to defend Google and rationalize these applications as defensive patents, assuming the best of Google, and gainsaying people who are worried about the broad effects of the patents by obsessing over the mostly irrelevant fact that titles & abstracts don’t determine the content of an award.
There are a few other aspects to the problem that are also very serious.
- the popular belief that Google has some patent war chest in neural network devices can create a hugely chilling effect on other research, especially by other corporate research teams, borne purely out of beauracratic risk-aversion. Compliance and legal teams can and often do put the kibash on whole research programs just out of unnecessary risk aversion. In computer vision, just consider the effects of the patents granted for SIFT and SURF, which are many orders of magnitude smaller in scope than even the most graciously assumed narrowing of these Google patent applications. And SIFT and SURF had big chilling effects on various lines of research back when manual scale-invariant feature construction was state of the art. I even worked for a company that required my team to abandon and remove all use of OpenCV period once SIFT was added, just because we could, in theory, be sued into proving that we weren’t using SIFT through OpenCV. Companies will take these bonkers extreme stances. It’s bad for innovation overall. The corresponding effects for these Google applications would be huge: no TensorFlow, no Keras, unless it’s used via vendor lock-in with cloud providers that either are Google or pay Google — to be clear even if the ultimately awarded patents are much narrower in scope.
- The other huge thing is that Google puts a burden on civic infrastructure here. They are requiring the PTO to analyze the applications, and the public is trusting that the PTO has adequate expertise, time and budget to get the answer right. But Google knows that by making the claims overly broad, when they are inevitably walked back to narrower claims, the likelihood for PTO to get this completely right is super low, and with lobbying any of the round of error is sure to go in Google’s favor.
So even if you don’t see it as a bad faith action from the specific patent applications, it’s clearly bad faith in terms of inducing secondary effects of extreme legal risk aversion and basically wasting taxpayer resources to play this stupid cat and mouse game of how to narrow down the claims.
When I ask myself what is the long term self-interest here, it seems clear and in lock-step with other Google strategies: to try to own all the pipes by which modern machine learning can be commoditized, and to try to create a future where machine learning is only consumed as a service from the hybrid cloud-and-consulting platforms of a few entrenched incumbent corporations.
Maybe it made sense before the internet? I'm definitely not arguing that these patents are a good idea, and the argument that they're defensive feels weak since just straight up publishing puts it into the public domain where it acts as prior art to prevent other patents. The only way defensive patents make sense versus just publishing in the open is if you're using them as a threat against other people with patents in your area. The whole thing is gross.
Patents never made sense for software, but the horse has bolted and the law doesn't look like it's changing soon.
Defensive patents made sense until it turned out that most offensive action was from patent trolls who are not trying to create a product so they don't need to defend anything.
You're right; DeepMind has applied for patents covering this work, but the patents have not yet been granted. I meant to write "covered work" but after some editing ended up with "patented work" instead, which is inaccurate. Thank you for pointing it out.
These aren't even software patents. They are patents on mathematical procedures.
Just skim over US20180075343A1 [0] to see what I mean. It's not a patent on a particular application of sequence processing using neural networks. It's not a patent on a particular neural network architecture. It's a patent on the generic use of neural networks for sequence processing. It's not a specific process, it's an idea that encompasses a broad class of processes. There is no sane patent system in which that should be patentable.
Frankly, I don't have a problem with patenting specific algorithms or software implementations. Those, to me, are specific processes. These DeepMind patents are on another level.
Edit: I'm not a patent lawyer. Please correct me if I've read that patent wrong. It's basically a neural network mini-textbook/literature review.
Often a patent will include background information to assist the reader in understanding the claimed invention. The text will usually also point out the shortcomings of prior inventions and how the present invention is an improvement. Neither of those are part of the invention being claimed.
To understand the scope of what is claimed, you always need to look at the claims. The claims will always specifically point out the scope of the invention. Note that all elements in a claim are necessary. If an accused infringer's activity is missing an element, then it does not infringe.
I think that if you look at the claims of the patent you reference, you'll find that the claims are limited to particular applications of sequence processing using particular neural network configurations.
Not sure about the patentibility of those patents (they are still all in the application phase, and I have not read them), but to my understanding patents a) have to be tested before court and b) often fall back to less generic claims in practice (claim no. N, with N > 1), if not being completely invalidated. Now, I expect Google to have good patent lawyers, but time will show the value of those patents.
Usually when patent applications are made they are written up to be as general as possible and will then steadily be whittled down until they're accepted.
That said, I really don't see how a number of these will get even close to validation. Quite a few are describing an absolutely generic, everyday process with so much prior art it's insane (recurrent neural network?). If I'm missing anything here I'd love if someone could correct and inform me.
Many times companies will file patents on any IP they design that is broad enough to disrupt their market. With the patent being exceptionally broad, it ironically leaves less room for questions when the patent request is filed. One would hope everyone on the Government side of patents would be well versed enough in the given technology but unfortunately that is rarely the case so having a broad patent is sometimes easier to push through.
You are reading it wrong. The things they are patenting look like the following:
1. A neural network system implemented by one or more computers, wherein the neural network system is configured to generate an output sequence of audio data that comprises a respective audio sample at each of a plurality of time steps, and wherein the neural network system comprises:
- a convolutional subnetwork comprising one or more audio-processing convolutional neural network layers, wherein the convolutional subnetwork is configured to, for each of the plurality of time steps:
- receive a current sequence of audio data that comprises the respective audio sample at each time step that precedes the time step in the output sequence, and
- process the current sequence of audio data to generate an alternative representation for the time step; and
- an output layer, wherein the output layer is configured to, for each of the plurality of time steps:
- receive the alternative representation for the time step, and
- process the alternative representation for the time step to generate an output that defines a score distribution over a plurality of possible audio samples for the time step.
This appears to me to be describing an autoencoder for audio compression.
There are two other claims as well, all related specifically to audio processing.
Now whether those actual claims are valid is a separate question. I don't know the state of the art in 2016, they could have been. They are still math, I agree. But that's how you tell what they are actually claiming, you can entirely skip the body of the patent and just read the numbered claims.
The ones that don't start with "The ... of Claim ..., wherein" are independent claims meaning that they stand on their own. The ones that do read that way are typically used to narrow the independent claim so that when they say the independent claim is too broad they can make those dependent claims mandatory.
So for instance, if the USPTO (probably) rightly claims that generating audio with a convolutional neural network isn't patentable, they can fall back to saying "The neural network system of claim 1, wherein the audio-processing convolutional neural network layers include one or more dilated convolutional neural network layers." is valid combined with claim 1 because no one has thought of that specifically, and perhaps that's all that is granted.
In that case, as long as you don't use a dilated convolutional neural network layer then it is not infringing.
It's still all ridiculous nonsense, but that's how you read the patent.
Yes, the ratio is pretty low for DeepMind. Higher for Google Brain.
Still, DeepMind open-sourced two major frameworks: Sonnet (general DL) [0] and DeepMind Lab (RL environment) [1].
Frameworks != reproducible results. Ideally, they'd release all the code you need to get the results from the DDPG paper. I recognize that the code probably depends on proprietary DeepMind tools, making this not feasible. Still, it sucks.
Do you know that number? I have yet to hear a single instance or even rumor of that happening. Your post is the first time I've even seen it mentioned. Do you have any proof to back it up? When you make bold claims like that, I think the onus is on you to back it up.
I am not aware of them ever doing that. There is only two cases where there has been any IP aspect that I am aware of?
One started by Moto before them purchasing and the other the Waymo trade secrets case with Uber.
Google has purchased tons of patents but they use for cross license and NOT for royalities.
Are you confusing Google and Microsoft? MS does use their patents for royalities and perfect example is getting more per Android phone sold from royalities than Google gets.
They will sue if they see an emerging threat to the survivability of their core buisness. No one can touch Google on their ads and search ecosystem so they won’t sue. This is an illusion of benevolence.
There is two cases of protecting IP in the history of Google. The first started at Moto before Google purchased and they let continue.
The other was the best known which was the Waymo suit with Uber.
Google gets patents to protect themselves from someone else getting but do not use.
Google instead gives away tons of IP. Like Borg with K8s. Or VP8 and then on top indicates they will protect anyone that uses from patent infringement and on top gives for free.
Or SPDY or map/reduce or GFS or so many others could have patent and hurt the industry but instead gave away and helped the industry push forward. Can't think of any other company that has given away more valuable IP than Google.
Maybe. But also realize you have to protect patents. So if they do not which clearly they do not then no issue.
Also realize Google culture was there when they started and weak and now they are dominate it continues.
Perfect example is owning Chrome and Android which are the dominate two clients. Then own the top 2 web sites with Search and YouTube.
They create a better protocol which saves money with SPDY. Gives a better user experience. But even thought they have NOTHING to gain sharing they do and give away to the standards committee. They do NOT use as a competitive advantage which is just standard good business practices.
That is just not what MS did when in power.
Instead they give it away to help everyone. Also so much very valuable IP they just give away to the industry to help everyone. Kubernettes is a perfect example. But there is so many more. Why on earth do you help your competitors?
But they help competitors that do just evil things against them. They give away Android to Amazon who uses on their Echo, Dot and Fire among other hardware. In return Google gets a ban for ALL companies being allowed to sell the competing Google products. Not just Amazon will NOT sell as that is understandable but they will NOT allow anyone to sell them!
But then SPDY is changed and Google goes back and pays the expense of supporting the changes.
> But even thought they have NOTHING to gain sharing
This is wrong. They would have been ripped apart by people on the Internet if they switched to a proprietary protocol specific to Google for Google properties. Tech users brought the world to Chrome, tech users would just as quickly take them away if they pulled a Microsoft.
>Kubernettes is a perfect example.
K8s is not Borg. It was rewritten in open source. Xooglers could have started it just as easily without Google because the idea is very simple to spread.
>They give away Android to Amazon who uses on their Echo,
It's was open source when they got it and they capitalized on that in the marketing. They didn't "allow" amazon to do anything. They don't allow amazon access to the Google play store so they aren't exactly playing friendly.
>In return Google gets a ban for ALL companies being allowed to sell the competing Google products.
No, the ban is for not allowing native Google apps on Android users (a.k.a Fire phones) that don't pay the tax to Google. Tit-for-tat, which is petty, but tit-for-tat nonetheless.
> Not just Amazon will NOT sell as that is understandable but they will NOT allow anyone to sell them!
I bought a Chromecast at Walmart. Seems like Amazon is only preventing them from being listed in Amazon's marketplace.
>But then SPDY is changed and Google goes back and pays the expense of supporting the changes.
Every browser pays this. That's the Internet. Walled ecosystems with closed protocols die. Google chose the correct move by going open standards, there was no benevolence here.
Google only allowed encrypted traffic with SPDY and why is was used for so long and nobody even knew they were using. So nobody would even have known as nobody did know.
Between their own browser and OS and their own services do not think they would be torn apart. It is not like their services would only work on Chrome or anything like that just better.
Really no different than any browser does proprietary things to make theirs work as well as possible.
But it does not matter. Google gave SPDY away as wanted to push the industry forward and how we got http2 so fast as they basically handed to the standards group not only complete but also since been using for a while they even had the data also to support the benefits.
Why so many companies adopted so quickly. If not for Google we be still debating this and that and not be getting the benefits of http2.
On Borg. Yes K8s is a rewrite from what was learned not only from Borg but another Google solution whose name escapes me.
The point is Google spent a lot of money building Borg and learned from mistakes and fixed and the knowledge gained is extremely valuable yet Google gave it away with K8s. Now Amazon, Microsoft and most other cloud players have it for free and do not have to invest into going through what Google had to in gaining the knowledge. For free and no strings attached and use to compete against Google.
Why? Because it helps move the entire industry forward. Google just looks at the broader tech industry very differently than Microsoft.
Given HN's recent bashing of Google for abandoning it's Don't be Evil mantra this seems like a rather absurd position to take. "Google totes won't abuse this at all!"
It is complete and utter horse shit they are trying to patent things invented decades ago.
2018/081089, "Processing text sequences using neural networks", Priority date: 26 Oct 2016: "In one aspect, a system is configured to receive an input sequence of source embeddings representing a source sequence of words in a source natural language and to generate an output sequence of target embeddings representing a target sequence of words"
From the universal lack of a response on this board of anyone excited to read the patent applications to learn how to get AI to make sounds and such, clearly the US patent system is no longer serving that purpose.
Is it that bad in CS? In my field I'm constantly reading old patents for recipes to solve my problems. But I guess a lot of patents in chemicals are expired comparatively.
Remember that patents are not judged based on title alone! You can't even judge based on the claims in these publications because they are in the application phase, these are not issued patents. The claims that may be eventually granted will define the scope of protection afforded by these patent applications.
They could likely be defensive patents. In a system where patents are abused by trolls and used as negotiation leverages by corporate lawyers (I'm looking at you, Oracle and IBM), its better to file a patent for everything and never actually sue anyone for using those technologies.
Of course this is assuming that Google management does not abuse it themselves.
It's interesting to see Google's AI principles [0,1] published on the same day as these patent applications. Of course, this could be unrelated and complete coincidence. However, this _could_ have been strategic depending on the possible breadth* of what they are trying to claim. (And to be clear, I am not saying this is why they released these principles -- plenty of other reasons for that.)
*Note on the applications - I have not reviewed the claim sets so cannot comment specifically on what they are attempting to claim. In any event (and as others have noted, as well) the devil is in the details (e.g, the claims in each application -- more specifically, what is eventually allowed/granted).
> It's interesting to see Google's AI principles [0,1] published on the same day as these patent applications. Of course, this could be unrelated and complete coincidence. However, this _could_ have been strategic depending on the possible breadth of what they are trying to claim. (And to be clear, I am not saying this is why they released these principles -- plenty of other reasons for that.)*
It's almost certainly not a coincidence. Patent applications with possible international impact are typically filed before any publication, so as to avoid forfeiting any potential foreign patent rights. [0] Patent attorneys get extremely busy before, e.g., trade shows, new-product announcements, etc. — they want to get applications on file to preserve foreign rights.
Totally, you are 100% correct w.r.t. public disclosures and patent filings -- that you want to get the latter done before the former so as to preserve foreign rights outside of the US. Also, more broadly, so as to preserve any rights at all -- following the America Invents Act (AIA) the US is now in a first to file system (like the majority of the rest of the world), not a first to invent system [0,1].
In this case of Google that we are discussing, however, the patents have already been filed but not yet published -- thus, the public did not/could not know what the patents were. Patents publish some time after filing, often times 18mo after the filing date for the USPTO [1] but this timeline can vary for a number a reasons, e.g. an earlier priority date, etc [2].
That all said, much of the work here by Google has likely been disclosed already, but the existence of the patents and what was in them is generally new info for the public and industry.
The referenced AI principles post by Google discussed more about "values" and less about any given tech. That said, my original comment was aimed toward the possible notion of strategic PR (e.g., saying we are going to do 'good' with not just this tech, but also these patents which you are just learning about), not strategic IP (e.g., preserving defensibility for any specific tech).
> following the America Invents Act (AIA) the US is now in a first to file system
This is partly true. If Alice invents a widget and publishes details without filing a patent application, she still has a one-year grace period in which to file a U.S. patent application [1], and if she's successful she'll get only U.S. patent rights (plus a few other countries, I forget which). But whether or not Alice publishes before filing, the longer she waits, the greater the risk that Bob will independently invent the widget, without deriving it from Alice, and will file first. Even after the AIA, we still aren't a "first to file" system — we're a "first inventor to file" system.
Ah — I didn't look at the dates and assumed that the publication was the same date as the patent applications' filing, as opposed to the same date as their publications. In the latter case, it would indeed be a coincidence.
They're almost certainly not as broad as the title would suggest. You've got to look at the claims to figure out what the patent apps seek to cover. The title does not define the scope of activity they seek to cover. Usually, it just specifies the field in which they purport to have invented something.
Did you actually read the patent? Or did you just read the title? The title of the patent is not the patent itself. The details of what they are patenting is described in (sometime very longs and numerous) pages of technical description. I'm an engineer at a big tech company and I'm an author on many patents that have pretty generic/vague names. The actual invention is described in depth in the patent application and that's what's important
Assuming a patent issues with the claims as presented, all of the elements need to be present for there to be infringement. Claim 1 of the application you cited takes up almost an entire page of text and comprises more than a dozen limiting elements, of which you've paraphrased two. If there's a devil here, it's a lot smaller than you're implying it is.
They can't be, unless Google decides to enforce them. It's not the first time they crowd-source massive amounts of specialized effort, only to steal all the data, close them off and get away with it.
Panoramio is the biggest one I know. They essentially stole 10 years worth of effort from a community collecting carefully positioning millions of geo-tagged photos.
Then you have Youtube. It's not completely closed off, but you can't query the video database, you can't discover any of the old content, and they "delete" (hide from the public) all issues they consider controversial or against their political beliefs.
Now it's possible they used the good old 3Es to stimulate academic research relevant to their targets, use it to create useful products, and then patent everything to choke out the competition. Of course they have better ways to kill competitors (imagine the massive potential for undercutting they have, and how nobody cares about undercutting in the online world).
One of the interesting disconnects I see between layperson understanding of patents and legal understanding of patents is how one person's "broad and obvious" is another person's "novel and innovative," and the system does its best to handle that but the tension is forever there.
It's illustrative to remember that Thomas Jefferson himself was sued for using a patented Archimedes Screw design on his property, that he was extremely certain he could have fought in court and overturned since the 'novel' technology in question was known to the Greeks... and he paid the patent-holder anyway because he wanted to encourage patent adoption and use. The system has forever had this tension baked in, and we just muddle through generation after generation.
“This could be... potentially disastrous for innovation.”
This is how I feel about the patent system in general. I genuinely think parents harm innovation by setting artificial limits on innovation in a misguided attempt to incentivise innovation.
I hope they don't get the patents at all. You shouldn't be tied to DeepMind/Alphabet's goodwill over patents in fields they're clearly not the only ones innovating in.
Trump's recent USPTO appointed seems like a patent extremist. He recently argued that the USPTO (which he now leads) should not be allowed to call patents invalid after issuing them. He said the Supreme Court was wrong to rule that.
And I believe the previous USPTO chief was a former Google employee. So, either way, I think Google will get these, unless some organizations like the EFF start going through the process of attacking these patents.
Good thing it's the Supreme Court then, and not the "Maybe do what we say Court."
I get a little excited every time some idiot from the current administration tries to take on US rule of law, because I know the slow machinations of the legal system will eventually spank them into orbit over it.
As if the people of the United States haven't been signing blank checks to the executive branch for decades, and then just flooring the accelerator after 9/11. Your inability to get past the sideshow face of Trump's administration seems to have caused you to completely forget about, say, the free reign the administration has over its drone assassination program? They literally drop bombs on the phone numbers that Palantir spits out. Or National Security Letters that basically give them the legal machinery to completely and wholly subjugate an entity in any way they wish, with rubber-stamp "judicial approval" from a secret court and zero recourse for the victim, including requiring the victim to lie for the government to cover up its crime? Don't let the bullshit with your favorite color make you lose sight of the absolute power "the slow machinations of the legal system" has been corrupted to allow.
This is extremely concerning. Processing sequences using CNNs is particularly bad. It covers using CNNs to process ANY sequences, text, audio, video, etc.. Basically covering any potential use for a CNN. This would like a company patenting Welding. It's just so insanely broad.
I am not saying people shouldn't be outraged, but please read the actual claims rather than just reading the patent titles. Titles are just a general description of the invention. Inventor don't patent titles, they patent a claim that is described in a long technical patent application. DeepMind is not patenting the general idea of a "recurrent neural network", they just have a claim that can be described as recurrent neural network. In the actual patent documents, they need an actual in depth technical description of a new process or invention that is not covered by any existing publication.
Now do people see why it matters that it was Google the one building military drone AI, and not just "Random Company X" working with the Pentagon on this?
Google is far far ahead of others in AI, with or without the patents. The patents just ensure they also get to have a legal monopoly on some stuff to ensure that nobody else can catch-up to them in the technology area.
If your work was "described in a printed publication, or in public use, on sale, or otherwise available to the public" before the patent's priority data, it is available as prior art.[1] That doesn't mean you're immune, per se, but it does mean that nobody can patent exactly what you had made available.
If so, perhaps it would be within your interest to submit a Preissuance Submission showing prior art to the PTO, like ridgeguy mentioned in a sibling to your comment.
By way of comparison, Microsoft is the single largest funder of academic CS research in the world, significantly larger than the NSF or US government, and unlike competing companies like Google and Apple they encourage academic researchers to publish papers on the work they fund to the point where if you don’t publish your work you don’t get more funding from them. Microsoft also doesn’t file patents on the academic research they fund.
But we all know Microsoft is the one that’s evil, because they bought GitHub unlike Google which is merely trying to stifle all use of and research in the most important arena in CS.
Nearly all the authors in that list are google employees, not independent academic researchers.
That blog post is about Google’s employees publishing papers on work google owns, the same sort of work that google is attempting to secure patents on. This is very different from supporting independent academic research.
Everything I’ve heard from academics is that the terms that come with funding from Microsoft are much better and more flexible for the academic than the terms that come with funding from Google, and there is certainly a great deal more funding available to academics from Microsoft than Google.
That said, given that my point was about the absurdity in difference in responses to Microsoft’s actions and Google’s actions, I’m particularly amused(?) that here we are, in a page about Google applying for patents that would cover a tremendous amount of essential CS research, and the primary thrust of your points here seems to be “Dude! Google’s Awesome! They’re so supportive of research!” which is pretty much exactly the response I was trying to highlight. Don’t be evil was a wonderful way to start their branding. It doesn’t matter that’s not the company they are anymore, it’s still how people think of them.
You are trying to frame people's responses into a "Google good, Microsoft bad" response, and I do not think that is fair to them at all. They are disagreeing with your idea that Google is non-existent on the academic research front, and Microsoft is doing way more than them. And frankly, as someone in academia, this does not pass my smell test either.
I’m baffled where you get the idea that I think google is non-existent on the academic front. Yes, Apple is not-existent, but that’s certainly not something I believe or would say about google, because it’s not the case. I did say Microsoft is the #1 funder of academic research in the world, which I believe is still true and which no one seems to be challenging [edit: it looks like there is some discussion of whether Microsoft’s funding is bigger or smaller than the government’s contribution, but all within a context of Microsoft certainly being the biggest NGO funding source and possibly being the biggest overall]
Again, I think it’s fascinating how in commenting on a post about google trying to file patents that would completely control the most important new field in CS, someone who correctly points out that Google is not #1 in funding academic researchers is branded as calling Google “non-existent on the academic front”.
This is precisely the sort of unproductive back and forth argument that the HN software is designed to discourage and punish, so I’m going to try to back off here and say hopefully we can agree to disagree, particularly since it doesn’t sound to me like we actually disagree.
I admit that I exaggerated with my wording that you think Google is non-existent on the academic front. It's unfortunate that your response focuses almost entirely on this one aspect, and not the message my post attempted to convey.
> someone who correctly points out that Google is not #1 in funding academic researchers
This type of wording is unnecessary. You are declaring that you are correct, while at the same time not providing supporting evidence when requested.
> This is precisely the sort of unproductive back and forth argument that the HN software is designed to discourage and punish
Welp. I am sorry this had such a negative effect on you. We can certainly agree to disagree. Cheers.
Since we asked you to stop using HN to prosecute agendas about Google and you've posted dozens of comments since then doing exactly what we asked you not to do, I've banned this account.
Obviously there's nothing wrong with saying nice or critical things about $Bigco on Hacker News, be it Google or anyone else. But when it becomes both predictable and proliferating, it stops serving intellectual curiosity and that violates the purpose of this site. You crossed that line a long time ago, unfortunately.
I'm not saying "Google's Awesome!". My point is that the view you presented is far more biased than what I get from my colleagues and friends that stayed in academia. People running around like headless chickens in the "MS buys GitHub" thread was unfounded, and it is unfounded here.
Yodon, The difference is Google gets the patents so nobody else can and does NOT use as a weapon or for royalities.
Microsoft just takes a different approach. They use the patents for royalities and how they make over a billion off of Android phones a year.
Google has tons of patents and even buys them up like what they did with Moto. Yet they do not charge royalities for using. They instead use for cross license agreements.
Just a very different culture from Microsoft.
But one of my favorite examples of Google and patents is VP8.
We had Mpeg-LA extorting license fees out of us for Mpeg2. Google said enough and created VP8 and gave away for free for anyone to use.
But then in addition also offered patent infringement protection for anyone using VP8. That is just not an approach you would see from Microsoft.
"It doesn’t matter that’s not the company they are anymore "
What has changed? I would say they are giving back more now than ever before. They use to give their "secrets" in papers like Map/Reduce, GFS, and so many others.
But now they actually give the software away with their "secrets". Perfect example is Kubernettes. They build Borg and learned from it and then with everything they learned now give away K8s.
Which is now used by Microsoft, Amazon and pretty much the entire industry. They do not charge a cent and help their competitors better compete against them.
They did it because it helps the entire industry move forward. That is just not the culture at Microsoft.
Google continues to give away just tons and tons of IP.
But you do have me curious. How do you think they have changed in terms of the technology community?
BTW, another example is giving away SPDY which became http2 which helped everyone including competitors.
Or all the money they spend finding all the major security flaws including Shellshock Meltdown, Cloudbleed, HeartBleed, Spectre and many others. Cloudbleed was with a competitor and helped them solve the issue.
Google finds many Microsoft vulnerabilities and tells them about them helping them make their products more secure.
I was going to link to exactly the same thing. Only considering these 2017 faculty research awards, that is over $22 million awarded to academic research. Not to mention Google frequently uses the model where professors work at google for a summer / semester / year on a funded project, but are not permanent employees.
Even if it was, publishing paper is still contributing to academic research. How is contributing yourself somehow worse than just putting your stamp on other people's work?
What is your source that Microsoft funds more CS research than the US Government? Let's ignore any DoE, DARPA, or NIH sources of funding, and just focus on the NSF. The computer science portion of their budget is almost one billion dollars.
> What is your source that Microsoft funds more CS research than the US Government?
Microsoft reported spending $14 billion on R&D in 2017. That year, NSF's CISE budget was somewhere around $950 million.
The comparison is apples and oranges, because NSF is funding public research and a lot of basic research, whereas one must assume the vast majority of that $14B at Microsoft is spent on the D portion of R&D.
Still, though, MSR is one of the more prolific research institutions in CS. If they were a university, they'd probably be top-ranked.
Still, though, MSR is one of the more prolific research institutions in CS. If they were a university, they'd probably be top-ranked.
I definitely agree. They have some top notch researchers, certainly.
I also agree that the budget comparison is apples and oranges, so I'm not quite sure why you brought it up. The OP made a grandiose claim about Microsoft's academic research funding, and made it quite clear with his discussion of Google that corporate research does not count. So I want a source for the amount of money Microsoft spends on pure academic research without the corporate ties.
Personally, I think if Microsoft spent anywhere close to a billion dollars on pure academic research, the funding situation in CS academia would be completely different. Microsoft grants would be pursued just as hard as NSF grants, because they would be such a major funding source. But that is not the case. Corporate grants are afterthoughts compared to the NSF for academics. Additionally, if Microsoft were plunking such a large amount of money into academic research, I assume they would at the very least publish that figure in a PR piece to cash in on some public good will.
> I also agree that the budget comparison is apples and oranges, so I'm not quite sure why you brought it up.
It's annoying that MSR's numbers aren't broken out from the R&D total. All we can do is speculate what portion of that goes to MSR et al, really.
But it's not surprising. There are all sorts of incentives for not breaking out those numbers. Still, the numbers are helpful. $14B is a lot of money. Even a small slice of that makes MSR one of the most prominent funders of CS research.
> Personally, I think if Microsoft spent anywhere close to a billion dollars on pure academic research... Microsoft grants would be pursued just as hard as NSF grants, because they would be such a major funding source.
I disagree with your perspective throughout this paragraph.
Funding grants for university faculty is NOT the only valid model for funding pure research. Therefore, we could very easily expect MSR to contribute a significant fraction of total US CS research funding without decreasing the competitiveness of the grant landscape for professors at universities.
To me, the distinction between "basic research" and "everything else" seems more important than the particular funding model used. So the criticism that Google has a bad track record with long-running research agendas is highly relevant, whereas 'how much money goes to academics in universities' is far less relevant.
> Additionally, if Microsoft were plunking such a large amount of money into academic research, I assume they would at the very least publish that figure in a PR piece to cash in on some public good will.
But of course, they already do that: "$14 Billion!"
> $14B is a lot of money. Even a small slice of that makes MSR one of the most prominent funders of CS research.
You are moving the goalposts. I am not saying Microsoft is not a prominent funder of CS research. I am challenging the notion they spend more than the NSF (or for an even stronger claim, the entire US Gov) on non-corporate CS research.
> But it's not surprising. There are all sorts of incentives for not breaking out those numbers. Still, the numbers are helpful. $14B is a lot of money. Even a small slice of that makes MSR one of the most prominent funders of CS research.
No, those numbers are not helpful. We have absolutely no idea what percent of that goes to academic CS research. It could be 10%, that would be incredible. It could be 1%, that is still great. It could be 0.01%, that isn't very notable.
> Funding grants for university faculty is NOT the only valid model for funding pure research. Therefore, we could very easily expect MSR to contribute a significant fraction of total US CS research funding without decreasing the competitiveness of the grant landscape for professors at universities.
To me, the distinction between "basic research" and "everything else" seems more important than the particular funding model used. So the criticism that Google has a bad track record with long-running research agendas is highly relevant, whereas 'how much money goes to academics in universities' is far less relevant.
You are again moving the goalposts. The OP specifically said Microsoft was the largest funder of academic CS research.
I think those numbers are useful. Microsoft spends $14B on CS R&D. We can go back and forth about the R and the D, but that's still a freaking huge number.
> The OP specifically said Microsoft was the largest funder of academic CS research.
Needless to say, I'm not OP, and I never made any of the claims OP made.
The only purpose of my original post was to try and share what numbers we do have on this question.
And I stand by reply calling BS on the distinction between research based upon whether money is flowing into professors' research groups. Some of the world's greatest patent lawyers are retained by universities, after all.
> unlike competing companies like Google and Apple they encourage academic researchers to publish papers on the work they fund
First, you cannot compare Google and Apple when it comes to academic research. Apple barely does any, Google is usually the largest source of papers in all major ML conferences (when it's not FB): NIPS, ICLR, ICML, etc.
I think you are confusing academic research and corporate research. Google has a huge stable of corporate researchers who publish papers in academic journals. Microsoft funds actual academic research.
And yes, my inclusion of Apple in that list was fully aware Apple doesn’t do significant funding of academic research. That’s the point.
So just because Google hires researcher and gives them resources, their publication is somehow worth less than Microsoft paying outside researchers?
Why is one somehow superior than the other? Both are publishing papers and pushing academia forward, why does it matter where the researcher are doing their work?
Frankly, when I read a machine learning paper without looking at the list of authors, it is impossible to tell whether it is written by "corporate researchers" or "actual academic researchers". I'm not sure the distinction you're making is really helpful. As long as the research is published openly, peer-reviewed, discussed in conferences, whether the researchers work for the company or for a university does not make much of a difference.
Most of these "corporate researchers" were top of their field in academia before joining Google, Facebook, Microsoft. Furthermore, Google also funds academic research [0]... I'm not even sure where you're getting all your information at this point.
To be fair the boundary between academic and industry research is quite gray now - lots of collaboration, many PhDs doing internships, etc. Still, it's true MS is not given enough credit.
Microsoft is also one of the largest patent royalty extortionists in the industry.
Can you imagine if Microsoft had these patents? Every single internet-connected device with "AI capabilities" would be paying Microsoft a royalty within 3 years after the company would obtain the patents.
Researchers at MSR can't publish everything they work on, you know. In a sense, MSR is also a patent mill. Sure they fund research and a good deal of it is published, but it's not simply out of the goodness of their hearts.
The comment was about academic researchers being encouraged to publish, not MSR researchers. People who work for MSR are Microsoft employees and therefore are corporate researchers not academic researchers.
Google has just given back so much more than Microsoft. Great example is Google dominates with Android and Chrome the client and server and creates SPDY and gives it away. Heck it was changed and they even switch from SPDY to http2.
Or a big one is MpegLA was commiting basically extortion and Google created VP8 and gave away for free. But then did a crazy thing of even giving people that use for free protection from patent infringement. Who does that?
That is why we do not pay what we had to for mpeg2 licenses. Just not aware of similar from MS?
Or a big one is Google finding Shellshock, Spectre, Cloudbleed, Heartbleed, Meltdown and others. MS has so much to lose and does not find anywhere near what Google does to help the entire industry. Plus Google even shares mitigations.
Instead Google finds MS vulnerabilities and gives them 90 days and MS b*thches about it instead of thanking.
But then all the papers of things that are now just defacto standards. Like GFS, Map/reduce, Bigtable, Borg with K8s and just so many more.
The big difference is Microsoft uses patents to get royalities out of people and use as a weapon in slowing down the industry versus helping push the industry forward.
Google does not play that game. They have tons of patents and purchased patents for cross license but they do not get paid anything in royalities. Instead they give away IP for everyone to use.
That is why Microsoft gets paid more for every Android handset through royalaties from patents than Google does.
Just a different approach between the two companies.
This is so disappointing. DeepMind deserves to make money off their research--they really are doing amazing stuff--but I feel like these patents will completely stifle any AI research outside of Google if they go through.
"Processing {text, video, audio, images} using neural networks" and fundamental concepts like reinforcement learning or RNNs shouldn't be patentable :/
The incentives in the USPTO process align to encourage companies to patent things they're using if for no other reason than defense against patent claims. Google may be patenting these things to ensure anyone can continue to use them (because "benevolent dictator" is the only model the US patent system really allows for).
Anybody already can use them because neural networks are already out there in the commons.
That in fact shows how unreasonable it is to claim patent rights here. The original function of a patent was to reward an inventor for their effort by granting them monopoly rights in exchange for all information about the invention to the public, that's why patents enforce that inventors need to disclose their inventions, that's what society gets in return of granting a monopoly.
In the case of neural nets this has already happened. Almost all of it is based on public research, especially the things those patents try to protect.
Google does not enforce patents. They are for protection so someone like MS does not get and use to get royalities. Heck MS gets more per Android phone for royalities than Google.
Could you imagine how much it would cost us if MS had the patents?
"Microsoft makes much more money from Android than Windows Phone. Every time you buy an Android smartphone or tablet, Microsoft is likely receiving $5 to $15. They likely make at least $2 billion per year from Android."
"Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for environment simulation. In one aspect, a system comprises a recurrent neural network configured to, at each of a plurality of time steps, receive a preceding action for a preceding time step, update a preceding initial hidden state of the recurrent neural network from the preceding time step using the preceding action, update a preceding cell state of the recurrent neural network from the preceding time step using at least the initial hidden state for the time step, and determine a final hidden state for the time step using the cell state for the time step. The system further comprises a decoder neural network configured to receive the final hidden state for the time step and process the final hidden state to generate a predicted observation characterizing a predicted state of the environment at the time step."
From "Recurrent Neural Networks" patent. This doesn't sound narrowly constrained to me. This just sounds like a RNN, which according to wikipedia, were conceived in 1982.
That is the abstract of an application--it does not define the protection of a patent. It is intended to aide someone browsing patents to determine if this document may be something worth looking into closer, it is supposed to be quite broad and vague.
That is what I see, as well, with the GENERATING AUDIO USING NEURAL NETWORKS patent application. IANAPL, but I do work in audio software and the application, seems to me, explains a broad model of audio processing using convolutional neural networks (in the claims an input is involved, so I am guessing the title tried to select an even broader set of problems - with & without input).
I think people pointing out that Google defensively files patents may be correct and I hope that's the intention with DeepMind's work, too. I am a fan of the audio work they did with WaveNet since day 1 and was so happy the paper was out for all of us to read. Still, I was hoping I may be able to (read: "daydreaming that I might be capable to") build on it, some day.
Organizations change leadership and they switch focus, too. I believe patents slow-down (even stop) progress and they are quite hypocritical, overall (show me a patented invention that was so unique, the author/owner didn't "stand on the shoulders of" 100's of years of freely available science to derive/discover/invent it and... I will agree for that specific case).
No, really, you shouldn't. We do not know what they are going to 'monopolize' yet. A patent application, even if published, is not enforceable, it is not a patent. I could file an application today claiming to have invented the wheel, and 18 months from now it would publish as an official looking patent publication. It means literally nothing.
Because of the way US patents work, it really is a "We should file these so somebody else doesn't claim-jump them and put us in the position of defensively proving prior art" system.
As a deep-pocket company, Google can afford to file patents and has a track record of being responsible about them in the community. It makes about as much sense to be mad at DeepMind for not filing them.
Since AI is the last great problem in computer science, I can't help but feel a tremendous sense of sadness and foreboding about this. I always wanted to dabble in AI in that far off mythical future when I have the time to do so. But if it all gets patented, what's the point? Might as well patent relativity, or quantum mechanics.
AGI is achievable, but we are along way away from it. There's still plenty of time to get involved! If you can't get involved now, at least try to influence lawmakers to remove the privilege of firms to patent algorithms, which will stifle research progress.
No, see... humans won't be doing any more problem solving when we have AGI. We'll just kick it and drink Mai Tais on the beach while they do all the work.
I love when people say "dabble in ai". The mathematical foundation to even begin truly understanding ai isn't something you can dabble your way through. - so am i being downvoted because im wrong or because i hurt your feelings? jesus fucking christ.
Probably it reads like textbook gatekeeping and doesn't add much to the discussion.
Granted this doesn't really either, but I'd rather people be interested in learning something rather than just be intimidated because they can't "really" learn anything about AI. That's just plain mean-spirited.
I upvoted you a little because I think the perception that the mathematical foundations of AI are too complex is valid. AI is actually pretty straightforward from what I can tell, but it has a problem with notation. I think that probability and quantum mechanics also suffer from (context-heavy?) notation and steep learning curves.
I'm hopeful though that as more people dabble in AI that the notation problems will be alleviated somewhat. My gut feeling is that its abstractions are still too low-level, like the early days of CS when people worked with binary, assembly or even LISP. Once we begin to see the forest for the trees (something analogous to MATLAB or Elixir but for AI), then I think we'll finally see the utility of AI realized for mainstream developers.
How are methods like these patentable when the research in these domains has been openly published and built upon by the community already? For example, the "Generating audio using NNs" patent abstract is:
Methods, systems, and apparatus, including computer programs encoded on computer storage media,
for generating an output sequence of audio data that comprises a respective audio
sample at each of a plurality of time steps. [...]
Patents are a weird beast. They are written to be as broad as human language could possibly be coaxed to allow. The abstract matters for very little. There will be a list of specific claims later in the doc. These count for everything.
The claims will usually begin obscenely broadly, practically "I have invented a way to perform calculations with electricity." Each subsequent claim will add an additional refinement/limitation to the previous one, and then lawyers will play games trying to see how broad of a claim can be admitted. The patent attorneys will eventually let the applicants claim the broadest claim for which no prior art can be easily discovered.
Filing a software patent is a weird undertaking. You describe to a lawyer your idea, which takes like 2 minutes. Then they have you brainstorm for every possible extension of it you can think of for an hour or so. Then the lawyer disappears and comes back with a 60 page document that seemingly makes almost no mention of your actual idea. Then you sign that you definitely invented whatever the word salad says.
The abstract doesn't really have all that much bearing on the actual contents of the patent. Remember the lessons of "How to read a patent in 60 seconds" [0]. Skip the title, skip the abstract, skip the dependent claims. Read just the independent claims.
"when the research in these domains has been openly published and built upon by the community already"
Well, mega-corps like Google lobbied for a first-to-file system (America Invents), for one thing. Community shmunity; whoever has the most resources to file the most applications wins.
Not necessarily; the major claim in the original WaveNet research was that they were the first to have a neural network directly generate audio samples.
(Not arguing that these aren't extremely broad and problematic patents, but Google may well be the first in many of these areas.)
How does an NN generate audio without generating samples? The outputs are somehow interpreted as a continuous function / polynomial with tons of terms? And nobody ever once tried to not do that?
Somebody at Alphabet or DeepMind decided that the research they produce wasn't tangible enough to justify their cost, so they have to demonstrate 'real business value' of their research. For bottom line business people, this means patents.
This is just speculation on my part, but from my experience, this is how/why this happens in a large org.
These are not enforceable anywhere -- they are applications. If and when they issue into patents, those will be enforceable in their respective territories. A PCT filing such as these, however, can go national stage in any of the PCT countries, which includes pretty much the whole world.[1]
(From a US perspective)
The disclosures I've flipped through reek of non-patentable subject matter (software/algorithms with no defined field of use), lack a strong inventive step (ok I'm not so sure but it seems like a lot of these are already being researched/used), and have a claims list that is LONG.
It only takes one claim to not be met for a product to not fall under that patent. So if the exact processes described in the claims are not followed or even altered, the offending party could see DeepMind in court (where they would, unfortunately, likely lose).
For a small(ish) company good alternative to patents is trade secret. If you really invent something valuable, never publish it.
Companies like MS, Google, Amazon can justify free academic research because they have infrastructure that uses the research. They don't make their profits directly from the research.
Deep Mind can only get value from fundamental research using patents or trade secrets. Being just a consulting gig company does not generate enough profits to justify basic research.
Its the same in electronics I'm more familiar with. Small design company can from IP licensing, but if you want to keep working with some fundamental research that does not fall under a copyright, you must choose between patent or trade secret to benefit from the research.
I’m very concerned to hear that Demmis Hassabis is a genus. I knew he was smart, and I was prepared to accept that he may be a different species, but I was hoping that he would maintain at least some backward compatibility with the primates.
One of the biggest issues with enforcing patents is "how do I recognize (and then later prove in court) that my competition is infringing on my patent?
Since you can't easily just go have a peek your competitor's code, it's going to be a challenge to come up with an enforcement strategy.
These are defensive patents. Google entire culture is different than say Microsoft.
Google believe in all ships rising also helps them. Versus lowering competitors ships helps them which is the Microsoft and other old tech companies approach.
That is why they buy up patents like what they did with Motorola but use for cross licensing NOT to collect royalties. We look at Android and Microsoft collects patent royalties such as they get more per Android device than Google. That is just the difference in cultures.
Google even lets one of their chief competitors, Amazon, use Android without any fees paid to them. Amazon turns around and uses for the Echo, Dot FireTV and majority of their other hardware and then bans everyone on their marketplace from being allowed to sell competing Google products.
Not just Amazon will NOT sell but bans every other company on their market place from being allowed to sell.
Yet Google still just gives it away.
Google owns Android and Chrome which are by far the dominate clients. Then owns the #1 and #2 most popular Internet sites with search and YouTube.
They create SPDY which is a better solution and save cost and really no reason needing to share. Yet they give it away to the standards committee. Who then changes and Google goes back and moves to the standard with HTTP2.
Google is doing this while being in the dominate position. When MS was dominate you would never see them doing something like this. Honestly it is not really smart business but glad Google cares more about the overall industry moving forward. They get all ships rise is good also for them.
Security is another perfect example. They invest and find all the major security vulnerabilities like Shellshock, Cloudbleed, Heartbleed, Metdown, Spectre among others.
They share these vulnerabilities and even mitigations to the entire market because all ships rising helps Google and just how they look at it.
They find vulnerabilities with Microsoft and give them 90 days to fix. MS fails to fix and b*tches instead of thanking Google.
One of the biggest is the silliness from the Mpeg-LA. They charge ridiculous license fees with mpeg-2. Google saw that and invested into creating VP8. That forced the Mpeg-LA to stop extorting money.
But Google does not keep it just for themselves and use as a competitive advantage like any "good" business would do.
Instead they not only give away for free but then also protect anyone from patent infringement. That is crazy.
Give away for free and then also protect?
Some of the IP they give away is just crazy. Borg was novel and just incredibly valuable and they give away Borg through Kubernettes (K8). Who does that? Well all ships rise and so do they.
They do not charge royalties for IP like Microsoft and the other do. They sure have some incredible IP but they just give it away. Map/Reduce, Text2Vec, GFS, WaveNet, or so many different incredible papers. I just love they share their "secrets" and help everyone move forward and do not just keep to themselves.
But they also do the "right" thing and not something that just gets them a buck or a billion bucks.
Perfect example is China. The China government tried to hack Gmail accounts of their citizens data. Google walked away from billion in China and said that is not acceptable.
But then we have Apple hand over all their customers data to the government but also the encryption keys. Just the perfect contrast you can have.
"Campaign targets Apple over privacy betrayal for Chinese iCloud users"
Google is a new tech type company that has a very different culture than the old tech companies. They believe in all boats rise helps also them instead of trying to hurt competitors with things that do NOT move the industry forward.
Exactly. Personally, i widh larry,sergey came out in public more often. I think Alphabet is getting hurt coz of their absence. Google gets really less appreciated. I guess people forget the Good when even a little bad is committed.
I think it's important to note, and keep in mind, that DeepMind is explicitly trying to create Artificial General Intelligence. In their words:
"solving intelligence, and then using that to solve everything else"
Even just five years ago, those of us who attended AGI conferences or discussed AGI would get kicked out of the room for publicly contemplating the topic.
Patented work includes:
* 2018/048934, "Generating Audio using neural networks", Priority date: 6 Sep 2016
* 2018/048945, "Processing sequences using convolutional neural networks", Priority date: 6 Sep 2016
* 2018064591, "Generating video frames using neural networks", Priority date: 6 Sep 2016
* 2018071392, "Neural networks for selecting actions to be performed by a robotic agent", Priority date: 10 Oct 2016
* 2018/081089, "Processing text sequences using neural networks", Priority date: 26 Oct 2016
* 2018/083532, "Training action selection using neural networks", Priority date: 3 Nov 2016
* 2018/083667, "Reinforcement learning systems", Priority date: 4 Nov 2016
* 2018/083668, "Scene understanding and generation using neural networks", Priority date: 4 Nov 2016
* 2018/083669, "Recurrent neural networks", Priority date: 4 Nov 2016
* 2018083670, "Sequence transduction neural networks", Priority date: 4 Nov 2016
* 2018083671, "Reinforcement learning with auxiliary tasks", Priority date: 4 Nov 2016
* 2018/083672, "Environment navigation using reinforcement learning", Priority date: 4 Nov 2016
...
Is there anyone from DeepMind on HN who could comment on this?