Why are people being so critical about this work? Sure, the blog post provides a simplified picture about what the system is actually capable of, but it's still helpful for a non-ML audience to get a better understanding of the high-level motivation behind the work. The OpenAI folks are trying to educate the broader public as well, not just ML/AI researchers.
Imagine if this discovery were made by some undergraduate student who had little experience in the traditions of how ML benchmark experiments are done, or was just starting out her ML career. Would we be just as critical?
As a researcher, I like seeing shorter communications like these, as it illuminates the thinking process of the researcher. Read ML papers for the ideas, not the results :)
I personally don't mind blog posts that have a bit of hyped-up publicity. It's thanks to groups like DeepMind and OpenAI that have captured public imagination on the subject and accelerated such interest in prospective students in studying ML + AI + robotics. If the hype is indeed unjustified, then it'll become irrelevant in the long-term. One caveat is that researchers should be very careful to not mislead reporters who are looking for the next "killer robots" story. But that doesn't really apply here.
I personally think they did great. They targeted the blog post at a more general audience so most people can follow and get an overhead view of the idea, and then put two giant buttons for "View code" and "Read paper" right at the top of the blog post for those who want more technical writing and working code.
Agreeed. As a non-ML developer, I think this is the paragraph that sells the work, even if it may be an oversimplification:
We were very surprised that our model learned an interpretable feature, and that simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment. We believe the phenomenon is not specific to our model, but is instead a general property of certain large neural networks that are trained to predict the next step or dimension in their inputs
I think it says something very interesting about human language and information processing in general.
As someone familiar with the field, you likely know this already, but the similarities between the Karpathy post from 2015 and this work from OpenAI is likely because Karpathy is a founder and lead researcher at OpenAI.
If they're getting better results than the previous state of the art, I think the most important point of this research is its critique of previous papers! The previous state of the art research needs to consider such old, basic techniques to improve their own results.
> Hype is an interesting thing especially when it comes from laymen.
Agreed, as can attributing value to said hype.
An argument can generally be made for many things for why they "can be useful, in X situation" (like 'a layperson understand ML'), doesn't mean it has value in every context.
(Or even that it's a particularly good example for its contrived purpose - just that it could/might suffice if nothing better exists.)
From my little experience with the AI community, I think people in it love to obfuscate things. Any attempt to make a topic approachable, even if some of the details are lost, get smacked around. I face this every day in my Masters. If you don't already come with a knowledge of AI + Stats, you're on your own. The community, including the teachers, don't want to teach the mundane.
Techy people, by and large, don't understand marketing. We think that technology should sell itself, and anyone who needs convincing of the superiority of some solution is just dumber than they are. Combined with the elitism complex we see all over academia and the genuine complexity of AI research as it stands today... recipe for sociopathic disaster.
I'm in the GA Tech program. Chose AI since I've never touched it. Every course has been terrible in this track. The lectures are awful. The readings take hours upon hours. The TAs pathologically refuse to answer guestions without channeling Confucius. Then I found old GA Tech videos and MIT open courseware. It's starting to make sense. I say all of this because intro to ai is highly rated. It can only get such a rating if it's coming from people that already know the material.
Are you in the OMSCS or on-campus program? I'm currently enrolled in CS 6601 (Artificial Intelligence) via OMSCS, and while it's extremely difficult material that requires a lot of personal investment to master, I've found both TAs and other students to be as helpful as one could reasonably expect.
OMSCS. The TAs refuse to answer questions because it could help cheating. They won't review homework because it could help cheating. They've created an environment so full of fear that it's impossible to learn from mistakes. I'm surprised the slack channel is allowed.
I've never had a direct question to a TA go unanswered (and I've asked my fair share of questions). They're not going to just give you the solution to the homework, though. I guess it depends on what kind of questions you're asking?
> next "killer robots" story ... doesn't really apply here.
Are you sure about that? We're talking about a model/robot which understand sentiments, and can generate fake reviews to boost fake products. I can easily see this being picked up by the AI hype journalists. In fact, this model could even be used for nefarious purposes.
AI hype journalists will find something to write about, regardless of the industry making their research accessible to the wider public.
Markov-chain generators have been around for a while, and have been used to throw off spam detectors. This should not stop research, but instead grow more research into adversarial usage of machine learning models.
>Why are people being so critical about this work?
Some people can't stand to miss an opportunity to remind everyone about how smart they are.
However... one criticism I have of the article is that their first graph doesn't start the y-axis at zero, giving a false impression of how much their method improves on others.
Imagine if this discovery were made by some undergraduate student who had little experience in the traditions of how ML benchmark experiments are done, or was just starting out her ML career. Would we be just as critical?
As a researcher, I like seeing shorter communications like these, as it illuminates the thinking process of the researcher. Read ML papers for the ideas, not the results :)
I personally don't mind blog posts that have a bit of hyped-up publicity. It's thanks to groups like DeepMind and OpenAI that have captured public imagination on the subject and accelerated such interest in prospective students in studying ML + AI + robotics. If the hype is indeed unjustified, then it'll become irrelevant in the long-term. One caveat is that researchers should be very careful to not mislead reporters who are looking for the next "killer robots" story. But that doesn't really apply here.