> Other than a handful number of people doing some fundamental research towards understanding the theoretical concepts of these methods, almost all the community now seems to target the development of more complex pipelines (that most likely cannot be reproduced based on the elements presented in the paper) which in most of the cases have almost no theoretical reasoning behind that can add 0,1% of performance on a given benchmark. Is this the objective of academic research? Putting in place highly complex engineering models that simply explore computing power and massive annotated data?
I last dabbled in image processing research around 2011. Probably most of the papers i read during the previous 5 years were small little epsilon papers that added no real value. I did some work in other fields and noticed a similar trend there. I always attributed it to the trend of PhDs being pumped through the system in ever greater numbers and the need for researchers to publish a paper every few months.
This happened to me. I developed a computer vision technique that achieved a well known result but without the same constraints. Although it never surpassed the other technique for most images, it worked in a wide variety of cases that the previous technique did not.
My major professor diluted the paper and added other content consistent with the previous method. Not just adding prior art to the introduction, but changing the meat of the paper so that it didn't seem like a departure.
He assured me that this would make it easier to publish, and publishing was all that mattered. There were no bonus points for publishing a novel technique, and there would certainly be extra work having to deal with referees.
I'm very glad to be out of that environment now. I noped out of academia and happily dealing with corporate B.S.
Having dropped out of University while studying Computer Vision, i've witnessed this much too often in former friends and colleagues.
Smart people doing remarkable things don't seem to have a place in our society anymore, neither in academia nor in the private economy. Sure there are exceptions.
Microsoft Research, Google X and of course a handful of universities that actually work as they are supposed to, but for most of my ex colleague's these weren't real options as noone ever instilled the courage in them to find their way there, or survive the competition.
It's strange that most of them are building CRUD Software, Writing Shaders for Game Engines or work in marketing, instead of pushing us towards breakthroughs in CV and with that AI.
I keep having that same thought. However, making a breakthrough requires 2 things:
1. Resources. Time and money. These research ventures might take years and sometimes need some funding in addition to your own salary. It is basically borderline impossible for most developers to contribute anything useful in an environment where they need to worry about paying their bills for this month.
2. Know-how. Right now, most industries have advanced so much that you need very specialized knowledge and a metric fuckton of math/stats to contribute to any particular field in any significant way. A developer with a BS in CS can most often not even understand the papers being currently published due to the high math/specialized required knowledge.
> Smart people doing remarkable things don't seem to have a place in our society anymore, neither in academia nor in the private economy. Sure there are exceptions.
I think smart people doing remarkable things have greater visibility than they have at any point in the recent past due to the internet and platforms like YouTube, etc. It is easier than ever to share and discover knowledge now than it ever has been in the past..
The biggest problem I see is that there is limited compensation for participating in these endeavors. I think this is an issue from the standpoint that some research is very costly or requires resources that the average person cannot obtain. There is still a great number of discoveries that can be made by people working in labs they have made wherever they found space.
In addition to that, not every publicly funded school provide these publicly funded software packages freely for public use, hoping by keeping them private to exploit their "business value" somehow. At least in this part of Europe.
Deep learning may have become the flavor of the season now, but ironically its pioneers like LeCun and Hinton too faced similar problems when they tried to publish [1].
> He assured me that this would make it easier to publish, and publishing was all that mattered. There were no bonus points for publishing a novel technique, and there would certainly be extra work having to deal with referees.
He was a terrible professor! Yes, publish or perish is real, but that's such a terrible, pessimistic attitude to have. Its like he had given up on trying to actually do research. Tell him to go become an adjunct.
My professor during my Master's adventure was insistent on publishing a lot. He didn't mind if it was little deltas. He burned the phrase "publish or perish" into my brain. But had I figured out something novel, he would have absolutely supported putting it out there. That would have been the whole point of doing this work! Deltas help you survive, but novel ideas are what you should strive for.
> Probably most of the papers i read during the previous 5 years were small little epsilon papers that added no real value. I did some work in other fields and noticed a similar trend there. I always attributed it to the trend of PhDs being pumped through the system
No, the reason for this is that all the easy stuff has already been discovered. 20 years ago it was still kind of easy for a single PhD student to make enormous progress in his field, but after a dozen of PhD generstions there is just not much left which can be discovered by a si gle student.
Just look at physics, they had to build a multi-billion dollar research facility below the ground to advance our knowledge. Or the satelites for discovering gravity waves, ... All of this can not be done in the classic PhD model where a single PhD student works on a topic.
tldr: After 150+ years of science there is almost nothing left which can be discovered by a single PhD student
> tldr: After 150+ years of science there is almost nothing left which can be discovered by a single PhD student
With 150+ years of science lots of new kinds of questions emerged that hardly anyone went deeply into. Just begin working there and make deep contributions. To make a few things clear:
* You probably won't get any academic recognition for this or probably no research funding agency will be willing to fund your research (far too experimental).
* It is quite possible that despite you being really talented your research into this will come to nothing. That is a prize one has to pay for the possibility of doing a deep contribution.
If you are looking for ideas where to start, just look around yourself and from what you see try to create a deep general theory (the more abstract and general the better IMHO) which has lots of predictive power and either allows you to formalize a theory where you can prove theorems about (my personally prefered way, since I'm mathematician) or has strong falsifiable predictions that one can in principle do experiments on.
This. Most of the comments on this thread support your statements - people don't vary because of the risks, the biggest one being not getting published.
I'm an advocate of publishing not just the paper, but the compilable source and executable binary code and datasets used for just that reason. Science is not science if others can't reproduce the results and tweak and permute the inputs, reliably and repeatedly, IMHO.
I feel like someone should start a group or meetup for people who are interested in novel, valuable techniques, rather than whatever will advance their scientific career. It could be filled by hobbyists, including retired people who could spend a lot of time on it.
Machine learning gives performance without theoretical undertstanding. Norvig discusses Chomsky on machine learning in linguistics: http://norvig.com/chomsky.html
> I mean actually you could do physics this way,
instead of studying things like balls rolling down frictionless planes, which can't happen in nature,
if you took a ton of video tapes of what's happening outside my office window, let's say,
you know, leaves flying and various things,
and you did an extensive analysis of them,
you would get some kind of prediction of what's likely to happen next,
certainly way better than anybody in the physics department could do.
Well that's a notion of success which is I think novel,
I don't know of anything like it in the history of science. [from the linked transcript]
>Machine learning gives performance without theoretical undertstanding.
that is good experimental data and the role of theorists here is to look how that performance achieved and why. For example, one can reasonably suspect that there is a good reason why the kernels in a well trained image recognition deep learning net do look like receptive fields of neurons in visual cortex. I'm pretty sure that there is some kind of statistical optimality in that, something similar to like normal distribution is maximum entropy distribution for a given standard variation. The same way i'd guess Gabor of neuron receptive field is something like maximum entropy on the set of all possible edges or something like this. The point here is that the great success of deep learning generates a lot of very good data for theorists to consume. You can do only so much theory without good experimental data, and in the decades before the availability of computing power (and resulting success of deep learning) there wasn't that much of the computer vision theory advances to speak about, really.
Here's a problem: you can argue that Gabor filters arise because we design neural nets to encourage them. Gabor filters mostly arise in CNN or things otherwise regularized to be like CNNs. Convolutional layers are a form of regularization that restrict the space of models that a network can conform to. The Gabor filters are learned but none of this is evidence they are globally "optimal" given that a human manually decided whether or not to include the presence of convolutions.
It also goes without saying that the phrase "statistically optimal" is meaningless in this specific context. You can claim they are a part of minimizing the cost function, but, again, you have to be very careful about the chicken and egg problem, because humans are the ones who manually craft the cost function.
You might find this 1996 Nature paper by Olshausen & Field interesting. In it, they describe how a coding strategy that maximizes spareness when representing natural scenes is enough to produce a family of localized, oriented, bandpass receptive fields, like those found in the early visual system of humans.
This fits in with my point: they imposed several restrictions about the model space: maximizing sparseness, and they also make several linearity assumptions.
those papers should be, if not 101, at least 201 for neural net track as it would help to establish common framework and basis for thinking, talking and analysis of neural nets machinery.
>you can argue that Gabor filters arise because we design neural nets to encourage them. Gabor filters mostly arise in CNN or things otherwise regularized to be like CNNs.
and do we know why? Usually it would mean some optimality. It should be relatively simple math here (back at the time at our University it would be given to a student as a thesis project and couple months later we'd have it), and that would give us 2 things - insight into biological visual cortex (which we suppose follows some optimality too and know we would have a very good candidate for the one) as well as to allow to start some primary convolutional layers with the (optimal set of) Gabors instead of going through learning them. Actually some of the best results i saw 15-20 years ago were produced by the simulation of visual cortex through such construction. And now image trained deep learning nets converge to the same.
No, we don't know why. You also have little basis to claim the biological visual cortex is optimal. It certainly works, which is enough for people who draw inspiration from it.
that is my point - we have very good suspicion and experimental data (successful deep learning CNN as well as visual cortex) that it is optimal, at least in some very wide class if not global, and that warrants investigation for a proof of it. Proven optimality would be very telling, especially for biological visual cortex. Even if optimality doesn't happen for CNN Gabor, there may happen to be discovered reasons for why not, and thus that would help to construct even better, probably optimal, approach.
Suspicion and empirical evidence cannot prove something is optimal. You cannot exhaustively empirically search an infinite space of models. You are seriously misunderstanding the definition of "optimal."
I am not seeing the chicken and egg problem. Isn't it always the case that when we consider the optimality of something it depends on some definitions?
The chicken and egg problem is that we design neural nets in order for Gabor filters to show up. If you used a different neural net architecture choice, they wouldn't show up. So the presence of Gabor filters indicating optimality is sort of begging the question.
In practical terms, performance without understanding can lead to highly surprising/counter-intuitive results when algorithms are applied to real-life problems. This doesn't matter much if you're doing movie suggestions or something like that, but it does matter in many other areas that could benefit from AI/ML.
But this is what the human brain is doing. A large complex statistical analysis of video that provided accurate predictions would contain an understanding of physics, just as our brain does. What you are essentially doing with this is creating a new researcher out of thin air, asking him to work out how to understand video and then when he/she/it succeeds not bothering to ask him how it was done. Then tossing the knowledge in his head away because you didn't come up with it.
I would hardly say that deep learning has taken over - some of the best results in the last few years have come from 'classical' domains like nonlinear optimization.
Deep learning / ML approaches certainly have a place, and they're getting a lot of attention right now, but the computer vision domain is about a lot more than segmentation and classification.
Maybe in coming years we'll see some more breakthroughs from the ML side on encoding priors - for example, teaching a network about projective geometry is a lot worse than just structuring it in a way that it 'knows' what projective geometry is. This could result in a closer collaboration between the two fields.
I would hardly say that deep learning has taken over - some of the best results in the last few years have come from 'classical' domains like nonlinear optimization.
Well, the article is arguing deep learning has taken most of the "mindshare", the attention of most researchers. If great results are coming from other parts of the field, that would be a reason to be concerned.
I was a Computer Vision Phd student until I dropped out in 2007. I wasn't very good, but a lot of what he says now was also true back then too. A field lacking any agreed on underlying theory, driven by fads, un-reproducable papers, massive volumes of forgettable papers by an inexplicably growing population of CV researchers with dim professional prospects.
Yet there are still people out there working against the tide trying to find a 'unified' theory of what's going on, granted with limited success or support. Some argued at the time that the problem is simply too difficult to tackle with our statistical 'tricks' and computing power. It's somewhat disheartening that folks are still grumbling about the same things 10 years after I left.
I've seen the exact trend in the field of optics where engineering work has replaced science.
The fundamental problem is that funding is given to those who promise the best outcome ("device that can recognize cancer") rather than the truth ("Where is the data located in an HBM").
Now, engineering work isn't bad, but today's university still has relics from a previous generation, like research papers. Hence, we're left with a bunch of research papers with little scientific content. The only fix I can think of is to offer useful alternatives to the PhD and prefer or mandate other markers of achievement like patents instead of research papers.
Is this a result of "hacker culture"? I.e. there seems to be a a pathological trend on HN of "fuck a CS degree, I can learn to code in a week" type of mentality. You have people who believe they can "hack" their way through complex fields without having a theoretical underpinning in it. No need to learn mathematics, you just need to "understand the main idea", code something up, run a bunch of simulations and tweak constants, run more simulations, etc. until you have some incremental improvement, write it up and publish.
It's the difference between giving a "brute force" computer proof like the four color theorem than try and come up with new theory where it's just a result from it.
No. I think its really the mindset of higher ranking people (professors) who due to funding or conflict-of-interest are motivated to do engineering rather then science.
Most folks I know are desperate to do actual science, experimental or theoretical. Instead, they optimizing some procedure/protocol.
It is an important point in principle that should be solved.
However, in practice that isn't an issue:
1) At least in this domain, all publications are de facto open access, as in, if you just google the name of a paper in a random citation in 99% cases you will get a non-paywalled full text version - if not from the actual place of publication, then on arxiv, author's home page, etc. It's not totally appropriate as there could be differences, but it's definitely enough to say "there is no one stopping them from doing science".
2) If you do actually need access to the university library databases for paywalled articles, then just go to the library. If you want to do science, there are options. Most people simply have or get some kind of university/college affiliation. If you don't, in many places you can still use the university library to access the data without the paywalls. If not, then you often can (depending on your country) "join" university to audit a single course, which would get you that affiliation and access to their infrastructure. I'm getting to more and more obscure scenarios, but even then there are options - the publications are accessible (though at some times not conveniently enough) and that is not a serious obstacle to doing science; it's still far less effort than actually reading and understanding these papers.
3) If you do need something that's really not available to you, just email the author. As a rule, people write articles because they want people to read them, use them and cite them. My advisor has a bunch of papers that he received that way in pre-internet time when that involved expensive mailing over the ocean. The only realistic case where an author wouldn't send a preprint version to you is because you're either rude or haven't taken the five seconds to click on the link in their homepage to get that paper.
I consider deep networks as experiments. People try incrementally different models , and the results keep getting better. At some point the theory behind them will advance to the point where we can analytically describe them.
How from a community where all fresh incoming PhD students have never and most likely will never hear about statistical learning, pattern recognition, euclidean geometry, continuous and discrete optimization, etc. new ideas will emerge.
Except they are learning this stuff prior to their graduate work...so I don't know where the author is coming from here. All of our Computer Vision people are very familiar with all of those topics - especially complex geometries and topology.
Really? My impression is that most new grad students have a superficial understanding of this stuff, which they promptly never use, since they just end-to-end deep learning solutions.
I think it depends on the student and researcher. I would agree that most are implementing ANNs to "brute force" around existing fundamental problems - but to me that's as much a solution as doing it another way - I'm not sure it's worth the effort to do it otherwise. Maybe some efficiencies, but I think getting towards are more generalizable ANN based machine vision system outweighs the negatives.
Nah it's just the latest hype to deep learn everything with lots of variations. It'll hit it a local maxima soon of what's achievable and then will begin the task of integrating actual computer vision knowledge.
Isn't DNN simply the way how we currently implement pattern recognition by statistical learning? I mean, you can do it a bunch of other ways too, but at the moment it seems that e.g. pattern recognition by a convolutional net trained on lots of unlabeled images is a better way that many classic algorithmic approaches to detecting specific patterns/features.
Deep learning is getting all the attention because it gets the best results- If you don't like this, you either have to provide (1) a different measure for results, or (2) give an objective mechanism for evaluating the "worth" of a technique that doesn't involve looking at results.
I would still like to hear what the author of the post recommends as a course of action- Maybe he can write a followup post that provides these details to clarify this.
It's currently getting the best results, which isn't exactly the same.
Deep learning undeniably works well. It works so well that it has almost completely taken over the field, sucking the oxygen out of other parts of the field. This is a little disconcerting since no one really understands how much of deep learning's success is due to a) the algorithms themselves vs. b) the massive amount of data and computational power being thrown at vision problems.
There is something to be said for some "counter-cyclical" efforts to encourage people to keep exploring approaches that aren't just slightly different network topologies, or least to ensure that they're not totally forgotten. That's the message I took away from the article.
For a time neural networks seemed dead and SVMs were sucking the oxygen out of other parts of the field. Then someone figured out how to train deep networks and the tides turned.
Sure, though the SVM tide never submerged competing methods quite as deeply as the incoming deep network tide, either in terms of performance or mind-share. Ensemble methods also became popular around the same time, for example.
In a way, you could take this as a perfect example of the authors' point: we might have had a lot of these advances sooner if we hadn't let interest in neural networks fall completely off a cliff.
We have unlimited money. But only when banks or the defense sector needed it. That's not a rant, that's an observation.
Think about it: Do we really, in the US of A or anywhere in the 1st world countries, have a lack of resources? In the 21st century? With hardly anyone actually working on the basics the population needs, like food, because our productivity is through the roof?
And "money" only is an issue when humans decide it is - it is an entirely made-up concept represented only in bits in computers (never mind that little bit of printed money, which can be produced at will too).
We do pay a huge amount of people in both private and public sector to do useless work. See dilbert.com cartoons and their popularity, and the results of research into the mood of working people, a huge amount of them thinking their work is useless to society or even just their own company.
The feeling (and reality) that we all have to work hard to get by is mostly an artificially created system outcome. With the productivity levels we have achieved, and the tools and knowledge at our disposal, we should have plenty of room for experiments.
> Think about it: Do we really, in the US of A or anywhere in the 1st world countries, have a lack of resources? In the 21st century?
Yes, I just say national debt. As long as this is not paid back (or we are working seriously to decrease it) we can't claim that we have an abundance of money - quite the opposite. Even more: As long as we aren't seriously working to decrease the national debt we are near to delayed filing of insolvency, which is not a good idea as any founder can tell you.
National debt? Really? You know that the modern money system IS debt? No debt - no money.
And no, debt does NOT have to be paid back. It can be rolled over indefinitely, or, if demands on "payments" really are an issue just go bankrupt. NOT A JOKE. It happened A LOT. I myself - and I'm not that old - have seen three currencies in my country in my life (Germany), my grandparents more. Germany is a horrible 3rd world place... no?
What are people going to do when all they get is "sorry, no more payments on the old debt. Are they going to emigrate to Alpha Centauri? Are they going to refuse to live from now on? No, they will grumble and get back to doing what they've been doing all the time. As long as the government is strong etc. Plenty of examples - see Germany. No that doesn't lead to Argentina - only Argentina leads to Argentina. A countries industry, culture, people, government don't suddenly stop working just because the currency is changed and old debts forgiven. Money does not exist, it's a made-up concept.
> we can't claim that we have an abundance of money - quite the opposite.
Uhmm... what should I tell someone like you? Except to repeat:
Money does not exist, it's a made-up concept.
Guess what happened after the financial crisis - we invented a trillion dollars. Out of nowhere. Suddenly it was there.
I don't think this an adequate analogy. SVMs never completely captured the field. There were also random forest, boosting and other promising algorithms that delivered results "in parallel" (within the same time-frame).
Also, the attitude that ANNs are "all we need" was there even before deep learning delivered all the recent SOTA results. I remember Patrick Winston commenting on that in one of his lectures in 2010(!).
By "how to train deep networks" are you referring to unsupervised pre-training, or the discovery/development of faster parallel training? (GPUs and distributed ASGD)
It gets the best results for what? Maybe for classification and recognition.
But the point of the article is that there is more to computer vision. Stereo, optical flow, geometry, and physics can only be aided by deep learning so much.
Another point not mentioned is the computational power required for deep learning. Consider programming the physics for a ball rolling down an incline. You could use (1) the math itself, vs. (2) a neural net. It's clear that the direct math approach could be orders of magnitude faster than neural nets. I wouldn't be surprised if directly coding the physics would achieve 1,000x the performance of a neural net.
I would posit that a deep learning network that learns to optimize parameters for a complex algorithm outside the convolutional network itself may have immense utility outside the classification problem. Call it a marriage of classic computer vision with deep CNN, or a hybrid approach. I don't think it's a binary decision. A deep CNN can find the optimal parameters (once trained) for a classic CV problem for a given image or video or other dataset, like superresolution, patch-based inpainting, or motion tracking. The training is the most computationally intensive part. As someone with way too many kids, I can testify...
Edit: Although what they seem to describe is replacing GPs with neural networks in Bayesian optimization which is supposedly more efficient.
Since the point of Bayesian optimization is to limit the number of times you have to evaluate a new set of hyperparameters, I am not sure how useful it is to "be able to scale" (i.e. even if maintaining the GP is O(n^3) with the number of evaluations, the costly part should be to evaluate the hyperparameters in the first place) but I haven't read the paper so they may show some high dimensional hyperparameter cases where performing a lot of evaluations pays off.
It is easy to introduce different measure which will make deep learning fail and which will improve the state of the art and humanness significantly. They are: 1) Learning from small amount of examples, make training database very small with 3-4 pictures by class. 2) Learning across classes. Train on evening pictures and learn to recognize daylight. Train on direct sunlight and learn to recognize on side light. 3) Learning of entities. Train on entity in white and learn to recognize a combined picture. Deep learning community intentionally do not consider these tasks, they prefer to increase the training data size to the limit only huge corporation could afford doing a research.
Deep learning is currently the best way to solve tasks such as your #1 example - there are great results with learning to identify classes from even a single labeled example; and the proper (only?) way to do this is to use basic feature-extraction layers made by showing the system a bunch of unlabelled images.
> (1) a different measure for results, or (2) give an objective mechanism for evaluating the "worth" of a technique that doesn't involve looking at results.
One does science also to understand why something works (i.e. what is the theory behind it). Deep learning gives something that (perhaps) works, but doesn't give an any clue to the question why it works - on what theory does the neural net internally compute in its black box.
Based on this criterion one can even argue that deep learning papers should not be considered as science as long as the authors of the deep learning paper make no really hard attempt to present at least a partly understanding of the internal theory that the trained neuronal network executes (i.e. a very detailed interpretation of the weights with ideally falsifiable predictions). This is hard work, I know, but isn't that nearly anything in science?
Completely agree with this viewpoint. Explanatory power and reproducibility are two of the most important aspects of a scientific paper. Most deep learning based papers lack both these properties.
> Explanatory power and reproducibility are two of the most important aspects of a scientific paper.
The reproducability problem should be solvable if the authors would simply release the training data and source code that trains the network (as yould scientific practise would require). Or is there anything else that is necessary to assure reproducability (I'm not that deep into CV papers)?
Benchmarks are indeed valuable, but I think one of the points in the article was that if the field focuses too much on them, research with theoretical depth gets excluded. However, those theoretical papers have value that is hard to quantify--it may take years to see it!
CV seemed stalled out before deep learning, at least in the late 90s. By that I mean classifiers had hit performance limits and weren't improving, and so on. As a non researcher, I'm glad to see it moving again.
I had very similar thoughts recently. Really glad someone took time to express them properly.
Even on this very website... I feel there is an immense bias in favor of anything "deep" and "neural" when it comes to AI. Can't recall any recent AI papers that made it to the front page without having those two words in the title.
And please, don't tell me there is nothing interesting going on in the field outside of deep learning. Even when an approach doesn't beat SOTA in terms of error rates, it can still contain valuable ideas or have interesting properties.
I worked in computer vision QA at a small startup named Neven Vision. (Google Goggles, Picassa fame) It's interesting to see how much things have progressed. I think I may have developed some of the first field tests in mobile computer vison. IR was very sensitive to lighting at the time. Though extreme angles always seemed to work. By the way I was testing on the best camera phone in town, the t-mobile sidekick.
It still is. Any system using structured light will have big problems when confronted with sunlight (windows, outdoors etc...).
I'm not confident there is a solution within the structured light domain to this as the beacons will (more than likely) never overcome sun intensity. We're doubling down on passive systems and reference maps.
One point that I've not seen mentioned yet, is that the neural revolution somewhat aggravates economic inequality. What recent progress basically has shown is that deep learning works better with more layers and more resources. Geoffrey Hinton has also recently conjectured that there exist a pretty much unexplored regime of applying gradient descent to huge models with strong regularization trained on relatively small (but still big) data. This inequality is alleviated to some extent by the fact that the machine learning community fully embraces online education and open science, but still, you need 50-150 GPUs to play human-level Go and having several grad students that explore a wide variety of complex and huge models is key for progress. I can only see this aspect getting worse in the years to come.
Research paper needs more creativity than improvement.
But it's quite difficult to come up with an innovative method, especially when the field is more explored and developed compared to decades before, and nowadays people need to publish a paper every few months.
Remember that throughout the eighties/nineties ANN classifier performance was as underwhelming as other approaches or even worse at many tasks. Now it reached a local optimum and all other approaches are being discarded.
When you search only in the immediately most promising direction you sure find local maxima - and you may miss the global one. Unless you can make guarantees about the search space which surely you can't make about life, the universe and everything.
If using loads of data and cpu is a bad thing, he should propose a new measure of goodness that takes these into account. A bit like information criteria in statistics which penalise using a bunch of extra variables.
> If using loads of data and cpu is a bad thing, he should propose a new measure of goodness that takes these into account.
You better separate between using lots of data (perfectly OK, in my opinion) and the number of parameters that the model uses (say, number of weights in the neural network) and where you better have good explanation for the existence of any variable and why it has this concrete weight and no other and why it is even necessary to introduce (consider any parameter that you have to introduce as some kind of physical constant - physicists invest lots of time to explain/reduce the number, so should CV researchers).
He should come up with a better benchmark then that requires the methods he fears are underexplored. Perhaps demonstrate something important that's being overlooked by the trendy crowd. If he can't come up with that, he's just being sentimental.
I last dabbled in image processing research around 2011. Probably most of the papers i read during the previous 5 years were small little epsilon papers that added no real value. I did some work in other fields and noticed a similar trend there. I always attributed it to the trend of PhDs being pumped through the system in ever greater numbers and the need for researchers to publish a paper every few months.