While it’s good information retrieval work, I’m super skeptical that it actually adds meaningful value for customers.
Having worked on several different search engines at several companies, my consensus on this stuff is that customers usually just want more simple boolean filters or multi-choice filters that they can use to precisely control thevresult set for their preferences, and they want you to clean up the data backing the filters so they are highly accurate.
Spending huge money on a fancy system to try to understand their intent is overwrought. Did you ask them? Did you try just empowering them to communicate their own intent with more filtering or ranking options they can tune or manipulate on their own?
This stuff annoys me sometimes because it’s dripping with so much marketing hype. Look at a sexy new embedding-based recommender, or a reinforcement learning agent that learns how to rank to gamify placed orders.
Maybe you could have better served your users by not spending big salaries + compute costs on this crazy query understanding model and instead just cleaned up your data so that you stop getting the opening hours of restaurants wrong, or you stop giving back BBQ places when somebody checks “vegetarian.”
In so many cases, just some simple filters + data cleaning is worth way more to the customer than all this.
> Having worked on several different search engines at several companies, my consensus on this stuff is that customers usually just want more simple boolean filters or multi-choice filters that they can use to precisely control thevresult set for their preferences, and they want you to clean up the data backing the filters so they are highly accurate.
It varied depending on the domain. In one company it was a high traffic online merchant, and customers were basically a random sample of the middle-class population at large, no better-than-average-person computer literacy.
In another case it was a multimedia company and the search users were producers, artists and technical directors, with much more of a power user mentality to searching for what they wanted, and jobs that required basic computer literacy and maybe familiarity with special GUI programs.
But I’m telling you, in every focus group or A/B test or user session we ever looked at, at any of these companies regardless of the user characteristics, the absolute number one thing every time was user requests for more and more categorical filters to allow them to efficiently exclude huge classes of results they know are not relevant to them.
Amazon (I did not work there) actually gets this right, with lots of brand, size, color, Prime-eligible, etc. filters for everything. That stuff requires no machine learning, is simple to implement and A/B test and measure usage efficacy for, and people really want it.
Search and recommendation is so interesting to me. The goal of it is fuzzy. If I search "pho" then after the highly reviewed pho places, how might you decide between really poorly reviewed pho places and really highly reviewed ramen places?
If you just return the pho, it's doing what I asked for, but not showing me things I might be interested in. If the other pho places are really bad, maybe you aren't showing me things I'd actually rather eat.
If you return the ramen too, then you're kinda not doing what I asked, which is frustrating, but you might be showing me things I'd rather eat, given the shitty alternatives. When there's no mechanism for me to say "but really, I just want pho" it can be even more frustrating.
In this case there's a clear line between bad and matching vs good and related, but in many real queries there's not such a clear line. The smarter the algorithms get, paradoxically, the more frustrated I get by real world examples of this dilemma. I guess I don't have much to add, but it fascinates the hell out of me.
I think the solution is providing results with more structure than just a list of options. Being able to return something like "here's the list of pho places, but they aren't very good. We also have some ramen places here that are pretty good if you are interested." I think just having some sort of explanation why I've gotten a bunch of ramen places in my results would go a long way. At least giving some sort of explanation of the results could help a lot with that frustration. You could go even further and add some level of interactivity, allow the user to respond with "sure, show me the ramen results," or "no, show me the pho results anyway" would be the ideal.
The difficulty is that a lot of these sorts of queries, especially when machine learning is involved, are pretty opaque. You can find some set of results that meet the maximization of some metric in your feature space, but it's hard to explain why a particular result shows up there without resorting to a bunch of equations.
One thing I've been annoyed at Google maps for is the inability to search delis, restaurants, takeout places, etc. all in one query. I want literally everything within 2 or 3 blocks of me. I usually search "food", "restaurant", and maybe "takeout" and different stuff pops up for each.
It's because their place data isn't so rich. Also they haven't considered it a priority to build out that kind of a search. (Despite it being a lucrative space)
20 years ago, the best we could do was boolean filters. Then Google came along and we've been charging forth down the fuzzy road ever since. At the first WSDM conference, intent was all anyone was interested in figuring out.
Now the population at large is far more computer literate than ever. I wonder if we still really need fuzzy search for everything in the same way we did 20 years ago. It seems like many geo-related problems would be better served by better boolean search.
Agreed, the loss of "programmer-friendly" features like wildcard and boolean matching is one downside of the progress made in the last couple of decades.
This isn't such a massive problem with Google as it is on specialized search engines like eBay's. A fuzzy search is worse than useless when you're looking for items with very specific names.
Reminds me a lot of the show Silicon Valley, where one of the characters has something like a Phd in artificial intelligence but ends up writing code for an app which dynamically turns your face into a smiley face (i.e. snapchat)..
If I remember correctly, the product in that episode was applying "snapchat filters" (deformation/modification/accessories) to animal faces. It was portrayed like this stupid job, but as an ML researcher myself my first thought was: "that's a pretty cool/challenging problem!"
It's funny how things that can seem hard can be so simple, and vice versa! I guess it depends on the person whether they're looking for an interesting challenge, or to make an impact - - because the two are often perpendicular.
I'm sure it is an interesting problem, but what happens when the majority of our technical talent goes towards solving useless but "interesting" problems?
Look at the engineers credited at the bottom of the article: "graduate of Carnegie Mellon University with a degree in Computer Science", "Ph.D. in Electrical Engineering and Computer Science from UC Berkeley", "Ph.D. in statistical machine learning from Duke University" these are some extremely intelligent and educated people but they are using their skills to help you choose between Chinese food and Indian?
We've already seen this situation described as the "Internet of Stuff Your Mom Won’t Do for You Anymore” and stuff like this doesn't really help. I really think that innovation is dead in silicon valley, its more about catering to needs of nerdy man-boys with lots of money to blow.
> I really think that innovation is dead in silicon valley, its more about catering to needs of nerdy man-boys with lots of money to blow.
I disagree. I think it's about catering to the needs of normals with lots of money to blow. The only nerdy aspect here is this post. The product itself? Totally for smartphone-using normies.
Its 2018, normal people do not have money to blow. The "normals" are the ones delivering your food, driving you around, doing your laundry, walking your dog, cleaning your house etc.
The "normals" are also people working in management, marketing, and other high-paying professions. SV might have extremely high density of tech people, but programmers are not the only (or even the majority of) high-earners in the West.
I wonder if/how they treat the 0 results cases, there isnt much discussion of this. I know some of the delivery companies are setting up commercial kitchens for outposts of popular restaurants or even entirely virtual restaurants. UberEats has a great visibility into this, ie if a lot of people are looking for pho and not finding results, encouraging a popular pho restaurant to setup in that delivery area makes sense.
In Singapore we have Uber Eats, Food Panda, and Deliveroo.
They all suck. Not a single one allows you to search menus.
Developers need to stop trying to make complicated search features and go back to a good old full text search that works.
While the argument may be valid for Udon and Soba... Making correlations between Japanese and Chinese food is not. Living in Singapore, if I want to order Chinese food, I do not want Japanese food, they are not similar at all.
> In Singapore we have Uber Eats, Food Panda, and Deliveroo.
Less of a problem in America, where all restaurants of a given type have very similar menus. For example, most Thai restaurants are going to have the same dishes, with (maybe!) a few unique ones at each location.
Chinese is one that has started to differ, more and more unique Chinese places are opening, but there are still few enough of them that anyone interested in those specialty dishes probably knows which restaurant to order from.
(I'm in Seattle for what it is worth. New York City might be a very different story!)
I don't want to detract from the message of the article, but:
> For example, an eater might search for udon, but end up ordering soba. In this case, the eater may have been looking for something similar to udon, such as soba and ramen, instead of only being interested in udon. As humans, it might seem obvious; Udon and soba are somewhat similar, Chinese and Japanese are both Asian cuisines.
The author's point is, Udon is similar with Soba, and Chinese cuisines is similar with Japanese food, compared with Burger and Noodle. But not saying, Udon is Chinese cuisine, and Soba is Japanese cuisine.
The author seems to be drawing a parallel between udon and soba, but instead of saying they're both Japanese, says that China and Japan are both in Asia. It's somewhat of a non-sequitur that only makes sense if (1) the author thought that udon is Chinese and soba is Japanese, or (2) the author didn't realize that the second phrase would not be read as a separate example but still in the context of "In this case ..." which set up the udon example.
Edit: Looking at the authors' names -- Ferras Hamad, Isaac Liu and Xian Xing Zhang -- I'm guessing the 2nd is what happened. Somehow I get the sense that Xian Xing Zhang and Isaac Liu know where udon comes from.
Hierarchies or simple category memberships are very simplistic: this is 1970s level natural language semantic processing, but I'm not sure it's appropriate. For example, Yoshihiro Murata is the 3 Michelin Star chef of Kyoto restaurant Kikunoi. In a June 2015 interview[0] in Mad Feed magazine he stated:
I think 50 years from now, the borders will be gone. We won't be talking about Nordic cuisine or Japanese cuisine anymore. It will be about what is delicious, without any sense of nationality. That will be the new cooking tradition.
The point being, perhaps rigid categories are the wrong approach and tastes are blending across traditional heirarchies. Last week at F&A Next[1] in Holland I met people are working on scientific flavor combination based recommendation engines already. Their market was chefs, but why not sell this feature directly to consumers?
There's another issue which is Halal in practice can mean anything from "doesn't include pork products" to "must be a certified Halal restaurant with certified Halal ingredients". They fail to differentiate this in the text. Basically it boils down to obtaining detailed personal preference and restriction information, which is good practice anyway in food service, in case of deadly allergies.
On the other hand, if you trust Michelin chefs, delivery is perhaps the wrong approach going forward. In March another three Michelin Star chef, Alvin Leung of HK's Bo Innovation was interviewed[1] by Salt Magazine and stated:
Maybe robots making fried rice is the future.
That's basically what we're working on at http://infinite-food.com/ .. we figure 3 minutes prep times mean 10-20x speed improvement vs. delivery plus full personalization ... raising Series A now to launch in three markets next year!
Having worked on several different search engines at several companies, my consensus on this stuff is that customers usually just want more simple boolean filters or multi-choice filters that they can use to precisely control thevresult set for their preferences, and they want you to clean up the data backing the filters so they are highly accurate.
Spending huge money on a fancy system to try to understand their intent is overwrought. Did you ask them? Did you try just empowering them to communicate their own intent with more filtering or ranking options they can tune or manipulate on their own?
This stuff annoys me sometimes because it’s dripping with so much marketing hype. Look at a sexy new embedding-based recommender, or a reinforcement learning agent that learns how to rank to gamify placed orders.
Maybe you could have better served your users by not spending big salaries + compute costs on this crazy query understanding model and instead just cleaned up your data so that you stop getting the opening hours of restaurants wrong, or you stop giving back BBQ places when somebody checks “vegetarian.”
In so many cases, just some simple filters + data cleaning is worth way more to the customer than all this.