(I'm a little worried about posting that on HN since it's unoptimized and will probably crash.)
I'm currently working on skin detection & exclusion during the color detection phase and am looking at using basic machine learning techniques. The key challenge I'm facing is differences in skin tones.
> I'm currently working on skin detection & exclusion during the color detection phase and am looking at using basic machine learning techniques. The key challenge I'm facing is differences in skin tones.
Try looking at the chromatic colour rather than the RGB values. You can get extremely far with just this, most skin colours fall into one of two peaks [0], no machine learning needed.
Once you've got this, edge detection & a few other bits should give you pretty reliable skin blocks. I've used it a few times before. Here's a presentation I did some years ago that I apparently still have on my desktop: http://files.figshare.com/1409002/1.pdf [1]
EDIT - I'm sure there are many good approaches for this, and many fancy ones. This is very simple and was researched/written purely for fun in a couple of weeks.
EDIT 2 - The final slide shows the more interesting part, where you use edge detectors to guide your estimation of what is inside or outside a shape. That plus an adaptive threshold (designed to stop if the number of pixels included jumped rapidly) got some good results, but I've not got the code any more.
awesome stuff :D Thanks for this. Will definitely look into some of that in more detail.
Another tricky part of skin detection is false positives. ie, what if the actual product is that color?
Some things I've noticed and will be taking into account are:
Skin areas tend to clump around the same locations in photos. The product is usually the focus and skin is near the edges.
Product types also tend to share similar photo layouts.
So with that, skin color in those zones score higher.
> awesome stuff :D Thanks for this. Will definitely look into some of that in more detail.
No worries, hope it helps, it was just a quick project back in the day at uni that ended up working a lot better than I expected.
Give me a shout if you want any work done on it (my email address is in my profile).
> Some things I've noticed and will be taking into account are: Skin areas tend to clump around the same locations in photos. The product is usually the focus and skin is near the edges. Product types also tend to share similar photo layouts. So with that, skin color in those zones score higher.
This kind of thing will really help you, small bits of knowledge about the specifics drastically simplify the problem. For example, you can estimate the skin tone by roughly segmenting the image into possible skin/not skin with the approach above, then look at segments which are more likely to be skin because of their positioning you can narrow your accepted parameters and hopefully help distinguish between the two.
Identification of unusual edges/shapes can help too, to classify regions as skin/not skin.
Beyond that, starting to look at estimations of pose to help guess the underlying shape (since you know it's on humans you can make a lot of assumptions).
Also, since you're detecting colours, mistaking very similarly coloured skin as the product wouldn't change your results much :)
Wow, looks awesome so far. At the moment we aren't dealing with skin tones. Mostly it isn't a problem but there are some cases (such as jewellery or swimwear) where we have to use humans
Color is actually much more difficult than most people realize! Even CIE2000 color distance still has fundamental "perceptual distance" problems, especially comparing non-saturated with saturated colors (although that's a blog post in its own).
Here's our engine linking "real world" photos into the Macys catalogue (based on color, shape, texture): http://www.pcsso.com/demo/macys.htm
It is impossible to create a three dimensional color space in which Euclidean distance corresponds directly to perceived color difference. There are dozens of papers on this topic, for anyone who wants some fun evening reading.
Yes, absolutely, which (I'm assuming) is why CIE2K difference is not-quite-euclidean. My opinion is the we still need to find a better difference metric, rather than improving the color space.
On that front, it's amazing to see how much of an effect "switching color spaces" can have on many algorithms.
Yeah, totally agree on that. We noticed this as well when approaching it naively via simple clustering then going through different color spaces and methods until finally getting some decent results.
Nice post. I'm a researcher in computer vision, so I'm glad to see that people find these problems interesting. It's often easy to underestimate how difficult vision problems are, since they seem so natural to us.
I think the recent posts by lyst (this color naming post, and the previous background subtraction post) are great introductions to problems in computer vision.
However, there is a lot more sophisticated work out there, and the techniques used in these posts are decades old (and contain some errors).
If you have a computer vision problem in mind, you can gauge the state-of-the-art, by searching recent papers on Google Scholar. Even if you're unfamiliar with the jargon, the introduction and conclusion of a paper can give you an intuition about the problem being solved and the steps the authors are proposing.
If you want to learn more about computer vision and its details, I highly recommend checking out online materials such as
Thanks for all the great resources. I think it must be obvious that we aren't image processing researchers! My background is combinatorial graph theory. But these HN comments have given us so much valuable material for improvements.
"To speed up the color difference calculations we decided to vectorised deltaE. The standard deltaE functions are already implemented in the colormath python package. We ported these to numpy and added a function to take a numpy array of Lab coordinates. When used with large data-sets the vectorised implementation is 25-180 times faster depending on which distance function is used. The vectorised delta E functions are now available in the colormath package and can be used in the following manner."
Uhm.. wouldn't it have been easier to just run a voronoi algorithm on the data set in the labspace? then you have a lookup table/cube. For a paltry 6mb of space, lookup becomes instantaneous. It's what voronoi is for.
Even better would be to take the full data set of all the survey responses, and use it to generate the partition of CIELAB space, instead of just using the single set of coordinates per name that Randall generated from those. But it would be more work.
Edit: do note they’re using a more complicated distance definition such that a straight Voronoi diagram in CIELAB space is not quite the same as their result. But the difference is so slight as to not matter, so your suggested solution would be substantially better than theirs.
I’d recommend using the ISCC-NBS system of color designation in cases like this. You’ll have to convert your colors to Munsell color space, and then use a lookup table to get the associated name. But it’s a pretty reasonable system.
Ideally you could get some kind of actual color measurements from clothes directly, instead of relying on (often very inaccurate) digital images, but maybe the’s too much to hope for.
Made me think we should write something similar for our approach, which is a bit more complex though leveraging semi-supervised learning, dynamic estimations and normalized color calculations with statistical bias.
Here is some simple demo if you are interested, where we translated it back also into color names to allow text search for estimated parent and real color.
Really cool techniques (this is the background removal one). I'm curious on how well this would work on hair color detection. My thoughts are that you could run face detection, and since detection almost always (anecdotal) find just the face, not the outline of the face, you could just grab around the face for hair.
Does someone at Lyst have a background in image manipulation? If so, I can't wait to see what else you guys are working on.
Most of the guys here have more experience with text processing but we are expanding into image processing. I think we could definitely detect hair colour but it isn't that useful for Lyst. Gender identification on the other hand...(watch this space)
I'll be curious about the gender classifier, mostly because I'm working on a statistical aging model for different gender/ethnicities. Keep on posting!
I'm building something similar in my spare time.
It performs multi-color image searching on street wear. You can select a bunch of colors and adjust the ratios.
Here's a very early in-development version. http://www.inthatstyle.com/womens?colors=73a1d3,e84b34&ratio...
(I'm a little worried about posting that on HN since it's unoptimized and will probably crash.)
I'm currently working on skin detection & exclusion during the color detection phase and am looking at using basic machine learning techniques. The key challenge I'm facing is differences in skin tones.