Depends what you're doing. I have a content-based recommender that uses clustering for diversity, basically if you want N items and have k clusters it picks the top N/k out of each cluster. [1] k-means works just great for that.
[1] for better or worse though, if 1/k of your articles are about some topic like soccer or cryptocurrency you will always get about 1/k articles about soccer or cryptocurrency no matter how you vote.
If 3/k of your items are about soccer or 2/k are about cryptocurrency then you get 3 and 2 clusters for them respectively whereas on some level you might want 1 cluster for each of them and also want one cluster for a topic which with prevalence 1/3k. On the other hand, having 2 clusters for a topic with 2/k could be seen as fair on another level... one way or the other I appreciate the system not having too many parameters and not being too sensitive to those parameters and I'd expect it to do roughly the right thing if k is set a little too high.
Early one I had a fight with the system where it kept showing me articles about soccer despite hating soccer because I had roughly 1/k articles about soccer (an RSS feed from The Guardian!). I wound up thinking a lot about feature engineering for sports and started reading soccer articles in detail and pretty soon I was amazed by games that were 0-1 and an own goal or that went 8-0 and pretty soon I became one of those people who is watching soccer at 9am on Saturday.
Related to that is the question of “should it show me more articles from clusters that I like better?” which defeats the point of diversity but certainly would improve most quality metrics (though I’d get sick of arXiv papers about recommender systems, which was most of what I got when k=1 in the beginning, and would be screaming for articles about cricket or something instead.)
I haven’t done a lot of work to try to improve that system because I like the feed it makes already and I’m not sure what to optimize to make it better. If it has a weakness it is that it has a lot of latency but there are certain things like sports articles and Ken Shiriff’s blog which are more timely and I’d like a way for some articles to “jump the line” without seeing too many of those articles overall…. But I’m not sure how to quantify what success is.
