Hacker News new | past | comments | ask | show | jobs | submit login

I spent a great deal of time writing a recommendation engine for a major web property, so I'll just offer some brief pointers for anyone interested in this stuff:

- If you know nothing about how recommenders might work, a good introductory book is Programming Collective Intelligence.

- To do recommendations at scale, your best bet is to write them in java and run them on Hadoop using map/reduce. You can write them in python too I suppose and use Hadoop Streaming.

- NumPy is awesome. Its a great way to prototype your ideas and if you come from a Matlab background (as I did), its very similar. I have not yet run anything in production using NumPy though.

- If you want a recommender that works out of the box, check out Apache Mahout (used with Hadoop) or the Weka project.




+1. If anyone is interested in map-reduce solutions in python, be sure to check out Yelp's Mr. Job (http://www.readwriteweb.com/cloud/2010/10/yelps-mrjob-poweri...)


Is ruby appropriate for writing recommendation engines? Or is there far more support for writing them in python?


As far as I am aware Ruby has no real equivalent to Numpy and Scipy for Python. This is why I swapped to Python from Ruby; the Ruby community gets extremely thin indeed as soon as you stray from web-related things.


have you looked into using Cascalog/Clojure with hadoop ?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: