Does this mean we are getting closer to being able to do typical inference search stuff straight in the db? I CTO ecommerce stuff and I am just now starting to look into similarity based recommendations where we can show related products based on a bunch of variables that infer similarity?
My knowledge of this topic is very tangential since I have not had a need to dive in, but isnt a completed / trained model just represented as a bunch of vectors?
We run Elixir so I could run the models in bumblebee but ideally we can stick to postgres only until vertical scaling starts to fall apart (which is double digits terabytes away).
Typically you have (1) an ML model to generate embeddings from the items in your db and (2) vector support in the db to order/compare those rows by similarity (i.e. cosine distance between embeddings). So this just gives you the second part in a more convenient / efficient package. That's super cool, but only the second half.
For the encoding model, you could use any ML model you want, from cheap/less complex models to expensive/incredibly complex models like GPT-3. You could even use a face recognition model to encode face images and sort/compare them the same, etc
So this just makes it a lot easier to roll your own similarity systen with an encoding model of your choice plugged in. If you have a lot of data to encode and aren't afraid of running your own model, it is a great part of a solution. But it is not an all-in-one solution.
Probably better to use an off the shelf recommender or Amazon personalize. Amazon personalize will turn your product text data into embeddings and use that + collaborative filtering to generate various types of recommendations.
I’ve worked on e-commerce recs. Typically you would represent each product with a vector. Then finding similar products becomes a nearest-neighbour search over these vectors. Depending on your use-case it’s feasible now to do this search in the db using pgvector, or using something like solr/elastic which both support vector search in recent releases. You could also use something like faiss or one of the many nearest-neighbour libraries or dedicated vector search engines. (Since you are working with Elixir, you might find ExFaiss interesting [1][2][3]).
But I would say that for recommendations, searching the vectors is the easy part. The main work in getting good recommendations is generating a good set of product vectors in the first place. The quality of the recommendations is directly related how you generate the vectors. You could use one of the many open-source language models to generate vectors, but typically that approach isn’t very good for product recommendations. It will just give you items that are textually similar, and this usually doesn’t give good product recommendations.
To get good product recommendations you’d probably want to build a custom embedding that captures some notion of product similarity during training using some signals you get from user behaviour. E.g. things like products clicked in the same session, or added to cart at the same time, gives a signal on product similarity that you can use to train a product embedding for recommendations.
This is a bit more involved, but the main work is in generating the training data. Once you have that you can use open source tools such as fasttext [4] to learn the embedding and output the product vectors. (Or if you want to void training your own embedding, I’d guess that there are services that will take your interaction data and generate product vectors from them, but I’m not familiar with any).
My knowledge of this topic is very tangential since I have not had a need to dive in, but isnt a completed / trained model just represented as a bunch of vectors?
We run Elixir so I could run the models in bumblebee but ideally we can stick to postgres only until vertical scaling starts to fall apart (which is double digits terabytes away).