I made this to experiment with embeddings and explore how different ways of displaying information affect your perception.
It gets the top 100 stories, sends their html to GPT-4 to extract the main content (this was not producing good enough results with html parsing) and then gets an embedding using the title and content.
Likes/dislikes are stored in local storage and compared against all stories using cosine similarity to find the most relevant stories.
It costs about $10/day to run. I was thinking of offering additional value for a small subscription. Maybe more pages of the newspaper, full story content/comments, a weekly digest or ePub export or something?
I think some of the highest value from HN comes from the comments, and it's much harder to find the "best" ones, since they might be in threads you might not have otherwise read.
Not sure if it's a "premium feature" so to speak, but would be very cool to extend this to comments generally.
Definitely, comments are usually better than the article. I thought of a 'Letters to the Editors' section that shows top comments (https://news.ycombinator.com/bestcomments) and references the parent story, but it might not be as useful without the context.
Maybe 'See Comments' here could load the comments on the same page? In a newspaper like style.
Nice – I like this a lot. I feel like I'd use this for slow-lane reading and the original HN site when I'm in a rush.
Regarding HTML to GPT-4, I seem to remember commenters here saying they got better results by converting the HTML to Markdown first, then sending to an LLM. Might save a bit of money too.
> Likes/dislikes are stored in local storage and compared against all stories using cosine similarity to find the most relevant stories.
You're referring to using the embeddings for cosine similarity?
I am doing something similar with stocks. Taking several decades worth of 10-Q statements for a majority of stocks and weighted ETF holdings and using an autoencoder to generate embeddings that I run cosine and euclidean algorithms on via Rust WASM.
It gets the top 100 stories, sends their html to GPT-4 to extract the main content (this was not producing good enough results with html parsing) and then gets an embedding using the title and content.
Likes/dislikes are stored in local storage and compared against all stories using cosine similarity to find the most relevant stories.
It costs about $10/day to run. I was thinking of offering additional value for a small subscription. Maybe more pages of the newspaper, full story content/comments, a weekly digest or ePub export or something?