1. Responsiveness 2. Excellent voice recognition for dictation of questions and device control 3. Web reading/browsing capability 4. Instant webpage to PDF conversion 5. Instant storage of those PDFs for use as custom LLM training data 5. Functionality to copy quotes from web pages or PDFs along with auto-generation of citations 6. Functionality to export of PDFs, quotes, and citations together into PKM app, say, Obsidian or other similar PKMs

BONUS: I know Kindle isn't set up for this, but if the same could be done audio and visual media, using voice control, beginning with podcasts (auto transcript save with cite, clipping with transcripts and cite, export of clips with transcripts and cit) and then YouTube (auto transcript save with site, auto clip with transcript and cite, export with transcript and cite)

IF I could get 1-6, I would strongly consider having this as part of my PKM kit. If I could get 7 too, I would have to buy it. This may be something I have to build, unless someone knows where these integrations can be found today.

That sounds like an awesome app, if you build I'd use it too! :)

Cool! This brings to my mind that it would be cool to have a local LLM chat that is able to point to certain places in a book I'm reading offline and have it recommend portions of the book I should focus on.

Yeah it would be nice if the LLM knew what you're currently reading - tricky to do with a web app though!

Concur- I’d love to have the book I’m reading — and anything before/after it in the series — available to an LLM for quick reference.

Made this to not have to constantly look things up on my phone while reading.

It's pretty handy for generating longer essays to read on the Kindle too.

I have the same problem, namely a constant urge to look up things while reading paper books. My personal solution to that is to copy whatever sentence that triggers my urge to look up anything and then set aside time to do the look up later. I think the value it brings is that I won't be distracted into deep rabbit holes while finishing my book.

I'm not sure what's the end goal here though. It seems like you would like to look up things as soon as you encounter them, but on the same device (Kindle)? Then the benefit is not switching devices?

You can replace Wikipedia on your Kindle without modifying any firmware or software on the device. Here's how it works:

1. Host an API which conforms to the Wikipedia search and document conventions (this can be LLM-powered) 2. Modify your router's DNS settings specifically for your Kindle so that it resolves wikipedia.org to your API's CNAME or IP address 3. You can now use "Wikipedia" to perform arbitrary operations on highlighted text. As a bonus, you can manage your API from your Kindle's built-in browser (assuming you've built out a management UI)

So you are saying Kindle doesn't have end to end encryption.

That sounds like a good system.

Yeah I find my phone and laptop quite distracting, so I don't like keeping them around when reading, especially at night.

Another use case I didn't plan for originally but is pretty fun is doing all your Kindle reading in Kindllm by having it write essays on random topics for you.

What are you using to run the Mixtral inference?

Using Anyscale Endpoints, cheap and seems stable: https://www.anyscale.com/endpoints

Very cool. I've no idea if this is possible without rooting the Kindle, but I wonder would it be possible to highlight sections of text in a book and send it to something like this with a request to summarise/explain?

It's a good idea! Amazon doesn't offer an API for fetching highlights but services like Readwise seem to get around this by using a Chrome extension to do it. Not sure how that could be done on a Kindle though...

Thanks for the tip on ReadWise, that looks interesting!

Really cool idea. Unfortunately, the proportions are somewhat broken on the Oasis (the text bar stretches a bit less than half way across the screen). Was the page designed to be pixel perfect for the Paperwhite?

I used percentages so the text bar and Send button should fill up the full width on any device. I saw the same thing happen before updating the Kindle version. I wonder if they haven't rolled out the upgrade for Oasis yet.

The demo video is great, I would consider putting it higher up on the page and possibly on autoplay. I wonder when Amazon will integrate LLM into the native reader. Nice work!

Thanks! Yeah the landing page might need some work

re: Native LLM: The fact that Amazon just upgraded the web browser shows that they at least have some engineers working on Kindles, so one can always hope

and yet, there are some obvious things on my scribe that they haven't done:

- no way to disable auto-rotate

- can only write on PDFs from send-to-kindle, but that's limited to I think 50MB so textbooks etc. are not an option

- only works with audible audiobooks, no normal m4b supported

i believe the web browser is also still crazy cut down. i like the device but they're evidently understaffed on software

Fair point, yeah it's definitely still cut down

Off topic, but why is the demo video so much brighter than the surrounding background? The page has a white background, but the video is substantially brighter.

Most likely the video was recorded with an iPhone. Playing it on an Apple device then automatically turns on some sort of an HDR mode and unlocks higher brightness - commonly seen while scrolling through reddit videos and other social media.

Ah I see. Yeah definitely filmed on an iPhone (with a cardboard box and a cutting board as a camera stand)

Apple devices render HDR content at higher brightness. It's actually quite interesting - see [0] and [1] for an intentional use of this!

[0]: https://notes.dt.in.th/HDRQRCode

[1]: https://news.ycombinator.com/item?id=36384625

Some sort of HDR effect.. very bright on a 14" MBP screen

Aggressive! I'm not saying the site or product is bad. Hope you have a better rest of your day!

Is it night shift on mac doing it?

This is very awesome, thanks for sharing!

I love this, but I wish there was a better way to enter questions. Using the kindle keyboard is not a great experience on my device, and no matter how good the suggested questions get, it wouldn't be able to handle my initial question

Agreed, I definitely write shorter questions on my Kindle than in ChatGPT on my laptop

This is cool!

I can't stop worrying about the fact that you only have 27% battery left, thou :)

27% on a Kindle with a healthy battery is a lot. Especially if this is a latest model paperwhite on the image which I think it is. It's probably their most impressive feature as a product

Haha oops didn't notice that

Doesn’t work on a Kindle Scribe - maybe the user agent string is different?

What does it say about Kindle in the user-agent string? https://www.whatsmyua.info

Looks great, but Kindle is the last device I am looking into when seeking responsiveness if I have to be honest.

Nice! (FYI this can be tested without a Kindle by switching the user agent of the browser to a Kindle UA).

Congratulations! Very cool stuff.

I had the very same idea after the browser update! Well done :)

How much is it costing you to run the LLM? Is it OpenAI?

Mixtral on Anyscale Endpoints: https://docs.endpoints.anyscale.com/pricing

Thanks! I'm not experienced enough in this domain to have an idea of what that means for the bottom line. I assume that at the scale of a niche app, the cost is close to negligible and you can pay it out of pocket, but I imagine that if Amazon would pre-install something like this on every Kindle, the costs would go through the roof. Speaking in broad terms, how much scale do you think an app like this could handle, while still being something an individual might consider worthwhile to pay for out of pocket?

Very cool!