Hacker News new | past | comments | ask | show | jobs | submit login
How I recovered a lost email from my email client’s memory (ctrl.blog)
155 points by weinzierl on May 3, 2020 | hide | past | favorite | 82 comments



> Evolution [..] has a bad data loss bug. It sometimes deletes the email body text in the compose window after changing the signature [..] it has bitten me about twice a month for the last two years.

and

> I might have been able to partially recover the message from the Draft folder if I’d retained my cool and acted immediately. It had been overwritten by an empty message instead. I must look into versioning my email draft folder at a later time.

This person has a much greater tolerance for shitty software than I do. I'm certainly not a perfectionist and appreciate that almost all software has bugs, but come on! Arbitrarily deleting draft emails twice a month for two years? Requiring convoluted versioned draft folders to work around this glaring issue? Why are they punishing themselves like this?! They must find something really awesome about Evolution to deal with this level of annoying.


Maybe they have an exchange mailbox? If you want exchange support on linux with full syncing of mail, calendar and tasks there are two options I know of: evolution and hiri. And hiri is paid abandonware. I’ve set up both, both are not good. Lately I’ve been using outlook web access, which is still bad, just not as bad.

Really I guess I miss outlook for windows. There I’ve said it. Judge me if you will. Its search feature is broken, but everything else worked well.


Option 3: davmail[1] acts as a translator between Microsoft's proprietary protocols and open protocols.

1. http://davmail.sourceforge.net/


Author here. My problem is related. However, I have a IMAP/CardDAV open-standards setup instead of ActiveSync/Exchange.


You can use Thunderbird with a paid add on. It’s well worth it, because evolution is just bad.


Had the same thought. Did this person consider using different software? Why hurt yourself like that?

Its no wonder that it will never be the year of the Linux Desktop.


Person/author here. Yes, I have considered other email clients. Evolution is the least bad of the available options and still sees active development. Most other clients don’t support CardDAV or LDAP so I can’t sync my address book. An email client is mostly useless without the contact details of the people want to contact.

Newscommers to the email client market have all been proprietary subscription-based middleware instead of actual email clients. The market has been standing mostly still for the last 10 years.


I agree and also use evolution because it feels more comfortable and well thought out than the alternatives.

Thunderbird feels wrong for some reason, and webmail doesn't let me have ten accounts in one place... I love being about to readily move an entire email folder to a different imap amount entirely with drag and drop.

I think it mostly just reminds me the most of Eudora so I just like it for being familiar.

(I've never encountered this bug, I guess I just don't have changing my signature in the workflow.)


Did you open a bug? I've opened 3 Evolution bugs over the past few years, and the devs are always helpful. They even changed some behavior that was annoying to me, which made me wonder if I was in fact the only user...


> which made me wonder if I was in fact the only user...

Nope, there's at least three of us :)

It's by far my favorite client. Not only is the UI nice (for some reason I can't stand Thunderbirds UI), but it works well with integrations and multiple accounts.



As others have mentioned, Thunderbird is the least worst option (in our opinions :). I use it with CalDAV/CardDAV synced to my Nextcloud instance.

The three good options are KMail, Thunderbird, and Evolution. Everything else is CLI or lacks features like DAV.

KMail and Evolution both bring in the entire KDE/Gnome PIM suite with daemons and other programs, making them not great unless you are using Gnome or KDE - as another HN commenter said about Java web applications, you wanted a banana and instead got the entire jungle and an angry gorilla. But they do integrate very well. Thunderbird is standalone.

I tried Evolution for a week, about a month ago, and have used KMail and Thunderbird a lot.

KMail is fully-featured, with native support for CalDAV/CardDAV and 'send later.' But it's incredibly complex and easy to misconfigure. When upgrading to a new computer recently, I tried doing an export => import to transfer data, but it apparently permanently borked the KMail installation on the new computer. Tried uninstalling/reinstalling and deleting all KDEPIM-related files in ~, and it still would not work... Even on my old computer I still kept Thunderbird installed along with KMail sometimes didn't work properly.

Thunderbird is very straightforward to use and is quite stable. I use the TbSync add-on (Thunderbird has an official add-on repository like Chrome/Firefox). You'll also need the 'Provider for CalDAV & CardDAV', which adds that functionality to TbSync but is distributed as a separate add on.

Set up cal/carddav account, then go to calendar, right click the toolbar => Customize, and drag the 'Synchronize' button onto the toolbar so you can force a synchronization if needed (in addition to the timer-based background sync).

There's also a 'send later' add-on available. Aside from that, I only have a few minor issues with Thunderbird:

To switch between HTML and plain-text emails, you need to shift-click the 'compose new email' button. Can't switch in the middle of composing; you'll need to make a new email and copy-paste over. Changing the default from HTML to plain text requires going into about:config. And you can't enable/disable text wrapping on the fly for plain text emails; it's an about:config pref.

With Gmail accounts, it incorrectly lists the inbox/sent/etc. folders in a subfolder of the account (functions properly, but ugly). You have to right click the account, go to Settings => Server Settings => Advanced and set the IMAP Server Directory to '[Gmail]'.

Finally, TB has its own Spam filtering mechanism. You can't fully disable it. Even if you go to account settings and disable junk filtering for that account, it still shows a button to mark as junk and overrides the J key for that. Annoying if you are used to vim controls and press J often. Also J has the homing nub on the keycap so I like pressing it a lot...

KMail has all of this stuff built in / fixed, but is just way too complex and brittle.

Actually, after seeing your article's screenshot of Evolution with KDE titlebars, and a person replying suggesting that Evolution is good for this purpose, I'm trying it out on KDE. Never thought that would happen! I don't use signatures so hopefully this bug doesn't affect me... I did have to separately install the gnome-keyring package to get it to remember the IMAP/SMTP password but aside from that it appears to work fine.


(replying where I have something semi-intelligent to add.)

> Thunderbird is the least worst option (in our opinions :).

I use TB when I use macOS. TB does weird thing to plain-text email formatting, though. E.g. it sometimes refuses to let me delete lines that contain "> ", and it sometimes freaks out when I try to insert al line break in a section of quoted text. (I reply inline, like a civilized emailer.)

> I tried Evolution for a week, about a month ago, and have used KMail and Thunderbird a lot.

I’ve used all three for years. KMail would be my preferred option if it was way more stable and less buggy. It has great features and I feel at home in it. But it works way less reliably than Evolution.

The version of KMail shipping on Flathub doesn’t even start. —and that’s when it’s running in a sandboxed environment that’s identical on everyone’s systems!

> There's also a 'send later' add-on available.

I know. https://www.ctrl.blog/entry/kmail-cve-2017-9604-openpgp.html On a related note, I couldn’t login to my IMAP account with KMail maybe ten years ago. My password back then contained an apostrophe. KMail didn’t encode it properly and would crash every time it tried to submit the password to the server. The bug also made it impossible to overwrite the saved password with a new one from the UI.

> Actually, after seeing your article's screenshot of Evolution with KDE titlebars.

The screenshot is manipulated, see disclaimer at the bottom of the article. It’s indeed running under Plasma, though.


I'm not saying Thunderbird isn't right for you, but evolution had all those things and sounds no harder to use... this is just one bug. It just reminds me of Eudora mail and outlook more I think.


Have you looked at Geary?

I haven't personally used it, I just know that it shares a lineage (?) with the elementaryOS project, and they seem to be making great stuff.


elementaryOS' mail client is now a separate implementation to Geary and is based on libraries that underly the Evolution mail client.

https://github.com/elementary/mail


Yep, but my understanding was the Geary's lineage was the same in that both projects originally grew up together before elementary decided to change direction. Not true?


Right, they used to share lineage but don't any more.


I love Linux for what it is, for the whole idea, being open, free and "democratic" but I tried using it while my Macbok was in service and oh boy, it's like having a Hackintosh 10 years ago, for work it was bearable but for "personal use"? I'd rather pay 5x more for something that "just works". And I did.


My personal machine runs Linux because my professional experience is that macOS doesn't 'just work', and Linux is easier and less opaque to fix.

I don't want to trawl through 'have you tried turning it off and on again' on Apple support forums, I want to find the text-based config solution in the Arch wiki, a man page, or unix.SE.

(Yes that order, not `man` first, typically. 'Sue me'. Other than for executables I don't find it that 'discoverable' for what's available or might be relevant. I only recently discovered `man [7] hier` - but how was I supposed to know the page is called 'hier' (for hierarchy of course, but even that)? I got it from a unix.SE answer.)


The name apparently comes from "Version 7 AT&T UNIX", which is where the page first appeared.


The main difference for me is that there are more ways to rescue, recover, fix, and work-around issues. I’m typing this on macOS now. I run into about the same amount of issues on macOS as I do on Linux. The big difference is that I’m just f*ed on macOS whereas Linux leaves me with multiple paths to save myself out of troublesome situations.


And that's exactly why I run linux, because it "just works", and has done for 20 years.


Alternative take: I've been using Evolution as my main email client since 2008-ish, mostly on Arch and Debian systems. I have never encountered this bug, and I'm willing to bet most people have not either.

The bug probably exists, and maybe with the magic set of configuration options, I could make it trigger too. But bugs can be finicky like that -- developers certainly don't like them, and it'll probably vanish pretty fast if they are able to reproduce it.


I tolerated a lot of crap from Evolution to get sync to my Gnome calendar stuff and notifications. Eventually it was too much, though. Sad. Overall I liked the software but data loss is hard to stomach.


It reminds me that we humans are capable of going to great lengths to resist change, even when it creates shitty outcomes.


Not quite arbitrary.

He new that he could avoid it by not changing the signature afterwards.


And yet it still bit him twice a month? After the second time that happened, I would have deleted that program and smashed the hard drive to keep it from returning.


Does anyone know of a good Linux email client that isn’t crippled by show-stopping bugs of this sort? I used Mailspring for a while, which has a nice modern interface, but quit after discovering that my drafts were only saved locally, not on the server. This has been an open bug for at least two years.

In the end I’ve always fallen back to Thunderbird as the least bad option.


I live in the KDE ecosysteem so I gave Kmail a serious shot, but it is such a pain to setup with multiple accounts that I gave up. It is very easy to misconfigure sth that I can't think of being a use case anybody needs, but then things like reply to all by default were removed because it ahem a certain group of FLOSS Devs is against such shenanigans. Pity, because apart from these issues I really like this mua.

Thunderbird is and will be for some time the only realistic least bad option. Once they replaced xul, perhaps updating its visuals will get easier.


Have you tried Geary? I helped a colleague get set up with ElementaryOS a while back and it looked really nice and, compared to Evolution, was much lighter and faster.

I haven't used it in anger though, I tend to stick to email in the browser these days.


I did use Geary for a while, I can't remember now why I stopped. I appreciated how lightweight it was but I think it was just a little too lightweight, missing one or two features that I'd rather put up with thunderbird than live without.


ElementaryOS now have a new email client that uses the Evolution libraries but has a Geary-like interface:

https://github.com/elementary/mail


I'm afraid the genre is pretty much dead as everyone seems to do email in the browser nowadays. Apart from TUI MUAs like Pine and Mutt - that may not be everyone's cup of tea - I found Sylpheed quite useful for a while. It is not as bloated as Evolution or Thunderbird but still has all the useful features I need and like. Its development seems to have slowed down though, so I don't know if it has any future...


Man, hearing this is a reminder of why I don't use Linux as my desktop OS and likely never will. My default mix of applications has shifted over the last 30+ years, but e-mail has always been one of the most fundamental apps for me, going back to the VM/CMS mail client (and before that VMS's mail program which was comparatively a disaster).


I'm not sure why you think this issue is specific to linux, it isn't.


seamonkey continues the Netscape Communicator legacy of having a browser and MUA together in one product. I use it for doing IMAP and POP actions to reorganize and back up my mail. I don't compose email in it much, but it does give you the choice of HTML or plaintext.

also, emacs can be used as an email client


Doesnt seamonkey basically bundle an old version of Thunderbird? Which itself, while very functional, isn't exactly slick and modern ;)


I highly recommend claws-mail. It is lightweight, very stable, and easily scriptable with Python.


I've used several, on linux and MacOS, and gmail. Now I use mutt, and am happy. Nothing else comes close. But the author writes that he needs integration with cardDAV, or something. I don't know if mutt can do that.


Emulate macOS in a VM and download Spark ;) I recently discovered this email client and it completely revolutionized how I do email.


A 5GB limit, and the $6/month (?!?!) has a 10GB limit??????


For their file storage/sharing service unrelated to emailing.


Past security issues aside, Bluemail.


Of course recoverin an email is an innocent disguise.

The same approach works for recovering any secret information that people used on a computer that an attacker can access. Of course there are plenty of possibilities. But it’s eye opening to see them in action.


Yes, encryption keys can persist in memory, too. That's why many law enforcement agencies use something like a HotPlug[1] + mouse jiggler to keep machines powered on when executing a search warrant.

[1] https://www.cru-inc.com/products/wiebetech/hotplug_field_kit...


If you let an attacker have physical access, it's game over anyway.


If you don't want to pay for HotPlug, you can also grab a full memory dump using FTK Imager or Belkasoft...


It's why operating systems should implement a lockdown option to restrict users from performing arbitrary access to memory or kernel, even if the user is root. I mentioned before that, on one of my computer, I completely disabled dynamic kernel modules, hotpatching, /dev/mem, no ptrace() to arbitrary process, etc., making it difficult for root to do any low-level access to memory or kernel. I also enabled IOMMU, it isolates the address spaces of different hardware from each other, so no external hardware cannot have arbitrary RAM access via DMA, hardware-based memory capturer won't work. The only way to attack is either an 0day or a cold-boot attack, the 0day threat can be reduced by using a security-minded kernel, like PaX/grsec (not available to the public anymore), OpenBSD, or HardenedBSD. As for cold-boot attack, future hardware may support full memory encryption [0] at the hardware level and fix this vulnerability. Mouse jiggler is a problem, but USB firewalls already exist [1], if proper policies is enforced by the firewall, unauthorized hardware cannot register as an input device.

There may be still some exploits, especially when you consider that Linux kernel is not designed with security as its first priority, and over the last 20 years a lot of black magic has been developed to insert bad things into the kernel, but at least doing the countermeasures I mentioned will make it difficult. Hence, it's impossible to do any low-level changing or debugging on the system without rebooting it - which will immediately revert the system back to a "at rest" state, and triggers full-disk encryption. Other people may choose to do the opposite, it's a tradeoff between uptime and security.

Unfortunately, any attempt to introduce such a lockdown will be accused of being an evil technology that enables DRM. However, ultimately, the question is not whether a computer is locked down, but who is in control of the computer and it's locked down to protect whom.

[0] Don't confuse "memory scrambling" and "memory encryption". The vast majority of PCs today already use memory scrambling - the memory controller will "scramble" the data in RAM to a seemingly-random pattern using a Linear Feedback Shift Register, but it's done for electrical considerations - if there are too many 1s or 0s in a row, excessive current spike (di/dt) is produced, and it reduces signal integrity and creating excessive electromagnetic interference - LFSR-based scrambling is not for cryptography purposes and trivial to decode. On the other hand, memory encryption is a true solution that provides cryptographic protection to the RAM, and many hardware vendors have roadmap to implement it. Currently, it seems that there are two types, the first type is a "full memory encryption" - protecting RAM from physical access, the second type is "per-application memory encryption", which allows an application to request a segment of encrypted memory with an unique key - protect sensitive data of one application from accidental access by other programs. Both are helpful.

[1] https://lwn.net/Articles/738306/


I forgot all about grsec - sad to see its no longer publicly available. Thanks for the tip on IOMMU and your other measures.


Just a note for you (and other readers), as I think it needs some elaboration.

> no ptrace() to arbitrary process

Traditionally, ptrace() restriction is a grsec feature. But in mainline kernels, the same feature is available in the Yama module, see [0]. Use Yama with "kernel.yama.ptrace_scope = 3" will permanently disable ptrace() for all users, including root, and it cannot be enabled again. Then, you should also compile your own kernel, so you can disable /dev/kmem (CONFIG_DEVKMEM), /dev/mem (CONFIG_DEVMEM) and /proc/kcore (CONFIG_PROC_KCORE) in the Linux kernel. Also, I forgot to mention kexec(), which allows the attacker to execute another kernel without rebooting, so CONFIG_KEXEC should be disabled as well. And the list goes on and on, I think it's necessary to download an old grsec kernel, and using the configuration section of grsec as a checklist (and try disabling them using mainline technique if possible) if your security is serious business.

If you do these things, it will block the technique described in the original article.

[0] https://www.kernel.org/doc/Documentation/security/Yama.txt


This reminded me I was still going to report this bug, but anyone else should feel free to do so before me: gpg-agent stores your password/-phrase in plain text in memory indefinitely. You can clear the cache with some command and they also expire after some time, and gpg-agent will pretend to have forgotten them and prompt you again for the password, but the memory that contains the password is not overwritten and you can still dump the process' memory and retrieve it. Email contents seem relatively benign by comparison...


That's huge and definitely should be reported. Getting rid of the private key is one of gpg-agents primary jobs.


Yes, eye opening. Non IT folks often believe they are safe because no one will find their secrets in a fast sea of information anyway. Nothing can be farther from truth. Most of the time key material and other secrets can be extracted automatically using widely and freely available tools.


> There are specialized tools you can use to analyze this data blob.

These are called file carving tools and two better known ones are foremost and it's successor scalpel [1].

[1] https://github.com/sleuthkit/scalpel


Scalpel, as good as it was back in its time, sadly has been stalled. Carrier and/or the folks in charge of The Sleuth Kit have taken it into their github repo[0] but there haven't been commits for ~7 years now.

I did a thesis on file carving some 10 years ago, and scalpels ideas where very good back then. Photorec[1], however, has been the gold standard for a long time on (open source) file carving. It can handle text based formats way better (scalpel is severely limited in this aspect due to the "header/footer" paradigm), and is a wonder with stream based formats (that can have boundaries on the bit level).

And it's not because they authors weren't good[2], I think what mainly happened is that they didn't have the time to keep maintaining the software they created (I know that has happened to me more than once).

There are also some commercial file carving tools, though most are aimed at having better integration with forensics software (like Encase, FTK, Oxygen, etc) or automate parts of the process, like document analysis. Still, if you just want to compare them by their ability to recover files, I'm pretty sure Photorec makes it to the top.

[0] https://github.com/sleuthkit/scalpel

[1] https://www.cgsecurity.org/wiki/TestDisk_Download (PhotoRec is part of TestDisk)

[2] They're some of the best in the field of digital forensics


Good to know! I always had ignored Photorec because I thought it is only for image formats.

To add to your list of options there is also YARA when used with appropriate rules. I don't know how it stacks up against specialized tools though.


Photorec supports a crazy ammount of file types (about 400 I think, but since they keeep adding it may well be over). Fun thing: Diablo II savefiles (and other games!) are carve-able with Photorec.

And it can also handle fragmentation (though I haven't tested the later versions to see how strong that is).


You can also manually carve Photorec using the qt ui.


I've noticed the same with web-able apps, you spend time typing-up some missive and then it freezes and refreshing the page loses it all. Could linux write everything to a file every 30 seconds. A bit like a keylogger, only you know it is there.


There used to be a Firefox add-on called Lazarus that did this.

https://www.pcworld.com/article/227948/Firefox.html

From Tom's Hardware:

"Lazarus: Form Recovery is a free downloadable Add-On for the Firefox web browser that automatically saves everything you type into forms of web pages you visit.

With Lazarus: Form Recovery, you will never lose what you write after a crash the browser or other technical problems. In the case when a problem, simply right click and select "recover form" to retrieve data previously typed."

However, that was for web forms, not an email client or other applications.


I’ve found that this can replace Lazarus in modern Firefox:

https://addons.mozilla.org/en-US/firefox/addon/form-history-...


Thats really buggy with Firefox ESR, Mozilla are adding more WebExt APIs to make it work better though so hopefully it will work at some point.


Yeah, but it would require an enormous amount of fast disk space. An actual keylogger would probably be more useful.


> Could linux write everything to a file every 30 seconds. A bit like a keylogger

yes, but probably not a good idea?


I was using Evolution on daily basis at work (around 2015-17) but I switched quickly to Thunderbird due to stability issues. I was using Evolution primarily for its support for Exchange Server, but it wasn't very stable at the time. On the other hand, the same was possible in Thunderbird through a very solid proprietary plugin (exQuilla).


I used to (when this was still possible) dump /proc/memory (or kmem?) to file and rummage through it looking for partially composed website submissions when Netscape decided to eat itself, back in the 1990s. Remarkably successsful.


Using Process Explorer at least, it's still just a right click on any process away.


There used to be a global system memory file in /proc, 2.0 / 2.2 kernel series, unless my memory's playing tricks on me. Even content from dead processes was still (briefly) available.


Yup..this is also how the fbi recovers stuff too. After Ross Ulbright was caught they did this to his laptop.


I thought they just kept the computer open?


Because of some reasons I use evolution and thunderbird. The search function in Evolution is abysmal.

May try out InScribe


nano ftw.

Modern web email clients have this since a long time and Gmail never crashed internally for me ;)


> Gmail never crashed internally for me

Some google services seem to be crashing the tab in my Firefox (but me only, since this bug has been there for many months now and I reported the crash with URL a bunch of times), I remember street view for sure but recently there was another, don't remember the name. (I don't use google services that frequently aside from pulling youtube content.)

Anyway, point is, browsers aren't infallible either and google is known to make their software work only in google-branded browsers. I'm not sure that a really stupid bug in $someSoftware is a good argument for why we should move everything into the browser, or a Google product in particular (why not bring up roundcube or runbox or something?).


Something tells me that if you are bent on using FOSS and/or care about privacy then Gmail isn't going work.

Can't argue that there must be easier ways to handle email.


I use nano, but I have to say I am very surprised that they mentioned it as a viewer for large binary files. It reads the entire file off of disk into memory and then tries to split it into lines! It can't even do well on files with large enough lines :(


Nano reads large files in chunks. Other text editors do indeed pull in the entire file before doing anything useful.



I misread this as: "I recovered a lost email from my client’s memory". It made me think of the Black Mirror episode 'Crocodile', and I was quite amazed. Then I saw 'email' client... haha


Author here. That was actually the working title up until two minutes before publishing.


Yep, and the title, once re-parsed is fine, but I also had - initially - the wrong impression, it flashed before me how you hypnotized your client (customer)and managed to retrieve from his/her memory the contents of an e-mail message he/she ddn't rememeber anything about.


That’s pretty much exactly how I thought the previous title might have been misunderstood!




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: