64-bit app support to allow exporting of all formats

[KeithM 1]

I've been trying to keep my Evernote backed up, and attempting to do so in multiple formats to ensure that I have something that can be parsed/imported later.

With 4000 notes, including a large number of scanned images and PDFs, the current 32-bit Evernote application crashes when exporting all notes in "Single-page html" or web-archive/mht. It took me a little while to track down what was happening, but essentially Evernote is consuming a large amount of RAM during exporting those formats. The used memory increases until it gets to about 3.5GB and then crashes.

The error messages in the activity log file look like this "date/time [pid] CBuffer: Failed to allocate xxxx bytes of memory." There should be a few of them before the app crashes.

The computer has 32GB of ram, and could certainly support a temporary increased usage to facilitate the export.

A 64-bit version would be nice to avoid these types of limitations.

Thanks

Keith

 

[Riptide360 7]

What platform are you on? All Mac and iOS apps are 64 bit. You might try exporting on a different platform to see if that helps, but I'm guessing the issue lies with Evernote's exporter or server side.

[KeithM 1]

Windows 10. I should have mentioned. On Windows, it's a 32-bit app. No 64-bit app is available (yet?)

Evernote Support confirmed the problem I was seeing was client side, with the client exceeding available ram for a 32-bit app during a large export. My "note database" is approaching 5GB, so depending on how they do the export, it's no wonder it is exceeding the ~3.75GB limitation of 32-bit apps.

I could probably break the notes into chunks for exporting.

The other thought is that maybe using a single HTML or MHTML isn't really a brilliant idea for exporting so many (large) notes. Maybe my browser would crash trying to display something that large.

Of course, Evernote should essentially not let the user select those options which are known to crash. They could, for instance, disable those options when the overall database is over a certain size.

Thanks.

[jefito 5,598]

19 hours ago, KeithM said:

Of course, Evernote should essentially not let the user select those options which are known to crash.

Since this only crashes under some circumstances, and determining fixed limits on diverse machines is hard, that's not always feasible. Also see https://en.wikipedia.org/wiki/Halting_problem. I'm guessing that this is a fairly rare use case; you're right in noting that exporting a large note database to a single HTML document probably isn't such a hot idea.

[KeithM 1]

It's not just "some circumstances", it's any time your evernote database goes over a fixed size. The size is approximately 4GB, but someone could narrow this down. Or heck, just ball park it a little low.

The 32-bit memory limitation of 32-bit apps doesn't change depending on the machine. It's 4GB. Ok, 4GB minus other memory the application is already using.

if (evernote_db_size > 3gb || free_memory < evernote_db_size) { radio_button_single.enabled = false; radio_button_mhtml.enable = false; }

Furthermore, how about a user friendly message when memory is almost depleted? "We're sorry, but the available RAM for this application is running low, so the export has been aborted. Please click here for a knowledge base article." 

Not quite the halting problem, is it?

Keith

 

[jefito 5,598]

3 hours ago, KeithM said:

It's not just "some circumstances", it's any time your evernote database goes over a fixed size.

Actually, I don't think that it's quite that simple. For instance, what *is* the database size? Remember that database size on disk is deceiving, because the database may be fragmented. And size of database probably correlates to amount of memory used in export, but that's likely not an exact an exact 1-for-1. BTW, in Windows, 32-bit apps get to use somewhat less than 3GB; it's 2GB normally, but 3GB if you flip a certain Windows switch (I forget exactly which one; it's been awhile). Ball-parking it low is probably useful (I'd put up a warning, up front, myself), but not conclusive either. And one of the problems with export is if you're reliant on 3rd-party libraries, then memory error handling may not be entirely in your hands. But yes, in general, you try to catch and recover and clean up when memory runs out, but it can be difficult to firewall all possible crash scenarios.

I'd agree that a conversion to 64-bits would be helpful (though again, they might be at the mercy of 3rd-party libraries). I still maintain that a wholesale export to any supported format (including ENEX) is probably a bad idea; for one thing, you're going to lose data (for sure your notebook names, and possibly others), and your point about what to do with a huge single export file still holds. 

I'd recommend at least exporting on a notebook by notebook basis. A pity that the ENScript tool only exports to .enex files, because you can at least automate that.

[jefito 5,598]

@KeithM: upvoted this for you, btw...

[EdH 1,670]

On 5/26/2017 at 9:18 AM, jefito said:

Since this only crashes under some circumstances, and determining fixed limits on diverse machines is hard, that's not always feasible. Also see https://en.wikipedia.org/wiki/Halting_problem. I'm guessing that this is a fairly rare use case; you're right in noting that exporting a large note database to a single HTML document probably isn't such a hot idea.

FWIW, I gave up long ago (3 yrs?) on Windows because it crashed. I never traced it down to RAM usage though, but makes sense. 32bit apps cannot see more than 3GB. EN really should be 64bit. No one should be running 32bit. I am not sure you can even by 32bit computers anymore - i.e. preloaded with 32bit Windows. Maybe some IT departments purchase those.