How to make proper and automated back-ups of Evernote v10 data [on Windows]? [SOLVED]

[MvdH 63]

Hi there peers and fellow Evernote users and enthousiasts where applicable,

How do you guys, galls and others make proper and automated back-ups of Evernote v10 data?

I just send the following support request to Evernote about this subject. I'm very curious how other professionals organise this. 

Thx in advance!
Greetings and a smile,

Mark

Hi there! 

Evernote is very important to me. It has been my paid second braind since earlier than 2012. It contains 15.000 notes and thousands of tags. I use Evernote v10 as a day to day driver, except for one feature. 

A good back-up is off-site, off-line, duplicate, right retention, checked, etc.. To be able to restore all or part of my data in Evernote, I've been creating automated back-ups via Backupery using Evernote Legacy. Those resulting back-ups are stored in off-site and offline locations (Frankfurt datacenter and rotating external harddisks). I check restoring every now and then, which is must-have for good back-ups. 

My data is too important to trust "well, we are in a datacenter and we got back-ups arranged for you".

I feel the end of Legacy is coming now Bending Spoons is forcing pop-ups on me trying to uninstall legacy. As a productivity tool, saving me time, being 100% reliable with fallbacks and such, I expect automated back-ups to be possible, meeting the criteria of offsite, automated, offline, etc.. 

I've checked your FAQ, blogs, vlogs, and forums. I have not found the right answers. How do I move forward without the need of Evernote Legacy? What is the professional route to take for this professional tool and my professional need for correct automated true back-ups? How long will Evernote Legacy be available for my back-up needs? Will making snapshots of the offline cache of Evernote v10 data be sufficient and how would it be restorable?

Thx in advance, greetings and a smile, Mark

[agsteele 1,858]

Evernote-backup from GitHub. I have a schedule to backup once a week. I also generate a set of ENEX files once a month but can manually generate the ENEX files of I need one sooner.

[eric99 638]

1 hour ago, MvdH said:

Hi there peers and fellow Evernote users and enthousiasts where applicable,

How do you guys, galls and others make proper and automated back-ups of Evernote v10 data?

I just send the following support request to Evernote about this subject. I'm very curious how other professionals organise this. 

Thx in advance!
Greetings and a smile,

Mark

Hi there! 

Evernote is very important to me. It has been my paid second braind since earlier than 2012. It contains 15.000 notes and thousands of tags. I use Evernote v10 as a day to day driver, except for one feature. 

A good back-up is off-site, off-line, duplicate, right retention, checked, etc.. To be able to restore all or part of my data in Evernote, I've been creating automated back-ups via Backupery using Evernote Legacy. Those resulting back-ups are stored in off-site and offline locations (Frankfurt datacenter and rotating external harddisks). I check restoring every now and then, which is must-have for good back-ups. 

My data is too important to trust "well, we are in a datacenter and we got back-ups arranged for you".

I feel the end of Legacy is coming now Bending Spoons is forcing pop-ups on me trying to uninstall legacy. As a productivity tool, saving me time, being 100% reliable with fallbacks and such, I expect automated back-ups to be possible, meeting the criteria of offsite, automated, offline, etc.. 

I've checked your FAQ, blogs, vlogs, and forums. I have not found the right answers. How do I move forward without the need of Evernote Legacy? What is the professional route to take for this professional tool and my professional need for correct automated true back-ups? How long will Evernote Legacy be available for my back-up needs? Will making snapshots of the offline cache of Evernote v10 data be sufficient and how would it be restorable?

Thx in advance, greetings and a smile, Mark

And related to that, how can we back up our beloved note links? What's the point of providing new note link functionality without backup capability?

As already mentioned before, the software fix is very simple, it's just a matter of saving the note-id in the enex file...

[gazumped 9,819]

I'm still running Legacy and Backupery too.  I've been looking at MS Power Automate which should allow me to automatically download and save notebooks from v10,  but haven't had time to get around that learning curve yet,  or test out the practicality of downloading large notebooks.  (61K+ notes here).  Please let us know what reaction you get!

[MvdH 63]

Hellow my friends. If I do "check for update" in Backupery, it doesn't say there is an update. However, I now looked at their website and they claim a much later version available: https://www.backupery.com/products/backupery-for-evernote/

Before I retract my support question here, at Evernote and at Backupery, I'll try this update first. Looks promising. If they now add chaning tags to 1000+ notes at once like in legacy, then I don't need legacy anymore.

[MvdH 63]

3 hours ago, agsteele said:

Evernote-backup from GitHub. I have a schedule to backup once a week. I also generate a set of ENEX files once a month but can manually generate the ENEX files of I need one sooner.

Sounds like work, but maybe it's only a little time of work. I want to spend next to zero time on technical tinkering, I've got other things to do 24/7. 🙂  But I'll check it out if I find no solutions. Thanks a million for thinking along!

[gazumped 9,819]

Thanks - I hadn't looked at the backupery site in some time;  I also downloaded and started the new version - initially disappointed;  looks like it could take a long time (24 hours plus) to download my whole account - and that's if Evernote don't limit the account download at some point.  I'll keep it going and comment back here.

[agsteele 1,858]

4 hours ago, MvdH said:

Sounds like work,

Well, it took me very little time and I'm not especially technical. Happy to share my solution of you would like to PM.

[MvdH 63]

2 hours ago, agsteele said:

Well, it took me very little time and I'm not especially technical. Happy to share my solution of you would like to PM.

Thank you. Very friendly! I'll keep your offer in mind. Investigating the Backupery answer/solution first. 🙌👊👍 We will see how that unfolds. Also work by the way. 😉😄😜  A few curious questions: How does the solution you use work? Is it for Windows Evernote? Does it need legacy or connect directly to the Evernote servers? Does it use ENscript, and does that exist? Basically: What is the technological principal?

[MvdH 63]

6 hours ago, gazumped said:

Thanks - I hadn't looked at the backupery site in some time;  I also downloaded and started the new version - initially disappointed;  looks like it could take a long time (24 hours plus) to download my whole account - and that's if Evernote don't limit the account download at some point.  I'll keep it going and comment back here.

Ok. I don't have the time this week to switch to the new backupery (working 24/7). Curious how your experiment will unfold. I have 25GB of data, 15000 notes. I hope the new Backupery will need less than one workday to perform the backup. The old Backupery with Evernote Legacy takes about 4-5 hours normally to finish a full back-up. Let's say 24 hours will work for me, though that's unhandy, but langer than that is practically undoable once a week. Or I buy a seperate computer specifically for the task of backing up Evernote. That could be a solution I could even live with. 😄 

[agsteele 1,858]

1 hour ago, MvdH said:

How does the solution you use work? Is it for Windows Evernote? Does it need legacy or connect directly to the Evernote servers? Does it use ENscript, and does that exist? Basically: What is the technological principal?

evernote-backup has a secure direct connection via the API. You login and run the backup. A download of your data is created in a location of your choice on your computer. If you run the command with a different set of commands a set of ENEX files will be created from the data.  You can run the commands on a schedule.  It certainly has a Windows and Mac version and possibly Linux too. It does not require any client application. The program runs on your device so a direct link from your computer to Evernote.

https://github.com/vzhd1701/evernote-backup

Instructions further down the page. Download from the almost hidden direct download link.

[gazumped 9,819]

Hmmn.  The new Backupery started at 12.25 GMT and timed out after 12 hours,  having downloaded nearly 6,000 notes but the download is 4GB of cache files,  not actual processed ENEX files.  I'll raise a support ticket with them.  🤔

EDIT: - And I just noticed that I left the desktop just before 6pm local and the backup stopped soon after. The app timed out after another 6 hours,  so it might have been my desktop power settings that caused the glitch.  Will try again tomorrow!

[johansan 37]

6 hours ago, gazumped said:

Hmmn.  The new Backupery started at 12.25 GMT and timed out after 12 hours,  having downloaded nearly 6,000 notes but the download is 4GB of cache files,  not actual processed ENEX files.  I'll raise a support ticket with them.  🤔

EDIT: - And I just noticed that I left the desktop just before 6pm local and the backup stopped soon after. The app timed out after another 6 hours,  so it might have been my desktop power settings that caused the glitch.  Will try again tomorrow!

I am running Backupery on a local server with no issues but I only have 1GB / 1600 notes. Took only 25 minutes for full backup.

The Windows server is connected to Ethernet on a 500 mbit Internet connection.

Maybe you are on a local WiFi with less-than-optimal connection?

[gazumped 9,819]

1 hour ago, johansan said:

Maybe you are on a local WiFi with less-than-optimal connection?

I think I have about 15% of your speed - quick,  but not super-speedy.  My settings put the desktop to sleep after 20 minutes inactivity,  which I just changed to 'never' - trying again...

[Danni P. 6]

Evernote has a history of being a secure and reliable place to store your notes and use for PKM. It baffles me that we have to opt for third-party solutions for something so important like this. For the price we pay, why would Evernote not offer similar simple backup protocols?  (https://www.backupery.com/products/backupery-for-evernote/ and https://github.com/vzhd1701/evernote-backup) 

IMO, features like this would continue to set apart Evernote from its competitors; being a more secure place to store your notes. Also, giving another incentive as to why you would pay a higher price? @Federico Simionato I hope it's ok that I tag you here. Maybe other offline/backup solutions are in the pipeline?

[agsteele 1,858]

You can do the backup on Evernote. The solutions offered are just niftier.

[gazumped 9,819]

26 minutes ago, Danni P. said:

we have to opt for third-party solutions for something so important like this

It depends on the degree of security you require.  Evernote - up to this point,  I think - relied on the fact that they had backups of each note back to its creation,  and a separate 'parent' copy of your notes on their server,  as well as (in Legacy) client copies on any connected Desktop.  In 15+ years of using Evernote I've never yet had to restore from an external copy of the database.  In v10 however,  the local copies of notes don't (IMHO) seem user-friendly. 

It is easily possible to export notes as backups on a notebook by notebook basis - Export notes and notebooks as ENEX or HTML - though when you have a lot of notebooks (I currently have 419 apparently...) that method is not so practical.

The other options,  as noted above,  are just different and much more convenient ways of doing the same thing - and I was looking into whether MS Power Automate could be persuaded to open the notebooks tab and export each notebook in turn if I left it running.  

[Danni P. 6]

@agsteele this is manual export, only a single notebook at a time? 

[Danni P. 6]

3 minutes ago, gazumped said:

The other options,  as noted above,  are just different and much more convenient ways of doing the same thing - and I was looking into whether MS Power Automate could be persuaded to open the notebooks tab and export each notebook in turn if I left it running.  

back to my point 🙂

[gazumped 9,819]

1 hour ago, Danni P. said:

back to my point 🙂

I think you missed mine about Evernote already having doubly-redundant backups built into the structure...

[Danni P. 6]

@gazumped nono, I got that. My neighbor was biking her way to work every day, for 15 years. Still, wearing a helmet would have saved her life when she got hit by a car.

You were using backupery, right? Why is that?

My point here is that we like to be safe and maximize personal control over our creations/valuables. Evernote makes it possible to do, but not practical - as you said. 

IMO, this is not a niche want-feature, but a need for many Evernoters. A reason why many similar types of note-takers would opt for something like Obsidian. Currently in Evernote; adding another step to solve the issue (third-party solutions), will add more friction, pose another type of risk, and might be costly for both customers and the Evernote corp. ?

So again, why not offer a similar simple protocol in Evernote? Can only see this type of feature (read: bulk notebook exports with options for automation) would build on the pillars of the Evernote culture, add more trust for the product/company, increase the value for Evernote customers, and attract new ones.

[agsteele 1,858]

2 hours ago, Danni P. said:

@agsteele this is manual export, only a single notebook at a time? 

I have it configured to run on a schedule, once per week, for my whole account. You can run it manually if you wish. I'm not sure if you can run it one notebook at a time since that wasn't my requirement.

[Danni P. 6]

5 minutes ago, agsteele said:

I have it configured to run on a schedule, once per week, for my whole account. You can run it manually if you wish. I'm not sure if you can run it one notebook at a time since that wasn't my requirement.

 Thanks for sharing this option @agsteele This was the solution offered by a github user, using Homebrew or PIPX install files, and Python? 

[agsteele 1,858]

Download from the not very obvious link for the installer for your OS. Follow the configuration instructions on the main page.

[Danni P. 6]

11 minutes ago, agsteele said:

Download from the not very obvious link for the installer for your OS. Follow the configuration instructions on the main page.

Adds to the point 🙃 my spouse would have absolutely no idea what to do - and if she did, would she be comfortable in doing so?

[gazumped 9,819]

Backupery has been running all day - about 14 hours now,  with occasional time-outs from Evernote for too much traffic (if I'm slowing sync for everyone I do apologise!) - still not complete though.  The process continues - more tomorrow!

[MvdH 63]

9 hours ago, gazumped said:

Backupery has been running all day - about 14 hours now,  with occasional time-outs from Evernote for too much traffic (if I'm slowing sync for everyone I do apologise!) - still not complete though.  The process continues - more tomorrow!

I apologise for the same. 😉

Summary:

Using the running logs, I measured how long it took to backup a sample of 100 notes. It took 142 seconds. That means 1,42 second per note. If these 100 notes are an average sample, then that means 15.000 notes of 10,84GB total resulting ENEX size, will take about 355 minutes = ~6 hours to make a full back-up on my fast laptop with fast internet.

This explains b.t.w. why my Android reinstallations that Evernote requests me to do every now and then when I have a problem (again), take DAYS to sync all notes offline. 🙂  I hope they improve sync speed on back-ups/all platforms soon by a factor 10.

Details/learnings so far:

Gazumped and everyone: Thx for all the updates. I have two laptops. One runs Evernote Legacy with the old Backupery. Backups on that laptop are HTML folder 321,35MB, Raw is 10.27GB and ReadyToImport folder is 10,84GB. So the true size of my back-up is 10,84GB. It takes the old backupery hours and hours to complete.

My 'feeling' is that this is the new Backupery a few hours longer than the old backupery on my old laptop, without doing HTML/raw back-ups alongside ENEX back-ups. If the back-up and restore is succesful, then I'll remove the old backupery from my older and slower laptop on wifi and replace it with the new Backupery and re-backup and measure and report here.

Reading Gazumpeds experiences so far, I installed Evernote v10 and the new Backupery on my 2 month old fastest and best Microsoft Suerface Pro with i7 and 32GB memory, directly connected to 1gbit up/down glassfiber Internet. I disabled battery saver, screensaver, etc.. 

The new backupery was finished in 19 minutes, it gives no errors in the UI. It did however if I open the detailed part of the UI, show that it stopped after 20 notes for the first notebook (containing thousands).  Same thing for my second notebook (I have only ttwo notebooks, with about 15000 notes in it. First I thought of a bug/problem. Then I thought it might have to do with the Trial status of that installation.

I registered this Backupery on my bought key and restarted a back-up. It now goes past first 20 notes per notebook. It seems very slow. I love however the running log it shows btw with what number note done of a total number, much more insight than the old Backupery.

 

[MvdH 63]

19 hours ago, Danni P. said:

Adds to the point 🙃 my spouse would have absolutely no idea what to do - and if she did, would she be comfortable in doing so?

Yeah, a good back-up experience/tool should be easy and reliable for any n00b and not cost a lot of time or attention or concentration.

[MvdH 63]

20 hours ago, gazumped said:

I think you missed mine about Evernote already having doubly-redundant backups built into the structure...

Good point, bút a good back-up from my point of view checks all of the following boxes:  (1) []local  (2)[]off-site (3) []offline (4) []duplicate (5) []encrypted (6) []AVG-proof (7) []restored+verified+interval (8) []retention+timeframe (no unique filenames) (9) []full must-have+frequency (10) []incremental if needed as add-on (11) []recovery duration

In addition to that it should be easy and trustworthy to set up, rely on as little human actions as needed. With what Evernote tells me, they don't make back-ups that check these boxes. They seem to have options to go back in time and have redundancy, mirroring. That's a totally different kind of solution other dan a back-up. So I conclude their methods are not enough for something that my personal life and business rely on totally. I could go broke if Evernote fails availability of my data or fails at disaster recovery of any kind.

[MvdH 63]

For those interested my dialogue with Backupery...

Quote

---------- Forwarded message ---------

Hi Denis,

 

Great answer! Thank you. Better answer than Evernote itself, ignoring my question, answering something irrelevant and closing my ticket. I found your update, I updated my question in the peer-to-peer forums of Evernote. I noticed you guys nót having the old backupery mentioning this update yet, and not spreading the news yet. Others had also not heard of the update yet. I suspect you guys are doing this on purpose, to get gradual feedback?

 

See the discussion here: https://discussion.evernote.com/forums/topic/145836-how-to-make-proper-and-automated-back-ups-of-evernote-v10-data/   I noticed two things. To me it was unclear in Trial mode it stopped after 20 notes. You should mention this prominently in the UI of Backupery. I first thought you were buggy. You can improve this yourselves. Second thing I notices is that Backupery takes 1.42 per note to back-up. For my set of 15.000 notes of ~11GB of ENEX file this means Backupery will need 6 hours to back-up. That should be 10x faster in my opinion. I believe this is due to slow API performance of Evernote itself. Your developers could consider having 10 parallel threads back-upping, if Evernote allows this "hammering" and not block you, it might improve things significantly. I believe this should only be done with approval of Evernote. Getting blacklisted or blocked is not the right route.

 

Thx for the tech insights of the workings of the New backupery through API's.

 

Greetings and a smile,

 

Mark

 

On Thu, May 11, 2023 at 8:46 PM Denis from Backupery <support@backupery.com> wrote:

Hi Mark,

 

Thank you for your message and I'm sorry for the late response!

 

Thank you so much for mentioning our app on Evernote forum, I'm really appreciating it.

 

The latest version of our app supports Evernote 10 and higher, so you are welcome to try it. 

Here is a quick story behind our app - previously the app was based on the scripting functionality of Evernote so it could work with Evernote Legacy only. The new version of the app uses Evernote Cloud API to make the backups so it does not require installed Evernote client anymore. Backupery for Evernote connects directly to Evernote and download all the necessary data.

 

Hope this helps. Please let me know if you have any questions!

 

Thank you,

Denis

Backupery Team

On Wed, 10 May at 9:01 AM

Hi there Backupery,

 

Very happy and content user of Backupery for Evernote! That first of all. Good practical tool, enough configuration and the foundation of saving me a lot of time.

I just posted the following question in the forums of Evernote.

 

You as Backupery, what is your stance in all this, your vision and are you able to help me to find answer? Paid answers/work could also be great. A good automated back-up tool for Evernote 10 as Backupery is would be great. I'm willing to pay €1000 euro's for one, for it will be worth that totally.

 

Greetings and a smile,

 

Mark van der Hijden

 

TO DISCUSSION FORUMS OF EVERNOTE

 

Hi there peers and fellow Evernote users and enthousiasts where applicable,

 

How do you guys, galls and others make proper and automated back-ups of Evernote v10 data?

 

I just send the following support request to Evernote about this subject. I'm very curious how other professionals organise this. 

 

Thx in advance!

Greetings and a smile,

 

Mark

 

 TO EVERNOTE SUPPORT:

 

Hi there! 

Evernote is very important to me. It has been my paid second braind since earlier than 2012. It contains 15.000 notes and thousands of tags. I use Evernote v10 as a day to day driver, except for one feature. 

A good back-up is off-site, off-line, duplicate, right retention, checked, etc.. To be able to restore all or part of my data in Evernote, I've been creating automated back-ups via Backupery using Evernote Legacy. Those resulting back-ups are stored in off-site and offline locations (Frankfurt datacenter and rotating external harddisks). I check restoring every now and then, which is must-have for good back-ups. 

My data is too important to trust "well, we are in a datacenter and we got back-ups arranged for you".

I feel the end of Legacy is coming now Bending Spoons is forcing pop-ups on me trying to uninstall legacy. As a productivity tool, saving me time, being 100% reliable with fallbacks and such, I expect automated back-ups to be possible, meeting the criteria of offsite, automated, offline, etc.. 

I've checked your FAQ, blogs, vlogs, and forums. I have not found the right answers. How do I move forward without the need of Evernote Legacy? What is the professional route to take for this professional tool and my professional need for correct automated true back-ups? How long will Evernote Legacy be available for my back-up needs? Will making snapshots of the offline cache of Evernote v10 data be sufficient and how would it be restorable?

Thx in advance, greetings and a smile, Mark

 

[gazumped 9,819]

14 minutes ago, MvdH said:

I love however the running log it shows btw with what number note done of a total number

My EXB file is 29.5GB and after 24 hours my Dell notebook is today showing 10.7GB completed.  That suggests (sigh) 22K notes out of my 61.3K total... and - currently - 4 seconds per note which looks about right from the reactions on the display.  The note numbers shown however don't seem to have any relation... maybe totals per individual notebook?

On @MvdH's "good" backups,  I have slightly mixed feelings.  As I said previously,   in 15 years or so of activity I've had no reason (that I can recall) to restore any notebooks from my own backups.  I kept records mainly in case I fat-fingered something and expunged data that I wanted to keep,  and (no disprespect intended,  Evernote) using someone else's overseas servers to store my data,  I wanted my own local copy "just in case".  

Pre-v10 I kept a set of Backupery and system backups that would go back 12 months or so but still regarded Evernote as 99.5% dependable.  The process was much quicker because it was copying files from the live legacy database on my desktop.

I still have that nagging feeling that it's my data,  so I should have a copy of everything in my control - and I should think it would benefit Evernote if I did so because a lot of the bad feeling expressed in the forums about 'lost' data is really down to User error.  If everyone had an easy way to keep their own local backup,  the number of Support requests and a degree of bad feeling should be reduced.  However Evernote was once described as holding "the largest cache of personal data in the world" and the app lives on speed;  any significant number of users downloading ALL of their notes is going to be a very significant bandwidth overhead.  So the 'lost data' support calls would probably switch to 'slow speed' calls...

Maybe an option to keep a complete local copy of your data* which is maintained on a drip-feed basis?  I don't want Evernote to put Backupery out of business (Hi Denis!) so maybe Backupery should be able to pause and resume backups when the system is dormant or shut down and operate in the background 24/7?  Or backup transparently to cloud storage?

I'm a little worried that my 'completed' files still show mostly as one very large cache file - there are no separate ENEX files... yet.  I assume that means there's still some processing to come.  I am impressed that despite the ongoing downloading Backupery/ Evernote are not detectably slowing my system down,  though they are taking up 1GB or memory each on my 32GB system.  If I can complete one full backup my next one should just be a 'changes only' follow-up and much,  much quicker.

I believe Evernote are watching this discussion more closely than some others,  so let's see what happens!

* NB some users don't want anything local because of limited storage space.  You can't please all of the people...

 

[MvdH 63]

@gazumped  Thx for your update. Let's compare notes (pun intended) coming days. I also never needed disaster recovery. But the reasons for back-ups is not the chance of frequency of trouble, it is the impact. @Danni P. said it very nicely, with his "why you where a helmet on your bike" reasoning. It is an insurrance.

About the slowing down your system, I believe this is because Evernote purposely throttles the API calls/minute that you can do. That explains 100% why your system is simply waiting for Evernote repsonses and doing nothing 99% of the time. At least I hope so, otherwise their back-end/servers are way to slow. 😄  I hope it is load spreading amongst all users. AS I said, I think those API calls should go 10x faster. 

I'll get back to y'all later on!

[agsteele 1,858]

😇 Meanwhile, I've created my backups three times albeit possibly smaller collection of notes.

[gazumped 9,819]

Show off... 

Like others have said - I looked at that Github page and thought it looked a tad complex to get it sorted out.. without doing the whole installation in detail,  is it simply just a case of downloading some code and installing the process?  Might give it a try in a couple of days - I'm determined to let the current operation finish first.

[MvdH 63]

6 minutes ago, gazumped said:

Show off... 

Like others have said - I looked at that Github page and thought it looked a tad complex to get it sorted out.. without doing the whole installation in detail,  is it simply just a case of downloading some code and installing the process?  Might give it a try in a couple of days - I'm determined to let the current operation finish first.

As I understand agsteele, I assume this script uses the same API's as Backupery. That means it will be exactly as fast, except when the overhead in Backupery is higher, or delays are build it (which would be silly). I don't expect that, so I think it will be as-fast, but Backupery easier UI and less tinkering, I think. That's why I parked the tool on GIT as an alternative. 

As I wrote the Backupery people, even with a slow API, sometimes you are allowed and not blocked/banned if you parallel process your API calls. 10 at a time. That would mean 10x the load on the Evernote servers, and 10x faster, for I suspect Backupery is waiting 99% of the time for responses.

This slowness is not a network/local laptop performance limitation, this is on the Evernote end...Per call... That idea is confirmed a bit by the fact how long a device takes to get all notes offline, like Evernote on Android. Downloading my data ~10GB compressed/encrypted should even over WIFI be done in much faster times. The time a single note takes in Backupery also strengthens this idea of mine.

Short update, Backupery new version has been working for a few hours now, looks promising. 

[agsteele 1,858]

One possibility for it being a tad quicker is that the Evernote-backup stuff runs on your device. Nothing gets in the middle of the process so a direct link from you to the API. Also, the ENEX generation is local to your device. 

But as I confessed, my data possibly occupies fewer electrons.

[MvdH 63]

49 minutes ago, agsteele said:

One possibility for it being a tad quicker is that the Evernote-backup stuff runs on your device. Nothing gets in the middle of the process so a direct link from you to the API. Also, the ENEX generation is local to your device. 

But as I confessed, my data possibly occupies fewer electrons.

I think you are totally right. Doesn't take away my desire for 10x faster API's! 😄 

Conclusion of my first test: So, my back-ups finished. It took 7 minutes less than 7 hours to finish up the fastest i7 32GB SSD newest Surface Pro of 2 months old, with 1gbit wired up/down Internet for 15.000 notes of 10,84GB total resulting ENEX size (in the old Backupery), it means it took 1,68 seconds per note to back-up. It resulted in an ENEX file of 10,3GB.  5.35GB of HTML files were in a separate folder. 

*) I predicted it would take about 355 minutes = ~6 hours to make a full back-up on my fast laptop with fast internet. It took 53 minutes longer

Good enough for me with my data, though 7 hours is longer than it should be I believe. People with 5 times my data amount exist, and they will need 5x the time, so 35 hours!

I'lll check a restore of the back-up later.

But before that, next I'll test on a laptop of 5 years old, i7, 16GB memory, and WIFI. We'll see what the difference will be later. I immediately have a benchmark difference test between wired/WIFI in my house with that. I'll get back on that.

UPDATE: I just installed Backupery on that slower WIFI and older laptop and started the back-up job. I'll get back on the results later.

@gazumped Your system is still crunching?

[Paul A. 419]

Following this thread with interest. I've always felt that Evernote should provide a native way to perform a full-fidelity backup (including attachments, tags, links, etc.) 

[agsteele 1,858]

RTE locks notes when a user is accessing them which prevents legacy apps from creating note conflicts. 

Backups via the API will be affected. Best to run Backuppery and Evernote-backup overnight or at times when Evernote is minimised or closed to avoid trouble. This MIGHT account in part for the slowness.

[s2sailor 1,169]

On 5/10/2023 at 3:36 AM, agsteele said:

Evernote-backup from GitHub. I have a schedule to backup once a week. I also generate a set of ENEX files once a month but can manually generate the ENEX files of I need one sooner.

I found this to be surprisingly easy to set up on a Mac.  Thanks for the pointer.  It does appear to be all notebooks only, which is fine for me.  The challenge I have is figuring out how to automate and schedule this.  I've never dug into AppleScript before, but maybe it is time.  Though setting a recurring task and running the backup manually once a week is not a big deal either, but automating it would be nice.

[agsteele 1,858]

On Windows I use the standard Task Scheduler and run

E:\evernote-backup.exe sync

every Sunday at 04h25  This updates the backup of my Evernote notes and notebooks.  Once a month I run

E:\evernote-backup.exe export e:\evernotebk\

As you can see the executable file is in the root of E: and I have a directory E:zevernotebk to hold the ENEX files.

That updates the local ENEX versions of my backup.  I have to manually tidy that collection of files since each month it adds a new copy rather than overwrites.  I don't doubt that could be automated too.  If I need to I can manually run the ENEX creation at anytime.

I expect that there is a similar mechanism for MacOS

[s2sailor 1,169]

As best as I can tell MAC OS doesn't have a utility similar to Task Scheduler, and to accomplish the same is a little more involved, but I'm plugging away at it.  I have noticed that the export option doesn't overwrite and is only all notebooks and not selected ones, but I'm ok with that. 

[agsteele 1,858]

I'm a little surprised to see that the often superior MacOS is missing this. Task Till Dawn seems to be the way...

https://www.addictivetips.com/mac-os/how-to-schedule-tasks-on-macos/

 

[s2sailor 1,169]

3 hours ago, agsteele said:

I'm a little surprised to see that the often superior MacOS is missing this. Task Till Dawn send to be the way...

https://www.addictivetips.com/mac-os/how-to-schedule-tasks-on-macos/

 

Thank you.  I actually understand that part of the problem.  The issue that I'm running into is creating an executable AppleScript that will run consistently.  What I have runs fine in the script editor, but is inconsistent when run stand alone.  I'll get it figured out.

I was surprised that Mac didn't have something similar either.  I was a Windows user since well, the beginning and switched to a Mac about seven years ago.  I'm a photography enthusiast and it seemed like Mac was the platform to be on so I thought I would give it a try.  I found nothing magical about the Mac.  It did seem like I was constantly googling to figure out things that I thought should be obvious, but weren't, at least to me.  We have iPhones and iPads and there is an advantage having all Apple devices, but other than that, I'm underwhelmed.  I think both OSes are fine.  It is just what you get used to.

That said, I've never had a Mac blue screen and did many times on Windows, but less often in the later years. 

Thanks for the Task Till Dawn pointer.  I'll check it out.

[MvdH 63]

Hi everyone,

Creating  a little noise in the Apple Script/Back-ups discussion. First of all, thx EVERYONE for thinking along, reading along, trying stuff and suggesting new idea's, other methods and spark creativity!

My original question was: How to make proper and automated back-ups of Evernote v10 data [on Windows]? [SOLVED]

Here are my final measurements/results from my new-Backupery tests, with the non-legacy version of the app, not relying on a local installation of Evernote anymore, using the Evernote API and doing a fresh download of your data, instead of using local data cached on your computer as the old Backupery did:

My final conclusion:

Backupery new version (published april 2023) is suitable for back-ups of my Evernote data [on Windows], of a little of 10+ GB of ENEX resulting file,15.000 notes. I'll get back if the restore test fails. The speed is too low. I believe because of slow API answers from Evernote servers itself, though I am not 100% sure. I tested on quite fast laptops and with WIFI or with wired Internet. 

 

Backupery can be found here: https://www.backupery.com/products/backupery-for-evernote/

 

It meets my criteria of a good back-up from my point of view and I can herewith check all of the following boxes, when Backupery is used in an automated script, and movement of resulting encrypted (via GPG in a timed batch file) files to Synology Cloud C2 back-up chain and Dropbox:  (1) []local  (2)[]off-site (3) []offline (4) []duplicate (5) []encrypted (6) []AVG-proof (7) []restored+verified+interval (8) []retention+timeframe (no unique filenames) (9) []full must-have+frequency (10) []incremental if needed as add-on (11) []recovery duration

In addition to that it is easy and trustworthy to set up, rely on as little human actions and reminders as needed. 

This conclusion results from the following report:

Newer machine on wired Internet:

On a newer laptop with wired 1gbit wired Internet (a 2023-03 Microsoft Surface Pro 9 i7 | 32GB |1TB) with Backupery NEW 

1. it needed 1,68 seconds per note
2. which for my 15.000 notes
3. resulted in 10,33GB ENEX files and 5,35GB HTML (that option was enabled)
4. it took 6,9 hours to complete

*) note that I use comma for decimals and point for thousands 😉

 

Older machine on WIFI Internet:

On an older (but good) laptop with WIFI Internet (a 2019-01 Asus RX362F i7, WIFI then 1gbit glassfiber) with  Backupery NEW

1. it needed 2,95 seconds per note
2. which for my 15.000 notes
3. resulted in 10,33GB ENEX and 5,35GB of HTML files (that option was enabled)
4. it took 12,3 hours to complete
5. conclusion: my older laptop via WIFI takes 175% of the time compared to my faster wired Internet laptop.

*) alternative setup measurement: older machine ENEX per notebook, without HTML via WIFI: 1,4 seconds per note, 5,9 hours total

 

I also tried:

*) alternative setup measurement: older machine ENEX per notebook, without HTML with seperate ENEX per note  «⚠️🟡 CHECK

started: 20230513 1423~?~ gestart

 

However, that job stopped with an errormessage: Evernote usage has been temporarily exceeded. Your data will be received in 37 minutes.

 

I guess I have been making too many API calls/back-ups past 24 hours. 😉

[s2sailor 1,169]

On 5/12/2023 at 6:59 AM, gazumped said:

is it simply just a case of downloading some code and installing the process?

Yes, surprisingly and it just worked, no fiddling required.  There are basically three commands.  

init - You run this initially.  It logs you in and creates a local SQL database

sync - this then downloads your EN database to the local copy.  First time is a full download.  Subsequent runs only downloads changes.  The initial download for me with 10k notes took about an hour.

export - this step creates enex files for each notebook.  This runs from the local copy so is pretty quick.  There is no option that I’ve seen for creating only selected notebooks.  It creates then all.

I’ll run the sync fairly often but will export only occasionally.  

[MvdH 63]

16 hours ago, agsteele said:

On Windows I use the standard Task Scheduler and run

E:\evernote-backup.exe sync

every Sunday ...

What I require is also automaticallty cleaning up after 10 local versions, and being ready for my automated Synology C2 Cloud back-up chain and such. It has a retention of 10 years and on a datacenter in another country of Europe.

I am curious, the script/tools you use from GIT, do they have the option to create one ENEX file per note? This helps me restore to older versions in time, when I need that. not must-have, but conveniant in case of disaster. Of course I have the per-note-history of Evernote, but I have experienced lost data or had unsaved/unfound older versions in their system. That's where my question comes from. Maybe you can shine some light on that.

[gazumped 9,819]

Still plugging away here.  I also fed back my initial struggles to Backupery and got a similar response - one point that  I hadn't registered from previous posts here is that Backupery doesn't need Evernote to be installed for it to work - so my largely redundant laptop which only runs Legacy so I have that version of a backup,  could now just run Backupery,  saving me about 30GB of storage space.  It can plug away in a corner,  (hopefully) undistracted by me and undistracting for me,  in a corner of the room.

I was specially incentivised to sort out my backup process because of Evernote's recent note format changes for Sync and RTE which might mean that a backup in Legacy might itself become useless if they decide to stop supporting the old code.

As to progress,  I'm now 3 days in and a guesstimated 53% complete... 

[s2sailor 1,169]

3 hours ago, MvdH said:

I am curious, the script/tools you use from GIT, do they have the option to create one ENEX file per note?

Yes. One of the options is to export to single notes.

[MvdH 63]

16 minutes ago, gazumped said:

Still plugging away here.  I also fed back my initial struggles to Backupery and got a similar response - one point that  I hadn't registered from previous posts here is that Backupery doesn't need Evernote to be installed for it to work - so my largely redundant laptop which only runs Legacy so I have that version of a backup,  could now just run Backupery,  saving me about 30GB of storage space.  It can plug away in a corner,  (hopefully) undistracted by me and undistracting for me,  in a corner of the room.

I was specially incentivised to sort out my backup process because of Evernote's recent note format changes for Sync and RTE which might mean that a backup in Legacy might itself become useless if they decide to stop supporting the old code.

As to progress,  I'm now 3 days in and a guesstimated 53% complete... 

Interesting to read you made the same choice as I did. Back-up was so slow and distracting me and blocking my main laptop to be taken with me, that I use my old laptop to work on back-ups for Evernote. It does that once a week, and I only check once a month if all goes well. Nice that it saves you 30GB of space! I don't envy you on how long a single backup takes on your set-up!

[MvdH 63]

9 minutes ago, s2sailor said:

Yes. One of the options is to export to single notes.

Nice, good (free) alternative for Backupery when people seek that. Does it automatically delete old back-ups (=can you set a certain amount of back-ups you want to keep)?

[s2sailor 1,169]

4 minutes ago, MvdH said:

Does it automatically delete old back-ups (=can you set a certain amount of back-ups you want to keep)?

No.  It just exports files.  We are on our own for cleanup and organization.

[agsteele 1,858]

Evernote-backup creates a local database. You can then schedule an export to the ENEX file(s) at whatever frequency you wish. The ENEX conversion is not required. You can get that to run locally without connecting to the Evernote servers. So you can limit the number of ENEX copies. The local data is updated whenever you run it. So an up to date copy of the data.

With a little effort I think I could create a batch file to today up the ENEX collection on a schedule.

[ianfm 34]

On 5/10/2023 at 7:57 PM, agsteele said:

evernote-backup has a secure direct connection via the API. You login and run the backup. A download of your data is created in a location of your choice on your computer. If you run the command with a different set of commands a set of ENEX files will be created from the data.  You can run the commands on a schedule.  It certainly has a Windows and Mac version and possibly Linux too. It does not require any client application. The program runs on your device so a direct link from your computer to Evernote.

https://github.com/vzhd1701/evernote-backup

Instructions further down the page. Download from the almost hidden direct download link.

Thought I would try evernote-backup, both on Linux and Windows. In both cases nothing works. I didn't find the instructions very helpful either. In Linux all I get when trying to run evernote-backup init-db is command not found (I installed this by extracting the tar.gz file for Debian. In windows when I execute the file there is a quick command window pops up then closes before I have chance to read it! Any tips would be appreciated.

[agsteele 1,858]

I can't answer for the Linux version but for Windows, run the program at a command prompt rather than double clicking on File Explorer. The Window will then stay open.

Once the initial configuration is completed you can then run the automated process detailed above.

[ianfm 34]

On 5/15/2023 at 2:04 PM, agsteele said:

I can't answer for the Linux version but for Windows, run the program at a command prompt rather than double clicking on File Explorer. The Window will then stay open.

Once the initial configuration is completed you can then run the automated process detailed above.

Thanks for that. I got the windows version running using the command prompt as you suggested.

[gazumped 9,819]

Update: Looks like we're going into Day 6 with the Backupery full database backup - now at 21.5GB and counting,  though I did have a small problem yesterday.  I'm connecting through VPN and that app wanted to update and restart.  The app flubbed the update so I had to reinstall it - all the while locked out of wifi by a rogue comms process.  Not sure whether it was Evernote being patient or Backupery being persistent - but when I kicked the VPN back into life my notes started ticking through like nothing happened! (And my heart started beating again!)

[Denis L 35]

Hello! 

Backupery for Evernote developer is here. Thank you for trying our app and for your feedback!

The new application has been released recently and now it does not require Evernote installed and support all Evernote versions (previously it used to rely on the Evernote scripting feature so it required to Evernote Legacy installed on the same machine). Now it connects directly to Evernote and downloads all the selected notebooks. It can also generate HTML exports if necessary.

Please let me explain why the app may work not so fast as expected. The issue is that third-party apps should honor Evernote API rate limits. In other words, Evernote reasonably protects their servers from overload so they put some limits on the API usage. So the application has to do some pauses between requests to Evernote in order not to exceed the number of requests per hour. That is why you can see the "Evernote usage has been temporarily exceeded. Your data will be received in xx minutes" message from time to time. We'll continue working on decreasing the number of API requests to make the export faster. 

Also, perhaps I should note that API rate limits are counted for each separate app. So even if Evernote usage is exceeded for Backupery app, your other Evernote apps continue working as expected without reaching the rate limits.

Please also note that the export process is supposed to survive the app or machine restart so you should not worry about interrupting your export process - the app will try to start the download process from the point where it was left off.

Also, at this moment the application makes full backups only. That means that on each backup it will download all the selected notebooks. We're working on the incremental backups feature so the app will download the changes only, though I don't have any estimations at this moment.

Please let me know if you have any questions or feedback. Any input is highly appreciated!

[gazumped 9,819]

1 hour ago, Denis L said:

Please also note that the export process is supposed to survive the app or machine restart so you should not worry about interrupting your export process - the app will try to start the download process from the point where it was left off.

Have to say, that after 6 days of continuous use,  the app is amazingly transparent to users while working - I haven't noticed any degredation of my device's response whilst also downloading (so far) about 24GB of database.  Plus I had my own brief interruption of connection which the app politely ignored and carried on (living up to my icon here!).  I'll try to keep it going until I get a result - from the size of my Legacy database,  I'm now about 80% completed.

[gazumped 9,819]

Just to round off here...  Backupery has completed!  61,380 notes from 0900 11 May to 1830 19 May - just over 200 hours or 5 seconds per note.  Of course some of those notes will have been edited / added / deleted during that time...

EDIT or maybe 11 seconds per note.  I think I dislocated my maths.  Total folder size is 34.5GB.

[MvdH 63]

9 hours ago, gazumped said:

Just to round off here...  Backupery has completed!  61,380 notes from 0900 11 May to 1830 19 May - just over 200 hours or 5 seconds per note.  Of course some of those notes will have been edited / added / deleted during that time...

EDIT or maybe 11 seconds per note.  I think I dislocated my maths.  Total folder size is 34.5GB.

Thx for your statistics! For you, the incremental backups @Denis L was talking about would really help. 👍

[Denis L 35]

5 hours ago, MvdH said:

Thx for your statistics! For you, the incremental backups @Denis L was talking about would really help. 👍

Absolutely. Having the incremental backup feature will make all the following backups going much faster. I don't have estimations at this moment, but I'll definitely post here if when we add this feature.

[Dave Green 69]

On 5/13/2023 at 11:29 AM, s2sailor said:

As best as I can tell MAC OS doesn't have a utility similar to Task Scheduler, and to accomplish the same is a little more involved, but I'm plugging away at it.  I have noticed that the export option doesn't overwrite and is only all notebooks and not selected ones, but I'm ok with that. 

I use 3rd party program Hazel for lots of things so I use it for this as well.

MacOS has corn and launchd as built-ins if you want to stick with native components.

evernote-backup does incremental backup after the first backup so it is very quick.  It does not appear to deal with the new Evernote Tasks.  I have filed an enhancement request with the developer.

[Dave Green 69]

As part of the evaluation, please try to restore a file with Tasks and two files with Links to each other.  evernote-backup does not presently support either of these new features in the recovered files.

[s2sailor 1,169]

15 minutes ago, Dave Green said:

I use 3rd party program Hazel for lots of things so I use it for this as well.

MacOS has corn and launchd as built-ins if you want to stick with native components.

evernote-backup does incremental backup after the first backup so it is very quick.  It does not appear to deal with the new Evernote Tasks.  I have filed an enhancement request with the developer.

Thanks for the Hazel pointer.  I'll check it out.  I was able to get the AppleScript working and scheduled. I just checked and can confirm that Evernote-backup does not support tasks.  Thanks for filing the request with the developer.

 

[s2sailor 1,169]

Just thinking but the developer may need to wait on an updated API from Evernote that supports these new features.