Feature request: Play more nicely with online backup software

[Julian 40]

I like to keep my data as safe as possible so, just in case there was ever some catastrophic issue at Evernote's data centre, I also have my local Evernote cache folder in my regular backup set which means that it gets backed up to various local destinations and also to my online backup services (Crashplan and Mozy).

 

Unfortunately however the local cache folder seems to be structured such that all the actual note data is contained in a single "...\Databases\<username>.exb" file that is currently 934MB for my account. Whenever I make even a modest addition to my Evernote account such as adding a brief text note the .exb file is changed and so my online backup software sends the entire almost 1GB file up to the backup server.

 

Would it be practical and would you (Evernote) ever consider restructuring the layout of the data in the local cache directory (a change that would be 100% invisible to users) so that the local <username>.exb file was broken into smaller chunks, maybe by replacing the <username>.exb file with a "<username>Exb" directory that contained "01.exb", "02.exb" etc files so that each individual .exb file was kept below a certain size and backup services would see less data changing when the user made even modest changes to his or her notes?

 

Edit: Another option that just occurred to me that might be simpler to implement would be to keep the single "<username>.exb" file but have it only contain unique references to attachments rather than the actual data for the attachments. The actual attachments could then be stored in separate file chunks within a "<username" Attachments" sub folder. I suspect that would keep the main .exb file down to a quite manageable size and simple edits might not touch any attachments at all and, if they did, would probably only affect a single file within the attachments folder. Since an attachment can be up to 100MB in size a maximum file size within the attachments folder of 100MB would seem to make sense.

 

- Julian

[gazumped 11,656]

I totally support the idea of backups,  but I suspect Evernote have too much invested in the single database file architecture to make any changes in the forseeable future.  I agree the single file becomes a bit of a monster (mine's now 12GB+) and if there are alternatives I will welcome them.  But this is the kind of development that Evernote tend not to discuss,  and would almost certainly not be willing to give any dates for.  In the meantime it might be sensible to exclude Evernote from your main backup systems and concentrate on a daily copy archived somewhere safe.

[BurgersNFries 2,407]

I agree with Gaz that it's extremely unlikely EN would break up the exb file into multiple files. I would suggest you remove it from your backup that syncs each change & instead, do a daily or weekly backup. Also, since EN syncs on a regular basis, you would mostly be concerned about your local/non-synced notebooks. Because if you do a nightly backup of your exb file & keep multiple versions, the chances of Evernote's servers having a catastrophic melt down & EN not having a good & very recent backup to revert to (b/c the do mirroring) and causing users to lose any changes done during a specific window of time coinciding with you not having an up to the minute backup are possible, but pretty slim.

[Julian 40]

I agree with Gaz that it's extremely unlikely EN would break up the exb file into multiple files. I would suggest you remove it from your backup that syncs each change & instead, do a daily or weekly backup.

 

I agree with Gaz as well although, as he himself says, "I agree the single file becomes a bit of a monster (mine's now 12GB+) and if there are alternatives I will welcome them" so there's never any harm in at least asking.

 

I think you're both right, for now I'll need to remove my Evernote database from my backup set. I currently have Evernote set to use a local folder on my data (D) drive (which is actually two mirrored drives). The entire contents of my D drive is what gets the full backup treatment. If I move the database back to its default location ("...\<user>\Application Data\Evernote" I think, I'll have to check) that will take it out of the scope of my backups.

 

What I might do is leave the "D:\Application Data\Evernote" folder location on my D drive but, since Evernote will have been reset to store its cache folder on my C: drive, the folder location on my D drive wouldn't be having any files updated as I used Evernote during the day so wouldn't generate any additional backup activity. I could then set up a scheduled task in Windows to copy the contents of the live Evernote cache folder from my C: drive across to the D: drive location either every night or maybe once a week at which point my Mozy and Crashplan services would see changed files and schedule them for transmission to the servers. This way I still get automated multi-destination backup of at least a reasonably up to date copy of all my Evernote notes but the big and tedious upload will only happen overnight as I'm asleep so shouldn't be too much of an issue.

 

Then again, if I ever get to Gaz's monster 12+GB file size then even the strategy above is going to be painfull for my 1 Mbs upload connection. Still, it might keep me going for a while.

 

I suppose another option might be to restructure my Evernote data since one reason for the big explosion in my file size was that there was a load of old stuff on my PC file system from things that I was interested in years ago. Rather than clutter up my day to day PC working environment I decided to archive this stuff to my Evernote account (the strap line is "remember everything" after all). Maybe I could sign up for a second free Evernote account and move the notes containing archived stuff there, possibly making it all available within a single search space by sharing the notebooks in my Evernote archive account with my primary Evernote account. This way the database file for my primary account would become much smaller and more manageable. I'll have to give that option a bit more thought.

 

Thanks for the comments, it's been a useful spur to get me thinking about a few solutions.

 

- Julian