(Archived) Note History / Sync implementation is poor, at best

[WDanW 3]

Earlier this week I was editing a good deal of important text (list of maintenance steps in a project) in a note. After each important set of changes I ran Sync. At one point I noticed a few lines text were suddenly missing. (This is an unrelated bug that I have occasionally experienced with the Windows EverNote client - I did not delete or overwrite this text). Regardless of how this text was removed, I wanted to review previous versions to recover the missing text.

The Note History, ironically, provided me almost no real history. Looking at this important note, I saw the most recent version and the previous version was from 4 days before. For this note, almost all days had one backup; a very few days had two backups, tops. None of my recent sync'd versions were present.

Where did all the sync'd versions go? They are in the log but not accessible in the history. Why bother providing sync/history functionality if a user can't get to those sync'd versions? I lost important text that I cannot easily recreate. I lost confidence that Evernote is actually saving my work (it certainly was losing my work because of that other bug). I lost hours in recreating the work and then lost a day, waiting for Evernote support to get back to me with the news that they couldn't get my text back. (My client lost confidence in me as the work I was supposed to deliver was 24 hours late as a result of this fiasco). I now will probably will lose many more hours moving over to Google Docs, which actually saves your revisions.

I know that every person values different functionality differently, but if Evernote isn't consistently saving and storing the content you are working on, what's the point of all of these plug-ins, Moleskin partnerships, etc? The most important reason I'm paying for Premium is for History to store my actual content; if that doesn't work I have to move elsewhere.

Can anyone tell me if Evernote plans on fixing it's note history/version functionality to show the actual sync'd versions and not a version we have no control over?

-Dan

[BurgersNFries 2,407]

As has been stated several times on the board, note history is only taken once about every 8 hours. It does not save each & every revision.

[heather 604]

We take Note History snapshots approximately every 8 hours when there is a change in content. It would be extremely odd to only see a version from 4 days prior. Please file a support ticket so we can examine your software logs and see what's happened.

Every sync/change to a note is logged. You can clearly see it in the activity log listed by GUID, so we're usually able to trace things down for you, even find lost content.

[WDanW 3]

Is there any planned change to how this is designed? Once every 8 hours, for me, is not sufficient - and I do not have control over which version is taken.

[jbenson2 2,147]

Evernote does not publish its roadmap.

Sync has worked fine for me.

Because I put my finished work (and my memories) into Evernote, I have seldom, if ever, required the history.

In my opinion, 8 hour capture seems sufficient for the majority of Evernote users.

[dlu 628]

Could you expand on the "unrelated bug that I have occasionally experienced with the Windows EverNote client" that seemed to have started this?

[ronbailey 0]

"Every sync/change to a note is logged. You can clearly see it in the activity log listed by GUID, so we're usually able to trace things down for you, even find lost content."

That sound good, if it works.

I began a note on my iPhone and about 2 hours later I added an extensive note on my iPad. I wanted to be sure it was safe so I synced at the settings screen. However the second, most important note is nowhere to be seen. All I have is the heading I created in my iPhone version. This was one of my main reasons for going Premium so I am pretty disappointed with it so far. This is the first time I have needed the facility to retrace my note history and Evernote has failed. Can you really trace my lost notes?

[Metrodon 2,184]

Have you opened a support case?

[BurgersNFries 2,407]

"Every sync/change to a note is logged. You can clearly see it in the activity log listed by GUID, so we're usually able to trace things down for you, even find lost content."

That sound good, if it works.

I began a note on my iPhone and about 2 hours later I added an extensive note on my iPad. I wanted to be sure it was safe so I synced at the settings screen. However the second, most important note is nowhere to be seen. All I have is the heading I created in my iPhone version. This was one of my main reasons for going Premium so I am pretty disappointed with it so far. This is the first time I have needed the facility to retrace my note history and Evernote has failed. Can you really trace my lost notes?

If you're going to be switching between devices/computers, it's important to understand how Evernote works.

[ronbailey 0]

No, I haven't opened a support case.

I suspect that what I did was sync from the iPhone settings and then, a couple of hours later, I synced from the same note on my iPad. Later in the evening I checked my iPhone and the updated note was not there. I then synced again from my iPhone and when I next looked at my iPad the note had reverted to the second iPhone sync and thereby eliminated by full notes from the iPad. I have checked the Web version and there is only one date in the history which seems to be the last time I synced from the iPhone.

I have read the suggested syncing discussion but it is all a bit complex for simpler souls who just wanted to use the app. I think there ought to have been some warning that I was about to overwrite my fuller but not more recent version on the iPad.

[jefito 5,589]

Simple analog to Evernote's syncing approach:

Your kid is designated as the person in your family who must go to the grocery store. Now, your kid is very bright (aren't they all?), but she has one drawback -- she can deal with one and only one shopping list at a time. So your wife gives the kid a shopping list in the morning, and as she's noticed that her beloved Ovaltine has out, she's made a point of adding it to her shopping list, which otherwise contains a common set of foodstuffs. Later on, and unaware of your wife's prior list (what, your kid is going to remember to tell you?), you hand your kid your own list, which includes that jar of Marmite that you want to give as a present to your British co-worker who helped you out earlier in the day. Later on, the kid goes to the store, drops the groceries on the counter and waltzes off to do whatever the youth of this country do with their free time. You're off in the den watching some old Rockford re-runs, and you suddenly become aware of a load wailing in the kitchen. You rush out, and it's your wife, wondering where her Ovaltine is and what she's going to do with a jar of Marmite, which as far as she's concerned is a palatable and nutritious as a container of rat poison.

So what happened? The answer is that your kid, as brilliant and talented as she is, just fulfilled the last grocery list she was handed, and therefore missed out on the Ovaltine. So it is with Evernote. If you edit a note on one device, and sync it to the cloud, and then go to edit the same note on a different device -- without having synced Evernote on that device -- you will wind up editing a version of the note which is not current, and when that note is synced to the Evernote servers, it overwrites the other edited version of the note from the first device (theoretically you get a note conflict, but you might not notice).

If you are going to edit Evernote notes on different devices, you must be aware of this, otherwise you face the potential problem of having your edits lost.

[Metrodon 2,184]

But the kid got the marmite? That is the important bit of the story isn't it?

[BurgersNFries 2,407]

But the kid got the marmite? That is the important bit of the story isn't it?

That's the real reason behind the loud wailing from the kitchen. Not the lack of Ovaltine.

[jefito 5,589]

But the kid got the marmite? That is the important bit of the story isn't it?

That's the real reason behind the loud wailing from the kitchen. Not the lack of Ovaltine.

Yes. And yes.

[Sugeeth Krish 475]

I just faced this issue, and am surprised for all that is publicized about the goodness of Evernote, and the features that inspired me to go premium on Evernote, today was the first time, i got disapointed. Evernote clearly needs to make this bug public or atleast find a fix to it.

And all those die hard Evernote fans, we all love Evernote all right , but let us not over see its flaws or make lame jokes about someone's loss. Let us criticize Evernote so that our greatest organization software continues to evolve and improve..

[GrumpyMonkey 4,319]

I just faced this issue, and am surprised for all that is publicized about the goodness of Evernote, and the features that inspired me to go premium on Evernote, today was the first time, i got disapointed. Evernote clearly needs to make this bug public or atleast find a fix to it.

And all those die hard Evernote fans, we all love Evernote all right , but let us not over see its flaws or make lame jokes about someone's loss. Let us criticize Evernote so that our greatest organization software continues to evolve and improve..

Hi. You are correct, that any loss of data is a serious issue. However, we also have to recognize that the issue is not a bug (as far as I can tell from the posts so far). It is user error.

Like any cloud service, Evernote syncs back to a central server, and generally picks the newest copy of a note. It doesn't read the content of a note and decide whether one ought to be synced and another ought not to be synced. Dropbox, iCloud, and every other service I know works exactly the same.

Does that mean it is perfect or easy to do? No. I am sure there is room for improvement, and I have reported issues and suggestions for improvement in the past on these forums. The key with any service is to become familiar with it, especially within your workflow, and (in my opinion) it is important to keep your own backups just in case. The cloud is relatively new territory, and we are all struggling to figure out the best way to deal with it. At least for now, the old advice to save, save, and save holds true.

Also, please see Heather's response above for Evernote's attitude about this. There is a way that Evernote ought to work, there is a system for reporting if it does not, and it is always possible that data can be retrieved.

"We take Note History snapshots approximately every 8 hours when there is a change in content. It would be extremely odd to only see a version from 4 days prior. Please file a support ticket so we can examine your software logs and see what's happened.

Every sync/change to a note is logged. You can clearly see it in the activity log listed by GUID, so we're usually able to trace things down for you, even find lost content. "

[BurgersNFries 2,407]

but let us not over see its flaws or make lame jokes about someone's loss.

First, no lame jokes were made about anyone's data loss.

Second, what GM says is correct...

However, we also have to recognize that the issue is not a bug (as far as I can tell from the posts so far). It is user error.

Third...

If you're going to be switching between devices/computers, it's important to understand how Evernote works.

http://discussion.ev...ow/#entry101095

It's a very simple concept. In a nutshell,

1. after making changes on one device/computer, sync the changes UP.
   
2. After switching to another device/computer, sync the changes DOWN before editing & go back to step1.
   

[WDanW 3]

The line between bug / correct application design / user error is a very narrow one. Earlier I said "If Evernote isn't consistently saving and storing the content you are working on, what's the point of all of these plug-ins..." and I stand by this. The singular most important focus of EverNote should be to store your notes. It should be rock-solid - it should never lose your work. All other functionality is gravy. The sync implementation is poor; this simplistic design has led to people losing their work.

I've been writing software for well over 20 years and have worked with many version control systems and there are plenty of ways of preventing work from being lost in a sync. Syncing content between systems is not a new concept; it existed long before "the cloud" came along. Hell, I've done this type of work on a bunch of systems; if I ever told one of my clients (especially the banks) that their data was lost because of my poor design I'd be looking at a lawsuit.

Please search the EverNote discussion boards for the word "lost", there are 10 pages of complaints for a variety of reasons. If you are paying for a service to record your notes it should record your notes properly with no loss, ever.

[GrumpyMonkey 4,319]

The line between bug / correct application design / user error is a very narrow one. Earlier I said "If Evernote isn't consistently saving and storing the content you are working on, what's the point of all of these plug-ins..." and I stand by this. The singular most important focus of EverNote should be to store your notes. It should be rock-solid - it should never lose your work. All other functionality is gravy. The sync implementation is poor; this simplistic design has led to people losing their work.

I've been writing software for well over 20 years and have worked with many version control systems and there are plenty of ways of preventing work from being lost in a sync. Syncing content between systems is not a new concept; it existed long before "the cloud" came along. Hell, I've done this type of work on a bunch of systems; if I ever told one of my clients (especially the banks) that their data was lost because of my poor design I'd be looking at a lawsuit.

Please search the EverNote discussion boards for the word "lost", there are 10 pages of complaints for a variety of reasons. If you are paying for a service to record your notes it should record your notes properly with no loss, ever.

Hi. As someone who has made those "lost" posts and has recorded lost data in the past, I agree that this happens, even though it shouldn't. Sometimes, it is the result of a bug. Sometimes, it is the result of user error. I am happy to call a bug a bug and get Evernote's attention to a problem.

However, there are currently no bugs that I know of for syncing, and this thread appears (to my eyes at least) to be about user error. I do not have your expertise in computers or designing systems, but I do have experience with the cloud, and the process outlined above for syncing is used by all of the other major services as well. Evernote has a longer period between syncs than some (a minimum setting of 5 minutes), but you can manually sync anytime. It can be tricky if you are using several devices at once (I have my Mac, iPad, and iPhone out in front of me now), but if you are careful and follow basic procedures (outlined above by BNF), then you ought to be fine. These are the same procedures you need to follow with any cloud service.

Could you be more specific about where Evernote sync implementation is poor, and what is not rock-solid with the service? That might help fellow users and Evernote developers understand a little better.

[WDanW 3]

My particular bug was with EverNote not providing full access to the history of versions and providing no control over which version within that 8 hour period get permanently saved. In every source/version control and content management system (CMS) that I've worked with (and EverNote is essentially a CMS), you have at least three basic actions:

- User submits changed content;

- Service records the version change;

- Service provides access to all previous versions and current version.

Depending on the rules of the CMS or the administrator, it might not store every version forever - maybe it only keeps the last 20 versions. Maybe it purges out versions older than 6 months or a year. But if you sync a version that exact version is stored and is available for later review, not a random version chosen by the server within an 8 hour period.

Looking at ronbailey's posts, I'm guessing content was edited locally on a out-of-sync client and that content was lost when the next sync occurred. There are plenty of ways of detecting and handling this. If you are looking for the basic design of a system that does this, well there are many - all major CMSs, wikis, source control systems, etc. If you want a specific server-based design then check out Subversion

This is not to say EverNote should expose the level complexity of Subversion - it definitely should not! But if, during sync, the EverNote client detects a local change on a out-of-date note, it should provide many visible warnings, the functionality to merge the changes would be nice and, if nothing else, a quick backup of the original made to a new note on the client and alert the user to check the note and make sure nothing is lost. For someone not in the software development field this might sound complex but for anyone who does this for a living (including the engineers at EverNote) this is real old technology.

Please don't get me wrong, I like EverNote a lot. I tested a number of different tools before choosing it. And I don't mean to insult the community or the engineers... but this functionality does not work the way that one would expect it to work.

[jefito 5,589]

Theoretically, as previously stated, Evernote should be able to detect conflicts and tell you about them (it puts them into a "Conflicting Changes" notebook, the several times it's happened to me). Then it's up to the user to merge changes or whatever (and I'm OK with that; it's a case of "my bad"). Still not sure what happened here, the details as related by posters here are too sketchy to tell, and I've not see this type of data loss myself.

[BurgersNFries 2,407]

I've been writing software for well over 20 years

Join the crowd. Some of us have over 30 years under our belts. So yah, we're a pretty tech savvy users group here.

but this functionality does not work the way that one would expect it to work.

EN is not Dropbox. My mantra to new users is that the EN servers are the source of all truths. I don't think that's all that different from a lot of computer systems. You have one place with the current/live/real/active data. EN was designed not as a collaborative tool but as a personal tool. They may be outgrowing that but that's how it started & that's how it pretty much still is. So although it may not work as one "would expect it to work", it works. But if you don't fully understand how it works & follow that path, then yah, you're going to probably experience data loss, since EN doesn't know if the most recently updated data is the right data or not. Just like any other computer system. That whole GIGO thing.

[Sugeeth Krish 475]

The issue at times, can even be that of a slow upload internet connection. So, if i make a change on one device and before it gets uploaded fully, and i have to make another change on the other device, evernote should be able to retain the first content in its cache and ensure that it is not lost. So, that when both devices are eventually able to upload, it should merge both the posts.

[JuanO 5]

It sounds like, to me at least, that this is an issue with "mappings" (to quote Don Norman). When I click on NOTE HISTORY a pop up shows up that says "Several times each day, Evernote makes a copy of any notes in your account that have changed since the last time the system checked."

To me this sounds like a snapshot is taken fairly constantly - more than every 8 hours. To WDanW this sounds like more constant than 8 hours a day. We can link and explain how Evernote works all day long, but the root of the problem is how the interface is explained to the user as they are using it. I would suggest a note under Note History that will show when the next snapshot is scheduled to occur if the note changes, as well as changing the text of the explanation.

I don't rely on the history feature much except as a backup. I use Dropbox for anything that needs to be versioned, i.e. code, etc. (Even IF Evernote took more frequent snapshots, I would still use Dropbox for code for obvious reasons.)

Understanding how the service works is important, I agree. But the way this feature is implemented is an opportunity for improvement. Which I think is WDanW's point.

[John in Michigan USA 129]

This is an important discussion!

My two cents and some useful terms/acronyms:

1) Data MUST NOT be lost. @WDanW (et. al.) is right when he says, "for anyone who does this for a living (including the engineers at EverNote) this is real old technology"

I am a former SQL database administrator. The technology to ensure against "silent" (i.e. no error message) data loss has been out there for a LONG time.

2) Replication or Save conflicts MUST be resolved intelligently (usually, this means informing the user; some try to do a smart merge but the unmerged, conflicting nodes are always there in the history)

To show how basic this is, check out any of these technologies:

Lotus Notes - RSC is their useful acronym for "Replication or Save Conflicts". It is defined here:

http://publib.boulde...onflicts_t.html

They've had this solved since about 1995.

Oracle's SQL * Net (home of the two-phase commit) or any RDBMS that has distributed transactions. Oracle has had it since c. 1992.

http://www.coolinter...interview/3394/

http://www.coolinter...nterview/43603/

http://www.coolinter...nterview/47738/

Read about Journaling File Systems. Unix has had this forever. Windows finally added a lite version of it in Win 7 (or possibly Vista...I'll never know :-)

Any of these technologies will accomplish what your users have a right to expect...either the save operation fails or the system notifies you there are conflicting versions of the node...never ever should it silently lose data.

[dlu 628]

This is an important discussion!

My two cents and some useful terms/acronyms:

1) Data MUST NOT be lost. @WDanW (et. al.) is right when he says, "for anyone who does this for a living (including the engineers at EverNote) this is real old technology"

I am a former SQL database administrator. The technology to ensure against "silent" (i.e. no error message) data loss has been out there for a LONG time.

2) Replication or Save conflicts MUST be resolved intelligently (usually, this means informing the user; some try to do a smart merge but the unmerged, conflicting nodes are always there in the history)

To show how basic this is, check out any of these technologies:

Lotus Notes - RSC is their useful acronym for "Replication or Save Conflicts". It is defined here:

http://publib.boulde...onflicts_t.html

They've had this solved since about 1995.

Oracle's SQL * Net (home of the two-phase commit) or any RDBMS that has distributed transactions. Oracle has had it since c. 1992.

http://www.coolinter...interview/3394/

http://www.coolinter...nterview/43603/

http://www.coolinter...nterview/47738/

Read about Journaling File Systems. Unix has had this forever. Windows finally added a lite version of it in Win 7 (or possibly Vista...I'll never know :-)

Any of these technologies will accomplish what your users have a right to expect...either the save operation fails or the system notifies you there are conflicting versions of the node...never ever should it silently lose data.

I thought the main problem was granularity of the versions in Note History?

[Brad inMinnesota 3]

Can I suggest that Evernote folk take a look at Google drive history?  Or perhaps look at most any wiki out there.  There is a revision history that can be pulled up for EACH and EVERY save/sync.

 

A simple 'Undo' command would solve 99% of problems.

 

The new 'checkbox' feature is what killed me.   I assumed it worked like the bullet and numbered list..  it does NOT!

I selected the lines that i wanted to have checkboxes.  Then selected checkbox.  And it DELETED the text and replaced it with a single, naked checkbox.   And no UNDO!  (using my iPod/Apple iOS device).

 

And I was almost going to pay for the Premium as this is something that seemed to solve my problem in 'restoring' my deletions.  Guess I don't have to go ask for my money back as the threads here outline what it really does('t) do.

 

And I am liking many/most other things about Evernote.  Hope there is openness to improve the product.

[BurgersNFries 2,407]

Hope there is openness to improve the product.

 

Of course EN is open to improving the product.  They always have been.  Although feature requests/improvements are always welcomed by EN, just because they are not implemented or are not implemented soon, doesn't mean they are not open to improving the product.  It just means they may choose to not add that feature request or else have prioritized other things ahead of it.

[Sugeeth Krish 475]

Hope there is openness to improve the product.

 

Of course EN is open to improving the product.  They always have been.  Although feature requests/improvements are always welcomed by EN, just because they are not implemented or are not implemented soon, doesn't mean they are not open to improving the product.  It just means they may choose to not add that feature request or else have prioritized other things ahead of it.

 

As much as i try not to crib and cry about EN's drawbacks here, i literally kind of am forced to. I mean this crazy. If there is no un-do button in a mobile client, this is asking for disaster. I am literally shocked to know this. And yeah, We've heard that a million times, Burgers. "It just means they may choose to not add that feature request or else have prioritized other things ahead of it."

 

I can show you hundreds of feature requests that have been lying in the forum for more than 3-4 years, and these features are already present across other services. I can accept lack of some of these features because i know that EN has +features that other services don't. But seriously, there are so many loop holes for users to lose data in EN, and that for me is unacceptable. EN needs to address these issues first.