(ALMOST) FIXED: 10/25/2024 (They made things right for the downtime too!) -- Is Evernote having some issues right now? 10/8/2024 (Update: This is completely a me problem, but also an Evernote server side problem)

[Jon/t 1,690]

23 minutes ago, mackid1993 said:

@Jon/t can you believe these answers?

Understand. Not an excuse, just an understanding but there's been a sync issue over the last few days and I would imagine a bunch of folk especially on the back-end team have been pulled off other stuff to get it fixed fast and it is fixed now.

I'm surprised its gone on this long. I would say more complicated/lengthy to fixed rather than any sort of incompetence. Everyone I've spoken to is pretty switched on and don't seem to mind hard work.

I really hope it gets sorted quickly for you.

[s2sailor 2,486]

They should definitely slow down development and work on the list of open problems. One way of triaging problems is to first work on those affecting the most people, with consideration to severity.

@mackid1993 if your issue is mainly affecting you, that unfortunately keeps you down on the list.

[PinkElephant 8,914]

Every larger problem starts with a single case ...

I think the engineers should know by now what sort of problem they encountered here. With what little influence we users have on the clients, and even less on the backend, it should virtually be impossible for a user to individually cause a situations as the one encountered here.

This means from my view that everything that has happened in this case can happen to anybody else as well.

[mackid1993 1,379]

I also wonder if this is a very simplified highly glossed over description of the problem I'm having.

  

[AlbertR 747]

25 minutes ago, Jon/t said:

there's been a sync issue over the last few days

Unfortunately it's not only a sync issue where data from clients have to be synced with the server(s).

In earlier times, clients did their job with no problem when offline: If I added a Tag to a note, the note was immediately available by searching with "tag:..."

Now it seems that all searches are based on a working online connection: searches that contain "tag:..." or "updated:..." either do not work (no answer) or work incompletely (not based on last changes) 😡.

For me EN10 is completely unusable this time 😤.
Even working completely offline is not possible... 

 

 

[mackid1993 1,379]

Finally! Good news!!

[PinkElephant 8,914]

"Failing Backend Infrastructure Component" ... makes me creepy !

[s2sailor 2,486]

Sounds like a failing shard which would explain the limited scope of the problem.

[mackid1993 1,379]

Just now, s2sailor said:

Sounds like a failing shard which would explain the limited scope of the problem.

Exactly. Good thing I take backups. Although I think shards are replicated.

[PinkElephant 8,914]

Just now, mackid1993 said:

Exactly. Good thing I take backups. Although I think shards are replicated.

Which is why I'm creepy. 

Usually you switch to an alternate, and the user won't even notice some piece of silicon is now a brick (or better a pebble). If there is a (working) alternate ...

[Jon/t 1,690]

3 minutes ago, mackid1993 said:

Although I think shards are replicated.

I think so... remembering back to 2017/18 when they moved to Google cloud, I think one of the benefits was automatic duplication of data to prevent issues like this.

One thing I've realised since becoming an EN expert is the software is way more complicated than it should be and one of Spoons big jobs is simplifying it. I suppose that's what they call technical debt.

[mackid1993 1,379]

Not to repeat post but I got this again and it made me laugh:

[mackid1993 1,379]

No word from support, but suddenly it started working... and this isn't usually when it would work. It's been going for like a half hour now normally. I don't want to say they fixed it until I hear from support though and jinx it.

[mackid1993 1,379]

Wow, support just gave me another 3 months of Evernote for free.

[PinkElephant 8,914]

Free EN is for everybody, don’t you think 🤣

[mackid1993 1,379]

37 minutes ago, PinkElephant said:

Free EN is for everybody, don’t you think 🤣

Well I guess if your shard sharts in its pants that's the least they can do!

[chilee 32]

I'm having this issue as well, and is getting worse. Before the problem was a delay in loading the first note in a tag--now it's all the notes. I'm losing my train of thought waiting for notes to load--and they are small notes. Not a good thing for note-keeping software. I never used to have all these problems!

***

10/25: The latency issue is everywhere on the interface, such as slow loading of the toolbar. Latency also on the app. Yesterday I created a note remotely, and the note can't be accessed. I had to recreate it. Every time I go to update a note I have to wait until it loads--sometimes minutes! What seems to solve the problem is selecting another note--then the note i want loads.

[AlbertR 747]

New day - new luck? My syncing problem seem to be away. With one exception? Don't know whether it came with the sync probs or with 10.112.2. I've never seen it before:

UpdateDate of notes is not visible in result of searches: Whereas updated notes show correct in notebooks, their UpdateDates are grayed on when listed as search results:

Web-App (now on 10.112.3) is OK - So I'll wait for Win-App 10.112.>2 before adding a support incident 😉

[Jon/t 1,690]

28 minutes ago, AlbertR said:

New day - new luck? My syncing problem seem to be away. With one exception? Don't know whether it came with the sync probs or with 10.112.2. I've never seen it before:

UpdateDate of notes is not visible in result of searches: Whereas updated notes show correct in notebooks, their UpdateDates are grayed on when listed as search results:

Web-App (now on 10.112.3) is OK - So I'll wait for Win-App 10.112.>2 before adding a support incident 😉

The syncing delay should be fixed now.

I've replicated the greyed out updated date issue. Only seems to happen in top or side view. Snippets and card views are OK. I've mentioned it.

[bradsayers 31]

On 10/10/2024 at 4:12 PM, mackid1993 said:

Maybe they are doing something... instead of constant 503's I'm seeing this now quite a bit.

I have been getting this on the web versions from three browsers. Have run through all the obvious steps and still get it. I run my business off evernote, so this is very alarming, since the problem appeared 4 days ago. Submitted a ticket then and getting AI responses. Not good. Already an Obsidian user, am thinking of moving mission critical content over there.

[mackid1993 1,379]

Some good news. Less scary than what @PinkElephant was afraid of. Now I monitor for a few days. Evidentially it was my shard like we thought. Oldest reply is on the bottom. Newest reply with the explanation is on the top.

[ForestD 1,542]

3 hours ago, bradsayers said:

I have been getting this on the web versions from three browsers. Have run through all the obvious steps and still get it. I run my business off evernote, so this is very alarming, since the problem appeared 4 days ago. Submitted a ticket then and getting AI responses. Not good. Already an Obsidian user, am thinking of moving mission critical content over there.

I've already moved to another solution for mission critical content. I still love Evernote and use it a little, but it's not stable enough for me at the moment to be my go-to. This thread is an example of several similar issues that I have faced over the last year with it.

[AlbertR 747]

44 minutes ago, mackid1993 said:

Some good news.

Fine. May I summarize? <envious>

• You get premium support (including contact to devs)
• You got discount on your account
• You have personally tuned server access

</envious>

But THX! You effort seems to help us all 🙂

[mackid1993 1,379]

11 minutes ago, AlbertR said:

Fine. May I summarize? <envious>

• You get premium support (including contact to devs)
• You got discount on your account
• You have personally tuned server access

</envious>

But THX! You effort seems to help us all 🙂

With all due respect, I literally spent weeks trashing them on Twitter until they finally listened to me. I also sent dozens of logs screenshots and finally a screen recording that got their attention. 

[tired... 683]

Thanks for your advice. I guess I also need to get far more persistent and direct, and also active on more platforms, to get the bugs and other problems solved.

[AlbertR 747]

11 hours ago, Jon/t said:

I've replicated the greyed out updated date issue.

FYI: Has gone away without a client update 👍- so it must have been a server problem? 🤔

BTW: For the moment EN-App seems to be much faster - also without a client update 🙂
EN-Web has been updated to 10.112.4 - and feels even faster than EN-App. Great!

... Or have I got new server connected, too? 😉 

 

[s2sailor 2,486]

1 hour ago, mackid1993 said:

With all due respect, I literally spent weeks trashing them on Twitter until they finally listened to me. I also sent dozens of logs screenshots and finally a screen recording that got their attention. 

I think this validates the squeaky wheel approach 😀.

[mackid1993 1,379]

Seems like they may still have to monitor things. From 2:40 to 2:50 PM CPU usage on my shard spiked to 97%. She is trying to figure out now if it's my account or my shard. So it's better, but they are still monitoring and looking into this.

[mackid1993 1,379]

This is an interesting find on the engineer's part:

She's working at midnight, so hopefully I'll hear from her tomorrow. I wonder why my client is calling deprecated APIs. The plot thickens. Say what you will about the direction of Bending Spoons and Evernote but it's almost midnight in Italy on a Friday and this engineer is still working--- on a problem for one guy. They have some damn dedication to their jobs.

[VincentC 361]

Also, I suspect when the boss says, "I would like you to see if you can fix this problem for this guy," it provides some additional motivation.

Also, given the size of Evernote's US and western hemisphere user base, it wouldn't surprise to learn that they had some people working shifts during US business hours.  I'd be surprised if they didn't, actually.  And Evernote's not their only significant software service, so US hours (or western hemisphere hours to be less egotistical) may be baked into their normal HR strategy across the company.

Glad they're making progress with you.

 

Vinnie

[mackid1993 1,379]

Just now, VincentC said:

Also, I suspect when the boss says, "I would like you to see if you can fix this problem for this guy," it provides some additional motivation.

Also, given the size of Evernote's US and western hemisphere user base, it wouldn't surprise to learn that they had some people working shifts during US business hours.  I'd be surprised if they didn't, actually.  And Evernote's not their only significant software service, so US hours (or western hemisphere hours to be less egotistical) may be baked into their normal HR strategy across the company.

Glad they're making progress with you.

 

Vinnie

As am I. She just got back to me to ask for some more info. She's going to let me know once they figure out what is going on.

[mackid1993 1,379]

4 hours ago, tired... said:

Thanks for your advice. I guess I also need to get far more persistent and direct, and also active on more platforms, to get the bugs and other problems solved.

If you don't make your voice heard, how can you expect to get anything done? They have 100 million users each with dozens of opinions. A lot of noise gets drowned out. It doesn't help that my issue was confused for a different bug that @Jon/t observed with client performance that was quickly fixed. That delayed me getting help by an entire week due to the confusion. I blasted them on Twitter nearly every day multiple times. I tagged them on Reddit directly as well, they have a support account on Reddit and Federico has an account as well. Two different Experts also were kind enough to flag this issue for me. Finally Federico responded to me one day on Twitter and a few days later I had a much higher up customer service person checking in with me every 2-3 days. Then yesterday they escalated the priority and today I heard from an engineer, we've been going back and forth all day, she's been asking me questions and collecting logs from me while she looks into things on her end. I even asked if they could give me a bit of free service for my trouble without specifying time and they gave me 3 months.

It also doesn't help that they just bought this company 2 years ago and didn't build it from the ground up, they took it on with 100 million users and had to rebuild it from scratch, so there's a disconnect and some inefficiencies between engineering and support that based on this thread we can clearly see are indeed improving with time. I'd say give them another year to refine their processes further and we'll start to see things become more efficient. It's all about having a positive outlook on things and life but also sticking up for yourself when you need to. We are a far cry from December 2023 where it would take 3 weeks to get a reply on a billing question. I needed sales tax refunded once and I waited nearly a month.

[mackid1993 1,379]

Woah! It's 2AM in Milan.

[mackid1993 1,379]

Pretty interesting, randomly it seems like activity on my shard is spiking I've been going back and forth with Daniela. She is looking into why my client is calling these deprecated APIs. Hopefully she finds a solution soon, but I'm hopeful now.

[eric99 1,083]

2 hours ago, mackid1993 said:

 She is looking into why my client is calling these deprecated APIs.

Maybe your Python Evernote backup? I don't use it myself, is that running in the background?

[PinkElephant 8,914]

The backup job is not running in the background, unless you create a task that executes it.

It is a command line tool. It’s pretty fast, because it saves incrementally only.

Yes, it may access the old API. But I doubt EN engineers could take it for the client by mistake.

[mackid1993 1,379]

17 minutes ago, eric99 said:

Maybe your Python Evernote backup? I don't use it myself, is that running in the background?

Seems like it was the webclipper. Somehow I had a few hundred sessions authorized. She ran a script to clear them out. Not sure how that could have happened.

[LaFlamme 91]

12 minutes ago, mackid1993 said:

Seems like it was the webclipper. Somehow I had a few hundred sessions authorized. She ran a script to clear them out. Not sure how that could have happened.

I'm still monitoring this thread. You guys have been into some heavy geeking. That webclipper situation is bizarre. All of this is making me think more of alternatives. I've been good for a couple weeks now on EN web but for how long? Plus, it would be nice to have a desktop client again.

[eric99 1,083]

20 minutes ago, mackid1993 said:

Seems like it was the webclipper. Somehow I had a few hundred sessions authorized. She ran a script to clear them out. Not sure how that could have happened.

uh, what does that mean: "a few hundred sessions authorized"? What are these sessions doing to the api when you're not using the webclipper?

[PinkElephant 8,914]

25 minutes ago, mackid1993 said:

Seems like it was the webclipper. Somehow I had a few hundred sessions authorized. She ran a script to clear them out. Not sure how that could have happened.

Do you close your browser tabs, or just let them hang around case you need them again ?

I would expect that closing the tab terminates all pending sessions.

[eric99 1,083]

1 minute ago, PinkElephant said:

Do you close your browser tabs, or just let them hang around case you need them again ?

I would expect that closing the tab terminates all pending sessions.

I would expect that closing the webclipper would terminate pending sessions as well

 

[mackid1993 1,379]

I have no idea. She killed them all and they started duplicating again. I think it might have something to do with sync in Vivaldi. She's going to look into it. She told me just to remove the web clipper for now until they can do some research into it. I legit had a couple hundred sessions though. 

[eric99 1,083]

8 minutes ago, mackid1993 said:

I have no idea. She killed them all and they started duplicating again. I think it might have something to do with sync in Vivaldi. She's going to look into it. She told me just to remove the web clipper for now until they can do some research into it. I legit had a couple hundred sessions though. 

So it's a webclipper or a vivaldi bug?

[mackid1993 1,379]

Just now, eric99 said:

So it's a webclipper or a vivaldi bug?

That's the question. The web clipper shouldn't let Vivaldi sync its session data. That's a security issue. 

[mackid1993 1,379]

I think what was happening was for some reason I had hundreds of webclipper sessions using their deprecated login API. She cleared them out. I just removed my webclipper extension and readded it. Now I can see when I sign out of the Evernote webapp I'm signed out of the webclipper which is how it should work now. I'm currently signed in on two browsers so I have two webclipers signed in. Evernote backup was also using the old API. I singed back into that using evernote-backup reauth --oauth.

[mackid1993 1,379]

She actually made some more config changes and restarted my shard a little while ago too. Hopefully fingers crossed Monday morning this'll finally be solved.

[mackid1993 1,379]

My webclippers replicated themselves again last night...

[eric99 1,083]

2 hours ago, mackid1993 said:

My webclippers replicated themselves again last night...

 

[eric99 1,083]

Where can you see this and can you remove these replicas yourself? Are you still on Vivaldi or is it also on a mainstream browser?

 

 

 

[mackid1993 1,379]

1 minute ago, eric99 said:

Where can you see this and can you remove these replicas yourself? Are you still on Vivaldi or is it also on a mainstream browser?

 

 

 

No Vivaldi, @Jon/t uses Vivaldi too so I'm wondering if he sees this also. The page is: https://www.evernote.com/AuthorizedServices.action

[eric99 1,083]

3 minutes ago, mackid1993 said:

No Vivaldi, @Jon/t uses Vivaldi too so I'm wondering if he sees this also. The page is: https://www.evernote.com/AuthorizedServices.action

Thanks for the link: btw I see hundreds of Evernote Web Clippers for Chrome on this page. It's not entirely clear to me what this page really shows

[Jon/t 1,690]

4 minutes ago, mackid1993 said:

No Vivaldi, @Jon/t uses Vivaldi too so I'm wondering if he sees this also. The page is: https://www.evernote.com/AuthorizedServices.action

I do and mentioned it a couple of months ago. From what I was told the account page is still being run by the old monolith but the actual system is on new stuff. The solution seemed to be to wait until the account stuff was rewritten. 

That being said I experienced no issues because of it. Just 100s of web clippers. I don't think it's a Vivaldi thing, just a mismatch of old and new code. 

[eric99 1,083]

1 minute ago, Jon/t said:

I do and mentioned it a couple of months ago. From what I was told the account page is still being run by the old monolith but the actual system is on new stuff. The solution seemed to be to wait until the account stuff was rewritten. 

That being said I experienced no issues because of it. Just 100s of web clippers. I don't think it's a Vivaldi thing, just a mismatch of old and new code. 

The weird thing is that I see 100s of web clippers for chrome, which I rarely use, and none for Firefox, my regular browser

[s2sailor 2,486]

2 minutes ago, eric99 said:

Thanks for the link: btw I see hundreds of Evernote Web Clippers for Chrome on this page. It's not entirely clear to me what this page really shows

Same here and I don't use Chrome.  I did a revoke all and then rechecked and I have 6 now ... weird. 

[Jon/t 1,690]

7 minutes ago, eric99 said:

The weird thing is that I see 100s of web clippers for chrome, which I rarely use, and none for Firefox, my regular browser

Same. From what I understood it was more of a reporting/display issue than an actual issue, down to the page still being run from the monolith code. I know authorisation is totally different from the old setup so maybe this has something to do with it.

[eric99 1,083]

13 minutes ago, Jon/t said:

Same. From what I understood it was more of a reporting/display issue than an actual issue, down to the page still being run from the monolith code. I know authorisation is totally different from the old setup so maybe this has something to do with it.

Since we all see these long lists, maybe the @mackid1993 problem has nothing to do with this after all?

[mackid1993 1,379]

She just confirmed it's a known bug in their backlog that doesn't affect anything. 

[eric99 1,083]

Just now, mackid1993 said:

She just confirmed it's a known bug in their backlog that doesn't affect anything. 

Does that mean that they are still in the dark?

 

[mackid1993 1,379]

39 minutes ago, eric99 said:

Does that mean that they are still in the dark?

 

No it's unrelated to my issue. She looked in their issue tracker about it. She just wants to see why my shard is calling a deprecated API and wants me to avoid using it until she can talk with her colleagues on Monday about it. 

[Feitz 270]

1 hour ago, mackid1993 said:

No Vivaldi, @Jon/t uses Vivaldi too so I'm wondering if he sees this also. The page is: https://www.evernote.com/AuthorizedServices.action

Thanks for the link, you are obviously not alone: 

1 hour ago, eric99 said:

Thanks for the link: btw I see hundreds of Evernote Web Clippers for Chrome on this page. It's not entirely clear to me what this page really shows

I recently had issues with web clipper not loading and your link showed hundreds of authorisations for chrome as well. Funny since I only use chrome-based Brave browser on my Android phone and Safari on my Macs.

-> 

 

There are literally hundreds of them:

[Feitz 270]

revoking them will not work: 

{"success":"false","reason":"csrf_failure"}

 

Edit: after several tries it seemed to have worked at last. 

No more Chrome authentications.