Corrupted server database: web client search returns notes which can't be loaded from the server!

[eric99 978]

I just noticed a very  serious problem with 3 of my notes: when clicking one of these notes in the search result list, the server returns "this note can not be loaded. Reload the page or try again".

I checked my HTML export and it was still OK in October 2021. In my last HTML export (February 2022), the notes are gone.

So the problem is that the search result list contains notes which can't be loaded from the server. These notes are only visible in a search list, not in the unfiltered list.

This happens in EN for WEB, windows and android (if online). Offline, these notes are not found.

You can find corrupted notes in your database by searching for '*' and then try to select all notes (increase your multiselect to 500 first). If the multi select doesn't work, there is a corrupted note in your search result list.

I can live with many bugs, but not with an unreliable database. It seems that our data is in danger now!

[agsteele 2,942]

3 hours ago, eric99 said:

You can find corrupted notes in your database by searching for '*' and then try to select all notes (increase your multiselect to 500 first). If the multi select doesn't work, there is a corrupted note in your search result list.

I can live with many bugs, but not with an unreliable database. It seems that our data is in danger now!

Thanks for this information. It will be good to discover of this is a common issue.

I think it might be too early to infer a cause and effect or that data is in danger. But I would agree that what you have discovered needs to be investigated.

It is possible that the problem notes were damaged on one of your devices and then synchronised to the cloud rather than damaged in the cloud. No doubt there could also be other explanations.

Do you have working versions/copies of these notes as well?

You will only get lots of conjecture here which may create more heat than light.

I'd suggest a support ticket.

[eric99 978]

As I already mentioned: These notes are only visible in a search list, not in the unfiltered list. Unfortunately I can't remember whether I deleted these notes myself. I accidentally  stumbled on this problem  when  multi-selection failed.

In any case, there is an inconsistent database behavior, deleted notes shouldn't be returned by a search.

So I hope that the server index contains stale references to normally deleted notes and not the opposite.

Have you checked your database for dangling references (You can do that  by searching for '*' and then try to select all notes (increase your multiselect to 500 first). If the multi select doesn't work, there is a corrupted note in your search result list)

Thanks

 

[gazumped 11,652]

16 minutes ago, eric99 said:

So I hope that the server index contains stale references to normally deleted notes and not the opposite.

Have you contacted Support yet?  They're the only people who are in a position to find out. Yelling 'fire!' in a crowded theatre is just going to generate lots of noise and bury your query in dozens of other - probably irrelevant - concerns.

[eric99 978]

3 minutes ago, gazumped said:

Have you contacted Support yet?  They're the only people who are in a position to find out. Yelling 'fire!' in a crowded theatre is just going to generate lots of noise and bury your query in dozens of other - probably irrelevant - concerns.

Although I have very bad experience with 'solved' tickets, I intent to contact them, but I hoped that other users could check their database for similar problems

[agsteele 2,942]

2 hours ago, eric99 said:

Have you checked your database for dangling references (You can do that  by searching for '*' and then try to select all notes (increase your multiselect to 500 first). If the multi select doesn't work, there is a corrupted note in your search result list)

I already have multi-select set to 500 BUT since I have more than 500 notes I will not be able to select all since that will exceed the limit. The maximum limit that can be set is just under 1,000 so even that won't work for me. So in my case, at least, all that does is confirm that the multi-select limit is working as intended.

As suggested, raise a ticket and see what support says.

2 hours ago, eric99 said:

I hoped that other users could check their database for similar problems

But perhaps with slightly less fear and trepidation until we know what is going. As you say, you don't recall the history of these errant notes so it is unwise to conclude only one reason.  Keep investigating and let us know what you discover.

[eric99 978]

6 hours ago, agsteele said:

I already have multi-select set to 500 BUT since I have more than 500 notes I will not be able to select all since that will exceed the limit. The maximum limit that can be set is just under 1,000 so even that won't work for me. So in my case, at least, all that does is confirm that the multi-select limit is working as intended.

Thanks for testing. To be sure, the multi-select test should be performed on a search result list, not on the unfiltered list since the dangling references are not in there.

You're right, it's not very clear what's going on but one thing is sure: the server database ( or database index) is corrupt since it returns an non existing note in the search result.

[PinkElephant 8,071]

Maybe a server update issue. I think support should take a look.

Just made a wildcard search of my account and checked - no problems at all.

[eric99 978]

47 minutes ago, PinkElephant said:

Maybe a server update issue. I think support should take a look.

The problem occurs in V10 evernote web, it has nothing to do with my desktop version, although it occurs there as well (I already deleted my local database and reloaded it again from the server without any result). It doesn't occur in legacy or offline V10.

[PinkElephant 8,071]

Tried a sample on the web client as well. Since it is 50 notes there, could no test all.

[eric99 978]

On 4/17/2022 at 2:29 AM, PinkElephant said:

Tried a sample on the web client as well. Since it is 50 notes there, could no test all.

Thanks for testing 🙂

The ticket has been issued (Ticket# 3502430) . Please feel free to tag it as high priority.

In the mean time, I noticed that a wildcard search returns 7 more notes than the total amount  of notes in the unfiltered list. I can test this only on my android phone since the Web client and windows client refuse to return the amount of filtered notes, instead it returns 1000+ 😕   Why do they want to hide the exact count?

Can anybody verify this on their phone please? Maybe this can help the developers to diagnose this issue...

 

 

[agsteele 2,942]

1 hour ago, eric99 said:

Why do they want to hide the exact count?

I really don't think that there's a conspiracy here. Nothing hidden other than because nobody considered it a useful piece of data. By all means suggest it as a UI improvement.

[eric99 978]

2 hours ago, agsteele said:

I really don't think that there's a conspiracy here. Nothing hidden other than because nobody considered it a useful piece of data. By all means suggest it as a UI improvement.

I'm not talking about a conspiracy, more in technical terms of misplaced OO information hiding. Why is a count not a useful piece of data once it exceeds 1000? This behavior has been implemented explicitly, since it works properly below 1000.  And why is it different from android where it is always shown?

[agsteele 2,942]

Who knows? Certainly nobody here... Sorry.

[eric99 978]

I just discovered another thread that describes similar problems with the server database (index).

Since legacy has workaround code to remove these "ghost notes" from the server result list, we could assume that this is an old known issue. Developers probably forgot to implement this workaround in V10.

The result is that my multi-select doesn't work anymore due to these ghost notes. I hope EN will at least re-index the server database for my account to make multi-select working again.

https://discussion.evernote.com/forums/topic/131743-deleted-notes-still-appearing-in-search/

[gazumped 11,652]

If we're still talking about the web client,  did anyone suggest clearing the browser cache yet?  Or trying a different browser?

[eric99 978]

1 hour ago, gazumped said:

If we're still talking about the web client,  did anyone suggest clearing the browser cache yet?  Or trying a different browser?

Thanks for the suggestion, but it happens also in two other different browsers, and also in the V10 online android and windows client (where I reinstalled my local database). As I said, it doesn't however occur in the legacy client, therefore I believe that legacy has a workaround to skip these zombie notes from the server resultlist before showing it to the user.

I noticed in the previous thread that a ticket was issued around januari 2021, apparently not solved yet. I hope my new ticket will remind development to finally fix it.  Maybe, you know how to increase the prio of my new ticket 3502430, since this issue affects the server database integrity for more than a year now...

 

[gazumped 11,652]

5 hours ago, eric99 said:

since this issue affects the server database integrity

We have a total of about 5 users and (maybe) a dozen notes that seem to be deleted,  but won't lie down.  I hardly think that 'server integrity' is at any risk. 

Plus - there's still no clear indication of what's causing this,  and that is usually the bugbear of fixing an issue [if there is one] finding it takes 98% of the time. 

Coding a fix - [if there is one] - about 5 minutes. 

All I can do is flag tickets for the Forum Admin to look at,  and I've done that - maybe there will be an answer to one or the other ticket...  

Let us know what you hear!

[eric99 978]

1 hour ago, gazumped said:

We have a total of about 5 users and (maybe) a dozen notes that seem to be deleted,  but won't lie down.  I hardly think that 'server integrity' is at any risk. 

Plus - there's still no clear indication of what's causing this,  and that is usually the bugbear of fixing an issue [if there is one] finding it takes 98% of the time. 

Coding a fix - [if there is one] - about 5 minutes. 

All I can do is flag tickets for the Forum Admin to look at,  and I've done that - maybe there will be an answer to one or the other ticket...  

Let us know what you hear!

Thanks 🙂

edit: if they can't pinpoint the root cause, they could always implement the legacy workaround in the V10 clients (removing  the zombie notes from the server  result list)

[eric99 978]

5 hours ago, gazumped said:

Let us know what you hear!

I just got an answer from support on my remark that it seems to be the same problem already reported before:

"...Thanks for pointing this out. I confirm that this issue has indeed been going on for a while and it is still open with the engineering team. We already pinpointed the root cause, but it is being blocked by another issue and it may take some time before this work is completed. That said, we're normally able to resolve this for each individual account, and the issue should not return after this.
 
I am still waiting for the engineers to fix your account, and I will get back to you with the confirmation once it is fixed. You can expect to hear back from me after the weekend (the ticket stays open)..."

[eric99 978]

The engineers  re-indexed my account and everything works fine again, no zombie notes, multi-select works again and the number of notes of a wildcard search is equal to the total number of notes (before I had 9 zombie notes according this test on my android phone)

Great support  👍 🙂