Bizarre Table Behavior Mystery

[jefito 5,589]

Evernote for Windows problem. Seen in 10.38.1-win-ddl-public (3408), Editor: v149.0.18202, Service: v1.53.0 (most recent update), plus the previous Windows version.

OK, so I need to make a table for some research. Easy! I've done it a million times in Evernote.

• Make a 4 column table, add a bunch of rows. Standard procedure.
• Add column titles to the top row. Give that a light background. I has a header!
• Add items to the leftmost columns of the rows under the header. Give each a background color.
• Copy simple Internet content from Wikipedia into various cells -- text only, no images. Some of it's italicized. Looking good, but not perfect yet...
• Change some of the cell text to use the Evernote monospace font. Italicize a few things. Change text sizes here and there. Great.
• Go to the top of the note, add in some web links. NP, standard procedure.

So OK, let's pop that note into a separate Window so it's out of the way of what I'm working on / researching so I can collect commentary and information on various items in my table. I'm ready to get some work done...

*** WHAM! *** In the separate window, my new table is now boxed up as 'HTML Content', i.e., it's no longer editable because [insert reasons here -- I couldn't think of any that made sense]. Meanwhile, back in the original version of the note, in the main Evernote  window, the table is there and still editable; and indeed, any changes I make are transferred to the separate window. I'll note here that I see this behavior if I just switch from my new note to a different one, and switch back again: tossed into 'HTML Content' jail.

OK, sure, I can choose to use the magic wand tool to "Simplify and make editable". Swish, flick! Yes, the result is editable all right, but all of the formatting that I put in - using Evernote tools btw -- is gone.

I do not understand this behavior. In particular, I do not understand the criteria by which a table, one which I created and edited using Evernote tools, is deemed to be uneditable by Evernote a short time later, notwithstanding the fact that I am actually editing the same content in a different window. I'm guessing that it may have something to do with something in text content I pulled in from Wikipedia, but that's not an uncommon use case for me. But hey, if I'm pasting in something that's going to make my table uneditable later on, shouldn't that generate a warning from the software?

Thoughts? Theories? I could try to replicate on a smaller scale case if need be.

[jefito 5,589]

Postscript...

OK, so I bit the bullet, and turned my table into a boring black & white frog ("Simplify & Make Editable), all in the main Evernote window. And it is indeed editable. Made some changes, then switched to a new note, and came back (also all in the main Evernote window). Table is now back as 'HTML Content'. Damn. Swish, flick -- Simplify and Make Editable-iarmus!. It's back to Evernote format again. Without making any further changes, switch to a different note, and back. It's 'HTML Content' again.

Smells like it's something in the copied content that Evernote doesn't really remove when it makes the note editable again.

I love magic software... (I write it, too)

[agsteele 2,942]

My guess, when you copy and paste from a website you are pasting the HTML code. Try pasting as plain text right from the beginning.

Edited June 2, 2022 by agsteele

[jefito 5,589]

Hi agsteele,

Thanks, and sure, that's how I'd see it, but obviously Evernote

a) let's you do stuff without warning you that it's going to -- at least potentially -- make your table uneditable,

b) it has to understand the clip and be able to convert it into something that Evernote can understand, so why not do that then and there, and

c) it still lets you continue editing until the point that you step away from the note original note (or do some other undetermined action), only rendering it uneditable when you return to it.

It just seems like there's some sync point where an internal format change takes place such that the editor (or Windows editor, at least) can't edit what it was happily and correctly editing earlier. I can paste HTML-sourced clips in fine, and edit/reformat them without problems, but at some point, the editor says "nevermore". I don't understand what Evernote's doing under the hood, so I can't explain why it's doing this. Mystery to me...

[agsteele 2,942]

If pasting as plain text doesn't work then I'd suggest a support ticket.

[Dave-in-Decatur 3,938]

@jefito, to me the weirdest part is that it only becomes uneditable in a separate window, not in the main program window. I agree that this needs a support ticket. I suppose you could (if you're inclined to put yet more time into this) perform the operation one step at a time, opening in a separate window after each step, to try and identify the step that produces the uneditable independent window.

[PinkElephant 8,072]

You asked for thoughts ?

From what I read, you say you copied some stuff from Wikipedia into the table. I think (sic) that EN believed it was a web clip now, and started to treat the whole thing as a web clip.

I never observed something similar, but I don’t remember copying web content into a table yet either. It is not what I use tables for, so probably it really came from the content you added, not from the fact it was a table at all.

If this is magic code or a user treadmill, I don‘t know.

[jefito 5,589]

7 minutes ago, Dave-in-Decatur said:

@jefito, to me the weirdest part is that it only becomes uneditable in a separate window, not in the main program window. I agree that this needs a support ticket. I suppose you could (if you're inclined to put yet more time into this) perform the operation one step at a time, opening in a separate window after each step, to try and identify the step that produces the uneditable independent window.

Yeah, yeah. Just need to bust it down to a simple case, repeatable case. My first post was mainly a what-the-heck? reaction...

OTOH, after nearly 40 years of making code for money, I've seen many wondrous strange things.

One of the first. On an early IBM PC, the C compiler (Lattice, before it was bought up by Microsoft) compiled a function similar to.

int calculate( a, b )
int a;
int b;
{
    int c = a * a + b;
}

One day, somebody looked at this and noticed that there's no return statement; the code is absolutely wrong, and the compiler should have told us so, but didn't. Yet calculate() worked perfectly, every time. Took a little bit of digging to figure out why...

 

[jefito 5,589]

15 minutes ago, PinkElephant said:

From what I read, you say you copied some stuff from Wikipedia into the table. I think (sic) that EN believed it was a web clip now, and started to treat the whole thing as a web clip.

Too simplistic.

My mystification comes from the fact that Evernote does not give any indications that it's treating the clip as a web clip, but as normal formatted text. I can continue editing happily away. At some point, under certain circumstances (e.g. switching to a different note and back, etc.) Evernote changes its mind, even though nothing in the note should change by switching away and returning. And then there's the weirdness of having the note open in two editing windows, with one treating it as HTML, and the other just as normal formatted text.

[PinkElephant 8,072]

Yes, makes not really sense - but in complex code there are things that don‘t make sense, can happen.

To tired today to run a test on it myself.

[Dave-in-Decatur 3,938]

4 hours ago, jefito said:

At some point, under certain circumstances (e.g. switching to a different note and back, etc.) Evernote changes its mind, even though nothing in the note should change by switching away and returning. And then there's the weirdness of having the note open in two editing windows, with one treating it as HTML, and the other just as normal formatted text.

When you switch away from a note to a different note and then switch back, Evernote may "redraw" the original note entirely; certainly it jumps back to the top of it instead of where you were when you left it (which I find irritating). The same thing might happen when opening it in a separate window. Perhaps something in that process triggers the "HTML Content" jailing. (Please understand that I say this as a guy who, years ago, used to write some BASIC code and batch files. No expertise of any sort is claimed here.)

[Mike P 2,624]

14 hours ago, jefito said:

*** WHAM! *** In the separate window, my new table is now boxed up as 'HTML Content', i.e., it's no longer editable because [insert reasons here -- I couldn't think of any that made sense].

I can't reproduce this effect. Perhaps a muminimum reproducible example would be good so that we can identify whether this is a general problem or something specific to your set up. What happens on the web version?

[jefito 5,589]

On 6/3/2022 at 1:10 AM, Mike P said:

I can't reproduce this effect. Perhaps a muminimum reproducible example would be good so that we can identify whether this is a general problem or something specific to your set up. What happens on the web version?

Hi Mike,

Curiously, I am having trouble reproducing it myself in a minimal fashion or otherwise, at least from scratch. I've tried copying content from the original table into a similar table in another note, and that table continues to be fine. But if I copy the entire table into my other note, it behaves as noted: switch away to a different note, even without editing the table, return, and it's become HTML content again. Meanwhile, the original table in that note stays editable throughout.

With respect to the web version, it appears to act the same. In the original problematic note, I use the magic want to make the HTML content editable, switch away, and switch back, and it's HTML Content again. To try to isolate things, in the web version, I've tried copying content from the bad table to a fresh new one, to see if I can trigger the behavior.

aaaand...

After a number of tries (including copying individual cells, rows, etc., I think I've identified some problematic cell content which, if I copy it from the bad table to a cell in the new table, the new table then exhibits the bad behavior: switch to a different note, return: the table is now HTML content. Not sure what the problem is; it's just some text I lifted from Wikipedia; looks innocent enough. Repeatable, too. I haven't tried to narrow it down much further than a short section containing 20-25 words, with no obvious suspicious formatting. Delete the text from the cell, all is well. Paste it in, table goes to uneditable. 

Seems to act the same in both the Windows desktop client and the web client, so yay??  It just seems to be something special about that text.

...

OK, Windows ClipSpy utility to the rescue? For the particular problematic chunk of text, the Windows clipboard's HTML format contains the following:

Version:0.9

StartHTML:0000000105

EndHTML:0000000459

StartFragment:0000000141

EndFragment:0000000423

<html>

<body>

<!--StartFragment--><div> Mask selection rules are checked only when <span style="display: none; clip: rect(1px, 1px, 1px, 1px); overflow:hidden; position: absolute; width: 1px; height: 1px; opacity: 0; font-size: 16.52px;"><span>{\displaystyle \dim(a)=\dim(b)}</span></span>, otherwise is false</div><!--EndFragment-->

</body>

</html>

If I just copy the bit "Mask selection rules are checked only when", then everything's OK. Pick up the '<span>' stuff, and the problem happens. It would seem that that's the culprit. In fact, if I paste that into the main body of a note -- i.e., not in a table -- it's marked as HTML content as well. And in a sense, it is HTML content, though its utility is doubtful. Question is whether that's something that can or should be filtered out when pasting into a note.

And finally, looking at the original source of the clip. It's on https://en.wikipedia.org/wiki/DE-9IM: it's in the section starting "a overlaps b". The part a little further on where it says "dim(a) = dim(b)"; that's the problem. Per ClipSpy, it's some kind of MathML expression.

That's all I know, at least today...

 

[Mike P 2,624]

1 hour ago, jefito said:

And finally, looking at the original source of the clip. It's on https://en.wikipedia.org/wiki/DE-9IM: it's in the section starting "a overlaps b". The part a little further on where it says "dim(a) = dim(b)"; that's the problem. Per ClipSpy, it's some kind of MathML expression.

 

Very interesting. I can reproduce this. Copying the text in the cell of the table which begins  "a overlaps b" into an EN table initially seems to work OK. Going out of the note and then back in results in the entire table being inside an "HTML content" box. If I simplify and make editable the box disappears but if I then go out of the note and back in it reappears. The dim a = dim b MathML bit  is not copied into EN at all.

If I paste and match style (ctrl+shift+V) (or copy into notepad) I get the raw mark up language for the equation

Just pasting in to the body of the note (no table) gives an even more bizarre result

Opening the 5kB attachment in notepad gives some sort of MathJax markup language??

Copying the Schrodinger equation from https://en.wikipedia.org/wiki/Schrödinger_equation gave very similar results

Here's another approach.

Select the text and then use the EN webclipper. Because you have already selected text you get the little known selection option

Wait until it syncs with EN and you find that it has converted the offending equation into a png image. The information can then be pasted into another note with no problems. The only difference from the orginal being caused by EN not allowing inline images.

 

 

[jefito 5,589]

Hey, that's even more fun that I thought. The MathML -> .png thingy is interesting. I didn't try Paste-and-match-style or the web clipper.

Took me a little while to narrow things down, as my narrative indicates (had to stop & get back to my real work). At least I know now how to fix the original note, and have a shot at  fixing any others that may pop up in the future.

Thanks for prodding me into further investigations.

[PinkElephant 8,072]

Probably it is the formula itself: Isn't this Schrödinger thing about Evernote tables being able to show content or formatting, but not both as the same time ? Quantum mechanics at work, so to say ...

Maybe ask Schrödingers cat for an explanation (I mean when it is eventually alive).

[Dave-in-Decatur 3,938]

I can confirm @Mike P's report. @jefito, maybe as a service to humanity you ought to edit that cell in Wikipedia to make it Evernote-compliant.

[jefito 5,589]

I'm guessing that the intersection of the set of geeky people interested in the finer details of DE-9IM and the set of Evernote users -- while demonstrably not empty -- does have a cardinality pretty near to 0.

Besides, we want to leave it as a test case for Evernote devs working on clipping and paste problems... 👩‍💻😈

[CalS 5,280]

35 minutes ago, jefito said:

Besides, we want to leave it as a test case for Evernote devs working on clipping and paste problems

Based on the HTML variants and coding preferences out there sounds like a lifetime job.  🤣

[jefito 5,589]

I'm here to help...  

[Mike P 2,624]

9 hours ago, jefito said:

I'm guessing that the intersection of the set of geeky people interested in the finer details of DE-9IM and the set of Evernote users -- while demonstrably not empty -- does have a cardinality pretty near to 0.

And as I demonstrated with the Scrodinger equation example it is a general issue for Wikipedia using whatever maths mark up language they use.

[jefito 5,589]

On 6/10/2022 at 12:28 AM, Mike P said:

And as I demonstrated with the Scrodinger equation example it is a general issue for Wikipedia using whatever maths mark up language they use.

Ummm. You did note the , right? I dug into the problem to a fair degree, found a pretty convincing suspect and reported my results here. After that, I'm just having fun. We can now see that pasting MathML into Evernote is obviously a problem, but I have no say in how its severity is rated by Evernote, nor Evernote's priority for fixing the problem. My guess would be that it's not a common use case, but that, like all guesses, is worth its weight in gold (how much does a guess weigh?). The fact that you replicated it was nice / confirmatory, and thanks for that, but doesn't really change my outside-the-Evernote-box (but longtime software developer's) assessment of its severity: I barked my shins on it, and I'd prefer not to again, but it's not a showstopper either. And I'm sure not gonna go and edit all the MathML in Wikipedia to make it Evernote compliant (pace @Dave-in-Decatur). 

[Mike P 2,624]

9 minutes ago, jefito said:

Ummm. You did note the , right? I dug into the problem to a fair degree, found a pretty convincing suspect and reported my results here. After that, I'm just having fun. We can now see that pasting MathML into Evernote is obviously a problem, but I have no say in how its severity is rated by Evernote, nor Evernote's priority for fixing the problem. My guess would be that it's not a common use case, but that, like all guesses, is worth its weight in gold (how much does a guess weigh?). The fact that you replicated it was nice / confirmatory, and thanks for that, but doesn't really change my outside-the-Evernote-box (but longtime software developer's) assessment of its severity: I barked my shins on it, and I'd prefer not to again, but it's not a showstopper either. And I'm sure not gonna go and edit all the MathML in Wikipedia to make it Evernote compliant (pace @Dave-in-Decatur). 

I agree. The webclipper work around makes it a pretty low priority to fix.