Is it possible to calculate the internal link of the note to create next?

[actant 22]

For offline notes, it's easy to simply add 1 to the last number of the internal link url.

Is there any info available to make it possible to calculate the internal links of an online note that is to be created next?

Thank you!

[PinkElephant 8,241]

Don’t think so - the links look rather cryptic. For security reasons links on cloud servers should never be computed in a way that a simple manipulation can create another valid link.

For local storage this is not relevant.

[gazumped 11,737]

No clue - that's a Support question,  but I doubt the answer's available for security (as above),  even if there is one.  

[actant 22]

Thanks for your replies! But it seems to be difficult to imagine that creating the internal urls simply with sequential numbers has something to do with data security. Encrypting the internal urls only makes it impossible to import notes with internal links while keeping those links still working.

[agsteele 2,964]

Whatever, but the straight answer to your question is that there is no method of calculating as you ask.

[actant 22]

Thank you all for your replies!

[PinkElephant 8,241]

For an internal link, data security is not needed, because you need access to the local drive to make the link work. This is protected by the device security.

But for content stored on a cloud server, it is important that from any link a person knows, there is no easy way to get at a second valid link. Usually the links are created by a hashing function that works very simple in one direction (on link creation), but is very difficult to break in the other. The link is no living link at all, but only a hash value. When it is entered, it is looked up in an internal table, and the related note is presented. The internal link on the server is never exposed, and the hash is semi random, not allowing to compute the hashing algorithm even if you learn to know several links.

That's more or less how it works.

[actant 22]

8 hours ago, PinkElephant said:

for content stored on a cloud server, it is important that from any link a person knows, there is no easy way to get at a second valid link.

If somebody have the link of a note in my account, he/she can access the note anytime and anywhere no matter whether I shared it or not?

[agsteele 2,964]

2 hours ago, actant said:

If somebody have the link of a note in my account, he/she can access the note anytime and anywhere no matter whether I shared it or not?

If you have enabled a public share then anyone can view the note if they are given or guess the link. However, guessing is exceedingly unlikely. So the link would need to be stolen or shared. if share is disabled the note cannot be viewed.

[actant 22]

1 hour ago, agsteele said:

If you have enabled a public share then anyone can view the note if they are given or guess the link.

Thanks for your reply! Then, sharing a note public simply use the same identifier that is used by its internal link. The design seems to be aweful. In most cloud services, the url of a shared resource uses a temporarily generated identifier that is destroied when the sharing is cancelled.

[gazumped 11,737]

2 hours ago, actant said:

the same identifier that is used by its internal link

Have you tried it?  What comes up in your browser is different from what's in the link (or was,  last I looked),  and notes need to have their sharing link enabled to be visible outside of Evernote's firewall.  Contents are not searchable even then,  and - while I wouldn't use the word 'unhackable' in this day and age,  working out on which shard your note is hosted,  plus the note's UID,  is going to require quite a lot of horsepower...

[actant 22]

2 hours ago, gazumped said:

Have you tried it? 

No. I never tried to share any notes publicly. I simly judged that according to replies suggesting that the internal links are based on encrypted strings for security considerations. In my personal opinion the internal links are encripted only to bring so much inconvinience unnecessarily.

[gazumped 11,737]

3 hours ago, actant said:

internal links are encripted only to bring so much inconvinience unnecessarily.

Seems unlikely that any coder would complicate things just for the fun of it - that makes it doubly complex - difficult to code in the first place and impossible to debug.

[eric99 995]

The internal link contains a Universally Unique Identifier.

See https://dev.evernote.com/doc/reference/Types.html :

"Typedef: Guid

Base type: string
Most data elements within a user's account (e.g. notebooks, notes, tags, resources, etc.) are internally referred to using a globally unique identifier that is written in a standard string format. For example:"8743428c-ef91-4d05-9e7c-4a2e856e813a"The internal components of the GUID are not given any particular meaning: only the entire string is relevant as a unique identifier"

For more details about GUIDs see also: https://en.wikipedia.org/wiki/Universally_unique_identifier

[actant 22]

@eric99   Thank you for sharing!

Sorry for another question:

1. In account X, there are two online notes: note A and note B. In note A there is a link to note B.

2. Note A and note B  are exported to test.enex

3. The file test.enex is imported to another account Y and synced.

Is it possible to write codes (e.g. in Python) to update the link in the new note A to make it jump to the new note B?

Thanks!

[agsteele 2,964]

No. 

[eric99 995]

7 hours ago, actant said:

@eric99   Thank you for sharing!

Sorry for another question:

1. In account X, there are two online notes: note A and note B. In note A there is a link to note B.

2. Note A and note B  are exported to test.enex

3. The file test.enex is imported to another account Y and synced.

Is it possible to write codes (e.g. in Python) to update the link in the new note A to make it jump to the new note B?

Thanks!

I'm glad you ask because few people on this forum seem to realize that note links are completely lost when moving your ENEX to another account. This is a serious limitation. What's the point of improving the notelink infrastructure as long as the links  are lost during export? 

This was already discussed  10 years ago but nothing changed so far: https://discussion.evernote.com/forums/topic/24557-import-old-notebooks-into-evernote-using-enex-loses-all-note-links/

EN may easily solve this as follows:

1. save the note id in the ENEX file (this is perfectly possible without breaking the format, because ENEX is an XML file (X stands for extendable)
2. when importing into a new account, the note ids in the note links should be substituted by the newly created ones (as you describe)

There is one bright spot: the HTML export in the legacy windows client (not the mac) keeps the note links intact by using relative hyperlinks to other notes in the export folders. You may fully navigate to other notes or any note attachment in your exported html database. It behaves as your own private website! Unfortunately, this isn't implemented in any other legacy or V10 HTML export.

To further answer your question:

Yes, it is possible to write a python or java program using the Evernote API to restore the notelinks in the new account:

This may be a good start: https://github.com/msbentley/everlink

When I have more time, I'll look at it in more detail

Please, let us know how it proceeds

 

 

[eric99 995]

1 hour ago, actant said:

@eric99  Thanks for sharing! The github project is "a python utility designed to help the transition from Evernote to another service whilst retaining full note-link functionality." And "currently only Joplin is supported." 

AFAIK, on Mac the notes in Evernote can be modified with AppleScript. This might make it easy to update the links, but I don't have a mac and so never tried it.

Sure, this github project is a python utility currently designed to help the transition from Evernote to another service, but this code is a very good starting point for your own code. It already does a big part of the job.

Do you intend to write it yourself? Maybe, you may join this github project to write an extension for evernote import?

 

 

 

 

[agsteele 2,964]

2 hours ago, actant said:

AFAIK, on Mac the notes in Evernote can be modified with AppleScript

The AppleScript support was supported by the old Legacy application for Mac. There is no scripting support for v10 on any platform.  If you can write your own working with the API then you have some potential options but I would still say that broadly speaking there are no scripting options available to achieve what you requested.

[actant 22]

1 hour ago, eric99 said:

this code is a very good starting point for your own code

Yes. It seems to be necessary to learn to use Evernote API.

[actant 22]

3 minutes ago, agsteele said:

There is no scripting support for v10 on any platform.

Thanks for sharing. But this is really a sad news.

[eric99 995]

1 hour ago, actant said:

Thanks for sharing. But this is really a sad news.

Yeah, very sad for anyone who uses note links because there is no backup or export provided (apart from the windows legacy html export)

[eric99 995]

12 minutes ago, actant said:

For so many years, no solution was developed for backup notes with recoverable links. I believe it's only a very small project if the EN team want to do it, but simply nobody did it. You can NEVER expect anything from the company. Almost everything people love with it was developed more than 10 years ago.

Indeed, why are these fundamental problems still there with a team of hundreds of employees working on just one product for 20 years? Compare this to Joplin or obsidian with a few engineers building the same product...

[gazumped 11,737]

2 hours ago, eric99 said:

Indeed, why are these fundamental problems still there with a team of hundreds of employees working on just one product for 20 years? Compare this to Joplin or obsidian with a few engineers building the same product...

All due respect - this isn't a 'fundamental problem',  it's something they didn't know would be required when they designed the database structure 20 years ago.  While they might be willing to change it now,  Evernote claimed to be the largest manager of personal information on the planet - 200 million+ current and past customers have their data stored in this old structure,  and a sizeable number of us connect with Evernotes servers every second of every day 24/7/365.

Changing the structure would be like walking across a busy racetrack to repaint the lines.

Of course all the new guys are starting from scratch,  or close to it - they have the advantage of Evernote's experience (want to bet most have been users at one time or another) - but none of the downside of hords of darn demanding users!

And I bet Evernote don't have hundreds of employees any more...  

[eric99 995]

4 hours ago, gazumped said:

have their data stored in this old structure

I did a small test: I extended an enex file with a noteId property.  I then imported this note in another evernote notebook. Everything went fine as expected because ENEX is an XML format. XML is designed to be extendable without breaking anything.

And they could always provide an additional option in the enex exporter whether to provide the noteId or not (the same as you can already specify which metadata should be exported).

  <note>
    <noteId>12345</noteId>
    <title>test note</title>
    <created>20230310T213530Z</created>
    <updated>20230310T213604Z</updated>
  /<note>

Conclusion: it is perfectly possible to extend ENEX with a note Id. This is fully backwards compatible.

This is the only extension needed to enable export of note links!

[gazumped 11,737]

11 hours ago, actant said:

A choice of to do or not to do.

Along with routine maintenance, bug fixes, developments that are already in the pipeline, and about 1,000 other requests from sync speeds to choice of thumbnails.  There's probably enough work in there for the next 10 years - why should this be first?  Or even soon? 

Evernote don't get involved with discussions about new features until they're released because they'd rather be working on code than arguing what's next and answering demands about when what's in progress will be released,  and why any particular feature was chosen over someone else's personal favorite.

[PinkElephant 8,241]

For any normal user calculating an encrypted link by reverse engineering is senseless.

And EN just needs to server their normal users. Nobody will waste dev hours on geek stuff.

[eric99 995]

Just now, PinkElephant said:

For any normal user calculating an encrypted link by reverse engineering is senseless.

And EN just needs to server their normal users. Nobody will waste dev hours on geek stuff.

The point is that notelinks from any normal user are now lost when exporting via enex... This is called a vendor lock-in!

[gazumped 11,737]

5 minutes ago, eric99 said:

This is called a vendor lock-in!

It's only an issue if you use links intensively and continue to do so after you know it doesn't currently carry through to ENEX back-up.  We're not arguing it should get done,  and under Evernote's new management that might happen sooner than otherwise;  but unless they were already planning to release it,  this won't happen for at least a year.  Meantime,  back-up differently or change the system (ToC's come to mind)...

[eric99 995]

3 minutes ago, gazumped said:

It's only an issue if you use links intensively and continue to do so after you know it doesn't currently carry through to ENEX back-up.  We're not arguing it should get done,  and under Evernote's new management that might happen sooner than otherwise;  but unless they were already planning to release it,  this won't happen for at least a year.  Meantime,  back-up differently or change the system (ToC's come to mind)...

afaik, ToC's are based on notelinks as well, so these will lock you in as well...

[gazumped 11,737]

1 minute ago, eric99 said:

ToC's are based on notelinks as well

Wouldn't know without testing - at a guess internal links are linked to a note's UID which may or may not change on being restored.  You do,  however have the accurate note title (unless you choose to change it) and could do a search...

[DTLow 5,737]

42 minutes ago, eric99 said:

This is called a vendor lock-in!

Not sure if "vendor lock-in" is the correct term
but we know data is lost on an export (notebook, note-id, reminders, tag hierarchy ...)

[gazumped 11,737]

9 minutes ago, DTLow said:

we know data is lost on an export

...and now I can confirm that Tables of Content are compromised too!  Just did a quick test with a ToC pointing to 2 test notes.  ENEXed all three notes,  deleted the originals and emptied the trash,  then restored the three notes.  The test notes are restored with the ToC but it no longer has access to the individual notes.  So my ToCs are now a casualty of this issue...  

Have to go back through in my copious free time (!) and tag the linked notes.

[Dave-in-Decatur 3,981]

On 3/10/2023 at 5:07 AM, eric99 said:

I'm glad you ask because few people on this forum seem to realize that note links are completely lost when moving your ENEX to another account. This is a serious limitation. What's the point of improving the notelink infrastructure as long as the links  are lost during export? 

So this thread has morphed from a discussion of whether it is possible to manually calculate and construct internal note links (something that it seems unlikely that more than a handful of users would ever think of doing) to this. Which, to me at least, looks like something that it seems unlikely that more than a double handful of users are likely to need. How often does one need to export all one's notes to .enex and then import all that to another account? (Another account of one's own, or belonging to someone else?) Compared to the sync issues that seem to affect almost every user at one time or another (some much more intensely than others, granted), and others issues affecting the Android app in particular, it's hard to see why Evernote would be expected to devote resources to this. Not saying they shouldn't do it, only not understanding why this should be a priority.

[eric99 995]

11 hours ago, Dave-in-Decatur said:

So this thread has morphed from a discussion of whether it is possible to manually calculate and construct internal note links (something that it seems unlikely that more than a handful of users would ever think of doing) to this. Which, to me at least, looks like something that it seems unlikely that more than a double handful of users are likely to need. How often does one need to export all one's notes to .enex and then import all that to another account? (Another account of one's own, or belonging to someone else?) Compared to the sync issues that seem to affect almost every user at one time or another (some much more intensely than others, granted), and others issues affecting the Android app in particular, it's hard to see why Evernote would be expected to devote resources to this. Not saying they shouldn't do it, only not understanding why this should be a priority.

Agreed, sync issues are top priority, but the capability to jump ship without data loss is crucial as well, especially now with Evernote's uncertain future and prices skyrocketing...

[gazumped 11,737]

11 hours ago, Dave-in-Decatur said:

How often does one need to export all one's notes to .enex and then import all that to another account?

Fair comment - and I'd think that links / tables of content etc. would not seamlessly translate into a different app with (presumably) a different structure.  There will inevitably be a lot of clearing up to do in the event of moving to another provider.  "Moving" in Evernote terms though is merely logging in from a different device so the server provides all.  Moving to OneNote (forinstance) would mean a major restructuring job.

In the (unlikely) event that one or more notes goes missing from an account,  there might be some work involved in importing and re-connecting it/ them to an interconnected linking network,  but barely more (I would have thought) than merely importing from backup.

[eric99 995]

On 3/12/2023 at 11:05 AM, gazumped said:

Fair comment - and I'd think that links / tables of content etc. would not seamlessly translate into a different app with (presumably) a different structure.  There will inevitably be a lot of clearing up to do in the event of moving to another provider.  "Moving" in Evernote terms though is merely logging in from a different device so the server provides all.  Moving to OneNote (forinstance) would mean a major restructuring job.

In the (unlikely) event that one or more notes goes missing from an account,  there might be some work involved in importing and re-connecting it/ them to an interconnected linking network,  but barely more (I would have thought) than merely importing from backup.

hmm, EN has still work to do 😏 :

"...

Even though millions of people have their most important information in Evernote, we still want to make sure that they can leave the service quickly and easily. There is no data lock-in with Evernote. We are committed to making it straightforward for you to get all of your data into, and out of, Evernote at any time. Our desktop software lets you export all of your notes and content in human-readable HTML, as well as a fully documented machine-readable XML format. We also have a full, free API that lets you access all of your data.

Our philosophy is that by making it possible for you to leave at any time, we’re forever-motivated to build great things so that you’ll want to stay. And we hope you do stay, for years and even decades. To this end we’re committed to investing in technologies that will make your data accessible far into the future.

..."  https://evernote.com/privacy/3-laws-of-data-protection

[gazumped 11,737]

1 hour ago, eric99 said:

EN has still work to do

Isn't this kinda like wondering why your Tesla battery won't fit in that Volvo diesel you bought?

Given that Evernote has something like 80+ competitors,  using different interfaces and many of them operating with different services like integral calendars - how much work would you expect Evernote to do towards making the transfer from their environment into a different architecture and a different coding?  Plus each one of those 80-odd companies may change or add features at any time,  and there will be new players - I found a few recently that I'd never heard of before.  (One based in a CN domain and politely called "high risk" by a reviewer!).  Any compatibility would be an ongoing and onerously expensive exercise.

Since Evernote has a published API,  I'd consider it the sign of a suitably ambitious and competent note-taking app to have provided an 'import' option that moved standard Evernote notes into their environment as completely as possible.  But fitting square pegs into round holes is always going to take some user (or AI!!) involvement.

 

[eric99 995]

1 hour ago, gazumped said:

how much work would you expect Evernote to do towards making the transfer from their environment into a different architecture and a different coding?

This: "We are committed to making it straightforward for you to get all of your data into, and out of, Evernote at any time"

That means that ENEX should be extended to export all data such as notelinks, notebooks and possibly tag hierarchy.  As I already explained before, these are very minor downwards compatible extensions, not changes. We've already spent far more time here discussing it than that it actually takes to implement it.

Obviously it's up to the competitors to provide importers, and maybe they have to best fit the data in their structures, but at least they need the full data to start with.

 

[AlbertR 562]

On 3/11/2023 at 11:31 PM, Dave-in-Decatur said:

How often does one need to export all one's notes to .enex and then import all that to another account?

Seldom for normal users - but maybe necessary if you try to seperate spaces of notebooks in Teams accounts to single account (BG: EN Teams accounts are somewhat expensive and nobody knows whether they will be supported in future...)

On 3/12/2023 at 11:03 AM, eric99 said:

especially now with Evernote's uncertain future and prices skyrocketing...

And there is another use case: "family" accounts that have been managed by parents - sometime the children grow up and should take "their" notes out of the family account...

3 hours ago, gazumped said:

how much work would you expect Evernote to do towards making the transfer from their environment into a different architecture and a different coding?

We're NOT talking about moving to different platforms. As @eric99 explained, it's easy to extend exports by UIDs. It's not so easy to extend imports to retain these UIDs (*) - OK. But it is possible - we've done similar within own software that handles (e)CAD data in a multi-user environments...

3 hours ago, gazumped said:

Since Evernote has a published API,  I'd consider it the sign of a suitably ambitious and competent note-taking app to have provided an 'import' option that moved standard Evernote notes into their environment as completely as possible.  But fitting square pegs into round holes is always going to take some user (or AI!!) involvement.

I'm not sure whether the API is really complete since it hasn't been updated for years. But there is a limiting parameter that makes it very hard to use it for mass data tasks: If you reach a limit of operations/hour, all further activity is denied for some time... (Filterize users know it 😉). The API is server based - so there is no way to work around this.

2 hours ago, eric99 said:

That means that ENEX should be extended to export all data such as notelinks, notebooks and possibly tag hierarchy. 

Full agree 👍 (with some hope):

• Notelinks are exported already (but do not work because UIDs are not retained. An improved import algorithm would make them living immediately.
   
• Notebooks might be added to ENEX easily (same way as Eric explained). An improved import algorithm should use it instead of placing all stuff in one "Imported" notebook.
   
• Tag hierarchies are harder to implement because there is no structure to describe it in ENEX format.
  • Currently tags of a note are written and read - but only as a reference based on the tag name.
    • A tag itself (including its style and hierarchy placement) is not written the ENEX because this has no meaning within a note.
  • Tag hieraries are easy to maintain and re-create interactivly if necessary
  • EN Teams users are familiar with using tags without hierarchy - it's not available there 😉.
    • I've had some discussions around this and saw, that it is hard to maintain global hierarchies that allow local users to add own tags and want to organize own hierarchies. So they decided to simply decline hierarchies at all - until anyone comes up with a nice idea to handle it anytime in future 🤔

(*) Some word around importing UIDs:

• An import process should re-use a UID if it is not used already. If it is used, a new UID is to be initialized. Because UIDs are stored in maps, this is easy to implement so far.
• During an import operation a second map should be used to destinuish between re-used and new UIDs. Also not very difficult...
• After all notes are imported, they have to be reworked regarding notelinks therein. The temp. map gives information regarding "re-used"/"new". In case of "new", the link has to be changed to the new destination UID.
• Remove temp. map
• Ready ✔