Behind the Scenes: Uniting the Evernote Apps

[Shane D. 1,826]

Hi All,

Since Evernote’s earliest days, the five clients (Android, iOS, Windows, Mac, and Web) have been evolving separately, tackling the challenges of software development in subtly different ways.In this installment of Evernote Behind the Scenes, CEO Ian Small talks to Mike, a senior member of the engineering team, about the work they’re doing to reunite the Evernote family.

Ian and Mike discuss the fundamental changes they’re making to how Evernote communicates between client and cloud, and how they’re tackling the difficult task of making a single shared code library work across the five different apps.

 

[DTLow 5,745]

Thanks for this look behind the scenes.   
I understand the need for the common interface

My concerns are more about the Evernote client apps

I'm looking forward to UI enhancements with the Web/Android/IOS platforms   
At the same time, I'm apprehensive for the Windows/Mac platforms     
What features will we be losing?   
For example, will Mac Applescript integration continue to be supported

[CalS 5,310]

Hard to ascertain how much is standardized EN data wrapper versus feature/function stuff.  Though it sounds like CONDUIT is wrapper with the functional programming still on the client.

"Any new features that we want to develop we can develop once in conduit to mange the data from the cloud and then our UI teams can focus on what they do best which is making a good user experience."

Reminds me of the three tier client server stuff of days gone by.  Not a bad thing.  IAC, I'm with @DTLow re are any features going bye bye as the data gets scrubbed through CONDUIT.  And it does make one wonder a bit about local data bases.  Not too worried, but I'd rather not be reliant on internet and server for my productivity.  I can't even use the current web version based upon performance.  Desktop rocks when it comes to getting things done.  

[MichaelvT 20]

Great update and is good to know that the core engine of the application stack is getting a facelift/refresh. 
now I am not a cloud native app specialist but, wording for a tech company myself I am not so worried about this new approach..

 I actually believe many cloud native applications make use of such a shared approach of resources to offer a cross platform solution 

this should save Evernote tons of operational overhead as well. This should not reduce but increase the pace of new features coming out once Conduit is at full swing. So looking forward to the developments ahead. 
 

I must say I am a bit (in a positive way) sad that we still have to wait for new features on the app but I am fully behind the idea to work on quality of the overall product and service rather than additional goodies. Let’s wait and see. 
cheers! 

[Jeremy England 141]

I have to image that a lot of stuff will be able to be implemented still right? At the most basic, Apple Script is just updating data which will then have to get parsed through the Conduit. Kind of like I can copy and paste a folder with right click, or I can copy and paste a folder at the command line. That's my hope at least lol.

[TomAshworth 3]

Looking forward to seeing the look and feel of the web client on the Mac client and a unified appearance on IOS. Speedy search is very important. Great work guys.

[DTLow 5,745]

24 minutes ago, TomAshworth said:

Looking forward to seeing the look and feel of the web client on the Mac client

I'm not happy with the "look and feel" of the web client(s)
I'd prefer to see the look and feel of the Mac client on the other platforms
- Window users may have a different opinion

[Rollie Cole 4]

I am all for making it easier for a developer to add features to many platforms at once. At the same time, I find a profound difference, not between OS's per se, but between (a1) devices with big screens, big keyboards & mice, and NO touchscreen -- MAC and Windows; (a2) devices with big screens, maybe big keyboards, maybe mice (iPad and other tablets); and (b) devices with small screens, tiny keyboards or keyboards on a touch screen (smartphones, whether Android or IOS). I would like my apps to display and receive input appropriately for those differences.

I note Evernote on Windows already displays differently on my portrait screen than on my landscape one (thanks, Evernote!). Where autotype and autocorrect are big hassles when I have a big keyboard (I am old enough to have started on a real, not even electric! typewrite), they can be very useful when trying to use my big thumbs on a tiny touchscreen keyboard on my iPhone (or my Samsung, when I had one of those; the touchstick helped a bit with accuracy at the expense of speed and convenience).

So please do not go "too far" in making the client "alike" on all devices. Please keep the size of the display and the form of input in mind (voice input and output is sometimes nice, but I am often in noisy environments or where noise from me or my device would disturb others).

Thanks for all you do. 

[macfixer 709]

I just hope to get my mitts on a real, working copy of Evernote that works properly.

[Rollie Cole 4]

Your experience sounds waaaaaay different than mine. I have had great luck with Evernote on Windows and IOS (and I think I used it on Android as well). My only real complaint is that the procedure for making some (but NOT all) notes available offline is a bit clunky (I would love to have that be a checkmark on all notes, rather than having to copy them to an offline notebook).

 

But these machines (and the humans that operate them) are still so different that YMMV (your mileage may vary) often applies.

[jefito 5,598]

1 hour ago, CalS said:

Hard to ascertain how much is standardized EN data wrapper versus feature/function stuff.  Though it sounds like CONDUIT is wrapper with the functional programming still on the client.

"Any new features that we want to develop we can develop once in conduit to mange the data from the cloud and then our UI teams can focus on what they do best which is making a good user experience."

Yes, what I was thinking as well. This seems like more plumbing (and the name "Conduit" is a little leading in that respect), though on the client side, and seemingly under the hood of the UI, as far as I can tell. But it's still a bit cloudy in terms of what it encompasses. But it seems as though Evernote applications still need to do device-specific work on whatever platforms are supported.

OK, so Conduit: seems as though the Evernote API, that still exists, and has to continue work because of existing 3rd-party applications. As I understood it, all of the existing Evernote applications also use (or used) the API, but will switch over to the Conduit. So is the Conduit mainly a wrapper around the API. possibly making use of the API more rigorous/regular (or maybe implementing Evernote policies) in terms of handling syncs and whatever else it does? Wondering whether Conduit also enforces policies on note content (e.g. providing available heading styles and other fonty stuff, which will be new). Does Conduit provide search facilities, so we get more predictable results on different devices? Does Conduit handle caching stuff as well? Does it handle local databases, which are different on different devices? Many idle questions...

Oh, and whither CEF? Is that still a thing, client side?

Is the Conduit going to be made available to 3rd party developers, a la the API?

[CalS 5,310]

6 minutes ago, jefito said:

So is the Conduit mainly a wrapper around the API. possibly making use of the API more rigorous/regular (or maybe implementing Evernote policies) in terms of handling syncs and whatever else it does? Wondering whether Conduit also enforces policies on note content (e.g. providing available heading styles and other fonty stuff, which will be new). Does Conduit provide search facilities, so we get more predictable results on different devices? Does Conduit handle caching stuff as well? Does it handle local databases, which are different on different devices? Many idle questions...

Oh, and whither CEF? Is that still a thing, client side?

Is the Conduit going to be made available to 3rd party developers, a la the API?

So many questions, so little time....  

[Paul A. 677]

I just want to know when they are going to expand the size of the preview groups. Having been a former Evernote beta tester for multiple platforms, the wait to get invited and get an opportunity to kick the tires myself is quite frustrating. Especially as I've heard very good things about the new mobile client (with the caveat of major functionality still missing, significantly including offline notes.)

[Ian Small 215]

Quickie clarifications for those following along...

Yes, per @jefito, Conduit is common *plumbing* on the client-side.

Conduit has no relation to the UX.  Every Evernote client built on Conduit could have a different UX and Conduit wouldn't mind.  Users might mind, and we would definitely mind, but Conduit wouldn't.  That's basically the whole point.

Where Conduit figures in is anywhere that information flows between client and cloud - whether that information is content moving back and forth or a service call of some kind.  We expect all of that information to flow through Conduit (on the client side).  The goal of conduit is to have common functionality behave in consistent and predictable ways across all of our clients.  Sync is the easiest function to understand this with:  we'd like all clients to sync a recently modified note (whether that's sync up or sync down) in a consistent way (allowing of course for variances in whether or not that particular note is supposed to be synced to that device).  

To the concern that @CalS raises about local data stores, part of Conduit's job is to interface to the appropriate technology that forms the local data store (which, unsurprisingly, is a pretty different technology layer on a browser, mobile device or on a PC).  That, along with the different memory / CPU / power profiles of the different devices on which our clients run, drives the need for per-device optimization that we talk about in the video, that makes Conduit such a beast to get right.

And finally, Conduit, in and of itself, does not close the door to device-specific customizations or integrations.  To the extent that device-specific functionality needs to get to the cloud, it will have to go through Conduit at some point.  But the functionality that fits that description is usually travelling a well-trodden (ie. pre-existing) path on its way to the cloud, rather than having to carve its own unique API pathway.

As @CalS says, yes, there are so many questions.  But I do not have time to answer them all (nor, quite honestly, have we actually converged on some of the answers yet - if only because hard experience teaches one not to constrain oneself to an answer before you actually need it).  Hopefully the elaborations above provide some clarity.

Back to lurking,
ian

[jefito 5,598]

8 hours ago, Ian Small said:

Quickie clarifications for those following along...

As always, I appreciate the insights, clarifications, and the videos. For my part, no worries about not getting answers to all of my questions -- being a software developer myself, a lot of them just stem from curiosity about application architecture in general, and since Evernote is an application I use a lot, that curiosity spills over to the Evernote's architecture as well. Sometimes understanding the stuff under the hood better can help me to diagnose other users' problems, but for my own usage, understanding how the soup is made is less important than how the soup tastes (if you'll forgive the metaphor). I'll let you get back to wrestling with the elephant. Cheers.

[CalS 5,310]

13 hours ago, Ian Small said:

...part of Conduit's job is to interface to the appropriate technology that forms the local data store (which, unsurprisingly, is a pretty different technology layer on a browser, mobile device or on a PC).  That, along with the different memory / CPU / power profiles of the different devices on which our clients run, drives the need for per-device optimization that we talk about in the video, that makes Conduit such a beast to get right.

Amen.  Got that t-shirt from from managing some turnarounds where redundant data was all over the place across apps and platforms, hence the wrapper comment.  Different context but same issue of sorts. 

Thanks for the sharing.

[jefito 5,598]

Answered my own question: the API is obviously still around, since Conduit is being used in the web browser, but the existing public Evernote applications do not. So they coexist..Even moreso, it just seems to make sense that Conduit  would be layered on top of it, but I have no actual evidence for that.

[Bev Adams 3]

If I understand correctly, Conduit works as a middle man between the data bases and the user interface. I share a notebook with a lot of people, some of them have no experience with Evernote or sharing files. One of my frustrations is helping these people over the phone. I know how to do things with my Windows PC, but their Mac or iPad look and in some ways function differently. I Does this mean that Evernote will look the same on all believe they can all do the same things. This Behind the Scenes video explains why they are different and now I understand better why it's difficult to make them the same.

Does this new Conduit system mean that soon my friends on their Macs and iPads will begin to see the same things I see on my PC?

[jefito 5,598]

35 minutes ago, Bev Adams said:

If I understand correctly, Conduit works as a middle man between the data bases and the user interface.

Actually, between the "client and the cloud":

21 hours ago, Ian Small said:

Where Conduit figures in is anywhere that information flows between client and cloud

 

37 minutes ago, Bev Adams said:

Does this mean that Evernote will look the same on all believe they can all do the same things.

No guarantees on that one, evidently:

21 hours ago, Ian Small said:

Conduit has no relation to the UX.  Every Evernote client built on Conduit could have a different UX and Conduit wouldn't mind.  Users might mind, and we would definitely mind, but Conduit wouldn't.  That's basically the whole point.

On the other hand, there is a push towards feature parity across all Evernote clients (e.g. https://discussion.evernote.com/topic/120633-behind-the-scenes-achieving-feature-parity-in-evernote-web/; check out the other videos in the Behind the Scenes series), but different form factors on the different device types don't make an identical interface an easy task, as you note.

 

[aukirk 368]

I can't wait for the new web editor features to make it to iOS and Mac versions...  I find myself essentially checking these forums everyday looking for some hint or indication about when they will be rolled out, as I plan to completely revamp my task management workflow.

I absolutely love the task lists vs. task boxes, as well as the highlighter colors.  It is great to combine a task list item with subtasks inline using task boxes... then when the entire thing is complete, check it off and the entire row with all subtasks get marked complete.  

The little things make me happy... but it drives me crazy knowing what is coming, yet not having access to the same features in iOS and Mac apps.

[Paul A. 677]

19 hours ago, aukirk said:

I can't wait for the new web editor features to make it to iOS and Mac versions...  I find myself essentially checking these forums everyday looking for some hint or indication about when they will be rolled out, as I plan to completely revamp my task management workflow.

I absolutely love the task lists vs. task boxes, as well as the highlighter colors.  It is great to combine a task list item with subtasks inline using task boxes... then when the entire thing is complete, check it off and the entire row with all subtasks get marked complete.  

The little things make me happy... but it drives me crazy knowing what is coming, yet not having access to the same features in iOS and Mac apps.

I totally agree. Eagerly waiting for the preview to be expanded in size and for these new features to come to desktop.

[lhb 66]

Will the redesigned conduit include some data-health/consistency/sanity/anti-corruption/checksum check?

At some time I had problems with notes on iOS, in that there were titles and content mixed up. Editing such a note would then of course "cement" such local problems into the database. It was unnerving to see new notes with mixed up titles and content. That kept me off using iOS all together, since I could not trust the data that was copied to the device. One could argue that one could always go back with note history, but for that one has to first see that there was a problem in the first place. In some notes this may be obvious, in others not. (I reported this issue and tried to work with support on finding the problem, but the issue was closed without getting to the bottom of it; much later I read in some release notes that such an issue was fixed.)

(There are also rumors of such problems occurring on other platforms, as well rumors of lost resources and the like. I suspected I had some lost resources sometimes, but it seems that this was a case of a resource being integrated into the note through a hyperlink rather than through embedding the resource into the note, so one could blame the particular clipping/copy/paste method (when copying from web sites) rather than a database problem; still, the issue is annoying when one goes offline with a computer for an extended time and finds that notes intended to be complete are not. But redesigning Conduit may not help there.)

Somebody is now going to suggest me to make backups and rely on note history, but the reason for using Evernote is so I don't have to rely on my own set-up, records, and fuzzy memory.

Anyway, it would be interesting to see whether note-health is now checked before viewing  or any edit starts. 

[CalS 5,310]

8 hours ago, lhb said:

Somebody is now going to suggest me to make backups and rely on note history, but the reason for using Evernote is so I don't have to rely on my own set-up, records, and fuzzy memory.

You should only need backups for two things; 1) local notebooks should you have any, and 2) recovering an accidentally deleted note.  If you don't have those issues all should be well.  

[jefito 5,598]

26 minutes ago, CalS said:

You should only need backups for two things; 1) local notebooks should you have any, and 2) recovering an accidentally deleted note.  If you don't have those issues all should be well.  

Well, #3 would be the unknown-but-nonzero possibility of Evernote going out of business...

[CalS 5,310]

3 hours ago, jefito said:

Well, #3 would be the unknown-but-nonzero possibility of Evernote going out of business...

Good point.  Though as long as you aren't using On Demand Sync you should be able to backup from the local data base should EN disappear in a nano second.

[lhb 66]

4 hours ago, CalS said:

You should only need backups for two things; 1) local notebooks should you have any, and 2) recovering an accidentally deleted note.  If you don't have those issues all should be well.  

 

3 hours ago, jefito said:

Well, #3 would be the unknown-but-nonzero possibility of Evernote going out of business...

Thanks both of you, and I agree.

But just to remain on track: My primary question was better protection in "conduit" against editing a locally corrupted note and submitting the result, something which has happened for me in the specific case of mismatched header and body.

[jefito 5,598]

5 minutes ago, CalS said:

Though as long as you aren't using On Demand Sync you should be able to backup from the local data base should EN disappear in a nano second.

I'm not, and I do.  

(backup to .enex, that is)

[CalS 5,310]

31 minutes ago, lhb said:

My primary question was better protection in "conduit" against editing a locally corrupted note and submitting the result, something which has happened for me in the specific case of mismatched header and body.

EN is going to have to answer that one.

[Dave-in-Decatur 4,006]

On another topic within the general uniting-the-UX theme: Recently yet another user in the Android forum raised the issue of not being able to write properly in right-to-left scripts (Arabic, Persian, Hebrew, and perhaps others). They broke that on Android several years ago, and frankly they must be leaving a large chunk of money on the table, since the millions of people who write in those languages would find Evernote difficult if not impossible to use. So, without a specific relation to  Conduit that I know of, this problem is one that really ought to be near the top of the list as the capabilities of the editors on various platforms are being brought together.

[salgud 12]

How about making the various apps and the online version more alike. And while you're at it, consolidate nomenclature, like "favorites" and "shortcuts".

[PinkElephant 8,815]

The current online version is the blueprint and testing ground for what the clients will look like. So this is exactly the Plan.

[Jason Trenkler 1]

I like the web version, however still looking for more functionality ... How about something as simple as Titles and Sub-Titles across all apps? 

[PinkElephant 8,815]

This is the plan. The new editor will be rolled out to all clients and apps, bringing this functionality to all EN applications.

[DTLow 5,745]

1 hour ago, Jason Trenkler said:

How about something as simple as Titles and Sub-Titles across all apps? 

Requests should be posted and discussed in the Feature Request forum
I understand notes and Titles   
What do you mean by sub-Titles?   
  
We are expecting UI changes, however I doubt there will be significant changes to the data infrastructure 

[PinkElephant 8,815]

When the old tribes wanted to know what the gods had planned for them, they threw bones into a kettle. From the position they tried to read the mind of their gods, and choose the right action for their well being.

Personally I do not believe that knowing what the gods have planned will do any good for you. So one day I will wake up, and there it is: The brand new EN client, right from paradise .... 

Or as Forest Gump said: Life is like a box of Pralinés: You never know what it will taste until you tried.

[Sterid 0]

Hi, is it possible to try new iOS app? I've already registered as beta user.   

[aukirk 368]

Any chance for an update or new "Behind the Scenes (from Home)"?  It feels like it has been forever since the new web editor beta was released, and it is GREAT... except it is not fully functional (without reminders or ability to easily file notes into notebooks).

I understand things are probably delayed due to COVID-19, but it would be great to provide some transparency and an update on anticipated timing for devoted Evernote users eagerly awaiting an update (for a long time)...

This migration to the common editor is clearly holding up any updates to all apps, as the iPad Evernote app still doesn't take advantage of the ability to open two instances of the same app side-by-side, which seems like a feature built for Evernote.

[Dasym 0]

On 3/11/2020 at 12:54 PM, PinkElephant said:

The current online version is the blueprint and testing ground for what the clients will look like. So this is exactly the Plan.

That's very worrying. The online version lacks lots of features from the Windows and Mac versions, eg. ability to view columns of metadata for the notes listed. I hope Evernote don't go down the route that OneNote has taken where their newer versions have less features than the one that was current over 4 year ago.

[DTLow 5,745]

1 hour ago, Dasym said:

That's very worrying. The online version lacks lots of features from the Windows and Mac versions, eg. ability to view columns of metadata for the notes listed. I hope Evernote don't go down the route that OneNote has taken where their newer versions have less features than the one that was current over 4 year ago.

Dropping functionality is one method if achieving platform parity
We also see signs that Evernote is adding functionality to the other platforms

I think the end result will be a combination of the two approaches

[Oberdada 28]

Have you thought about unifying the sort order on the different platforms? I am particularly concerned with the alphabetical sort order of special characters in tags and note titles.

The background to the question is that many Evernote users use special characters as prefixes, e.g. to identify different types of tags and to display them one after the other in a specific order. Or to display important notes in a certain order at the top when sorted alphabetically. But if, for example, the character "#" on Windows is sorted before the characters "." and "@", but on Android after these two, this destroys the users' order system. It is particularly bad with special characters, which are sorted on one platform before the numbers, on another between numbers and letters and on the third platform after the letters ...

[Chriswaterguy 3]

On 5/6/2020 at 7:23 PM, Oberdada said:

many Evernote users use special characters as prefixes, e.g. to identify different types of tags and to display them one after the other in a specific order. Or to display important notes in a certain order at the top when sorted alphabetically. But if, for example, the character "#" on Windows is sorted before the characters "." and "@", but on Android after these two, this destroys the users' order system. It is particularly bad with special characters, which are sorted on one platform before the numbers, on another between numbers and letters and on the third platform after the letters ...

I'm glad it's not just me using it this way! My way of using Evernote depends heavily on sort order of special characters.

I installed Evernote 10 today and it's now much worse - the sort order seems to vary depending on whether you're in Notebook view or search results (sorting by Title, ascending in both cases). I could try to work it out, but for other reasons (notably the new version being sooo slooow) I'm reinstalling the legacy version.

I hope this gets fixed. 

[organizing 3]

Enjoyed the Behind the Scenes video update.

Does anyone know which language / libraries are being used to build the five clients?

 

[PinkElephant 8,815]

On desktop it is said to be the Electron framework.

https://www.electronjs.org

On mobile something else (it was mentioned somewhere).

[Paul A. 677]

19 hours ago, organizing said:

Enjoyed the Behind the Scenes video update.

Does anyone know which language / libraries are being used to build the five clients?

 

They are using JavaScript on all platforms. They appear to be using the React framework (v16.13) on Web, Electron (presumably combined with React) on desktop, and presumably also React on mobile.

JavaScript tends to be a slower language, but optimizations in both software and hardware in recent years has raised performance to acceptable levels in many cases. However, I am concerned about the experience on Android. Some users like @PinkElephantsay that iOS performance has got quite good as of version 10.5. iOS users likely benefit from the impressive JavaScript performance of recent Apple mobile chips.

In contrast, Android users are typically stuck with Qualcomm chips which have had comparatively mediocre JavaScript performance in recent years. I hope Evernote has a plan to optimize the heck out of their software such that it eventually runs well on Android, but I am a bit dubious. I fear that EN on Android will continue to be an unsatisfying experience until Qualcomm and other Android SOC vendors raise their game on single-core performance, which could take years for that to trickle down to a majority of the Android user base.

[PinkElephant 8,815]

Probably you have a point with the Apple SOCs. We run the new EN on an iPhone 6S+ and on an iPhone 11 Pro Max. The difference in performance is significant, on the 6 it is much slower than on the 11. The chip of the 6S+ has only about 1/10 of the transistors of the 11 (if you believe in Wikipedia, Apple itself does not tell much). The 6 is good for retrieval and an occasional note, the 11 is good for work with EN iOS.

Another aspect is that iPhones are typically running on a current OS. EN needs at least iOS 13.3, which was released a little more than a year ago. But nearly all iPhones will run it, back 5 years of production, starting with the iPhone 6S. So all the software routines will be there to support the EN framework.

This is a very different situation with Android, where many phones are never upgraded, and updates often trickle down to security fixes after a short time. This is a much more diverse environment.