(Archived) Implementing Offline storage in Evernote 3

[panther.jd 3]

One of the things I like about 3 is that it's online. MY notes are all backed up without me doing anything, and I can access the notes from any browser, add notes from my phone, and it's all synced to my account without me doing anything.

But I can see that some do need the ability to store the notes offline without the data going on the net. So I thought a thread on the best way to do this would be good. The Evernote team can take the best ideas from here to make it better.

A lot of people have said that there should just be a check box in the client on each note when you make it, however it strikes me that this might not be the best way to to implement it. You would end up with a notebook that has some notes on the web, and some notes off, I think it could lead to problems with managing your notes.

In fact I think combining offline and online mode into Evernote 3 will create two types of problems, that of managing the offline and online data, and the problem how to handle offline and online data in the interface.

First, let's think about how the information is handled on the PC. Right now, if you run the client, you have a database file with all of you notebooks in it on the PC which is synced to your account by the client. I think that if you want offline notes, the client should create a new database for it. This will make it easier to manage your offline data, you'll know it's all in the one file. But, to make this work smoothly for the user, the client will need to load both the online and offline databases, and present all the information to the user.

In fact, you could allow more than two database files to be loaded. Now you may say that there's no need, since you can just have multiple notebooks in each database, but in some instances, this could be a useful power feature. Imagine that you want to share some offline notebooks within a work group, or you want to separate offline work and personal data... By allowing the client to simultaneously load an arbitrary number of databases, one online and the others offline, you can have a different set of notebooks for each purpose, however you want to do it. You could even choose to load only offline databases if that suits you.

*********

Now let's think about the interface a little. We want the user to be presented will all of their data, regardless of where it is. At the same time, it should always be clear to the user whether a note is offline or online and (this is particularly important) whether a new note is going to an online or offline database.

Now if we go with the what I suggested above, allowing an arbitrary number of databases, then on startup, the client would need to load all of the databases. If a database is not in the default location, the user would need to tell Evernote in the options where the database is (such as a database stored on a network computer for off the net sharing within a workplace). If some of the databases are password protected, the the client would prompt for a password and offer to save the password.

But what if you want to load different set's of databases at different times? Now you need profiles. In this case a profile is a set of databases. The profile could have all of the passwords in it, and be password protected itself so you only need to remember one password instead of all the passwords for all the different databases you use. The client will have a default profile that just loads the online notes, so users who don't need this level of controle don't have to deal with the complexity of managing profiles.

Now let's think about how all off and online notes would look sitting together in the client.

One easy way to make the difference clear is by color one color for online and another for offline. You could also allow different colors for different databases. (If you want to do this for notebooks too, it could get confusing figuring out the difference between off and online, it might be better to give the different databases a different shape of border, or some other design element to make what is what clear.)

When it comes time to add a note, the user would need to select which notebook it should go in, and here the client will need to make it clear to the user which notebook is selected, and whether that notebook belongs to an off or online database. With the web clipper, the user can either set a notebook as default or have the clipper ask which notebook to use.

Well, that's just some ideas for how offline and online could live together. Maybe you have an idea on how to make it easier & better.

[engberg 89]

Thanks for the detailed feedback. We agree that it's important that people can choose whether their notes should be synchronized to the service or only stored on their local system. I think we tried to achieve your objective by offering both "private" and "local" notebooks.

If you want your notes to be available across the web and multiple computers, you can put your notes in a "private notebook" that is fully synchronized with the Evernote service. This gives you a backup of your data, and allows you to access your notes from any web browser, mobile phone, etc.

If you have notes that you only want to access from a single computer, you can put them into a "local notebook", which is never synchronized with the service. This does not give you the availability and redundancy of a private notebook, but gives you an option for some of your notes which may better meet your privacy needs.

(Of course, there's even another privacy option available ... you can put notes into a private notebook for availability, but then encrypt sensitive text like passwords so that they can only be accessed by someone who knows your secret passphrase.)

As you mention, we should represent "private" and "local" notebooks differently in our client UI. We're working on this for an upcoming release -- these notebooks should be visually distinct so you can tell which are which.

New notes are also created in the notebook that you have currently opened ... so if you're working in your "Cooking" notebook, you can clip new recipes into that notebook without having to manually categorize each one.

Thanks again for the feedback!

[Elbert 0]

I know everybody's needs are different, but the last thing in the world I want is a bunch of different databases. I'd like the option to have EN3 automatically sync what's on the local computer with the on-line database when I start EN3, and again when I exit from EN3. If I enter some notes at home, I'd like to be able to access them at work. If I forget to sync, I won't be able to.

[Citizen_d 0]

I know everybody's needs are different, but the last thing in the world I want is a bunch of different databases. I'd like the option to have EN3 automatically sync what's on the local computer with the on-line database when I start EN3, and again when I exit from EN3. If I enter some notes at home, I'd like to be able to access them at work. If I forget to sync, I won't be able to.

Doesn't EN3 already do this (auto sync)?

Thanks dengberg - I was looking for the way EN3 distinguishes data to be uploaded, and local only data, and was hoping this was at a notebook level. An example of how I plan to use this is that I use EN3 at work and want a notebook per client where I can clip client related info - both private emails and/or even generic web based. I want these client notebooks to be local to my work PC only. Then I have notebooks for things that I want to access at home too and aren't sensitive or work related. Looks like I'll be able to do this. Looking forward to the visual differentiation of them too.

-CD

[tokjdm 0]

I don't know if this makes any sense but I would find it easier if this was handled through tags. For instance, a tag "offline" would ensure that a note is never put online, a tag "personal" would ensure that the note is never shared, etc...

[s2sailor 2,193]

I created a notebook by importing my 2.2 database. I'm assuming by default this is private, but I would like to make it local. I don't see a notebook option to change this. Is there one? Thanks.

[engberg 89]

I created a notebook by importing my 2.2 database. I'm assuming by default this is private, but I would like to make it local. I don't see a notebook option to change this. Is there one? Thanks.

If you have imported data into a private (synchronized) notebook, and want it in a local notebook instead, you should create a local notebook and then move your notes from the private notebook to the local one. That will permanently remove the notes from the service, and they will only be available in that local notebook on that computer.

(This may not work 100% correctly on the current Mac beta ... we're checking that and will have it working right in the next build. Windows should work fine.)

[Citizen_d 0]

I don't know if this makes any sense but I would find it easier if this was handled through tags. For instance, a tag "offline" would ensure that a note is never put online, a tag "personal" would ensure that the note is never shared, etc...

Then you would have inconsistency in notebooks as you would have the notebook available everywhere (synced with the service) but notes would drop in and out depending on what PC you were using and what was tagged as private for that PC. A mess.

A question I have for the EN3 guys - is it possible to transfer local notebooks? E.g. I know I'm getting a new PC and I want to transfer my sensitive local notebook to the new machine. Is there a nice wasy way to do this? Is it achieved with the Export option?

-CD

[tokjdm 0]

Yes but I am not sure that creating separate notebooks is necessary. It creates a new level of complexity and you would still need to remember the specific properties of each notebook. Working with tags is simpler in my view and should give you more flexibility.

[engberg 89]

A question I have for the EN3 guys - is it possible to transfer local notebooks? E.g. I know I'm getting a new PC and I want to transfer my sensitive local notebook to the new machine. Is there a nice wasy way to do this? Is it achieved with the Export option?

We're going to be introducing a cross-platform export file format that would allow you to do this. (You could also use it as a "back up" of some local notes.) This will be supported in both the Mac and Windows client before the end of the beta.

Thanks

[shimra 14]

I don't know if this makes any sense but I would find it easier if this was handled through tags. For instance, a tag "offline" would ensure that a note is never put online, a tag "personal" would ensure that the note is never shared, etc...

Then you would have inconsistency in notebooks as you would have the notebook available everywhere (synced with the service) but notes would drop in and out depending on what PC you were using and what was tagged as private for that PC. A mess.

A question I have for the EN3 guys - is it possible to transfer local notebooks? E.g. I know I'm getting a new PC and I want to transfer my sensitive local notebook to the new machine. Is there a nice wasy way to do this? Is it achieved with the Export option?

-CD

I think you can actually copy the notebook file and put it on the new pc in the default directory after you install Evernote on the new pc. I had luck copying the notebook file to transfer non synching notebooks to Evernote portable when I tested it.

[siddalr 0]

I created a notebook by importing my 2.2 database. I'm assuming by default this is private, but I would like to make it local. I don't see a notebook option to change this. Is there one? Thanks.

If you have imported data into a private (synchronized) notebook, and want it in a local notebook instead, you should create a local notebook and then move your notes from the private notebook to the local one. That will permanently remove the notes from the service, and they will only be available in that local notebook on that computer.

(This may not work 100% correctly on the current Mac beta ... we're checking that and will have it working right in the next build. Windows should work fine.)

I tried this:

1. Right-click on Notebooks, select 'New Notebook', specify that it should be Local, and give it a name.

2. Select all notes in the Notebook imported from vsn 2.2 (right-click, Select All or Ctrl+A)

3. Drag and drop all the selected notes into the newly created Local notebook.

4. Delete the old (now empty) Private notebook.

5. Ensure Account/'Work Offline' is not ticked and click the 'Synchronize' icon.

Results

I also had a second Private (synchronised) notebook with the same name as my Evernote web login in addition to the notebook I had imported from Evernote 2.2. This notebook was still listed in my PC client but it contained zero notes - they had been lost.

Synchronisation occurred and percent completion was displayed. When 100% complete, I logged on to the web version of Evernote. As expected, Evernote had removed the private (synchronised) Notebook I had deleted and my Local notebook was not listsed. The Private notebook bearing the name of my login ID was displayed, containing zero notes.

Comment

I consider the Local (unsynchronised) Notebook feature to be essential as I wish to keep tight control over some notes (e.g. bank details) and would not risk putting them on a web server no matter how good the security. Access via an HTTP connection would also make data vulnerable to interception. The limitation of Local notes is clearly that they would not be available via a web browser or be able to capture information from mobile devices such as cameras.

Presumably the loss of the notes contained in the notebook bearing my EN web login ID is a consquence of all notebooks being held in a single file.

What would happen if I had not deleted my old (now empty) notebook? Presumably synchronisation would have restored all the notes from web Evernote?

A notebook 'Properties' box would be useful. One property could be notebook disposition: Local (unsynchronised) or Private (synchronised) - and anything else. Changing the disposition from Private to Local could be made to remove synchronisation and delete all associated data in web Evernote. Changing the disposition from Local to Private could be made to mark the notebook to be synchronised with web Evernote. A 'Warning' screen could be displayed explaining the consequences of the user's action before he commits.

[engberg 89]

When you move a note from a private (synchronized) notebook to a local notebook, it converts that to a local note and the server will remove it from its original notebook the next time you sync. This means that you don't really need to delete the notebook at all ... you just move the notes and sync to get the note to be local.

You're right that the UI doesn't make it clear which notebooks are local vs. private vs. public. This should be improved soon.

Thanks for the feedback

[siddalr 0]

You're right that the UI doesn't make it clear which notebooks are local vs. private vs. public. This should be improved soon.

Thanks for the response. One of my notebooks in the PC client (the one I referred to in my earlier post, auto-generated with a name derived from my account name) is displayed slightly inset with a black dot in front of it. Why is its name displayed differently and what is its significance?

[engberg 89]

One of my notebooks in the PC client (the one I referred to in my earlier post, auto-generated with a name derived from my account name) is displayed slightly inset with a black dot in front of it. Why is its name displayed differently and what is its significance?

I think you're looking at the "Default" notebook. This is the notebook that will receive new notes that you create when you don't have a specific notebook selected ... for example, if email a note in to your account. It's sort of like an "Inbox" for notes.

[Toodle 0]

One of my notebooks in the PC client (the one I referred to in my earlier post, auto-generated with a name derived from my account name) is displayed slightly inset with a black dot in front of it. Why is its name displayed differently and what is its significance?

I think you're looking at the "Default" notebook. This is the notebook that will receive new notes that you create when you don't have a specific notebook selected ... for example, if email a note in to your account. It's sort of like an "Inbox" for notes.

I've actually named my 'default' notebook as "--Inbox--". That way it appears at the top of the list and is easily recognizable as something other than a typical notebook.