How much metadata do you apply to notes?

May · March 3, 2012

There are tons and tons of attributes and depending on what we're interested in - any attribute of an object is a potential way to group notes... It's practically impossible to define all attributes whenever you process notes in Evernote. And you never really know how you're going to search for the note later and in what case it might be useful and what topic it might be related to. I personally have a lot of thoughts on this topic but would like to know what do you think.

How do you deal with this?

How many notes do you keep? How many tags do you use?

The more the better? the less the better?

What about adding new tags/attributes/any metadata to a new group of notes but not adding those new tags to all your previously created relevant notes because it's just not practical when you have thousands of notes.

What are your thoughts about the best way to use metadata and store and retrieve personal information overall? (Not technically but in general idea.) yeah, I know, it's a very broad question I think it's still a fairly new thing for most people to try and capture all meaningful information in their life so we're all pioneers in this in a way.

do you know any good books, articles on this topic? I'm not interested in newbie/ tips&tricks kind of stuff though but interested in more of a holistic methodology/concepts/models, e.g. Gtd.

May · March 3, 2012

Also another question inspired by recent discussion about using random codes instead of tags to group related notes...

Does every attribute you group notes by has to be a tag? Do you use temporarily tags? I mean you could just use searches all the time and keep all metadata/keywords in the body of notes. And you could just use saved searches and search history to create and navigate groups/categories of related notes instead of using tags at all... So why use tags at all? Where do you draw the line in general? What is the purpose of tags in your view/workflow?

I know GrumpyMonkey for example stopped using tags at all and I find it very fascinating and I'm seriously reevaluating my own usage of tags at the moment

I think using index notes to list some groups of notes is a very viable alternative to tags and it doesn't have to be one or the other, i.e. it's possible to use both tags and index/master notes but each for different purposes...

GrumpyMonkey · March 3, 2012

hi may. mine is a minimalist approach, and as jmichael is fond of pointing out, it fails to take full advantage of what evernote has to offer. i can certainly see it being too stripped down for aome people. it works for me, though.

i think it would be good to hear from jbenson on this, because he uses the random codes and tags. burgersnfries has a lot more notes than me, and she uses tags a lot as well. i am too lazy, but if you could put links to posts where they explain their systems, that might help generate discussion.

May · March 3, 2012

What I have seen people post - is more techical details of how they use Evernote, you know... What I'm interested in is more of a general idea behind their organizational approach, i.e. not how to technically organize stuff but what's the whole idea behind their methodology in general. E.g. When they process a note what thinking process they go through and etc.

May · March 3, 2012

Btw here are some quotes from the book I'm reading at the moment, David Weinberger - Everything is Miscellaneous - The Power of the new Digital Disorder

"the solution to the overabundance of information is more information."

"When we come across the paper photo from 2005 of Aunt Sally on a beach in Mexico at sunset celebrating cousin Jamie’s birthday, with the twins in the background playing badminton, we have to decide which one spot in one album we’re going to stick it into. If it were a digital album, we wouldn’t have to make that choice. We could label it in as many ways as we could think of: Aunt Sally, Mexico, 2005, beach, birthday, twins, badminton, sunset, trips, foreign countries, fun times, relatives, places we want to go back to, days we got sunburned. "

I like this... The question how do you apply this in Evernote, ie would you just tag everything in every possible way? seems like too much work for me.

But that's definitely interesting...

Maybe delegating stuff like this to AI is the future.

BurgersNFries · March 3, 2012

I often include the date in my titles inYYYYMMDD format. I use the 4 digit year b/c I've retroactively added info from prior to 2000. I include it in the title b/c as GM pointed out, if I ever export the data, I'm not relying upon the EN created date. Also, whe retroactively adding info, IMO, it's faster/easier to just type the date than change the created date.

I also include any misspellings, I think I may use when searching for the note. IE, if the note is info about someone with the last name of Shafer, I'll include Shaffer as a keyword.

anjoschu · March 4, 2012

Of Evernote's provided metadata fields, I actively use

- Title (not really metadata)

- Tags

- Creation date (I use this a lot, e.g. for the date of letters I receive or send -- I don't always scan incoming mail the same day it arrives)

- URL (in clippings, obviously, but sometimes also explicitly)

I rely on Evernote to automatically fill these, because I refer to them, but don't set them explicitly:

- Changed date

- Location (on mobile)

I've never used the following:

- Author (I use tags for this)

Oh, and in the body, I often add some search keywords, e.g. splitting up "Einkommensteuererklärung" (tax return statement) into "Steuer" "Einkommensteuer" and "Steuererklärung", and adding stuff I might search for, like "fiscal", to the mix. Often, when I'm capturing a note on mobile, I do this right at the end of the note title, leaving the body empty.

May · March 5, 2012

I would be very interested to know how people with 500+ or even 1000+ tags manage their tags! Please share how many tags do you have and if you create any overviews/index notes to create lists of tags? Do you organize tags at all? Do you ever rename/delete tags?

jbenson2 · March 5, 2012

It's kind of tough to get a count on the number of tags.

Do I ever rename or delete tags? Yes, occasionally.

Even though Evernote makes it easy to rename, I would estimate 98%+ of my 850 tags have the original spelling.

I might have deleted 20 tags over the past couple years.

The number of my personal / employment related tags is probably around 150.

The majority of my tags apply to my "hobby" of collecting political news - tags for names of politicians, locations, and political issues.

.

May · March 6, 2012

How do you tag notes? For example do you just tag each note with everything that comes to mind and the more tags you add the better? What if you forget to add all relevant tags to a note at first, do you add more tags later? Do you put any limit on how many tags per note you use?

jbenson2 · March 6, 2012

I assign the relevant tags when I create the note.

I only add tags that I consider appropriate.

Most notes will have 2 to 4 tags. Some notes can get up to a dozen (especially if several politicians are mentioned)

I don't have a limit on the number of tags.

I also have a note of saved searches that I run once a month or so that will look for missing tags. Tedious, but it finds notes I missed.

Here are 3 for Obama.

notebook:Politics Obama -tag:"Obama Barack"

notebook:Politics "President Obama*" -tag:"Obama Barack"

notebook:Politics "Obama administration" -tag:"Obama Barack"

.

May · March 6, 2012

Thanks, I like this approach, it makes sense even though could be tedious... What do you think about adding a feature to define synonyms in Evernote database?

it could be as simple as a plain txt file in the database folder or online "note" in the web client of Evernote or something... Would probably be relatively easy to implement,it's nothing groundbreaking.

Basically it would define synonyms for search. E.g. You could define Obama Barack = Obama; president obama; obama administration or vice versa and etc.

Or another example - father = dad and etc.

So for example each time you search for "dad" you would also get the results with the word "father". This way you'd just define synonyms once and wouldn't have to manually add them to each note all the time or do custom searches for missing tags/keywords.

jbenson2 · March 6, 2012

Super idea.

May · March 8, 2012

Some more of my thoughts on this topic, i.e. tagging and organizing in general:

First of all Metadata is essentially data about data.

Metadata is infinite - metadata itself is data, it is possible to create metadata about metadata, metadata about metadata about metadata and so on.

In digital world everything could be both data and metada though, depending on the context...

Data and metadata can change their roles... the only distinction between metadata and data is that metadata is what you already know and data is what you’re trying to find out.

From David Weinberger:

There used to be a difference between data and metadata. Data was the suitcase and metadata was the name tag on it. Data was the folder and metadata was its label. Data was the contents of the book and metadata was the Dewey Decimal number on its spine. But, in the Third Age of Order (see the previous issue), everything is becoming metadata.
Now take a closer look at these information objects. They look like contents tagged with lots of metadata, but in fact they're all metadata. If I'm looking for an article about hydrocarbons written by Barbara Rodriguez, then the article's topic ("hydrocarbons") and author's name ("Rodriguez, Barbara") are metadata, and the content is the data. But, I could just as well be trying to remember the name of the author who wrote an article that included the phrase "Hydrocarbons are the burros of the the cosmos" sometime in the 1960s, in which case the content and date are metadata and the author's name is the data. What's data and what's metadata depends on the person doing the asking.
So, in the Third Age of Order, all data is metadata. Contents are labels. Data is all surface and no insides. It's all handles and no suitcase. It's a folder whose content is just another label. It's all sticker and no bumper.

So what?

Now here is an interesting thought - is there any difference between notes and tags then? in Evernote there is always a difference between them technically, but in concept sometimes there is no difference. And techical difference between them in Evernote doesn't really matter much.

Notes are both data and metadata because you can use any word or phrase from a note to find that note. So for example if you have an article about an iPad you don't need to tag it with the word "iPad" or "tablet" or "apple" and so on because those words are already in the article anyway. Any word in the article could be already used as tag in search.

Tags are metadata in most cases. They describe the information, what the note is about.

A tag is metadata when you know it and use it to find the data. However a tag could also be data when you don't know the tag and want to find the tag. For example you could use content from a note to find a tag, so in this case you used the note as metada and the tag as data.

So there is no real difference between the note and its tag because either could be both data and metadata.

There is still technical difference between them in Evernote though, for example you can assign tags easier and faster compared to typing keywords manually in the body of a note. And of course there is a separate list for tags and a separate list for notes but both lists become fairly useless when you have thousands of notes/tags so manually navigating/browsing those lists to find anything is not a viable option anyway.

The main advantage of using tags instead of just keywords in notes is that tags are just more convenient. They are always user defiened only but you could also just type some rare characters, e.g. "###this is tag [tag name]" in notes or random codes to define tags

other than that there is not much difference between tags and notes.

Btw this leads me to the idea of organizing notes and tags in Evernote differently. Notes could be used to organize tags/categories just as well as tags/categories could be used to organizied notes. There are no limitations at all in terms of structure basically. This is something I'm experimenting at the moment... It's something I overlooked in the past because I was used to the concept of organizing with tags in Evernote... But it's an artificial limitation. There is another viable option, i.e. to use notes as metadata to organize tags/categories into hierachical or non-hierarchical networked structure. It's a workaround; it's crazy; but it works.

It might seem confusing but what is causing this confusion is that we think about tags and notes in a certain way but in reality there is no single way to define/categorize them and they could change their roles at any time or they could be both things at the same time and etc., e.g.

A quote from David Weinberger

“People keep pretending they can make things deeply hierarchical, categorizable, and sequential when they can’t. Everything is deeply intertwingled.” So said Ted Nelson, the eccentric visionary who coined the term hypertext in the mid-1960s. In the third order of order, information not only becomes intertwingled, intertwingularity enables knowledge. And unique identifiers enable intertwingularity—although there can be so many unique identifiers for the same thing and at various levels of abstraction that the identifiers are all a-twingle also.

May · March 8, 2012

Does this make any sense at all btw? I think I'm onto something with this...

logandb · March 9, 2012

It does make sense. Thinking about how I assign and use tags, I agree that you may be onto something, but I'm not sure what it is just yet!

I have 3000 notes, made up of around 500 for work and 2500 personal. I have around 80 tags defined. Taking away the 20-25 used for categorising scholarly articles and recipes, I'm only using 55-60 for the bulk of my notes, at no more than two levels of nesting. My goal when assigning tags has always been to try to fit the note into an existing tag. If it doesn't fit, then I assign it the 'Misc' tag. Every so often I browse the Misc set and try to identify any similarities that might warrant a new tag.

However, I've been finding lately that ad hoc searches within my five notebooks yields the results I need the majority of the time. If I get too many search results to find the note I'm after, I might refine the search with one or more tags to make the list a little shorter. The only time I really need tags is if the note itself doesn't yield sufficient information retrievable from a search - notes consisting of photos alone (with no OCR'able text) are one example of this.

Perhaps, given that 'data' and 'metadata' are becoming indistinguishable, we should be referring to 'added data' instead. Each note's data, whether it be a photo, a pdf, an audio note or text, is only retrievable if we remember enough about it in order to find it again. The paradox I see is that while Evernote 'remembers everything', we do not, but we do need to remember *something* in order to retrieve *everything*. With a large number of notes, as EN encourages, there is no way of browsing the entire list to find a particular note. Therefore, we need to remember enough about the note required that we can provide a search term, tag or date range that yields an manageably browsable search result to fill the gap between what we remembered and what we want to find. So, when we create a note, we have to predict what 'added data' is required to find it in the future. The nature of this added data is highly individual and undoubtedly related to how our individual minds work. Some people use comprehensive tags (@jbenson2 if I recall) or dates in titles (@BurgersNFries?), while others put extra information into the note itself. The Evernote designers have tried to anticipate the nature of 'added data' by providing for tags, locations, dates of creation/updating based on their perception of how human memory works and we the users try fit our own memory schemas into this structure.

I'm thinking that a study of how people work store and retrieve in Evernote would make a brilliant research study....

May · March 9, 2012

I like your example with the "misc" tag. I also used to do something similar, except instead of misc tag I just didn't add any tags at all and then searched for notes without tags and looked at similarities to tag/group them with.

However when you really think about it - what's the purpose and outcomes of this?

Why do we want to organize things in the first place?

There are different purposes, e.g. with mind mapping, outlining - we use those tools mostly to understand things and etc. rather than to just find them later and while Evernote could also be used for brainstorming/outlining - it's not really what it's designed for.

In case of Evernote we usually organize things to find them later.

Now I think there are basically 2 main ways to find things - searching and browsing... Both are useful in different cases.

Tags/keywords are generally flat (even though nesting is possible, it's limited in usefulness and it limits you to organize tags in only a single way, more on that later) and are most useful for searching.

But sometimes you don't know what you're looking for and discovering what you want is at least as important as finding what you know you want.

So basically whenever we organize things in Evernote - we organize them for 2 reasons - to find them later by searching and/or by browsing.

But here is a interesting realization - whenever you use just tags to organize notes with the purpose of finding them later by browsing you're going to inevitably run into problems! I think one of the main reasons people want to organize notes with tags is to define some categories and then to browse those categories and to discover what they want, i.e. to discover things when they don't really know what they're looking for... in those cases searching is useless since you don't even know what to search for. When we do not know exactly what we are looking for, it is much easier to be able to broaden and narrow our area of interest with some categories than to perform some sort of random walk from idea to idea.

Now you can say that you can organize notes into categories with tags and then browse those tags and find notes by browsing - so what's the problem? You can even nest tags into hierarchies which is supposed to solve the problem of having a lot of tags, right? In practice though it still doesn't work.

Yeah, there is navigation possibility. But it's limited and also it's unhelpful most of the time, especially when you have a lot of tags/categories to browse through which is what you're going to end with anyway unless you just avoid tags/categories (which is also not helpful and doesn't solve the problem when you want to find information by browsing).

The problem with using tags as categories is that there's no way to conveniently organize tags themselves in multiple ways.

It is a problem because every time you organize matters in one way, you are disordering them in others.

Sure, any note can have multiple tags but if you use tags as categories then eventually you would want to organize those categories with parent child relationships as well, i.e. tags and sub-tags.

Otherwise browsing flat tag list is just as helpful as browsing flat notes list. The list itself becomes useless.

So this is why some people tend to avoid/limit tags I guess.

Nesting tags into hierarchies is limited and not helpful because every time you organize matters in one way, you are disordering them in others. It's similar to tagging a note with only a single tag so you have to make a decision about what is the most important attribute of the note. This is a limitation of a physical world though and it has no place in a digital world.

Btw on a side note - there is no one single most important thing in reality at all, everything could be the most important thing, it really depends on the context. For example at this moment when I'm writing this post - it is the most important thing in my life - but after that I'll move on to something else which would become the most important thing.

It's silly when people say that something is the most important thing in their life, e.g. even health or money and whatever. Everything is important, what's the most important thing really depends on the context...

But back to the topic...

Yep, it's impossible to organize everything in every possible way... No one person or group is going to be able to organize it in all the useful ways, hanging all the leaves on all the branches where they might be hung.

but we do still get value from organizing things.

The solution is to give up control.

Things have their places, not a single place. Doesn't matter if they're not in all of their places. The more ways you can find a relevant note the better. Link/group things whenever it's appropriate and helpful, don't attempt to interlink/group everything in every possible way.

Heck even amazon.com or wikipedia doesn't organize everything in all the useful ways but they still have categories and those categories are still very useful. You just don't want to rely on one thing only, i.e. either browsing with links/categories or searching for exact keywords. You want to do both. The more ways to find stuff the better but there is no best/ultimate way to organize everything.

This is how you solve the scalability issue, basically just give up control and perfectionism.

Evernote is good mostly for one way of information retrival - search. Browsing is possible and in fact everything could be done with some imagination.

But it's not possible in Evernote to organize things to find them later by browsing. unless you use workarounds. (which I do use and will write about them later, but in short - it's about using notes to organize tags and tags to organize notes, i.e. using both notes and tags as data and metadata depending on the context, this gives unlimited flexibility and makes it possible to avoid tags limitations).

Each note's data, whether it be a photo, a pdf, an audio note or text, is only retrievable if we remember enough about it in order to find it again. The paradox I see is that while Evernote 'remembers everything', we do not, but we do need to remember *something* in order to retrieve *everything*. With a large number of notes, as EN encourages, there is no way of browsing the entire list to find a particular note. Therefore, we need to remember enough about the note required that we can provide a search term, tag or date range that yields an manageably browsable search result to fill the gap between what we remembered and what we want to find.

"Each note's data, whether it be a photo, a pdf, an audio note or text, is only retrievable if we remember enough about it in order to find it again."

It's true but since metadata is infinite we always remember "something" about anything...

The problem is when you search for "something" that is a only part of the note itself. This is a problem because you might not remember anything from the note itself.

But If you add metadata to the note, i.e. define all the possible categories/topics (you could think of) it relates to and then also organize those topics into broader and narower topics then it's pretty much impossible to not find anything. It's really just a matter of time then.

"manageably browsable search result to fill the gap between what we remembered and what we want to find."

This really nails it in my view, especially when you consider that a search result could also tell you what you might want to search for additionally. In other words the search doesn't have to be for the note exactly, you could search for the information about the note, i.e. its category/metadata and then use this metadata to find the note itself. But to do this you have to use notes to define categories/metadata instead of and/or as well as tags, (this is just based on Evernote design though) this is what I'm experimenting with at the moment.

Might sound confusing but in practice it's actually not, the only confusing thing is when you try to define what a tag is and what a note is because there is no one ultimate way to define what is what, i.e. it depends on who you ask and when..

deverill · March 10, 2012

Hi May,

I would like to reply to some of your questions in hopes it may help in your quest.

I use less than 3 dozen tags, most are abandoned but I haven't removed them yet. You see, I started out with one notebook and tags for "everything". In retrospect, over time it seems I have abandoned most, but not all, tags in favor of notebooks. My current thinking about tags is that a few tags can help group things but adding a tag for every little thing is all but useless. In your example of the photo, IMO, who would benefit from a tag: Badminton? Is that to find pics of us playing, snips from the local store to buy a new set, physics studies of how a birdie flys, etc? It would be useless to me and that is assuming I remembered that we played the game on that trip!

The tags I use now are probably around a dozen, plus a date. I have tags for each month "1 - Jan" and for each year "2012". I also have tags for each member of my family that is relevant and certain select other people that are prominent.

I have a few separate notebooks but for the most part, the majority of them are in a stack called Reference (ala GTD). Within that stack are "Computer Info", "Food", "Legal", "Photography", etc.

I rarely browse my notes. I'd say at least 90% of the time I am looking for something I know that I used to know. Therefore, most of my use in Evernote is searching for some topic. I usually am pretty good about getting to the note I want via search and sometimes timestamp if I know when it was added so I often don't use the more advanced search features.

In the rare occasion I go browsing for something I rely on good titles to help me find it. For instance, I have a notebook of stories and quotes. I will sometimes browse through that to find a pertinent story or quote about something being discussed. I just search for the topic in that notebook or, if the topic is somewhat nebulous I will simply step through the titles to find what I need.

I add scanned documents, emails, etc. into my Inbox and when I move them from the inbox to their appropriate notebook I add the proper tags if applicable. If it is one of my active tags, such as Medical, I'll add that, the month and year tags when appropriate, the family member and anything else fitting such as Taxes. I don't always add month tags, for example, unless the item added is specific to a certain month - like an electric bill.

If I need to find something that happened on a certain month I can do that. I can find things relating to a certain family member. I can find what I need for taxes. etc.

The more tags the better is definitely not my method. Too few is equally unhelpful. Basically, if you need it - use it, but no more than what is necessary and useful.

If I add a new tag I will almost never go back and add it to older notes. The biggest exception to this is, for example, when I started using Month and Year tags - going through old electric bills is worthwhile to have those tags added.

I sure hope this helps in some small way and didn't just waste your time.

Deverill

mattdaddym · March 15, 2012

I'm a noob. First post. Been using Evernote for awhile, but haven't found my moment of Zen yet. I've read most of David Allen's book, seen Daniel Gold's stuff, and have done my share of online research around various methodologies for using Evernote. So, a few comments on what I've seen here so far.

I think we all agree our "system" needs to be just complex enough to fit your needs. I am a bit OCD about things and have certainly suffered from tag-overload. Mommy told me not to touch the hot stove, but I certainly never did it again after I burned myself. Shame on me...live and learn.

Regarding note searching, it would be cool to see a "Did you mean?" or "Related Searches:" suggestion thingy.

Regarding the benefits of tags, for me it is the consistency of them. Believe it or not, one day I might tag something "email" and the next day "mail". There are a million other examples of this. By having a not-too-crazy list of tags, I am sure to use the same name every time. Also, tags can be like an umbrella over the concept of your note. Your note may cover a concept without ever actually using the name/description of that concept. Tagging solves this, but I guess adding this to the note body does as well, so...um.

Some of the ideas/concepts in this thread remind me of an M.C. Escher drawing (data/metadata, tags/notes), lol. This is one of the great things about Evernote. It is as simple or complex as you need it to be. I think beginners need examples and suggestions to help them get started. It's like martial arts for organization. Most of us need to "paint the fence" and "wash the car" before we are ready or can even understand what the true masters recognize and practice.

Regarding my organizational structure, I am experimenting with the who, what, when, where model. I can't remember where I saw this. All of my "real" tags are nested under who, what, when, and where. I know that every note I have should have at least one of each of these tags. However, some things have multiple what and multiple who tags. I've only got about 308 notes at the moment. A note might be tagged like this (!!reference #noWhen .anywhere @noWho). This note has a when, where, and who tag even though it really doesn't need it. I'm sure many will find this offensive For me, it's about following a process for each note I create. The only thing I am certain of right now, is that my approach will evolve

deverill · March 31, 2012

Belated welcome Matt,

I want to reassure you that whether others may think it is offensive or not, the only thing that matters is "Does the system work for you?"! Anyone offended by your system is paying too much attention to others and should keep their noses to themselves. Now a healthy exchange of ideas about how to best do something is good, but condemning someone who wants to do it differently is not.

My "Was on, wax off" (see, I'm old so I refer to the first movie. LOL) way to introduce someone to Evernote is this:

Add everything you can into Evernote. If you ever need it then you can get it. It is easier to just add it than to worry about whether you will ever need it. Then if you have any need to group things consider notebooks or tags, whichever feels best. Then after that you should explore all the corners of the system and refine it to meet your needs and run with it.

I would caution the beginner, however, against adding 30 tags to each note to cover every little thing. For instance, I could add a lab report from my last blood work and tag it: Lab, blood, doctor, cholesterol, iron, sugar, diabetes, HA1C, triglycerides, etc. but the best thing for me is to simply label it: Jim, Lab, 2011, 12 - December. The rest of the information is in the note itself and can be searched when needed.

To me the beauty of Evernote is that it has all of these organizational tools for us to use in whichever way we want.

jbenson2 · March 31, 2012

Regarding the benefits of tags, for me it is the consistency of them. Believe it or not, one day I might tag something "email" and the next day "mail". There are a million other examples of this. By having a not-too-crazy list of tags, I am sure to use the same name every time. Also, tags can be like an umbrella over the concept of your note. Your note may cover a concept without ever actually using the name/description of that concept. Tagging solves this, but I guess adding this to the note body does as well, so...um.

I use a mnemonic trick to help stick with the same tag. Many of my tags have a 3 letter prefix.

http://discussion.ev...dpost__p__80338

I agree with the umbrella concept for using a tag when the word is not actually used. (Nanny_state, Politically_correct)

A single tag also helps when there are different spellings.

For example, for the following 5 spelling variations I use the single tag (tag:"Gaddafi Muammar")

Moammar Gaddafi

Muammar Gaddafi

Moammar Gadhafi

Muammar el-Qaddafi

Muammar al-Gaddafi

JMichaelTX · April 16, 2012

Some more of my thoughts on this topic, i.e. tagging and organizing in general:

First of all Metadata is essentially data about data.

Metadata is infinite - metadata itself is data, it is possible to create metadata about metadata, metadata about metadata about metadata and so on.

In digital world everything could be both data and metada though, depending on the context...

Data and metadata can change their roles... the only distinction between metadata and data is that metadata is what you already know and data is what you’re trying to find out.

From David Weinberger:

There used to be a difference between data and metadata. Data was the suitcase and metadata was the name tag on it. Data was the folder and metadata was its label. Data was the contents of the book and metadata was the Dewey Decimal number on its spine. But, in the Third Age of Order (see the previous issue), everything is becoming metadata...

Hi May, I'm coming late to this party, and I haven't carefully read everything after the above quote.

You pose an interesting question.

However, I have to strongly disagree with your assertions about metadata.

It is a simple matter of definition. I subscribe to your first definition: "Metadata is essentially data about data"

Therefore, by definition, everything is NOT metadata.

Simple example: The author and date of an article, book, etc do NOT change the contents of the article. If you change the author or date, the contents are still just as valid or invalid. The author and date may influence your evaluation of the contents, but the contents stand alone. It is like taking a blind testing of wine. The wine tastes the same regardless of its label.

Next, metadata is NOT infinite. We choose what metadata we think is important.

In the end, that is what really makes metadata important and useful.

We reduce thousands/millions of words down to a few select fields/values that are useful to us.

When you take a digital picture, the camera encodes selected metadata.

When you process the picture later, that is all the metadata you have.

Now you may analyze the picture and determine some technical attributes, but that is NOT the same as pre-defined, provided, metadata.

Having selected, limited metadata is both the benefit and limitation of metadata.

So, when a system is designed, content and metadata are clearly separated.

Don't confuse the ability to do a full text search with metadata.

The metadata that Evernote provides is very, very limited, IMO.

For example, many of us would very much like Evernote to add a metadata field: Due Date.

Metadata is only useful to us in retrieving data if we can remember (or be prompted) the metadata values when making the search.

This is why, IMO, that use of Tags is far superior to use of keywords in the Title.

I can easily get/see my list of Tags to select the tag I want to search on.

I don't know how one does this with Title keywords. That doesn't work with my memory.

peterfmartin · April 24, 2012

I would caution the beginner, however, against adding 30 tags to each note to cover every little thing. For instance, I could add a lab report from my last blood work and tag it: Lab, blood, doctor, cholesterol, iron, sugar, diabetes, HA1C, triglycerides, etc. but the best thing for me is to simply label it: Jim, Lab, 2011, 12 - December. The rest of the information is in the note itself and can be searched when needed.

I'm someone who adds a lot of tags (arguably far too many, based on the utility and efficiency of the time spent adding them and later using them to retrieve notes). I've been trying to think about whether I can trust both Evernote and myself to remember and find tags notes just with keyword searches. While I've been coming around to trusting myself a bit more (for example, now I clip a lot of web pages without tagging them, trusting myself to do keyword searches on them and to add the necessary tags in time), I've been losing trust in Evernote's ability to find keywords in notes. Those of you who rely on keyword searches have never had problems of Evernote not returning all notes correctly?

jbenson2 · April 24, 2012

I'm getting older so I let Evernote tags help me in the memory arena.

For instance, the cable modem service went down this morning. I know it went down a couple times earlier this month, so before calling the tech people, I searched for Internet. No luck. Then I searched for tag:Com-Charter and found the 2 other mentions (I had typed in the term "cable down").

Back up and running again.

May · May 2, 2012

Hi May, I'm coming late to this party, and I haven't carefully read everything after the above quote.

You pose an interesting question.

However, I have to strongly disagree with your assertions about metadata.

It is a simple matter of definition. I subscribe to your first definition: "Metadata is essentially data about data"

Therefore, by definition, everything is NOT metadata.

Simple example: The author and date of an article, book, etc do NOT change the contents of the article. If you change the author or date, the contents are still just as valid or invalid. The author and date may influence your evaluation of the contents, but the contents stand alone. It is like taking a blind testing of wine. The wine tastes the same regardless of its label.

Next, metadata is NOT infinite. We choose what metadata we think is important.

In the end, that is what really makes metadata important and useful.

We reduce thousands/millions of words down to a few select fields/values that are useful to us.

When you take a digital picture, the camera encodes selected metadata.

When you process the picture later, that is all the metadata you have.

Now you may analyze the picture and determine some technical attributes, but that is NOT the same as pre-defined, provided, metadata.

Having selected, limited metadata is both the benefit and limitation of metadata.

So, when a system is designed, content and metadata are clearly separated.

Don't confuse the ability to do a full text search with metadata.

The metadata that Evernote provides is very, very limited, IMO.

For example, many of us would very much like Evernote to add a metadata field: Due Date.

Metadata is only useful to us in retrieving data if we can remember (or be prompted) the metadata values when making the search.

This is why, IMO, that use of Tags is far superior to use of keywords in the Title.

I can easily get/see my list of Tags to select the tag I want to search on.

I don't know how one does this with Title keywords. That doesn't work with my memory.

metadata is data about data but anything could be data about some other data...

For example if you want to find out a book's title/author/date it was published/etc. and search for some words/quotes from that book - you used the book's content as metadata(label) and the book's title/author/etc was actually data.

The author and date of an article, book, etc do NOT change the contents of the article.

I see your point. But it doesn't change that anything could be metadata, e.g. author is data about book (metadata) but the book itself could also be data about author, as in my previous example (hence, it also could be metadata)

Yes, from the software's(Evernote) perspective tags, notebooks and etc. are always metadata and notes are always data(I guess). But from the user's perspective anything can be both data and metadata regardless of the format it's stored in.

I can store a list of categories/tags/ in plain notes instead of relying on theTag List (just an example). It'd be data from Evernote's perspective, but not necessarily from my own perspective.

Metadata is something that you already know. Data is something that you want to find. What is data and what is metadata depends on who you ask and when.

jefito · May 2, 2012

Metadata is something that you already know. Data is something that you want to find. What is data and what is metadata depends on who you ask and when.

This is not necessarily true. Metadata is just data attached to other data. I may not know what the exact date I created a note (created date is metadata, is it not?), but might be interested to know what notes I created in a certain date range. I can find out by querying against the created date metadata.

BTW, it may be of interest to read Wikipedia's definition of metadata: http://en.wikipedia.org/wiki/Metadata. There are apparently a couple of different meanings. The first one is the one I use in this context.

JMichaelTX · May 2, 2012

However, I have to strongly disagree with your assertions about metadata.

It is a simple matter of definition. I subscribe to your first definition: "Metadata is essentially data about data"

Therefore, by definition, everything is NOT metadata.

Metadata is something that you already know. Data is something that you want to find. What is data and what is metadata depends on who you ask and when.

May, I don't think we can have an informed discussion until we agree on terms and definitions.

You have stated several different definitions of "metadata".

As I stated above, I subscribe to this definition: "Metadata is essentially data about data"

Here's a quote from TechTerms.com:

Metadata describes other data. It provides information about a certain item's content.

For example, an image may include metadata that describes how large the picture is, the color depth, the image resolution, when the image was created, and other data. A text document's metadata may contain information about how long the document is, who the author is, when the document was written, and a short summary of the document.

Web pages often include metadata in the form of meta tags. Description and keywords meta tags are commonly used to describe the Web page's content. Most search engines use this data when adding pages to their search index.

I don't see any value in your last definition: "Metadata is something that you already know. Data is something that you want to find. What is data and what is metadata depends on who you ask and when"

May · May 2, 2012

I don't see any value in your last definition: "Metadata is something that you already know. Data is something that you want to find. What is data and what is metadata depends on who you ask and when"

Don't confuse the ability to do a full text search with metadata.

a search term is metadata (data about data which you want to find).

You're confusing data/metadata and organizing information for **browsing** or **searching**. It doesn't matter how the data/metadata is organized, I mean the way we organize information doesn't define what is data and what is metadata from the user's perspective.

Web pages often include metadata in the form of meta tags. Description and keywords meta tags are commonly used to describe the Web page's content. Most search engines use this data when adding pages to their search index.

Yeah, but I'm not talking about what's metadata for a search engine, I'm talking about a user's perspective.

May · May 2, 2012

Metadata is something that you already know. Data is something that you want to find. What is data and what is metadata depends on who you ask and when.

This is not necessarily true. Metadata is just data attached to other data. I may not know what the exact date I created a note (created date is metadata, is it not?), but might be interested to know what notes I created in a certain date range. I can find out by querying against the created date metadata.

BTW, it may be of interest to read Wikipedia's definition of metadata: http://en.wikipedia.org/wiki/Metadata. There are apparently a couple of different meanings. The first one is the one I use in this context.

I don't see any contradiction, I mean you used a date range instead of a certain date as metadata... the date range is something that you know (metadata) and notes within this range is something that you want to find(data)...

If your point was that you didn't know metadata then who entered the date range as a search query?

I may not know what the exact date I created a note (created date is metadata, is it not?)

You're missing my point...

Created date is not always metadata. Note is not always data. Sometimes it's data, sometimes it's metadata, it depends on how you look.

As you said it yourself:

Metadata is just data attached to other data

"created date" is data attached to a note. But it also works the other way around, i.e. a "note" is also just data attached to a "created date" data. Hence my point about what's data or metadata really depends on who you ask and when.

If you use note's content to find out the date it was created on then the "created date" is data and the note itself is metadata.

JMichaelTX · May 2, 2012

Yeah, but I'm not talking about what's metadata for a search engine, I'm talking about a user's perspective.

The definition of metadata is the same for both. The user may not know the term "metadata", but they understand the difference between the content they are looking for, and the data which describes that content.

The definition that both jefito and I quoted has been in use for a long time, and is well-accepted within the computer and IT communities.

In fact, the basic question you posed with this thread, "How much metadata do you apply to notes?" has no meaning if "metadata" and "data" mean the same thing.

May · May 2, 2012

JM, my "definitions" don't contradict each other. The problem with "Data about data" or "data attached to data" (by jefito) definition is that anything could be data about (or attached to) some other data, hence "the only distinction between metadata and data is that metadata is what you already know and data is what you’re trying to find out. " ( a quote from David Weinberger)

It's not a definition but it's the only distinction.

I agree my thread title could be reworded to "how much additional data you manually add to Evernote notes in order find them later by search or browsing"

JMichaelTX · May 2, 2012

@May: Well, then it appears that we will have to agree to disagree.

May · May 2, 2012

What are you disagreeing with exactly?

check this out

There used to be a difference between data and metadata. Data was the suitcase and metadata was the name tag on it. Data was the folder and metadata was its label. Data was the contents of the book and metadata was the Dewey Decimal number on its spine. But...

http://www.hyperorg.com/backissues/joho-oct15-04.html#data

JMichaelTX · May 2, 2012

The definition of "metadata".

May · May 2, 2012

I see... Metadata is data about data. I haven't contradicted it though.

You're missing my point because you don't consider relying on search as a way to organize information.

You consider organizing for browsing only in this case.

When you organize for browsing then you have to think carefully about which metadata you'll capture because the physical world limits the amount of metadata you can make available.

But with **search** everything could serve as metadata. I.e. every word in a book could serve as metadata, instead of just author/title/etc.

.

JMichaelTX · May 2, 2012

I see... Metadata is data about data. I haven't contradicted it though.
You're missing my point because you don't consider relying on search as a way to organize information.
You consider organizing for browsing only in this case.
When you organize for browsing then you have to think carefully about which metadata you'll capture because the physical world limits the amount of metadata you can make available.
But with **search** everything could serve as metadata. I.e. every word in a book could serve as metadata, instead of just author/title/etc.
.

May, you are drawing inferences and conclusions about my thoughts that are simply wrong.

You may want to pose these type of assertions as questions.

IAC, I don't see how we can have a meaning discussion, or communicate clearly, unless we can agree on terms and definitions.

The accepted standard definition of metadata does NOT make any assumptions about how it is used (browsing vs searching).

The fact that one can search for both metadata, and for text contained in the content, does NOT make metadata and data the same.

A "word in a book" is NOT metadata. It is part of the content. Just because I can search for it does NOT make it metadata.

May · May 3, 2012

A "word in a book" is NOT metadata. It is part of the content. Just because I can search for it does NOT make it metadata.

You're disagreeing with the definition of metadata then ("data about data").

a quote from

http://www.thebusinessintelligenceguide.com/data/Data_Defintion.php

"In simple terms, Metadata is essentially data about data. A more sophisticated defintion may be - Metadata is structured, encoded data that describe characteristics of information-bearing entities to aid in the identification, discovery, assessment, and management of the described entities."

"Data and metadata can change their roles - a poem would be regarded as data, but if used as lyrics for a song, the entire poem could be attached to an audio file of the song as metadata. Thus data / metadata is very much contextual in defintion."

You're basically not accepting some "data about data" as metadata...

I mean, suppose I want to find out the name of some song... And I use a word from this song lyrics (which is not metadata according you) to find out the song's name/artist's/etc.

In this example a word from a song is data about some other data, i.e. metadata. The same applies to books and so on.

or in other words:

suppose I want to make a list of artists (for whatever reason) and I'm going to use all of their song lyrics as metadata to find those artists later, why not?

This wouldn't work in a physical world because physical world limits the amount of metadata you can make available (since searching is impossible). But in a digital world metadata can be expansionist. It can include the entire text of the book.

And it works both ways, i.e. I can use artist to find lyrics or I can use lyrics to find artist. Hence data / metadata can change their roles.

Does this make sense?

IAC, I don't see how we can have a meaning discussion, or communicate clearly, unless we can agree on terms and definitions.

I think one could say that indexed contents, e.g. song lyrics or words from a book - is low quality and unstructured metadata and there is also high quality structured metadata.

would this work for you?

a quote from David Weinberger:

"Traditionally, metadata has been used to help people find data. The metadata on a card in a library catalog is there to help patrons locate books on shelves. Metadata could do little more because metadata was a reduction of information: A 600-page book is boiled down to a few facts that fit on a 3x5 card.

But in the digital world, there's little reason to boil things down. Metadata can be expansionist. It can include the entire text of the book. It can include the text of a biography of the author of the book. It can include a map of the places the author of the biography of the author once visited. Why not? Suppose someone wants to find the book written by the subject of the biography written by that guy who used to live in Slough?"

another quote:

"If I'm looking for an article about hydrocarbons written by Barbara Rodriguez, then the article's topic ("hydrocarbons") and author's name ("Rodriguez, Barbara") are metadata, and the content is the data. But, I could just as well be trying to remember the name of the author who wrote an article that included the phrase "Hydrocarbons are the burros of the the cosmos" sometime in the 1960s, in which case the content and date are metadata and the author's name is the data. What's data and what's metadata depends on the person doing the asking."

In a connected world all information is interrelated, hence everything is metadata.

People keep pretending they can make things deeply hierarchical, categorizable, and sequential when they can’t. Everything is deeply intertwingled.” So said Ted Nelson, the eccentric visionary who coined the term hypertext in the mid-1960s.

May · May 3, 2012

Ok, so here is another example of how data / metadata change their roles.

Suppose I have a list of to do items with due dates.

I could ask: "What **tasks** do I have?" and then look at my to do items. In this case tasks are data and their (due) dates are metadata (data about my tasks).

However I might just as well be asking a different question, e.g. "What is due **tomorrow**?" and then look at the list of all of my to do items which are due tomorrow. In this case a due date (tomorrow) is data and my to do items are metadata (i.e. data about the date, or in other words what needs to be done until tomorrow). This is how calendar works.

As you can see I can organize information for different purpose and in different ways, e.g. I can organize tasks by projects/contexts. Or I can add tasks to a calendar.

In a digital world I can do everything at the same time, of course, i.e. I can input projects, contexts, dates, tasks and then filter everything in any way I want. There is no single correct hierarchical way to organize and look at any information.

This also applies to everything else, e.g. books, articles, song lyrics and so on.

JMichaelTX · May 3, 2012

@May: I've already stated several times that I disagree with your definition:

In a connected world all information is interrelated, hence everything is metadata

Sorry, but until/if we agree and the definition of metadata, I don't have any more to say on this subject.

May · May 3, 2012

That's fine. I'm actually trying to explain things (from point of view) instead of asking anything at this point.

After providing many examples, articles and book quotes - I don't know what more could be said without repeating it over.

I mean all you do is just disagree without providing anything other than a definition of metadata (which I neither contradict nor disagree with). Hence, I don't think we even disagree, it's more like you're just not seeing my point.

JMichaelTX · May 3, 2012

I mean all you do is just disagree without providing anything other than a definition of metadata (which I neither contradict nor disagree with). Hence, I don't think we even disagree, it's more like you're just not seeing my point.

I don't understand this. I just quoted your definition of metadata which is in direct conflict with the standard definition provided by both jetito and myself.

I don't see your point. It appears to me that you are confusing how one searches for data with how one structures the data.

Let's bring this back to Evernote.

In EN, ONLY the Note Contents field is the actual "data". All of the other EN fields, like Tags, Title, Created Date, are *metadata*.

No matter how you search or browse, this remains the same.

The Note Contents field is what we are actually recording as the primary object.

All of the other fields simply describe some attribute of Note Contents.

The fact that you can search for text in the Note Contents does NOT make Note Contents metadata.

I don't see any way this can change.

It *is* a matter of definition. Metadata describes the primary data object. Period.

May · May 3, 2012

It *is* a matter of definition. Metadata describes the primary data object. Period.

Yeah, but there is no primary data object in reality. What's primary depends on who you ask and when.

there is no single correct hierarchical way to organize and classify stuff. That's the whole point. Information doesn't have a map.

Things have many different places not a single place. Even data / metadata change their roles. It doesn't contradict anything. it just means one thing could be many different things depending on how you look at it.

"Why does this matter? It changes the primary job of information architects. It makes stores of information more useful to users. It enables research that otherwise would be difficult, thus making our culture smarter overall."

"Because a third-order miscellany is digital, not physical, we no longer have to agree on a single framework. Things have their places, not a single place. We get to create our own categories, ones that suit our way of thinking. "

"The basic fact that order often hides more than it reveals has sometimes itself been hidden within the art and science of organizing our world. "

JMichaelTX · May 3, 2012

there is no single correct hierarchical way to organize and classify stuff. That's the whole point. Information doesn't have a map.

I didn't say anything about hierarchical organization. I don't think you are reading what I am writing, or at least that is how it appears to me.

I already stated that the segregation of info into metadata and data does NOT imply nor assume any type of data organization.

As you know Tags are metadata that can be used to find data that has NO hierarchical relationship.

Things have many different places not a single place. Even data / metadata change their roles. It doesn't contradict anything. it just means one thing could be many different things depending on how you look at it.

I don't know what this means. You say that you are not contradicting the definition of metadata, yet you state in the same sentence "Even data / metadata change their roles".

You ignored my request to bring this back to Evernote. I get the idea that you are discussing information management theory. It may or may not be practical or useful, but in any case I don't see how it applies to Evernote.

Evernote has a defined database, which has primary data (Note Contents) and metadata (all the other fields).

Maybe others are interested in debating the definition and use of metadata with you, but I really don't have anything further to say on that subject.

This is why I stated earlier that we need agree to disagee about the definition of metadata.

If there is something that directly applies to Evernote then I might be interested.

May · May 3, 2012

Maybe others are interested in debating the definition and use of metadata with you, but I really don't have anything further to say on that subject.

I'm not interested in this either and I'm not the one who started arguing about data / metadata definition...

If there is something that directly applies to Evernote then I might be interested.

This whole topic applies directly to Evernote, you just focused on the data / metadata definition aspect, which is fine, but it wasn't my main point at all.

How this applies to Evernote? In many ways...

For example:

1. Any organizational structure could be created and all you really need is just notes and search, that's it.

Evernote provides special fields for inputing structured metadata which is very useful but that doesn't mean it must end there and that it's all you can do to organize Notes.

Not only note contents **could** organize notes themselves sufficiently in the presence of a good search. But you can also use notes themselves to organize metadata such as categories/tags/what to search for/etc. instead of relying on auto-generated Tag/Notebook/etc. lists and avoid some of their limitations. (just an example).

More importantly:

2. There is no single correct way to organize anything.

"Give up control. Build a tree and you surface information that might otherwise be hidden, just as Lamarck exposed information left hidden in Linnaeus’s miscellaneous category of worms. But a big pile of miscellaneous information contains relationships beyond reckoning. No one person or group is going to be able to organize it in all the useful ways, hanging all the leaves on all the branches where they might be hung. "

This means Notebooks and hierarchical tags in Evernote Tag List (while still being useful) are still too limited. Hierarchy implies that there's a single correct way to organize information when in reality information doesn't have a single correct map.

It's not something that you can put on a map, or organize "correctly" in the Tag/Notebook List with hierarchically nested tags/notebooks.

JMichaelTX · May 3, 2012

@May: the "who started it ..." is, well, just irrelevant.

More importantly:
2. There is no single correct way to organize anything.

This means Notebooks and hierarchical tags in Evernote Tag List (while still being useful) are still too limited. Hierarchy implies that there's a single correct way to organize information when in reality information doesn't have a single correct map.

But some ways are better than others.

IMO, embedding metadata in the Note Contents field is NOT as efficient and easy to use as designing the metadata as separate fields.

If by your statement you mean each person needs to determine what works best for him/herself, I agree.

Well, this is your thread. Have you gotten out of it what you wanted to get?

May · May 3, 2012

But some ways are better than others.

There is no better in this case, I was talking about organizing things topically and not technically.

There is no single "proper" or "the best" place to put anything into. Things have their places, not a single place.

Well, this is your thread. Have you gotten out of it what you wanted to get?

Yes and no

lol

JMichaelTX · May 3, 2012

Maybe you want to look up the definition of "better". It is a *relative* term, not an absolute.

May, I am just not understanding you, or your logic.

So I don't think I have anything further to contribute to this thread.

Good luck.

May · May 3, 2012

Maybe you want to look up the definition of "better".
May, I am just not understanding you, or your logic.

I can totally see that... I've tried to explain myself better but it was futile in this case since you miss my relevant points and focus on the definitions of data / metadata (and that a note content itself can't be metadata by definition) and then focus on technical aspects of using specific Evernote features.

That's not what I'm talking about. This is not a thread about using keywords in notes vs tags (Similar to threads like MBA vs iPad).

This is more about a general idea behind any organizational system overall.

It's not just me or my logic that you're not understanding since I'm providing lots of quotes from David Weinberger and other sources as well.

It's hard to **get** this stuff regardless of the language it's communicated in.

I mean if you don't get some of my points - I can live with that

JMichaelTX · May 3, 2012

May, if you can give me a concrete, but simple, example in Evernote of applying your theory, then I will listen.

peterfmartin · May 3, 2012

I'm a total noob about all of this, but it's been interesting to read the arguments in this conversation, if not the back and forth about who's being reasonable and who's being unreasonable (so I've skipped a number of the posts). Let me share one thing I've gotten out of this discussion.

In the analog age, when very little data was searchable, metadata was organizable, searchable data that would help people locate data, which was non-searchable. The simplest example is something like the Dewey Decimal System: A book's content was its data, and its Dewey classification number was its metadata. Why? Because only the latter was searchable. Books were ordered by number, so someone could say, "I want to find a book about Portuguese literature" and find it by the Dewey system. There was no way to search content, meaning no one could search book content to find books; no one could say, "I want to find all books with Dewey numbers of 869.52" and find them by searching for "Portuguese literature." it was impossible to consider the content metadata that would help readers find anything else.

If I understand him right, May's argument is that now, with digitized, searchable content, a user can search content, which traditionally could not be metadata, to find information that traditionally could only have been metadata, since it was the only searchable data. He has given lots of examples of this, but one more I can think of is the GrumpyMonkey school of organizing: GrumpyMonkey adds data to the "content" field to find other data in the same fields. Thus, some of his data in those fields is metadata—if it all isn't, if we don't subscribe to the May school of what metadata is.

I think it's pretty clear why JMichael is insisting there are clean lines dividing data and metadata: because once that was true, and a lot of digital information (like that in Evernote) is inspired by analog models where data and metadata were totally separate things. And I think it's pretty clear why May is saying that that distinction isn't true anymore: because now, with everything searchable, any data can be used as metadata, used to find the target data.

I don't think you two will ever agree on the other's definition, but I do think you two can understand each other.

May · May 3, 2012

@peterfmartin, I'm glad you get what I've been trying to communicate.

I totally understand JM's point about data / metadata, it's clear and simple. It makes everything seem simple and easy to define and classify. The only problem is that real world is not like that and it misses the point that I'm making.

I even provided additional classification of metadata, e.g. high quality and structured metadata - this would be tags in Evernote for example, and low quality unstructured metadata (data about data in general) - this would be indexed content of everything, which means it'd be basically everything because in a connected world all information is interrelated in some way.

Both are useful. I'm not suggesting to give up structured/user defined metadata such as tags just because we can use search btw, just to be clear.

The data / metadata definition itself doesn't really matter.

it just makes some points about how we can organize things. E.g. with a good search we can always filter on the way out instead of trying to organize everything in all of the useful ways on the way in.

And that we don't have to rely on any special field to input "metadata".

It just makes everything much more flexible and means that there are no limitations in terms of organizing notes into any kind of structure, that's all.

I think a lot of people don't see this when they start using Evernote and/or want to replicate some organizational structure which they are used to. Any organizational structure is possible in Evernote even with just notes and search, but you have to let go of old concepts and ways of thinking, or in other words concepts of the physical world.

JMichaelTX · May 3, 2012

@peterfmartin, I'm glad you get what I've been trying to communicate.
I totally understand JM's point about data / metadata, it's clear and simple. It makes everything seem simple and easy to define and classify. The only problem is that real world is not like that and it misses the point that I'm making.

There are plenty of examples in the real world:

Books
- Author
- Publisher
- Publication date
[*]Cars
- Manufacturer
- Model
- Manufacture date
[*]Photos
- Photographer
- Date
- Location

Any organizational structure is possible in Evernote even with just notes and search, but you have to let go of old concepts and ways of thinking, or in other words concepts of the physical world.

I'm not sure that "any organizational structure is possible", or perhaps I should say that not all organizational structures are practical.

We are constrained by the search engine grammar/syntax.

Perhaps you would like to provide an example of your assertions.

May · May 4, 2012

a book is not always a primary object and its author is not always metadata. Sometimes an author is the primary object and sometimes a book is.

examples I provided previously

Ok, so for example with books, if you use the book itself as the "primary object" you will run into problems eventually because there is no ultimate "primary object" in real world.

Book

- author

- author biography

- author's other books

- book 2

- coauthor

- coauthor's daugther

- pictures of coauthor's daugther on a vacation in Thailand

When you try to organize information into some single hierarchical structure with some single ultimate "primary object" it eventually doesn't make any sense.

That's because information itself doesn't have a map. There is no single "proper" or "the best" place to put anything into. Things have their places, not a single place.

All information is interrelated in a connected world.

Yes, as long ad you use a closed and very limited system in which you have only books and their authors as metadata and that's it, then JM example works.

But when you add more information such system breaks. E.g. I just burried information about "pictures of coauthor's daugther on a vacation in Thailand" under some "book" which is not the primary object anymore and it's not helpful. Not to even mention that basically the whole thing just doesn't work.

I will write about how I organize information in Evernote in general and into any kind of structure and etc. but it's too long for me to write in a forum post at this moment. I don't think I can provide just some quick example to explain everything without oversimplifying or overcomplicating things.

JMichaelTX · May 4, 2012

May, I was not describing a hierarchy, but giving examples in real life where we have objects (Books, Cars, Photos) and we have attributes that describe them.

I guess we are having a definition of terms issue again.

In the *real* world we deal with *primary* objects all the time. The primary object is the one we have chosen to focus on at the moment.

So, I can tell you that I have new TV. For now, that's the primary object, or, the object with focus.

I can then describe the TV, by listing some of its attributes:

HD
Black border
60-in
Panasonic
3D

Yes, "Panasonic" can also be the object of focus, and we can describe it. But then we would be changing our focus to another object.

When we are talking about a Book I understand it also has contents.

So if one wanted to design a system for storing electronic books, we can include the books actual contents, as well as attributes that describe the book (author, publisher, etc). If we use a SQL DB, we can search on both specific fields (the attributes) as well as the full text of the book.

Both have their uses.

For example, I can use the Author field to search for all books by a specific author, let's say "John Smith"

My results would be 100% accurate (assuming I had entered all books into the DB).

However, if I search book contents for the author's name, I might return a lot of books, some which were NOT written by the author.

In fact, the results might leave out some books by "John Smith" because the author's name does not appear in the book contents.

There could be other books whose contents referred to "John Smith" in citations or quotes, or even a list of books.

So if I did not have a specific field for "Author" it would be hard, maybe even impossible, to get just a list of books by a specific author.

I run into this issue all the time on Amazon.com. When I enter an author's name into the Amazon search block, I get books by the author, but other books as well. Sometimes it is not even clear why the other books were returned.

Same thing is true with Google. If I search Google for "John Smith" I get much, much, much more than just a list of his books.

I can even search for "Books by John Smith" and I still get much more than just his books.

So my whole point is that there are uses and benefits to being able to search BOTH the full text content as well as specific fields.

But I grow tired of theoretical discussions.

This is an Evernote forum, so can we focus on how all of this relates to Evernote?

Here's my Evernote example: Due Date.

While we can enter the "Due Date" in the Note contents or Title, it is not the same as, nor as good as, having a specific field for "Due Date".

Without a "Due Date" field, we cannot accurately:

Find a list of Notes due within a given range
Find a list of Notes due after a given date
Find a list of Notes due on a given date
Find all Notes that even *have* a due date
etc.

So May, If you have some clever new theory of how to do all this better, please give us an example.

May · May 4, 2012

well, I think we're on the same page now. Almost.

May, I was not describing a hierarchy, but giving examples in real life where we have objects (Books, Cars, Photos) and we have attributes that describe them.

When you say that one entity is an object and another entity is its attribute - you're using a strict hierarchy right there already. Again, book and author could be both either an attribute or a primary object depending on who you ask and when.

But other than that...

I.e.

So my whole point is that there are uses and benefits to being able to search BOTH the full text content as well as specific fields.

and I said previously

1. Any organizational structure could be created and all you really need is just notes and search, that's it.
Evernote provides special fields for inputing structured metadata which is very useful but that doesn't mean it must end there and that it's all you can do to organize Notes.
Not only note contents **could** organize notes themselves sufficiently in the presence of a good search. But you can also use notes themselves to organize metadata such as categories/tags/what to search for/etc. instead of relying on auto-generated Tag/Notebook/etc. lists and avoid some of their limitations. (just an example).

Both are useful. I'm not suggesting to give up structured/user defined metadata such as tags just because we can use search btw, just to be clear.

I never said that we should give up structured metadata such as tags and etc. We should be using high quality structured metadata whenever it's useful in addition to making sure the content is organized for search.

This is an Evernote forum, so can we focus on how all of this relates to Evernote?
Here's my Evernote example: Due Date.
While we can enter the "Due Date" in the Note contents or Title, it is not the same as, nor as good as, having a specific field for "Due Date".
Without a "Due Date" field, we cannot accurately:
Find a list of Notes due within a given range
Find a list of Notes due after a given date
Find a list of Notes due on a given date
Find all Notes that even *have* a due date
etc.

This is an example of filtering and not organizing btw. I said that there are no limitations in terms of organizational structure and I never said anything about filtering.

Filtering would always have some limitations (unless maybe there is some sort of artificial intelligence added)

You can't filter information in any imaginable way regardless of what software (Evernote or anything else) you're using (at this point).

I can provide lots of examples like this, but here is a much more simple example

Show me all notes which are tagged with "tagA" or "tagB" and not "tagC". Evernote can't do even that. (in a single step and without workarounds)

Due date field can't be replicated in Evernote conveniently either.

That's why I use Omnifocus for task management instead of Evernote. Omnifocus is just much better at filtering actionable information, it has not only due dates but also start dates, repeating projects (in any imaginable way), sync with calendar, special dedicated forecast mode, review mode and etc. you can filter and also display and sort everything **conveniently** by project or by context and date at the same time and etc. etc.. While it's possible to replicate most of this functionality in Evernote - it's far less convenient and practical.

Now whether Evernote should add "due date" functionality and/or other specific fields is a good question.

You can do most things with tags but task management is not Evernote's strong point.

I think good filtering tools (and this is a task many are working on, not just Evernote) are essential to be able to filter things on the way out instead of trying to organize everything in way which we're used to in a physical world.

jbenson2 · May 4, 2012

Now whether Evernote should add "due date" functionality and/or other specific fields is a good question.

You can do most things with tags but task management is not Evernote's strong point.

The Evernote CEO still has hopes for Due Dates.

mcd78 - How are Due Dates coming along

Phil Libin - @mcd78 We're putting in a lot of work on reminders / due dates / to-do lists. Aiming for a release this year, but don't have the specifics nailed down yet.

JMichaelTX · May 4, 2012

well, I think we're on the same page now. Almost.
May, I was not describing a hierarchy, but giving examples in real life where we have objects (Books, Cars, Photos) and we have attributes that describe them.
When you say that one entity is an object and another entity is its attribute - you're using a strict hierarchy right there already. Again, book and author could be both either an attribute or a primary object depending on who you ask and when.

Well, I'm not sure if we have language problem (Russian/English) or a technical terms problem.

But we have a completely different understanding of the term "hierarchy".

Do you have any experience in designing databases?

Object-Attributes (or Entity-Attributes) are NOT part of a hierarchy.

Hierarchy refers to different records (of the same type) in the same DB table that have a parent-child relationship.

For example:

Folders and Subfolders
Notebooks and SubNotebooks

A Folder and its date are NOT a hierarchical relationship.

1. Any organizational structure could be created and all you really need is just notes and search, that's it.

You keep making this assertion, but I don't buy it.

How can you (as an end-user) create *any* organizational structure in Evernote?

Not only note contents **could** organize notes themselves sufficiently in the presence of a good search. But you can also use notes themselves to organize metadata such as categories/tags/what to search for/etc. instead of relying on auto-generated Tag/Notebook/etc. lists and avoid some of their limitations. (just an example).

I disagree. You can't do this in a reliable and accurate way.

Whatever text you might type into a Note as "metadata" might also exist in other notes (think of a large PDF) as normal text content.

So you have no way to execute a search that looks ONLY at your "metadata".

This is an example of filtering and not organizing btw. I said that there are no limitations in terms of organizational structure and I never said anything about filtering.
Filtering would always have some limitations (unless maybe there is some sort of artificial intelligence added)

Data structures (organizing) and searching (filtering) are closely linked.

When you design a DB, you must consider how you will want to search the DB.

So, if I have key attributes, like "Author", then I must create a DB field for "Author" so that I can reliably search/filter on it.

Using a full-text search can be helpful, but you cannot rely on it for accuracy (as I stated in my previous post).

May · May 4, 2012

Well, I'm not sure if we have language problem (Russian/English) or a technical terms problem.

Nah, you're just missing the point

But we have a completely different understanding of the term "hierarchy".

Hierarchy:

"Noun:

A system or organization in which people or groups are ranked one above the other according to status or authority."

Aren't you using a hierarchy here?

There are plenty of examples in the real world:

Books

Author

Publisher

Publication date

JMichaelTX · May 4, 2012

May, you often tell both GM and me that we are missing the point.

Have you considered that it is you who is missing the point, or not understanding what we are saying?

Or that you are not clearly expressing yourself?

You didn't answer my question: Do you have any experience in designing databases?

JMichaelTX · May 4, 2012

Hierarchy:

"Noun:

A system or organization in which people or groups are ranked one above the other according to status or authority."

Aren't you using a hierarchy here?

There are plenty of examples in the real world:

Books

Author

Publisher

Publication date

Did you even think about the quote you made????

'People" or "Groups" are ranked ....

So, "People" are ranked.

"Groups" are ranked.

In both cases the things being ranked are of the same type. Do you get that?

"Books" and "Author" are of two different types.

In order to be in a hierarchy, the objects MUST be of the same type, and be in the same DB table with a parent-child relationship (this is rank)

Do *you* get this point? It is very important.

You still have not answered my question about your DB design experience.

May · May 4, 2012

"Books" and "Author" are of two different types.

information is just information. I'm not talking about any specific database design.

When you say that some information is an attribute of some other information - you already use a hierarchy (in general).

Other than that we're not really disagreeing on anything.

Did I say that :

we should give up structured metadata just because we can search? No.

hierarchies should be avoided in all cases? No.

Evernote shouldn't improve its search/filtering ? No.

In fact I did stress that all of those things are very useful. But you just keep disagreeing with me, hence I say that you just miss the point. (and drag out the discussion for no benefit)

You still have not answered my question about your DB design experience.

I used FileMaker for a project I had 2 years ago. (for example)

http://www.filemaker.com/products/filemaker-pro/

I'n not using it currently because I don't have very specific needs and Evernote is better for managing all general information, (instead of just specific information related to a specific project) and Omnifocus is also better for task management and GTD.

May · May 6, 2012

May, on 04 May 2012 - 06:29 AM, said:
"Not only note contents **could** organize notes themselves sufficiently in the presence of a good search. But you can also use notes themselves to organize metadata such as categories/tags/what to search for/etc. instead of relying on auto-generated Tag/Notebook/etc. lists and avoid some of their limitations. (just an example)."

I disagree. You can't do this in a reliable and accurate way.
Whatever text you might type into a Note as "metadata" might also exist in other notes (think of a large PDF) as normal text content.

So you have no way to execute a search that looks ONLY at your "metadata".

First of all I never said that notes contents **do** organize them suffciently in all cases. I said that note contents **could** organize notes sufficiently.

**Could** indicates a possibility and not something that must be done (AFAIK).

But having said that, In practice - you can actually make it as reliable and as accurate or as broad and general as you want.

For example, suppose I have a category "ways to organize information".

Now suppose instead of using a tag "ways to organize information" to group all notes related to this topic - I'm just going to write down a phrase "ways to organize information topic" in each note.

And then also instead of relying on the automatically generated Tag List I'm going write down and organize categories in separate notes with ".metanote - " in titles.

you have no way to execute a search that looks ONLY at your "metadata".

I do have.

I actually tested this with Google which has far more content than 100 000 notes that you can store in Evernote. And even with Google I can get 100% reliable results at all times.

Here is why:

So basically if I search for an exact phrase "ways to organize information topic" then I'm not going to see any unrelated notes **at all** even if I'm using Google and searching all internet (not just some large PDFs or whatever)... And that's even without inventing any special unique keyword or random codes/etc.

And I can also search for any word or phrase in my categories only (i.e. search for structural metadata only).

For example If I search in notes with ".metanote - " in titles then I get exactly the same functionality as search for Tags only.

Now, just to be clear, I'm not saying that we should avoid tags or anything like that, and I'm personally still using Tags because they're more convenient to add and rename.

But this is just an example of how one could create **any** organizational structure (hierarchical or networked) with just notes and search and that's it.

(As long as one's not relying on specific software features to hold his hand at all times of course.)

Notes could be both data and metadata.

Can you seriously say that that notes themselves can't be or can't contain "data about data" (metadata)? Think about it.

Everything is just information. Notes are just information. Information could be anything. This is important because it means that there are no limitations in terms of organizing notes into any structure.

May, on 04 May 2012 - 06:29 AM, said:
This is an example of filtering and not organizing btw. I said that there are no limitations in terms of organizational structure and I never said anything about filtering.

Filtering would always have some limitations (unless maybe there is some sort of artificial intelligence added)

Data structures (organizing) and searching (filtering) are closely linked.
When you design a DB, you must consider how you will want to search the DB.

So, if I have key attributes, like "Author", then I must create a DB field for "Author" so that I can reliably search/filter on it.

Using a full-text search can be helpful, but you cannot rely on it for accuracy (as I stated in my previous post).

At least you're not disagreeing here. Yes, organizing and searching are closely linked.

When you design a DB, you must consider how you will want to search the DB.

This is correct.

I just provided an example of how you can organize information into any kind of structure. You can organize it in any way you want.

This doesn't mean any one person can organize everything in **all** of the useful ways and then be able to filter in **any imaginable** way.

Hence, my point that

Filtering would always have some limitations (unless maybe there is some sort of artificial intelligence added)

I mean there's no magic, I can't conveniently search/filter and ask for things like "show me all movies which I haven't watched and which are also available on dvd with Milla Jovovich and rated higher than 6 on IMBD" unless I have organized and defined all of those attributes.

But do I need any particular specific software feature such as Tags to define all of those attributes?

No. I can organize it with just notes and search and then ask such questions and get 100% reliable results (for example).

I can't do the "rated higher than 6" though but you can't do it with Tags or any other specific field either unless you have some semantic search functionality.

JMichaelTX · May 7, 2012

May, we have had some good discussions. However, I don't have anything further to contribute to this thread.

In order to communicate, especially about technical matters, it is essential that we have agreement on key terms, and on the fundamental principles of the technology involved. You have redefined "metadata" and "hierarchy" in a way to fit your needs, but totally inconsistent with the well-accepted definition by IT professionals, and you don't seem to understand the basic principle of entity-attributes used in database design.

See below for some examples.

So, I don't see how we can have a meaningful discussion.

Good luck.

Metadata is something that you already know. Data is something that you want to find. What is data and what is metadata depends on who you ask and when.
This is not necessarily true. Metadata is just data attached to other data.
BTW, it may be of interest to read Wikipedia's definition of metadata: http://en.wikipedia.org/wiki/Metadata. There are apparently a couple of different meanings. The first one is the one I use in this context.

"Books" and "Author" are of two different types.
information is just information. I'm not talking about any specific database design.
When you say that some information is an attribute of some other information - you already use a hierarchy (in general).

When you say that one entity is an object and another entity is its attribute - you're using a strict hierarchy right there already.

Hierarchy:
"Noun:
A system or organization in which people or groups are ranked one above the other according to status or authority."
Aren't you using a hierarchy here?
There are plenty of examples in the real world:
Books
Author
Publisher
Publication date
Did you even think about the quote you made????
'People" or "Groups" are ranked ....
So, "People" are ranked.
"Groups" are ranked.
In both cases the things being ranked are of the same type.
"Books" and "Author" are of two different types.
In order to be in a hierarchy, the objects MUST be of the same type, and be in the same DB table with a parent-child relationship (this is rank)

May · May 7, 2012

You have redefined "metadata" and "hierarchy" in a way to fit your needs

I disagree with this. I use standard and accepted definitions. You're just missing the point that data / metadata is contextual in definition. One thing could be both data / metadata depending on the context.

What you're saying is that metadata is always metadata regardless of the context and vice versa, thus you're using a strict hierarchy to classify information.

What I'm saying is that some information could be both data and metadata depending on how you look at it. This doesn't contradict anything because I'm not using a strict hierarchy to classify information.

totally inconsistent with the well-accepted definition by IT professionals

Again, a quote from thebusinessintelligenceguide

http://www.thebusinessintelligenceguide.com/data/Data_Defintion.php

It is often difficult to distinguish between raw data and metadata because:
Something can be data and metadata at the same time - the headline of an article is both its title (metadata) and part of its text (data).
Data and metadata can change their roles - a poem would be regarded as data, but if used as lyrics for a song, the entire poem could be attached to an audio file of the song as metadata. Thus data / metadata is very much contextual in defintion.
Metadata is infinite - metadata itself is data, it is possible to create metadata about metadata, metadata about metadata about metadata and so on. For this reason, it is essential to archive metadata about metadata, to keep track of where the metadata originated from when merging two documents.

I mean I just provided an example of how you can organize anything into any kind of structure (for both browsing and searching) with just notes and search and without relying on anything else.

This clearly showed how one can use notes as both data and metadata.

Something can be data and metadata at the same time - the headline of an article is both its title (metadata) and part of its text (data).

But you're still insisting that notes themselves can't contain any metadata...

my #1 reason for participation in such discussions is to contribute and to help people. But sometimes it leads to lengthy (and mostly unproductive) debates.

I don't see how saying that notes themselves can't contain some "data about other data" (metadata) (because this goes against definition or whatever) adds anything useful or leads to learning new ideas and refining old ones.

How much metadata do you apply to notes?

Recommended Posts

May 268

Link to comment

May 268

Link to comment

GrumpyMonkey 4,319

Link to comment

May 268

Link to comment

May 268

Link to comment

BurgersNFries 2,407

Link to comment

anjoschu 67

Link to comment

May 268

Link to comment

jbenson2 2,147

Link to comment

May 268

Link to comment

jbenson2 2,147

Link to comment

May 268

Link to comment

jbenson2 2,147

Link to comment

May 268

Link to comment

May 268

Link to comment

logandb 227

Link to comment

May 268

Link to comment

deverill 54

Link to comment

mattdaddym 2

Link to comment

deverill 54

Link to comment

jbenson2 2,147

Link to comment

JMichaelTX 4,117

Link to comment

peterfmartin 221

Link to comment

jbenson2 2,147

Link to comment

May 268

Link to comment

jefito 5,589

Link to comment

JMichaelTX 4,117

Link to comment

May 268

Link to comment

May 268

Link to comment

JMichaelTX 4,117

Link to comment

May 268

Link to comment

JMichaelTX 4,117

Link to comment

May 268

Link to comment

JMichaelTX 4,117

Link to comment

May 268

Link to comment

JMichaelTX 4,117

Link to comment

May 268

Link to comment

May 268

Link to comment

JMichaelTX 4,117

Link to comment