(Archived) REQUEST: Boolean searches

[Delaware 0]

This is my first post. Thank you for Evernote. I especially enjoy the ability to use it on my desktop, iPhone, and the Web. It would be much more useful if one could do Boolean searches, as InfoSelect offers.

[jefito 5,589]

Not all of us know about InfoSelect. Could you please provide some examples?

~Jeff

[engberg 89]

Thanks for the feedback. For Evernote's consumer service, we use global intersection of criteria (AND) by default, but you can switch the queries to use global union (OR). This is the same level of complexity offered by many other consumer applications such as iTunes smart folders. Most consumer applications don't mix in arbitrarily complex boolean logic because it is hard to make a good UI that shows the state of the query that has been executed. A global union or a global intersection tends to meet the needs of 99.5% of consumers without any complicated query-constructor UI, etc.

Thanks

[BurgersNFries 2,407]

Most consumer applications don't mix in arbitrarily complex boolean logic because it is hard to make a good UI that shows the state of the query that has been executed.

BTW, I LOVE the ribbon in the Win 3.5 client that shows what the query is. If something looks goofy, that's one of the first things I check, to make sure I'm looking for what I think I'm looking for.

[engberg 89]

Thanks - that was actually one of the things we spent the most time on when we were building the service, although we were stealing a lot of ideas from predecessors like the Flamenco folks at Berkeley:

http://orange.sims.berkeley.edu/cgi-bin ... l/Flamenco

http://flamenco.berkeley.edu/

After spending a lot of time on usability testing recently, I'm not completely convinced that the "search explanation" is completely intuitive for new users, but it's probably the one UI thing we did at launch that was trying to be a little innovative.

[crane 40]

This is the same level of complexity offered by many other consumer applications such as iTunes smart folders. Thanks

One thing I like about iTunes smart folders/playlists is that you can actually use other smartlists in queries. In other words, you can build arbitrarily complex search expressions, combining all three boolean operations, AND/OR/NOT.

[Delaware 0]

Thank you, engberg, for your prompt reply. Very impressive responsiveness. After reading your post, I tried entering a search of two words without the "and," and it worked! Problem solved! I hope this helps those who have trouble with the technical language, as I do.

[nmjk 4]

One thing I like about iTunes smart folders/playlists is that you can actually use other smartlists in queries. In other words, you can build arbitrarily complex search expressions, combining all three boolean operations, AND/OR/NOT.

This is what brought me to the forums today in the first place. The roadblocks I've hit with the lack of tag inheritance and the low complexity of boolean searching would be solved, so I hoped, by the ability to refer to other saved searches in the search grammar. I now consider this my #1 request because this ability would allow me to hack together pretty much everything else I need.

[hyloguy 0]

Dave Engberg said: "For Evernote's consumer service, we use global intersection of criteria (AND) by default, but you can switch the queries to use global union (OR)."

Dave, how do I do that? I see nothing in Preferences about switching the queries to use union instead of intersection.

EDIT: Never mind, I figured it out. I wanted to save a search that would match either one tag or another tag. Solution query:

any: tag:fitness tag:health

[Big Cloits 3]

I just spent a half hour going nuts because I foolishly assumed I could mix logic in an Evernote search. Not so much. It’s amazing how often I’m frustrated by this limitation. I really should not have been so surprised I couldn’t do it in Evernote.

Grouping and mixed logic really needs to be there, in this and many other apps: the availability of the required syntax is invisible and harmless to beginners, but its absence is exasperating to those of us with the dazzling mental prowess required to understand something AND (this OR that).

[mentalplex 2]

Despite the snark, paulingraham is right.

I'm really not sure why it would be hard to make a good UI for more complex search queries. People do it all the time. The NCBI website is a good example. If you're worried about maintaining that slick, streamlined design, just use a dialogue box for the 'advanced search'. Users who want complex search options won't mind the added visual clutter of a dialogue box, and users who don't want the feature won't have to see it.

The comparison with "most consumer apps" doesn't make sense either. The iTunes comparison is especially weird. If you're not Rob Gordon from High Fidelity, the useful relationships between music files are simple. With Evernote's flexibility and notebook/tag organization, it's BUILT for creating a "life-database" with complex relationships between files. An application that builds a complex database but can't run complex searches just doesn't make sense. The power of the flexible design is wasted.

[rfbriggs 1]

Dave Engberg said: "For Evernote's consumer service, we use global intersection of criteria (AND) by default, but you can switch the queries to use global union (OR)."

Dave, how do I do that? I see nothing in Preferences about switching the queries to use union instead of intersection.

EDIT: Never mind, I figured it out. I wanted to save a search that would match either one tag or another tag. Solution query:

any: tag:fitness tag:health

Could someone please elaborate on how this is working? Here's what I'm trying to do:

I am trying to get my tax forms together for the year. I have all of my year-specific files tagged "2011". I also have tags for "taxes" and "donations". So what I tried to do was click on the 2011 tag in the browser, then do a search of "tag:donations OR tag:taxes". Didn't bring me back anything, even though each of those tags individually works within the 2011 tag.

Is this really not possible to do? Is there some other way to do the OR operation?

Thanks.

[BurgersNFries 2,407]

Dave Engberg said: "For Evernote's consumer service, we use global intersection of criteria (AND) by default, but you can switch the queries to use global union (OR)."

Dave, how do I do that? I see nothing in Preferences about switching the queries to use union instead of intersection.

EDIT: Never mind, I figured it out. I wanted to save a search that would match either one tag or another tag. Solution query:

any: tag:fitness tag:health

Could someone please elaborate on how this is working? Here's what I'm trying to do:

I am trying to get my tax forms together for the year. I have all of my year-specific files tagged "2011". I also have tags for "taxes" and "donations". So what I tried to do was click on the 2011 tag in the browser, then do a search of "tag:donations OR tag:taxes". Didn't bring me back anything, even though each of those tags individually works within the 2011 tag.

Is this really not possible to do? Is there some other way to do the OR operation?

Thanks.

You cannot combine ands & ors.

[jefito 5,589]

I am trying to get my tax forms together for the year. I have all of my year-specific files tagged "2011". I also have tags for "taxes" and "donations". So what I tried to do was click on the 2011 tag in the browser, then do a search of "tag:donations OR tag:taxes". Didn't bring me back anything, even though each of those tags individually works within the 2011 tag.

Is this really not possible to do? Is there some other way to do the OR operation?

BurgersNFries is correct. However, as a workaround, you could temporarily move all of your 2011 notes into a separate notebook, then do the following search on that notebook:

any: tag:taxes tag:donations

Another workaround might be to rename your tax-related tags to have a common prefix, say 'tax', so you have tags 'tax-taxes', 'tax-donations', etc. Then you can perform the following search:

tag:2011 tag:tax*

This will, in effect, find notes tagged with '2011' and any of your tax related tags.

[TekMason 3]

I'm honored to find out that I am part of a very exclusive group of people like mentaplex who have the "the dazzling mental prowess required to understand something AND (this OR that). " LOL.

I have to agree with paulingraham and mentalplex on the importance of providing grouping and mixed logic in searches/queries. IMHO, the UI design argument is a red herring. Allowing AND OR and brackets to be used/recognized within a query should not be a UI issue at all. It is a shame that we can not utilize the incredible power of Evernote's engine because of a restricted query functionality/syntax.

I was hoping to use EN for our entire IT Team to manage ToDo items and notes but I am not sure how I will be able to overcome the limited query functionality to achieve that.

TekMason

[jefito 5,589]

I haven't found ot to be as big a deal as you may think. Surely it would have been useful in some situations, but those situations are pretty rare for me. I'm not disagreeing that it would be a good thing to have, but I think that it would entail a pretty big change to the search grammar, which is all AND by default, but changes to all OR only in the presence of the any: search modifier. They'd need to come up with grouping, and they'd need to support it across all of their clients.

[TekMason 3]

Thanks for your quick reply jefito.

I guess I'm just thinking out of the box but if you're not living on the edge you're just taking up space

I am an EN noob so I don't really know how the guts of it work. Maybe the queries don't happen in the back end, like I initially thought they would. Regardless, all clients already understand AND and OR logic. I don't think the grouping really isn't much of a leap from what we already have, it could be treated as a query on a query. The possibilities of what you could do with this product with a robust query engine to compliment the power of the engine/db are almost limitless.

TekMason

[jbenson2 2,147]

In my opinion, changing the search code to allow "something AND (this OR that)" sets the bar rather high for Evernote.

Why?

For the past 2 years, Evernote has been unsuccessful in figuring out a considerably less demanding endeavor - how to add a Due Date column.

[Wern 246]

Jbenson,

we all know, due dates WILL be coming, but don't know when, sigh...

As for the "true Boolean logic" I think it will not happen any time soon, if at all, because:

• Boolean Logic is not easy to handle for beginners, especially if we have more than very few parameters
  
• EN's servers will be severely overtaxed by a true Boolean search, thus we'd "pay" for it with a much slower service
  
• All of this could only happen after a major re-write of the current search logic and allocation of considerable resources to make this happen.
  
• Other things are at the top of the list (i.e. due date)
  

Wern

[jefito 5,589]

I guess I'm just thinking out of the box but if you're not living on the edge you're just taking up space

??? argument by buzz-phrase?

I am an EN noob so I don't really know how the guts of it work. Maybe the queries don't happen in the back end, like I initially thought they would.

Queries happen locally on the desktop clients, as far as I know. Not sure about the mobile clients; I'd expect that they need to round-trip to the cloud.

Regardless, all clients already understand AND and OR logic. I don't think the grouping really isn't much of a leap from what we already have, it could be treated as a query on a query. The possibilities of what you could do with this product with a robust query engine to compliment the power of the engine/db are almost limitless.

There's a big difference between being able to use AND and OR logic (obviously they can), and making it accessible to users, particularly in light of the existing query syntax. The question in my mind would be how would they change the query syntax if they were interested in adding mixed AND/OR logic? What do you think?

[TekMason 3]

I don't want to turn this into an argument about what can and can't be done but I must say that I am a little surprised about the negativity and "can't do" attitude about EN development. I have worked in the development community for almost 20 years and I can't imagine how implementing things like boolean logic and adding a due date field can be that difficult.

??? argument by buzz-phrase?

Sorry, I should have said "...creative and innovative (out-of-the-box) ideas could be developed for EN with Boolean logic query functionality."

There's a big difference between being able to use AND and OR logic (obviously they can), and making it accessible to users, particularly in light of the existing query syntax. The question in my mind would be how would they change the query syntax if they were interested in adding mixed AND/OR logic? What do you think?

The existing query syntax does not have to change, just expand it so that it will accept ANDs, ORs and brackets. You could even use existing syntax (any:) and just allow brackets to do it as long as EN did some pre-processing on a query string something like this:

todo:false any:(tag:"this" tag:"that")

As for the "true Boolean logic" I think it will not happen any time soon, if at all, because:

• Boolean Logic is not easy to handle for beginners, especially if we have more than very few parameters
  
• EN's servers will be severely overtaxed by a true Boolean search, thus we'd "pay" for it with a much slower service
  
• All of this could only happen after a major re-write of the current search logic and allocation of considerable resources to make this happen.
  
• Other things are at the top of the list (i.e. due date)
  

The implementation of boolean logic can be totally transparent to beginners as has been previously noted.

Boolean logic is something that computers perform very well and efficiently. Whether there is a string of ANDs, ORs or ANDs and ORs would have a negligible impact on processing resources or time.

Sorry, I can't see how it would require a major rewrite of the current search logic.

Yes I suppose the dev team has priorities but it makes me wonder what the heck those 100 employees do all day if they can't do something as simple as adding a due date.

Evernote is an incredible product but I am a little disappointed to find out that such basic enhancements seem to be such an obstacle for the dev team to address.

TekMason

[jefito 5,589]

I don't want to turn this into an argument about what can and can't be done but I must say that I am a little surprised about the negativity and "can't do" attitude about EN development. I have worked in the development community for almost 20 years and I can't imagine how implementing things like boolean logic and adding a due date field can be that difficult.

I never said that they can't do it, and I certainly haven't been negative about the idea (I've been advocating for changes to the search grammar myself, for nearly as long I've been in the forums, about two years, I think). On the other hand, I have alluded to a couple of obstacles that would make it difficult for Evernote to deliver this as quickly as some people seem to believe that they should be able to. A third issue, of course, is that maybe it's just not that high a priority for them, or maybe don't have plans to do something of the kind at all..

Yes I suppose the dev team has priorities but it makes me wonder what the heck those 100 employees do all day if they can't do something as simple as adding a due date.

A statement like this makes me wonder if you really know how a software development company actually works. Not all employees are developers, and in Evernote's the changes need to occur across an estimated 14 clients, as well as avoiding backwards compatibility issues (for the 14? million existing customers), and satisfying any other internal requirements that they have for the field that they haven't talked about.

[Metrodon 2,184]

A third issue, of course, is that maybe it's just not that high a priority for them, or maybe don't have plans to do something of the kind at all..

Probably the most important issue.

I'm guessing that the vast majority of Evernote users carry out very very simple searches and so adding complexity to the grammar will make power users happy but will produce relatively little bang for the bucks.

[Owyn 457]

A third issue, of course, is that maybe it's just not that high a priority for them, or maybe don't have plans to do something of the kind at all..

Probably the most important issue.

I'm guessing that the vast majority of Evernote users carry out very very simple searches and so adding complexity to the grammar will make power users happy but will produce relatively little bang for the bucks.

Agreed, even though I am one of those "power users" who would use enhanced search syntax.

[jefito 5,589]

Agreed, even though I am one of those "power users" who would use enhanced search syntax.

Well, we knew that.

[JMichaelTX 4,117]

A global union or a global intersection tends to meet the needs of 99.5% of consumers without any complicated query-constructor UI, etc.

Frankly, I have a hard time believing this. Can you cite a survey, poll, or other source for "99.5%"?

[jbenson2 2,147]

I don't think Dave Engberg lurks around here anymore.

I think his last post was on May 18, 2011

[TekMason 3]

Even if it is only 0.5% if the people that would need this (which I think is severly underestimated) relatively simple to implent feature...that small percentage consists of the technology mavens who are be responsible for creating a (paying) community that makes software companies like EverNote a success.

Take a quick peek at Malcolm Gladwell's book, The Tipping Point sometime

All IMHO,

TekMason

[roschler 158]

You can now do searches that mix AND/OR and NOT logic together with BitQwik 2.0, a free app for Windows Evernote users. Mac users that have Parallels or VMWare Fusion should be able to run it too:

Disclosure: I am the author

-- roschler

[GrumpyMonkey 4,319]

You can now do searches that mix AND/OR and NOT logic together with BitQwik 2.0, a free app for Windows Evernote users. Mac users that have Parallels or VMWare Fusion should be able to run it too:

http://discussion.ev..._20#entry174749

Disclosure: I am the author

-- roschler

Cool

Mac users can also use Spotlight.

http://www.princeton.edu/~cmayo/evernote-spotlight-search.html

[James Barton 0]

Add my name to the list of (premium) users who would like this feature to be implemented.