(Archived) Searching for @

[crane 40]

In the new spirit of peace and love and good two-way communication, I'm putting out another question to the EN team

Right now, I can't use @ in a saved search, e.g., "@next", "@waiting", errands. Neither can I use other strange characters, like #.

I know it has been suggested that we not use this weird characters, instead, using something like _next. However, I just did a search for _next, and it gave me all the "next"s in my document.

Anyway, will there be a way, sometime in the future, where sometime < decade, that we will be able to use these strange characters in our searches? I'm okay with having to escape it, e.g., "\@next", or something like that - as long as I can just write something like "do this @next" in my notes.

Thanks.

[ruudhein 29]

+1

GTD is growing. Fun to ride on the waves, maybe

[engberg 89]

The behavior of "_next" on Windows is unintentional (my fault, didn't communicate well internally). I.e. this should be searchable without finding "next".

As discussed in a previous thread, the '@' character is currently treated like punctuation, so it is used to separate words instead of being part of a word. This means that you can search for the word "evernote" in "whomever@evernote.com" in spite of the fact there is an '@' next to it. If we treated '@' as a character in a word, then that would contain two words "whomever@evernote" and "com".

More detail than you wanted: this isn't really an issue of "searching" but rather of "indexing", which is the process of taking each note and pulling out the searchable words (in order) and putting them into a fast search index. Without doing this sort of indexing in advance, searching would be extremely slow on the desktop for large numbers of notes, and would be basically impossible on the web. This indexing takes a while and needs to be done in advance. This means that it's not a problem of how you type in your query, it's a question of how we build the index that makes your query possible hours later.

I.e. this is another case of: we hear what you're requesting, the implications of your request would break something else that reasonable people would expect to work, so we're not sure what the right solution is yet.

In the short term, I think that the underscore thing ("_next") may be addressed in the next Windows build. (Or, for completeness and as mentioned in the previous thread on this topic, you can use tags.)

Thanks

[crane 40]

In the short term, I think that the underscore thing ("_next") may be addressed in the next Windows build. (Or, for completeness and as mentioned in the previous thread on this topic, you can use tags.)

Thanks for your detailed response.

Just so you know - tags are really only useful for stuff that you're willing to manually tag. I was trying to recapture some of the smart tags by just have a few saved searches, e.g., @next, @library, @waiting, etc. The point is to reduce how much ***** I have to do after typing the note. Unfortunately, all of my personal keywords use things like @, #, etc.

Sigh.

[dormouse 3]

I understand the explanation - and the issue is not a problem for me. I never did like the GTD use of @.

But I assume that this worked in 2.2, so why has it changed for 3?

[crane 40]

I understand the explanation - and the issue is not a problem for me. I never did like the GTD use of @.

The @ is just one character that is broken now. Another I use is #, e.g., #password# to identify notes that contain, well, passwords. I use the # to distinguish terms that I know I'm going to want to search for, so that I can find those specific notes, especially for words that are pretty common.

I've tried a whole bunch of other symbols in the desktop version, e.g., -, =, +, !, $, %, etc., but I can't get anything to be searchable. Or rather, indexable. It's kind of limiting.

[dormouse 3]

so what you really need is a list of symbols that are treated as something other than punctuation. The issue seems to be that punctuation is ignored for the purposes of indexing. When searching for docs on my computer I do expect to use * as wildcard and to find words in quotes (and only in quotes) etc.

[engberg 89]

I understand the explanation - and the issue is not a problem for me. I never did like the GTD use of @.

The @ is just one character that is broken now. Another I use is #, e.g., #password# to identify notes that contain, well, passwords. I use the # to distinguish terms that I know I'm going to want to search for, so that I can find those specific notes, especially for words that are pretty common.

I've tried a whole bunch of other symbols in the desktop version, e.g., -, =, +, !, $, %, etc., but I can't get anything to be searchable. Or rather, indexable. It's kind of limiting.

Transitining to a "word" and "phrase" based search syntax has a lot of performance advantages, and it also matches up much more closely with what people expect from consumer-friendly search engines like Google. If you search for "@home" or "#home" or "$home" in Google, you'll get pages that just have the word "home" in them. This isn't because Google is particularly lazy or malicious in their search implementation, but rather they've found that people become pretty annoyed if search engines only work when you type the exact Unicode character sequences from the target documents. My dumb example from the previous thread was that people expect to be able to search for "Murder, She Wrote" and find a list of documents that might have "murder - she wrote" in some of them.

That said, I understand that there's an existing usage pattern for people to put non-word character sequences in their docs and expect to find them, and you'd prefer not to use 000Password or some other alphanumeric sequence. Our (somewhat arbitrary) inclusion of the '_' character into the list of word characters was one attempt to help support this use case ... although, as I mentioned, it's currently broken.

Thanks

[crane 40]

How about improving the search so that we can search for non-word characters? (I'll admit to occasionally being frustrated with the fact that google will not let me search for special characters.) For example, if we went towards a more regex kind of search, we could finetune exactly what we were looking for

[engberg 89]

Regular expression searches are the sort of thing that common search engines like Google avoid for both performance and usability reasons ... they're slow on large data sets (e.g. you need to search character by character through thousands of notes), and they are really confusing for most users.

If I were to paraphrase the original requirement: "I want to type special non-word sequences in the body of my note so I can find them later via a text search without getting back similar words. This is faster for me than tagging, and I would prefer not to achieve this via made-up alpha-numeric words like zzzCooking or 0Cooking0."

I'd like to help achieve this (e.g. via "_Cooking") without making the search engine slow or unfriendly for the more common use cases.

[dormouse 3]

Makes sense to me.

Certainly seems as if the made up alphnumberic is the most straightforward solution - at least for everyone that's not already used to something else. I can't see how the @ can be made to fit, even if you move from _ since it has a much bigger usage in email addresses that people would expect to search in.

At least it is clear that the made up alphanumeric will always work, and that other solutions will be you trying artificially to make things fit for people who want to use symbols.

[crane 40]

If I were to paraphrase the original requirement: "I want to type special non-word sequences in the body of my note so I can find them later via a text search without getting back similar words. This is faster for me than tagging, and I would prefer not to achieve this via made-up alpha-numeric words like zzzCooking or 0Cooking0."

I'd like to help achieve this (e.g. via "_Cooking") without making the search engine slow or unfriendly for the more common use cases.

That would be a good paraphrasing. What I don't particularly like is the limitation to _ as a special character. I can see not using the @ because of its placement in email addresses. But how but giving us some more to play with, e.g., #, $, %, &, -, =, +, etc.?

$ would be particularly useful, not just as a special char, but also to find money amounts, and programming terms

[ruudhein 29]

To those reading the thread to deal with the specific case of GTD contexts written as @context ... What I've started doing is CTRL + SHIFT + C to get a checkbox, then use and/or specific action verb or context.

Examples:

[ ] Call John to resolve order delay

[ ] Email Peter with EverNote invites

I then have saved searches named like "@Call", "@Email". Syntaxt: todo:false call or todo:false email etc.

Once an action is complete, check the checkbox and the item is moved off your open actions list.

[wsp 11]

May I put in a good word also for brackets ([] <> {})? Many of my automatic categories are based upon them. I haven't so far used @, which I gather is favored by disciples of GTD, whereas I think of myself as belonging to the school of RGTD (Rarely Getting Things Done).

[jttpost 0]

In the short term, I think that the underscore thing ("_next") may be addressed in the next Windows build.

If "_" could be used as a special char in future builds, it would be fantastic. I personally would immediately shift all my old @ to _.

For safety though. can we depend on it being a special char past beta?

That would be a good paraphrasing. What I don't particularly like is the limitation to _ as a special character. I can see not using the @ because of its placement in email addresses. But how but giving us some more to play with, e.g., #, $, %, &, -, =, +, etc.?

$ would be particularly useful, not just as a special char, but also to find money amounts, and programming terms

I personally agree with crane.

"_" is a good start, but if there is some acceptable "group of chars" that can be made "special" without compromising the index performance, then that would be fantastic.

In the meantime, if "_" is a new official special char, then I am personally going to use "_" with wild abandon when it is available in future builds.

Thanks for all your help, Dave.

[engberg 89]

If "_" could be used as a special char in future builds, it would be fantastic. I personally would immediately shift all my old @ to _.

For safety though. can we depend on it being a special char past beta?

Yes, the "_" character will be a "word character" for indexing and searching for the indefinite future (i.e. unless we find out that's completely breaking something for some of you.) The flip side of this is that you can't find "engberg" in "dave_engberg", etc. I.e. saying that you *can* search for "words" that contain underscores implies that you can't find strings that have underscores near them, because these are part of the "word".

This is the primary consideration that would prevent us from adding additional punctuation to the "word" character set ... e.g. if '[' and ']' where interpreted as word characters for indexing and searching, you couldn't find find the word "emphasis" in a document that contained:

... [emphasis added -ed.] ...

[ruudhein 29]

I have a bunch of email addresses in my 2.2 collection which contain _

Question: is this kind of behavior of choice maybe best served by an advanced option?

[engberg 89]

I have a bunch of email addresses in my 2.2 collection which contain _

This means that you can find "dave_engberg@..." by searching for "dave_engberg" or "dave*", but not by "engberg".

We'd like to follow the lead of all of the other successful consumer search technologies that people are used to like Google, etc. and not introduce a lot of heavy complexity into pattern matching, etc. That's why we're leaning against making things more complicated with a lot of variable punctuation behavior, etc.

[ruudhein 29]

Makes sense.

Maybe it would be possible then to default to search for words, treating _ as punctuation unless you do a phrase based search? ("_office" vs "office or _office).

This way we could have both, maybe.

Otherwise, is there a combination solution that makes things easier? If the special character and word aren't written as one, would that make things easier to implement? So: @ Office or _ Office instead of @Office or _Office?

[helix 10]

It seems that right now only "_" will be available for a search, but as said above, even this one could be in question. Here are some temp solutions in this regard:

1. Adding a single non-protruding character e.g "i" in front of the @next will make it unique since the program is now looking for a (single)i(space)next which is probably not found elsewhere. This still keeps the @ visual to the user. So search for i@next will find only the @-associated next`s and none of the standalone next`s. A search for i@ will find all i@`s like i@next, i@todo etc. i at the end of a word before next (e.g."gemini next") does not interfere. In this spirit the old good # could be revived too but that will require of course a different character like "j".

2. In any case of "_" or "i@", a find\replace add-on for EN 2.2 is a must for migration to EN3 and should be supplied to us by the EN team.

3. Another idea is to use Greek symbols such as Ω ,π , ∑. They are different, catching the eye, there are many of them, and best of all, they are recognized by EN as characters, not spaces. So Ωnext and ∑todo can do the job. Unfortunately they are not readily available, but one can get them by:

- Copy\paste from a permanent text on the desktop.

- Paste from a clipboard-retention utility such as ClipMate.

- Assigning them to a specific keys or specific key combinations on the keyboard using some key editor utility.

- Adding "Insert symbol" to EN3 editing manu.

Not perfect, even far from perfect... just trying to help..

[salgud 12]

I don't have the "@" problem, nor any good suggestions as to how to deal with it. But I find this thread very encouraging because there's an open dialog about what is causing the problem for the developers and what the possible solutions might be, involving the end users. I've seldom seen a problem in my career that couldn't be solved with this approach. I hope this continues and that an honest effort is made by the EN team to "make it work" for your power users. I think the payout, though hard to measure in accounting terms, will be huge.

[dormouse 3]

But I find this thread very encouraging because there's an open dialog about what is causing the problem for the developers and what the possible solutions might be, involving the end users.

I couldn't agree more.

[crashing_sux 0]

Has any thought been given to either storing two matches in the index (similar to what you do when recognizing text in images), one for @TODO and one for just TODO?

What about just doing simple pattern matching in the indexer? It sounds like @ is heavily used and much differently used to start a word than it is in an email address. It's pretty easy to treat @ as a word character at the beginning of a word, and then ignore it when it is in between other characters, or followed by other characters and then a "." as in me@email.com.

Even then, I think I prefer the ability to have it treated both ways. I might like the ability to find all notes I put an email address in, which would be easy if I could just search for @*. or @*.com. Often email addresses don't contain a persons name, searching for Shane would not find my email address because my email address is not FirstNameLastName@company.com. It could be because I'm a dirt biker but most of my friends email addresses are something like DirtBikeRider@email.com or MXRacer@email.com. Searching for John isn't much help there.