Prevent Duplicate Clippings

[jwstevensii 2]

It would be nice if Evernote could provide feedback and let me know if I already have captured data from a URL.  I sometimes don't pay attention and re-capture and article previously captured.  Something like "You already have a note created from this URL, would you like to capture anyway?"

[heylonghair 1]

The frustrating part is that this functionality has already been developed by Evernote themselves and was working fine in their previous iteration of Clearly.  If they don't want to implement this functionality into Web Clipper so be it but then bring back Clearly.  I'm still using Clearly precisely for that functionality and it still works fine.  I will not commit to Web Clipper until this functionality is implemented.  It is the NUMBER 1 reason why I am hanging onto Clearly.

[ryanjm 1]

This is an old thread, but I want to add my two cents. Hopefully Evernote will reevaluate this option.

 

This feature would be super helpful for me as well. I bookmark / screen grab a lot of pages and it'd be nice to know if I already have it. Big bonus would be to be able to edit that bookmark on that page, maybe there is a new tag I need to add or change which folder it is in.

 

As for the speed issue, I don't mind not having the icon change. Just let me know when I'm about to bookmark it. Once I click the web clipper button, then fire off the request to the server to see if I've got that exact url. It would be awesome to check just the base URL (fixes issues with www vs none, http vs https, /pricing vs /) but I see that as more of a 2.0 feature.

 

As for a basic bookmarking feature, the ability to tell if you've bookmarked something before without going back to EN and searching, seems like a necessary thing. This is apart of Evernote doing the thinking for me. Letting me know I've already done this action before.

 

Thanks!

 

Related thread: https://discussion.evernote.com/topic/54349-web-clipper-and-duplicate-clips/

[drcursor 2]

I've just created a very simple extension that does exactly this - https://chrome.google.com/webstore/detail/evernote-already-there/iidpfiphaljkpnbhmkamheklfpgbmheg .

 

Please tell me if it fits what you we're asking for. 

 

Evernote Moderators - this is something that should really be on the WebClipper plugin!!

[JMichaelTX 4,117]

I've just created a very simple extension that does exactly this - https://chrome.google.com/webstore/detail/evernote-already-there/iidpfiphaljkpnbhmkamheklfpgbmheg .

 

Please tell me if it fits what you we're asking for. 

 

Evernote Moderators - this is something that should really be on the WebClipper plugin!!

 

Thanks for sharing.  It's a good idea, but it has two major problems:

1. It requires unnecessary access to my Evernote account, including update of Notes
2. When I cancel the requested EN authorization, it is stuck in a loop that keeps re-requesting access
   1. I had to quite Chrome, and then remove the extension to stop it.

[GreyGray 15]

I don't think this belongs in EN, it is a complicated issue as far as design decisions are concerned and has many things that would need to be addressed.  It would potentially slow EN down, as there are only a few ways to deal with it.  You can get to the URL easily as you clip, but then you have to search the entire EN library to see if that URL exists.  It would be better to create a fast index of all URLs and just see if that URL is contained in the index, but you have to match the index to the note UUID.  And what happens if you change things, say you move a URL to another note, or you want a duplicate.  Now, that index has to track a Url all over the place.

 

If this ever happens, I think the best way to do it would be to grab the favicon of the site, do a md5 hash on it and store that.  Next time you clip, look for the hash if it is there, some alert could be shown to the user.  That alone is wrought with troubles as there are user1.blogger.com and user2.blogger.com type sites, some could have different favicon files, some may not.  URL's are ever changing, moving, http response codes that tell you to redirect, what do you do in the case of a http temp versus permanent redirect?  Use the original or the final?  What if the admin accidentally, as very often happens, shows a temporary redirect http code when they mean for a permanent redirect http code?  Or vice versa?

 

I vote nope on this one, too many variables for too much potential downside and slowdown off the app.

[JMichaelTX 4,117]

I don't think this belongs in EN, it is a complicated issue as far as design decisions are concerned and has many things that would need to be addressed.  It would potentially slow EN down, as there are only a few ways to deal with it.  

 

It is NOT complicated at all, and would NOT slow down the process.

 

The Note URL is already a part of the Note metadata database, and a very fast SQL search will return whether or not the URL exists.

The Evernote clipper is already doing a lot of much harder, more processing-intensive work, such as pulling your NBs and Tags, determining suggested tags based on web page content, etc.

 

A simple visual indicator, such as the EN icon changing to a different color would be very useful to a lot of us.

[Loose Cannon 4]

On 8/11/2015 at 4:49 PM, JMichaelTX said:

 

It is NOT complicated at all, and would NOT slow down the process.

 

The Note URL is already a part of the Note metadata database, and a very fast SQL search will return whether or not the URL exists.

The Evernote clipper is already doing a lot of much harder, more processing-intensive work, such as pulling your NBs and Tags, determining suggested tags based on web page content, etc.

 

A simple visual indicator, such as the EN icon changing to a different color would be very useful to a lot of us.

BINGO.  The URL's could be hashed in a new column in the evernote backend.  Then the evernote clipper would be allowed to re-clip the same note at the browser or client end.  When the new duplicate clip is received, it's URL can be hashed and compared to the existing hash table.  If the clip is a duplicate, the new clip should be discarded.  It's as simple as that, no user interaction is required.  Though as mentioned in some of the posts above, it would be nice if the clipper's color could be changed to different color after clipping, at least that way, if you came back to an open tab that you've already clipped (at least in the same browser session), you'd know that you already clipped it.  No back end to client communication is necessary for that.  The web clipper could maintain a session based ephemeral flag for URL's clipped in the current browser session, or it could maintain a permanent local hash set in a local webclipper db.

 

/rant on

BTW IMHO "it's too hard" isn't an answer that I would expect from "evangelists".  An evangelist is defined as a person seeking to convert others to their religion.  How useful is it for an evangelist then to resist product suggestions by saying "it can't be done that way" or "it isn't possible".  It is possible and it can be done, should be their attitude.  Now mind you I understand that there are many many product feature requests from consumers that are simply inane, superfluous, or not possible from a business economics perspective.   This particular feature isn't one of those.

/rant off

[gazumped 11,652]

Reading this 4-year old thread again (yawn) I can't actually find the point at which anyone says "it's too hard".  I freely accept that checking the URL of a page is pretty easy,  especially since (unless you opted out) Clpper is already checking your notes index to see what similar clips already exist in your database.  But suppose the content of that URL has changed?  Or you clipped an image last time and want a different image or an article or some text now?

There are some smart people at Evernote so I'm sure getting something together would be technically feasible.  But planning it means a discussion of several people to decide exactly what tests to include;  one or more coders to get things working in various flavors of browser,  testing,  approval by Apple,  Mozilla etc...  All of which costs considerable time and money,  and takes resources away from current firefighting on other,  maybe more urgent,  issues.

Evernote has to decide whether any gains they get from developing this feature - and how many prospective new customers have said "but does Clipper warn me if I've been here before?" as part of their due diligence before signing up? - against the overall likely cost of producing it.  The fact that after four years nothing has happened in this direction apart from Context suggests that it isn't exactly regarded as critical.

Evernote currently tests the relative popularity of feature requests in the 'feedback' thread.  It might be worth posting the bare bones of the idea there - with a link back here for the previous discussion - to see how popular this feature would be.

[robinne 1]

Well, I'm quite surprised this topic has been going on for so long and still not resolved...

I'm using Evernote extensively for my PhD and it basically has replaced the use of bookmarks to me, but I happened to duplicate, when not more, some pages or domains that I have visited a while ago. 

And as it seems that Evernote Desktop doesn't manage duplicates or seemingly close entries, a "warning" feature in the clipper would be handy, and would save a lot of time!

[JMichaelTX 4,117]

I'm using this kind of phrase a lot lately,  but "If it was commercially convenient,  viable and easy to do,  I imagine Evernote would have done it by now"....

 

I know you are, but it's not always, or even likely, to be correct.

Evernote has made lots of decisions to NOT do something, even though it's easy to do.

 

 I don't see how it contributes to anything, other than contributing to the frustration of the OP.

I don't know why you continue to defend Evernote for having bad design.

IMO, you are just making excuses for Evernote.

[gazumped 11,652]

If you support this request - please vote for it at the top left of this page.  Current votes:  zero.

Entirely due to my own cupidity I happen to know that I created a situation where I currently have just over 2,000 duplicate notes in my database.  It's a big database - though not the biggest around here by any means.  The dupe notes are virtually all quite small ones.

All those extras appear not to be a problem - I do my searches and have a side list view on a laptop and if I sort by title and look at the note size column,  two notes with the same title and very similar sizes are good candidates for dups.  A quick check and a delete or two will clean up my search.  I'll usually check my list for old grocery orders and the like anyway,  just to minimise future hits,  so this is admin work (sorry; "curation") that I'd do anyway.

On a whim I did get to the stage of doing a 'select all' on my list view of all notes (sorted by title) and pasted that into Excel,  then went back to my selected-all list view and used Ctrl+Copy Note Link to get the direct links to all notes,  and re-posted the names as links into the note-title column in my spreadsheet.

Using Excel's features I could now isolate a list of all my (apparent 2,000-some) potential dups and jump straight to each candidate to sort out the problem.  Which I might have a go at if my current addiction to Solitaire loosens its grip.

But given the lack of hassle in just leaving the dups there until they annoy me,  I see no great urgency in pursuing this clean-up.  It would be nice to have a one-click solution to find duplicates rather than the 20 minutes or so I spent copying and pasting and (mainly) waiting for the system to catch up with "how many lines???" but IMHO it is strictly a 'nice to have'.

I could spend however long it takes to look at 4,000 notes and delete half of them - a couple of hours I'd guess - or I could deal with that extra workload by spending an extra couple of minutes with search results several times a day.  And get 2 hours' more work out of my day.  If it's not time-efficient for me to do it with my 2,000 dups,  a few extra notes in any database is unlikely to hurt anyone.  But your choice - if you prefer to excise duplications and you have Windows,  the above should help.

I also have Clearly - and I'd forgotten that it does show a nifty little check mark when you've clipped a page - but only if you stay on that page.  I just used it to clip a page,  closed and re-opened the page and clipped again.  Result - two notes.  The difficulties of deciding when a duplicate clip really is a duplicate have been raised before.

A lot of the above may not be applicable to other platforms and devices outside of a Windows laptop,  but I'd imagine there are other ways around the issue if you want to find them....

And Context is also an option - it's not foolproof,  but if you have your settings correct,  you'll get a pop-up after each clip which will show 'similar' notes.  If you see the exact same note title in that display,  that would be an 'Ooops' moment.

It's just occurred to me that if I attach that spreadsheet into a note,  having deleted all the unique note names,  that note is highly likely to come up in all search results (I normally use intitle: searches) which also include my duplicate suspects.  So if I see the BEWAREDUPES note title anywhere in my list view...   Hmmn...

 

 

[GreyGray 15]

8 hours ago, gazumped said:

I could spend however long it takes to look at 4,000 notes and delete half of them - a couple of hours I'd guess - or I could deal with that extra workload by spending an extra couple of minutes with search results several times a day.  And get 2 hours' more work out of my day.  If it's not time-efficient for me to do it with my 2,000 dups,  a few extra notes in any database is unlikely to hurt anyone.  But your choice - if you prefer to excise duplications and you have Windows,  the above should help.

Was this duplicate items set created all at once?  Or even if not all at once, you have 2000 items, and then 2000 more were created.  You should be able to find all the items, there must be some commonality that they have in which you can do a search to locate just the 4000 in question.  Then sort by date created, date added, date modified, date last modified, or whatever criteria makes sense given your knowledge.  Now you have a huge chunk of the list items in order that need deleting.

I am not sure how many items you can show on the page, you may want to move to the web version of EN in order to use Javascript, or you can use Applescript if on the Mac, and I am sure something else if on Windows. Somehow, you need to iterate thought the found set and select 1-2000 and then hit the delete key.  Or delete them one at a time.  Search the forums, the applescript is out there in some form or another and a little hacking to it will work it out for your case.

[gazumped 11,652]

Thanks for the thoughtful response @GreyGray

It's a good point that a batch of duplicates added at one time would share the same created date,  and/ or tag(s) but...

The duplicates were from my adding an email library to my main database.  I carved it up over a period of weeks and imported anything that seemed important - unfortunately there are two problems with that;  for various reasons duplicates crept in with virtually every upload,  and since there are several emails in some conversations,  there are more duplications than I mentioned earlier,  in that each email includes the history of each previous reply.  They have the same title but different sizes.

Plus - if I go by note title and size,  there's also an issue - I use a 'smart title' setup,  so each email from one source about the same subject will likely have the same title.  If someone sends me two very different but identically sized pictures though,  I'll have same title,  same size but actually different emails.

I hesitate to mess with the database outside of Evernote,  so any changes have to be via the Trash notebook...  and I really really need to have an eyes-on inspection of two notes to verify they are a duplicate before I hit the nuke button.

There is an app that may be able to handle the duplications - http://www.onenotegem.com/evernote-batch.html but there are some interesting issues regarding an install...  I'm not sure my installation is still valid (a lot of changes since its last use) and it would take a while to repair that.

But again - I'm happier to deal with dups when I find them in normal activity than to spend time working out how to clear all of these in one go.

[JanKO 0]

I really appreciate this idea and would find it extremely useful. Clearly, the person who is consistently looking for dups in his/her EN database lives in a different universe from me. I sometimes find myself clipping a second time just to make sure I actually have the content and would love the warning. Or I am doing a similar search at another time and don't remember whether I clipped the content. 

[michaelbierman 0]

On 5/8/2013 at 12:46 PM, Wern said:

While I can see a need for this, this would be difficult to implement and not useful after some time. Websites change, URLs change and implementing this would then give you useless or conflicting messages, as changes will be more likely. Probably not on the radar screen of EN developers.

Which is why basing it on the URL or URL alone wouldn't be smart. But there are cases (e.g. pictures) where the content could be analyzed to look for matches based on size content (color maps, EXIF data, etc.. Similarly PDFs could be matched with a fingerprint of sorts based on file size, content, meta data. 

[gazumped 11,652]

Still think a hash on content would be difficult to fly - pictures get changed / posts updated / twitter feeds change - and you may not have clipped all the page last time... plus there's still the need to do a search on the database after scanning the site to check for duplicates...   means a slow(er) clip after you make the decision to go ahead.

[JMichaelTX 4,117]

2 minutes ago, gazumped said:

Still think a hash on content would be difficult to fly

Maybe not.

The EN Web clipper could provide an option to identify potential dups in its Options.  Then the dup search could be limited to the page URL + Title -- that should be very fast.  The Options could also allow the user to specify the period of time to search: last NN days, last week, last month, last year, etc.

[jefito 5,589]

On 1/15/2017 at 8:58 AM, Loose Cannon said:

BTW IMHO "it's too hard" isn't an answer that I would expect from "evangelists" <snip>

Historical note: you're a little behind the times. There are no Evernote "evangelists" anymore. The title was one given to some forum members who were pretty active at the time, but that program was turned off some years back. Probably just as well, as hardly any of us took on the true evangelist role as you describe it, not that we were required to: we were allowed to express opinions about Evernote, pro- and con-, and participate in discussions and disagreements, and even rant, just like anyone forum user. I always thought that it was kind of a goofy title, myself.  Anyhow, nowadays anyone can be a "guru" if they post enough times, but any of the old evangelists who are left around mainly continue on doing what they were doing before. I think that @Wern is no longer here though, if that's who you were referring to; in which case your rant may have been a little in vain. Anyways, carry on...

[Damian IOE 1]

I was hoping this thread would address the problem of some notes getting duplicated not by my stupidity, but EN's. Not able to determine when or why, but it happens with the Clipper extension in Chrome.

 

When it happens, it would be nice to easily compare the notes to see if one has 'better' tags, or additional comments that I added to it.

[Wern 246]

While I can see a need for this, this would be difficult to implement and not useful after some time. Websites change, URLs change and implementing this would then give you useless or conflicting messages, as changes will be more likely. Probably not on the radar screen of EN developers.

[gazumped 11,652]

Plus if my clippings are useful I'll pretty soon find that I have two copies of something.  This is where my gardening gloves come out,  the pruning shears - er - shear,  and I gain a sleeker,  happier database again...

[Nathan T 49]

While I can see a need for this, this would be difficult to implement and not useful after some time. Websites change, URLs change and implementing this would then give you useless or conflicting messages, as changes will be more likely. Probably not on the radar screen of EN developers.

 

I don't know, it seems pretty straightforward to implement a basic version of this.  You wouldn't have to catch every duplicate; if you miss some, it's no worse than now.  So you could focus on eliminating the false positives you describe.  For example, the warning could appear only if you're clipping an identical url to the same notebook where you have the same url clipped already.  (As opposed to simply the same site or a similar url.)  I can see how trying to match the contents of a full article clip might be too resource-intensive to give this immediate feedback, but it could at least check whether you've clipped the same page recently; say within the past month.  If so, it's likely you've simply forgotten you already clipped it, and a notification would be helpful.  Seems to me checking for an identical url, notebook, and date range is something that could be done in under an hour by a knowledgeable developer.  The slightly more time-consuming part would be the UI, but they already have the notification on clip option, so it should be pretty straightforward to put something in there.  (And indeed, it already shows you 'similar' pages you've clipped, which is very similar to this request; this would just be a more obvious warning for an identical page.)

 

Edit: All that said, I'm intimately familiar with the tradeoffs developers have to make, and with the fact that there are nearly infinite features out there that would be both useful and not too hard to code, so some always end up taking priority over others.  Just lending my voice as someone who would like to see this particular feature move up a notch or two.

[Nathan T 49]

You know what would be ideal?  If for every page you visited, the evernote browser plugin sent a request to your library to check whether you've clipped that exact url.  If so, the web clipper icon would change color.  (Just like the bookmark star in the browser.)  That way there'd be no back and forth "oops, I already clipped that!" or whatever.  It'd just be immediately obvious.  If you haven't been to the page in a long time, you'd have a reminder that it was clipped at some point in the past, which might even prompt you to search for related stuff you clipped the first time around.  Or to check whether the page has changed.  Or just to not bother clipping it again.  Super handy, and unobtrusive.

 

Until then, maybe there's room for a third party developer to make an add-on that does just that?  A toolbar icon that would automatically run an evernote search for web clippings with the url of any page you visit, and light up if matched, with the ability to click on the icon to open that note.  I'm no plugin developer, but maybe someone out there is and would like to take a crack at it?

[jbignert 473]

Thanks for the feedback Nathan,

We've considered this before and the main thing holding us off at the moment is exactly what you describe above that we would have to send a pretty expensive request for every website that any Clipper user loads. We've considered using the browser history but that will not be reliable since it may be frequently cleared by the user. I'm not sure how useful the feature would be if it had a limited memory (you mentioned a month f.a.e). 

 

I see the rationale for the feature though. 

[Nathan T 49]

Thanks for the reply! I was actually thinking a bit more last night about how this might be implemented without that overhead. What about an option in the web clipper to automatically add any page you clip to a folder in your browser's bookmarks? You could then use bookmarks instead of history in the way you describe, which would be much more reliable. And in fact, you wouldn't necessarily even need to do anything else, since the built in browser interface would then show that the page had been clipped (or bookmarked).

A bit trickier would be keeping that bookmarks folder entirely in sync with EN, but that wouldn't be necessary for the feature to be useful. It could certainly be left out of the initial version, for example. The auto-add to bookmarks folder option alone would be pretty cool.

[gazumped 11,652]

@jbignert - wouldn't it be easier to check existing notes for the URL and just pop a warning "you've downloaded from this page before - sure you want to go ahead?"

 

@NathanT - I clip from a lot of places including my phone and a work PC; don't see this extra folder being of much help in preventing dups from more than one source...

[Nathan T 49]

@jbignert - wouldn't it be easier to check existing notes for the URL and just pop a warning "you've downloaded from this page before - sure you want to go ahead?"

 

@NathanT - I clip from a lot of places including my phone and a work PC; don't see this extra folder being of much help in preventing dups from more than one source...

 

For sure, I agree that part alone wouldn't be useful for everyone, but it'd be a start for some of us.  On second thought though, it wouldn't seem to be too resource intensive to just keep the bookmarks folder in sync with EN.  The web clipper would need to send a request at login and periodically afterward to check whether there were any web clippings added to or deleted from the EN library since the last check.  Since the polling wouldn't have to be too frequent, and the queries would be limited to a short time index, I would expect it to work pretty well.  Bookmarks in that folder could then be added or deleted as necessary to sync with recent changes in EN.  (Perhaps less often and/or manually it would also want to check the whole library in case the bookmarks folder has been mangled somehow.)

 

If you are pruning bookmarks from the folder as well, the tricky part then becomes the UX.  You'd want to make sure the user doesn't go messing with that bookmarks folder, without the option seeming overly complex.  Maybe just a warning modal when the option is enabled?  Or the web clipper could monitor the folder and give the user a message if they manually add a bookmark there.

 

Anyhoo, just some ideas in case they help!

 

Edit: And yes, it would definitely be technically easier to just check and give a warning when the user goes to clip, but then you have that wasted mental energy I mentioned.  Seeing that it's clipped as soon as you land on it would accomplish the same thing without the back and forth, and might even jog your memory to related stuff.

[gazumped 11,652]

"Evernote has made lots of decisions to NOT do something, even though it's easy to do."

 

Unless you've been to meetings that the rest of us missed,  I don't see how you can know that EN made decisions one way or the other;  and 'easy to do' doesn't mean it's commercially convenient or viable for EN to throw in a quick update in the middle of whatever priorities they've already set for themselves.  Big companies,  as you know,  tend to have about the same agility as a fully loaded oil tanker when it comes to crash stops and sharp turns - even when they want to take action.  Whether or not it pleases you or the OP,  this is Evernote's party and they'll plan what they want to..

[Nathan T 49]

@jbignert - wouldn't it be easier to check existing notes for the URL and just pop a warning "you've downloaded from this page before - sure you want to go ahead?"

 

@NathanT - I clip from a lot of places including my phone and a work PC; don't see this extra folder being of much help in preventing dups from more than one source...

 

BTW: You could use something like X-Marks to sync your bookmarks between browsers and devices, which would solve that part of the problem.  I'm a little bit tempted to just make a habit of hitting the firefox bookmark star (which just dumps things in 'unsorted bookmarks', which I don't use anyway, every time I clip something actually.  It won't be perfect, but I'll avoid a lot of those accidental re-clippings, and it'll be nice to see at a glance that I clipped a page before.  It won't catch everything (probably won't do it on other devices since it's more of a pain to bookmark), but won't hurt.

 

Makes me think, along with this, it'd be cool to have an analogous 'reject' button.  Wouldn't have to be something associated with Evernote, just an idea that came to mind.  Like an anti-bookmark, so next time you land on that page you can see immediately from the icon that you've already parsed it and decided it isn't worthwhile.

 

Edit: Ironically, X-Marks does exactly that as well!  They have a 5-star rating feature for individual URLs.  It links to your X-Marks account, so you can see your rating next time you hit the page.  So instead of bookmarking things I clip, I might just star them instead.  Anything worthy of clipping would be 5 stars, and I'll only 5-star things I've clipped.  And junk that I don't want to waste time on can be one star.  Hmmm...

[jbenson2 2,147]

Thanks for the feedback Nathan,

We've considered this before and the main thing holding us off at the moment is exactly what you describe above that we would have to send a pretty expensive request for every website that any Clipper user loads. We've considered using the browser history but that will not be reliable since it may be frequently cleared by the user. I'm not sure how useful the feature would be if it had a limited memory (you mentioned a month f.a.e). 

 

I see the rationale for the feature though. 

 

If you implement this, please be sure to avoid any slowdown in general search speed or clipping speed.

I've got 22,000 notes and 12,000 of them are web captures of political commentary websites.

Is the verification process for 12,000+ notes is instantaneous, fine.

If it slows down Evernote, please reconsider.

[gazumped 11,652]

Evernote can (apparently) do a 'related notes' search without too much of an impact on speed and overhead,  so I'd hope a very similar process could produce a "you've clipped here before" match - and that would seem to be in line with Evernote's advertised wish to make remembering and connecting things as easy as possible.  Otherwise why have a related notes feature in the first place?  There are bound to be cases though where the warnings would come thick and fast - if you regularly clip from a new site for instance - so you'd need a way to option it off.  But I agree with jb2 - whatever (if anything) is possible mustn't get in the way of searching or clipping.

 

It is possible to duplicate clips,  but storage is cheap,  upload limits are generous and even my flaky memory works well enough most of the time.  I don't see that this feature would save anything other than the occasional confusion over a duplicated page - which only occurs when you find the duplication,  and so can eliminate it..

 

Nope - this is a 'nice to have' but there are many other things more pressing!

[Nathan T 49]

"...this is a 'nice to have' but there are many other things more pressing!"

 

I don't disagree with you there, especially if all it does is warn you when you try to clip a duplicate url.  However, I think knowing immediately when you land on a page whether you've clipped it before (with the clipper icon lighting up or changing color or something) would be considerably more valuable and even more in line with the Evernote philosoply you mentioned.  Plus, it would be far less obtrusive, avoiding the 'thick and fast' warnings problem.

 

The only downside is the technical issue jbignert mentioned.  I suggested one sort of work-around to that if they wanted to crank it out quickly.  Another option would be to keep a dedicated index in the EN database to allow for quick url-matching, making that operation less resource-intensive.  Plugins like the Xmarks one I mentioned and social bookmarking ones like Delicious do those sort of operations on every visited page, so it can definitely be done.  And since it happens completely asynchronously with the flow of clipping, even if it took a few moments to return the info, it wouldn't delay clipping pages (or slow down any aspect of evernote).  Worst case, it would take a moment for the clipper icon to light up upon hitting a visited page, which obviously would be no worse than now.

 

Anyway, the EN devs seem like smart folks, so I expect this is all old news to them and they've got a set of priorities that make good sense!

[gazumped 11,652]

There's a related discussion going on over here I think - http://discussion.evernote.com/topic/38099-on-demand-related-notes

[haws 1]

A Bloom filter is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set.

Couldn't there be a client-side check for duplicate URLs? Evernote can't be less smart than the bookmarks feature of the browser...

[elifarley 1]

Indeed. The Web Clipper should perform a client-side test of the url against the user's bloom filter (regardless of which browser he/she is using).

 

Since a bloom filter can return either "possibly in set" or "definitely not in set", the Web Clipper doesn't have to confirm the set membership if the bloom filter says "definitely not in set", thus saving Evernote both bandwidth and CPU cycles.

 

Whenever a new url is saved, Evernote servers should update the user's bloom filter.

[gazumped 11,652]

I'm using this kind of phrase a lot lately,  but "If it was commercially convenient,  viable and easy to do,  I imagine Evernote would have done it by now"....