(Archived) Import full text of delicious?

[yesno 0]

Evernote's delicious import is very nice, but I'm more interested in saving the actual full text of every linked page in Evernote. Is there an easy way to pull this off?

[engberg 89]

There isn't any way to automatically do this without opening each link, selecting the part of the page you want to keep, and then clipping that.

[wkoffel 0]

I'm wondering if anyone has figured out a good method for this? I've been a long-time Evernote user, and a longer-time delicious user, and I've recently gotten obsessed with the idea of consolidating all my bookmarks into Evernote.

However, like this poster, I don't just want all the bookmarks, I actually want a web-clip of the whole page, or a searchable PDF of the page. I'm considering writing a script against the API to do this, but am hoping someone else has tackled it.

I have one concern with performance, I've seen so many other complain. Do the latest versions of Evernote (mostly Mac and iOS for me) handle larger data sets like this okay? I've got just shy of 1,000 delicious bookmarks, nothing record-setting, but enough that it could drag down performance if Evernote isn't robust for that still.

Thoughts from anyone who's made this leap?

[Veritrope 3]

I'm going to take a shot at writing an AppleScript to do this -- it certainly seems achievable looking at the Delicious export code!

[wkoffel 0]

Thanks Justin! I agree, it seems doable, I was too shy to go after it, my raw applescript skills are rusty (spent too much time in rb-appscript recently), and I'm unfamiliar with both APIs. Short of an elusive day of free-time, I was secretly hoping someone with your kung-fu had given it a shot. :-)

Excited to hear if you get around to it!

-Will

[Veritrope 3]

Okay Will -- here's an early draft for you to kick around:

http://veritrope.com/code/delicious-bookmark-importer-for-evernote/

CAUTION -- This is very early-stage code which I haven't thoroughly tested. I am posting it in its current form to get feedback from users who have asked for a script like this.

Please proceed slowly until you have a comfort level with what the script is doing.

Here's how this version works:

If you have an exported HTML file from Delicious of your bookmarks, this script will allow you to select it and process its contents into individual, web-clipped notes from each link of the whole page — saving them all in a notebook called “Delicious Import".

After locating the pertinent information within the export file, the script conducts a basic test of whether the link points to a working page. (This is done to avoid the Evernote WebClipper timing out while looking for a non-existant page.) If it page seems to be working, the script then creates a new note using Evernote’s “Create note from URL” function, while attempting to preserve the tags from Delicious and the bookmark’s original creation date.

I'd recommend starting with a small export of only a few bookmarks to see if it works for you.... and, if you need it to do something else/different, please let me know where you think the AppleScript should go from here.

[wkoffel 0]

Looks like a great start, Justin!

I'll schedule some time this weekend to play around in much more depth. Some early feedback:

- You probably want to use curl -siL, the -L will cause it to follow redirect (HTTP 3xx) codes. Typically for older bookmarks, it won't be uncommon for a blog to have refactored their URLs or changed hosts and be sending "permanently moved" responses. Note that curl will return the headers from both requests in the output, and you are scanning the first one for the response code. Maybe there's some fancier way to do that, but if a redirect causes a 4xx response, the code as currently written will still think linkExists = true

- On bookmarks where the host is completely gone, curl returns an exit code (7), and the AppleScript bails, 'error "The command exited with a non-zero status." number 7'. I don't know enough applescript to know if you can wrap the curl in a try clause, maybe print a warning to the console, set linkExists to false and move on to the next one? That'd probably catch quite a few random edge cases.

On a non-technical note, what's the best way to give feedback on this? Happy to do it through this board, but maybe people don't want to listen to us wax-poetic about the intricacies of curl and Applescript. :-) If you'd prefer feedback through veritrope or to a personal email address, let me know.

I really appreciate the initial volley on this! As long as Evernote performance issues don't get in the way, I predict I'll have all my bookmarks in there in the not too distant future.

[Veritrope 3]

Will,

This is great feedback and I've just incorporated your comments back into the source code.

We should continue this at Veritrope.com on the script's page so that other people who are using it can chime in and participate in this. As published, it's a pretty bare-bones script for me -- no GROWL, no real error handling, etc. -- but I thought it would be fun to collaborate on this one and build it out over several iterations.

Looking forward to more of your thoughts/observations/suggestions!

[karakumi 1]

A thought for an addition to this script: emit a list of all unretrievable URLs to a separate file. Those URLs can then be retried later (for temporary outages) or retrieved from versions in the Internet Archive. Initially, the retries/archive searches would probably be a manual process.

[Veritrope 3]

A good idea!

I'll see if I can make it happen sometime soon... but, if you aren't averse to tinkering around, you can change the code yourself and share it at the Code Library!

If you need help getting started, contact me through the website.

[ybakos 15]

Hi Veritrope, just wanted to say thanks for the script, it's the best way to import delicious bookmarks with tags and create a Web clipping.

[digitalformula 1]

Hi - I've made a few modifications to Justin's script and documented them on my website (with credit to Justin for writing the original).

If you're interested in the changes, which I think make the script better (shameless self-plug) please see http://digitalformula.net/articles/deli ... pplescript.