Python SDK (A Blast from the Past?)

[hefty-pie 5]

Greetings!! I have been trying to incorporate an Evernote Oauth flow into my application. Sadly, the python SDK will not install due to dependency conflicts it appears. Using a python flask server. If I am being honest, I truly don't mind writing my own requests to implement the flow. Does anyone know how to do this? The examples on the Oauth API page have not proven to be helpful. Perhaps I misinterpret them. But the RFC indicates a POST request while the examples given uses GET. Furthermore, the RFC indicates the importance of the authorization header, which the examples do not mention at all!!! I have made attempts employing the GET method based off of examples on the Oauth page, and even the POST method based off of the RFC (and also some of the Javascript SDK code that I encountered which appeared to also use the POST method). All requests have been met with a 401. If anyone has any insight on this issue, it is greatly appreciated!! Cheers!!

[david_navigator 5]

I'm finding that just trying step 1 Generate a Temporary Token through Postman is also returning a 401. Let me know if you get anywhere.

[gazumped 12,078]

Hi.  We're mostly just users here - you need to talk to Evernote for more on this...

[david_navigator 5]

Quote

Hi.  We're mostly just users here - you need to talk to Evernote for more on this...

That's what I thought, but Evernote Developer Support points developers to this forum.

[hefty-pie 5]

19 hours ago, david_navigator said:

I'm finding that just trying step 1 Generate a Temporary Token through Postman is also returning a 401. Let me know if you get anywhere.

I will absolutely do that!! Good luck to us both!! Cheers!

[hefty-pie 5]

Ok! Mr. Navigator!! I would like to offer some preliminary help! I have succeeded in obtaining the temporary authorization token after much digging and experimenting! Based on my findings, I believe that the 401 is being returned because of an inaccurate calculation of the signature for the request. And this inaccuracy may be due to an error in calculating the base-string. I will attach a python code file, in which you will find some helper functions that should be able to get you that token! Things to note: 1) On the official Evernote OAuth page where they do provide example requests for going through the OAuth flow, I do not see any mention of the 'oauth_body_hash' parameter. Based on my experiments, this is an absolutely essential piece of data to include in the request! 2) When constructing the base-string note that 2 values need to be DOUBLE-ESCAPED! These are the 'oauth_body_hash' and also the 'oauth_callback'  3) As I continue working through this OAuth flow, I suspect that the signature calculating function will need to be made more dynamic so that it can be used in subsequent calculations for obtaining the Access Token. Anyway, I hope that this code will prove helpful to you and I shall continue to post updates here as I have them. Cheers!! 

 

Here's what you want to generate:

https://www.evernote.com/oauth?
  oauth_body_hash=<ESCAPED BODY SHA-1 HASH>&
  oauth_callback=<YOUR ESCAPED CALLBACK URI>&
  oauth_consumer_key=<YOUR CONSUMER KEY>&
  oauth_nonce=<RANDOM NONCE>&
  oauth_signature=<ESCAPED CALCULATED SIGNATURE>&
  oauth_signature_method=HMAC-SHA1&
  oauth_timestamp=<CURRENT EPOCH TIME>&
  oauth_version=1.0

evernote.py

[PinkElephant 8,845]

Just as a reminder: This is a user forum - I doubt only a tiny fraction will have any interest in these findings.

If you feel there is something broken with the API, you should convince support to provide it to the EN devs.

[AlbertR 713]

52 minutes ago, PinkElephant said:

you should convince support to provide it to the EN devs.

Sorry. This advice is nonsense

• Support points to Forum (see @david_navigator's posting from Wednesday)
• Even Feedback@Evernote.com is a black hole and answers with "If you require support or have any questions, please check out the articles in our Help & Learning or get assistance from other Evernote users in our discussion forums.")

So all offical ways to Evernote point back to here. And even if here are very few developers, I'm highly interrested in such workaround postings like seen from @hefty-pie

 

 

[gazumped 12,078]

6 minutes ago, AlbertR said:

Sorry. This advice is nonsense

Now.  What you could  have said might be some thing like "Actually I'd be interested to know more about this" but what you went with was criticism of someone else's opinion - which IMHO we're all very entitled to have. 

As it happens,  I also couldn't give a stuffed animal what third-party developers want to do with Evernote,  and I think (much like the quasi-official 'bug' reports that I keep on seeing) that such information is far better sent to Evernote directly,  rather than cluttering up the pages of a Forum which seems to me to be mainly for users asking for help,  and for other (theoretically more experienced) users to supply it. 

But hey - that's just me.  And my opinion.

[hefty-pie 5]

Ok!! Here will be my final update on this thread! I hope that David Navigator will find it helpful if still applicable. I will simply share an updated code file which will assist in getting through to the Access Token retrieval. Figuring out the Evernote API from there, will be left for another day, if at all. And yes. I am officially answering my own question. It was worth a shot to post here none the less!! For all who find this thread tedious and misplaced, I have no intention of arguing about any of that. The link at https://dev.evernote.com/support/ literally directs to this Forum and is accessed by a button that is labelled 'Go to the Developer forum'. Please take it up with Evernote staff if this ruffles your feathers. I enjoyed my time here!! Until the link changes, I shall continue to post similar concerns in this channel (hopefully unnecessary). Cheers, all!!! 

evernote.py

[david_navigator 5]

@hefty-pie many thanks for this. I'll try it tomorrow when I'm back in the office. Please post more info as you manage to progress.

Everyone else, this is the place that Evernote tells developers to post questions & issues - I wish there was a more specific topic.
This is the kind of response you get from an email to DevSupport@ - and all I was doing was reporting an error in the API documentation. The Developer Forum mentioned below is a hyperlink to this topic.
 

Quote

Hello David,
 
Unfortunately, I can't provide in-depth support on how to use the SDK. Please refer to the Developer forum and StackOverflow. 
 
Thank you for building with our API,
 
Alessandro
Evernote Developer Relations

I'm guess there used to be a specific developer topic as googling for help pointed to these two forum posts

http://discussion.evernote.com/topic/30584-here-is-a-net-oauth-assembly/
https://discussion.evernote.com/topic/18710-access-token-secret-returning-blank/

- but clicking on either simply gives a "You do not have permission error". I raised this with regular Evernote support and they just sent a boiler plate response which didn't have anything to do with answering my question about how to access these posts.

As far as I can see the developer documentation and SDK's are full of errors and omissions.
I've been a developer for over 30 years and I can hand on heart state that I've never come across such a poor level of support for third party developers. 

For those of you who aren't interested in these API specific questions, I'm sure there's a way to ignore them, but for the time being, it's the only place that developers can get some help.

Cheers

David 

 

[PinkElephant 8,845]

We don’t say „Don’t post here“.

We just say „Few devs here, so it may not be the best place on the planet to search for devs help“.

In general I think EN should do more themselves to support devs who build an ecosystem around the service. First stop should be the API, which was left behind when new features were introduced, and the synchronization changed. I think the „current“ documentation is still from 2014.

There are apps that stopped to build EN integrations out of this reason. Others have folded completely (like Filterize).

I wouldn’t guarantee that the API will still work as before when the legacy clients will stop syncing. In 14 days we all will know more

[gazumped 12,078]

I do know the developer of another third-party integration that had notes that could sync to and from Evernote has also decided to deprecate that feature.  I tried it out but couldn't authenticate with my Evernote account.

[david_navigator 5]

@hefty-pie I'm not a python developer, but I'm looking through your code trying to understand it.

Something I don't get is 

oauth_body_hash = generate_body_hash(body)

Haven't you set the value of body to an empty byte string 3 lines previously  ?

[david_navigator 5]

After much toing and froing with Evernote support, I've eventually got some node.js code that does what I want. Posted here for the next poor developer that comes along...

This code prompts the user for a Notebook Name and a Note Name and then sets/resets the ContentClass field, which effectively changes the note between read-only & read write.

There's no UI, but as this is just for my use, I'm happy doing everything via the command line.

 

const express = require('express');
const session = require('cookie-session');
const Evernote = require('evernote');
const app = express();
const port = process.env.PORT || 3000;
const prompt = require('prompt-sync')();

app.use(session({
    name: 'session',
    keys: ['a-secret-key'],
    maxAge: 24 * 60 * 60 * 1000 * 365
}));

// Use / to redirect to login and obtain oauth token
app.get('/', (req, res) => {
    console.log('login');

    var callbackUrl = "http://localhost:3000/oauth_callback";

    var client = new Evernote.Client({
        consumerKey: "XXXXXX",
        consumerSecret: "YYYYYY",
        sandbox: false,
    });

    client.getRequestToken(callbackUrl, function (error, oauthToken, oauthTokenSecret) {
        if (error) {
            console.error(error);
        }
        req.session.oauthToken = oauthToken;
        req.session.oauthTokenSecret = oauthTokenSecret;
        res.redirect(client.getAuthorizeUrl(oauthToken));
    });
});

// Callback for oauth
app.get('/oauth_callback', (req, res) => {
    console.log('oauth_callback');

    req.session.oauthVerifier = req.query.oauth_verifier;

    var client = new Evernote.Client({
        consumerKey: "XXXXXX",
        consumerSecret: "YYYYYY",
        sandbox: false,
    });

    client.getAccessToken(req.session.oauthToken,
        req.session.oauthTokenSecret,
        req.session.oauthVerifier,
        function (error, oauthToken, oauthTokenSecret, results) {
            if (error) {
                console.error("Get Access Token error");
                console.error(error);
            } else {
                // store the access token somewhere
                req.session.oauthToken = oauthToken;
                req.session.oauthTokenSecret = oauthTokenSecret;
                // Redirect to tests
                res.redirect('/tests');
            }
        });

});

// Page to run tests to query UserStore to get basic user information and NoteStore to read the first note in the default notebook
app.get('/tests', async (req, res) => {
    try {
        authenticatedClient = new Evernote.Client({
            token: req.session.oauthToken,
            sandbox: false,
            china: false,
        });

        console.log("Get UserStore");
        const userStore = authenticatedClient.getUserStore();
        console.log("Get User");
        const user = await userStore.getUser();
        console.log(`Hello ${user.username}`);
        console.log(user);
        console.log("Get NoteStore");
        const noteStore = authenticatedClient.getNoteStore();
        console.log("List notebooks");
        const notebooksList = await noteStore.listNotebooks();
//		console.log(notebooksList);
 //       const notebook = notebooksList.find(n => n.defaultNotebook);
 const NotebookName = prompt('What is the notebook name?');
 const notebook = notebooksList.find(n => n.name === NotebookName);
 if (notebook == undefined)
 {
	console.log(`Could not find notebook with name ${NotebookName}`);
	res.send("Test could not be completed");
	return;
 }
//		"Example: 01. Application Review"
        console.log(`Get notebook ${notebook.name}`);

        var filter = new Evernote.NoteStore.NoteFilter({
            notebookGuid: notebook.guid,
            ascending: true,
        });

        const spec = new Evernote.NoteStore.NotesMetadataResultSpec({
            includeTitle: true,
            includeAttributes: true,
        });

        const metadata = await noteStore.findNotesMetadata(filter, 0, 500, spec);

        console.log(metadata);

        // Read first note
//        const note = metadata.notes[0];
        const NoteName = prompt('What is the note title?');
        const note = metadata.notes.find(n => n.title === NoteName);
 if (note == undefined)
 {
	console.log(`Could not find note with title ${NoteName}`);
	res.send("Test could not be completed");
	return;
 }		
        console.log(`Read note ${note.title}`);
        const noteContent = await noteStore.getNote(note.guid, true, true, true, true);
        console.log(`Read class ${note.attributes.contentClass}`);
		if (note.attributes.contentClass == undefined) {
        note.attributes.contentClass = "navigator.test";
		} else {
        note.attributes.contentClass = undefined;
		}
        await noteStore.updateNote(note);

        console.log(`Note updated`);

        console.log(noteContent);
    } catch (e) {
        console.error("Error");
        console.error(e);
    }
    res.send("Tests done. Take a look at the console.");
});

// Start the server
app.listen(port, () => {
    console.log(`Server is running on port ${port}`);
    console.log(`http://localhost:${port}`);
});

 

[Ibere Ranieri 0]

I quit, the python solution still not working for me.

[Sean C 0]

It may be possible to run the OAuth steps using the OAuth1 client from OAuthLib. Hypothetically, once the oauth_token is retrieved at the end, this could be plugged into any Python bindings for the Evernote Thrift API.

If anyone gets this working out, maybe someone could drop a line here? Using the Qt networking API with requests signed with oauthlib, I've been able to get to the "Retrieve Access Token" step. However, it keeps failing here, with Qt emitting a message: qt.network.http2: stream 3 finished with error: "Host requires authentication"

My code for the this step (failing) is something like the following

 

from oauthlib.oauth1 import Client as AuthClient

# for the following, self.netmanager: QNetworkAccessManager
#
# this is being run via an intercepted HTTP redirect
# via a handler on the <page>.navigationRequested signal (PyQt* or PySide* via qtpy)
# e.g
#     <page>.navigationRequested.connect(partial(self._handle_auth_load_url, auth_widget, callback))
#
#
# the following should receive the URL for the redirect, then parse the URL for the verifier token
# ideally to run the last stage in the oauth process (failing however)
#
# contents of def _handle_auth_load_url(self, widget: AuthWidget, callback: Callable[[str], Any], navrequest: QWebEngineNavigationRequest):

navtype = navrequest.navigationType()
qurl = navrequest.url()
host = qurl.host()

if (navtype is QWebEngineNavigationRequest.NavigationType.RedirectNavigation) and host == "localhost":
    client = AuthClient(key, secret, tmp_token, verifier=verifier, encoding=None, signature_type='QUERY')
    next_qurl = QUrl(urlstr, QUrl.ParsingMode.StrictMode)
    print("-- FINAL REQ", urlstr)
    request = QNetworkRequest(next_qurl)
    for k, v in headers.items():
      # no headers now
      print("-- HDR %r %r" % (k, v))
      request.setRawHeader(k.encode(), v.encode())
      reply = self.netmanager.get(request)
      reply.finished.connect(partial(self._handle_auth_token_reply, widget, callback, reply))
      # reject the redirect to http://localhost/ (callback URL provided in ealier oauth stages)
      navrequest.reject()
else:
    navrequest.accept()
    if qurl.scheme() == "data":
      # server may respond with a data: URL encoding an HTML page
      dbg_info = "(data URL)"
      else:
        dbg_info = qurl.toString()

        print("-- DBG handle URL : call", dbg_info)

It's failing, though I've been able to code it to this point at least

 

Some dangling variables In the previous section:

- Key, Secret: Evernote API key and API key secret, via API key request (Application: Personal, literally "Personal" in the API key request)

- Callback: Function that should be called with the access token, once received

- AuthWidget: Instance of a subclass of QWebEngineView. The instance has been initialized with a specific page item that is an instance of a subclass of QWebEnginePage, such that the subclass was defined to intercept any redirects to 'http://localhost' and run through the code above (not working out, at this point)

[Sean C 0]

It seems that the indentation got mangled in the code snippet, there, and there is a limited time allowed for edits.

I would reassure the reader, I"m not running a 'get' for each header key, value. There might not be any headers introduced for the auth albeit, when using the 'QUERY' signature type for OAuth1 in oauthlib. This signature type is generally request-generic.

Missing from the snippet, oops

OAUTH1_TOKEN_ENDPOINT: str = "https://www.evernote.com/oauth"
OAUTH1_VERIFY_ENDPOINT: str = "https://www.evernote.com/OAuth.action"

CALLBACK_URL: str ="http://localhost/"
  
# [sic]

urlstr, headers, _ = client.sign(OAUTH1_TOKEN_ENDPOINT)

 

[Sean C 0]

HTTP/2 may need to be disabled, for at least the final stage in the auth process. e.g if running the final request via a Qt Network Manager, given the 'request' initialized in the (incomplete) snippet above:

request.setAttribute(request.Attribute.Http2AllowedAttribute, False)

Adding this, I'm not seeing the same error.

Regardless, it seems the `_handle_auth_token_reply(self, [sic], reply: QNetworkReply)` handler is still seeing `reply.errorOccurred`.

With this failing reply, it's receiving a page encoded in a data URL rather than encoded in the reply body. This might be related to how my build here is intercepting the earlier redirect to localhost. I'll try to test out the unabridged stuff, with HTTP/2 disabled

[Sean C 0]

For Python 3 support with Evernote (Unofficial) I've created a fork of the Evernote SDK for Python 2. The fork has been partially updated (tentatively) for Python 3 support. The project may be renamed eventually, if Evernote might not like the project name to include one of their trademarks. Presently it's available as evernote-py3. GitHub would probably still show the right project for that link, if the project was renamed later. This uses a now unmaintained oauth 1 library named oauth2.

Using some sample code together with the HTTP client API used by the underlying oauth library, it seems to be usable to an extent. Presently, it's producing an error similar to what I was seeing with the hacks under Qt. The issue appears to be related to a redirect URL from the Evernote API, providing a redirect destination that is a data URL. With the Qt hacks, the content of the data URL was automatically decoded as HTML and displayed in the Web Engine widget. It's an error message, "Sorry, we've encountered an unexpected error"

Concerning the logic of passing the error message in the data URL, it might be related to how the Evernote oauth support is designed there. The client might be expecting a URL query fragment (not an HTML page) within the HTTP response body. So, they've passed the error message in a data URL with a redirect. I'm not sure of all of the details of how this was handled under the Qt Web Engine support. It's simply breaking the oauth parser, with the forked code there.

Maybe it's easier to reproduce the error, with that sample code at least?