(Archived) Mac vs. Windows Recognition Operation

[ghamma 0]

Greetings,

As an experiment, my son, using his Mac, and I, using my PC, submitted the same images as notes.

On the Mac, there was an immediate synchronization and the file was apparently sent to the server. On a following synchronization the recognition results were returned to the Mac.

On the PC, the recognition ran within the PC and the synchronization sent the file to the server. The recognition results from the PC were subsequently sent to the server.

The results of the recognition were clearly different. On some images, tilted text was recognized more reliably on the Mac result from the server. On others, of handwriting, the recognized text varied between the two system results on the same image.

Our impression is that, while we have identical CPUs, the server recognition result was available much more quickly on the Mac than on the PC.

-George Hamma

[engberg 89]

George -

Thanks for the interesting analysis. You're right that the processing steps on the Mac are different than on the Windows client ... the Mac version sends all images to the service for processing, while Windows includes a local image processing capability. As you've noticed, this can result in differences in the speed of recognition on the two platforms. (Although you might perceive different results if you weren't manually synchronizing ... in this case, the Windows client might have local results before your Mac client bothered to ask the service for updates.)

I'm a little surprised that the search results from the image processing were different in your test, since the recognition libraries used on the service are identical to the libraries used in the latest version of the Windows client. I.e. for the exact same image file, the current Windows application should produce an image recognition index that is identical to the service's.

Would you be able to provide the version number from your Windows desktop client? It's available under the "Help" menu, option: "About Evernote". The current update is version 3.0.0.550.

Thanks again for the feedback

[ghamma 0]

Greetings,

My version of the Windows client is

3.0.0.550 (27500)

downloaded and installed today.

-George Hamma

[engberg 89]

Ok, you've got me stumped. Would you be willing to send us one of the images that you tested which produced different results on the Windows and Mac? Alternately, you could put it in a notebook and then make that notebook public (under the notebook Settings on the web).

Thanks

[ghamma 0]

Greetings,

No problem with providing an image. Where would I send it (you can email it if you want)?

-George Hamma

[ghamma 0]

Greetings,

An image that gave distinct recognition results between Mac (found Platform) and Windows (didn't find Platform) is posted at:

http://preview.evernote.com/pub/ghamma/SampleEntries/

I have more.

-George Hamma

[engberg 89]

I'm guessing that the image in your public notebook was the one from your Windows client? The image recognition data I see on this note does not find the word "platform" ... you can see this when you search on that public notebook page.

Would you be able to able to put the comparison note from the Mac (with the different search results) into this notebook, too?

Also - how are you getting the image into the Mac client? E.g. are you starting with this JPEG file on your hard drive and then dropping it on our icon in our dock, or are you pasting it from Mail.app, etc.?

The reason I'm wondering is: My best guess is that something on the Mac side is transforming the image slightly so that we're actually processing a different JPEG file from the Mac than we are from Windows. I've seen lots of places on my Mac where it decides to be helpful by resizing or converting images as you move them between OS X apps. The images get converted to TIFF, then back again, etc. The end result looks the same to the naked eye, but the low-level representation has changed in ways that may produce different recognition results, particularly for tricky images with skewed or blurred text.

That's why I'm curious what steps you took on the Mac side to get that JPEG file into your Mac client.

Thanks

[cdhamma 0]

OSX version 10.5.2, Mail version 3.2 (919/919.2).

The image was sent to me from the Windows client using the Evernote software's email function.

I dragged the image from the OSX mail client onto the Evernote dock icon, and synchronized with the evernote servers.

http://preview.evernote.com/pub/coryhamma/CoryHammasNotebook

In this copy of the file, the word Platform is recognized. This is kind of interesting, because if the two image processing engines are the same, then I would offer this up as an opportunity to improve the engine by simply accounting for stretched images. I mean, it is one thing to go back into the recognition code and try to account for this, and another thing to just stretch the image horizontally or vertically and see if the engine is able to pull out more words.

[engberg 89]

Aha, that definitely explains it. The image in George's notebook is the original JPEG ... it's a 2029x2283 image, 1.7MB. In our web UI, we display a reduced version of the image (to look better in a browser), but if you click on it, it will expand to show the full sized image.

When you email a note from the service with big images (more than a megapixel), we do the same thing in the outbound email that we do on the web ... we reduce the image down to a size that would look good in the recipient's email client. This reduction also helps avoid email size limits at a lot of email providers.

That means that Cory received the reduced 568x640 "display version" of the image. Our image processing libraries got a different set of results on this image than they did on the original.

Why? Well ... that's not a simple question. We believe that we have the world's highest quality text recognition algorithms for processing text (and handwriting) in digital images, but this process is about 75% science (neural nets, splines, etc.) and about 25% art. Image changes that are subtle to the eye can often result in different recognition due to imperceptible changes in the color space, pixel boundaries, etc. In particular, resizing an image like this can frequently result in a different set of discovered words.

We rely on images like yours to help us improve our algorithms. Our R&D team can use this to determine why the algorithm failed to find the word "platform" in the large image, but succeeded in the smaller version. This gradually helps improve our technology by reducing false positives (finding words that aren't there) and false negatives (failing to find words that are present). This means that our recognition technology should keep improving with each release.

Thanks again for taking the time to send these to us.

[ghamma 0]

Thank you for clarifying what happens during a sequence of operations that, superficially, would be similar submittals to EverNote.

You noted that the browser-displayed version of a submitted image is of reduced resolution when the original resolution is greater than 1 megapixel. When I view the subject image in a browser, I can see the opportunity to expand the image when the magnifying glass icon is shown.

When I view the version of the image in the Windows client, however, I get a lower resolution version of the image without the opportunity to expand it.

Now that it's clearer what happens when sharing information from a set of notes, I have a request. If you're not going to provide the original (full-resolution) image, can you indicate it somehow?

-George Hamma

[engberg 89]

Thanks for the feedback, we'll give that some thought.