Problem with the watched folder only importing the title and not the actual file to a new note.

[Fowleronline 7]

I've searched the forums, but cannot seem to find an answer to this problem. 

I came back to EN recently, after a period of not using it.  I have tried to return to being paperless and I use a Scansnap 500 to send PDFs to a folder on my Windows 10 PC (fully up to date March 2022). 

I am using 10.29.7-win-ddl-public (3186)
Editor: v140.0.17554
Service: v1.46.2
© 2019 - 2022 Evernote Corporation. All rights reserved

I have a folder containing 800+ PDF files of various sizes, and where I used to use an IFTTT recipe to sent them to individual notes in EN, I now find I cannot do it for some reason.  Instead I have designated my "watched folder" as the output folder of my Scansnap, names.... ScanSnap.  And EN sees them immediately a new note is created, ... BUT it only imports the title of the file and not the actual PDF.  

Please point me in the right direction, I've retried 3 times now and I am just getting duplicate notes, without any file to view and it is so frustrating to know that I have certain information, but cannot search for it, because the source file is not in EN, just the title.

Thanks in advance

Martin

Admins, Apologies if this is in the wrong place, please relocate as necessary.

[PinkElephant 8,819]

V10 does not support scripting, and maybe IFTTT will not do the job any more.

My workflow is (ix500, ScanSnap Manager):

As a preparation I have set up an import folder and connected it to my EN Inbox. You can create several import folders, and connect each one to a different notebook if you want.

https://help.evernote.com/hc/en-us/articles/209004967

Scan to folder, OCR by Abby active

Change the file name created automatically to a name that will serve as note title after importing. Set the target folder where to save to be the import folder. Save …

… and seconds later the scanned file appears as a new note in the designated notebook, file name converted into note title.

This works perfectly, and it makes sure the OCR is done before the import starts. When I scanned directly into the import folder, I got 10 or so notes with a corrupted pdf file, because the import was faster than ScanSnap to create the scan and finish the OCR. Only the last note was with a working pdf, but still with a note title that I needed to change.

My current process creates one note per scan, and avoids any rework. I just need to add my tags, and I am done.

[Fowleronline 7]

Thank you, we seem to have a very similar setup.  I seem to be doing it right with the import folder.  When I scan a file I give it a meaningful name and that is reflected in the note heading.  Individual notes are created in my inbox which is were I want them.  The folder contains 935 files which total just over 1gb.  My upload limit is 10gb so it's not that.  

So... now I have hundreds of empty notes, many appear to be duplicates.

I can only delete 50 at at time!

and I am still stuck with a folder full of scanned pdf files, that I want to import, to be able to search within them and no clear way to do it.

Could the OCR be an issue, if so, how do I import and OCR these documents?

Is there a way to search for and remove duplicate notes?

I'm a pretty low lever user, so I know I have a lot to learn, but is there a way to use Scansnap manager to re-scan all the PDF,  and OCR them again, and send them to a new folder, which I can then make my watched folder?

New scans seem to be fine, it's just the existing ones that are blank!

Thanks for taking the time to help me out.

Cheers

Martin

PS, just as a quick test, I tried a fresh scan, and EN imported it immediately and correctly... twice! once with the default scan name, and a few seconds later again when I made the name meaningful!!  Am I going to have to create a temp folder to then move what I want to import into a watched folder? Not exactly office automation as I want it...

[PinkElephant 8,819]

Don’t scan right into the import folder !

ScanSnap usually sets up a folder for itself, can be that one for the first storage. Whatever folder you use, don’t use the Import folder as scanning target.

Set up a completely separate Import Folder. I have mine on the desktop, because I drop other files there as well.

I have ScanSnap Manager OCR right after the scan, by activating the option in the scan profile used. It only takes a few seconds extra. The file in the SSFolder is already OCRed.

Now in the „new file“ dialogue I give it a meaningful name. In the target folder field I have the Import Folder selected. When I save it there, it gets imported.

With the corrupted notes, you can only delete them. There is a way to set the note selection limit up, which works with at least 500, but did not accept 1.000 as new limit. But it is not simple, I did it on my Mac.

EN has no duplicate finder. Sometimes sorting by title can help.

When you set up the separate Import folder, make it a new one. Since there is no import history, it should import any file you drop there. Just drop your existing files there (in batches, not all at once), and they will import.

Hint: We are already at release 10.32. 

[Fowleronline 7]

Thank you ... maybe the Mac works differently..

"You’re up to date! Evernote 10.29.7 is currently the newest version available".

I cannot seem to find a way of increasing the number of files I can delete on the app or the web version, could you point the way please, that would save hours!

I seem to remember Scansnap came with Abbey or something for OCR, but it does not seem to have that option now... do I need to add something do you think?

I will follow your advice on the watched folder and re import, thanks.

NB: I have now found a setting for ABBYY for Word, ABBYY for Excel and ABBYY for Powerpoint... I presume this is the OCR, but there is no PDF option, and that would be my preference.

Thanks again

 

 

[PinkElephant 8,819]

OK, but a word ahead: You should start to read software manuals, or play with software to learn about its functions.

About the EN version: The update information in the app unfortunately is sometimes not reliable. The version 10.32 was announced in the forum by EN staff, and when you go to the EN website and the download page there, it will download 10.32.

Yes, ScanSnap software came with Abby, and it will OCR pdf as well. The options you mentioned are to convert scans into files of the type Word, Excel or PowerPoint  - they are not the one we will use. We want a pdf, fully OCRed.

I assume you use ScanSnap Manager, not ScanSnap Home. I use ScanSnap Manager myself, Version 7.2 L30. To make it OCR your pdf, these settings are needed:

First open ScanSnap manager settings: Click on the program symbol (or on Windows somewhere in the Start menu), then choose "Settings" (red arrow).

When it is open, select the correct profile top right. I most time use the "Scan to folder" option, not the one with Simplex in the profile name. So let us check this profile. You see different tabs below. You should look through them, they all allow to adapt the profile to your need, or even create new profiles.

We now focus on the "File type" tab (arrow). In this tab the option "Convert into searchable pdf" (arrow) must be checked to enable OCR. Furthermore the radio button "All pages" must be set (arrow), else it will only OCR page 1.

 

Before scanning, make sure this profile is selected. On my Mac I do it by a right click on the program icon, and choose "Scan to folder" from the list of profiles.

 

 

 

 

[PinkElephant 8,819]

Now I have done a scan. It went to a folder ScanSnap Manager has set up for itself, NOT the EN Import folder. This window opens, and shows the scan result. If everything has worked with the OCR, before showing this window a dialogue box will briefly tell OCR is ongoing, and it will take a few seconds. 

Now I change the file name to whatever I think should be the note title later. So I click on the automatic file name, and change it. (Upper arrow)

After that I save the file into my EN Import folder. I have named my Import Folders "EN + notebook name", so the "EN Inbox" is the Import folder sending files into my Inbox notebook. After selecting the Import folder, hit "Save" at the bottom, and it is saved in the folder. Because it is a local function, the EN client needs to be running to import. Then usually a few seconds later a new note appears in the EN account, holding the file. It will sync to the server, where the OCRed content is read and integrated into the search index. This can take a little.

 

[PinkElephant 8,819]

My description about raising the number of notes that can be selected is for the Mac. Probably it works similar on Windows.

You first need to find the folder where the local Evernote copy is located, and in that folder the file "config.json". 

1. Quit EN v10 (not only close it, quit by the File menu)
2. Locate the folder titled Evernote and open it
3. Save a copy of the config.json file with an addition like .OLD as a safety copy.
4. Open the config.json file in a text editor
5. The "qa" section must be changed to this. If there is no QA section, add this code before the end of the config.json file, inside of the last bracket:

"qa": {

"multiSelectionLimit": 500

}

 

6. Set the number of notes you wish to allow.  There is a limit of about 1,000 maximum so there's no benefit in going for a larger number.  I have it set at 500.

7. Save the amended file and restart Evernote 10.

[Fowleronline 7]

Again my thanks.

I have read the manual (some time ago) and I regularly play with functions as I know they can change too with updates, but there still seems to be a difference with mac and windows.  Plus, having been away, you don't know what you don't know... 

I am using Home, but it was my understanding that Manager was part of home, and as I cannot get it to load independently, I'm downloading it again in full. 

SShome refuses to upgrade as I am already using the latest version according to the popup. 

SS Manager is not listed as an option in Windows. 

ABBYY fine reader  will not install either from the download as it says it is already installed.

I am going to assume it is corrupt and uninstall the whole lot and re install from the new downloads.

For EN, I should not have to check on a forum to see if I am up to date, that is what the "check for updates" button should do! Again, I did not know that was an optioin, and for my very basic level of use, I do not think the latest update would make much of a difference to my functionality... not enough to cause me a problem anyway... there is only me using it, with a small number of notes.

I'm sure it's safe but I haven't played with a config file since DR_DOS. 

Thanks again, I'll see how the re install goes.

Cheers

Martin

 

[PinkElephant 8,819]

Yes, it is either Home or Manager. Both are available for Windows. Fujitsu has posted a document to help decide whether to prefer Home or Manager on the same page where it offers the downloads. This link would be for my ix500 scanner and Windows 10OS:

https://scansnap.fujitsu.com/global/dl/win-ix500.html

To pick scanner and OS, there is this page. It lists all scanners and OS supported. First pick the scanner, then the OS, then the available software will show.

https://scansnap.fujitsu.com/global/dl/

From the first page I have linked, here is a screenshot. The help document Home/Manger is linked on top, the software downloads are below. The Manager software is listed below the Home option.

 

[PinkElephant 8,819]

In general with all ScanSnap software the trick is to set up the profile to support the own use case. The window where to select the relevant options starts from Home as well as from Manager.

It opens on my Mac when I have SSHome open and click on the "Scan" button. I can't proceed further, because having SSManager installed, Home will not work. But if you have the settings right and "Scan to folder" chosen, it will generate a pdf file with OCR in that target folder.

Set up your EN import folder, have EN v10 open and simply drop the scanned file on that folder. It will import, etc.

[PinkElephant 8,819]

Further remarks:

About the EN update button, I agree: It should work, but from my own experience and other forum posts I know it does not do the job reliably. Not good, but between 10.29 and 10.32 the differences are not very large (unless you have a new Mac with Apple Silicon chip).

About tweaking the config file: I fully understand everybody who does not want to mess with the system settings in that way. And it may not survive an update - I need to tweak it again from time to time.

There is another trick to make it faster when working on a multiple selection of notes: Go offline with your PC before you start. Quit EN (but stay logged in before, do not log out before quitting). Disable WiFi, pull the network cable, or use the network settings, whatever.

Open EN again. When you were logged in, and you have "Keep data when leaving" active, it should show all of your notes as always. Start selecting multiple notes, and modify (delete) them. Once offline changes will be processed by EN only on the local database, which is significantly faster. After changes are done, quit EN, go online, and open EN again. Changes will now sync in the background with the server. Best practice is not to use EN while the sync goes on.

It speeds up operation while you are doing them, and thus reduces your time spend waiting for the client to sync.

[Fowleronline 7]

That was actually the page I used, although I did not see the bit at the top... I am rather red/green colour blind and it did not stand out to me, though I can see it now I look carefully.

I don't think Manager offers me any extra value, as I do not use any of the other features it supports, as I said I am a basic user so Home will be fine.  I've opened ABBYY and set my defaults so I will see what happens. Nothing looks any different and no new options are available, but I'll give it a run later.

I am deleting hundreds of notes, 50 at a time, and EN is soooooo slow at deleting - it's taken all afternoon, but I'm nearly done.  A few more empty notes and I am back to proper ones.  How many can I put into the import folder at a time, so as not to create problems for myself? I would not have expected a problem, but clearly you think there might be an issue if I do 900 in one go lol!

I did try it off line for a while, but I didn't notice any real speed improvement, and I couldn't do other things in the background while off line, so I gave it up.  Once I finish, that will be it until tomorrow, so that should allow the servers to update.  My computer will get a reboot again too, it had one earlier and that helped, but it feels clogged up again with 50% memory (over a GB) being used by EN, and 25=60% processor on an i7... The fan noise is driving me crazy!!

Thanks again for your support.

 

[agsteele 3,059]

The most recent versions of Evernote do seem to process multiple note operations more speedily that prior to 10.30.

You can move multiple notes to an import folder but you shouldn't then shutdown your computer until you are certain everything has been imported. If you import 900 items then Evernote has tyo create 900 new notes each of which will take a finite amount of time. I'd do it all in small batches.

You could also it in the Legacy version of the software. The initial import will probably be quicker but then the Legacy application has to sync to the servers in the backgound. So still don't shutdown until you are sure the process is completed.

Import folders isn't designed to be a mass import function. Rather for importing individual files and attachments as you go along. So you might find you get a smoother, less troublesome result if you install the Legacy app and set up some different folders which will do it all in the background. You can have Evernote v10 installed and then install Legacy alongside.

[Fowleronline 7]

OK thanks, my tiny brain is already at it's limit, so I'll soldier on with the version I am running I think...  All I am doing is deleting 50 notes at a time and it's taking 1.3gb of memory and 30% of my processing power!  I've never heard the system fan work so hard!!  Thank you for your suggestion though.

But back to my original issue... Do you know why is it only importing the title and not the whole file, when it is clearly there and available in the watched folder.  A long time has passed so it's not a case of it catching up or processing, it's just done a hopeless job of importing my pdf files!

How many files would I put into the watch folder to have them imported correctly, 50? 100?  It might take a while before I can tell if it has done it properly.

 

[PinkElephant 8,819]

For me it imports correctly - although if I watch the process with EN open, first comes the note with the file name as title, and then a tiny lag, and then the pdf appears. The system seems to have some work to do in the background.

So probably start with a single file, see if it works, then raise the number slowly. Going all in means it may choke, and you get no feedback on when this happens. When doing it step by step, you will notice what works, and can go back to it once it starts failing.

Good luck !

[Fowleronline 7]

Thanks lol. 

I'm still deleting - 1,400 and counting, 50 at a time!! 

I found some earlier uploads that were just one line and no content, from an earlier attempt to correct this EN error.

I can't start with one file, not with 900+ to import!  Or is there a better way?  These are just my scanned files...

[PinkElephant 8,819]

You know the tale of the king and the chess champion. The king wanted to learn the game, and asked for the salary. The chess player said „Place a grain of wheat on the first field, 2 on the next, 4 on the 3rd, and so on, doubling the number with each field, and I will be happy to teach you“. The king thought it was a great deal, and accepted.

There was not enough wheat in the whole kingdom to pay the chess champion.

Only 10 cycles already takes you above 1.000 files (2¹⁰ = 1.024). And if it fails, you will know how much is too much.

[Fowleronline 7]

Then the King had the chess champion killed, and lived happily ever after playing draughts! 😁

If I find a number, I'll post it here. Thanks

[Fowleronline 7]

This is so bloody annoying.... A red banner appears at the top of the screen for about 4 seconds, then goes...

"Something went wrong when adding your attachment. Please try again".

What went wrong?

Which attachment?

Where is the list of failed uploads?

Why is the appropriate note not marked in some way?

Will it automatically retry? If not, why not?

So I painstakingly visit each note in the Windows app, to see if it's populated. I delete the empty note and in a separate window locate the file, edit the name to add a space, and EN re-imports it without an issue...

THEN EN CRASHES...

And on restarting it, it starts to import all the files in the watched folder again... ALL 550, that I had painstakingly added in small batches to avoid issues.

FFS...

More duplicates, more "something went wrong" messages.

For mature software, this is pretty *****... BTW, I'm not asking for help here just venting and if there is an more appropriate forum, or some way to flag it for the attention of staff, I would appreciate it. 

 

 

 

[agsteele 3,059]

As you have realised, these forums are used to user. So, whilst we sympathise and hope your venting had made you find some relief, none of us can do anything other than say that were not experiencing the issue you describe. If the options presented here haven't helped then you can only open a support ticket. 

[Fowleronline 7]

Thanks... Do EN monitor or look at the forums they provide?  Looking for help/support was what brought me here, I did not see a way to raise a support ticket... if you know how to do that, would you please point me in the right direction or provide a link so that I can contact them. 

I now have 1220 notes in my new notebook, of which half are now duplicates and many of them will be empty - I do not want to weed that again at 50 notes at a time if they can help me avoid it.

Thanks again.

 

[agsteele 3,059]

2 hours ago, Fowleronline said:

Thanks... Do EN monitor or look at the forums they provide?  Looking for help/support was what brought me here, I did not see a way to raise a support ticket...

The answer is that we believe that they are watched but it is unusual to see any responses from staff. Sometimes but not often.

Support Ticket – Evernote Help & Learning

2 hours ago, Fowleronline said:

I now have 1220 notes in my new notebook, of which half are now duplicates and many of them will be empty - I do not want to weed that again at 50 notes at a time if they can help me avoid it.

You can tweak the program configuration to increase the number of notes you can process at a time. I have it set to 500 but you cannot exceed 1,000. @PinkElephant noted how to do that earlier in the thread.

You might be better off installing the Legacy version of Evernote to bulk stuff. At the moment, whilst it remains available, you get access to the old ways of handling multiple note actions.

https://help.evernote.com/hc/en-us/articles/360052560314-Install-an-older-version-of-Evernote

Overall, though, I think your issue may be arising from the way you are adding these notes. I'm sorry that I haven't got a clear idea of the process that you're following. So before you add more notes you will want to get the duplicates cleared away. I saw that you were/are using an IFTTT recipe and that may well not be the best way forward in Evernote 10. @PinkElephant suggested scanning to Import Folders which is what I do with total success on a Windows 11 PC with Evernote 10.32.4

Otherwise, if you would treat me as an absolute idiot and describe step by step how you you are processing these documents I can see how things might be different for my successful approach.

[Fowleronline 7]

Thanks, but I won't trouble you further.  I'll raise a ticket.

I was doing as PinkElephant suggested and it was working well until the app crashed and then decided, on it's own, to import ALL the files in the watched folder, despite that nearly 600 had already been added to EN, and had been checked.  

I've created a notebook with all these recently uploaded notes in it and at least it will be searchable.  And start a fresh as of tomorrow with all my new files only and forget my archive.

Cheers

Martin

 

 

[PinkElephant 8,819]

It must have a record somewhere which files already have been imported. If this record gets deleted or corrupted, it doesn’t know that files already have been imported. I think this happened here - will be interesting what support tells.

[agsteele 3,059]

Are you using these forums to promote your personal link for Filemail? You seem to be posting in multiple threads virtually identical posts. Most of the threads you've selected are a year or more old.

Even if this is well intended, I'm afraid that this all feels like spam.

[Dave-in-Decatur 4,006]

It's absolutely spam. I've reported the 3 posts I've seen. If you've got an inside track, you might try to get their account deleted.

[PinkElephant 8,819]

It’s plain forum spam, I reported all of them and proposed the account to be blocked.