Thanks for replies! I got documents saved in harddisk and moving them to Evernote for better usability. Sometimes have them also in memory sticks, etc. When they're in Evernote, no more need them in hardisk.
This how to reproduce in Evernote Windows:
1. + New Note
2. Attach file (icon) or Attach files (File menu) from a folder in harddisk or usb-memorystick
3. When note is ready, close Evernote
4. Folder can't be deleted or memorystick can't be removed as they are locked/reserved by Evernote (for no reason)
5. Kill all Evernote processes and now deleting folders and ejecting sticks becomes possible
This doesn't disturb me a lot as I don't do this very frequently, but I liked to report it here if this could get fixed...