Jump to content
EP2008

Database approaching 10gb... any way to compress/optimize this?

Recommended Posts

I've got a lot of photos in my database - many are simply receipts, but they add up (after 14,000+ note).  Is there an EASY way to compress the images or the database as a whole?

Share this post


Link to post

There are ways to re-index your database, but I don't think you will save much space.

In Evernote Windows, Optimize Database: hold down the Ctrl button and press the Help icon in Evernote.

 

Instead of photos, have your considered storing your receipts as PDF"s? 

You could use a dedicated scanner (ScanSnap) or a phone app (CamScanner) and send them to Evernote

 

For photographs, I store them in Flickr and Google Photos. Pictures take up way too much space in Evernote

  • Like 2

Share this post


Link to post

I've got a lot of photos in my database - many are simply receipts, but they add up (after 14,000+ note).  Is there an EASY way to compress the images or the database as a whole?

hi. a more aggressive solution might be this:

http://www.christopher-mayo.com/?p=127

the only ammendment i would make is that i now use spideroak instead of dropbox for security reasons.

Share this post


Link to post

There are ways to re-index your database, but I don't think you will save much space.

In Evernote Windows, Optimize Database: hold down the Ctrl button and press the Help icon in Evernote.

 

Instead of photos, have your considered storing your receipts as PDF"s? 

You could use a dedicated scanner (ScanSnap) or a phone app (CamScanner) and send them to Evernote

 

For photographs, I store them in Flickr and Google Photos. Pictures take up way too much space in Evernote

 

Thanks for that tip, I wasn't aware of this hidden menu.

 

I do use pdf's for multipage documents, but find jpg files to be far more convenient because the previews will show up on mobile.

 

I recently purchased a program called JPEGmini, which does an amazing job losslessly shrinking jpg files, but there seems to be no easy way to use it on the images stored on Evernote - I'd love it if there was a "batch shrink / resize" option in evernote.  That could half the storage used by my database. I would imagine in the next 5 years, this will be an even bigger problems, since I expect my database to grow considerably by then. 

  • Like 1

Share this post


Link to post

I've got a lot of photos in my database - many are simply receipts, but they add up (after 14,000+ note).  Is there an EASY way to compress the images or the database as a whole?

 

AFAIK, there is no way to compress images AFTER they are in your EN DB, other than editing the Note, and editing the Image with external tools.

 

However, you can definitely take steps to reduce the image storage size BEFORE you enter into EN:

  1. When scanning receipts:
    1. Use the lowest resolution that still allows good readability.  You shouldn't need more than 200dpi, and even lower may work.  Run some tests to determine what works for you.
    2. Scan in B&W (not greyscale or color)
    3. Scan as GIF rather than JPG
  2. When scanning or saving Photo images:
    1. Resize to the smallest size that you find acceptable
    2. One option is to store only a small thumbnail in Evernote, with link to the high res image in one of the cloud services (like Flickr).
    3. Reduce the JPG quality field to the lowest level that produces images acceptable to you.

Over time both the EN Win and EN Mac database/folder seem to grow unnecessarily.

If you really want to go the effort to minimize your EN DB, then you can periodically do a clean reinstall of Evernote:

Although there are other advantages to scanning documents to PDF rather than JPG/GIF, I don't think doing so will materially reduce the resultant file size.  The PDF will still have to contain the image, plus the PDF wrapper.  So it might even be larger.

 

Finally, I'd suggest you might want to step back and take long term view of this issue.

 

Do you have the time, or want to spend your time, constantly working on (an worrying about) reducing the size of images you put into Evernote?

    OR,

Might it be a more productive choice to just buy a larger hard drive, or even better a large SSD?

Maybe buy an external drive (even a NAS) to offload other stuff on your computer to give more room for Evernote?

 

Each of us have different budgets and priorities.

I have taken the approach which requires the least amount of time and requires only a modest investment of $:

  1. Set my scanner to use a moderate resolution, and scan everything but actual photos to PDF.
  2. Store all of my high res photos in Cloud storage and/or external NAS (Network Attached Storage).
  3. While I store all scanned docs in Evernote, I put only a few selected photos (< 10%) in EN.
  4. My primary photo storage is #2.
  5. Bought as large a SSD (500GB) for my primary Mac that had a reasonable price.
  6. Transferred all infrequently accessed files to my NAS.

So, in general, image size in Evernote, and my Evernote DB size, is not something I worry much about or spend much time dealing with.

I do NOT do clean installs that I mentioned above.

  • Like 2

Share this post


Link to post

Thanks for the detailed post @jmichael.

 

I do many of those things, but I'll try to optimize my workflow when it comes to scanning in images (I use CamScanner).  I'd prefer to keep the high-resolution for many of my scans, and the tool JPEGmini I use does not result in image loss (like compression does), and it maintains the files full quality/resolution.  I've been looking at my largest notes, and saving attachments -> running them through JPEGmini -> saving them back to my note.  It's saves quite a bit, but it's the single images in single notes (thousands of them!) that has me at a standstill. 

 

It's also not only about keeping the database small on my PC, but my smartphone, too. having a 5GB+ database on mobile is very cumbersome.

 

Thanks again - I will work on long-term optimization. 

Share this post


Link to post

Keep in mind, too, that there's no reason to keep receipts for longer than 10 years, and that's only for mortgage payments for your house. Most other receipts can be destroyed or recycled after 2/3 years, or at most 7 (for anything the IRS might want to see).

I find it difficult to believe that you would really need to keep thousands of receipts for long enough to make your database grow to this size. Consider purging your oldest and no-longer-needed receipts to save space.

Share this post


Link to post
16 hours ago, Jason Aydelotte said:

Keep in mind, too, that there's no reason to keep receipts for longer than 10 years, and that's only for mortgage payments for your house. Most other receipts can be destroyed or recycled after 2/3 years, or at most 7 (for anything the IRS might want to see).

I find it difficult to believe that you would really need to keep thousands of receipts for long enough to make your database grow to this size. Consider purging your oldest and no-longer-needed receipts to save space.

Just to follow up with that, since my OP was over a year ago.

I've now got over 16,000 notes and my database is over 11gb large(!!).

If I calculate all my receipts going back to 2006 (approx. 4000), they total at most 1gb, so removing them won't make a difference.

We still need a way to optimize our database - build in file compression would be great, especially for jpeg files (which I try to make as small as possible before importing them anyway).

Share this post


Link to post
11 minutes ago, EP2008 said:

Just to follow up with that, since my OP was over a year ago.

I've now got over 16,000 notes and my database is over 11gb large(!!).

If I calculate all my receipts going back to 2006 (approx. 4000), they total at most 1gb, so removing them won't make a difference.

We still need a way to optimize our database - build in file compression would be great, especially for jpeg files (which I try to make as small as possible before importing them anyway).

Are you experiencing any performance problems - or is it just disk space

My Mac is performing nicely with this volume, and I know of other users with larger numbers/sizes
(Macs handle database performance better than Windows)

For disk space, you might look at some of your larger notes and move your attachments offsite

Share this post


Link to post
23 minutes ago, DTLow said:

Are you experiencing any performance problems - or is it just disk space

My Mac is performing nicely with this volume, and I know of other users with larger numbers/sizes
(Macs handle database performance better than Windows)

For disk space, you might look at some of your larger notes and move your attachments offsite

I keep my database on an SSD, and my computer is quite high-end, so no performance issues. 

But.... it's unusable on my ipad and barely usable on my high-end Android phone.

The largest file in my database is 80mb and the next largest is about 25mb, but most are well under 1mb.

Considering the monthly allowance is 10gb, I can't imagine how things would be if I uploaded that much. FWIW, my current usage this month (with 4 days left) is 74mb... but after eight years even that can add up to over 7gb of data.Other months use more data, especially when I'm attaching PDF documents and images.

Share this post


Link to post
59 minutes ago, EP2008 said:

I keep my database on an SSD, and my computer is quite high-end, so no performance issues. 

But.... it's unusable on my ipad and barely usable on my high-end Android phone.

The largest file in my database is 80mb and the next largest is about 25mb, but most are well under 1mb.

Considering the monthly allowance is 10gb, I can't imagine how things would be if I uploaded that much. FWIW, my current usage this month (with 4 days left) is 74mb... but after eight years even that can add up to over 7gb of data.Other months use more data, especially when I'm attaching PDF documents and images.

Unless you have any of your notebooks marked as offline there isn't a major storage hit on your iPad.  If you do use offline notebooks, provided you typically have an internet connection, consider a strategy of only having certain notebooks offline.  This requires far less space. 

I actually created a notebook in which to place notes that I wanted to be sure and have available if I was without a connection.  Otherwise I just let EN download the notes when needed.  Which shouldn't be too bad at all if most of your notes are < 1MB.  FWIW

Share this post


Link to post
18 minutes ago, csihilling said:

Unless you have any of your notebooks marked as offline there isn't a major storage hit on your iPad.  If you do use offline notebooks, provided you typically have an internet connection, consider a strategy of only having certain notebooks offline.  This requires far less space. 

I actually created a notebook in which to place notes that I wanted to be sure and have available if I was without a connection.  Otherwise I just let EN download the notes when needed.  Which shouldn't be too bad at all if most of your notes are < 1MB.  FWIW

I don't use offline notes on my ipad since it's always on WIFI. But it's slow and unusable.

On my Android phone, I don't use offline notes (database is too large) and the database is approx. 400mb, but it's very fast. 

I like your idea of keeping a specific notebook for only offline notes, but I'd rather do it by tag since I have very specific notebooks for very different things.

Database size is really what I'm hoping can be optimized since my daily drivers are my PC and Android phone.

Share this post


Link to post
5 minutes ago, EP2008 said:

I don't use offline notes on my ipad since it's always on WIFI. But it's slow and unusable.

Not sure what the problem is but Evernote runs ok on my iPad, even with my entire database stored 

You might try some self help
- delete and re-install (Warning: Hold off on this - the current release has issues)

Share this post


Link to post
14 minutes ago, EP2008 said:

I don't use offline notes on my ipad since it's always on WIFI. But it's slow and unusable.

On my Android phone, I don't use offline notes (database is too large) and the database is approx. 400mb, but it's very fast. 

I like your idea of keeping a specific notebook for only offline notes, but I'd rather do it by tag since I have very specific notebooks for very different things.

Database size is really what I'm hoping can be optimized since my daily drivers are my PC and Android phone.

Yeah, v8 on the iPad is sloooooow.  Instantaneous on the iPhone though.  Slowness may have to do with DB size or note count, but I don't really think so.  My iPhone DB size lists at 143MB and my PC version is close to 15GB.  So just the titles and the like on the phone, other than the aforementioned special notebook..

You really can't use tags for offline notes.  Only notebooks can be designated as offline.

  • Like 1

Share this post


Link to post
On 21.10.2015 at 10:53 PM, jbenson2 said:

There are ways to re-index your database, but I don't think you will save much space.

In Evernote Windows, Optimize Database: hold down the Ctrl button and press the Help icon in Evernote.

 

Thanks a lot for this - I wonder why this functionality isn't more easily available in the options.

My database is in use for quite some time but I almost only use text - I just put all kinds of ideas/thoughts brainstorm results etc. into EN and I also have a few longer notes that are kind of my actual scratchpad - they're an ever mutating blog of unsorted text. But in general I don't even use EN that much - still, given I just recently changed my Backup tool and while creating my backup set I wondered why my EN database is almost 1GB in size.

Luckily I found this thread, did the optimization and ... it's down to less than 40MB, much more reasonable given the actual content. 1GB still wasn't that bad but if it grows linearly with the amount of usage and data one packs into EN - it might be really interesting to those doing lots of writing primarily (obviously this only relates to those where the database growth wasn't just caused by the actual data stored like Photos etc. but due to what unused overhead was created due to many changes of existing notes).

Share this post


Link to post
4 hours ago, Nelvin123 said:

Thanks a lot for this - I wonder why this functionality isn't more easily available in the options.

Per Evernote, those tools are mainly intended for the use of technical support folks when trying to diagnose / fix problems remotely (over the phone, etc.). Most people don't need them, so they're semi-hidden, though maybe Optimize Database could be made more generally available. I haven't used it in a long time; I'm not particularly space-constrained. 

I have to say that 1 GB reducing down to 40 MB is surprising though...

Share this post


Link to post

I recommend using Google Drive, OneDrive, Dropbox, and other cloud services to store pictures. If you are really struggling to get data on Evernote, try temporarily storing the data to a cloud service, wait until the day of the limit break, and simply put all the pictures and other data into Evernote.

Share this post


Link to post
2 hours ago, Nelvin123 said:

I wondered why my EN database is almost 1GB in size.

Luckily I found this thread, did the optimization and ... it's down to less than 40MB

This indicates a serious problem.  I'm not sure how 40MB can possibly expand to 1GB
Unfortunately, there's not much to investigate in Windows; just a single .exb file.  
On Mac's the notes are maintained in separate folders, and the actual database is quite small

As well as optimization tools; I also find it usefult to rebuild my database from the servers.
Warning: the rebuild does not cover Local Notebooks or unsync'd notes

Share this post


Link to post
2 hours ago, J_Chi said:

I recommend using Google Drive, OneDrive, Dropbox, and other cloud services to store pictures. If you are really struggling to get data on Evernote, try temporarily storing the data to a cloud service, wait until the day of the limit break, and simply put all the pictures and other data into Evernote.

The problem being discussed here is not about the monthly storage limits. It's about the size of the resultant database. The context is most likely the Windows client, if as DTLow says, there is no central database storage on the Mac.

Windows client users, if they want to reduce the size of their Evernote database can now use the On-demand sync (Tools / Options / Synchronization / Enable on demand sync) to reduce the amount of note content stored on your local database. See:

 

Share this post


Link to post
1 hour ago, jefito said:

Windows client users, if they want to reduce the size of their Evernote database can now use the On-demand sync (Tools / Options / Synchronization / Enable on demand sync) to reduce the amount of note content stored on your local database.

This might be a good option, provided search works just as well.  Do you know of any downsides to enabling this feature (other than not having the data stored locally)? 

Share this post


Link to post
7 minutes ago, EP2008 said:

This might be a good option, provided search works just as well.  Do you know of any downsides to enabling this feature (other than not having the data stored locally)? 

You'd need to follow the various discussions. I haven't used it, as storage isn't a problem for me.

  • Like 1

Share this post


Link to post
23 hours ago, EP2008 said:

This might be a good option, provided search works just as well.  Do you know of any downsides to enabling this feature (other than not having the data stored locally)? 

I haven't made the jump yet because once I do, then there is no way to backup the database (exb file with ALL my notes) and while I like the cloud as much as anyone, I use it as backup, not as my primary source of everything. 

Share this post


Link to post

I just read in this thread that MAC has the notes within folders and a small database file. I wonder why they couldn't do that with Windows? Cloud services can play nice with WIndows folders (Dropbox, Google Drive...).

Share this post


Link to post
Just now, lisec said:

I haven't made the jump yet because once I do, then there is no way to backup the database (exb file with ALL my notes) and while I like the cloud as much as anyone, I use it as backup, not as my primary source of everything. 

I'm with you on that. But my (now 13GB+) database is quite hefty!

I think the most ideal solution would be if Evernote allowed for image compression/resizing either individually or in bulk (or both). 

Share this post


Link to post
35 minutes ago, lisec said:

there is no way to backup the database (exb file with ALL my notes)

I was wondering about this - not being able to do a full backup with “Demand Sync”

I’d like to hear from other users confirming this?

It’s not be critical for me; I do daily incremental backups (changed notes), so the data is captured.  I do a weekly full enex backup but it’s doubtful that I’ll ever use it

Share this post


Link to post
39 minutes ago, EP2008 said:

I'm with you on that. But my (now 13GB+) database is quite hefty!

I think the most ideal solution would be if Evernote allowed for image compression/resizing either individually or in bulk (or both). 

That would definitely help. I started a thread about that a while back - you can vote for the idea here: 

Another option might be to sync-on-demand by folder (or tag, which would be much better).  If I could tag notes as "archive" and know that those go to the cloud but everything else stays put, I would consider making the jump.

  • Like 1

Share this post


Link to post

True.  This, the delay in search, and offline note unavailability are enough for me not to use the option.  However, if you have space and/or speed issues it seems to help.  No space/speed issues for me so no need.  

Share this post


Link to post
57 minutes ago, DTLow said:

I was wondering about this - not being able to do a full backup with “Demand Sync”

I’d like to hear from other users confirming this?

I can't confirm it first-hand, since I don't use on-demand sync, but it makes sense. If all of the note content isn't in the .exb file, then backing it up won't be a full backup in any sense...

58 minutes ago, DTLow said:

It’s not be critical for me; I do daily incremental backups (changed notes), so the data is captured.  I do a weekly full enex backup but it’s doubtful that I’ll ever use it

...and that would apply to .enex backups as well.

Share this post


Link to post
7 minutes ago, cduong13 said:

I want to limit the Evernote Space Storage also.

Which platform/Device?

  • As noted in this discussion; Windows has a "Demand Sync" feature
  • Macs maintain a full copy of the database
  • Mobile platforms don't maintain a copy, but have an Offline feature for selected Notebooks

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...