This issue is fixed for me since updating the app. Thanks guys! You wasted your time doing that. The only difference between Windows 7 and 10 in terms of usability is the start menu. You could have just installed Start10 to have a Windows 7 start menu in Windows 10.
I'm also having this issue with the duplicate icons. No other app is presenting the same issue in Windows 10 for me. Yeah I remember having the problem frequently with iTunes back when I still used it, but that was always a quick fix. You basically just had to play around with pinning and unpinning the icons separately until it worked. Nothing I do will get this Evernote icon to stop duplicating itself though.