(Archived) How Evernote Should Have Responded to security issue

RobertF57 · March 2, 2013

I live in Evernote. Evernote is mission-critical to my business. Here's how they should have handled this situation:

Send out an email to their 45M users before they configure the client to pop up a "change password" message. Maybe not everyone will have read that email before they use Evernote but at least Evernote will have made an attempt at warning them.
The client message should say that the user needs to change their Evernote account password. It should not display the email address associated with their Evernote account with the wording "Your password seems to have changed...". That message is completely inaccurate and confusing to the ordinary user.

Heinrich · March 3, 2013

Hi,

>>>

Hi Bankrobber, You mean those e-mails are being sent by hand. That's definitely a lot of typing. I'll better wait then, don't I. H.
You assume that it's premium users in Europe, but I am a premium user in the U.S., so I doubt that it's anything to do with location, though I wonder whether you folks in non-English speaking countries get translated emails in your preferred language...

Jeff, Do I sound like I need my English e-mail translated to me or that I run my messages through Google Translate before posting them here?

I do apologise for this unfriendly post to the other users on this forum but the utterly condescending replies of some of those 'EN Evanglists' are annoying. Regards. H.

Sugeeth Krish · March 5, 2013

@jmpsfs I'd care too hoots about whether the Pentagon was hacked or not, when my own a*s is on fire.

jmpsfs · March 5, 2013

Wow. This is what I saw:

1. Evernote got hacked.

2. No passwords (other than salted ones) were stolen.

3. They found the problem.

4. They immediately forced a password reset for all users, essentially blocking anyone who might have decrypted the passwords. (applause!)

5. At about the same time, they started sending explanatory messages.

I'd changed my password before my email arrived.

What in all this noise am I missing? I think they handled it fine. Props.

Nobody likes hackers. Nobody trusts anybody who's been hacked. Let's shut down the Pentagon.

dlu · March 3, 2013

Evernote's broadcast that was emailed to users sadly fits a classic phishing message.

Rather than being mailed from evernote.com and having all links pointing to evernote.com, the message and the links actually originated from and connect to
another site: mkt5374.com

The registrar for mkt5734 is MarkMonitor.com
The administrative and technical contacts are
Silverpop Systems
200 Galleria Parkway Suite 750
Atlanta GA 30339
This domain was only registered on 26 September 2012

The IP addresses are also different, if you check the DNS record A listing.

mkt5374.com maps to 74.112.69.20
evernote.com maps to 204.154.94.73

The evernote.com domain is registered to and have administrative and technical contacts at:
Evernote Corporation
305 Walnut Street
Redwood City
CA,94063

Evernote has violated a major principle of internet security that has been widely touted since phishing incidents began!

Never using a link that points to a site in a different domain than the one from which the message and the content is supposedly are sent is an extremely basic security rule.

Shame on Evernote for mailing a security broadcast with password reset links that are indistinguishable from phishing links!

Shame on Evernote for dragging their feet and not implementing Two Factor Authentication (2fa) in a timely manner!
This breach would not have been prevented by (2fa). However, 2fa would prevent against phishing emails - such as those just emailed by Evernote itself.

;-(

We used to send our announcements through software we run locally, but we're in the middle of a switch to SilverPop for delivering newsletters and announcements. They were the only way we could deliver 40 million emails in less than 24 hours, and we didn't have the experience to configure that mailing the way we should have. In the future, we'll absolutely make sure that we don't send similar emails with sketchy-looking links.

TonyKlein · March 3, 2013

A Sophos blog entry worth reading:

Evernote shoots itself in foot over “never click on ‘reset password’ requests” advice

jbenson2 · March 3, 2013

Evernote's broadcast that was emailed to users sadly fits a classic phishing message.

Rather than being mailed from evernote.com and having all links pointing to evernote.com, the message and the links actually originated from and connect to
another site: mkt5374.com

The registrar for mkt5734 is MarkMonitor.com
The administrative and technical contacts are
Silverpop Systems
200 Galleria Parkway Suite 750
Atlanta GA 30339
This domain was only registered on 26 September 2012

The IP addresses are also different, if you check the DNS record A listing.

mkt5374.com maps to 74.112.69.20
evernote.com maps to 204.154.94.73

The evernote.com domain is registered to and have administrative and technical contacts at:
Evernote Corporation
305 Walnut Street
Redwood City
CA,94063

Evernote has violated a major principle of internet security that has been widely touted since phishing incidents began!

Never using a link that points to a site in a different domain than the one from which the message and the content is supposedly are sent is an extremely basic security rule.

Shame on Evernote for mailing a security broadcast with password reset links that are indistinguishable from phishing links!

Shame on Evernote for dragging their feet and not implementing Two Factor Authentication (2fa) in a timely manner!
This breach would not have been prevented by (2fa). However, 2fa would prevent against phishing emails - such as those just emailed by Evernote itself.

;-(

Thanks for the very interesting behind-the-scenes information.

The Evernote email triggered my GMail to move the msg to my Spam folder with the following warning:

Be careful with this message.

Similar messages were used to steal people's personal information.

Unless you trust the sender, don't click links or reply with personal information.

By the way, this security incident was reported by the BBC in Europe yesterday.

And it was on the Drudge Report today.

"Massive online company hacked; 50 million passwords reset..."

.

BurgersNFries · March 3, 2013

Hi Bankrobber, You mean those e-mails are being sent by hand. That's definitely a lot of typing. I'll better wait then, don't I. H.

Snide 'a lot of typing" comment aside, how long do you think it would take to send out 45+ million emails...???

jefito · March 3, 2013

Jeff, Do I sound like I need my English e-mail translated to me or that I run my messages through Google Translate before posting them here?

I do apologise for this unfriendly post to the other users on this forum but the utterly condescending replies of some of those 'EN Evanglists' are annoying. Regards. H.

Sorry -- my reply was not meant to be condescending in any way. I should have probably quoted your other post:

Premium users in Europe have obviously not been included in the e-mail notification programme. Regards. H.

All I was saying was that I, an American Evernote premium subscriber, had not yet received an email at that time either. I didn't think that you could therefore conclude that absence of the notification email had anything to do with location of the user.

Martin Packer · March 3, 2013

@Feedback Good detective work. I hope this fits into my points 2 and 3: Someone made the decision to use this mailer. And there is probably a trade-off in play and a lesson of some sort to be learned. (I'm being vague as a lot of this is in the "none of my business" category and I'm not party to what Evernote went through as this unfurled.

Feedback · March 3, 2013

Evernote's broadcast that was emailed to users sadly fits a classic phishing message.

Rather than being mailed from evernote.com and having all links pointing to evernote.com, the message and the links actually originated from and connect to

another site: mkt5374.com

The registrar for mkt5734 is MarkMonitor.com

The administrative and technical contacts are

Silverpop Systems
200 Galleria Parkway Suite 750
Atlanta GA 30339

This domain was only registered on 26 September 2012

The IP addresses are also different, if you check the DNS record A listing.

mkt5374.com maps to 74.112.69.20

evernote.com maps to 204.154.94.73

The evernote.com domain is registered to and have administrative and technical contacts at:

Evernote Corporation
305 Walnut Street
Redwood City
CA,94063

Evernote has violated a major principle of internet security that has been widely touted since phishing incidents began!

Never using a link that points to a site in a different domain than the one from which the message and the content is supposedly are sent is an extremely basic security rule.

Shame on Evernote for mailing a security broadcast with password reset links that are indistinguishable from phishing links!

Shame on Evernote for dragging their feet and not implementing Two Factor Authentication (2fa) in a timely manner!

This breach would not have been prevented by (2fa). However, 2fa would prevent against phishing emails - such as those just emailed by Evernote itself.

;-(

Martin Packer · March 3, 2013

@dlu Thanks for your posts on this thread. I guess that makes you even more of a lightning rod than ever. :-)

I would like to see some kind of "post mortem", including 3 things:

How this happened.
Decisions made as the situation unfurled.
Lessons learned.

I realise - having dealt with similar things myself in the past when I did Security more than Performance for Enterprise customers - that some of this has to remain private. That's fine. You have to balance that against reputational and trust considerations. I'm confident Evernote will get it right and be able to genuinely give the message "we learned a lot, changed things that should reduce the risk and impact in future, but realise there WILL be further threats to the service we'll have to deal with".

C6REW · March 3, 2013

I am a Business User in the UK and had all my emails.
I think it is just a timing issue with some 50,000,000 emails!
Best regards
Chris

50,000,000 users and a company that claims to want to be a 100 year company and in 2013, we don't have 2 factor authentication.... Guess that Evernote wants to continue to live in the middle ages.

I know little or more precisely nothing about 2 factor authentication. I am sure I can guess what it means.

But I guess the good folk at Evernote will be on to this very soon.

Best regards

Chris

Sugeeth Krish · March 3, 2013

I am a Business User in the UK and had all my emails.
I think it is just a timing issue with some 50,000,000 emails!
Best regards
Chris

50,000,000 users and a company that claims to want to be a 100 year company and in 2013, we don't have 2 factor authentication.... Guess that Evernote wants to continue to live in the middle ages.

dlu · March 3, 2013

Hi Bankrobber, You mean those e-mails are being sent by hand. That's definitely a lot of typing. I'll better wait then, don't I. H.
You assume that it's premium users in Europe, but I am a premium user in the U.S., so I doubt that it's anything to do with location, though I wonder whether you folks in non-English speaking countries get translated emails in your preferred language...

Oy localization. Yeah we did our best to get as much of the content localized as possible. I believe we've localized the blogpost into all major languages as well as the emails, but that's not really my department.

jefito · March 3, 2013

Hi Bankrobber, You mean those e-mails are being sent by hand. That's definitely a lot of typing. I'll better wait then, don't I. H.

You assume that it's premium users in Europe, but I am a premium user in the U.S., so I doubt that it's anything to do with location, though I wonder whether you folks in non-English speaking countries get translated emails in your preferred language...

Heinrich · March 2, 2013

Hi Bankrobber, You mean those e-mails are being sent by hand. That's definitely a lot of typing. I'll better wait then, don't I. H.

skellam · March 2, 2013

We're actually trying to do both those things. There are quite a few moving parts and getting a bunch of clients to suddenly patch and release (regardless of where they are in their development cycle) can be challenging.

Some of the clients are updated to help guide you through the process and others will be updated soon. I'm sure there are edge cases where we don't give the most graceful UI, but we've tried to be as helpful as possible. We've also begun sending out email, check your inboxes!

cheers!

Certainly, doing all this on short notice can be challenging. With all the high profile security breaches in cloud services in the past year, was any consideration given that this might be a problem for which a plan should be in place before it happened? It's almost a given that a security breach will occur. I know bad things happen to good companies but I hope that changes will occur quickly and even that two factor authentication will be prioritized in the near future.

C6REW · March 2, 2013

I am a Business User in the UK and had all my emails.

I think it is just a timing issue with some 50,000,000 emails!

Best regards

Chris

Metrodon · March 2, 2013

Premium users in Europe have obviously not been included in the e-mail notification programme. Regards. H.

As I understand it, emails are still being sent. Just because you haven't received an email yet doesn't mean you have been excluded in any way.

Heinrich · March 2, 2013

Premium users in Europe have obviously not been included in the e-mail notification programme. Regards. H.

robsonj · March 2, 2013

As soon as I saw the message, I assumed a breach, but I am in IT so guess thats the way I'm wired. What I am impressed with and shouldn't be lost is that Evernote responded swiftly and a forced change of passwords globally which both fixed the issue and was even possible in a timely manner.

I've only been using Evernote for a week, having spent it digitizing our filing cabinet to Evernote. After reading much about Evernote security online & in forums, this response tells me I made the right choice in Evernote.

Of course things can be done better and improved all the time, but good job the Evernote team imho!

dlu · March 2, 2013

We're actually trying to do both those things. There are quite a few moving parts and getting a bunch of clients to suddenly patch and release (regardless of where they are in their development cycle) can be challenging.

Some of the clients are updated to help guide you through the process and others will be updated soon. I'm sure there are edge cases where we don't give the most graceful UI, but we've tried to be as helpful as possible. We've also begun sending out email, check your inboxes!

cheers!

Imagenomad · March 2, 2013

I must admit that my first thought was, "WTBH???" ("what the blinking heck???", family forum 'n all that).

My second thought was, "Trojan keystroke logger hell doom".

My third thought was, "Nah. It's the Evernote team getting things not quite right again but their intentions are good so I'll forgive them but my patience is beginning to wear a little thin".

shimra · March 2, 2013

The client message was confusing, and I think its because the clients were never designed to handle a password reset due to a security breach and they had no way to transmit a clear message. Hopefully this won't happen a second time

spg SCOTT · March 2, 2013

Hi,

1. They are sending out emails, but it does take some time. There are quite a few users.

2. I haven't seen this message myself (my be client specific or because I changed it in the Web Client first, I don't know) but it does sound a little confusing, given the chosen wording.

That said, the blog post does mention that they are updating the clients to handle password changes better, so this may come into that.

We are also releasing updates to several of our apps to make the password change process easier, so please check for updates over the next several hours.

(Archived) How Evernote Should Have Responded to security issue

Idea

RobertF57 4

Link to comment

25 replies to this idea

Recommended Posts

Heinrich 1

Link to comment

Sugeeth Krish 475

Link to comment

jmpsfs 23

Link to comment

dlu 628

Link to comment

TonyKlein 3

Link to comment

jbenson2 2,147

Link to comment

BurgersNFries 2,407

Link to comment

jefito 5,589

Link to comment

Martin Packer 162

Link to comment

Feedback 4

Link to comment

Martin Packer 162

Link to comment

C6REW 416

Link to comment

Sugeeth Krish 475

Link to comment

dlu 628

Link to comment

jefito 5,589

Link to comment

Heinrich 1

Link to comment

skellam 114

Link to comment

C6REW 416

Link to comment

Metrodon 2,184

Link to comment

Heinrich 1

Link to comment

robsonj 2

Link to comment

dlu 628

Link to comment

Imagenomad 20

Link to comment

shimra 14

Link to comment

spg SCOTT 736

Link to comment

Archived