Quality of moderators

[DutchPete 247]

The regulations for appointing moderators were changed recently: anyone with more than 300 posts can be appointed. While the 300 threshold is arbitrary & in principle is as good as any measurement, it is certainly not a measure of the quality of the moderator. Furthermore, with the "old" system there were clearly a number of very good moderators, as well as mediocre ones, plus some non-moderators who were very good.

 

My proposal is to bring in some form of quality by measuring the moderators' accumulated likes as a percentage of his total number of posts. A like is an indication of the usefulness of a post for the other users, and in a way is an approval rating. Only posters with at least 300 posts under their belt get measured. My idea is that someone with a percentage of somewhere between 60% and 75% (exact number to be agreed upon) is asked to be a moderator. Acceptance by that person remains voluntary of course.

 

Conversely, when a moderator's likes drops below the agreed level she/he loses the status of moderator. There should also be a page with a list of current moderators and their approval rating. Everything should be out in the open.

 

In my opinion this will prevent moderators & aspiring ones from posting just for the sake of posting to reach that numerical level of 300 posts or to maintain their status. It will also improve users' confidence level in Evernote's concern about its user & the quality of this forum.

 

I hope Geoffrey Barry reads this too & will give it some thought. We really need quality here!

[Wordsgood 526]

Interesting suggestion. I very much miss the old team of Mods, but your suggestion sounds a bit like a popularity contest. No offense meant.

p.s. The only thing I've used my so-called powers for is to remove blatant spam. Escorts, gurus and snake oil cure sales pitches. I have a very strong hate on for the third named spammers... They are, IMO, the lower than pond scum.

*Edited* Sept. 14

[DutchPete 247]

Interesting suggestion. I very much miss the old team of Mods, but your suggestion sounds a bit like a popularity contest. No offense meant.

p.s. The only thing I've used my so-called powers for is to remove blatant spam. Escorts, gurus and snake oil cure.

 

One could look at it like a popularity contest, but one where "campaigning" is not possible or hardly possible, which helps keeping the quality in place.

Evernote has created the "like" feature (like many other websites) so why not go one step further and actually make good use of it, rather than keep it as a "nice to have" thing.

[gazumped 11,661]

Not objecting to the idea at all,  but I wonder about the practical details.  With a couple of years' experience at doing this I can testify that lots of users who have been helped and have expressed their written thanks,  don't think to click the 'like' button.  On your proposed scale I'd score 25% forinstance with just over 1,900 likes.

 

One good thing about the new system is that it's pulled in a few people who haven't posted much before,  who are giving some really good answers (and occasionally the appropriate snarky cynicism),  but even with say 10 likes,  their ratio is going to be below 5% for a while.

 

I have seen a couple of posts where I thought about contradicting/ adding to the comments made,  but things haven't been so bad that I needed to jump in and warn someone about a dumb or dangerous process.

 

I think it's part of the moderating job to look at all posts and make sure something isn't being proposed that would make a situation worse,  and jump in to the rescue if that's happening.  Healthy discussion,  I'm sure,  would then ensue.  I'm sure Geoff and Charlotte and a few other employees will also do so if they need to - and we can always use the Report button if there's something blatantly wrong going on.

 

If someone is unhappy with the advice they receive,  OP's are also free to use the Report button to express their displeasure.

 

I agree newbies to the forums might be reassured if they got comments from someone with some kind of a Gold (or green..) Seal of approval,  or even if moderators had that title officially - but the system is still in its early days,  so no doubt it will be tweaked again.

 

Good suggestion though - I Liked it for you..  

 

 

[DutchPete 247]

Not objecting to the idea at all,  but I wonder about the practical details.  With a couple of years' experience at doing this I can testify that lots of users who have been helped and have expressed their written thanks,  don't think to click the 'like' button.  On your proposed scale I'd score 25% forinstance with just over 1,900 likes.

 

One good thing about the new system is that it's pulled in a few people who haven't posted much before,  who are giving some really good answers (and occasionally the appropriate snarky cynicism),  but even with say 10 likes,  their ratio is going to be below 5% for a while.

 

I have seen a couple of posts where I thought about contradicting/ adding to the comments made,  but things haven't been so bad that I needed to jump in and warn someone about a dumb or dangerous process.

 

I think it's part of the moderating job to look at all posts and make sure something isn't being proposed that would make a situation worse,  and jump in to the rescue if that's happening.  Healthy discussion,  I'm sure,  would then ensue.  I'm sure Geoff and Charlotte and a few other employees will also do so if they need to - and we can always use the Report button if there's something blatantly wrong going on.

 

If someone is unhappy with the advice they receive,  OP's are also free to use the Report button to express their displeasure.

 

I agree newbies to the forums might be reassured if they got comments from someone with some kind of a Gold (or green..) Seal of approval,  or even if moderators had that title officially - but the system is still in its early days,  so no doubt it will be tweaked again.

 

Good suggestion though - I Liked it for you..  

 

Good points. Perhaps the 60-75 bracket is too high. Also, the like buttons could be made more prominent so as to draw people's attention to it & encourage them to vote.

[BurgersNFries 2,407]

IMO, unless & until Evernote gets their ship together, it's fruitless to ask for anything.  There is nothing to indicate they are listening to their userbase or improving their product(s).  There are several things that indicate they have divorced themselves from their users.  (Making significant changes with no warning & no confirmation...users have to come to the board & discuss it amongst themselves & come to an opinion based on their collective experiences.)  The only positive thing I've seen in the past ~four months has been an increase from 1 gig to 4 for premium users.  But big whoop.  For one thing, even with my large database, I've only once or twice ever needed that many gigs & that was when I was adding a lot of things that I had scanned before using Evernote.  So I doubt there are very many users who even will ever use between 1-2 gigs, so 4 is overkill.  And those who do start using this extra upload limit, will begin to experience the usability problems I've already encountered.  So good luck with that y'all. 

 

Instead, give us something we can use, like a product that actually works with my existing database on the clients I use.

 

(This is another reason I don't post much anymore b/c this would be my reply to a lot of the threads.  And once I get going, the negativity just keeps flowing...I'm going to stop now.)

[DutchPete 247]

@ BNF: I understand your points. I just hope that soon they will realise something is wrong & needs to be fixed, especially when someone like you is bailing out. I thought the idea of the moderator reorganisation was to give people like G. Barry more time to devote to the forum & provide internal feedback.

If they do as they say they would, instead of paying lip service, then maybe some ideas/proposals fired at them will hit home & come to some sort of fruition.

That is my hope anyway.

[jefito 5,589]

Seeing as how the main duty of moderators at this time is to clean up spam, with secondary duties for moving content to more suitable locations in the forum, this doesn't really resonate with me as being particularly useful. Here's the whole of the moderation "rules" (from https://discussion.evernote.com/index.php?app=forums&module=extras&section=boardrules):

 

 

 

Reporting and Moderation
Our most active members (with 300+ comments in the community) will be allowed to help organize and tend the community’s grounds. When members reach this level they gain the ability to hide spam content as well as the ability to move content around in the forums. The “remove spam” ability should only be used to remove spam content, and the “move” ability should only be used to move content to its correct forum or sub forum. There is no requirement or expectation that these members perform these functions, however, their status allows them to do so if they wish.

Any member, no matter the level, can use the reporting tool to notify an Evernote employee to review particular content in the forums.

 

Moderator status doesn't confer any quality ratings whatsoever, or seal of approval or really much of anything except the ability to perform the above functions that you can perform, if you wish, at this point. You're not even asked to be a moderator; it's just a pure function of post count. Congratulations. You've been promoted to voluntary housekeeper. It's useful to have folks do that, and I don't mind doing it or having other people help keep the forums tidy, so that's not a bad thing. It's just not that much of a great thing.

 

It's up to individual users to make quality judgements about things other people post here, and anyone can look up any other user's "like"/post count ratio as input to that. I'd fail the 60% mark by a large percentage, by the way, and I'm betting that there are precious few here, if any at all, who have a rating higher than that figure.

 

There is a list of moderators available from the main forum (it's down towards the lower right hand side): https://discussion.evernote.com/index.php?app=forums&module=extras&section=stats&do=leaders. Not sure how up-to-date it is, or whether it reflects the actual moderator standings here.

[Metrodon 2,184]

Jef, I'd definitely give you a rating of 61%

[jefito 5,589]

Geez, thanks. A real vote of confidence.

[DutchPete 247]

Seeing as how the main duty of moderators at this time is to clean up spam, with secondary duties for moving content to more suitable locations in the forum, this doesn't really resonate with me as being particularly useful. Here's the whole of the moderation "rules" (from https://discussion.evernote.com/index.php?app=forums&module=extras&section=boardrules):

 

 

 

Reporting and Moderation

Our most active members (with 300+ comments in the community) will be allowed to help organize and tend the community’s grounds. When members reach this level they gain the ability to hide spam content as well as the ability to move content around in the forums. The “remove spam” ability should only be used to remove spam content, and the “move” ability should only be used to move content to its correct forum or sub forum. There is no requirement or expectation that these members perform these functions, however, their status allows them to do so if they wish.

Any member, no matter the level, can use the reporting tool to notify an Evernote employee to review particular content in the forums.

 

Moderator status doesn't confer any quality ratings whatsoever, or seal of approval or really much of anything except the ability to perform the above functions that you can perform, if you wish, at this point. You're not even asked to be a moderator; it's just a pure function of post count. Congratulations. You've been promoted to voluntary housekeeper. It's useful to have folks do that, and I don't mind doing it or having other people help keep the forums tidy, so that's not a bad thing. It's just not that much of a great thing.

 

It's up to individual users to make quality judgements about things other people post here, and anyone can look up any other user's "like"/post count ratio as input to that. I'd fail the 60% mark by a large percentage, by the way, and I'm betting that there are precious few here, if any at all, who have a rating higher than that figure.

 

There is a list of moderators available from the main forum (it's down towards the lower right hand side): https://discussion.evernote.com/index.php?app=forums&module=extras&section=stats&do=leaders. Not sure how up-to-date it is, or whether it reflects the actual moderator standings here.

 

If the function of moderator remains limited to cleaning up then that people will soon lose interest in that "congratulations" from Evernote & the quality of this forum will also suffer. Time will tell. I am in any case not interested in that kind of activity only.

[jefito 5,589]

If the function of moderator remains limited to cleaning up then that people will soon lose interest in that "congratulations" from Evernote & the quality of this forum will also suffer. Time will tell. I am in any case not interested in that kind of activity only.

But that's what's on offer; did you think that it was something different? If that were to change, then I might have another opinion, but this is the situation now as I know it.

 

The fact of the matter is that spam removal is not very enlightening, but someone needs to do it, and for the sake of the forums, I don't mind doing my part; I've been doing it for some years now. The prior evangelist role had a few more capabilities available (e.g. reading and commenting on reported content, ability to lock topics), but they were needed less often than just plain old spam removal and general cleanup. If you don't want to do it, then you don't need to, but as I'm interested in having a relatively smooth-running forum here so that we can offer help to people who have problems with Evernote, I don't mind pushing a broom every now and them.

[Metrodon 2,184]

I may be speaking a little out of turn here as I'm going to speak for all the ex-evangelists....

I haven't spent a stupid amount of time over the last few years on here cleaning up spam, fixing thread titles, occasionally helping people (and sometimes just being a bit of a dick for my own entertainment - hello Linux thread!!!) in order to get anything back.

I've never expected thanks from Evernote or from other users. I've done it because I mostly enjoy it and feels good to do something that helps people out.

If you expect more than that from being a mod on here then you are going to be disappointed I reckon.

Personally, I think I'm at least an 87% kinda guy.

[GrumpyMonkey 4,319]

Congratulations to our mew mods! I don't mind having a million moderators, because I am taking a break now from pushing brooms. I have slow Internet in the single-digit MB realm these days, so I avoid any extra page loads. I'll let you Google Fiber gigabyte folks pick up the slack

I agree that you'll be disappointed if you expect anything more than good Internet karma from your contributions here. And, unless those selected mods get extra perks like free trips to the Evernote mother ship for a chance to spend a week or two haranguing developers and management with their ideas for improvement, I don't know if elevation to a higher-status mods is going to be desirable to anyone.

Don't I have the most likes of anyone here? I think so, but I don't want to spend a few page loads to go find out. If I were elevated to some super status, I'd want to get Libin's phone number emailed to me and three Evernote wishes (I'd spend one of those on encryption, another on selective sync, and the third one on a big orange head)! Personally, I want to see high-level, substantive discussions here on the forums, and I'd like to do more to foster that, but I am a little too grumpy these days. Maybe after I've calmed down and rested a bit more.

metro and jefito get a solid 65% each from me

[jefito 5,589]

Personally, I think I'm at least an 87% kinda guy.

I 'liked' your post to help nudge you up a bit. You still have a ways to go...

[BurgersNFries 2,407]

(and sometimes just being a bit of a dick for my own entertainment - hello Linux thread!!!)

That's always a fun one.

[Metrodon 2,184]

Personally, I think I'm at least an 87% kinda guy.

I 'liked' your post to help nudge you up a bit. You still have a ways to go...

I 'liked' yours and then 'unliked' it because I am tremendously childish.

[GrumpyMonkey 4,319]

I used to like you a lot when I started this thread. Now I unlike you all, and I think I am the only one left I like

[BurgersNFries 2,407]

What are you all drinking/smoking? And why aren't you sharing? Oh wait, because the sharing is broken, again.

[jefito 5,589]

 

 

Personally, I think I'm at least an 87% kinda guy.

I 'liked' your post to help nudge you up a bit. You still have a ways to go...

I 'liked' yours and then 'unliked' it because I am tremendously childish.

 

It's what I like about you. You make me look good by comparison.

[ScottLougheed 1,316]

I may be speaking a little out of turn here as I'm going to speak for all the ex-evangelists....

I haven't spent a stupid amount of time over the last few years on here cleaning up spam, fixing thread titles, occasionally helping people (and sometimes just being a bit of a dick for my own entertainment - hello Linux thread!!!) in order to get anything back.

I've never expected thanks from Evernote or from other users. I've done it because I mostly enjoy it and feels good to do something that helps people out.

If you expect more than that from being a mod on here then you are going to be disappointed I reckon.

Personally, I think I'm at least an 87% kinda guy.

I'm with you on this Metro. We all came here as regular folk, and by chance were asked to be Evangelists. Now that's gone, and we are all still here, contributing largely to the same degree as we were as Evangelists, hopefully as well as we always have. 

 

I hear what you are saying DutchPete. In general though, I haven't noticed a significant decline in the quality of participation, largely because the quality didn't come from Evangelists alone (or at all...), it came from the whole community. Really I think the changes to moderation have been largely technical in nature with the biggest problem being a bit more spam lingering while that end of things gets sorted out. 

[DutchPete 247]

I am not against getting my hands dirty & helping in the clean up of the forum as & when required. But that is not what moderating is about. Moderator is a misnomer in this context. Maybe that was done on purpose to have people participate more willingly. A better title for the job would be "cleaner", nothing more nothing less. I did not realise that that was all there is to it.

But, if the new moderating function is just going to be cleaning, then I agree with the views expressed here that we don't need to raise the Quality. Might as well keep going the way are. We can close this topic.

[GrumpyMonkey 4,319]

We can't close topics anymore

Indeed. "Cleaners."

[DutchPete 247]

Oh well, such is life. I normally expect Kaizen (= continuous improvement, as you know better than anyone else) but the moderator issue is  going the other way, IMO. But we'll have to accept it, just like all the other things that are decided for us unilaterally by Evernote. It's a matter of "take it or leave it". Pity.

[jefito 5,589]

I am not against getting my hands dirty & helping in the clean up of the forum as & when required. But that is not what moderating is about. Moderator is a misnomer in this context. Maybe that was done on purpose to have people participate more willingly. A better title for the job would be "cleaner", nothing more nothing less. I did not realise that that was all there is to it.

But, if the new moderating function is just going to be cleaning, then I agree with the views expressed here that we don't need to raise the Quality. Might as well keep going the way are. We can close this topic.

Hey, I do appreciate the sentiment that started this, just so you know.

 

On the other hand, quality added to the forums comes from participation by interested and informed people. I'm all for that, but you don't need a "Moderator" title to add quality, as has been proven over and over here. Cleaning up also improves quality, but you do need to be a moderator to do that (cleaning up has always been a duty of forum moderators here, at least at some unspecified level of participation; now it's purely voluntary). People will continue to stay interested so long as Evernote provides tools that help them in their lives.

 

Whitewashing Aunt Polly's fence does not require moderator status, though.

[gbarry 2,658]

I had this awesome response and then I up and accidentally hotkeyed out of the tab and removed it all. Hate when that happens. What you see below is but a shadow of its former self

 

Thanks for the topic and great discussion.

 

We've found that quality (to the extent that you can measure something so subjective) tracks pretty well with number of comments made from a community member. Meaning, if you comment up to 300 while never really contributing, only providing short snippets of useless info and/or causing flame wars, you're going to end up getting removed long before you reach 300. Commenters who make it to 300 are almost always great commenters and will have shown that they are committed both to the space and to the commentary involved. I don't know of any member in this forum who wants these higher level member rights so much that they'd be willing to put in the time and effort to reach 300 non-spammy posts without months of on-topic commenting--that's a lot of work! But if someone happened to achieve that in short order? More power to them, and welcome to the community.

 

This is not to say that we couldn't incorporate "likes" or some other form of reputation system in the future, I think that's a good idea. But in terms of measuring quality, the "Like" system here hasn't proven to be the most robust way to identify a commentator's quality, at least not in a way that's immediately implementable. It actually tracks well as a delayed indicator for how often one comments--the people with the most "Likes" here also tend to have the most comments, it just takes a lot of comments to hit a lot of "likes" because the "Like" button isn't used all that often.

 

One of the principal reasons for the new moderator system was to give a clear path towards achieving moderator status, which we didn't have previously. Since we were now working with a more open path to moderation, we chose a conservative, limited set of moderator abilities. I think we've at the very least achieved a clear path for other members to become more active members of the community (and to achieve moderator rights if they would like to help out), and I hope to improve and expand on that in the future.

 

To the extent that these changes limited some of their anti spam powers, the ex-evangelists will be the first to tell you that on the spam fighting front things are still a work in progress as we bring more resources to bear on handling it 24/7. As a rule, we believe that this is something that we should be handling internally anyway. No one really joins the forums out of their love for deleting dubai escort service and trex muscle booster spam--you're here because you enjoy helping others or getting more information about Evernote. And we'd like to keep it more about *that* than the escort services.

[DutchPete 247]

Hi Geoffrey, thanks for the detailed explanation from Evernote's point of view. The points you make are valid & I think we can all live with them. At the end of the day we all want to have an enjoyable & fruitful forum experience, which is the primary reason for joining, like you say.

Also, your closer involvement, or in any case your closer visible involvement, is great to see. Many thanks for this response !!

[jefito 5,589]

One of the principal reasons for the new moderator system was to give a clear path towards achieving moderator status, which we didn't have previously. Since we were now working with a more open path to moderation, we chose a conservative, limited set of moderator abilities. I think we've at the very least achieved a clear path for other members to become more active members of the community (and to achieve moderator rights if they would like to help out), and I hope to improve and expand on that in the future.

Relative to this, I see that we now have the ability to change topic titles. Thanks for adding that (back, for some of us ).

[gbarry 2,658]

Yup. I view that as similar to moving a thread, you're simply helping it get reviewed and handled by adjusting titling.

 

Something I forgot to mention is that we actually have moderator notifications for all actions from the expanded moderator group. So, in addition to how community's naturally self-police, we also keep a relatively watchful eye on actions as well to make sure no one gets a little too...liberal...with moderation.

[jefito 5,589]

Something I forgot to mention is that we actually have moderator notifications for all actions from the expanded moderator group. So, in addition to how community's naturally self-police, we also keep a relatively watchful eye on actions as well to make sure no one gets a little too...liberal...with moderation.

"liberal with moderation", heh, I see what you did there...

 

 

Good to know, but I figured that something like that was in place -- charboyd has quizzed me on a spam removal operation that wasn't obvious (it was a duplicate post). BTW, I try to make my spam comments amusing for y'all. No need to thank me; it's all part of the service.