Dev.SN ♥ developers
Sections
Dev.SN
So, given my last journal, a writeup on how they work today. For the most part, my original story on this topic is true, but I changed a fair bit since then and now, nor did I go much into the thought process in how it was divined.
In contrast to the original system, the current one wants to keep a specific number of moderation points always in circulation, with the concept that mod points are a constantly moving and fluid item. Moderation simply doesn't work if there isn't enough of the damn things, and having too many wasn't a problem at all (Overrated exists for a reason).
The original idea is we should dynamically generate our pool of modpoints based on our activity levels, so the original implementation of this script took the comment counts for the last 24 hours, with the basic notion that every comment should have the potential to be moderated at least once. This number was multiple by two, and provided our baseline moderation count. Since we were based our mod point count on a 24h window, mod points were set to expire every 24 hours instead of every 72. At this point, I hadn't realized the fundamental problem with the slashcode moderation system; my thoughts were "need lots of mod points", "this is incredibly complex, I can do better". That realization came as I was stripping the old one out of slash.
As part of this, I also changed the eligibility requirements for moderation. Instead of having a specific number of tokens, I wanted only users who were active to get mod points. The ability to retain drive by moderations by lurkers was something worth maintaining, and part of what I suspect makes up the bulk of Slashdot moderations.
I also wanted to avoid the problem of "moderator burnout", or users getting mod points too frequently, and just being turned off from moderation. I know that happened to me on slashdot, and others as well who ignored modpoints (or chose to become ineligible). As such, I wanted there to be a cooldown on how frequently someone can get modpoints.
That being said, I didn't want everyone and their mother being moderators all at once, so I decided that 30% of all active users (defined (at the time) as anyone active within the last 24 hours) who had neutral or better would be eligible for modpoints.
Version 1 was fairly simple. It basically took the comment count for the last 24 hours, multiple by 2, this is the minimum number of modpoints that exist at all times. Take all users who were active in the activity_period, take mod_points_to_issue/(elligable_moderators*.3), and hand out those points equally. As a failsafe, the system will hand out ten mod points minimum (the underlying thought here being that I don't just want to get one or two modpoints; more is better, so lets take Slashdot's 5 and multiple it by 2).
And for the most part it worked. When we were in closed alpha on Thursday, we opened the test site to 100 users to try and test it in something resembling real world logic. And, for the most part it worked, because everyone was very highly active. You might see the mistake with that logic when applied to a production site.
Come go-live. User counts surge through the roof, active users are flowing in (can't believe we hit 1k users in a single day), and the moderation script starts handing modpoints in the thousands. At one point, there was close to 2000 modpoints in circulation at any given time).
For that moment, moderation was working well. Then users started going offlining, and EODing, or worse, users were getting modpoints when they signed off, and not seeing them until they signed in. The script was happy, 30% of users were moderators, but there were a lot of +1s. When I looked at the database, most people who had modpoints hadn't been signed in for hours.
Suddenly in a flash of inspiration, I saw the mistake. Slashdot could get away with handing out users with no activity level because even with 80% of their system being moderators, most people would be inactive at any given time. With our 30%, there simply weren't enough modpoints in the hands of active users.
So, in an attempt to salvage the situation, I did a critical adjustment on how the damn thing works. Activity periods for users was seperated into a new variable, and dropped to 1 hour (then five minutes, so any logged in user has a chance), and process_moderation had its crontab shorted to five minutes (it used to run hourly).
To keep modpoints constantly in circulation, expiration time was dropped to four hours, so only people who are active RIGHT NOW were moderators, especially since our editor team had posted 20 articles that day already. Whenever a user looses his points (via expiration or using them all), their slot is freed up, and a new user immediately gets modpoints.
That change in logic underpins version 2 of this script. Now the minimum count is what we hand out, except in the very rare case that we need more modpoints in circulation, in which case, the active users start getting more and more (up to a cap of 50, then it spills past 30 of users). For the most part, it seems to be working, comment moderation scores are generally going up, but it may still require further tweaking to make it work well. I generally am not seeing as many +3-5s as I like, but its right now a whole hell a lot better than it used to be.
I'm open to any thoughts, criticisms, or whacky ideas relating to how mod points are being dished out. Let me hear them below.
(Score: 2) by NCommander on Wednesday February 19 2014, @04:26AM
You said:
Observation 1: I saw mention of a problem with people getting mod points just before they logged out. Would it help to have a trigger of some sort that fires when a user logs in that checks whether mod points are available and whether or not this user is eligible? I'm thinking of this as an interrupt-driven *addition* to the polling mechanism currently employed.
Yeah, that's a definitive problem, and one I knew would exist going in. This is one of those cases where we would need to predict the future to properly fix. Its less of an issue on the other site due to their mod points sticking around for days.
Observation 2: A pet peeve of mine is that I'd get a message that I have mod points and they expire, say, on 02/18/14. Yeah, but WHEN on that day? Do I have 5 MINUTES or 23 HOURS left? I try to use my mod points judiciously and would rather let them expire than shotgun them out there just because I got them. OTOH, if I'm in a pretty good discussion, and I know I've only a short while left, I'd be more generous in their distribution. If I've got several more hours, I might wait a bit to see if I contribute something to the next story or two that will post. IOW, full timestamp, please!
A known (and annoying) bug. The thing that displays the text is actually completely separate from the rest of the moderation code, and the bit of code where it does lie is in a section I'm hestiant to touch until we have a full dev environment up and running again. It's on my TODO list pretty much on the top for when I return to hitatus if none of the other devs got it.
OOOOH, I like this idea. Its actually rather trivial to implement as we have the lastgranted data always available. Just needs a new column in the database. Right now, the userbase as it is means that there's about a 24 hour gap between getting points (more or less depending if and when how many users are signed in at a given day). Consider this one on the TODO list.
Still always moving
(Score: 2) by NCommander on Wednesday February 19 2014, @04:26AM
Meant to click preview on that one :-)
Still always moving
(Score: 1) by martyb on Wednesday February 19 2014, @09:06AM
Thanks for the feedback! I realized as I submitted it that I really should have made 3 different posts, but lacked the time to do it over; still learning after all these years! I can, at least, split up my replies!
Observation 1 (assignment time of mod points delayed until login). Yeah, I knew you'd rewritten big chunks of that. I'd seen a long history of fiddling with the code on /. too. Glad it's on your todo list...
That said, I've thought of a possible abuse and workaround. "Hey! I get mod points when I login... maybe if I keep logging out and it, I can get some more!!!"
Blegh. How about waiting, say, until 10 minutes AFTER login before those points are considered for distribution?
(Score: 1) by martyb on Wednesday February 19 2014, @09:11AM
Yeah, take that well-deserved break!!!
Reading your reply, I vaguely recall having seen something about this, before. Thanks for the explanation.
I understand not wanting to break something that currently works well enough to get things started. Glad you find the idea a viable one, but agree it's best to keep things stable before making thing better.
(Score: 1) by martyb on Wednesday February 19 2014, @10:09AM
Glad you liked it!!
Further considerations on implementation and avoiding possible abuses...
What happens if I "delay" for, say, 12hours. Do I still have my mod points? That is what I was originally thinking, but I've got a gut feeling that could cause some problems.
If I ModPointDelay (MPD), 1 hour, I'd expect I'd still have the points when the time is up.
If I MPD 2 Days, and upon delay expiration still have the points, then I wonder how you would balance that against the pool of assigned/available mod points?
Mod point availability surge? Here's a thought experiment:
As you can see, these are all going to become available at the same time. In this case causing a surge in availability of 60 mod points. Take a few thousand users, add in various delays, and ummm, I'm not sure what would happen, but it could be interesting :-)
Then again, what if NONE of them log in during the period? Sure, their mod points will expire, but we'd need to plan around the system-wide impact between these extremes.
Random thoughts: Have mod point "buckets" for each 15-minute period? MPDs put the user in one of those buckets. When system is looking to put out mod points, it should look though the MPD'd users who are now available?
This is NOT simple! Here are some factors that come to mind:
It's an interesting puzzle! I'm curious to see what your thoughts are on this!
Off topic to this thread, but... "Just needs a new column in the database." Hmmmm... could you add a column to the user's table for when their Mod Points expire? Query THAT when presenting the mod point expiration to the user (as both a duration and as a date/time.) Sounds too easy, so I must be missing something!!?!
(Score: 2) by NCommander on Wednesday February 19 2014, @12:25PM
The system always considers 30% of logged in users as potential moderators. If there aren't enough modpoints to comment counts, it starts handing out more modpoints. If it hits the upper gap, it causes a spill to go beyond 30%.
Still always moving
(Score: 1) by Reziac on Thursday March 06 2014, @11:12PM
I take moderation seriously. That means I *don't* want to have to rush around just to spend mod points... which would make it stressful and no fun. When I get 'em, I'd very much prefer to have 3 days to dole them out as I come across good comments (tho I may spend them all on one article, too, who knows)... to feel like I can take my time and think about what's being said.
If I know I've got just four hours to spend 'em... which by dumb luck had better coincide with when I'm here... I won't have time to dig in the bowels of comment chains.
I went thru a period on /. where for several months I seemed to have permanent mod points. I had 15 points no matter how many I spent. The only difference it made was that since I knew there was no rush, I could spend them very judiciously. I did a lot more dumpster-diving in long comment chains, dredging up the worthy. I didn't fret about which post more deserved my last mod point. -- And on days when I didn't feel like moderating, I just ignored 'em.
Incidentally, with very rare exceptions, I only mod up. There are enough people willing to mod down already.
[I've had mod points here a couple times already. Didn't realise how lucky I was. ;) ]