Stories
Slash Boxes
Comments

Dev.SN ♥ developers

posted by NCommander on Wednesday February 12 2014, @11:42PM   Printer-friendly
from the e=mc-hammertime dept.
As the to-do list of tasks continues to be whittled bit by bit towards launch, one large item remains: distribution of moderation points. While I wish I could reuse the existing Slash code, the fact is that this code is completely inappropriate for a site smaller than Slashdot. So, I've been working with a couple of users in chat to rework the underlying math so that mod point distribution actually works in a reasonable way. If you're interested in the potential algorithm behind this, read on. Be warned, it is a bit dry and technical.
The current thoughts are that slash should enforce that a specific number of mod points must ALWAYS be in circulation, and in the hands of our potential moderators. Without going into too much detail, we're running off the assumption that the total number of moderations applied in an article should approximately equal the number of comments. To account for inactive users and for the constant flow of new articles/comments, twice as many mod points will be distributed.
Have I lost you yet? To reiterate, two mod points must exist for every comment in active articles. As far as problems go, having too many mod points in circulation is far preferable too little. If you don't understand why, I recommend trying to find +5 posts on other slash sites like Slashdot Japan or BarraPunto.

Here's an example:
Assuming that we have two articles with approximately 200 comments each, that means we need to have 800 points in circulation. However, if we stuck with the old Slashdot method of 5 points to a user, we'd end up needing to have 80 people read an article and not comment on it. Obviously, that's not going to work, and as we have more articles/comments, that number will only increase.

Instead, we'll limit the number of moderators to approximately 30% of active accounts who haven't moderated relatively recently. In this context, active means that you have logged in within the last 5 days. The process_moderators script will calculate how many mod points are currently in circulation, how many need to be in circulation, and how many, if any, it needs to add to the pool. It then looks at the list of eligible moderators, selects 30% of them, and hands them out. To prevent moderation fatigue, we'll cap the maximum number of points a user can receive, and by the same logic, also set a minimum. I don't know about you, but I'd be a little annoyed if I found I had gotten a whopping 1 whole mod point.

This should allow a relatively fluid system, and I hope, allow us to have something very similar to the moderation system we've all grown to love. I'm open to suggestions or even radically different ideas on how to improve this below.

Related Stories

Welcome Testers ... 23 comments
By the time this post goes live, we'll be proudly serving our first few users. Right now, we are still working on nailing some outstanding issues with slash (most notably, moderation), but we're at the point that I'd like to get some early feedback, and allow others to run through the system and test stuff. Please make sure to read the preceding posts, feel free to comment, and enjoy!

EDIT: Struck a naught word -- NCommander
Massive Site Progress - Status for 02/16 59 comments
So, since this site went up three days ago, we've made tremendous progress in getting this site up and functional, and many of you gone above and beyond to try and give us content and provide meaningful comments. I'm completely blown away to say the least. I've got some big news to share, and a list of site improvements made over the last few days. Read on for more details ...
End of Day 1: Systems Update 149 comments

So, as I write this, day one has officially come to an end. I'm still somewhat in shock over it. Last night when I was editing the database to change over hostnames and such, I was thinking, man, it would be great if we got 100 regular users by tomorrow. Turns out I was wrong. By a factor of ten. Holy cow, people. I'm still in a state of disbelief, partially due to the epic turnout, but also because our very modest server hardware hasn't soiled itself from the influx (the numbers are, well, "impressive" is a way to put it). Anyway, I wanted to do a bit of a writeup of where we stand now, what works, and what doesn't. Check it out (and some raw numbers) after the break! Warning, it is a bit lengthy.

This discussion has been archived. No new comments can be posted.
Display Options Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 1) by LaminatorX on Thursday February 13 2014, @12:34PM

    by LaminatorX (14) <reversethis-{moc ... ta} {xrotanimal}> on Thursday February 13 2014, @12:34PM (#57)

    I'm very glad to read you went forward with this. Are 2:1 and 30% educated guesses, or derived from analysis of /. threads?

    I wonder would it be possible for some positive or negative feedback to be added to the process? Perhaps periodically measure how much moderation is happening on the current front-page stories, and adjust the mod-point assignments accordingly.

    --
    This is our news site. There are others like it, but this one is ours.
  • (Score: 2, Interesting) by AudioGuy on Friday February 14 2014, @12:45AM

    by AudioGuy (24) on Friday February 14 2014, @12:45AM (#64) Journal

    I am having trouble seeing how you can tune ANY moderation system with so few people as we are likely to have at first.

    One possible short term idea - give the submitter of the article a fairly massive number of mod points. They may have at least an interest in keeping things on topic. There is also an easy hook or identifier of them since they are the submitter.

    Pick a few of the best moderators and just let them moderate constantly for a while?

    When you are speaking of moderation 'not working' you means these coefficients etc, right? Not actual buggy code?

    I wonder how Malda handled this originally. I imagine it started with just him, then adding a few people, then a few more, etc.

    Might be interesting to find an 0.9 version or similar and see what its moderation was like.

    This problem needs more heads applied to it.

    • (Score: 2, Informative) by unitron on Sunday February 16 2014, @02:42AM

      by unitron (70) on Sunday February 16 2014, @02:42AM (#214) Journal

      Malda started with the people running the site moderating, as I recall, and then added more and more privately invited users to help and then moved on to rotating through the list of registered users with halfway decent karma.

      But he was racing to catch up with the growing number of both comments and users, adding on the moderation and karma system after the site had been up and running for a while.

      Here, the situation isn't quite the same.

      --
      something something Slashcott something something Beta something something
    • (Score: 5, Insightful) by Non Sequor on Monday February 17 2014, @07:46PM

      by Non Sequor (1005) on Monday February 17 2014, @07:46PM (#1179)

      I'm of the view that the people with the strongest opinions on a topic are most likely to abuse power. Random passing samaritans are more likely to do good deeds, albeit with some random but mostly harmless counterproductive mods done for amusement.

      I remember reading that minor edits by unregistered wikipedia users are primarily positive. I take from this that people will typically make a positive contribution if: there aren't major barriers to action and there isn't much at stake.

  • (Score: 2, Insightful) by mrbluze on Friday February 14 2014, @04:25PM

    by mrbluze (49) on Friday February 14 2014, @04:25PM (#97)
    In the beginning the first people joining will likely be active, so distributing mod points to them is fair. By my reckoning it should scale well.
    --
    Do it yourself, 'cause no one else will do it yourself.
    • (Score: 1) by mrcoolbp on Saturday February 15 2014, @09:07PM

      by mrcoolbp (68) <mrcoolbp@dev.soylentnews.org> on Saturday February 15 2014, @09:07PM (#184)
      I agree, there should be some dished out to the alpha testers so we can play around with mods and the like.
      --
      (Score:1^½, Radical)
      • (Score: 1) by linsane on Sunday February 23 2014, @02:19PM

        by linsane (633) on Sunday February 23 2014, @02:19PM (#5252)

        Don't know if this thread is still active, but:

        gut feel is there are too many mod points kicking around at the moment and not enough commentators.

        a) there are few trolls around at the moment requiring down-modding (thus fewer points required to be in circulation at any one time)
        b) giving people mod points dissuades ppl from commenting
        c) comments are the make or break here
        d) with /. there was a nice middle ground where folk would leave your comment at +1 and not bother spending a point to up or down it. Here it seems the balance punts the comments straight up to +3/4/5 or straight down, with few left at 1
        e) combined with b), the short time period does also encourage holding back from commenting imho.

        The system will have to evolve over time but hope that helps for the moment...

  • (Score: 1) by zocalo on Monday February 17 2014, @05:32AM

    by zocalo (302) on Monday February 17 2014, @05:32AM (#528)
    Should be fairly easy to tune as the site grows and the number of active users/posts goes up. Looking forwards to watching that happen too.
    --
    UNIX? They're not even circumcised! Savages!
  • (Score: 1) by SurvivorZ on Monday February 17 2014, @04:42PM

    by SurvivorZ (792) on Monday February 17 2014, @04:42PM (#1053)

    Give the first 10,000 accounts 25 mod points every single day starting today and forever.

    You'll get 10,000 users really quickly!

    • (Score: 1) by migz on Wednesday February 19 2014, @07:44AM

      by migz (1807) on Wednesday February 19 2014, @07:44AM (#2344)

      Mod up! Er...

      But seriously don't complicated things, fixed number of mod points that expire at the end of the day. E.g. 25 per person per day.

      Those who feel strongly will burn their mod points on what they think is important. Those who are lurkers, won't affect anything anyway. Any wierdness in unforseens resulting outcomes can be remedied tomorrow without upsetting anybody who had been hoarding their mod-points.

      This system might encourage sock-puppets, but seriously who has time for that?

      Once karma gets going the allocation can get tweaked to take this into account. Eventually points can be tweaked start carrying over e.g. 10% points carry over to next day for active top karma scorers. Random allocation of mod-points can be added later.

      But KISS and get it going ASAP. Please ...

    • (Score: 3, Insightful) by Non Sequor on Wednesday February 19 2014, @08:18PM

      by Non Sequor (1005) on Wednesday February 19 2014, @08:18PM (#2997)

      10,000 accounts... 8,000 of which are controlled by a botnet upmodding spam posts.

  • (Score: 3, Insightful) by caffeine on Monday February 17 2014, @07:01PM

    by caffeine (249) on Monday February 17 2014, @07:01PM (#1153)

    I'm sure am not the only one who finds it annoying on slashdot when people hijack an early comment to get a position high on the page and take the thread in a completely different direction. Valid comments that get posted directly in reply to the article get lost by members hijacking threads above them.

    I'd like to see a change to the moderation system on Soylent News to help address this before it becomes an issue here as well. Perhaps a hijack moderation, if a reply post is considered to be unrelated to the parent. If a post gets enough hijack ratings, it is hidden and all it children suffer the same fate.

    I think a change like this would let us stop this behaviour and make a clear point of distinction with slashdot.

    • (Score: 1) by LowID on Tuesday February 18 2014, @03:26PM

      by LowID (337) on Tuesday February 18 2014, @03:26PM (#1823)

      A hijacked thread souks not be hidden, but simply re-parented at the base depth level. It may have an interesting conversation after all, and up votes should show it-just not high in the threads order.

      • (Score: 1) by LowID on Tuesday February 18 2014, @03:31PM

        by LowID (337) on Tuesday February 18 2014, @03:31PM (#1826)

        s/souks/should.
        Dammed text prediction.

        • (Score: 2, Insightful) by caffeine on Tuesday February 18 2014, @05:07PM

          by caffeine (249) on Tuesday February 18 2014, @05:07PM (#1914)

          I can see the benefit in keeping the good points but think allowing a hijacked message to be moved would not discourage the behaviour enough. I'd imagine people would quickly work out to post new discussions in the right spot.

          IMO, this has been building on slashdot for years and is to the point where people who make good points relating to the article being discussed are pushed off the first page by later replies hijacking earlier threads. I think this behaviour is part of the downwards spiral of slashdot. People with valid points are becoming less likely to comment on the actual article drowned out by attention seekers.

    • (Score: 1) by SpallsHurgenson on Tuesday February 18 2014, @10:34PM

      by SpallsHurgenson (656) on Tuesday February 18 2014, @10:34PM (#2099)

      Amusingly, such a system would have prevented the "fuckBeta" posts that resulted in the creation of SoylentNews.org.

      Some discussions are worth having even if they are only tangentially related to the topic at hand. What you are suggesting sounds like an attempt to censor portions of the discussion.

      If people don't like the direction others are taking the comments, they can mod it down. If enough people don't like it, it gets put below the default threshhold and most readers won't see it and that direction of discourse will die, but it will still be available to people who want to continue the argument.

      • (Score: 1) by caffeine on Wednesday February 19 2014, @02:13AM

        by caffeine (249) on Wednesday February 19 2014, @02:13AM (#2178)

        I'm not trying to censor any topics. As far as I am concerned people should be free to make any comment they wish. Just do it in a new thread if it is not really related to a current thread. The fuck beta comments often seemed to be the starting point of a highly positioned thread so would have shown anyway.

        My issue is with people pretending to reply to an early comment as a method of getting their comment listed high in the threaded display in order to get noticed. I'd just like the moderation system adapted to help discourage people hijacking a thread purely for position. I think the off topic mod is not really suitable for this.

        Of course, discussions tend to wander and deciding what is a hijack and what is just a diversion will not always be easy.

        • (Score: 1) by evilviper on Wednesday February 19 2014, @04:23AM

          by evilviper (1760) on Wednesday February 19 2014, @04:23AM (#2235) Journal

          My issue is with people pretending to reply to an early comment as a method of getting their comment listed high in the threaded display in order to get noticed. I'd just like the moderation system adapted to help discourage people hijacking a thread purely for position.

          Simple solution then... Change the default sorting to show readers the NEWEST comments FIRST. In the mean time, just use the "Off Topic" mod as intended.

          --
          Do YOU see ALL home-page stories?
          dev.soylentnews.org/search.pl?tid=1
          github.com/SoylentNews/slashcode/issues/78
    • (Score: 1) by evilviper on Wednesday February 19 2014, @04:19AM

      by evilviper (1760) on Wednesday February 19 2014, @04:19AM (#2233) Journal

      I'm sure am not the only one who finds it annoying on slashdot when people hijack an early comment to get a position high on the page and take the thread in a completely different direction.

      A good comment is a good comment. If the story sucks, or there's a tangentially related topic that's notable, it can be a better discussion than the topic.

      Anybody that has anything productive to say, on-topic, will still say it. Those going off on a tangent will stay in their own little thread, and will only be modded-up if they have something insightful/informative/interesting to say. I don't see the harm, and I've never been upset that part of a discussion went off the topic the subby/editor wanted it to go.

      --
      Do YOU see ALL home-page stories?
      dev.soylentnews.org/search.pl?tid=1
      github.com/SoylentNews/slashcode/issues/78
      • (Score: 1) by akinliat on Wednesday February 19 2014, @03:28PM

        by akinliat (1898) <akinliatNO@SPAMgmail.com> on Wednesday February 19 2014, @03:28PM (#2755)

        Agreed.

        The whole idea of off-topic always seemed a bit absurd to me. I mean, do we really need to be told what we can or can't discuss? I can't remember the last (long) conversation I've had with any friend that hasn't wandered all over the place, and it's one of the things I prized about /.

      • (Score: 1) by caffeine on Wednesday February 19 2014, @06:17PM

        by caffeine (249) on Wednesday February 19 2014, @06:17PM (#2913)

        >A good comment is a good comment. If the story sucks, or there's a tangentially related topic that's notable, it can be a better discussion than the topic.

        I agree.

        My issue is not with the comments being off topic, it is with the practice of placing them as a reply to an earlier discussion purely for page position. IMO, it is akin to pushing in in front of a queue. This is the reason why I think the off-topic moderation is not useful for this issue.

        I'd like the comments to be either related to the parent comment, or started as a new comment thread to the top level. This will make it easier to read and I think encourage more and better comments.

      • (Score: 1) by rcamera on Thursday February 20 2014, @03:20PM

        by rcamera (2360) on Thursday February 20 2014, @03:20PM (#3645) Homepage

        not only will they be off in their own little (or enormous) thread, but they may be modded as "Off Topic". if someone doesn't want to read through off-topic modded posts, they are completely able to weigh off-topic as very negative on their own.

        i've gone through and spent 5 "Off Topic" points in a day on a single off-topic thread. some of it was funny as hell, as i recall, but off-topic anyway.

        ot; i read at -1.

        --
        /* buck feta */
    • (Score: 2) by daffmeister on Thursday February 20 2014, @06:24AM

      by daffmeister (2141) on Thursday February 20 2014, @06:24AM (#3310)

      Wouldn't that just be "Offtopic"?

      • (Score: 1) by caffeine on Thursday February 20 2014, @06:59AM

        by caffeine (249) on Thursday February 20 2014, @06:59AM (#3333)

        I think off-topic relates to the comment as compared to the actual article. I'm talking more about how the comment is related to its parent.

        In this thread people have rightly pointed out that the wandering of discussions is a good thing. Off-topic seems more a tool to stop wandering rather than the behaviour of hijacking an early comment to get an early position.

        For me, I see a fairly structured threaded discussion as far superior to flat, time based discussions like youtube etc. Perhaps I am just getting old, but to me slashdot was moving slowly towards youtube and reddit. I just hope SN can do a better job of fighting this trend and these early days are a great time to discuss these issues.

    • (Score: 1) by marcello_dl on Friday February 21 2014, @10:14AM

      by marcello_dl (2685) on Friday February 21 2014, @10:14AM (#4303)

      If you want to discourage hijacking, maybe making sub posts less visible than root posts would balance the fact that discussions are mostly followed depth first. Just make the subposts only a click/tap away.

    • (Score: 1) by jmoschner on Tuesday March 04 2014, @02:14AM

      by jmoschner (3296) on Tuesday March 04 2014, @02:14AM (#10472)

      Maybe instead of a hijack rating, simply (or not so simply) make On-topic and Off-Topic seperate from the normal mod system and allow any logged in user to mark a comment as on or off topic without using mod points and while still able to comment in a thread.

      This way mods don't have to waste points on marking posts off topic and if a person makes an off-topic comment it won't ruin their karma.

      Then people can choose weather or not to show posts labeled off topic. It may even allow full threads of off topic discussion that are hidden to all but those who wish to see/participate in them.

  • (Score: 2) by wjwlsn on Saturday February 22 2014, @05:13PM

    by wjwlsn (171) on Saturday February 22 2014, @05:13PM (#4955) Homepage Journal

    Twice now I've had mod points disappear unexpectedly. Is that normal? Both times, I had used a few of the points, and then posted a reply in anotherhread (that I had not moderated in). After that, the points went missing.

    (I'm not complaining, but this does seem weird.)

    --
    I am a traveler of both time and space. Duh.
  • (Score: 1) by Pete (big-pete) on Tuesday February 25 2014, @04:22AM

    by Pete (big-pete) (1612) on Tuesday February 25 2014, @04:22AM (#6479)

    I have mod points today, and I just went to moderate posts in 2 stories, and in both cases there were a LOT of score 4 and 5 posts - all reasonably moderated, but they were comments I'd normally have expected to see at score 3 or so.

    I think you could decrease mod points by half and have a functioning mod system at the moment.

    Also it seems that because threading is the really old style where you need to click through to another page to see the comments, replies are not receiving anywhere near as much moderation love as top level comments. I think that it's valuable to fix that fairly soonish. Sure, I can go through to distribute my mod points at that level to try and balance things, but there's only so much one person can do.

    -- Pete.

    • (Score: 1) by Pete (big-pete) on Tuesday February 25 2014, @10:49AM

      by Pete (big-pete) (1612) on Tuesday February 25 2014, @10:49AM (#6629)

      Wow - or rather I HAD mod points...they don't last long do they? I didn't use them and they've gone already...I didn't have time to find stuff that needed moderating, and parked the thought until this evening - but now they've gone.

      Ho hum.

      -- Pete.

  • (Score: 1) by mrcoolbp on Wednesday February 26 2014, @08:33PM

    by mrcoolbp (68) <mrcoolbp@dev.soylentnews.org> on Wednesday February 26 2014, @08:33PM (#7665)

    Posted on behalf of user: linsane (633):

    "Don't know if this thread is still active, but:

    gut feel is there are too many mod points kicking around at the moment and not enough commentators.

    a) there are few trolls around at the moment requiring down-modding (thus fewer points required to be in circulation at any one time)
    b) giving people mod points dissuades ppl from commenting
    c) comments are the make or break here
    d) with /. there was a nice middle ground where folk would leave your comment at +1 and not bother spending a point to up or down it. Here it seems the balance punts the comments straight up to +3/4/5 or straight down, with few left at 1
    e) combined with b), the short time period does also encourage holding back from commenting imho.

    The system will have to evolve over time but hope that helps for the moment..."

    --
    (Score:1^½, Radical)