Overhauling the Points System

I was going to put this in a blog entry but decided that it would look more community-oriented if I put it in a forum post. V and Manoki, especially the former, have been beating me over the head in email about the points system--how I'm never going to be able to keep up with the points stories at this rate. I fear that V is right.

Now let me say this right off the top:

NO ONE IS GOING TO LOSE THE POINTS THEY HAVE NOW, NOR THE RIGHT TO REQUEST STORIES WITH THE POINTS THEY HAVE NOW.

OK? OK.

What we need to change is the way points are awarded. Right now, you get a point a post. People also get 5 points per referral.

This is what I wrote V and Manoki:

My original goal:
Encouraging participation, period
--I think we've achieved that goal, and as long as I keep writing and participating, that should continue.

My goals now:
1. Encouraging traffic
2. Encouraging referrals and widening the story base (see #1)
3. Encouraging me, but that's third by a wide margin.

So I'm throwing the floor open for ideas. What say you, gals and pals?

TheBoy's picture

Embodiment

I like the current point-a-post system.
--If you could cap it daily with minimal coding (no more than 6 points/day for posting) it might be a way to (at the margin) minimize post-whoring. Idunno whether that would reduce posting atawl.
--I think you should add rewards for referrals who join. Also, perhaps, a smaller bonus for their referrals, or for their persistent reading (i.e., active for three/six/twelve months, or 5 points for every bonus story one of you refer-ees earns).
--I think the bonus-story scale shouldn't remain an arithmetic progression. Your big-time posters (see, e.g., V, me) don't really need to have more questions than we've got on the list. Even if you just went to 75/150/300/500/700/900 (or somesuch), with the increased availability of points (plus bonus-points for service beyond the call of duty), the rate of questions would likely decrease somewhat.

This is not an entirely "deeply thought out" post, so much as what's percolated over the last day or two.

MeiLin's picture

Most High

--Post-whoring really isn't a problem now.

--Referral points right now are ONLY for referred folks who join.

--Geometric progression = interesting thought.

TheBoy's picture

Embodiment

the advantage to capping points for posting isn't just that it restricts postwhoring (fun as it might be...).
It also encourages people who really want stories to consider alternative sources of points more seriously.

This from a guy who's only been so effective at referrals...hard to do to pretty much anyone here. We're all supposed to be so "professional."

MeiLin's picture

Most High

I can award fractions of a point. So perhaps that's what to do--award fractions for comments?

Gudy's picture

Embodiment

A geometric progression will sooner or later bite the most active posters in the butt. OTOH, there are 52 weeks in the year, meaning roughly 50 bonus point stories per year, max. OTGH, scaling bonus point rewards with the number of active members would, if anything, discourage referrals. I'm not sure how to solve this tangle of partially conflicting ideas, but two things seem to be clear to me: a) reward those things more that you value more, i.e. traffic (how?) and referrals, b) decrease the average number of comments needed for a bonus story to a level that doesn't accumulate an endless backlog of bonus story questions at current or projected membership levels.

Fractional points for comments or a slightly more geometric points progression (e.g. 50, 100, 250, 500, 1000, 2000, 3000, ...) might indeed be worth considering.

Requiem's picture

Petitioner

"OTGH"

Wow, not only did you use the phrase, you abbreviated it. And here I thought I was a geek.

I'm for geometric progression, mostly for the reasons stated above, and also because I'm selfish and that still lets me get at least one bonus story.

TheBoy's picture

Embodiment

I had a teacher use OTOH/OTOH (on the one hand/on the other hand), but I don't get the G.

Gudy's picture

Embodiment

On The Gripping Hand. It's the usual hacker/SF geek way to continue the "one the one hand" - "on the other hand" progression. It was coined in a series of novels by Larry Niven and Jerry Pournelle featuring three-handed aliens with two hands on one side and a third, stronger hand - the gripping hand - on the other.

V's picture

Embodiment

5 of the stories earned over the past 2 weeks were people's first questions, the 50 point ones. If we take that as typical, then the audience is already about double what MeiLin can handle at 1, maybe 2 points stories per week. It certainly won't work if her audience triples. It's no longer a question of "How do we spread bonus stories around so everyone can get some?", it's "What actions or behaviors do we want to reward, and how can we reward them with the resources available?"

I'd suggest rewarding good, quality posts. TheBoy has a good idea later in the thread for rewarding good, quality (or at least, popular) questions. I'll flesh out an idea to reward quality posts down lower in a new commenting thread, too.

MeiLin has stated that we can all request stories with the points we have now. That means even if she calls an immediate halt to new points while we discuss and implement a new system, the 57 outstanding questions, at 1 question/week, could take us until late February 2010.

Capriox's picture

Embodiment

Instead of fractions for comments, inflate the value of referrals (10 points?) and do something like the geometric progression for bonus stories? I was thinking of that idea too when I saw you were re-considering the system. My thought was 50 points (so the first one is still easy?)/150/300/600/900, or whatever variation you like.

Would awarding an individual a point or three for reviews on Good Reads, etc. be more motivating for people to do so than the group goal you currently have now?

Maybe hand a couple points out for people who can show you where they've prominently displayed your banners or otherwise noticably advertised you?

You have the ability to remove points from comments that are obviously no-content-filler/post-whoring, yes? You could use that to take care of that problem if it arises, especially if the person already has gotten their first 50 points, instead of capping daily points. Not sure if that would be more or less work than setting up a daily cap.

Just my random early AM thoughts.

manoki's picture

Supplicant

that another crucial component of the points system is that it be easy to administer. So, nothing that requires her to evaluate the quality of posts and change points by hand.

We did discuss the idea of having the group evaluate points (like on Amazon when they ask how useful was this review to you?) we would be asked something like "how interesting was this post"? and people would get more points for the most interesting posts.

Katie's picture

Embodiment

I think the forum discussions will continue regardless of whether or not people get points, so...maybe only points rewarded for story comments?

The Vixen's picture

Devotee

I agree with that. Also, perhaps only one point per chapter. I was a bit surprised I was getting points every time I replied to someone elses post with "yeah, that's a good point."

TheBoy's picture

Embodiment

That, however, puts a serious disincentive on moving non-chapter-related comments to the forum.

The Vixen's picture

Devotee

you could just say that people don't get a points story till every 100 pts, and grandfather in anyone over 40pts currently.

MeiLin's picture

Most High

getting to 50 seems to be a toughie, but after that, it's all downhill. Smile

MeiLin's picture

Most High

Yeah, this needs to be as automatic as we can make it; I can't evaluate every single darn comment for merit. I don't do that now, though I threatened to! Smile

Having said that, so few reviews get posted that I'm happy to take time awarding points for reviews (even bad ones) and promotional efforts like flyers (I have a flyer half made-up). BUT YOU HAVE TO TELL ME! I can't go around the internets looking for stuff and triangulating user names etc.

Tigger's picture

I"m hoping this works. I've tried several times to comment, or make an account on the forum, and I keep having issues. Something about an SQL error. I don't quite understand the "points" except that it's a way to get bonus stories. I'm pretty basic though - just tell me what I have to do and I'll do it. Smile I love the little stories, the little insights into people. Of course, I am also LOVING the big stories - besides this, only ToMU and MToMU have managed to keep my interest. Well done!!

MeiLin's picture

Most High

Are you still getting the error? That should have been fixed a few days ago. I see you've got an account now, so yes?

V's picture

Embodiment

The sole purpose of this post is to provide raw data for any of those people who like to crunch numbers. Thoughts and interpretation will be provided separately. As I talked to MeiLin about posts, I took a snapshot of how many stories were earned, written, and asked. I broke it down by category starting from the top...people who earned 5 stories (400+ points), then people who earned 4, etc. The numbers of "written" stories is fuzzier because I just did a quick count on what was linked on http://www.meilinmiranda.com/bonus-stories which 1) contains non-points-stories and 2) isn't always updated immediately.

Stats as of 1/14/09
1 poster x 400+ points (5 questions) = 5
2 posters x 300+ points (4 questions each) = 8
3 x 200 (3) = 9
12 x 100 (2) = 24
24 x 50 (1) = 24

Total questions earned = 70
------------------------
Total written = ~24
Total requested/unwritten = 21
Unasked = ~25

Stats as of 1/22/09
2 x 400 (5) = 10
1 x 300 (4) = 4
4 x 200 (3) = 12
13 x 100 (2) = 26
26 x 50 (1) = 26
----------------------
Total earned = 78
Total written ~25
Total asked/unwritten = 24
Unasked = ~29

Stats as of 1/30/09
2 x 400 (5) = 10
2 x 300 (4) = 8
4 x 200 (3) = 12
14 x 100 (2) = 28
25 x 50 (1) = 25
----------------------
Total earned = 83
Total written ~26
Total asked/unwritten = 26
Unasked ~31

TheBoy's picture

Embodiment

1/14
400+ = 1
300+ = 3
200+ = 6
100+ = 18
50+ = 42

1/22
400+ = 2
300+ = 3
200+ = 7
100+ = 20
50+ = 46

1/30
400+ = 2
300+ = 4
200+ = 8
100+ = 22
50+ = 47

V's picture

Embodiment

OK--now, the interpretation. I really like bonus stories. The chance to ask a real author a question and get it answered, personally, in story...that is hella cool. But the thing is...there's lots of us, and hopefully soon many MORE of us, and we have to keep this reasonable. It's a simple balance. 1 story written = 1 story earned. If we earn stories faster than they're written, we'll build an ever-increasing backlog.

So...how many bonus stories would we like MeiLin to write per week in addition to the regular chapters? Be realistic here. It seems that 2-4 entries per week seem to be what she's capable of providing right now...so that means 1-2 bonus stories per week, maximum.

Between 1/14 and 1/30 (16 days) we earned 13 stories under the existing system, almost a story (60-80 posts) per day. That's a big problem, and more traffic will only make it worse.

1/14 stats: The top 6 posters earned 22 stories. The next 12 posters earned 24 stories. The bottom 24 posters earned 24 stories. The top 1/6, middle 1/3, and bottom 1/2 all represent similar slices of the pie.

1/30 stats: Top 8 posters: 30 stories. Next 14 posters: 28 stories. Bottom 25 posters: 25 stories. The distribution is skewing slightly towards repeat earners, but first-time questions still account for 30% of all stories.

I make this comparison to illustrate that point limits increasing in geometric progressions will not be helpful when you have an increasing fan base...there's more and more points being made, and many of the stories go to first time posters. It would almost have to be "one and done", one question per person, and even then it could fail when MLM hits 10-20,000 readers.

If we want a points system, we need to design one that 1) can limit newly-earned stories to 1-2 stories per week and 2) can maintain this no matter if there's 2,000 or 20,000 readers, plus 3) doesn't make further demands on MLM's time. That is our challenge.

TheBoy's picture

Embodiment

possible alternative:
each week, there's a drawing for a bonus story.
Base the chance to win on points.
Move the points into a separate "category" of points (expired).
Restrict posters' ability to win sequentially.

TheBoy's picture

Embodiment

flaw: too much favoritism to frequent posters?

bonus: it allows for exactly 1 awarded per week.

That means that Angel occasional bonus stories could be awarded for meritorious service, and (b) people would have time to make their selection (thanks to the backlog).

Problematic:
The transition (as with any scheme)

TheBoy's picture

Embodiment

I have no code-fu.
I have no knowledge as to this complexity.

Anyone may ask a question.
Points may be expended to support any question.
The highest-scoring question each week is moved from the list of "pending questions."

Again, it's a system that may over-favor big-posters.

MeiLin's picture

Most High

People post their question and the rest of us vote on them? Is that what you're saying?

TheBoy's picture

Embodiment

people post their questions.
People "vote" for them by assignation of points (again transferring them from "active" to "spent").
Weekly, you take the highest-scored question, and move it to the list of pending questions.

V's picture

Embodiment

So if I read you, points aren't "fractions of a story" so much as "currency". Someone (everyone?) can post a question, and then everyone else can spend their points trying to vote that question upwards...the one that draws the most interest is the one that's answered?

TheBoy's picture

Embodiment

right.

And, if you're persistent enough, you can just keep assigning your points to your question, and potentially get it advanced...(or you can convince V or me to put all our posts to it for a week...by donating to MeiLin. ;))

Capriox's picture

Embodiment

Ooh, I'm intrigued too!

I'm not sure I understood one thing - the question or two that "wins" each week - are they the ones that MeiLin would answer next? Or are they added to the pool-to-be-addressed, like what's in the bonus stories thrad now?

I like the second interpretation better, because it leaves MeiLin the flexibility to write what story she feels inspired by that week, or to choose one that, unbeknown to us, is particularly relevant to the main story.

TheBoy's picture

Embodiment

the latter.

MeiLin's picture

Most High

yes? interesting!

TheBoy's picture

Embodiment

si

Andrea can't login right now's picture

Hm.. but I'd still want you (Mei) to be able to choose what story to do when, at least to a degree. Say a certain question/prompt is really inspiring, but a different one is picked by the votes?

V's picture

Embodiment

I propose a system that rewards people for making posts that other readers find particularly helpful, insightful, or otherwise well-posted. The most similar system I'm familiar with would be Craigslist ("Best of Craigslist") with some traces of the Slashdot scoring system, although that part wouldn't be visible. It would require some custom coding, but hopefully it would be an investment for the longer term.

Every comment gains a "great comment" link beside "reply" and "quote" that anyone can click. We click good posts that we think meet whatever goals MeiLin sets out. It doesn't matter if people have different judgement...their judgements will average out. Then, posts that meet a certain threshold (number of clicks) earn a point. They might also get a little star by their title so other people can see what kind of posts are getting rewarded.

Example: Audience of 5 people. 3 people click posts generously, 2 people click only the great ones. "1" or "2" posts are bland, "4" are probably worth a point and "5" easily are. So the threshold for "point" posts could be set at 4

When the audience goes up 10x, you can do the same thing with your scale...posts scoring 10 or 20 are bland, but anything over 30 is going to be very good.

Now MeiLin has a throttle, a control valve...she adjusts the threshold where points are awarded so that posts earn points at about the same rate points are "used" by getting their questions answered. She can make infrequent adjustments in the background so that the queue doesn't get away from her.

There are a number of other intricacies/potential pitfalls to a system like this, but I think they could be solved with proper planning.

It seems that Drupal's voting system could be hooked up to posts and you could vote 1-5 on each post, but that carries problems. First...you need to avoid the comments off in the fringes that garner a few "5" votes and dominate the field. Second, even if you limit it to comments with a minimum number of votes, you still have issues when the audience and posting rate grow, resulting in more "good" comments. First the bar might be a 4-out-of-5 vote, then a 4.5, then 4.8, and eventually you'd need a 4.99973 before you could get a point. This system would focus on everyone agreeing exactly whereas the first system would better measure general agreement.

MeiLin's picture

Most High

...and people being afraid to post for fear of getting less-than-stellar marks. I have seen that happen. I'm looking through modules and capabilities and we'll see what I can come up with. I'm kinda leaning toward TheBoy's votes-for-points, myself...

V's picture

Embodiment

In the one I advocate, the scores are and always will be hidden. All that you'll know is that a post did (or didn't) reach the magic threshold that puts it in the top 20% of all posts (and that percentage will continue to shrink).

But yes, the vote system is very tempting if it's coded well.

gjh42's picture

I like TheBoy's idea. It takes some of the directness out of the current system (people are no longer solely responsible for getting a points story they want), but as the readership increases, that would become increasingly rarer anyway.

Voting for a question that intrigues you gives the average poster a feeling of influencing the course of events, while active posters would still get more influence on what gets moved to the "pending" list.

With the stories still identified as "X's points story", there is incentive for people to be the ones to come up with interesting questions, and questions that provoke general interest will get "pended" first. MeiLin could always elevate questions that she felt were particularly relevant regardless of the votes they were getting.

This also eliminates the non-content-generating activity of rating posts by whatever method. People with a lot of posts don't need the ego-boost of getting yet another personally chosen question. (I speak as a member of an e-commerce support forum with the third highest number of posts, where there are 30-150 *members* online at any time of the day or night) They can ask a question they feel strongly about and support it with their points, or they can support an intriguing question asked by a newbie.

Another matter would be limiting the number of questions asked, so there was not a flood of possibly trivial ones thrown out. There might be a (low) post count threshold before a question can be proposed, and some limit on how frequently one person can post questions.

MeiLin's picture

Most High

I've created some new user roles.

Petitioner: 50 pts
--Right to ask a question.
--Right to vote on questions.

I made some others, too, but I don't know what to do with them just yet. Here's what I'm thinking. At 50 points you get the right to ask a question and vote on questions. But when you vote, you "spend" a certain number of points. So you don't keep accumulating points; you lose some when you vote.

What think you?

seia's picture

Devotee

Sounds like a decent idea, but there's one thing that makes me confused about it: If somebody were to get 50 points and become a Petitioner, would they lose the rank because they voted and lost (random number) points, dropping to below 50?
It's not even that big a problem, you could solve it by making the rank dependent on total points earned instead of total points in possession.

TheBoy's picture

Embodiment

I like the points earned/points still possessed theory.

Also:
I think that the way to handle pre-existing accounts (i.e., all us registered now), particularly those who've requested bonus stories already, would be to count those bonus points that have been redeemed as already spent. E.g., I've got about 450 points, and have requested stories up through 300. If the policy went into effect -immediately-, I'd instead have about 150 unspent points.

viruslife's picture

Supplicant

I have a hard time coming up with something to ask, but would love to vote on other questions for one that I really like.

Andrea can't login right now's picture

Would the number of votes a question gets be the sole determining factor of whether or not that question gets answered?

TheBoy's picture

Embodiment

that had been my thought...

Andrea can't login right now's picture

I'm just wondering what would happen if there was a questions Mei found interesting but it never got enough votes to be answered.

Capriox's picture

Embodiment

Well, she still is the Author and Website Mistress... claim executive privilege and answer it anyway?

MeiLin's picture

Most High

I've set up petitioner, devotee, postulant, supplicant...and Most High.

Guess who's the only one ranked Most High? 8)

Capriox's picture

Embodiment

I was active in another forum that used a similar system of ranks/titles for more frequent posters and those who had particular jobs (it was a Dark Angel fan site - don't judge me! - so there were special titles for fanfic authors and fanart illustrators). I found that a helpful/enjoyable nudge to post more often to reach the next rank up, so I like this idea also, in combination with the points = votes system.

I don't like the rating posts system just because I've been to several sites with post ratings and either it doesn't seem to help improve to posting (Youtube, anyone?) or it's just a plain hassle, for me anyway, to have to go through and rate every post I read. Especially when my computer is being ultra-laggy like it is right now, that would really cut down on my enjoyment/amount of posts I read.

(Of course, my computer has gotten *so* bad now - no thanks to the recent trojan war - that I finally caved and am getting my first new computer in over 5 years. I'll be setting it up next weekend, eee!)

GreenGlass's picture

Supplicant

Oh man, that adds a whole new dynamic for me. I already struggle with my idealizing tendencies. Are you telling me to indulge them, my Most High?

gjh42's picture

I'm already on record in favor of this idea, and I think the proposed ranks are fun.
Tracking total points earned (for ranking) alongside points available for voting should not be hard.
Once set up, this would be essentially a self-regulating system which would filter story questions at a known rate no matter the readership/activity level. What's not to like?

Add new comment

Get an exclusive free ebook from the world of the Intimate History! Exclusive content, contests, new releases and more.