QuizMe: New Player Skill Rating

Thanks to Daniel, there will be a new world order on our quiz's leaderboard. In his own words (mostly), here's how our new play skill rating will work:

Currently, we rank players by the total number of points earned. After seeing the results of our different trials online, it was obvious that a major problem with using the total number of points was that it is largely dependent on the number of questions answered. It would only be an accurate comparison between users that have answered a similar number of questions.

Other flawed measures: Point average and percent correct -- both ignore the difficulty of a question and percent correct ignores how fast a question is answered.

Our new skill rating addressees all of these problems. Similar to the Elo rating system, our skill rating system takes into account a number of factors to determine how good a player is at answering questions.

The system works something like this:

Each player starts with a skill rating of 1500. Whenever a player answers a question, their current skill rating is compared to the difficulty rating of the question (which is determined by how other people answered the same question) to determine what the expected score for this particular user answering this particular question is. This predicted score is then compared to the score the player actually earned, and the player's rating is adjusted up or down depending on if the player scores higher or lower than expected.

For those of you who are interested in the actual formula:

To determine the expected score:

P = Player's rating before answering the question
Q = The questions difficulty rating

Expected score = 1 / (1 + 10^((Q-P)/800))

The adjustment to the players rating is made with the following equation:

P = Players original rating
E = expected score
S = actual score
K = a scaler that is dependent on the number of questions a player has answered: it ranges from 25 for new players to 10 for players who have answered a lot of questions.

New player rating = P + K(S - E)

To Like, or Not to Like, That Is the Question

More quiz features coming soon...

We've added a couple of new features to our development pipeline. They include commenting and question rating, allowing players to comment on a particular question/ answer/ topic and also allowing them to rate questions up or down, like or not like respectively. The first will help users connect with each other as comments will link to user profiles. The latter will allow our hard-working editors and our algorithms to serve better questions in the future. Until then, we'll keep you in the loop.

A Vote of Confidence

Big day in the U.S. today ... The whole world is paying attention to what may be the most important presidential election in recent history. Though at this time the race is not over, it's clear that the Internet has played a central role the 2008 campaign and positions itself as a big (huge) winner. It's possible that this view from our San Francisco-based office is slightly subjective, but there is no doubt that the access to real-time information and mobile devices have encouraged people to participate, speak up, act, share, link, comment, connect.

We're happy to be part of this.

In the past month, our 2008 Elections Quiz has gotten noticed in a few client sites around the States: The Detroit Free Press, The Washington Times and the Miami Herald. Across sites, more than 10,000 new player accounts have been created... People from different party lines have given our quiz their vote of confidence by playing for hours at a time and for more than 30 minutes on average. No, we're not kidding. :)

We've been checking out our stats between the sites running the Election Quiz and it's interesting to note that though Democrats overall beat out Republicans on the Detroit Free Press and the Miami Herald sites in percentage of answers correct, percentage-wise Republicans on the Washington Times site beat out all groups on all sites.

In terms of questions, most of you did not know that Harry Truman (and not George Bush) had the lowest approval ratings in the history of the US presidency. Give 'em hell, Harry's rating was all of 22%, compared to Dubya's 24% from this September. The hardest question was that William Taft fared the worst in seeking re-election. The guy only had 8 electoral votes in 1912!

Many of the most difficult questions were historical trivia, but fun tidbits from the candidates, like Joe Biden claiming to never have had a drink, proved fairly difficult as well. Though a few questions were answered correctly by all users, there weren't any questions that all of you got wrong (rats!).

No one got all 50 buttons, but "mark b," an Independent immortalized on Mount Rushmore on the Detroit Free Press came the closest with 49.