The goal of 100 Leaders is to encourage thoughtful conversation about leadership and the qualities that make an effective leader. Rather than using a simple vote (leaders receive one point for each vote received) or an up/down rating system (users vote to promote or demote individual leaders), this site invites users to rate leaders on five qualities of leadership, using a scale of 1 to 5 with half point increments.
How it Works
The ranking is based on the algorithm outlined in Building Reputation Systems (Randy Farmer and Bryce Glass). This calculation takes into account not only the average rating a leader receives, but also the number of ratings received. Sites like Yahoo and Yelp use a similar formula to order ratings.
The formula uses a constant “adjustment factor” applied to all items. It establishes a “floor” for the minimum number of votes necessary to be on the list. Major League Baseball statistics do something similar: players have to come up to the plate a certain number of times to appear in the stats. This eliminates the scenario of a player who bats only once, gets a hit (thereby “batting a thousand”), and is listed above a player with a .250 average who comes up to the plate many times per game.
The formula also establishes a “ceiling” for the number of votes so that there is no benefit to the ranking once the item being rated goes beyond that ceiling figure. For example, if the ceiling is set at 1,500 and the item has been rated 1,600 times, the item does not receive any additional bump. Given the large number of votes, the average score should be representative at that point. If the item is below the ceiling, it will receive a slight “bump” in its score to reflect the impact of the votes. The higher the number of votes, the greater the potential bump.
To promote transparency, 100 Leaders lists the rating and the number of votes so users can see the impact of the crowd on the final scores.
Note: Final overall scores are displayed to the nearest hundredth. To break apparent ties, scores are ranked to nearest thousandth.