At the Visualized conference, I had a spark of inspiration when a speaker revealed one of his visualizations that was based on constantly-updating data had a very high return visitor rate because the updated data gave visitors a reason to return. This idea, while obvious, got me excited. If I could get a dataset that updates I could get returning visitors, which would allow me to take a more iterative, data-driven approach. As a product manager working in an agile environment, I had been wanting to apply this iterative process to my data visualization side projects.
A friend invited me to join a Survivor fantasy league (think fanatasy football, but for the TV show Survivor). After each week's episode the person running the league publishes the results in a data table on a blog post and league participants fill the comments with lively discussion and analysis. I thought this would be the perfect dataset for my iterative data viz: the data changed weekly, there was a group of people highly interested in the data (based on the comments) and there was an opportunity to provide a much better way to explore the dataset than the data table provided:
Unlike a fantasy football league consisting of a group of friends, the Survivor league is filled with 150 people. So, unlike fantasy football players wanting to see if they beat their friend, I hypothesized my users would care only about their score and rank relative to the performance of the league overall not to specific strangers. This informed my decision to make the page a "scorecard" where users first pick their team and then see their performance, with others' performance in the background only as context.
To inform the layout and content of the page, I thought of what questions the user would likely want to pose, and what analysis would answer these questions.
One of my complaints about the league's data table is that it's hard to see how well you're really doing compared to the other 150 teams. In the first few weeks, when teams hadn't earned many points yet, you could be in 50th place but only a few points away from first place. To show this better I chose a histogram. To answer the user's second question, performance over time, I tried repeating these histograms:
I really liked this approach, because it showed how the distribution spread out each week. After the first week (the top histogram) only 7 points separated first and last place. By week three the spread almost tripled to 20 points. Despite my enthusiasm, after testing with a few friends, it was clear this series of histograms was confusing. I was overloading the graphic. Instead, I decided to simplify by showing only one histogram and showing performance over time with a separate chart:
I wanted to give users insight into which of the 4 survivors on their team was contributing most to their performance. Instead of showing this single metric by player, I wanted to add the second metric of popularity - the percentage of teams who had that survivor. This is important, because your performance is relative to the rest of the league. If you have a great survivor, but everyone else has that same survivor, you aren't doing better than anyone else. So, to show these two metrics, I chose a scatterplot. I love scatterplots because they're easily understable by most viewers and they allow you to compare items on two different metrics without having to juggle both metrics in your head. In my past job, I often presented clients with user insights accompanied by quotes from their customers. I noticed these quotes were much stickier and more memorable when accompanied by a photo of the user. For my scatterplot, I did the same. Instead of a circle, each datapoint was a photo of the survivor.
I also labeled two of the quadrants ("Underrated Players" and "Overrated Players") to show the user the insight they could gather from this visual.
My initial hypothesis that users cared solely about their performance turned out to be wrong. I integrated behavior analytics into the site and captured each time a user changed the team. Analyzing the data in Mixpanel, I created a funnel to see how many times people changed teams. It turned out over a third of all users changed teams 3+ times:
Reflecting on my own behavior, this made sense. I kept asking myself "Now that I know how well I'm doing, who's doing the best and why?" I wanted to know which players I *should have* chosen. To accomplish this I added a leaderboard showing the top five teams and the players they have, in the form of a bubble chart. Each player was sized according to the number of points they contributed to the team.
The first few iterations of the design had a light palette. I wanted to try a darker color scheme to give it more of a dashboard-feel. This also allowed me to make the user's own datapoints pop.
The darker background also brought more attention to the team dropdown:
I thought this would increase the number of visitors to change their team, but analyzing behavior in Mixpanel, it didn't seem to have a big effect on behavior.
In my first iteration, I chose to focus on desktop. For mobile, I had a simple landing page. I wanted to increase the rate of mobile users returning on desktop, so I included a button that would allow a mobile visitor to email themselves a link so they could return on desktop:
I tracked this behavior and was dissapointed to find out less than 7% of mobile visitors emailed themselves a link:
But, before altering the design to be mobile-friendly, I looked at my traffic by browser and saw that less than 6% was on a mobile browser. I decided it wasn't worth the effort.
The success couldn't be measured in terms of traffic alone becaue the information was only relevant to 150 people. I measured the success instead based on return rate, using Mixpanel's cohort analysis. I was happy with the outcome: I had fufilled my goal to generated repeat visit behavior. Roughly one third of visitors returned a week later.
Due to the nature of the dataset, my sample size wasn't large enough to do all the analysis I wanted to do. The cohort analysis aabove shows I was only aquiring ~25-50 users per week. Had this have been more, I would've loved to have seen if cohorts coming in later weeks (when the design was refined) were more likely to return. I loved using analytics. For my next project I'd like to take the analytics a step further and A/B test design variations. This will require a dataset that has more appeal.
My biggest regret was deciding too early to design the entire visualization around the assumption the user would first pick their team from a dropdown. This limited my possibilities for adding insights and visualizations that weren't team-specific without a major design change. Going back, I would've prototyped a different interface that allowed for a better balance of team-specific insights and league-overall insights.← Previous Case Study