.png){kind=logo}
You've made it to the third and final part. Incredible. I applaud you.
We love writing trivia and we love turning that trivia-love into trivia-money, but we also love making software. At the end of the day, without software I’m just a man yelling out random trivia facts into my webcam.
Here’s the four key technical areas where we invested Ryan’s brainpower:
Remember how earlier I wrote that co-founder Ryan had written 100% of our code? Well, we decided to make life easier by moving the front end “marketing site” off Ryan’s plate and onto mine.
Inspired by this Tyler Tringas tweet, we started looking into what it would take to migrate our existing “marketing site” onto a “no-code” platform like Webflow. Well, not like Webflow. Specifically Webflow.
We found a contractor through Webflow’s market of “Experts” and in two weeks we had fully migrated our site.
Our site. Not our entire app. We created a subdomain so that everything behind “auth” (i.e. logging in) remains app.watercoolertrivia.com. That is our web application, and Ryan builds those pages with our other engineering contributors.
Moving the marketing site to my plate meant I could build new pages quickly, run promotions, and set up design tests without slowing down Ryan from building more important and more impactful features. Plus I got to up-skill. I’m no Webflow expert, but I can scratch together a fun side project every now and then on the platform now.
A/B testing is one of the many deities in the polytheistic church of Silicon Valley. Or something. So we hopped aboard.
Specifically, we’d heard for a few months that groups were disengaged during their free four-week trial of Water Cooler Trivia because the trivia questions were too hard. We heard this feedback, but didn’t act. That makes us bad entrepreneurs. Oops!
Then we did act.
Making that change to half the groups meant we could measure the impact.
Ultimately, the easier difficulty setting had a statistically-significant impact on increasing the share of free trials that converted to paying Water Cooler Trivia customers. Specifically,
So we rolled out the winning treatment. And eventually we launched even more difficulty settings. Accessibility rules.
Technically our Slack integration was released in late 2019 and our Teams integration was released in early 2021. Editor’s note: the integration is in the final stages of Microsoft’s approval process as of Jan 25, 2021. However, we still consider integrations a key part of our year.
Groups that sign up via Slack engage with Water Cooler Trivia about 2.5x more than groups that sign up via email. Realizing that, we added “Sign up with Slack” directly to our homepage. The popularity of both Slack and Teams exploded during the shift to remote work, so we doubled down on our product marketing and experience in the Slack and Teams apps.
Free-text trivia rules. It’s more satisfying to pull that European capital or Indonesian ingredient from the back of your mind than randomly guessing multiple choice option C.
Of course, grading free-text responses is significantly more challenging than multiple-choice and true-false. We increase the challenge by declaring “spelling doesn’t matter” on each quiz. It’s cool with us if you don’t know the difference between Schwarzenegger and Schwartzenegger.
%20(1).png)
Historically, we’ve used a few clever tricks to keep false negatives and positives low (<0.5% of all responses). As the business has grown, we’ve had more manual grading on our plate, often requiring grading in the middle of the night.
We worked with a data scientist to assess eight different free-text matching algorithms on a set of thousands of our own hand-graded trivia responses. Some of the concepts:
Unsurprisingly, we quickly discovered a tradeoff between
At one extreme, the “fuzzy partial ratio” matching algorithm (more detail here) was the most “aggressive” variant. Although it would have reduced our manual grading by ~50%, it was likely to introduce hundreds of grading mistakes per week.
We opted for a more conservative approach: “fuzzy token sort ratio.” We expect it to reduce our manual grading burden by ~10-15% without adding an increase in errors.
Well, in way more detail than you wanted to know, that was our 2020 at Water Cooler Trivia HQ. We’re thrilled to have you with us as we roll out more features and keep the weekly trivia fun rolling for your team in 2021.
