Ever since publishing the Wait Wait Stats API, I have been working on some of the technical debts that had been accruing throughout the development and release process for both the API and the underlying libwwdtm data access library.
One of those items in the technical debt list was to refactor the libwwdtm modules and make the code easier to manage and maintain. Instead of having every function related to a module reside in one code file, those functions are now broken up into core, details, info and utility sub-module under each major module (guest, host, location, panelist, scorekeeper and show).
Once the libwwdtm code refactor was completed, it was time to update the Stats API to make use of the refactored modules and make the code a little bit more readable. After doing a thorough set of regression and performance testing, I unleashed the changes to the world.
After the refactor project was completed, I went on to complete another item from the technical debt list: adding in Bluff the Listener stats to the panelist details. To be honest, this is something that I completely forgot to include for the first release and realized that I missed it after the initial release. That said, it was a pretty simple task to complete and finished updating libwwdtm and the documentation updates in under a day.
Since the inclusion of the Bluff the Listener stats was incorporated into the panelist details endpoints, no new API endpoints were implemented and the returned response now includes a new “bluffs” object containing the number of times the panelist has had the correct Bluff the Listener story and the number of times the panelist had their story chosen*.
Sample panelist details API response that includes the new “bluffs” object
The documentation for the Stats API has also been updated to reflect both the code refactor and the new Bluff the Listener data in the response. The documentation is available at https://api.wwdt.me/docs/. If you would like to look at the code for both libwwdtm and the Stats API, check out the libwwdtm and the api.wwdt.me repositories on GitHub.
*: Please note that I have not gone through previous shows to collect Bluff the Listener information for most of the show’s history; thus, the statistics provided are incomplete. Eventually, I will backfill the information as time permits.
For the past few years, people have requested access to the data that I have been collecting for the Wait Wait Stats Page to play around with the data or build a brilliant, interactive infographic. Each of those requests would require that a data export of the requested tables or a MySQL/MariaDB database dump to be created and sent over to the requester. Of course, those exports and database dumps are just snapshots of the data and another request was needed to get update data each time.
That got me thinking about what it would take to create an API service that people can use to get data directly from the Stats Page database. At that time, I wasn’t sold on the idea of extending the PHP codebase that is used for the current Stats Page; as, I was looking at doing a completely re-work of the Stats Page at some point and wanted the code that I would write for an API to also be re-usable for the next version of the Stats Page. I was waffling between writing the new codebase in Python, something that I was already quite familiar with, and TypeScript. While the latter is something that would be interesting to learn, I didn’t have the spare time to pick up a new language and the surrounding frameworks and toolsets; so, Python it was.
Over the past couple of months while continuing to work on code that would power the next projects for the Wait Wait… Don’t Tell Me! Stats Page, I wanted to figure out a way to see how well a panelist has fared against another panelist (say: Adam Felber vs Faith Salie and Luke Burbank vs Roxanne Roberts). After a number of sleepless nights, a lot more than I am willing to admit, I worked out how to pull the data from the Stats Page database and pieced together some of the logic behind it.
Since the overall development on the code for the current version of the Stats Page has ceased and any work has moved over to the new data access libraries and web application frameworks written in Python, the new report would also have to be developed in Python. Thus, I started working on a couple of proofs of concepts that would just output raw JSON so that I could validate the data before moving on to a proper web-based presentation. Once the resulting data was verified, I started writing the HTML template that would be used and get the design ironed out.
Instead of using a full design framework like Materialize, I decided to give a CSS-only framework called Pure a shot. This would give me a chance to learn how it works and how easy it would be to make the report be a mobile-first, responsive design. While it does provide a lot more features and functionality than I would need for this report template; I’m not completely sold on it being the sole framework for the next version of the Stats Page.
Anyway, after a few hours of playing around with a design that was readable on mobile and on larger screens, and doing a good bit of code cleanup and optimization, I present to you the new report.
The new Panelist vs Panelist report
You can view the report by going to wwdt.me/pvp or clicking on the “P vs P Report” link in the side navigation on the Stats Page.
For the time being, the Python script that generates the report runs every 30 minutes. Since panelist scores and ranks aren’t added more often than once a week, I think running it every 30 minutes is quite sufficient. There are a few things that I will probably need to add, such as a link back to the Stats Page and possibly links to each of the panelist pages.
Update: Side navigation menu with links to each panelist have been added, along with a link back to the Stats Page.
After spending a few hours listening to the Wait Wait… Don’t Tell Me! audio archive that I have found ways to download from the NPR site over the years, I have gone through and updated the show location for all shows in the Stats Page database.
*: Of course, there are a few shows where Carl Kasell didn’t say which location a Best Of show was recorded or the venue in which a show was taped from (be it at a public radio station or a location with a live audience).
Ever since I started working on version 3.0 of the Wait Wait… Don’t Tell Me! Stats Page, one of the design decisions that I made when it came to the individual host, scorekeeper, panelist and guest pages was to use the specially encoded names in the URL. The encoding process would convert certain characters, such as a space, into their URL-encoded version, or “%20” in the case of a space.
Well, looking back, that was not really the smartest way of doing it; and, ever since version 3.0 was published back in 2014, I was afraid of breaking any links that may have been created with that URL format and did not have a lot of spare time to re-design the whole thing. As months and years went by, that problem lingered in the back of my head.
Fast forward to this month, where I found a few extra hours between work and everything else that has been going on in my life to actually sit down and committed myself to fixing it. Since I had been planning on developing a set of publicly-accessible APIs that can be used to query the database that the Stats Page pulls its data from, I knew that I had to work on making host, scorekeeper, panelist and guest names more URL friendly.
With that in mind, I decided to extend the corresponding tables to add a new column for each one: a slug for each entry. To “slugify” a name or some text is to convert that string by replacing special characters like spaces, quotes and parentheses with a simple hyphen, converting everything to lowercase, and even shortening the strings. You’ll often see this with blogs, Facebook or other sites that need to present a search engine friendly URL that is also fairly human-readable. For example, “Peter Sagal” would turn into “peter-sagal” and “P.J. O’Rourke” would turn into “p-j-o-rourke”. Both slugified strings are still pretty readable and don’t include any funky encoded chunks in them.
The next step was to update the URL routing and views on the site to accept both the original URLs and the new slugified URLs. That required a good amount of re-jiggering of the code. After a few hours of repeatedly coding, testing, and cursing, I was able to get everything working properly. So, if you had bookmarked any of the individual host, scorekeeper, panelist or guest page, you should still be able to get to them. What you will now notice is that all of the links to those pages will now point to the new, slugified URLs. For instance, you can go to Peter Sagal’s host page by going to either http://wwdt.me/hosts/Peter%20Sagal or http://wwdt.me/hosts/peter-sagal.
This will probably be the last major change to the version 3 branch of the Stats Page. Any new work will (eventually) be put towards the new version 4 branch of the site; which, will focus primarily on a mobile-first design and incorporating features that have been in the backlog over the past year or so.
Although Wait Wait… Don’t Tell Me! officially reached their first 20th anniversary milestone back in January of this year, the show has been celebrating their anniversary throughout the year and have had a number of shows on the road as part of their 20th anniversary tour. On October 25th, they had their 20th anniversary extravaganza at The Chicago Theatre in downtown Chicago, their fair city.
Months prior to the special show, I had already worked out plans to travel to Chicago for the big show. The anticipation and excitement continued to build as the date got closer and started peaking when I landed in Chicago a few days before the show.
Thursday evening, I made the short walk from the hotel to the Chicago Theatre to pick up my tickets at the box office and anxiously waited for the doors to open. During that time, I checked to see where I was going to be seating and saw that it was on the main floor and about 10 rows back from the front. While waiting for the show to start, I was trying to see if I could recognize anyone that I’ve met at previous shows and/or anyone that follows Luke Burbank’s podcast, Too Beautiful to Live. Alas, I didn’t recognize anyone.
At 7:30 PM Central Time, the show was about to start and there were two people, one from WBEZ and one from the show’s sponsor, Wix.com, that thanked everyone that came to the show. Afterwards, as with any other Wait Wait live taping, the lights went down and a multi-color spinning light show started up with Paul Friedman announcing the start of the show, along with Peter Sagal and Bill Kurtis. As part of the pre-taping part of the show, Peter reminisced a little bit about the show being on for 20 years and would then describe how the special show would work.
Peter talking to the audience before they brought out the first set of panelists
Unlike a regular taping of the show, in which only three panelists would be on the panel, the special 20th anniversary show would have a rotating set of panelists that would change up throughout the show. Peter Grosz, Paula Poundstone and Roy Blount, Jr. would be the panel for the first segment, Who’s Bill This Time? For the first panel round, the panel consisted of Tom Bodett, Roxanne Roberts and Maz Jobrani. For both the Bluff the Listener and the Not My Job segments, Amy Dickinson, Mo Rocca and Faith Salie were on the panel. Afterwards, Adam Burke, Negin Farsad and Adam Felber would be the panel for the second panel round. The panel changed again for the Listener Limerick Challenge segment, consisting of Tara Clancy, Luke Burbank and Brian Babylon.
Below is a breakdown that I created, listing each segment and panelist, and whether they answered a panel round question correctly. Also, I have noted which panelist had their Bluff the Listener story chosen and which one had the correct story.
Panel rotation for each of the segments
After each panel rotation, they would ask each panelist a question relating to something related to something they did or thought 20 years ago as a means to make sure the recording levels were good and that there weren’t any issues with the headsets. Even with that bit, there were still a number of technical difficulties with the panelist headsets working properly. Sometimes, the microphone would cut out and they would have to spend a moment or two to fix them. This can be attributed to having to switch panelists a lot more often than usual.
When it came time for the Not My Job segment of the show, NPR legends, Nina Totenberg and Robert Siegel, who have been on the show a couple of times, especially during the first few years where most of the guests were other NPR journalists and personalities.
For this special show, they really changed things up for the Lightning Fill-in-the-Blank segment. They had brought back all of the panelists back on stage and there would be three teams of five panelists that would each be answering one Lightning Fill-in-the-Blank question. For each team, there was a team captain that was chosen based on being the panelist with the most losses on the show. Peter mentioned that they determined that based on data from my Wait Wait… Don’t Tell Me! Stats Page. By the way, that will be brought up again later.
Due to the unique nature of the rotating panel and the special panelist teams for the Lightning Fill-in-the-Blank, I will not be keeping track of the various points that each panelist would have accrued during the show. Instead, I refer you to the following breakdown of which panelist answered their Lightning Fill-in-the-Blank question correctly and the totals for each team.
Scoring breakdown for the special Lightning Fill-in-the-Blank segment
Once the segment was completed and they wrapped up the show, without the usual Panelist Predictions segment due to the number of panelists and Peter doing a few pick-ups, Peter introduced everyone involved with the show’s production and logistics team on stage. Before Peter finished, he made a callback to the mention of the Wait Wait… Don’t Tell Me! Stats Page when they were setting up for the Lightning Fill-in-the-Blank segment. Peter mentioned my work on creating and maintaining the site for many years and that it has been a resource that they have used over the years. At that point, he was looking for me in the audience, I stood up and they brought out a framed poster featuring special artwork for the 20th anniversary and it had been signed by everyone on the show. I was surprised and amazed that they said that it was for me as a thank you for all of the work that I’ve put it and continue to put into the site.
Update: I received the signed and framed poster earlier this year and below is a photo of it:
After that, the show wrapped up and people came by to say hello and to thank me for the work on the site. I was then taken backstage to meet and mingle with everyone on the show and those behind the curtains, so to speak. Thankfully, I also learned that they would be shipping the signed and framed poster to me, rather than having to carry it around with me and fly it back home. Unfortunately, I didn’t get a chance to look at it up close by the time I got backstage.
Between the banter between segments, some of panelists really hamming it up (including the number of times that Paula interrupted Peter or when Mo Rocca did a cartwheel while holding a drink), and some of the surely-will-be-edited-out blue comments and quips, the experience of being at the live taping of the 20th anniversary extravaganza was one of the best experiences that I’ve had of any live taping of any show that I’ve been to.
20 years is a huge milestone for Wait Wait… Don’t Tell Me! and the show lived up to and exceeded my expectations. It was great to see many of the current panelists on stage and to hear memories from the earlier years of the show. I am entirely humbled by the experience and for the appreciated that was heaped upon me for my work on the Stats Page.
I want to thank the Wait Wait… Don’t Tell Me! crew, WBEZ and NPR for all of the hard work in preparing for and producing the extravaganza. I also want to add a special thanks to Colin Miller for making sure that I could come out to Chicago and be part of the show.
