Share on twitter
tweet
Share on linkedin
share
Share on email
Send
Share on facebook
share

An Exploratory Analysis of Super Bowl Ad Performance Through Twitter

Comedic ads reign dominant in procuring “positive” buzz during the Super Bowl, but balance needs to be trodden.
Adam Whalen

Co-Author

Colten Hoth

Co-Author

Matt Pecsok

Co-Author

Table of Contents

1 Introduction

While the popularity of marquee cultural and sporting events across America have ebbed and flowed as trends continually evolve, the Super Bowl has stood the test of time in persistence and prominence. According to AdWeek, the 2020 Super Bowl peaked at 102.1 million multiplatform viewers making it once again the largest televised sporting event over the last 12-month period, surpassing both the NBA finals as well as the World Series. While the viewing habits may have changed as folks shift from linear viewing methods toward more connected television devices (i.e., streaming services such as Hulu live sports), the appeal of this event from a marketer’s perspective still stands due to the huge, diverse crowds it continually draws in. 

As such, Domo, Firetoss, Molio, and the David Eccles School of Business at the University of Utah have come together to analyze the 2020 Twitter data from Super Bowl LIV to better understand and extract insights around top performing advertisements. With this overarching goal in mind, and knowing that we our audience is most specifically marketers, we isolated two key research objectives that guided our decision-making for which analytical methods to employ. 

  • Research Question 1: To what extent can Twitter & secondary data sources inform whether or not purchasing an ad in the Super Bowl is effective?
  • Research Question 2: What aspects or features of a Super Bowl ad make it effective?

Due to the nature of these questions, we determined that we would need various methodologies to answer them. The following research paper first establishes those methodologies, followed by a comprehensive section on our results, and concluding with said business-pertinent takeaways that satisfy the above stated research objectives.

2 Methodology

In total, 494,127 tweets were collected from the Super Bowl that are included in this analysis. In addition, we utilized data from the 2019 Twitter data set to procure further insights. Lastly, due to the omnichannel landscape that we currently live in, we also pulled in front-facing viewership data from YouTube, popularity scores from Google Trends, and reported ad performance on Facebook & Instagram from various advertising publications to further enrich and strengthen all analysis being completed. Similarly, we operated off an average reported cost of $5.6 million dollars per thirty seconds to calculate any and all return on investment numbers encompassed within this report. 

With this data in tow, we evaluated a number of analytical methods to answer our research questions and further trim the data sets. Ultimately, we settled on one rough method of refinement (for trimming purposes) and multiple methods of measurement that include sentiment analysis, year-over-year performance comparisons, return on investment by frequency of tweets, covariance analysis looking at commercial and brand keywords through Google Trends data, and topic modeling methods for clustering purposes. The range and diversity of these analytical techniques resulted in a slew of KPIs that transformed into business-pertinent takeaways in the summary section of this report to determine which brands had the most effective overall performance. 

The method of refinement incorporated weighted averages across our various data points (Twitter, YouTube, Google Trends, and Facebook/Instagram rankings) to narrow down which ads were best and worst performing. The goal with this method was never to declaratively state which ads were the best; rather, focus the discussion on a group of roughly twenty ads/brands from which more detailed and thorough analysis could be completed, as well as drawing qualitative insights from the style and tones of the ads themselves.

3 Results

3.1 Top Performing Ads

Refining the Data Set Through Weighted Analysis of Brand Frequency on Twitter and a Custom Omnichannel Performance Ranking

With more than 50 brands placing advertisements in the Super Bowl there was an immediate need to “refine” the field and focus solely on the best and worst performers to procure business actionable insights. Knowing that Twitter is just one of the many platform’s folks use to receive news and voice their opinions, we opted to create a weighted algorithm ranking the performance of these ads across the various channels of interest. We ultimately curated an omnichannel ranking system that was composed of 1) Twitter volume/mentions at a 40% weight, 2) total YouTube views at a 30% weight, 3) the Google Trends popularity score at a 20% weight, and 4) AdWeek’s Facebook/Instagram poll data of best performing ads at a 10% weight. The final result was a rough final score on a scale of 1 to 100 designated the overall performance of said ad across all the channels, meant to simply slim the field (not make declarative statements about performance) for further analysis.

Top Superbowl Commercials
The dual axis measurement above shows the volume of tweets (blue bars) against our custom omnichannel ranking (yellow line) to roughly determine top performing ads. The top 11 are shown above, while we considered the top 20 to 25 for further analysis.

The results are certainly imperfect (such is the nature of a subjective weighted system), but did narrow our focus down on the top ads without having to solely focus on just Twitter data. We still see that all the ads – apart from Trump’s Criminal Justice Reform and The Cool Ranch Doritos spot – were generally high performing on Twitter, showing that our strong weighting of that platform bled through. However, this did give more preference to ads that had strong Google Search trends (visualized later in this results section) and YouTube viewership data such as Smaht Pahk, Can’t Touch this, and Groundhog Day which otherwise show up as middling in just the Twitter data. This also limited the effect of something like Mama Test 5G from overtaking the other ads simply because a Twitter competition was run.

3.2 Brand Return on Investment Calculated

by Tweet Frequency

Marketing campaigns are ultimately asked if the ad was worth the money spent. To determine the ROI we created a ratio to look at cost per tweet. We looked up the amount of ad time the brand purchased and used $5.6 million per 30 second ad to determine the cost. The top three brands that got the most tweets generated per dollar spent are Pringles, MGM, and T-Mobile. We found that there is not a correlation between amount of money spent and the number of tweets. It is not enough to throw money at the ad, but the brand needs to exhibit some creativity to connect with viewers.

Total Cost per Tweet by Brand

3.3 Year Over Year Tweet Growth & Loss

The next piece of analysis looked at brands from 2019 and 2020 that were repeated advertisers. We computed tweet volume for both of the years and then analyzed the difference in tweets from year over year. As expected, brands that had a huge impact (such as Marvel) which had much less anticipated ads suffered the worst.

Brands that lost tweets since last year
2019 brands that lost tweets year over year, led by Marvel who had and Avengers Endgame commercial last year which ended up being the highest grossing movie of all-time.

Clearly people are not talking as much about Black Widow as they were about Endgame in 2019. Also, other brands such as Pepsi and Verizon also had a drop in tweet volume, so if we are looking at year over year comparison as a gauge of ad success for these brands the ads were substantially less successful than the previous year.

Similarly, we looked at advertisers who were able to increase the number of tweets from 2019 to 2020. Here we see T-Mobile again perform strongly with their free phone giveaway that doubled their tweet volume from ~20,000 to ~45,000. Olay also had a substantial improvement over their previous year’s ad with a new message that generated far more buzz than 2019 focused on women empowerment, that also premiered earlier in the night whereas in 2019 the commercial debuted later in the game.

Brands that increased tweets since last year
Among brands that increased in tweets since last year include T-Mobile with their free phone giveaway and Olay with a new women empowerment message.

3.4 Tweet Volume Over Time

The below plot(s) track tweet volume grouped into 2 minute intervals during the 2019 and the 2020 Super Bowls organized by brand. The graphs clearly illustrate how much the Bud Light and Marvel commercials surpassed (by 3 times) all other brands in 2019. In fact when we look at total tweet volume in 2019 (which was ~ 700,000) and in 2020 (which was ~ 450,000) we can note that difference is almost entirely due to those two massive brands getting 150,000 tweets as compared to only ~50,000 for the next closest brands in 2020. One thing to note about the Marvel ad however is that while the commercial could be called a success by the number of tweets, it was probably mostly due to anticipation of the upcoming Avengers Endgame movie, not because the commercial itself was particularly effective.

brand tweets over time in 2019
Above we see the brand tweet counts over time in 2019, with notable spikes among Marvel and Bud Light who had the top performing ads.
brand tweets over time in 2020
For 2020, the overall volume is not only lower, but there are no elongated spikes like we saw in 2019, with Bud Light coming the closest to replicating such behavior.

3.5 Brand Resonance by Gender

Another angle we pursued was to look at genders of people tweeting to see what the gender proportions were in the data and which of the top 10 ads had large differences when compared to the average overall. We used code to extract first names, hit a web api to classify the names by gender then graphed the ads by proportion of gender. We found that 67% of the tweets were male while 33% were female. 

When we looked at the proportion in specific ads we found the following results by Brand. This is interesting because it clearly shows that ads appeal to different genders at different rates. Specifically the Olay “Make space for women” ad was hugely successful in getting women to talk about their ad, though it’s probable their audience was actually intended to be males. On the other hand Hulu’s ad featuring Tom Brady did a poor job of attracting female responses.

Brands by Gender proportions
With the above we see that the Hulu ad involving Tom Brady resonated with males at a vastly higher clip than Females, while the Olay ad involving female empowerment has inverse relationship indicating these folks may (or may not) have hit their target demographic.

3.6 Sentiment Analysis

In addition to the year over year analysis that was completed, we also analyzed the sentiment of the tweets to better understand what kind of responses were occurring. We noted that those with the most positive responses were T-Mobile, Jeep, and Bud Light. The T-Mobile giveaway encouraged the use of positive terms – explaining its prominence – while Jeep had Bill Murray who is a prominent comedian and actor. The Hulu and MGM ad’s produced the most apathy, meaning to say folks simply don’t resonate with those ads indicating the dollars may have been waster.

Tweets by sentiment 2020
The most positively received ads included T-Mobile’s free phone giveaway and Bill Murray’s appearance in the Jeep commercial, while Tom Brady’s Hulu ad and MGM’s ad fell flat with predominantly neutral responses.

3.7 Topic Modeling

Building off the prior analysis, we also partook in some Topic Modeling and clustering techniques to isolate which terms were connoted most often with the brand. Ideally, we would see that a brand’s name and the commercial tag line/messaging work in tandem, and as you can see below. The Jeep ad did a good job of this as the conversation is focused around Bill Murray, whereas the Tide commercials (due to their multi-faceted nature of working with brands and communicating multiple message) have a noisier picture that perhaps loses itself by trying to do to many different things.

The first step in the text analysis of the tweets was to simply remove all retweets. We then removed all links, special characters, as well as stop words, followed by reducing a word to its stem so words like “dad” or “daddy” are counted as one. After that, hashtags and mentions were extracted for separate analysis, with the remaining tweets then analyzed by how often a word appears. For those words such as Super Bowl or Commercial, we opted to filter them out as they were not relevant for the analysis.

Top 10 Tweeted Word Pairs; Brand: Jeep
Top 10 Tweeted Word Pairs; Brand: Tide
The word pairs for Jeep show that the brand tethered more strongly to its message than Tide did. This is examined further as we take this beyond Twitter and incorporate Google Trends data to analyze brand/message overlap.

3.8 Brand Resonance

Through Google Trends Data

“Second screening” is an activity that has risen substantially over the last three years, simply meaning to say that folks are using the web on a phone or tablet while consuming content on their big screens. This trend is exacerbated when it comes to linear television. As such, we took a deep dive into Google Trends data to support the Twitter analysis that was completed for the last three measurements. In short, we averaged the Google Trends score across the three hours since when the commercial went live for both the commercial terms (i.e., for Google’s ad, “Loretta” is the term being measured) as well as the brand associated with said term (i.e., “Google” is the brand term for the Loretta ad). Google Trends data is scaled from 1 to 100, where 100 means that there are a peak number of searches for that term, 50 means half as popular as 100, and 1 means little to no popularity. On the x-axis below are the commercial terms, and on the y-axis are the brand terms. 

Brand Resonance Through Google Trends Data

In essence, what this analysis shows is which brands were most successful at tethering their messaging and commercials terms to their actual brands. For example, the TMobile ad performs poorly here, in that the “Mama Test 5G” tagline had little to no search volume at all. Similarly, this shows that the Hyundai commercial perhaps did a poor job of connecting the Hyundai Brand to the “Smaht Pahk” tagline, as most folks are searching for said tagline and not registering that it is connected to Hyundai. Microsoft was the top performer, as both the “Be The One” message and Microsoft brand term had high popularity in search terms.

4 Conclusion

In conclusion, we accrued a good amount of exploratory data in attempt to answer our two primary research questions, which again were:

  • Research Question 1: To what extent can Twitter & secondary data sources inform whether or not purchasing an ad in the Super Bowl is effective?
  • Research Question 2: What aspects or features of a Super Bowl ad make it effective?

In answering question 1, we hypothesize multiple answers to this question. The first being that Twitter data alone is likely not satisfactory to fully inform whether or not purchasing an ad is effective, and what data we do have suggests that purchasing said ads is not as effective as it once was. When we analyze the year over year data, we see that overall Twitter usage is down substantially, lending credence to the idea more folks are consuming content in different ways and discussing such content on platforms other than Twitter. This is proven by an ad like The Cool Ranch, which had low Twitter activity but strong Google Trends data and YouTube views.

For research question 2, we see some noise in that the T-Mobile giveaway ad performs well on Twitter but did not perform as effectively on Google Trends, YouTube, and general rankings, indicating that while giveaways are flashy and generate instantaneous buzz they aren’t as effective in producing long-term awareness and persistence for the brand itself. Additionally, we see through the sentiment analysis that comedic ads still reign dominant in procuring “positive” buzz about a brand, but this balance needs to be trodden lightly as something like Hyundai’s “Smaht Pahk” ad lean too heavily into the comedy and the brand retention is lost as seen in the Google Trends covariance data.

As can be seen, there are a vast amount of business practical insights and takeaways that resolve the above research questions in satisfactory and compelling ways. Should there be any further questions about this research, please reach out to any of the four leads on this project by email.

This report was created as part of the Game Day Analytics Challenge hosted by the Informations Systems Department at the David Eccles School of Business at the University of Utah.

The teams where between three and four students. Together with my teammates Adam WhalenMatt Pecsok, and Colten Hoth we received third place in the graduate student category.

Find the infographic here and more information about the challenge here.

Header Image by Timothé Lejeune on Unsplash

Twitter Data was supplied by organizer.

Feel free to share:

Share on twitter
tweet
Share on linkedin
share
Share on email
Send
Share on facebook
share

More to explore

Mask
Analysis

A look at COVID-19 Data and the Importance of Testing

Since the outbreak of COVID-19 there have been over 1.6M confirmed cases and 97k deaths globally.[1] Some countries have fared better than others, which lends the question, what are the best actions a governmental entity can take to mitigate – or wholly eliminate – the spread of a virus like COVID-19 without creating long-lasting issues?

Game Day Analytics Challenge
Infographic

[Infographic] Super Bowl Ad Analysis

Infographic to Super Bowl Ad Analysis. Comedic ads reign dominant in procuring “positive” buzz during the Super Bowl, but balance needs to be trodden.

Jonas Vitt

About Jonas Vitt

Born and raised in Karlsruhe, in the south-west of Germany, Jonas is currently pursuing a Master of Science in Business Analytics at the David Eccles School of Business at the University of Utah in Salt Lake City, UT. He finished his Bachelor’s degree in Business Administration with a focus on marketing at Pforzheim University in July 2019.

This website uses cookies to ensure you get the best experience on our website. Cookies are sometimes also set by third party services. You can find further information in our privacy policy.