Dr. Nehemiah Mabry (“Dr. Nee”), a professional engineer and former NASA researcher, hosted the weekday Twitch live streams from June to October 2024. He engaged gamers via virtual field trips, interviews, collaborative building challenges, and gameplay. Together, they solved engineering puzzles, culminating in a four-night escape room event, streamed and solved live.

To study whether his efforts brought engineering to life for viewers, we conducted interviews, surveyed participants, and analyzed Twitch chat messages using a novel machine learning approach. We then compared live streaming engagement to that of the corollary “Building Stuff” documentary series. We found that collaborative learning tripled the odds that viewers would engage in chat discussions about engineering and science.

Analyzing fast-moving social media data requires both technical rigor and methodological creativity. Our researchers conducted a kickoff workshop, literature review, 32 in-depth qualitative interviews, a survey of 76 participants, and analysis of 10,000-plus Twitch chat messages from 450 unique users. We used these analysis methods to identify:

• Whether community-centered streaming—an ongoing series of interactive live streams that incorporate intentional host engagement, intentional moderation of the stream and its chat, and an active chat environment—could reach broader or different audiences than traditional media
• Best practices for building an online learning community
• Factors influencing deeper engagement and learning, such as stream format, collaborative streaming, and other qualities

Notably, to process the immense volume of Twitch chat messages, the Social Data Collaboratory team developed a machine learning classifier—an algorithm that uses text data to classify content—to categorize each message across 10 to 12 themes of interest, including STEM engagement, collaborative learning, peer support, and emotional tone. To ensure accuracy, researchers manually labeled and validated a representative sample of data.

“By alleviating the need for manual labeling, our novel machine learning classifier accelerated Twitch chat analysis,” said Simon Page, a data scientist with the Social Data Collaboratory. “This opened the door to more scalable, time- and cost-efficient social media research.”

One challenge was defining STEM engagement for a Twitch chat environment, where messages are typically short and informal. However, the result was a validated, systematic classification of thousands of chat messages—a methodology that is both novel and replicable.

NOVA project staff actively facilitated conversations. Their contributions were treated as independent variables to explore how different engagement approaches influenced the development of the learning communities we observed. Although they only represented 4.5 percent of all chat participants, NOVA staff generated nearly one-quarter of all messages. Their contributions frequently posed catalyzing questions that shaped and guided subsequent discussion.

“Our team iterated across different stream data sources, combining qualitative and quantitative methods to better understand people’s experiences and the unique impact of interactive community learning,” said Emily Bray, a research scientist in Arts & Culture at NORC.

Streaming platforms offer many opportunities to reach broader audiences. Community-centered streaming naturally aligns with participatory learning and collective problem-solving as long as science communicators consider that people use and have different expectations of individual platforms.

Two essential strategies for capturing and keeping viewers are designing “on-ramps” for STEM newcomers that show how the content is relevant to them, and “stay-ramps” that give first-time visitors compelling reasons to return. Partnering with streamers who already have established audiences and scheduling streams with intention are important because, according to our findings, it can draw larger audiences and deepen the quality of chat engagement. Museums, libraries, and cultural organizations can use these same principles to meet audiences where they are to build meaningful, lasting live streaming engagement.