Building AI That Listens

The Problem: When Communication Breaks Down, Learning Stops

"Without dialogue, there is no communication, and without communication, there can be no true education."

Paulo Freire's words hit differently when you're watching a 5th-grade math class struggle through Zoom.

During the pandemic, I watched teachers try to gauge understanding through glitchy screens, while students—especially those from linguistically diverse backgrounds—were getting lost in translation. The verbal cues that make classroom communication work were disappearing into digital static.

Between 2023 and 2024, our research team at Columbia University tackled this challenge head-on. That work led to a question that changed my approach to AI: What if AI could help teachers and students communicate more effectively, rather than replacing human connection?

The Vision: AI as Communication Bridge

Traditional educational AI often focuses on replacing human judgment—automated grading, algorithmic assessment, one-size-fits-all feedback.

We flipped the script.

Our research focused on a deceptively simple question: Can we build AI that helps teachers understand when students actually "get it," and helps students decode what their teachers are really saying?

The answer led us to develop a speech analytics system that analyzes classroom dialogue in real time, identifying the verbal signals that indicate true comprehension and the communication patterns that inspire learning.

The Technical Challenge: Making AI Understand Context, Not Just Words

Building AI for classroom communication meant solving problems that don't show up in typical NLP applications.

1. Context-Aware Classification

We developed binary classification models using transformer-based architectures that could distinguish between "effective" and "ineffective" verbal signaling cues across five categories:

• Inclusion: Making every student feel seen and valued
• Integrity: Building trust through consistent communication
• Courtesy: Showing respect for diverse perspectives
• Translation: Helping students decode complex concepts
• Charisma: Creating engagement and motivation

Instead of just transcribing speech, we focused on turn-level context analysis: what was said, who said it, and how it functioned within the conversational flow. Our models processed 30-second dialogue windows to capture the full context of teacher-student exchanges.

2. Dialect and Cultural Sensitivity

Traditional speech recognition often fails students who speak African American Vernacular English (AAVE) or come from multilingual households. Left unchecked, that failure becomes structural bias.

We integrated specialized benchmarks for AAVE assessment, dialect-aware evaluation metrics, and culturally responsive training data so the system wouldn't pathologize the way students naturally speak. The goal was simple: ensure the AI worked for these students, not against them.

3. Human-in-the-Loop Design

Rather than replacing teacher judgment, our system amplified it.

Teachers and students became annotators, helping the AI learn what good communication actually looks like in their specific context. This human-in-the-loop design was essential—not just for performance, but for trust and model reliability.

What We Learned: The Power of Inclusive Communication

Our research revealed something profound: The way teachers communicate doesn't just affect what students learn—it affects whether they believe they can learn.

Through our pilot implementations across multiple 5th-grade classrooms, we discovered measurable patterns:

When teachers increased "translation" language—phrases like "What I hear you saying is..." or "Let me put that in different words"—students were 40% more likely to attempt follow-up questions and showed significantly higher engagement in problem-solving discussions.

The Bigger Picture: Responsible AI in High-Stakes Environments

This project taught me something crucial about AI development: The higher the stakes, the more intentional we have to be about ethics.

In classrooms, you're not just processing data—you're influencing a child's relationship with learning. Every algorithmic choice can either reinforce bias or break down barriers.

Bias Mitigation at Every Level

• Evaluation metrics that explicitly tracked performance across demographic groups and linguistic backgrounds
• Continuous testing for dialect-based bias using fairness-aware evaluation frameworks
• Safeguards against censorship of culturally relevant language or expression

Transparent Data Practices

• Clear consent processes in both English and Spanish
• Our conversational assistant, "Divy Assist," explained data collection to families in plain language
• Community engagement sessions with parents, teachers, and students in underserved districts

Accessibility-First Design

• Adherence to W3C Accessibility Guidelines (WCAG) 3.0
• Multiple modalities for student expression and feedback
• Tools that promoted self-sufficiency rather than creating new technological dependencies

Why This Matters for AI's Future

This work convinced me that the most powerful AI applications won't be the ones that replace human capability—they'll be the ones that amplify human connection.

The real breakthrough wasn't just the technology. It was demonstrating that AI can be designed to strengthen one of the most fundamentally human activities: learning through dialogue.

As we build increasingly sophisticated AI systems, the lessons from this classroom work are only becoming more relevant:

• Context matters more than raw accuracy. Perfect transcription is useless if the system can't understand cultural nuance or conversational intent.
• Human oversight isn't a limitation—it's a feature. Human-in-the-loop design makes AI more reliable, more equitable, and more effective.
• Ethical design isn't optional. In high-stakes domains, it's the foundation that determines whether the technology helps or harms.

What's Next

The research continues at Columbia University, where the team is expanding the framework to support more grade levels and subject areas.

For me, this project sparked a broader mission: helping organizations implement AI in ways that amplify human capability rather than replacing it.

The same principles that guided this classroom work now guide how I help startups and enterprises implement AI in complex, human-centered environments:

• Start with human needs and outcomes, not just technological possibilities
• Design for inclusion and equity from day one
• Build systems that make people more effective and confident, not obsolete
• Measure impact through human flourishing, not just efficiency metrics