TRAIL Logo

TRAIL Lab

Publications

2025

Teacher Agency in Implementing Automated Writing Evaluation Systems in a Science Classroom: A Focus Group Study

Yuxuan Fu, Shamya Karumbaiah, Xiaoying Gao, Ipshita Dey, Debbie Gnesdilow, Sadhana Puntambekar

Proceedings of the 18th European Conference on Technology Enhanced Learning (ECTEL)

Teacher Feedback on Linguistic Variations as a Source for Evaluating Language Models

Kaycie Barron, N. Tseng, Shamya Karumbaiah, Cynthia Baeza

International Conference of the Learning Sciences (ICLS)

Exploring Out-of-Tutor Interventions by Teachers in Supporting Student Learning with AI Tutors

Shamya Karumbaiah, Anand Syamkumar, Vincent Aleven

International Conference of the Learning Sciences (ICLS)

View Paper

A Comparative Analysis of LLM and Specialized NLP System for Automated Assessment of Science Content

X. Gao, Shamya Karumbaiah, A. Dalal, I. Dey, D. Gnesdilow, S. Puntambekar

Proceedings of the 26th International Conference on Artificial Intelligence in Education (AIED)

Authenticity or alienation: Latine students' perceptions of stereotypes in AI-generated educational materials

Jihyun Rho, Shamya Karumbaiah

Proceedings of the 26th International Conference on Artificial Intelligence in Education (AIED)

Teacher Perceptions on AI Support for Bi/multilingual Learners: Superpowers and Constraints

Alina Guha, Shamya Karumbaiah, Cynthia Baeza, K. White, Mariana Castro

Proceedings of the 26th International Conference on Artificial Intelligence in Education (AIED)

Bias or Insufficient Sample Size? Improving Reliable Estimation of Algorithmic Bias for Minority Groups

Jaeyoon Choi, Shamya Karumbaiah, Jeff Matayoshi

ACM Learning Analytics and Knowledge (ACM LAK)

View Paper

Toward Teacher-Centered AI Design: Exploring the Role of Pedagogical Values and Contextual Factors in K-12 Teachers’ Perceptions of Responsible AI

Yaxuan Yin, Shamya Karumbaiah, Ju Lim, Schwartz, S., Acquaye, S.

Computer Supported Collaborative Learning (CSCL)

AIBAT: AI Behavior Analysis Tool for Teacher-Driven Contextual Evaluation of Language Models in Education

Shamya Karumbaiah, Yaxuan Yin, Aayush Bharadwaj

26th International Conference on Artificial Intelligence in Education (AIED)

Responsible AI in Education: Understanding Teachers’ Priorities and Contextual Challenges

Yaxuan Yin, Shamya Karumbaiah, Acquaye, S.

ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT)

Prompt vs. Supervise: A Case of Using Language Models to Assess Students’ Science Explanations

Shamya Karumbaiah, Ananya Ganesh, Yang, S., Tseng, N.

In European Conference on Technology Enhanced Learning (ECTEL)

2024

Leveraging Multimodal Classroom Data for Teacher Reflection: Teachers’ Preferences, Practices, and Privacy Considerations

Kexin Bella Yang, Conrad Borchers, Ann-Christin Falhs, Vanessa Echeverria, Shamya Karumbaiah, Nikol Rummel, Vincent Aleven

In European Conference on Technology Enhanced Learning (ECTEL)

Revealing Networks: Understanding Effective Teacher Practices in AI-Supported Classrooms using Transmodal Ordered Network Analysis

Conrad Borchers, Yeyu Wang, Shamya Karumbaiah, Muhammad Ashiq, David Williamson Shaffer, Vincent Aleven

14th ACM Conference on Learning Analytics and Knowledge (ACM LAK)

View Paper

Evaluating Behaviors of General Purpose Language Models in a Pedagogical Context

Shamya Karumbaiah, Ananya Ganesh, Aayush Bharadwaj, Lucas Anderson

25th International Conference on Artificial Intelligence in Education (AIED)

Analyzing Transitions in Sequential Data with Marginal Models

Jeff Matayoshi, Shamya Karumbaiah

Journal of Educational Data Mining (JEDM)

View Paper

Gaming the System Mediates the Relationship Between Gender and Learning Outcomes in a Digital Learning Game

Ryan S Baker, J Elizabeth Richey, Jiayi Zhang, Shamya Karumbaiah, Juan Miguel Andres-Bray, Huy Anh Nguyen, Juliana Maria Alexandra L Andres, Bruce M McLaren

Instructional Science

View Paper

2023

Multimodal Analytics for Collaborative Teacher Reflection of Human-AI Hybrid Teaching: Design Opportunities and Constraints

Shamya Karumbaiah, Pinyang Liu, Alisa Maksimova, Lea De Vylder, Nikol Rummel, Vincent Aleven

Proceedings of the 18th European Conference on Technology Enhanced Learning (ECTEL)

View Paper

Optimizing Parameters for Accurate Position Data Mining in Diverse Classrooms Layouts

Tianze Shou, Conrad Borchers, Shamya Karumbaiah, Vincent Aleven

Proceedings of the 15th International Conference on Educational Data Mining (EDM)

View Paper

Teacher Noticing and Student Learning in Human-AI Partnered Classrooms: A Multimodal Analysis

Shamya Karumbaiah, Conrad Borchers, Ann-Christin Falhs, Kenneth Holstein, Nikol Rummel, Vincent Aleven

Proceedings of the 18th International Conference of the Learning Sciences (ICLS)

View Paper

A Spatiotemporal Analysis of Teacher Practices in Supporting Student Learning and Engagement in an AI-enabled Classroom

Shamya Karumbaiah, Conrad Borchers, Tianze Shou, Ann-Christin Falhs, Pinyang Liu, Tomohiro Nagashima, Nikol Rummel, Vincent Aleven

Proceedings of the 24th International Conference on Artificial Intelligence in Education (AIED)

View Paper

Using Demographic Data as Predictor Variables: a Questionable Choice

Ryan S Baker, Lief Esbenshade, Jonathan Vitale, Shamya Karumbaiah

Journal of Educational Data Mining (JEDM)

View Paper

2022

Upstream Biases in Adaptive Learning Systems

Shamya Karumbaiah

Graduate Summer School on Algorithmic Fairness

Quick red fox: An App Supporting a New Paradigm in Qualitative Research on AIED for STEM

Stephen Hutt, Ryan S Baker, Jaclyn Ocumpaugh, Anabil Munshi, JMAL Andres, Shamya Karumbaiah, Stefan Slater, Gautam Biswas, Luc Paquette, Nigel Bosch, Martin van Velsen

In Artificial Intelligence in STEM Education: The Paradigmatic Shifts in Research, Education and Technology

View Paper

Using Neural Network-Based Knowledge Tracing for a Learning System with Unreliable Skill Tags

Shamya Karumbaiah, Jiayi Zhang, Ryan S Baker, Richard Scruggs, Whitney Cade, Margaret Clements, Shuqiong Lin

Proceedings of the 14th International Conference on Educational Data Mining (EDM)

View Paper

How does Students’ Affect in Virtual Learning Relate to Their Outcomes? A Systematic Review Challenging the Positive-Negative Dichotomy

Shamya Karumbaiah, Ryan Baker, Yan Tao, Ziyang Liu

Proceedings of the 12th ACM International Learning Analytics and Knowledge Conference (ACM LAK)

View Paper

Analyzing Transitions in Sequential Data with Marginal Models

Jeff Matayoshi, Shamya Karumbaiah

Journal of Educational Data Mining (JEDM)

View Paper

Improving Phishing Reporting Using Security Gamification

Matthew L Jensen, Ryan T Wright, Alexandra Durcikova, Shamya Karumbaiah

Journal of Management Information Systems (JMIS)

View Paper

2021

Optimizing philosophies for predictive models in learning analytics

Shane Hutt, Shamya Karumbaiah, Joshua Ocumpaugh

Proceedings of the Workshop on Towards a Philosophical Framework for Learning Analytics at the 11th International Learning Analytics and Knowledge Conference (LAK)

Decoloniality as a Lens of Ethical Foresight for Learning Analytics

Shamya Karumbaiah, Jillian Brooks

Proceedings of the Workshop on Towards a Philosophical Framework for Learning Analytics at the 11th International Learning Analytics and Knowledge Conference (LAK)

Using Marginal Models to Adjust for Statistical Bias in the Analysis of State Transitions

Jeff Matayoshi, Shamya Karumbaiah

Proceedings of the 11th International Learning Analytics and Knowledge Conference (LAK)

View Paper

Using Past Data to Warm Start Active Machine Learning: Does Context Matter?

Shamya Karumbaiah, Andrew Lan, Sachit Nagpal, Ryan S Baker, Anthony Botelho, Neil Heffernan

Proceedings of the 11th ACM International Learning Analytics and Knowledge Conference (ACM LAK)

View Paper

How Colonial Continuities Underlie Algorithmic Injustices in Education

Shamya Karumbaiah, Jamiella Brooks

Proceedings of the IEEE Research in Equity and Sustained Participation in Engineering, Computing, and Technology (IEEE RESPECT)

View Paper

Investigating the Validity of Methods Used to Adjust for Multiple Comparisons in Educational Data Mining

Jeff Matayoshi, Shamya Karumbaiah

Proceedings of the 13th International Conference on Educational Data Mining (EDM)

View Paper

Four paradigms in learning analytics: Why paradigm convergence matters

Ryan S Baker, Dragan Gašević, Shamya Karumbaiah

Computers and Education: Artificial Intelligence

View Paper

Context Matters: Differing Implications of Motivation and Help-Seeking in Educational Technology

Shamya Karumbaiah, Jaclyn Ocumpaugh, Ryan S Baker

International Journal of Artificial Intelligence in Education (IJAIED)

View Paper

A Re-Analysis and Synthesis of Data on Affect Dynamics in Learning

Shamya Karumbaiah, Ryan S Baker, Jaclyn Ocumpaugh, Alexandra Andres

IEEE Transactions on Affective Computing (IEEE TAC)

View Paper

2020

Accurate and Interpretable Sensor-free Affect Detectors via Monotonic Neural Networks

Andrew S Lan, Anthony Botelho, Shamya Karumbaiah, Ryan S Baker, Neil Heffernan

Proceedings of the 10th International Conference on Learning Analytics and Knowledge (LAK)

View Paper

Relationships Between Math Performance and Human Judgments of Motivational Constructs in an Online Math Tutoring System

Rurik Tywoniw, Scott A Crossley, Jaclyn Ocumpaugh, Shamya Karumbaiah, Ryan Baker

Proceedings of the 21st International Conference on Artificial Intelligence in Education (AIED)

View Paper

Studying Affect Dynamics using Epistemic Networks

Shamya Karumbaiah, Ryan S Baker

Proceedings of the 2nd International Conference on Quantitative Ethnography (ICQE)

View Paper

Affective Sequences and Student Actions Within Reasoning Mind

Jaclyn Ocumpaugh, Ryan S Baker, Shamya Karumbaiah, Scott A Crossley, Matthew Labrum

Proceedings of the 21st International Conference on Artificial Intelligence in Education (AIED)

View Paper

Predicting Math Identity through Language and Click-stream Patterns in a Blended Learning Mathematics Program for Elementary Students

Scott A Crossley, Shamya Karumbaiah, Jaclyn Ocumpaugh, Matthew J Labrum, Ryan S Baker

Journal of Learning Analytics (JLA)

View Paper

Adjusting the L Statistic when Self-Transitions are Excluded in Affective Dynamics

Jeff Matayoshi, Shamya Karumbaiah

Journal of Educational Data Mining (JEDM)

View Paper

2019

Using Participatory Design to Facilitate In-service Teacher Learning in Computational Thinking

Shamya Karumbaiah, Sugat Dabholkar, Jooeun Shim, Susan Yoon, Betty Chandy, Andy Ye

Proceedings of the 13th International Conference on Computer Supported Collaborative Learning (CSCL)

View Paper

Temporally Rich Features Capture Variable Performance Associated with Elementary Students’ Lower Math Self-concept

Shamya Karumbaiah, Joshua Ocumpaugh, Michelle Labrum, Ryan S Baker

Proceedings of the Workshop on Online Learning and Social-Emotional Learning at the 9th International Conference on Learning Analytics and Knowledge (LAK)

View Paper

Affect Sequences and Learning in Betty’s Brain

Juliana Ma Alexandra, L Andres, Jaclyn Ocumpaugh, Ryan S Baker, Stefan Slater, Luc Paquette, Yang Jiang, Shamya Karumbaiah, Nigel Bosch, Anabil Munshi, Allison Moore, Gautam Biswas

Proceedings of the 9th ACM International Conference on Learning Analytics and Knowledge (ACM LAK)

View Paper

Using Reinforcement Learning to Optimize the Policies of an Intelligent Tutoring System for Interpersonal Skills Training

Kallirroi Georgila, Mark G Core, Benjamin D Nye, Shamya Karumbaiah, Daniel Auerbach, Maya Ram

Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)

View Paper

Predicting Math Success in an Online Tutoring System Using Language Data and Click-stream Variables: A longitudinal analysis

Scott A Crossley, Shamya Karumbaiah, Joshua Ocumpaugh, Michelle Labrum, Ryan S Baker

Proceedings of the 2nd Conference on Language, Data and Knowledge (LDK)

View Paper

The Influence of School Demographics on the Relationship Between Students’ Help-Seeking Behavior and Performance and Motivational Measures

Shamya Karumbaiah, Joshua Ocumpaugh, Ryan S Baker

Proceedings of the 11th International Conference on Educational Data Mining (EDM)

View Paper

The Case of Self-Transitions in Affective Dynamics

Shamya Karumbaiah, Ryan S Baker, Jaclyn Ocumpaugh

Proceedings of the 20th International Conference on Artificial Intelligence in Education (AIED)

View Paper

Using Epistemic Networks with Automated Codes to Understand Why Players Quit Levels in a Learning Game

Shamya Karumbaiah, Ryan S Baker, Amanda Barany, Valerie Shute

Proceedings of the 1st International Conference on Quantitative Ethnography (ICQE)

View Paper

2018

Is Student Frustration in Learning Games More Associated with Game Mechanics or Conceptual Understanding?

Shamya Karumbaiah, Seyedahmad Rahimi, Ryan Baker, Valerie Shute, Sidney Dmello

Proceedings of the 13th International Conference of the Learning Sciences (ICLS)

View Paper

The Implications of a Subtle Difference in the Calculation of Affect Dynamics

Shamya Karumbaiah, Juliana Ma Alexandra L Andres, Anthony F Botelho, Ryan S Baker, Jaclyn Ocumpaugh

Proceedings of the 26th International Conference on Computers in Education (ICCE)

View Paper

Engaging with the Scenario: Affect and Facial Patterns from a Scenario-Based Intelligent Tutoring System

Benjamin D. Nye, Shamya Karumbaiah, S. Tugba Tokel, Mark G. Core, Giota Stratou, Daniel Auerbach, Kallirroi Georgila

Proceedings of the 19th International Conference on Artificial Intelligence in Education (AIED)

View Paper

Predicting Quitting in Students Playing a Learning Game

Shamya Karumbaiah, Ryan S Baker, Valerie Shute

Proceedings of the 11th International Conference on Educational Data Mining (EDM)

View Paper

2017

Analyzing Learner Affect in a Scenario-Based Intelligent Tutoring System

Benjamin Nye, Shamya Karumbaiah, S Tugba Tokel, Mark G Core, Giota Stratou, Daniel Auerbach, Kallirroi Georgila

Proceedings of the 18th International Conference on Artificial Intelligence in Education (AIED)

View Paper

Addressing Student Behavior and Affect with Empathy and Growth Mindset

Shamya Karumbaiah, Rafael Lizarralde, Danielle Allessio, Beverly Woolf, Ivon Arroyo, Naomi Wixon

Proceedings of the 10th International Conference on Educational Data Mining (EDM)

View Paper

2016

Phishing training: A Preliminary Look at the Effects of Different Types of Training

Shamya Karumbaiah, Ryan T Wright, Alexandra Durcikova, Matthew L Jensen

Proceedings of the 11th Pre-ICIS Workshop on Information Security and Privacy (WISP)

View Paper