Teachers as Makers
Effecting change in schools is often times mediated through teachers, the trained individuals who interact with students on a daily basis. While teachers are tasked with bringing out innovation and creativity in their students, teachers should also see themselves as innovative and creative individuals. Hence, a portion of my work both domestically and internationally is aimed towards training teachers to not only be capable at using digital fabrication and invention, but also helping teachers see their own creativity. Furthermore, in addition to fostering a sense of innovation among teachers, this work also involves helping teachers develop curricular units that truly embody the ideas of constructionism.

Semi-Supervised Object Tracking
There are a range of interesting materials available for students to use when building and inventing. Moreover, their building process highlights their level of cognition and engineering knowledge. This project uses a collection of computer vision techniques in order to capture and track the movement of objects without the need for instrumentation.

Low-Cost Student Localization and Collaboration
Student collaboration and use of external resources play a central part in many learning environments. This project aims to provide a low-cost solution for storing this information by using wireless signals in conjunction with Machine learning and Probabilistic Modeling in order to approximately localize students and detect student collaboration.

Multimodal Data Capture for Situated Settings
A primary challenge of multimodal learning analytics is collecting data that is of sufficiently high quality for computational analysis, while also being collected with the goal of doing rich analyses. This particular projects aims to bridge the learning sciences and computer science community by developing guidelines, and sample hardware/software tools that can be used to capture and analyze multimodal data using cutting edge artificial intelligence, and in the support of constructs that are relevant to education researchers. This work is currently funded by a NSF EAGER.

Building as Assessments
Building Assessments studies ways that expertise is evidenced in the microscopic and macroscopic actions that students take when building physical artifacts. A combination of building actions, student gesture data and spoken language can be used to identify student competencies and opportunities for additional learning.

Programming Pathways
Students approach computer programming with different intuitions and approaches. Often times the approaches utilized suggest underlying cognitive processes, and misconceptions. This project analyzes student programming pathways by using novel machine learning algorithms in order to exam common pathways that students follow both within a given assignment and across assignments. This project also studies qualitative learning data about each student as well as they help seeking behavior.

Sentiment, Speech and Drawing Analysis
This work focuses on the intersection of speech, sentiment and drawing for studying short-term and long-term engineering proficiency in constructionist learning environments.

Identity in Construction
Identity formation is a primary area of development in constructionist and project-based learning environments. This project studies how these changes are manifested through students actions and speech.

Studying Practices of Effective Learning Strategies
One affordance of multimodal analysis is the ability to capture and compare the multimodal behaviors associated with different learning/ teaching strategies. This work does just that by identifying the underlying behaviors that differentiate different teaching/learning strategies.

Deciphering Complex Instruction via Multimodal Learning Analytics
Complex instruction includes opportunities for mixed ability groups to engage in co-construction of knowledge through in-depth discussions and group collaboration. However, in the context of complex instruction, a variety of socio-emotional behaviors are certain to emerge. In order to validate and better support these behaviors, we are leveraging multimodal learning analytics, with a particular focus on the discourse, sentiments and non-verbals cues that are exhibited across repeated group interactions.

OpenGesture
OpenGesture is an open-source, low-cost, easy to author, application framework for collaborative, gesture and speech based learning applications. The platform is designed to enable students, teachers, researchers, practitioners and parents to author innovative learning applications that are built on the cutting edge of human computer interface technology. Furthermore, the platform supports extensive opportunities to collect data about how users are engaging with the various applications.

MAGIC TV & MAGIC MAP
MAGIC TV and MAGIC MAP are two applications that enables the user to use speech and gestures to interact with their television. These applications leverage Nintendo Wiimotes, microphone arrays and proprietary speech recognition technology to create an intuitive and easy to use environment for the average user. One particular feature of interest is the system's ability to automatically differentiate between system directed speech and non-system directed speech (i.e. speech that is intended for another person).