To be successful in today’s rapidly evolving, technology-mediated world, students must not only possess strong skills in areas such as reading, math and science, but they must also be adept at “21st-century” skills such as critical thinking, communication, problem-solving, persistence and collaboration. Building learning and assessment systems that facilitate the development of such skills requires the recognition and extraction of patterns of human behavior and activities. Advances in computer vision, machine learning, educational data mining, and affective computing have made it possible to analyze a much wider range of student responses and activities. This is enabling the expansion of the types of tasks to elicit the knowledge, skills and attributes of learners and, hence, an expansion of the forms of learning and assessments systems we can use. Inexpensive and ubiquitous sensors like webcams and microphones enable comprehensive sensing of the user’s behavior with real-time capture and recognition of gaze, facial expressions, gestures, body pose, spoken responses and paralinguistics. Such different modalities can then be integrated to model the user’s cognitive thought processes and non-cognitive behavior such as engagement, motivation, persistence and affective (emotional) state during interaction with learning or assessment systems such as educational games and automated tutors. CMLA 2016 held in conjunction with CVPR 2016 provides an excellent platform for the sharing of knowledge and ideas across disciplines including computer vision, educational measurement and social science. Our hope is that by bringing together the best minds in these fields we will be able to further the state of the art and generate further interest and excitement in this area.