eFiL: e-Feedback for interactive Lecture

Expectations and experiences of today's students seem to diverge increasingly from the traditional teaching practices. In a large classroom, student-teacher interactions are rare, and students are more spectators than actors during the lecture being taught. Similarly, teachers cannot assess understanding and real-time assimilation of their lectures by students.

Today, new digital devices open many opportunities for innovation in the field of learning. The range of digital devices is very large and still largely unexploited: digital tablet, pen-based computer, electronic pen, notebook, interactive screen, multi-touch table, ... All these digital tools induce potential for new uses that are yet to be imagined: digital handwritten annotation or drawing, multi-user and multi-media interaction, interaction with peers, collaborative learning, real-time feedback... Our learning methods are being reinvented, and one of the main challenges for researchers is to design and assess content and learning processes for 21st century higher education. Pen-based hybrid notebooks connected with interactive displays offer potential uses that are in harmony with our ‘usual’ learning behaviours: taking notes, annotating a document, drawing a diagram, filling in a form, answering questions... Pen based hybrid notebooks or usual learning behaviours are therefore important promoters of memorization, understanding, assimilation, synthesis, but also interaction, explanation, thinking…

The aim of eFIL project is to go beyond ‘classical’ active learning methods to improve learning in higher education using and evaluating a new digital learning environment (see challenge 1) based on CAMIA/KASSIS.

CAMIA/KASSIS is a new digital learning environment (workbook) for active learning in higher education for face-to-face lectures. It was designed and developed during the last 2 years at the INSA Rennes in collaboration with lecturers and the IntuiDoc team of IRISA laboratory.  The concept is based on one of the learning keys which is to make the learner active during his learning. The handwritten note-taking is one of the first elements to make the learner active through a continuous effort of synthesis and emphasis which allows a better appropriation of the delivered discourse. The CAMIA/KASSIS solution is based on the networking (autonomous network or existing network) of a set of pen-based hybrid tablet. It is an interactive digital solution based on several innovative concepts that are integrated within the same digital learning ecosystem:

  • it offers to students and lecturers a direct handwritten note-taking on the slides to facilitate the composition of formulas, diagrams, sketches, drawings, annotations (going beyond limited keyboard input);
  • it offers handwritten advanced graphical quizzes composition for teachers: questions can be expressed on-the fly, by sketching or drawing directly on the tablet display (going beyond rigid and planned interaction based quizzes or push button);
  • it offers real time aggregation, analysis and restitution of the responses collected for immediate feedback for students and teacher.

CAMIA solution has been recently transferred to Learn&Go company to finalize the development and to operate the operating system migration (windows, android, ios): the result is  packaged in a new derived application named KASSIS. One of the goals of eFIL project is to scientifically evaluate the learning impact of this concept and to improve and extends the CAMIA/KASSIS functionalities and design. For instance, in eFIL project we will add to main functionalities:

  • An automatic synthesis of the collected hand-drawn responses by generating in real time an interactive saliency map summarizing the distribution of graphical responses of all the students in response to a graphical quizzes submitted by the teacher.
  • A Lecturer’s dashboard to provide the Lecturer with a posteriori analysis of student activity and to allow the Lecturer to monitor in real-time the students’ rate of activity through the trace analysis automatically collected (handwriting, quizzes, navigation...) and to steer the unfolding of the lecture accordingly.

CAMIA/KASSIS is an integrated digital teaching environment that can be considered as a complementary tool to traditional individual note taking systems like OneNote and Evernote that allow a long-term archiving of the notes and offer recognition possibilities of the notes (OCR).

In the long term, it is therefore expected that the students will use the interactive CAMIA/KASSIS ecosystem during the lecture and that they will export their notes under existing software "OneNote / EverNote" for archiving and managing them.

In eFIL Project, this new environment will be designed, developed, and tested in classroom settings during lectures (see challenge 2). Traces will be collected and analysed both to explore learning dynamics in classroom, and to provide a dashboard allowing instructors to monitor their activities during lectures as well as in-between sessions (see challenge 3). To summarize, three main challenges will be addressed in eFil Project.

Challenge 1: Going beyond ‘classical’ active learning methods to improve learning in higher education

The first challenge of the eFIL project is to go beyond the classical active learning methods in using digital devices, and more particularly the CAMIA/KASSIS digital learning environment based on pen-based tablets. The objective is not only to improve note-taking on CAMIA/KASSIS during lectures stimulating the note-takers to actively process the course material, but also to improve students learning in using real-time feedback tools helping students to gauge their understanding of complex concepts from interactive activities based on Peer Instruction (an instructional method for engaging students during class through a structured questioning process). It consists of asking each student to first respond to the poll independently, then to discuss the question with peers, and then to re-vote. Although this method has been recognized as an effective instructional strategy enhancing students’ understanding of difficult concepts, the feedback given to students comes from their responses using clickers, and questioning is limited to conceptual multiple-choice questions (or quizzes). One of the objectives of the eFIL project is to go beyond the Peer Instruction method mainly based on clickers to increase the richness of questioning using pen-based tablets and shifting from conceptual to graphical quizzes to provide students a richer collective feedback than a simple histogram of responses. Thus, tools providing collective feedback such as saliency maps and dashboards, will be evaluated in a series of studies.

 

Challenge 2: Optimizing eFIL learning environment with user-centered design

In all steps of the project, efforts will be devoted to usability and acceptability evaluation using a human-centered design (ISO 9241-210) in all work-packages. This approach aims at making systems usable by focusing on the users, their needs and requirements, and by applying human factors, ergonomics and usability knowledge and techniques. When prototypes will be designed, they will be used to collect user feedback while and after carrying out tasks. These user tests will serve to assess whether usability objectives including measuring usability performance and satisfaction criteria have been met. These tests will also analyse the influence of the interface design and quality of human-computer interaction on user experience, acceptance, learning, and motivation. The information gathered will be used to drive the design of subsequent prototypes.

To conduct the experiments associated with human-centered design approach, we will rely on institutions partners of eFIL project: INSA Rennes, University of Rennes 2 and University of Nantes. This study will also be done in partnership with UBL (University of Loire Bretagne) which has confirmed its great interest in this project.

This multidisciplinary partnership will provide the opportunity to have a sample of students with different profiles on a variety of pedagogical contents (e.g. human sciences and engineering sciences). Studies will be conducted both in a user-lab environment (in early stages of each work package to evaluate first prototypes, see www.Loustic.net ) and in natural setting during lectures for later stages.

 

Challenge 3: Exploring and analyzing traces to monitor learning activities in classroom with Educational Data Mining and Learning Analytics

Most of the studies will imply the collection of CAMIA/KASSIS traces, their analysis and mining to detect interesting statistics and patterns that help compute indicators of the learners’ activity. Such indicators will be parts of interactive dashboards aiming at sustaining the tasks of

  • scientists who carry out experiments, and benefit from trace data analysis;
  • instructors who propose learning activities and need to monitor their unfolding both in real time during the activity (for course and students’ activity steering) or after (for reflective analysis of pedagogical strategy, student progress measurement, course evolution).

Designing these indicators and dashboards will imply careful modelling of traces (meaningful events in learners’ activity with CAMIA/KASSIS), trace collection, trace analysis (from basic statistics to mining), and the iterative design of meaningful interactive visual representation of trace data.