Development of a Multi-Metric, Multi-Perspective Evaluation and Certification System for a Lifelong Learning Society
| Title | Development of a Multi-Metric, Multi-Perspective Evaluation and Certification System for a Lifelong Learning Society |
| Lead Institution | Tohoku Gakuin University |
| Principal Investigator | Tadashi Inagaki |
(A-3) Development of a Multi-Metric, Multi-Perspective Evaluation and Certification System (a-4-①)
In this research, we aim to capture aspects of learning that have traditionally been difficult to assess by combining diverse evaluation perspectives—such as self-evaluation, peer evaluation, and teacher evaluation conducted in schools—with log data provided by the system.
As part of the “new learning” envisioned by this SIP project, we are working on developing an evaluation framework, support systems, and methods for visualizing the learning process and outcomes through data—targeting inquiry-based learning conducted during periods of integrated studies and in individual subjects, or across subject areas.
(A-3-1) Development of Evaluation Metrics from Multiple Perspectives
We are developing new metrics that integrate multiple perspectives, including learning logs, self-evaluations, peer evaluations, teacher evaluations, and other forms of assessment.
The evaluation metrics are expected to include both competencies—such as the ability to identify and solve problems, language skills, information literacy, and non-cognitive abilities—and the content knowledge acquired as a result of inquiry-based learning.
Figure 5 summarizes the approach to evaluation in inquiry-based learning. Traditionally, tools such as portfolios for tracking the learning process, rubrics for indicating the quality of learning, and performance tasks that demonstrate acquired knowledge and skills in real-world contexts have been used.
In response to these existing methods, this project focuses on: (1) visualizing the inquiry process as educational data, (2) visualizing the relationship between assessments and educational data, and (3) developing methods to visualize the learning outcomes of inquiry-based activities.
Regarding the visualization of the inquiry process in (1), we are particularly focusing on the information-gathering stage. Students collect information from a variety of sources, including books, the web, and field investigations.
By visualizing who is gathering what kind of information and how far they have progressed, it becomes possible to provide timely evaluations and targeted support.
RefNavi, currently under development, supports the visualization of information collected by students by storing it in a database and displaying it using various indicators such as timeline, media type, genre, and difficulty level (see Figure 6).
Regarding (2), the relationship between assessment and educational data, various assessment tests provided by private companies evaluate a wide range of competencies and abilities (see Figure 7).
We aim to identify the learning behaviors and conditions that contributed to the assessment results by analyzing how those results relate to the learning logs accumulated throughout the learning process.
In the future, we aim to extract indicators from learning logs that are useful for viewpoint-based evaluations, develop methods for constructing digital twins based on other evaluation approaches and learning outcomes, and link these to knowledge maps so that they can be stored within the OKLM Digital Twin (see Figure 8).
In the development of methods for visualizing the learning outcomes of inquiry (3), we focus on identifying what kinds of learning content are included in students’ inquiry processes and the outcomes they produce.
By linking learning outcomes to frameworks such as the Course of Study codes and the Nippon Decimal Classification (NDC), we aim to establish a method for openly recognizing their value. This includes referencing classifications of information gathered through RefNavi, developed in (1), and conducting pilot implementations of a system for awarding open badges as certification for learning achievements.
(A-3-2) Development of Methods for Customizing Evaluation Metrics
The perspectives and weighting of evaluation vary depending on factors such as the type of human resources sought by the local community, each school’s individual school policy, and the characteristics of the educational content and services.
Along with providing methods for customizing the indicators developed in (A-3-1) to suit the needs of local communities and educational institutions, we are also working on developing a framework that enables integration with the evaluation metrics used by various educational content and service providers.