DigATaL – Framework

Methodolgy of the self-assessment tool

The self-assessment is based on the DigATaL framework, which describes four core competences and seven cross-linked competences and four different levels of proficiency. The tool is structured so that four questions target the core competencies needed to create and use digital assessments. Each answer option represents a progression level. In addition, there is a question at the end of the tool on the cross-linked competencies, with each question representing one of the competencies.

In education, assessment refers to a variety of methods or tools that educators use to assess, measure, and document students’ academic readiness, learning progress, skill acquisition, or educational needs. The self-assessment consists of five questions and can be completed within five to ten minutes. Afterwards, you will receive an assessment and a recommendation in which areas you could develop your competences. It is intended to provide you with an orientation as to which competences are necessary for the use of digital assessments and how pronounced they are in your case. Your results are only visible to you and will not be used for other purposes. At the end of the test, you will receive brief feedback on each individual competence, as well as a recommendation for the development of this competence and an overall assessment of the level of your competence.

On Digital Assessments in General

Assessments serve as critical tools for assessing performance, involving the gathering and analysis of data and information (see Bartholomeyczik 2012, p. 110). It’s important to distinguish between assessments conducted within the healthcare sector, which are diagnostic in nature, and those carried out in educational settings, where they assess a learner’s fit for a course and inform the didactic approach. Such assessments enable learners to evaluate their own abilities, identifying strengths and weaknesses, while providing educators with insights into individual and collective learning progress. Based on these insights, educators can tailor their instructional methods (refer to Franke & Handke 2012, p. 147). The AHEAD project specifically concentrates on the role of digital assessments in education, rather than diagnostic evaluations, setting the criteria for selecting appropriate assessment tools.

In the educational sphere, assessments perform multiple functions, including selecting suitable learners, providing feedback and diagnostics, documenting progress, and motivating students. However, they also pose potential stressors for both teachers and learners, due to the preparation involved and the implications of their outcomes (Franke & Handke 2012, p. 147 ff.).

Assessments are integrated at various stages within the teaching and learning process, categorized into diagnostic, formative, and summative assessments (Franke & Handke 2012, p. 149). Diagnostic assessments occur before learning begins to gauge learners’ existing knowledge and suitability. Self-assessments are particularly relevant here (Franke & Handke 2012, p. 150). Formative assessments monitor progress throughout the educational process, aiding in the adaptation of teaching strategies (Franke & Handke 2012, p.151). Summative assessments evaluate the achievement of learning objectives at the end of a course, often through exams or projects (Franke & Handke 2012, p. 151 f.). The discussion also extends to digital and self-assessments as alternatives to traditional assessment forms.

The terms e-assessment, digital assessment, and online assessment are used interchangeably in literature. This text will consistently use “digital assessments” to refer to assessments in an educational context (Thiele & Kauffeld 2020, p. 4). Digital assessments provide advantages such as flexibility in time and location, anonymity, and objectivity in evaluation (Ott, Ulfert & Kersting 2017, p. 216 ff.). While various generations of assessments have been identified (Siarova, Sternadel & Masidlauskaite 2017, p.58), this text will not categorize digital assessments, instead focusing on examples like online peer assessments and video sharing for practical skills (Kapsalis et al. 2019, p.17ff.; Seery et al. 2017, p.403f.).

Regarding competency frameworks, the term is often used interchangeably with competency models and grids, despite conceptual differences. Competency frameworks provide a broad structure for classifying competences, while competency models and grids are more specific to organizations or for detailed competence assessment. This article adopts the term “competency framework” in line with EU frameworks and project collaboration, despite the nuanced distinctions (Neumann 2013).

Neumann (2013) discusses the resolution of competency models, emphasizing the balance between granularity and the measurable distinctiveness of traits. Competency frameworks developed for digital assessment competency, such as those discussed in the AHEAD project, aim for a higher resolution than general educational goals would require.

Self-testing and assessment tools facilitate personal evaluation of competences, skills, and knowledge. They promote self-awareness and self-directed learning, though challenges such as personal bias and the Dunning-Kruger effect must be navigated carefully to ensure accurate self-evaluations. These tools translate competencies from frameworks into assessable statements, aiding in personal and professional development.

On the Importance of Digital Assessment Competencies

Digital assessment competence refers to the array of skills and knowledge necessary for leveraging digital tools in evaluating performance, utilizing digitally collected data to chart learning progression, and identifying instructional challenges through digital insights. This competence extends to the consistent monitoring and adjustment of teaching methodologies based on digital feedback, integral to a teacher’s professional skill set.

Additionally, insights from Ireland on mean values and standard deviations related to this competence are provided for a comprehensive understanding.

In Baumert and Kunter’s model of professional action competence for educators (2011, p. 32), the essential teacher knowledge incorporates subject-specific pedagogical knowledge, which includes both reflective and diagnostic abilities essential for improving instructional quality (as discussed by M. Kramer et al., 2020; Schmelzing et al., 2009). Furthermore, expertise in learning diagnostics and performance evaluation forms a core component of pedagogical knowledge (Voss & Kunter, 2011).

The Hamburg Competence Framework (Hamburger Landesinstitut für Lehrerbildung und Schulentwicklung 2018) illustrates how digital assessment competence is interlinked with other competency domains, particularly digital and subject-specific skills. This framework outlines the creation and application of digital assessments, offering a detailed view of the requisite capabilities. For instance, at the foundational level, an educator may recognize the basic potential of digital assessments for student evaluation. However, fully utilizing digital evaluations to enhance teaching across various contexts signifies a more advanced level of competence, within the scope of instructional diagnostics, which has become increasingly significant for evidence-based teaching practices.

The framework defines four levels of progression in digital assessment competence, categorized into four areas of content. It acknowledges that proficiency in one area does not necessarily imply expertise across all. For example, an educator might be well-versed in data protection and copyright due to training in another field, yet may only possess foundational knowledge in crafting and utilizing digital assessments.

The progression from “Foundation” to “High-specialized” levels covers a spectrum of competencies:

At the “Foundation” level, capabilities include basic content creation, user competence with standard software and hardware, and elementary data literacy focusing on data storage, retrieval, and organization while considering data protection.
The “Intermediate” level introduces integrated use of digital assessments, comprehensive evaluation methods, ethical considerations, and a broader understanding of digital technologies.
At the “Advanced” stage, educators engage in cooperative use of digital assessments, facilitate peer-assessment, and encourage self-reflection among students, alongside advanced user competence and data literacy for organizational needs.
The “High-specialized” level involves designing novel assessments, utilizing cutting-edge technology, pioneering data management, and innovating assessment methods.

• At the “Foundation” level, capabilities include basic content creation, user competence with standard software and hardware, and elementary data literacy focusing on data storage, retrieval, and organization while considering data protection.
• The “Intermediate” level introduces integrated use of digital assessments, comprehensive evaluation methods, ethical considerations, and a broader understanding of digital technologies.
• At the “Advanced” stage, educators engage in cooperative use of digital assessments, facilitate peer-assessment, and encourage self-reflection among students, alongside advanced user competence and data literacy for organizational needs.
• The “High-specialized” level involves designing novel assessments, utilizing cutting-edge technology, pioneering data management, and innovating assessment methods.

This competence framework aligns with several existing frameworks for educator professional development, including:

• The European Framework for the Digital Competence of Educators (DigCompEdu), emphasizing digital technologies in teaching and learning.
• The Technological Pedagogical Content Knowledge (TPACK), focusing on integrating technology into teaching.
• National and international standards, such as the National Educational Technology Standards (NETS) for Teachers by the International Society for Technology in Education (ISTE).
• In the context of German education, the framework connects with the KMK’s “Education in the Digital World” strategy, highlighting content creation, communication, data protection, and innovative problem-solving in assessment.

This comprehensive view of digital assessment competence underscores its pivotal role in modern educational practices, providing a framework for developing and enhancing teacher capabilities in digital evaluation methods.