Assessment and Analysis
Connection and Belonging: Citizenship and Identity in the Primary English Classroom View Digital Media
Paper Presentation in a Themed Session Ana Patricia Ferreira
Teaching English as a foreign language may be seen as a vehicle for the fostering of a sense of belonging and connection, for sharing a social and cultural heritage, for creating and strengthening the notions of identity and citizenship among young learners. This teaching / learning process, when enriched with intercultural and plurilingual factors, digital resources and opportunities for reflection and discovery of self and of “the other”, may lead to a growing self-knowledge, greater reflexive capacity and critical spirit.
Improving the Indirect Assessment of Globally Applicable to 21st Century Engineering Skills: A Re-examination and Re-design using Rasch Methods View Digital Media
Paper Presentation in a Themed Session Noela Haughton, Carmen Cioc
The goals of this study are to: examine the construct validity of an indirect self-assessment of essential engineering skills; and re-design of the instrument to improve the relevance of the questions. The survey was developed to support the assessment of the re-design of mechanical engineering technology (MET) core courses. Target coursework was third- and fourth-year Advanced Fluid Mechanics, Applied Thermodynamics, and Mechanical Design II courses, all part of a MET program. Curricular changes integrated active learning and assessment strategies, including problem-based learning (PBL) pedagogy, the Kern Entrepreneurial Engineering Network’s 3C’s; and additional use of indirect assessments (peer- and self-assessment) to guide further assessment and curriculum development. Each goal will be supported by the analysis of three years of longitudinal data from multiple courses. The current analysis will enable construct validation and re-design using Rasch Methods. The engineering design framework that helped to guide the development, validation, and the refinement of the survey is the classical engineering design model, an iterative process that engineers use for problem solving, ask, research, and imagine, plan, create, test, and improve. A supplementary step across the framework involves communication. In addition to the engineering framework, the authors superimposed the KEEN’s 3Cs curiosity, connections, and creating value, and thus refined the survey to better emphasize the engineering skills the students need across the curriculum and before graduation.
Changes during the Transition from High School to College: A Longitudinal Case Study of the Development of Students' Cognitive and Metacognitive Processing Skills and Task Perception in Academic Writing View Digital Media
Paper Presentation in a Themed Session Run Mu
In today's information-driven knowledge economies, learners face a heightened need to effectively synthesize information from diverse sources. The capacity to attain communicative competence in writing plays a crucial role in language development and academic achievement across all educational levels. However, previous studies and examination papers showed that secondary students are lack of text-based writing practice and they generally produced a composite of disconnected parts. Transitioning to tertiary education, these students may face pressing issues as almost all academic writing requires them to synthesize information across multiple texts to construct a good argument. Therefore, the current study investigates the transition of students' cognitive and metacognitive processing skills and task perception in academic writing from high school to college and attempts to answer three research questions: (1) What is the nature of the novice writers’ perceptions of academic writing task, and how do these perceptions transit from high school to college? (2) What is the nature of the novice writers’ cognitive and metacognitive processing skills while writing, how do these skills develop from high school to college? (3) What caused the changes of task perception and processing skills? By analyzing 22 students' essays, questionnaire responses, and semi-structured interviews at three distinct time intervals, we observed favorable transformations in novice writers' cognitive, behavioral, and emotional aptitudes related to academic writing. Additionally, we noted a progressive enhancement in students' utilization of cognitive and metacognitive processing skills. The implications for pedagogical support in fostering academic writing skills among novice writers are discussed.
Assessing Ourselves: Inter-rater Reliability Issues in Assessing Student Learning Outcomes View Digital Media
Paper Presentation in a Themed Session Lauren Mandel
Student learning outcomes assessment requires two levels of assessment: assessing student work and assessing how well the program is assessing the students’ work. In a professional master’s degree program, a rubric was created for assessing student work on achievement of program-level educational outcomes. That rubric was used from 2018-2023 (six years). Over that time, the inter-rater reliability scores varied widely. One issue is that student work is designed to demonstrate achievement of course learning objectives, and the Assessment Committee needs to apply the rubric to assess that student work for achievement of program learning outcomes, which differ from the course learning objectives. Another is that multiple types of reviewers have used the rubric over the past six years: members of the committee who are full-time department faculty, as well as members of the department’s advisory board, some of whom are part-time department faculty and some of whom have no faculty experience. This paper reports a secondary analysis of inter-rater reliability scores for the rubric, with recommendations from the literature on how to improve reliability in assessing student learning outcomes.
