This ninth volume in the series deals with students' misunderstandings and mistakes in physics and mathematics. This book reports on a study that examined student misconceptions and errors across education systems and over time.

## Introduction

The research in this book explores patterns of student misconceptions, errors, and misunderstandings across educational systems, grade levels, gender, and time using 20 years of data from Trends in International Mathematics and Science Study (TIMSS) and TIMSS Advanced Assessments. Using TIMSS and TIMSS advanced assessment data to investigate student misconceptions, errors, and misunderstandings has many advantages.

## De ﬁ ning the Terminology

*Performance Objectives**Misconceptions in Physics**Errors in Mathematics**Misunderstandings in Physics and Mathematics*

TIMSS and TIMSS Advanced data have been used in a number of secondary analyzes conducted to address the topic of student misconceptions in different countries (Angell 2004; Juan et al. Performance measures are based on the set of TIMSS and TIMSS Advanced items selected for the study.

## Core Concepts in Physics and Mathematics

The TIMSS 2015 Advanced Mathematics Framework (Grønmo et al.2014) expects students at the end of secondary school to solve linear and quadratic equations, as well as systems of linear equations and inequalities, and to use equations and inequalities to solve contextual problems. To solve linear and quadratic equations and inequalities as well as systems of linear equations and inequalities.

## Research Questions

While this report focuses on student misconceptions, errors, and misconceptions related to the two topics of gravity and linear equations, the general methodology described in the report can be applied to a variety of math and science topics covered in TIMSS and TIMSS Advanced. Highlights from TIMSS and TIMSS Advanced 2015: Mathematics and science achievement in the U. Washington, DC: US Department of Education, National Center for Education Statistics.

## Introduction

Abstract Many diagnostic methods have been used to analyze data from large-scale assessments, such as the Trends in International Mathematics and Science Study (TIMSS), and the previous research on misconceptions and misunderstandings in physics and mathematics has been expanded. A comprehensive review of previous research on student misconceptions and misunderstandings in physics regarding gravity and in mathematics regarding linear equations links the established literature to the current study.

## Diagnostic Models Overview

Another study (Choi et al. 2015) also used a CDM approach to compare performance on the TIMSS mathematics assessment between United States and Korean eighth grade samples. Other studies conducted by TIMSS participating countries have taken a similar approach to describe students' understanding and misconceptions based on their responses to individual TIMSS and TIMSS math and advanced science items at different grade levels (Angell2004; Juan et al.2017; Mosimege et al.2017; Prinsloo;

## Misconceptions in Physics

Many students believed that the force was directed upward during the ball's upward motion, and that the force was zero when the ball was at the top of its flight (when it stops momentarily and changes direction). Although students identified the force as downward as the ball moved downwards, most were unable to justify this answer correctly, with many students believing that the force must be directed downwards as the ball moves downwards.

## Misunderstandings in Mathematics

Exploring high school students' reasoning strategies when solving linear equations. Journal of Mathematical Behavior. An analysis of student errors in answering a TIMSS domain algebra problem. Journal of Physics: Conference Series.

## TIMSS and TIMSS Advanced Data

2TIMSS Advanced courses are defined as courses covering most of the topics covered in the TIMSS Advanced assessment frameworks for physics and mathematics. 1The math coverage index is the percentage of the corresponding age cohort covered by the TIMSS Advanced target population (students in their senior year of high school who are taking or had taken TIMSS Advanced eligible math courses).

## Methodology

*Assessment Framework Review and Content Mapping**Evaluation of Item-Level Performance Data**Reporting Patterns in Percent Correct and Percent with**Statistical Comparisons*

Percentage of Students with Misconceptions, Errors, and Misunderstandings Two different types of item-level analysis have been used to determine these percentages:. Differences in the percentage correct and the percentage of students with misconceptions, errors and misunderstandings were calculated (1) between each country and the mean across the five countries, (2) between female and male students within each country, and (3) across assessment years for the trend items.

## Addressing the Research Questions

### Research Question 1

To answer the first research question, we examined items from TIMSS and TIMSS Advanced administered at each grade level that demonstrated specific types of student misconceptions, misunderstandings, and errors. We presented sample items released on each grade level to illustrate the different types of misconceptions, errors, and misunderstandings.

### Research Question 2

What are the common types of misconceptions, errors, and misunderstandings among students in fourth, eighth, and senior high school (TIMSS Advanced students) and how do they compare across countries. Response patterns provided evidence of the nature and extent of students' misconceptions, misunderstandings, and errors.

### Research Question 3

Sets of assessment items at each grade level illustrate the nature and extent of student misconceptions, errors, and misunderstandings across grade levels in five countries (Italy, Norway, the Russian Federation, Slovenia, and the United States). The frequency of specific types of student misconceptions, errors and misunderstandings at each grade level varied across the five countries.

## Introduction

In addition to examples of items, the results include tables and figures that show patterns in the percentage of students exhibiting certain misconceptions, errors, and misunderstandings by country and grade based on the set of items they measure. 3 Tree graphs show differences between women and men by country at each grade level. 4 In grades four and eight, separate trend graphs for each country show the percentage of students with misconceptions, misunderstandings, and errors across multiple years of assessment.5. With the exception of trend item scores, all data reflect the most recent assessment in which each item was used from 1995 to 2015, which is, for example, the year the item was published (Supplementary materials providing standard errors for all estimates are available for download at www.iea.nl/.publications/RfEVol9).

## Physics Results

### Student Performance on TIMSS and TIMSS Advanced

Three MC items, namely items 4, 5 and 6 from performance objective 3 ("determine the effect of gravity on objects in motion or on objects at rest") were among the most difficult items in all countries. Physical performance objectives (PO): PO3 = determine the effect of gravity on objects in motion or on objects at rest, PO4 = identify the direction of the force due to gravity.aData not available for item 6 (see annex for country-specific notes).bData not available for item 5 (see annex for country-specific notes).

### Common Types of Misconceptions and

Draw arrows on the figure that indicate the direction of the ball's acceleration at points P, Q, and R. The international average is therefore based on the other four countries (Norway, the Russian Federation, Slovenia and the United States).

### Patterns in Misconceptions and Misunderstandings

Grade eight item 9 ("force that causes a ball thrown upwards to fall," Fig.4.10), measured the misconception that "gravity alone cannot cause an object that is initially at rest to start moving; it requires another force/push" (P3B). However, misconceptions or misunderstandings related to the direction of gravity were quite common in grade four (Fig.4.19).

### Gender Differences in Misconceptions and

Across the five items measuring the P3A misconception ("gravity acts only on falling objects, but not on objects at rest or on objects moving upward"), there were two items in Italy and the Russian Federation and only one item in the United States, where a significantly higher percentage of females demonstrated the misconception. Table 4.19 Percentage of female and eight students with misconceptions and misconceptions about gravity, by country and 2015 Misconceptions and Misconceptions Item Year Percentage of students with a misconception or misconception (%) Italy NorwayRussian Federation FemaleSlovenia Female, Female, Female, Female, United States of America Performance objective 3: Determine the effect of the gravitational force on objects in motion or on objects at rest P 3A: Gravity only acts on falling objects, but not on objects at rest (on the ground or sitting on another surface) or on objects moving upward.

### Patterns in Misconceptions and Misunderstandings

P3B = gravity alone cannot cause an object initially at rest to start moving; requires another force/thrust, P4B = gravity pushes up on objects sitting on a solid surface and on objects moving up. This reflected an 8% decline in the share of students in Italy and a 5% decline in the United States between 2011 and 2015.

### Summary of Physics Results

In both the Russian Federation and Slovenia, the percentage of students demonstrating misunderstanding decreased over time, and the difference between the first assessment cycle (2007) and the third (2015) was statistically significant. In KR item 11 (“the force that causes a marble to roll down an inclined track”), the percentage of students demonstrating the misconception decreased significantly in the Russian Federation (from 67 to 47%) and Slovenia (from 90 to 84%) between the 2011 and 2015 assessments.

## Mathematics Results

### Student Performance on TIMSS and TIMSS Advanced

One of the most difficult items in fourth grade, item 22, is from Performance Objective 7 (“given pairs of numbers in tables or ordered pairs, create a verbal description of the relationship”). Mathematics Performance Objective (MO): MO7 = Given pairs of numbers in tables or ordered pairs, produce a verbal description of the relationship, MO8 = Given a verbal description of the relationship between a set of numbers, produce pairs of whole numbers that follow that relationship (rule), MO9 = Use algebraic thinking to solve simple real-life problems involving unknowns.

### Common Types of Errors and Misunderstandings Related

More than 60% of students, averaged across the five countries, were able to identify the correct equation of the line (option B). On average across the five countries, 45% of students could identify the correct description of the rule (option A) from the four choices given.

### Patterns in Errors and Misunderstandings Related to

Many students of both grades could not correctly apply the procedure (error M2A). On average, two-thirds of students in the five countries (and at least half in each country) were unable to solve these problems correctly.

### Gender Differences in Errors and Misunderstandings

The Russian Federation had significant gender differences in favor of men on both items in TIMSS Advanced. There were no significant gender differences on any grade four items in the Russian Federation.

### Patterns in Errors and Misunderstandings Related to

Trend data for item 17 show no statistically significant difference in the percentage of students exhibiting this misunderstanding from 2003 to 2007. For item 22, the percentage of students exhibiting this misunderstanding decreased from 2003 to 2007 in the Russian Federation (by 15%) and the United States (by 7%).

### Summary of Mathematics Results

Abstract Assessment Issues from Twenty Years of TIMSS and TIMSS Advanced Assessments enabled the identification of specific types of student misconceptions, errors, and misconceptions related to two core concepts (gravity in physics and linear equations in mathematics). Errors and misconceptions about linear equations were extremely common in all five countries; on average >50% of students demonstrated errors at each grade level.

## Summary of Results Across Physics and Mathematics

### Patterns in Misconceptions, Errors, and

(1) we summarize the results for physics and mathematics; (2) we discuss the limitations and further applications of our methodology; (3) consider the consequences associated with teaching physics and mathematics; and (4) outline some implications for future TIMSS assessment design and reporting. "down towards the surface of the Earth") in Norway. In contrast, in eighth grade, 50% of students in any country showed no misconceptions.

### Gender Differences in Misconceptions, Errors,

Table 5.2 Summary of math errors and misconceptions related to equations on all pages from the high school level, by country, 1995–2015 Performance target Errors and misconceptions Items Percentage of students with errors or misconceptions (average across added items) Countries ITANORRUSSVNUSACountry Avg. There was also one misconception in fourth grade (M8, “unable to identify a correct set of numbers that follow a certain relationship/rule”) in the United States.

### Trends in Patterns of Misconceptions, Errors,

By eighth grade, there were some significant decreases over time in the frequency of misconceptions, errors or misunderstandings, and errors related to gravity (P3A and P3B) and linear equations (M3A and M5). The number of items with trend differences in the United States was the same as in the Russian Federation (total of 10 items in both countries).