A recent large-scale study involving over 5 million students from 58 countries examined whether the content of math test questions could unintentionally disadvantage students from lower socioeconomic backgrounds. The study found that math problems related to money, food, and social interactions, which were assumed to be more relatable for low socioeconomic status students, actually hindered their performance compared to students from higher socioeconomic backgrounds. The research was published in npj Science of Learning.
Socioeconomic status is an individual’s or family’s social and economic position within society. It is assessed by combining income, education level, and occupation of an individual or their family members. This status influences many aspects of life, including access to resources and opportunities, which significantly impact educational outcomes.
Children from families with higher socioeconomic status tend to perform better at school and demonstrate superior academic performance overall. Scientists explain this by the fact that children from these families have access to better tutoring, more enriching educational activities, more supportive environments, stable housing, and better healthcare. All of these factors contribute to cognitive development and academic success.
On the other hand, children from families with low socioeconomic status often have limited access to education, experience more stress due to everyday challenges, and have fewer opportunities for academic enrichment.
Study author Marjolein Muskens and her colleagues note that in countries worldwide, math tests play a critical role in determining certification and admission to secondary and tertiary education. Such tests are often pivotal in shaping a student’s academic and professional future. Math tests are among the tests where children from families with high socio-economic status tend to perform better than children from families with low-socioeconomic status.
The researchers wanted to find out whether this difference might be reduced if tasks used in the math test contained contents that were closer to everyday challenges faced by children from low socioeconomic backgrounds. They describe how families with low socioeconomic status depend more on other people for their basic needs. Due to this, people with low socioeconomic status tend to be more attuned to other people. They may also prioritize external social factors (e.g., wishes of their surroundings) over internal, individual factors (e.g., their own preferences). Because of these social dynamics, these children might perform better on math tasks that refer to their everyday challenges. The study aimed to test whether this assumption held true.
The researchers analyzed data from the Trends in International Mathematics and Science Studies (TIMSS) from 2007 and 2011. Data came from 58 countries and included over 5 million children in grades 4 and 8.
The study authors identified math tasks within this dataset that they considered particularly relevant for children from families with low socioeconomic status. These were tasks that involved money, food, or social interactions. They called them “low-socioeconomic status ecologically relevant content.” The researchers categorized all other available math tasks as neutral regarding socioeconomic status. They then statistically compared the performance of children from families with different socioeconomic statuses on these targeted tasks.
Contrary to the study authors’ expectations, children from the lowest socioeconomic status families performed 16% worse in grade 8 and 18% worse in grade 4 on these items compared to children from the highest socioeconomic status families. The performance gap was most pronounced on tasks about food (where low socioeconomic status children scored 22% lower) and least pronounced on tasks about social interactions (where low socioeconomic status children scored 12% lower).
“Our study provides an important contribution to investigating sources of social inequality in education by showing that content in math items related to money, food, and social interaction may contribute to unintended biases in math tests for students from low-SES [low socioeconomic status] backgrounds,” the study authors concluded.
“This raises the question of whether items with this content should be avoided in math tests. Simply excluding items with this type of content from tests is neither desirable nor feasible. In addition, since equipping students with critical life skills is an important goal of elementary education worldwide, conceptual and procedural understanding of money is a crucial part of what students need to learn. Consequently, when the goal of a math test is to assess the ability to engage in monetary transactions, omitting items with money is not feasible as well. Therefore, it is important to design interventions that could reduce or remove the bias of this content.”
The study sheds light on the links between socioeconomic status and performance on specific types of math tasks. However, the authors note that students with the lowest socioeconomic status often had missing data about their parents’ education. To address this, the researchers used the reported number of books at home as an indicator of socioeconomic status, which might have somewhat affected the socioeconomic status classifications.
The paper, “Math items about real-world content lower test-scores of students from families with low socioeconomic status,” was authored by Marjolein Muskens, Willem E. Frankenhuis, and Lex Borghans.