IEA International Large Scale Assessments

Table of contents

•Our mission•Our network•Our goals•Implementing our studies•Our current studies & new developments•Conclusions

International Association for the Evaluation of Educational Achievement (IEA)

Mission

IEA is a non-governmental, non-profit cooperative of national research institutions and governmental research agencies that originated in Hamburg, Germany in 1958.

IEA develops and conducts high-quality, large-scale comparative studies in education to support countries’ efforts to engage in national strategies for educational monitoring and improvement.

IEA continues to promote capacity building and knowledge sharing to foster innovation and quality in education. IEA studies inspire fruitful dialogue on critical educational issues facing the sector today, informing the development of evidence-based policies and practices across the globe.

A Worldwide Network

IEA draws on the collective strength of its institutional members.

Most of these represent a national system of education, and all are associated with a strong research tradition linking educational researchers and practitioners.

A Worldwide Network• Armenia • Australia • Austria • Belgium (Flemish) • Belgium

(French) • Botswana • Brazil • Bulgaria • Canada • Chile • People’s Republic of China • Chinese Taipei • Colombia • Croatia • Cyprus • Czech Republic • Denmark • Egypt • England • Estonia • Finland • France • Georgia • Germany • Greece • Hong Kong SAR • Hungary • Iceland • Indonesia • Iran • Ireland • Israel • Italy • Japan • Jordan • Kazakhstan • Republic of Korea • Kuwait • Latvia • Lithuania • Luxembourg • Macedonia • Malaysia • Mexico • Morocco • Netherlands • New Zealand • Norway • Palestine • Philippines • Poland • Portugal • Qatar • Romania • Russian Federation • Singapore • Slovak Republic • Slovenia • South Africa • Spain • Sweden • Thailand • Turkey • United Arab Emirates • United States

Our goals• provide a forum for cooperation, development, and implementation

of comparative studies in education IEA studies are joint ventures, relying on a spirit of cooperation between partners to achieve common goals. • promote and advance an evidence-based global dialogue on access

and equity in educationIEA’s studies are an important data source for those working to enhance student learning at the international, national, and local levels.• encourage the exchange of knowledge, expertise and insights to

improve the provision of quality education worldwideIEA has, over the past 55 years, built an international network of scholars, researchers, policy analysts, and technical experts in the field of large-scale educational research.

Why international assessments• Monitor the effectiveness of educational systems in a global context.

• Evaluate progress in educational achievement over time from both international and national perspectives.

• Work towards closing gaps in educational achievement and achieve equity among ethnic, social, or regional groups.

• Examine educational achievement and growth in learning outcomes from primary through secondary school.

• Use the international database to research the factors associated with high learning outcomes.

Why international assessments

• Benchmarking function– Provide comparable indicators on student

performance and schooling practices across countries

• Analytic function– Suggest hypotheses about

• relationships between student performance and factors that might influence performance • areas where students have particular strengths or

weaknesses

IEA Studies‘ design

IEA Studies are • imbedded in school structure (grade based)• curriculum based

IEA Studies consider the processes and products of education: Focus on Content and Cognitive DomainsCapture school, teacher, student, parent contextHave a wide focus (grade levels and subjects)Follow a Trend DesignInclude regional options

IEA Studies’ designUse curriculum as the major organizing concept. IEA studies are investigating:

the intended curriculum (what society intends for students to learn and how the education system should be organized to facilitate this learning)

- Thorough review of countries’ curricula- Thorough review of assessment materials involving all countries - Test curriculum matching analysis

the implemented curriculum (what is actually taught in classrooms) - Teachers are asked what and how they taught students

the achieved curriculum (what students have learned and what they think about these subjects).

- The assessment results are broken down by subjects, areas, and cognitive processes- In-depth analysis of mismatches between countries’ curricula and student

achievement

Influence on the curriculum

• Then it is up to the ministries and policy makers to decide if this is what they want and what is desired for their country

• Countries then might decide on – changes in the curriculum– policies to change the implementation of the

curriculum (for example in-service teacher training programs, or the teacher education)

IEA Studies design (cont)Keeping continuity with the past and relevance to the changeLinking assessments over time

- using the same set of items in successive assessments- restricting changes in design - employing statistics that allow putting data from successive

assessments on a common scale (IRT scaling)- interpreting results cautiously

Bridging study- some students surveyed under old or new procedures

Framework development- maintaining strong connection to constructs fundamental for the study subject- reflecting results of previous data collection and other studies- responding to emerging questions and needs

Implementing IEA Studies

International Study Centers Australian Council for Educational Research (ACER), Australia •

Boston College, USA • Educational Testing Service (ETS), USA • Michigan State University, USA • National Foundation for Educational Research (NFER), UK • Roma Tre University, Italy National Study Centers IEA SecretariatIEA DPC International advisory committees of scholars, researchers, policy analysts, and technical experts

Implementing IEA Studies (cont)

Source of data administering assessment of student achievement in one or more school subjects or within an interdisciplinary area of learning (TEST BOOKLETS).

collecting extensive, internationally comparable information on the background characteristics of participating students and their parents, teachers and schools in order to allow a relation of students’ background and school environments with their performance (QUESTIONNAIRES).

Current IEA studies

IEA TIMSSTrends in International Mathematics and Science Study

Assesses the mathematics and science achievement of grade 4, grade 8, and grade 12 students (grade-based assessment)Conducted every four years since 1995 in up to 70 countries

TIMSS 1995 1999 2003 2007 2011 2015

Participants 45 38 50 59 63 ~ 70

Current cycleTIMSS Numeracy 2015

o For countries where students are still developing fundamental mathematics knowledge, skills, procedures and problem solving strategies; students assessed are in grade 4th, 5th, or 6th

TIMSS Advanced 2015 o Assesses final-year secondary students' achievement in advanced

mathematics and physics, or as an option offered in 2015 for the first time, at the start of their STEM coursework in universities.

IEA PIRLSProgress in International Reading Literacy Study

o Assesses reading ability of 4th-graders (at the time of moving from „Learning to Read“ to „Reading to Learn“)

o Conducted every five years since 2001 in up to 60 countries

PIRLS Literacy o assessment of foundational reading skills

ePIRLS o innovative online assessment of reading achievement; designed

to be responsive to the information age

IEA ICCS

International Civic and Citizenship Education Study

o Investigates the knowledge and understanding of civics and citizenship of 8th-graders

o The second cycle is currently being conducted in 24 countries

IEA ICILSInternational Computer and Information Literacy Study

o Assesses computer literacy competencies of grade 8 students

o Conducted in 21 educational systems

New developments • ICT literacy has become a major emphasis in more and more countries• Schools are equipped with computers and their use is embedded in the

teaching and learning

eTIMSS will assess the students’ science knowledge in a computer-based environment.

Data for School Effectiveness Research

In spite of limitations, large-scale assessments have a great potential to study school effectiveness

A great range of educational factors can be examined

Major endeavor: TIMSS+PIRLS Relationships Report (published in October 2013)

How do homes support literacy and numeracy?Are schools more effective in different subjects? What are the characteristics of effective schools in reading, mathematics, and science?

Result Highlights from TIMSS/PIRLS 2011

Strength

High BenchmarkMathematics – 82%Reading – 67%Science – 46%

High Benchmark All Three – 39%

Low Benchmark All Three – 97%

Capacity building• Assessment is not a goal in itself• Secondary analysis to understand educational systems

• IEA emphasizes capacity building in the countries to• find what works in the country context• find answers to pressing policy questions • find differences within the country• find ways to improve the educational system

• Opportunities: all data available for download; software support for analysis; seminars for countries’ researchers or practitioners

IEA data

All data can be downloaded for free at http://rms.iea-dpc.org/

Tools for analyzing the dataThe IEA has developed the IEA IDB Analyzer - a plug-in for SPSS (for free at http://www.iea.nl/data.html)

What makes us unique• Unique international assessment in primary school• Methodology/design/framework independently defined • High integrity of the IEA activity• Long-term cycle of studies track changes in curriculum and instructional practices,

enabling countries to measure progress over time and identify issues relevant to reform efforts

• E-assessment supports paper-based assessment • Strong network of long-term expertise to leverage. NRCs are integral to the studies’

development process. IEA studies start up national research teams and build capacity that didn’t previously exist.

Thank you!