UnitedNationsEducational,Scienti?candCulturalOrganization

?Shutterstock,AVAVA

Measuringglobaleducationgoals:

HowTIMSShelps

MonitoringprogresstowardsSustainable

DevelopmentGoal4usingTIMSS(TrendsinInternationalMathematicsandScienceStudy)

Educat'on

2Measuringglobaleducationgoals:HowTIMSShelps

Whythisbooklet?

ThisbooklethasbeenpreparedfortheinternationalreleaseofTIMSS(TrendsinInternationalMathematicsandScienceStudy)2019andinthecontextofthestudy’seffortstoadvancetheglobalSustainableDevelopmentGoal(SDG)4,monitoringprogressandpromotingappropriatenational,regionalandinternationaltoolsformeasuringlearningoutcomes.

TIMSSisaflagshipstudyoftheInternationalAssociationfortheEvaluationofEducationalAchievement(IEA),andTIMSS2019markstheseventhcycleofthestudy,providing24yearsoftrends.Conductedeveryfouryearssince1995,TIMSShasbeenavaluabletoolformonitoringinternationaltrendsinmathematicsandscienceachievementatthefourthandeighthgrades.Fromthe2019cycleonwards,aninnovativecomputerizedversionofTIMSSenablescountriestoinvestigatecomplexareasofthemathematicsandscienceframeworkthataredifficulttomeasurewithtraditionalpaperandpenciltests.

TheanalysespresentedinthisbookletarebasedonresultspresentedintheTIMSS2019international

report,TIMSS2019InternationalResultsinMathematicsandScience1,andadditionalanalysesconductedbyIEAandUnitedNationsEducational,ScientificandCulturalOrganization(UNESCO).Thisbookletprovidesfurtherinsightsonhowlearningassessmentsnotonlycanhelpmonitor,butalsoimprovelearningoutcomesthroughinterventionsrelatedtoequity,schoolviolence,learningenvironmentorteacherqualifications.Inaddition,itshowshowlearningisassociatedwithparticularfactorsthatalsoneedtobecloselymonitored.Theseincludechildren’searlychildhoodexperiences,theirmotivationandeducationalaspirations,theirhomebackground,aswellastheirlearningandteachingenvironments.

UNESCOandIEAhavereleasedthisbooklettoincreaseparticipants’understandingofTIMSSfindingsandtheirrelevanceforpolicy-making,theapplicationoflearningassessmentstomeasureglobaleducationtargets,andactionsneededtotranslatetheEducation2030Agendacommitmentsintonationaleducationdevelopmentefforts.

?istock,monkeybusinessimages

1FortheTIMSS2019internationalresultsvisit:

/timss2019/international-results/

Measuringglobaleducationgoals:HowTIMSShelps3

IEA’sTIMSS

Forover60years,IEA(

iea.nl

)hasbeenconductinginternationallarge-scaleassessments(ILSAs)oneducationalachievementandotheraspectsofeducation,includingTIMSS,withtheaimofgainingin-depthunderstandingoftheeffectsofpoliciesandpracticeswithinandacrosssystemsofeducation.

TIMSSisdirectedbytheTIMSS&PIRLSInternationalStudyCenterintheLynchSchoolofEducationatBostonCollege(

),workinginclosecooperationwiththeIEAandthenationalcentreoftheparticipatingcountries.TIMSSandPIRLS(ProgressinInternationalReadingLiteracyStudy;aninternationalassessmentofreading),togethercompriseIEA’scorecycleofstudiesmeasuringachievementinthreefundamentalsubjects—mathematics,scienceandreading.

TIMSSprovidesinternationallycomparativedataonhowstudentsperforminmathematicsandscience.LikethepreviousTIMSSassessments(conductedin1995,1999,2003,2007,2011and2015),TIMSS2019collecteddetailedinformationaboutcurriculumandcurriculumimplementation,instructionalpracticesandschoolresources.

TheTIMSSschool,teacher,studentandhomequestionnairesgatherextensiveinformationaboutthecontextualfactorsatschoolandhomewhichareknowntobeassociatedwithlearningandstudents’achievement.Theseincludedetailsonhowtheeducationsystemisorganizedtofacilitatelearning,students’homeenvironmentandsupportsforlearning,schoolclimateandresources,andhowinstructionusuallyoccursinclassrooms.TIMSSalsopublishesanencyclopediathatprovidesrichdataabouteachcountry’seducationalcontextforlearningmathematicsandscience2.

Seventy-twoeducationalsystemsparticipatedinTIMSS2019,including64countriesanddependentterritories,and8benchmarkingentities.Conductedeveryfouryearsatthefourthandeighthgrades,TIMSShasaquasi-longitudinaldesign,withthefourth-gradestudentcohortassessedfouryearslaterattheeighthgrade.Assessingfourth-gradestudentscanprovideanearlywarningfornecessarycurricularreforms,andtheeffectivenessofthesereformscanbefurthermonitoredattheeighthgradefouryearslater.

THEROSETTASTONEPROJECT

RosettaStoneisacollaborativeprojectproposedbyIEAandTIMSS&PIRLSInternationalStudyCenter,andledbytheUNESCOInstituteforStatistics.

TheRosettaStoneprojectaimstoprovideaconcordancetableandconfidenceregions3thatwillallowforassociatingcountries’achievementonregionalmathematicsandreadingassessmentswith

benchmarksontheTIMSSandPIRLSachievementscales,enablingcountriestomeasureprogresstowardSDGtarget4.1(seepage6ofbooklet).

Theconcordancetablewouldrepresentthe‘RosettaStone’,analogoustotheoriginalRosettaStonewhichprovidedalinkbetweentheGreekscriptandEgyptianhieroglyphics.

2FortheTIMSS2019Encyclopedia:

/timss2019/

3ForfurtherinformationontheRosettaStoneproject:

/wp-content/uploads/sites/2/2019/08/GAML6-REF-4-

Rosetta-Stone-IEA.pdf

4Measuringglobaleducationgoals:HowTIMSShelps

EducationalresponsetotheCOVID-19crisis

Theyear2020hasbeenmarkedbytheCOVID-19pandemic,aworldwidecrisisthatledtoalevelofeducationaldisruptiongreaterthananythingseenbefore.AccordingtoUNESCO,morethan90%ofalllearnerswereaffectedbyschoolclosurescausedbyCOVID-19attheearlypeakofthedisruption.

Attheonsetofthepandemic,UNESCOsupportedandfacilitatedpolicylearning,knowledgebuildingandsharingthroughdifferentinitiativesandpolicyfora—forexample,theGlobalEducationCoalition,whichprovidesaplatformforcollaborationandexchangetoprotecttherighttoeducationduringtheeducationdisruptionandbeyond,theGlobalEducationMeeting,whichprovidesaplatformforexchangeamonghigh-levelpoliticalleaders,ministers,policy-makers,multilateralorganizations,developmentpartnersandglobaleducationactorstoprotectandreimaginethefutureofeducationandtheachievementofSDG4.

Asthecrisisamplifiesinequalities,andpolicy-makersarerequiredtomakedecisionsonreal-timebasis,whatCOVID-19hastaughtusistheneedforquickandreliabledata.Ithasalsoneverbeenmoreimportantto

stayconnectedandcollaboratewithandlearnfromoneanother,acrossandwithincountries.

IEA’sstudiesarerecognizedforfocusingonthehighestqualityandcomparabledata,andthewealthofcontextualdatafromstudiessuchasTIMSScanofferinsightsatatimewheneducationisfacinguncertainty.AmongsomeofthedatacollectedfromTIMSSincludeshowcrowdedteachersreportedclassroomstobe;studentsreportingarrivingatschoolhungry;schoolhygiene;availabilityofcomputerresources;teacherprofessionaldevelopment,especiallyrelatingtotheintegrationofinformationandcommunicationtechnologyinteachingandlearning;andstudentresourcesathomeandiftheyhaveaquietplacetostudy.

Fortunately,TIMSS2019datacollectionstartedinthefirsthalfof2019,beforeanytracesoftheCOVID-19pandemic,andwaslargelyunaffectedbyschoolclosures.ThismeansthatTIMSScanprovideaneffectivebaselinemeasure,forexampleofstudentperformanceorattitudes,againstwhichtogaugehowchangesimplementedduringandafterthepandemicmayhaveaffectedstudentsandschools.

RoleofILSAsinmonitoringSDG4

In2015,UNMemberStatesadoptedanewsetofambitiousgoalstoaddresspoverty,inequality,disease,unsustainablepatternsofconsumption,climatechangeandotherdevelopmentchallengesby2030.TheSDGsproposeacomprehensiveframeworktoensurethatnooneisleftbehind.TheSDG4aimsto‘ensureinclusiveandequitablequalityeducationandpromotelifelonglearningopportunitiesforall’,anditscorrespondingtargetspromotearenewedandbroaderfocusoninclusionandequity.SDG4alsoexpressesaclearconcernwiththeoutcomesofeducationalprocesses—intermsofbotheffectiveacquisitionofbasiccompetenciesaswellasofrelevantlearningforcivicparticipation,socialandculturallife,andemployment.Thisglobalcommitmenthasimportantimplicationsforeducationpolicy,planningandpractice,andparticularlyinareasthatarecriticaltoimprovinglearningprocessesandoutcomes.Suchareasincludeteachereducationandcontinuousprofessionaldevelopment;curriculumdevelopmentandthedesignofteachingandlearningmaterials;schoolmanagement;pedagogicalapproaches;andlearningassessment.

Learningassessmentreferstoawiderangeofmethodsandtoolsusedtoevaluate,measureanddocumentlearningoutcomesandlearningprogress.Assessments

gatherinformationfrommultiplesourcesonwhatlearnersknow,whattheycandowithwhattheyhavelearned,andprovideinformationontheprocessesandcontextsthatenablelearning,aswellasidentifyingfactorsthatmaybehinderingprogressinlearning.Morerecently,therehasalsobeenagrowinginterestinlearningassessmentforaccountability,todeterminetheextenttowhicheducationsystemsyieldthedesiredlearningandsocialoutcomes.

SDG4isasignificantdeparturefromtheMillenniumDevelopmentGoalsinthatthereisamuchgreateremphasisonlearningoutcomesasopposedtoparticipation.Thisimposesnewchallengesfordatacollectionandreporting.TheEducation2030AgendarequiresefficientandaccuratesystemstomeasureprogresstowardsSDG4commitments.Internationallearningassessmentscanmakeimportantcontributionstothisprocessbycollectingcross-nationaldatathatassessprogresstowardslearninggoals,creatingacommonlanguagefordefininganddiscussingcompetencylevelsinlearning.Further,thesedataalsoprovideimportantmeasurementsofqualityeducation,complementingthecommonlyusedproxyindicatorssuchasstudent-teacherratios,percentageoftrainedteachersorexpenditureperstudentasapercentageofgrossdomesticproduct.

Measuringglobaleducationgoals:HowTIMSShelps5

Policyimplications

TIMSSisnotexplicitlydesignedtomeasureSDGsandthereforeTIMSSdataarenotalwaysanexactmatchforthetargetdefinition4.Thereare,however,importantoverlapsbetweenwhatTIMSSmeasuresandwhatisincludedintheSDGs.Therefore,theresultsoftheanalysespresentedinthisbookletcanbeconsideredproxymeasures.

TIMSSdatacanshedlightonandbenchmarkcountryprogresstoachievesomeSDG4targetsrelatedtostudentachievement,accesstoearlychildhoodcareandeducation(ECCE),reducinginequities,knowledgeandskillsacquisitionforsustainabledevelopment,andbuildingbetterlearningandteachingenvironments.BasedonthedataofTIMSS2019,educationpoliciesneedtocontinuedevelopingevidence-basedtargetedstrategiestoaddressexistingchallenges.

WhilesimilarproportionsofGrade4boysandgirlsreachedminimumproficiencyinmathematics,awideninggendergapbasedonaveragescoresfavouringboyscomparedto2015datasuggeststheneedforarefocusedstrategy,additionalresourcesanddifferentiatedinstructionbeginningatearlyprimarygradestoeliminatethegendergap.Meanwhile,atGrade8,morecountriesshowmoregirlsmeetingminimumproficiencythanboyssuggestingtheneedforongoingeffortstoreducedisparity.

Datashowedthatwherestudentshadgreaterdisparityofaccessto‘resources’(particularlyinmiddle-incomecountries),theproportionofstudentsthatmetminimumproficiencydecreases.Effortstoimproveaccesstovariousresourcesshouldnotmimicor

exacerbatedisparitiesbutaimtonarrowthem.Digitaldividesshouldalsobeminimizedwithteachersandstudentshavingaccesstoarangeoftechnologiesincludingradioandtelevision.

Basedonthefindingoffewerin-servicetrainingopportunities,Grade4students’teachersmayrequireadditionaltrainingorthatthelimitedresourcesbespreadoutmoreequitablybasedoncriticalneeds,especiallywherehybridlearningmodelswillbeused.Morebroadly,improvementtoteacherqualitywouldbeacriticalresponseincludingenhancedstandardsforinitialteachereducation.Italsoincludesmoreandbetter-qualityin-serviceteachertraining.Thisistrueforthosehigh-incomecountrieswherein-servicetrainingislesscommonandforsomemiddle-incomecountrieswhereitisneededtoimproveuponteachers’lowerinitialqualificationsandskills.

Onlinelearningallowsforgreateranonymityincommunicationbetweenstudentsandteachersandstudentsthemselves.Duetothenegativeeffectsofallformsofbullying,betterresponsestocombatit(includingcyber-bullying)areneeded,suchasawareness/sensitizationcampaignsforstudents,trainingforteachers,andmechanismsforhandlingcomplaints.

Basedoncountries’varyingprogressinstudentoutcomes,includingknowledgeofsustainabledevelopmentandaccesstoECCE,governmentexpenditurestoeducationshouldbeincreasedoratleastsustainedtotargetongoingchallenges,particularlyinlightoftheongoingCOVID-19pandemic.

?istock,SDIProductions

4

/metadata/

6Measuringglobaleducationgoals:HowTIMSShelps

SDG4Targets

Target4.1:By2030,ensurethatallgirlsandboyscompletefree,equitableandqualityprimaryandsecondaryeducationleadingtorelevantandeffectivelearningoutcomes.

Globalindicator4.1.1Proportionofchildrenandyoungpeoplein(a)Grade2or3;(b)attheendofprimaryeducation;and(c)attheendoflowersecondaryachievingatleastaminimumproficiencylevelinmathematics,bysex.

Analysis:Percentageoffourth-andeighth-gradestudentsabovetheTIMSSLowInternationalBenchmarkinmathematics.

Target4.2:By2030,ensurethatallgirlsandboyshaveaccesstoqualityearlychildhooddevelopment,careandpre-primaryeducationsothattheyarereadyforprimaryeducation.Globalindicator4.2.2Participationrateinorganizedlearning(oneyearbeforetheofficialprimaryentryage),bysex.

Analysis:Participationinpre-schoolbyyearsofattendance.

Target4.5:By2030,eliminategenderdisparitiesineducationandensureequalaccesstoalllevelsofeducationandvocationaltrainingforthevulnerable,includingpersonswithdisabilities,indigenouspeoplesandchildreninvulnerablesituations.

Globalindicator4.5.1Parityindices(female/male,rural/urban,bottom/topwealthquintileandotherssuchasdisabilitystatus,indigenouspeoplesandconflict-affected,asdatabecomeavailable)foralleducationindicators.

Analysis:Minimumproficiencylevelinmathematicsbygenderandresourcesathome.

Target4.7:By2030,ensurethatalllearnersacquiretheknowledgeandskillsneededtopromotesustainabledevelopment,including,amongothers,througheducationforsustainabledevelopmentandsustainablelifestyles,humanrights,genderequality,promotionofacultureofpeaceandnon-violence,globalcitizenshipandappreciationofculturaldiversityandofculture’scontributiontosustainabledevelopment.

Globalindicator4.7.1Extenttowhich…(ii)educationforsustainabledevelopment…[is]mainstreamedin…(b)curricula.

Thematicindicator4.7.5Percentageofstudentsinlowersecondaryeducationshowingproficiencyinknowledgeofenvironmentalscienceandgeoscience.

Analysis:Percentageoffourth-andeighth-gradestudentsabovetheTIMSSLowInternationalBenchmarkinscienceandEarthScience.

Target4.a:Buildandupgradeeducationfacilitiesthatarechild,disabilityandgendersensitiveandprovidesafe,non-violent,inclusiveandeffectivelearningenvironmentsforall.Globalindicator4.a.1Proportionofschoolsofferingbasicservices,bytypeofservice.

Thematicindicator4.a.2Percentageofstudentsexperiencingbullyinginthelast12months.Analysis:Incidenceofbullying,availabilityofcomputersandaccessibilityofschoolstostudentswithdisabilities.

Target4.c:By2030,substantiallyincreasethesupplyofqualifiedteachers,includingthroughinternationalcooperationforteachertrainingindevelopingcountries,especiallyleastdevelopedcountriesandsmallislanddevelopingStates.

Globalindicator4.c.1Proportionofteacherswiththeminimumrequiredqualifications,byeducationlevel.

Thematicindicator4.c.7Percentageofteacherswhoreceivedin-servicetraininginthelast

12monthsbytypeoftraining.

Analysis:Availabilityofqualifiedteachersandprevalenceofteacherin-servicetraining.

Measuringglobaleducationgoals:HowTIMSShelps7

Target4.1:PrimaryandSecondaryEducation

TheEducation2030Agendaplaceslearningoutcomeindicatorsattheheartoftheinternationaleducationmonitoringframework.Oneofthetwoglobalindicatorsoftarget4.1isthepercentageofstudentswhomeetaminimumproficiencylevelinmathematics(a)inearlygrades,(b)attheendofprimaryeducationand(c)attheendoflowersecondaryeducation.

Grade8studentsachievingminimumproiciencyinmathematics(%)

100

80

100

98

96

94

92

90

9092949698100

60

40

20

0

TIMSSdatainformglobalindicator4.1.1aforcountriesthatadministertheassessmenttofourth-gradestudents,4.1.1bforcountriesthatadministertheassessmenttofifth-gradestudents,and4.1.1cforcountriesthatadministertheassessmenttoeight-andninth-gradestudents.FollowingaprocessmanagedbytheUNESCOInstituteforStatistics5,whichisthecustodianagencyofmostoftheSDG4globalindicators,theTIMSSLowInternationalBenchmarkof400scorepointsisprovisionallyconsideredequivalenttotheminimumproficiencylevel6.

TIMSS2019participatingentitiesreflectedinthefigurehavebeenselectedbyand

namedaccordingtoUNESCO.

JapanSingapore

Rep.ofKorea

RussianFed.

HongKongSAR,China

Ireland

Lithuania

Finland

Portugal

Italy

AustraliaSwedenNorway*

Forinstance,accordingtotheTIMSSLowInternationalBenchmarkdefinition7,fourth-gradestudentsatthatlevelhavesomebasicmathematicalknowledge;canadd,subtract,multiply,anddivideone-andtwo-digitwholenumbers;solvesimplewordproblems;andhavesomeknowledgeofsimplefractionsandcommongeometricshapes.Theycanreadandcompletesimplebargraphsandtables;arefamiliarwithnumbersintothethousands;canorder,add,andsubtractwholenumbers;havesomeknowledgeofmultiplicationanddivisioninvolvingtwo-digitnumbers;cansolveone-stepwordproblemsandnumbersentences;andcanrecognizepictorialrepresentationsofsimplefractions.Finally,theycanrecognizebasicmeasurementideas;recognizeandvisualizecommontwo-andthree-dimensionalgeometricshapes;andreadandcompletesimplebargraphsandtables.

FIGURE4.1.1:Percentageoffourth-andeighth-gradestudentswhoachievedminimumproficiencyinmathematics,2019

France

Kazakhstan

NewZealando

OTurkey*U.A.EmiratesooGeorgia

Iran,Isl.Rep.ChileQatar

Oman

SaudiArabia

SouthAfrica*Morocco

020406080100

Grade4studentsachievingminimumproiciencyinmathematics(%)

Notes:Thefigureincludesonlycountriesthatparticipatedinbothfourth-andeighth-gradeassessmentsin2019.ThescalesforGrade4andGrade8arenotequivalent.*Norway,SouthAfricaandTurkeyadministeredthefourth-gradeassessmenttofifth-gradestudents,NorwayandSouthAfricaadministeredtheeighth-gradeassessmenttoninth-gradestudents.

5

/metadata/

6TakingtheminimumproficiencyasdefinedbyUNESCOInstituteforStatisticsonpage16of

/sites/default/files/

documents/sdg4-databook-global-ed-indicators-2019-en.pdf

,whichmatchTIMSS2015statisticsonperformanceattheTIMSSLowInternationalBenchmarkfoundhere:

/timss-2015/mathematics/performance-at-international-benchmarks/

percentages-reaching-international-benchmarks-across-assessment-years/

7ForfullbenchmarkingdefinitionsseetheTIMSS2019InternationalResultsinMathematicsandScience:

/

timss2019/international-results/

8Measuringglobaleducationgoals:HowTIMSShelps

AmongparticipatingcountriesandterritoriesinTIMSS2019,86%ofGrade4studentsand80%ofGrade8studentsachievedminimumproficiencyinmathematics.Whilethevastmajorityofstudentsinhigh-incomecountriesachievethislevel,farfewerdosoinmiddle-income,andespeciallylower-middle-incomecountries.Forexample,inMorocco,43%offourth-gradeand41%ofeighth-gradestudentsachievetheLowInternationalBenchmark(Figure4.1.1).

TIMSShasbeenimplementedforaquarterofacentury.While10,mostlyhigh-income,countrieshavetakenpartineachofthesevenroundssince1995,manymorehavetakenpartinatleasttworounds.Valuableinformationcanthereforebeextractedtoassesscountries’potentialtoensurethatallstudentsachieveminimumproficiencylevelby2030.Among

countriesthattookpartinthe2015and2019rounds,theaveragegrowthinthepercentageofstudentsthatachievedminimumproficiencyinmathematicswas0.3percentagepointsatfourthgradeand0.5percentagepointsateighthgrade,inannualterms.

SomecountriesareoncoursetoachievetheSDG4targetby2030andseveralcountrieshaveexperiencedrobustgrowth(Figure4.1.2).Forinstance,inChilethepercentageofeighth-gradestudentsthatachievedminimumproficiencyinmathematicshasgrownfrom41%in2003,to57%in2011and70%in2019.Ifthecountrywereabletomaintainthesamegrowthrate,itwouldbeoncoursetoachievetheSDG4targetby2030.However,othercountrieshavestagnated.Forinstance,79%ofRomanianstudentsachievedminimumproficiencyin1995and78%in2019(Figure4.1.2).

FIGURE4.1.2:Percentageofeighth-gradestudentswhoachievedminimumproficiencyinmathematics,selectedcountries,

1995–2019

100

Grade8studentsachievingminimumproiciencyinmathematics(%)

90

80

70

60

50

40

30

UnitedStatesKazakhstan

NewZealand

Romania

U.A.Emirates

LebanonJordan

SaudiArabia

Morocco

←2030 SDG4deadline

CountriesmakingfastprogressCountriesmakingslowornoprogress

Lithuania

Turkey

GeorgiaChile

Iran,Isl.Rep.Qatar

Oman

←2030 SDG4deadline

199520032011201920272035199520032011201920272035

TIMSS2019participatingentitiesreflectedinthefigureabovehavebeenselectedbyandnamedaccordingtoUNESCO.

Note:ThefigureincludesselectedcountriesthathaveparticipatedinmorethanoneGrade8TIMSSroundsince1995.Percentagesareestimates;standarderrorsnotdisplayed8.

8Fordetailedtrendresults,pleaseviewtheTIMSS2019InternationalResultsinMathematicsandScience:

/

timss2019/international-results/

Measuringglobaleducationgoals:HowTIMSShelps9

Target4.2:EarlyChildhood

OneoftheEducation2030agendagoalsistoensurethatallchildrenhaveaccesstoqualityearlychildhooddevelopment,careandpre-primaryeducationsothattheyarereadyforprimaryeducation.TIMSS2019continuedtodemonstratethepositiverelationshipbetweenearlychildhooddevelopmentofskillsandhigheraverageachievement9.WhileTIMSSisaprimaryandlowersecondaryeducationassessment,parentsoffourth-gradestudentsareaskedtoreporthowmanyyearsofearlychildhoodeducationtheirchildhadattended(includingearlychildhoodandpre-primary)10.Thisquestionoffersusefulinsightsonthedistributionofexposuretoearlychildhoodeducation,whichcomplementinformationavailablefromadministrativedata,evenifitisavailablewithafour-yearlag.

Inthemajorityofcountries,fourth-gradestudents

participatedfor