Lessons from education experiments

280Jaargang 101 (4732) 14 april 2016
Education policy in progress
I
n the Netherlands, there has in recent years
been a reversal in the way educational re-
forms are viewed, so that now there is an
important place reserved for experimental re-
search. The idea behind experimental research
is to study its effects by comparing a group of
students who have experienced such an ‘inter-
vention’ (experimental group) with a group of
students who have not (control group). In that
way, one hopes to prevent drawing board-devised education innova-
tions from turning out a failure in the classroom. See Borghans et al.
(2015) for a description of this development.
Good education is becoming more and more important. But because
there are fewer opportunities to provide pupils with additional and
longer tuition, it is becoming increasingly urgent to use the educa-
tion years of pupils effectively. Besides, the costs of education re-
search – especially since the advent of ICT – are steadily going down.
Many test results of school kids are already in the computer systems
and can therefore be very easily interconnected. Thus, without much
effort, one is able to observe the effects of experiments over longer
periods of time. Due to this, there is an ongoing upsurge as to the
value of education experiments.
Also on an international level, experimental education research has
grown considerably in recent decades. In the nineties, economists in
academic circles became aware that research into the effects of cer-
tain types of education was likely to seriously distort its outcomes if

it was based on a comparison between people who made different
choices. For example, if less bright students receive more hours of
language teaching, one might get the impression – should no experi-
ment be carried out – that additional language tuition does not lead
to better results. To rule out such selection ef-
fects, studies were carried out using natural
diversification or randomly but sharply defined
policy limits in education. A classic example is
the study into the effects of class size by An-
grist and Lavy (1999), in which one has used
the strict and unequivocal rules as regards the
allowed size of classes. However, the problem
with natural and quasi-experiments has been
that important educational questions remained unanswered be-
cause there was no natural experiment available. In order to answer
these questions, field experiments were arranged.
From these education experiments, interesting findings emerged.
First, it is very important to have good data on education. For ins-
tance, in an experiment in an elementary school you want to tell how
pupils are going to do in secondary education and after. If one only
monitors the pupils in the one experiment, then experimentation be-
comes very costly. Also, in that case there are often a considerable
number of drop-outs. That is why developments making it easier to
gather the data of all students in the Netherlands for research, may
truly stimulate experimentation.
Second, the success of an experiment requires a good collaboration
between researchers and those involved in educational practice.
Ideas that look good in theory are often hard to realise in practice.
It is challenging for researchers to develop creative solutions to such

problems of everyday practice.
Fionnuala O’reilly
Advisor at the UK Behavioural Insights Team
“The Somerset Challenge” comprised a suite
of randomised controlled trials by the Beha-
vioural Insights Team in the UK involving
both teachers and students. One trial tested
different methods to raise the aspirations of
youths who opt against attending university
despite having the grades to gain admission.
Students were randomly allocated to a con-
trol group who did not do anything, or to
one of three intervention groups. In the first
group, students received information on fi-
nancial support, loan repayments and future
earnings. In the second group, parents were
given the same information. In the third group, students received
a talk from a relatable
role model –someone
who had grown up in So-
merset and had attended university. The
results indicate that giving students a short
talk from a relatable role model increases
their stated interest in attending university
(7.8 percentage points higher than the con-
trol group), as well as their likelihood of ap-
plying (8.4 percentage points higher).
In a separate trial, we tested the efficacy of
sending different types of messages to re-
cruit teachers to rural schools in Somerset.
Teachers were randomly assigned to receive
a pro-social message emphasising the social
impact of teaching, or to receive a challenge
message which acknowledged that teaching
can at times be difficult. We found that tea-
chers who received the challenge message
had 2.58% more click-throughs to a vacancy
website than those who received the pro-
social message.
The above are just two examples of how be-
havioural insights can aid in making current
practices more effective, but the Behavioural
Insights Team believes there is ample oppor-
tunity to use behavioural insights to sculpt
education policy more broadly to the benefit
of students, parents and teachers alike.
Lessons from education experiments
Case 1: The Somerset Challenge
LEX BORGHANS
Professor at Maastricht University
TRUDIE SCHILS
Researcher at Maastricht University
INGE DE WOLF
Professor at Maastricht University
Interested
Control
Proportion in percent
KidsParentsTalk
Likely
60
40 20
0
Significance: the talk group is different from the control group at the
five percent confidence level.
48 44 48 56
3934 3747
ESB visited a conference on evidence informed policy of the European Com
mission and the Dutch Ministery of Education on April 6 and 7

281Jaargang 101 (4732) 14 april 2016
Education policy in progress
DENNIS VAN GESSEL
Policy advisor at the Dutch Ministery of Education
From 2005 onwards, the government has
been dealing with youngsters prematurely
leaving school by way of the prevention pro-
gramme ‘Attack upon early school leaving’
(Aanval op de schooluitval). Dropping out is
not just a problem for young people them-
selves, but also has undesirable economic
and social consequences for society. The
programme’s aim is to reduce the number of
drop-outs. This has been a success: the num-
ber has gone down from almost 60.000 a year
to 35.000 in 2012, and to 25.000 in 2015.
The considerable decline in the number of
drop-outs is mainly due to the new approach.
An important part of the latter is to make the
existing data more accessible and to use it in
a better way. As brief spells of absenteeism
turn out to be an important predictor of even-
tual dropping out of school, the digital absen-
teeism portal that provided realtime insights into pupils’ absen-
tee behaviour hel-
ped guide parents,
schools and munici-
palities in directing
efforts. Because of
this information fa-
cility, one is able to act upon absenteeism
faster, so that youngsters can be kept aboard.
Furthermore, the government has closed per-
formance contracts with schools and munici-
palities. As part of these contracts, schools
receive additional resources if they achieve
certain objectives. These performance con-
tracts, among other things, have resulted in
professionals from a region frequently mee-
ting to tackle premature school leavings. The
municipality serving as a regional contact
takes on the part of ‘director’ here, and by
way of a regional problem analysis each re-
gion decides by itself what measures it takes.
The Ministry provides support in the form
of account managers frequently touring the
country to share best practices and transpa-
rent data products.
Hessel Oosterbeek has played a major role in raising awareness
as to the importance of experimental educational research. Since
2005, the CPB Netherlands (CPB) also has laid greater empha-
sis on experimentation. At CPB, especially Dinand Webbink has
propagated using both quasi- and natural experimental research
methods. Apart from their own impact studies, the CPB is incre-
asingly furthering what they call a ‘promising education policy’
(
kansrijk onderwijsbeleid), which summarizes what Dutch and inter-
national quasi-experimental research has shown to be both effec-
tive and efficient.
Partly because of these developments also the Ministry of Educa-
tion has more and more started to ponder the importance of expe-
rimentation. In 2009, the vast research programme ‘EducationEvi-
dence’ (OnderwijsBewijs) arrived. In 2010, the Ministry initiated the
project ‘Insight into effectiveness’ (Zicht op effectiviteit), while the
Top Institute for Evidence Based Education Research (TIER) was
set up with subsidies from the Ministry with the aim to generate
evidence-based knowledge about education. In 2011, the ‘Innova-
tion Impulse Education’ programme (InnovatieImpuls Onderwijs;
IIO) set out, in 2012, the National Directive Body for Education Re-
search (NRO) was established, and education councils agreed to
give schools more freedom to decide for methods that countered
bullying and were proven to be effective by experiments. In order to advance education, the right experiments must be carried
out. Now, the choice of experiments is largely determined by costs
and practical feasibility. Actually, lacking here is the research as well
as the research methodology that may establish what experiments
might provide the information critical to improving education. With
this knowledge, one would be able to make experimentation more
targeted. And thus one would sooner discover how education can be
improved.

LITERATURE
Angrist, J.D. and V. Lavy (1999) Using Maimonides’ rule to estimate the effect of
class size on scholastic achievement. The Quarterly Journal of Economics, 114(2),
533–575.
Borghans, L., T. Schils and I. de Wolf (2015)
Experimentalism in Dutch education
policy: experiences and lessons learned . Maastricht: Maastricht University.
Cornet, M., F. Huizinga, B. Minne and D. Webbink (2006) Kansrijk kennisbeleid.
CPB Document, 124.
Case 2: Attack on school
dropouts
Experiments and policy in the Neterlands
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0’0 1-
’02 ’0
2-
’03 ’0
3-
’04 ’0
4-
’05 ’0
5-
’06 ’0
6-
’07 ’0
7-
’08 ’0
8-
’09 ’0
9-
’10 ’10-
’11 ’11-
’12 ’12-
’13’13-
’14 ’14-
’15
Target
In thousandsRe
alis a tion
Early school leavers
Vocational education students receive information at the jobparty Almere
On Stage. Aim of the
event is to prevent early school leaving.
Photo: Goos van der Veen, Hollandse Hoogte