Supporting the right workplace experience: a dynamic evaluation of three activation programmes for young job seekers in Slovakia

Abstract

This paper investigates three alternative active labour market policy programmes available to young job seekers in Slovakia who were registered in 2011. All of the programmes facilitate gaining work experience and share a comparable design; however, they differ mainly in the collected workplace experience and the composition of participants. Using administrative data, we first explore the selection into each programme; second, we estimate the treatment effects on job seekers’ postparticipation absence from the registry of the unemployed. For this we argue that we have sufficiently rich data to control for selection into programs. We use a dynamic estimator and report the average treatment effects of participation in different periods between the sixth and fifteenth month after starting unemployment. For participation in earlier stages of unemployment, we confirm differences in the programmes impacts, with patterns described by previous literature; workplace experience collected in either the private sector or in a regular workplace appears to improve the chances of absence from registered unemployment of young job seekers relatively more than does participation in a public works type of programme. When compared later in the unemployment spell, the between-programme differences level out. Despite its ambivalent average impact, the public works type of programme positively impacts specific subgroups of participants.

1 Introduction

Early career unemployment appears to have a lasting, scarring effect that worsens labour market prospects in terms of future employment (Schmillen and Umkehrer 2017) or income (De Fraja et al. 2021). The economic cycle or exogenous economic shocks, such as the COVID-19 pandemic, could therefore generate lifetime damage to the cohorts that graduate during the affected periods (Bell and Blanchflower 2010; O’Higgins 2012; Grotti et al. 2018). The absence of work experience penalises fresh graduates with no labour market history (Topel and Ward 1992). Moreover, potentially longer unemployment periods are linked to an additional disadvantage in the hiring process (Kroft et al. 2013). The precariousness of this situation justifies publicly funded interventions channelled predominantly through active labour market policies.

Comparing the active labour market policy (ALMP) programmes in terms of their impacts on postparticipation absence from registered unemployment, we aim to extract internationally relevant, policy-design information on what works in regard to activating unemployed youth. In our case, each of the three evaluated programmes not only accommodates a different type of participants but also serves a different purpose in the portfolio of Slovak ALMPs. One is provided in the earlier stage of unemployment and is conditioned on sufficient resources; the other is provided later in the unemployment under an obligatory provision. Therefore, we do not estimate the treatment effects for the composition of participants commonly present in all the three evaluated programmes (e.g. as in Sianesi (2008) or Biewen et al. (2014)); instead, we estimate the average treatment effects on the treated (ATETs), acknowledging the programme-specific composition of participants. Taking advantage of a rich dataset, we identify the ATETs under the unconfoundedness assumption, which is critical for the causal interpretation of our results. In considering the programme-specific timing of participation, we rely on a recently developed dynamic estimator which uses never-treated jobseekers as a comparison unit, accounting for the fact that not-yet-treated jobseekers may enter the programme later on. We also explore heterogeneity in our ATETs.

The programmes focused on herein are all aimed at activation by providing workplace experience to the unemployed. They are comparable in terms of the type and extent of the support received. The main difference lies in the type of workplace into which job seekers (JSs) are placed, accompanied by a pattern of selection into the programmes that drives major between-programme differences in the composition of the participants. Workplace experience gained in public clerical jobs or private firms supported by the Graduate Practice (GP) programme enhances the future labour market prospects of participants. A more pronounced positive impact is estimated for the Voluntary Activation Works (VAW) programme, which fosters workplace insertions in the nonprofit and community service sectors. In contrast, participation in an alternative programme called Activation Works (AW), which includes municipality-organised community services, has an ambiguous impact on postparticipation absence from registered unemployment. The differences in the impacts of the three programmes, combined with the selection into the programmes, add to the damaging effects of early-career unemployment, which results in worsened employment prospects for predominantly low-skilled participants selected for AW in the earlier stages of their unemployment. Comparing the impact of participation taking place after twelve months of unemployment, the differences between programmes level out.

Our contribution adds to the existing evidence on the impact of youth-oriented ALMPs by documenting a case study with policy-design relevance from the underexplored region of central and eastern Europe. We build our analysis on a comparison of the impacts of three ALMPs offered to young JSs during 2011. Comparing these alternative programmes that subsidise the collection of workplace experience allows a generalisation of policy-relevant experience. To produce a reliable comparison of the impact of the three ALMPs, which are implemented at different stages of the unemployment period, we adopt a dynamic impact evaluation scheme introduced by Vikström (2017). Our implementation of this method¹ that uses Slovak administrative data adds to the list of the few recent impact evaluations that use a dynamic impact evaluation framework (see Lombardi et al. (2019) and Albanese et al. (2020)).

The remainder of this paper is structured as follows. A brief overview of the literature on youth unemployment and related policy responses is provided in the next section. We outline our identification strategy in the second section. A description of youth unemployment trends in Slovakia and the measures under evaluation can be found in the third section. The data with the evaluation sample are introduced in the fourth section. Our main results are described in the fifth section. We conclude by discussing our main findings in the final sixth section.

2 Policies tackling youth unemployment

The transition from school to the labour market plays a crucial role in shaping individuals’ careers (Crosnoe and Elder 2002). In particular, long-term unemployment in the very early career stages appears to have a scarring effect on the subsequent career path (Ellwood 1982; Schmillen and Umkehrer 2017; De Fraja et al. 2021). In developed countries, the effect of early-career unemployment on future income seems to be more pronounced than the effect on future employment (Ellwood 1982; Gregg and Tominey 2005; Goldsmith et al. 1997). A negative employment effect is nonetheless observable, especially in the case of low-skilled individuals (Burgess et al. 2003). In a recent empirical study, Schmillen and Umkehrer (2017) estimated a direct effect of early-career unemployment on the chances of being unemployed in prime age. Furthermore, they claimed that the damaging effect of youth unemployment is greater for those who have longer unemployment experience. De Fraja et al. (2021) showed that unemployment of up to 2 years after graduation is linked with a greater damaging effect in terms of prime-age income.² In addition to the impact on labour market outcomes, youth unemployment has immediate negative implications at the social and individual levels, such as increased crime rates, the obsolescence of recently acquired education, and greater pressures on social policy budgets, among other outcomes (Bell and Blanchflower 2010).

The negative effects of youth unemployment have motivated a massive policy response. In terms of extent, the EU-wide Youth Guarantee represents an unprecedented initiative (Escudero and López 2017). ALMPs are an important channel through which support is funnelled under this initiative (Eichhorst and Rinne 2018). Nevertheless, the evidence on their impact remains ambiguous (Caliendo and Schmidl 2016; Eichhorst and Rinne 2018). At the programme level, successful examples can be identified, but systematic overviews have suggested that success certainly does not predominate (Card et al. 2010). In a follow-up meta-analysis, Card et al. (2018) claimed that ALMPs exerted a low impact on the labour market outcomes of both younger and older participants.

Caliendo and Schmidl (2016) conducted a meta-analysis of 37 impact evaluations of ALMPs that specifically target unemployed youth or the reported estimates for this subgroup. Summarising the evidence, which is predominantly from western European countries and Scandinavia, they also concluded that the available evidence is ambiguous. They classified ALMPs into four groups and pointed to zero or negative effects of public sector work programmes, a heterogeneous impact of labour market training programmes and a predominantly positive impact of job search assistance and monitoring and wage subsidies. The labour market training programmes that focus on workplace-based learning have shown more promising results. However, applying typologies at the programme level does not generate informative comparisons. Kluve et al., in a more recent meta-analysis on the impact of youth-oriented measures, concluded that rather than identifying successful programme types, the available evidence allows the identification of successful elements of programme designs (Kluve et al. 2019).

2.1 Youth-oriented ALMPs

Based on a meta-analysis of 113 impact evaluation studies of youth-oriented ALMPs, Kluve et al. (2019) concluded that the intervention type is less important than its particular design and delivery. Successful programmes that target unemployed youth usually rely on providing work experience to recent graduates (Caliendo et al. 2011; Ehlert et al. 2012; Caliendo and Schmidl 2016). Compensating for the lack of work experience, therefore, appears to be a feasible activation strategy. Caliendo et al. (2011) evaluated seven different German ALMP programmes that target youth. They reported positive employment effects for programmes that support job searches and short-term and further training measures. Strong, statistically significant, positive effects of employment on participants were estimated for measures that subsidise employment (acquisition of work experience) combined with a skills-upgrading component. Supporting employment in areas of public interest did not yield employment effects statistically significantly different from zero.

In practice, mixed programme designs appear to yield more favourable results (Kluve et al. 2019). Mixed designs based on supporting the acquisition of workplace experience combined with skills upgrading/training are quite widespread in youth-targeted ALMP programmes.³ The training components range from classroom to on-the-job training, with various levels of formalisation. A mixed approach, which provides classroom training together with work experience, has shown a positive impact on postprogramme employment probability in Germany (Ehlert et al. 2012) and the United Kingdom (Dorsett 2006). Holtmann et al. (2021) addressed the question of the trade-off between formal education and workplace experience and showed that unemployed dropouts from formal education benefitted from second-chance formal (prevocational) education, while it was only graduates with a school certificate whose employment chances were improved by programmes that foster workplace experience. Auray and Lepage-Saucier (2021) defined atypical work as less-desirable, temporary or part-time work and concluded that starting atypical work during unemployment significantly increases the chances of finding regular employment. Additionally, the workplace experience provided under the ALMP programmes evaluated here is not sheltered by a standard employment contract; it consists of only 20 workplace hours per week, is associated with “pocket money” remuneration, and the participant lacks the legal status of an employee. We aim to add to the existing research on ALMP-provided workplace experience by generating information about the differences in the employment impact linked to the different types of workplaces.

2.2 Evidence on the impact of the youth-oriented ALMPs targeting youth in central and eastern Europe

A predominant share of the available evidence on the impact of ALMPs has been collected in Western European countries. Evidence from Germany, Scandinavia, and Switzerland is overrepresented in meta-analyses of ALMP-focused impact evaluation studies (see, e.g., Card et al. (2010); Caliendo and Schmidl (2016) and Card et al. (2018)). The countries of Southern Europe are slightly lagging, and the countries of Central and Eastern Europe (CEE) are barely represented at all. In a more recent meta-analysis by Kluve et al. (2019), evidence for CEE countries was completely absent. The context of ALMP provision in CEE countries differs from that in Western Europe, Scandinavia and Southern Europe. For example,

• In CEE countries, unemployment did not exist before 1990; thus, all of the labour market policies were created during the 1990s by mostly copying successful examples from the rest of Europe;

• The demographic pattern of CEE is specific, with extensive cohorts born in the 1980s turning into extremely small cohort sizes born in the 1990s; and

• The Roma population comprises a larger share of the unemployed in CEE countries than in other European countries, and this subpopulation is often more segregated from the majority.

Although available evidence is scarce, valuable instances can be found. One randomised controlled trial was organised to evaluate the impact of monitoring the job search effort of JSs registered with the Hungarian public employment service (Micklewright and Nagy 2010). Otherwise, impact evaluations that rely on observational data have predominated among the studies from the CEE region. Of the studies that identify the impacts of policies on youth labour market outcomes, Horn (2016) evaluated the impact of secondary school programmes with apprenticeship training on the employment chances of youths in Hungary. Csillag and Scharle (2020) evaluated the success of the Hungarian public employment service in increasing the registration rates of youth not in education, employment or training. Krekó et al. (2023) estimated a positive employment effect of a job trial subsidy, which is a measure comparable to GP, stressing the importance of its screening function.

In the Czech Republic, Hora and Sirovátka (2020) used a quasi-experimental design to evaluate a programme comparable to the GP initiative analysed here. They identified a positive impact on unemployment duration that is more pronounced over the long term and for medium-skilled young JSs. Evidence from Poland also confirms a negative or statistically nonsignificant employment effect of public works-type programmes in general (Wišniewski and Maksim 2012) and specifically for young unemployed (Madoń et al. 2021). The positive employment effects of the Slovak GP programme were estimated by Štefánik et al. (2020) and Svabova and Kramarova (2021). An older study by Lubyová and Van Ours (1999) estimated negative effects of the predecessor of the Slovak Activation Works programme. The findings presented here aim to complement this evidence by comparing the employment effects of three programmes that facilitate workplace insertion for young JSs in Slovakia.

3 The context of Slovak ALMPs

Between 2000 and 2006, Slovakia experienced one of the highest unemployment rates in the EU. This experience can be explained by an underperforming production sector combined with relatively stronger inflows of young-age cohorts into the labour force due to demographic changes. During a short period that preceded the economic crisis in 2008, the GDP and employment growth in Slovakia were among the highest in the EU member states. Slovakia’s labour market reaction to the economic crisis was one of the most severe in the EU. The period ranging between 2011 and 2019, which is covered by our sample, was characterised by steady growth in total employment, accompanied by a reduction in the unemployment rate to levels less than the EU average. Although disrupted during the COVID-19 pandemic, the country seems to have resumed this trajectory again in 2021.

Despite its turbulent labour market situation, Slovakia’s ALMP spending (as a share of GDP or per registered JS) remains one of the lowest in the EU. Herein, we mainly focus on 2011 because it is a year during which the three ALMP programmes were implemented under comparable settings. With a high number of JSs and limited resources, Slovak employment policies focus on programmes with low costs per participant, with the aim of maximising the availability of activation measures. The unemployment benefit covers fifty percent of the preunemployment working income; it is paid monthly during the first six months of unemployment to those unemployed individuals who worked for at least two out of the last three years. Alternatively, the social assistance benefit is means-tested, but its monthly payments are not time-limited. Contributions associated with ALMP programmes are paid on top of the unemployment or social insurance benefit.

3.1 Description of the measures under evaluation

In 2011, only one ALMP programme was specifically designed to target young JSs registered with the Slovak public employment agency, namely, the Central Office of Labour, Social Affairs and Family of the Slovak Republic (COLSAF); this was the Graduate Practice programme (see Table 1). A total of 6.55 percent of youths under the age of 26 who were included in the unemployment registry in 2011 were participants in this programme (see Table 2). Out of the ALMP programmes with no age restrictions, the AW programme had the highest portion of youth participants at this time, with 1.33 percent of the youths included in the unemployment registry during 2011 participating in this programme. In addition, 0.56 percent of youths included in the registry of the unemployed participated in Voluntary Activation Works, a subprogramme offered under the framework of Activation Works. During the evaluation period of 2011, the ALMP options available to young JSs after they registered with COLSAF were limited to the three programmes evaluated herein. Since 2014, more ALMP programmes that target unemployed youth have been introduced under the EU-wide Youth Guarantee initiative. Most of these programmes combine supported employment with a training component. One of the programmes aims to assist youths in the transition from unemployment to self-employment.

Table 1 Overview of the main features of the evaluated programmes

The Graduate Practice programme⁴ covers the workplace insertion of registered JSs only if they are younger than 26 years of age, regardless of their previous work experience or the time elapsed since their graduation. No conditions concerning the length of previous unemployment are applied. Participants spend up to 20 h weekly at one employer for a period of 3 to 6 months. During 2011, participants received approximately 190 euros monthly, and they were paid by COLSAF based on two contracts established between COLSAF and the JSs and between COLSAF and the corresponding employers. In 2011, GP was the most accessible ALMP programme, enrolling almost one in five registered JSs under the age of 26. Previous impact evaluations have pointed to a small (but statistically significant), positive impact on the employment of participants (Institute of Fiscal Policy 2016; Štefánik et al. 2014a; Svabova and Kramarova 2021), accompanied by a negative income effect (Štefánik et al. 2020). Workplace insertions sheltered under this programme mostly occur in clerical jobs in public institutions (associated with negative income effect), but there have also been insertions in private firms included in the programme. Therefore, the job performed by the participants at the organisation into which they were inserted is similar to a regular job.

The Activation Works programme⁵ aims to provide work experience and contact with the workplace mainly to the long-term unemployed⁶ JSs from all age groups. Participants work directly for municipalities and deliver 20 h of weekly labour for a maximum period of 6 months. The financial remuneration received is comparable to that received under the GP programme.

Originally, the programme was designed as a passive labour market policy, with eligibility linked to the means-tested minimum subsistence allowance and status of a long-term unemployed. Later, a parallel system for distributing this support evolved in which these eligibility criteria were not applied. In 2011, the parallel system was already in operation (Mytna Kurekova et al. 2013). In cases when the participant received the minimum subsistence allowance, the AW was added to the allowance; in other cases, the AW contribution was paid directly by the municipality.

Municipalities are responsible for organising the workplace insertion. Therefore, the organisation of public work differs dramatically from work under the GP programme because of the nature of the duties performed and because municipalities efforts to utilise this source of labour differ from the efforts made by regular employers. Some municipalities organise AW labour into community work projects, while others use AW participants in municipal firms or administration. In some of the municipalities, AW participation in the first 12 months of unemployment was possible; in others, it became available and claimable only after 12 months of unemployment. Available evaluation studies have pointed to a stigmatising effect of participation in AW (Institute of Ethnology 2009; Mytna Kurekova et al. 2013), accompanied by a negative impact on postparticipation employment (Štefánik et al. 2014b; Hidas et al. 2016). Dependence on the social security scheme itself is often linked to lower employment outcomes and longer unemployment periods (Guzi 2014). A higher share of AW participants are Roma, which is an ethnic group that faces multilayered discrimination not only in access to public policies (Mikula and Montag 2022).

The Voluntary Activation Works programme⁷ is a variation of AW for individuals in registered unemployment regardless of the length of their registered unemployment period or their eligibility for minimum subsistence benefits. Combined participation in AW and VAW is not allowed. VAW participants perform 20 h of labour weekly for six months for an employer, which must be a nonprofit organisation. The financial remuneration received is the same as that in GP (in 2011, approximately 190 euros per month). The main difference between AW and VAW is in the organisation of workplace insertion. VAW offers insertion into a regular job within a nonprofit organisation, whereas AW insertion is, with only a few exceptions, limited to occasional cleaning tasks for municipalities or other community services.

In 2011, all three evaluated ALMPs (GP, AW and VAW) had already been in effect since 2004. Additionally, all three of them share a common design, with a workplace insertion component in place to compensate for the lack of workplace experience among young unemployed JSs. All three programmes are also comparable in terms of the intensity of support (20 h per week), the duration (maximum of 6 months) of workplace insertion, and the remuneration related to participation (approximately 190 euros per month). These programmes took some form of a direct financial contribution provided by COLSAF directly to the participant, without a standard employment contract existing between the participant and the employer or an obligation for the employer to employ the participant after receiving the support. The main difference lies in the type of workplace where JSs are inserted. In the case of the GP and VAW programmes, participants are inserted into a regular workplace (GP) or a nonprofit organisation (VAW), while in the case of AW, participants are asked to participate mostly in community service organised by municipalities. The organisation of AW varies widely across municipalities (Mytna Kurekova et al. 2013)), which drives the high heterogeneity in the estimated impact of AW participation on employment (Štefánik et al. 2014b).

3.2 Mechanisms behind the selection of programme participants

The three programmes also differ in the composition of the participants, with the GP and VAW initiatives focusing more (but not exclusively) on registered JSs with higher education (ISCED 3 or higher). The timing also differs, with GP and VAW participation offered at earlier stages of the unemployment period (see Fig. 1). AW participation mostly starts after 1 year of registered unemployment. Differences in the timing of participation also constrained the periods for which we were able to successfully identify the effect of participation. Namely, we explore the third, fourth and fifth quarter of unemployment, for which our data provide a sufficient overlap of participants observations in all three compared ALMP programmes.

Fig. 1

Density of the start times of ALMP participation based on the unemployment duration Note: Lines refer to the estimated kernel relative density of participants based on the time that elapses between the start of unemployment and entrance into the programme. The area display the histogram of the relative density of all unemployment spells of JSs younger than 26 years old and registered during 2011 by the final duration of the unemployment period in days. Source: COLSAF database

All three programmes have a form of a financial payment (contribution) for which the potential participant and the “employer” need to jointly submit an application to the regional COLSAF office. COLSAF caseworkers assist in submitting the application and distribute the information about available support to the JSs. In the case of GP, potential employers are often proposed with which the practice could take place. These referrals might partially explain the bias in the composition of GP participants towards the more easily employable and better educated JSs. Additionally, self-selection might also play a role, with the more educated and skilled JSs being more likely to find an employer inclined to cosubmit their GP application. In the case of VAW, applications are cosubmitted with a nonprofit organisation with which VAW participation should take place. VAW is widely utilised and supported by the nonprofit sector. As a result, potential VAW participants are outreached beyond the usual services provided by COLSAF. Caseworkers from nonprofit organisations recruit potential VAW participants from disadvantaged subgroups or neighbourhoods. Finally, AW is a programme for those who have been unemployed for longer, which biases the composition of participants towards being less employable. AW applications are submitted jointly by the potential participant and the municipality.

Although applications are submitted jointly by the JS and the receiving organisation (employer/nonprofit organisation/municipality), the roles of COLSAF caseworkers and institutional factors (e.g., availability of funding) in selecting among the programmes are instrumental. Caseworkers are responsible for distributing the information and assisting through the application process. They steer JSs towards particular programmes based on the policy applied by the regionally managed COLSAF office and the momentary availability of resources.

AW and VAW are universally available to all eligible applicants, and COLSAF is obligated to provide positions upon receipt of a complete application. The provision of positions to a GP applicant is subject to the decision of regional employment councils and may be affected by the availability of resources. In our empirical analysis, we control for a long list of regional variables, including the regional office of registration. Nevertheless, the obligatory provision of AW and VAW might add potential unobserved factors into the mechanism of selection of participants. While in the case of AW and VAW, these factors might be driven rather by the arbitrariness of participants, in the case of GP, they are more likely driven by the arbitrariness of the caseworker. Since all applications assume some activity from the potential participants, non-take-up is not significant and is not penalised. De-registration from the unemployment register is only possible if a JS refuses a job offer or is not cooperative for a sufficient period of time. One motivation to register is the insurance-based unemployment benefit. For persons not eligible for the insurance-based unemployment benefit, the motivation to seek registration also relates to the publicly covered health insurance individuals who are not registered and not employed are obliged to pay their health insurance contributions monthly.

4 Identification strategy

We identify the average treatment effects on the treated (ATETs) under the unconfoundedness assumption. The advantage of our strategy lies mainly in allowing for an interpretation of the acquired ATETs with respect to the differences in the composition of participants and the mechanisms behind the selection into the programmes. We find this advantage particularly relevant in our context, with a heterogenous mechanism of selection into the programme resulting in a heterogenous composition of programme participants. Additionally, to compare the three programmes for which participants are recruited during different periods of the unemployment period, we must account for the chances of exiting registered unemployment varying over the unemployment period. This problem is often addressed in impact evaluations of ALMPs by adopting a so-called dynamic evaluation framework (Fredriksson and Johansson 2008; Sianesi 2008). In our identification strategy, we implement the dynamic estimator based on inverse probability weighting (IPW) introduced by Vikström (2017). By implementing this technique, we add to a rather short list of recent studies. Albanese et al. (2020) implemented the same method in evaluating an early retirement scheme in Belgium that enables work time reductions during the years preceding retirement. This approach was also used in a comparison of dynamic estimators (Lombardi et al. 2019; Thomas et al. 2020) and in the context of job-search assistance (Muller et al. 2017). The method was further elaborated in (Van den Berg and Vikström 2022) in an application on the income effect of a training programme.

4.1 Dynamic evaluation framework

A static evaluation framework refers to situations when a treatment is exogenously offered to a JS at a particular time. Such situations, however, do not capture typical ALMPs. The static evaluation framework would not be appropriate in our context because participants are dynamically selected into the treatment. More concretely, a decision to participate in a programme depends on the length of a candidate’s previous unemployment and, more importantly, expected future unemployment. JSs who are unemployed for longer have a different probability not only of entering one of the programmes but also of leaving the register of the unemployed. JSs who are early in their unemployment periods are less likely to participate in certain programmes; this could be partly their decision or because case workers are reluctant to assign/recommend a programme early as the JSs would enter employment anyway. This condition suggests that it is important to control for the length of the unemployment spell before the start of the programme and possibly allow for treatment effect heterogeneity. Knowing that it is important to account for the length of the previous unemployment period, it is challenging to construct a meaningful comparison group for the observed JSs treated at a particular time. This is because not-yet-treated individuals may be treated later. Consider the following example: if we naively compare the residual unemployment of a JS who entered the programme 3 months after the unemployment start with unemployed JSs who did not enter the programme within 3 months after becoming unemployed, then the difference in the outcomes will be partly driven by the fact that some of the not-yet-treated JSs will be treated in the future. However, discarding these observations is not a solution, as it would result in a sample selected based on outcomes (Crépon et al. 2009). This example shows that adjusting for the time spent in unemployment alone does not lead to a meaningful comparison.

That is, there are two interdependent random processes: one that models exit from unemployment and the other that models selection into the treatment. Ignoring the interdependence between these two processes would lead to biased results. The sequential IPW method of (Vikström 2017) that we employ explicitly addresses this problem by sequentially using not-yet-treated JSs to estimate the counterfactual unemployment probabilities of never being treated. This method allows the exploration of heterogeneous treatment effects regarding the length of previous unemployment.

Different types of objects of interest can be estimated in the policy evaluation framework (for an overview, see (Abbring and Heckman 2008)). Here, we focus on the average effect of the treatment received after a period of unemployment and compare it with outcomes after no treatment exposure; thus, the comparison group is the group of JSs who will never be treated. More specifically, we use the average treatment effect on the treated s months after becoming unemployed on the probability of exiting unemployment earlier than t months after the JS is registered as unemployed versus the effect of never receiving the treatment. This average effect has also been considered in previous works, e.g., (Fredriksson and Johansson 2008; Crépon et al. 2009).

Fredriksson and Johansson (2008), Crépon et al. (2009), and Vikström (2017) used not-yet-treated participants to obtain the counterfactual probability of exiting unemployment in the absence of treatment exposure. As explained by Albanese et al. (2020), Vikström (2017) applied the IPW approach to dynamically adjust weights to account for selection into the programme and employed them with the Kaplan–Meier estimator of the likelihood of surviving in unemployment.⁸

In what follows, we denote our outcome of interest the indicator of being potentially absent from the unemployment register at time t (which is measured in quarters since the beginning of the unemployment period) conditional on participation in the programme at time s as $Y_t(s)$, where $Y_t(s)=0$ means that a JS is in the unemployment register (proxy for being unemployed) and $Y_t(s)=1$ means that a JS is not in the unemployment register (proxy for being employed). The potential outcome if never treated is $Y_t(0).$ $\overline{Y}_t(s) = \{Y_1(s),\dots ,Y_t(s) \}$ denotes the sequence of potential outcomes, and $\overline{Y}_t(s)=0$ represents that $Y_1(s) = \dots = Y_t(s) = 0$. The start of the programme is denoted with S, $X_t$ stands for a vector of additional covariates, which is measured at least slightly before time t, and $D_t$ is the binary indicator for treatment at time t.⁹

Our interest is in the average effect of participation in a programme at time s. We examine the effect on the probability that a JS finds a job before time t and compare it with the probability that a JS who is unemployed in periods $1,2,3,\dots ,s-1$ and treated in time s finds a job if he or she never participates in a programme.¹⁰:

$$\begin{aligned} ATET_t(s) = - \left( \text {Pr}(\overline{Y}_t(s)=0|S=s, \overline{Y}_{s-1}(s)=0) - \text {Pr}(\overline{Y}_t(0)=0|S=s, \overline{Y}_{s-1}(0)=0) \right) . \end{aligned}$$

(1)

The identification of $ATET_t(s)$ rests on the following identification assumptions:

Assumption A1

Sequential unconfoundedness

$$\begin{aligned} \left\{ Y_k(s); \forall k, s \ge t \right\} \perp D_t | X_t, S>t-1, \overline{Y}_{t-1}(0)=0. \end{aligned}$$

(2)

This assumption relies on information about participants, which is captured by the vector of covariates $X_t$, being sufficiently rich that controlling for this information renders the treatment assignment as good as had it been random.

This assumption is likely violated if selection into treatment is mostly driven by the individual motivations and heterogeneity that are correlated with the potential outcomes. In line with multiple earlier policy reports, we argue that in the Slovak ALMP context, selection into treatment is substantially driven by the availability of ALMP funding at the particular time and in the corresponding region of the potential participant (Svabova et al. 2022; Hidas et al. 2016; Štefánik et al. 2014a). In the context of scarce resources and poorly managed access to ALMP programmes, JSs’ individual characteristics become relatively less relevant compared to the time and space of their appearance in the registry of JSs. Attempting to capture this variability, we put substantial effort into generating variables capturing important aspects of the implementation context at the level of regional COLSAF offices and individual labour market histories, where individual skills and motivations could be partially manifested. Despite our efforts, we acknowledge that the causal interpretation of our results crucially rests upon the validity of the sequential unconfoundedness assumption.

We control for a rich set of individual characteristics, including labour market history, complemented by regional-level indicators (a complete list of the covariates can be found in Table A1 of the empirical appendix¹¹). The regional-level indicators used in our model were specifically designed to capture the moment of selection into treatment. For example, Roma ethnicity might play an important role in the moment of selection into one of the programmes. Members of this ethnic group, especially those who live in segregated settlements, face discrimination in access to public services (Mikula and Montag 2022). Considering the importance of ethnicity in selection into one of the programmes, we use not only individual-level information on ethnicity but also high-granularity regional information on the presence of a segregated Roma community in the particular village.¹² Additionally, JSs from segregated Roma communities more often leave school early and enter the labour market or unemployment at an earlier age. As a result, they already have an extensive labour market history at a young age. We exploit this information through a list of variables that covers unemployment history and employment and income history up to three years before the start of the unemployment period.

Being aware of the youths’ shorter labour market history, we utilise information pertaining to the level and field of attended education at the highest available level of detail. Individual-level information is coded upon being presented with a diploma or certificate, thereby distinguishing ten fields at four levels of education. At this level of granularity, all the key educational tracks of the Slovak educational system are distinguished, each of which is associated with a different length of study, and vocational and general educational programmes are differentiated between. Unfortunately, our database does not include educational outcomes, such as test scores or grade repetitions collected during the study.

Caliendo et al. (2017) test the unconfoundedness assumption in a data context comparable to ours. They conclude that the usually unobserved variables do not threaten the validity of estimates acquired when using unconfoundedness-based estimators, especially if a comprehensive set of control variables is used. Our list of control variables copies the sets usually used in studies based on unemployment register administrative data. In particular, the variables related to the outcome history should capture a substantial part of the unobserved differences and motivation that drive the selection into the programmes. The history of individuals’ labour market outcomes is complemented by evidence from the social insurance records, which allows for tracking preunemployment status and working income.

Since our sample consists of individuals under the age of 26, their observed labour market histories are shorter. In reaction, we aim to cover them with a more detailed list of variables, available monthly, during the two years preceding the unemployment spell. Social insurance-based employment records observed during this period also include part-time, temporary and student-type working contracts. In contrast, we consider unemployment histories over a more extended time since, especially for the low-skilled, registered unemployment often occurs shortly after age 15. For this reason, we include a set of variables describing past unemployment (the length of the first unemployment spell, the number of previous unemployment spells and the length of the previous employment in months).

Assumption A2

No-anticipation assumption

$$\begin{aligned} \text {Pr}(Y_t(s_1) = 1) = \text {Pr}(Y_t(s_2)=1), \ \forall t < \min (s_1,s_2). \end{aligned}$$

(3)

This assumption states that even if JSs have information about the timing of a future treatment (programme participation), then this does not result in a change in their behaviour related to job seeking, e.g., job search intensity (Abbring and Van den Berg 2003). That is, JSs do not anticipate future treatments.

The share of JSs participating in ALMPs in Slovakia in 2014 was approximately one-quarter of the share of those participating in such programmes in Sweden.¹³ The overall low level of access to programmes limits anticipation of JSs and, therefore, differs from the situation described in Sianesi (2008).

Two additional aspects of the evaluated measures speak against the potential violation of Assumption A2. First, the received financial support is relatively small compared to the opportunity costs associated with finding regular employment. Participation in all three evaluated measures is associated with receiving a monthly payment of approximately one-half of the minimum wage and less than one-quarter of the average wage. Second, because of the parallel implementation of AW by municipalities (Mytna Kurekova et al. 2013), low-income household members are effectively eligible to participate in AW from the beginning of their unemployment periods. Thus, the members of the group most tempted by the financial support associated with participation in AW are unlikely to have been motivated to change their job-search efforts based on the anticipated change in future income.

As we are aware of the limited options in supporting our assumptions empirically, we support the credibility of our results with a placebo test, using a randomly generated variable copying the observed distribution of the time elapsed between the start of unemployment and the start of participation in one of the ALMP programmes. Obtained placebo results are uniformly not statistically significant. They can be found in the empirical appendix (Subsection 3.4), accompanied by codes and a more detailed description of the estimation procedure.

4.2 Estimation technique

Under Assumptions A1 and A2, Vikström (2017) derived a consistent estimator for $ATET_t(s)$:

$$\begin{aligned}{} & {} \widehat{ATET_t(s)}= - \left( \underbrace{\prod _{k=s}^{t}\left[ 1 - \frac{\sum _{i=1}^{N} Y_{k,i}\cdot \mathbbm {I}(\overline{Y}_{k-1,i}=0)\cdot \mathbbm {I}(S_i=s)}{\sum _{i=1}^{N} \mathbbm {I}(\overline{Y}_{k-1,i}=0) \cdot \mathbbm {I}(S_i=s)}\right] }_{\begin{array}{c} \text {Estimator of observable probability} \\ \text {of presence in the unemployment register until }t\text { if treated in }s \end{array}} \right. \\{} & {} \quad - \left. \underbrace{\prod _{k=s}^{t} \left[ 1 - \frac{\sum _{i=1}^{N} {\hat{\omega }}_i(s,k)\cdot Y_{k,i}\cdot \mathbbm {I}(\overline{Y}_{k-1,i}=0)\cdot \mathbbm {I}(S_i>k) }{\sum _{i=1}^{N}{\hat{\omega }}_i(s,k)\cdot \mathbbm {I}(\overline{Y}_{k-1,i}=0) \cdot \mathbbm {I}(S_i>k)}\right] }_{\begin{array}{c} \text {Estimator of counterfactual probability} \\ \text {of presence in the unemployment register until }t\text { if never treated} \end{array}}\right) \end{aligned}$$

with

$$\begin{aligned} {\hat{\omega }}_i(s,k) = \underbrace{\frac{{\hat{p}}_s (X_{i,s})}{1-{\hat{p}}_s (X_{i,s})}}_{\begin{array}{c} \text {Propensity score} \\ \text {weight} \end{array}} \frac{1}{\underbrace{\prod _{m=s+1}^{k} (1-{\hat{p}}_m (X_{i,m}))}_{\begin{array}{c} \text {Estimated proportion of } \\ \text {not-yet-treated individuals} \end{array}}}, \end{aligned}$$

(4)

where subscript i stands for the i-th observation, N is the sample size, $\mathbbm {I}(\cdot )$ represents an indicator function, and $p_s(X_{i,s}) = \text {Pr}(S=s|X_{i,s},S \ge s, \overline{Y}_{s-1}=0)$ is the probability that not-yet-treated unemployed individuals in period $s-1$ will enter the programme in the next period. We implement Vikström (2017) by first discretising the time by quarter-of-year increments and then changing the risk set in every point in time. We find this to be a practical choice, balancing the precision of the estimated probabilities ${\hat{p}}_s(X_{i,s})$ and losing the precision in measuring the outcome and treatment timing (by month).

The intuition behind the expression for $\widehat{ATET_t(s)}$ is as follows: although we can estimate the quantity $\text {Pr}(\overline{Y}_t(s)=0|S=s, \overline{Y}_{s-1}(s)=0)$, which is the first term, directly from the data, we have to rely on Assumptions A1 and A2 to estimate the counterfactual probability $\text {Pr}(\overline{Y}_t(0)=0|S=s, \overline{Y}_{s-1}(0)=0)$, which is the second term. To do so, we use the individuals who have not yet been treated and adjust for their likelihood of receiving treatment using weights ${\hat{\omega }}_i(s,k)$ in a sequential manner. The term $\prod _{m=s+1}^{k} (1-{\hat{p}}_m (X_{i,m}))$ in the expression for the weight is the estimated proportion of not-yet-treated individuals and serves as a normalisation.¹⁴ We used a logistic regression to estimate ${\hat{p}}_s (X_{i,s})$.

For an illustration, consider a JS who enters the programme in the third quarter ($s=3$) after he or she becomes unemployed. We are interested in comparing the probability of this person exiting registered unemployment in the fifth quarter ($t=5$) after the beginning of the unemployment period with that of another treated person who would, counterfactually, never enter the programme. While we observe the first person, identifying assumptions must be used to recover the probability of exiting registered unemployment for the second person. We provide a graphical illustration of the approach in the appendix.

4.3 Balancing the subgroups of interest

To assess whether the IPW approach that we employ succeeds in producing a balanced sample, we develop our own routine that generates sample balance diagnostics. An R function that implements the routine is available online.¹⁵

We consider a subsample of JSs eligible for the programme at time k, which is denoted as ${\mathcal {S}}_k = \{i: \overline{Y}_{k-1,i}=0 \}$, of size $n_k$. This sample consists of JSs who enter the programme (treated - ${\mathcal {S}}_k^{t} = \{i: S_i=k, \overline{Y}_{k-1,i}=0 \}$) and those who do not (control - ${\mathcal {S}}_k^{c} = \{i: S_i>k, \overline{Y}_{k-1,i}=0 \}$) of sizes $n_k^{t}$ and $n_k^{c}$, respectively.

If the reweighting works, then we should observe that the empirical moments of the treated group are close to those of the control group weighted by the following:

$$\begin{aligned} {{\hat{\theta }}}_{i,k} = \frac{{\hat{p}}_k (X_{i,k})}{1-{\hat{p}}_k (X_{i,k})} \bigg / \left( \sum _{i \in {\mathcal {S}}_k^{t} }\frac{ {\hat{p}}_k (X_{i,k})}{1-{\hat{p}}_k (X_{i,k})} \right) , \end{aligned}$$

(5)

where the weights are normalised to the total weight.

For the j-th covariate, let us denote the means of the eligible, treated, control and weighted control groups as $\overline{X}_k^{(j)}, \overline{X}_k^{(j),t}, \overline{X}_k^{(j),c}$ and $\overline{X}_k^{(j),cw}$, respectively, so that

$$\begin{aligned} \begin{array}{l} \overline{X}_k^{(j)} = \frac{1}{n_k} \sum _{i \in {\mathcal {S}}_k } X_{i,k}^{(j)}, \\ \overline{X}_k^{(j),t} = \frac{1}{n_k^{t}} \sum _{i \in {\mathcal {S}}_k^{t} } X_{i,k}^{(j)}, \\ \overline{X}_k^{(j),c} = \frac{1}{n_k^{c}} \sum _{i \in {\mathcal {S}}_k^{c} } X_{i,k}^{(j)},\\ \overline{X}_k^{(j),cw} = \frac{1}{n_k^{c}}\sum _{i \in {\mathcal {S}}_k^{c}} {{\hat{\theta }}}_{i,k} \cdot X_{i,k}^{(j)}. \end{array} \end{aligned}$$

(6)

To quantify and visualise the balancing performance of IPW for every time period k, we introduce a balance measure that we call the standardised absolute bias, which is henceforth denoted as $\text {s.a.b}^w_k$. It is defined as the mean (across different covariates) of the absolute differences between the treated and weighed control means scaled by the standard deviation of the variable based on the eligible sample:

$$\begin{aligned} \text {s.a.b}_k^w = \frac{1}{J}\sum _{j=1}^{J} \frac{\left| \overline{X}_k^{(j),t} - \overline{X}_k^{(j),cw} \right| }{\widehat{\text {sd}}_k^{(j)}}, \end{aligned}$$

(7)

where

$$\begin{aligned} \widehat{\text {sd}}_k^{(j)} = \sqrt{\frac{1}{n_k} \sum _{i \in {\mathcal {S}}_k } \left( X_{i,k}^{(j)} -\overline{X}_k^{(j)} \right) ^2}. \end{aligned}$$

(8)

We can similarly define the balance measure for the j-th variable as an average across different time horizons k:

$$\begin{aligned} \text {s.a.b}^{(j),w} = \frac{1}{T}\sum _{k=1}^{T} \frac{\left| \overline{X}_k^{(j),t} - \overline{X}_k^{(j),cw} \right| }{\widehat{\text {sd}}_k^{(j)}}. \end{aligned}$$

(9)

For comparison purposes, we also define the standardised absolute bias for the unweighted control sample and denote it as $\text {s.a.b}_k$ and $\text {s.a.b}^{(j)}.$

In our application, the IPW estimator leads to a notable improvement in the balance measures for all three programmes (see Graphs A3-A5 of the online empirical appendix).

5 Data and the evaluation sample

The aim of this paper is to study the differences in the impacts of participation among the three ALMP programme alternatives offered to young JSs. The outcome of interest is postparticipation absence from the registry of the unemployed, which is used as a proxy for postparticipation employment. For this purpose, we explore a rich administrative dataset of the entire population of JSs younger than 26 years of age. We observe all JSs registered with the Slovak employment service, COLSAF. Additionally, our database is linked to the Social Insurance database, with data on all formally employed and self-employed persons in Slovakia, but this linkage applies only to the preunemployment period. Our observation period starts in January 2007 and ends in December 2014. For the period of 2007 to 2010, we can reconstruct the employment and unemployment histories of all individuals in our data. Outcomes are observed from 2011 to 2014, when we observe only the presence or absence from the registry of unemployed JSs. By participants, we mean those who participated only one time in one of the programmes of interest during 2011. The entire population of GP, AW and VAW participants from 2011 is observed, with no sampling. Those who participated multiple times or in other ALMP programmes from 2007–2014 are excluded from the analysis (included under “Other ALMP” in Table 2). To identify the contrasting, unbalanced control group of eligible nonparticipants, we use the total eligible population of JSs who were younger than 26 years of age and registered as unemployed during 2011.

Table 2 Structure of individuals younger than age 26 in registered unemployment in 2011, based on their ALMP participation

Most of the individual characteristics of JSs are reported at the time of registration through the Application for Registration in the Database of JSs and subject to later updates. In addition, we construct a comprehensive set of variables that refers to the employment and unemployment histories of JSs from 2007–2010. Here, we utilise information about the frequency and duration of employment and unemployment, the economic sector of employment, earnings, and ALMP participation. The set of covariates is complemented by a list of regional-level characteristics.

The main differences between the eligible JSs and participant groups are observable in their ethnicity, education levels, declared skills and labour market histories. The educational structure of GP and VAW participants reflects that, of the eligible JS group, the education and skill structures of AW participants are biased towards low-skilled workers, with a high overrepresentation of individuals with only elementary or lower secondary education and without computer or foreign language skills; they are also more likely to be Roma (not Slovak). A similar picture is drawn by the evidence on labour market histories, with AW participants displaying less employment experience and longer past unemployment periods. Additionally, women are overrepresented among the GP and VAW participants (Table 3). A complete list with descriptive statistics on the subgroups of interest can be found in the online empirical appendix (Table A2).

Table 3 Mean values of selected characteristics of eligible JSs and participants

Nevertheless, the individual characteristics (including the level of education) are not the strongest among our observable predictors of selection into the three programmes of interest. A look at the explanatory power of the variables used to estimate the propensity score reveals an interesting picture. The programme’s availability to a particular regional inflow cohort of JSs explains an even greater proportion of the observed variability than the observable individual characteristics of JSs. Variation in the programme availability explains 10.3 percent of the total variability in participation in GP, 11.6 percent in the case of VAW and 36.6 percent in the case of AW (Table 4). In our probability model, the time and place of the start of unemployment are even more important in predicting participation in the programmes than participants’ individual characteristics. The availability of ALMP funding to the programmes of interest presents an important predictor of selection into treatment at the relevant time and in the corresponding region of the potential participant. At the same time, the lack of resources contributes differently to the accessibility of the programmes. It appears to have a stronger predictive power in the case of “exclusive” programmes. Predicting the propensity to participate in GP, the share of variance explained by the full model increases from 8.7 to 17.3 percent when adding the programme availability variables. In the case of AW, the increase is relatively smaller (from 48.8 to 53.5 percent), with individual and regional characteristics explaining a higher share of variance. These outcomes document the differences in the selection into the programmes described earlier in Table 3.

Table 4 Pseudo R-squared of the PS model specifications

In contrast, the characteristics of the regional labour market, including the regional unemployment rate and the average wage, explain a relatively smaller proportion of the variability in the model of the propensity of selection into the programmes of interest (Table 3). In the case of AW, the characteristics of the regional labour market and the individual labour market history explain a greater proportion of the variability than in the cases of GP and VAW. This evidence speaks in favour of interpreting AW as the programme of last resort to which JSs are sent only after they do not fit with GP or VAW (in line with Mytna Kurekova et al. (2013)).

Note that these predictive characteristics of the propensity score models alone cannot provide direct evidence for the validity of the unconfoundedness assumption (A1) because they are only based on observable characteristics; thus, important unobservable confounders could still be omitted. Moreover, the Pseudo R-square values reported in Table 3 need to be interpreted with caution as unobserved confounders might also bias the predictive power of the arbitrarily clustered sets of explanatory variables. Despite these risks, we believe that these results may provide indicative evidence that we are able to capture a non-negligible part of the variation in the selection into the programmes.

5.1 Achieved balance of the groups of participants and eligible nonparticipants

In our identification strategy, we use IPW to adjust for the observed differences between participants and eligible nonparticipants. To assess how the IPW-based estimation technique succeeded in this task, we plot the mean standardised absolute bias before and after weighting is applied. This assessment documents a significant balance improvement with satisfactory postweighting levels of standardised bias averaged for the postparticipation observation periods (Fig. 2) and covariates (Fig. 3). More detailed evidence of the mean standardised balance improvement achieved by weighting can be found in the online empirical appendix (Graphs A3).

Fig. 2

Improvement of the mean standardised absolute bias by postparticipation quarters (GP, AV, and VAW). The dotted vertical line indicates the level of the mean standardised absolute bias of 0.1, considered as the threshold level of a satisfactory balance (see, e.g. Morgan (2018)). Source: COLSAF database

Fig. 3

Improvement of the mean standardised absolute bias by covariates (GP, AV, and VAW). The dotted vertical line indicates the level of the mean standardised absolute bias of 0.1, considered as the threshold level of a satisfactory balance (see, e.g. Morgan (2018)). Full labels of covariates can be found in the empirical appendix (Table A1): http://www.lmevidence.sav.sk/data_uploads/Empirical_appendix.html Source: COLSAF database

6 Summary of the results

We trace the entire population of JSs younger than 26 years of age registered with the Slovak public employment service in 2011. The more educated among them, with more working experience and skills, are selected for participation in the GP programme. These JSs represent approximately 6.65 percent of the eligible population; 65 percent of them are female, and more than 90 percent of them are sent to the programme within the first nine months after the start of their unemployment period. In terms of the observable characteristics of GP participants, the composition of participants remains biased towards the more educated and skilled, regardless of the period when they were recruited for the programme (see Table A2.1 of the empirical appendix). At the same time, time and spatial programme availability explain a substantial proportion of variance in access to the programme (see Table 4), which suggests that in situations with sufficient resources, the bias towards the more employable may become less pronounced. We underline that the causal interpretation of our results rests on the assumption that your data are sufficiently rich to control for selection (Fig. 3).

We examine the ATETs on the postparticipation absence from the unemployment register. First, we look at the GP participants whose participation took place six to nine months after the start of their unemployment period in the 3rd quarter of their unemployment (top left corner of Fig. 4). Within the additional twelve months after the start of their GP participation, their chances of exiting registered unemployment improve by approximately 2.5 percentage points compared to the outcome in the absence of GP participation. Eighteen months after participation, the net gain in terms of unemployment exit chances peak at 3.3 percent. After a slight decline, the ATETs remain steady and statistically significantly different from zero throughout the observed postparticipation period of two and a half years that is, ten quarters after the start of the program participation (Fig. 4—first row at left). The pattern and the magnitude of the coefficients estimated for the third quarter are comparable to earlier studies on the impact of GP (Institute of Fiscal Policy 2016; Štefánik et al. 2020; Svabova and Kramarova 2021). In addition to these studies, we show that as the flows of cream-skimmed participants channelled to their future employers through GP early in their unemployment dry-out, the statistical significance of the positive ATETs disappears because of not only a decrease in the sample size but also an increase in the heterogeneity of the ATETs.

Fig. 4

Estimated average treatment effects $\widehat{ATET_t} (s)$ of those treated in the third, fourth and fifth quarters $(s=3,4,5)$ after the start of their unemployment periods by programme of interest. Note: Outcomes are measured based on presence in or absence from the registry of unemployed persons. Ninety-five percent confidence intervals are acquired based on 500 bootstraps. The horizontal axes show the number of quarters since the start of the programme (t-s). Extended results can be found in the empirical appendix (Table A3.4): http://www.lmevidence.sav.sk/data_uploads/Empirical_appendix.html Source: COLSAF database

Apparent evidence that supports the early intervention approach can be found in the case of the impact of the VAW programme (bottom row of Fig. 4). VAW appears to have a high impact on the absence from the unemployed registry for participation that takes place in the third quarter after the start of the unemployment period, whereas the ATETs decrease for participation starting after a longer period of unemployment. For participation that takes place in the fifth quarter after the start of the unemployment period, the ATETs become different from zero only at the very end of the observation period of one and a half years-six quarters after the start of the programme). The composition of VAW participants is less selective than in the case of GP; in fact, it aligns quite well with the average composition of the eligible group (see Table A2 in the empirical appendix). Overall, the gains from the less selective VAW are higher than from the highly selective GP. More pronounced evidence of higher gains of earlier VAW participation provides an example of an early ALMP provision being relatively more important in the case of less selective ALMP programmes.

Participation in AW does not show a definite impact on participants’ chances of absence from registered unemployment, regardless of its timing during the unemployment period. The only exception is the positive impact of participation in the third quarter on absence from the registry 2 years after the start of participation, which also provides subtle support for the early intervention approach. Nevertheless, the estimated ATETs hide substantial regional heterogeneity in the implementation of the programme, which is also described by previous studies (Mytna Kurekova et al. 2013; Štefánik et al. 2014a). Our example provides evidence that supports the conclusions of earlier meta-analyses (Caliendo and Schmidl 2016; Card et al. 2018; Kluve et al. 2019) and indicates a zero or even negative employment effect of participation in the public (community) works type of programme. At the same time, we add to the literature on Slovak ALMPs by documenting that the negative employment effect observable under a static evaluation strategy (Hidas et al. 2016; Štefánik et al. 2014a) disappear once a dynamic evaluation framework is applied. While a negative impact of AW participation on the absence from the PES register is reported by some of the earlier studies, our estimates join the evidence pointing at its more ambiguous impact. Additionally, we show that differences in the impact of AW, its alternative VAW and a more selective GP disappear with an increase in the amount of time elapsed between the start of the unemployment and participation in the programme. Considering only participation between the twelfth and fifteenth month of unemployment (right column of Fig. 4), the relatively better performance of GP over AW disappears. Although the wider confidence intervals of estimates acquired for VAW participation in the fifth quarter are already driven by a smaller sample size, a comparable pattern also appears in the case of this programme.

Our dynamic evaluation framework assumes the timing of participation to have an impact not only on the outcome of interest (absence from the registry of unemployed) but also on the composition of the participants. The presented ATETs quantify the average net effects on the population of participants. In our case, the interprogramme differences in the composition of participants exceed the composition change observable for different periods of participation. Table 3 reveals the characteristics that are strongly differentiated among the participants in the three ALMP programmes of interest through the considered periods of participation. We select a few of them to help us inspect the heterogeneity of our ATETs. Specifically, we report the ATETs estimated for participation in the third and fifth quarters of the unemployment period for the following subgroups of participants:

• Gender;

• Aged older and younger than 20 years old;

• Level of education.

• Living in a village or town; and

• Living in a municipality with the share of Roma population less than or greater than 5 percent.

The ATETs estimated for absence from the unemployment registry can be explored in Table A3 in the online empirical appendix.¹⁶

In Table 3, we observe that female participants dominate GP and VAW. Based on the existing evidence, ALMPs tend to have a higher impact in the case of female participants (Card et al. 2018). However, in the case of GP and VAW, the average impact is not higher for female participants than for male participants. While in the case of VAW, both genders yield an impact of a comparable magnitude, in the case of GP, female gains from participation are even smaller than male gains. Moreover, the gains from early participation in VAW are driven by the impact on female participants. In contrast, male participants tend to yield a higher impact for participation in the fifth quarter of the unemployment. Please note that our outcome of interest is absence from registered employment, with differing gender-specific patterns of existing registered unemployment.

Programme participants also differ substantially in their educational level. We observe that the positive ATETs of GP and VAW are driven by individuals with higher secondary and tertiary educations. In the case of individuals with no or only primary education, the estimated ATETs are greater in magnitude but also linked to a larger heterogeneity, which results in positive but statistically nonsignificant coefficients during most of the observation period. Nevertheless, in the long run, participation in GP also appears to have an impact on less-educated individuals. This finding is in line with evidence on the impact of comparable programmes in neighbouring countries, such as the Czech Republic (Hora and Sirovátka 2020) and Hungary (Krekó et al. 2023).

In the case of VAW, the estimated ATETs appear to be driven by the positive ATETs for participants with secondary and tertiary education. JSs with no or only primary education are strongly over-represented among the AW participants. Negative ATETs of AW are observed mainly for participants with secondary education. For the most numerous elementary-educated AW participants, the estimated effects are ambiguous. In contrast, for tertiary-educated job seekers remaining in the database for at least 12 months, we observe positive ATETs of AW, which become statistically significant 15 months after participation.

Interestingly, participation in AW also yields positive long-term effects in terms of absence from the unemployment register in municipalities with an above-five percent share of Roma individuals in the population. Although an even more pronounced impact in this segment is also observable for participation in VAW, a positive impact of participation in AW, which is higher than the impact observable for participation in GP, contrasts with the overall picture drawn by our comparison.

AW appears to mildly but positively impact participants with tertiary education, as well as participants residing in municipalities with a higher share of Roma individuals in the population. This finding presents an interesting contribution to the stream of the Slovak impact evaluations of this programme (Lubyová and Van Ours 1999; Štefánik et al. 2014a; Hidas et al. 2016), which identify the effects of the programme exclusively under a static framework and consistently indicate its overall negative or ambiguous employment effects.

For later participation, the gain in terms of postparticipation absence from registered unemployment is indisputably reduced for VAW. We document an example where early participation in a nonselective programme yields a higher impact. In contrast, early participation in the cream-skimming GP programme does not have a different impact than later participation in regard to the unemployment spell. While the gains from early GP participation are driven mainly through female and more educated participants, an increased impact with later GP participation is observable for males and participants living in settlements with an increased share of Roma individuals in the population. Additionally, we estimate higher ATETs for GP participation during the sixth quarter, i.e., 16 to 18 months after the start of unemployment (see Graph A3.4 in the empirical appendix). This evidence aligns with the conclusions of a previous study on the impact of a programme similar to GP that is implemented in the Czech Republic, which claimed that the highly selective programme is poorly targeted because it yields higher gains in the case of previously long-term unemployed participants (Hora and Sirovátka 2020).

7 Concluding remarks and discussion

There is rich empirical evidence on the negative impact of early-career unemployment on future employment outcomes, especially for low-skilled individuals. Herein, we describe three comparable and alternative ALMP programmes in Slovakia, their allocation to registered young JSs, and how these programmes differ in terms of their impacts on postparticipation absence from the unemployment registry. The design of all three programmes aims to enable the acquisition of job experience by inserting unemployed individuals into a workplace environment. The programmes differ in the composition of participants, which results from a selection process administered by the public employment service provider, COLSAF. Moreover, they vary in the character of the workplace environment into which participants are inserted. Although the GP and VAW programmes predominantly insert participants into an existing job within an organisation, AW inserts them into a “community service”-type job organised by a municipality.

Our identification strategy is specifically chosen to compare the impact of the ALMP participation that occurs in various stages of the unemployment period. Nevertheless, the outcome of interest (the probability of absence from registered unemployment) is influenced not only by the duration of previous unemployment but also by the composition of participants that potentially changes with the prolongation of the unemployment period. The analysis of the determinants of the selection of young people registered as JSs into one of the evaluated ALMP programmes suggests that the programmes are used as alternates, with each covering a different type of JS. Moreover, the between-programme differences in the composition of participants overcome shifts in the composition observed with prolonging the period of elapsed unemployment. GP remains the most exclusive of the programmes, whereas VAW and especially AW are leveraged in situations in which GP is not available because of budgetary limitations. Moreover, AW is more frequently provided to less-skilled JSs with less favourable employment histories.

Because of the different functions of the inspected programmes in the portfolio of provided ALMPs, we choose an empirical strategy that recognises the persisting programme-specific composition of participants. Subsequently, we explore the development of the ATETs as the start of the participation shifts to later stages of unemployment. Please note that, interestingly, the observed composition of participants in each of the programmes does not differ substantially between the various participation periods. For example, the GP graduates remain more educated and with richer working experience regardless of whether they participate in Q3 or Q5 (see Table A2.1 in the empirical appendix). The between-programme participants’ composition differences remain more substantial than those between periods of participation. The unobserved characteristics are likely to change with an increase in the amount of time elapsed since the start of the unemployment spell as the more motivated and skilled find jobs. The same outcome occurs for both participants and the eligible control group. Still, the PES client workers consistently tend to select particular types of JSs for GP. Considering this situation, we choose an empirical strategy different from Sianesi (2008) or Biewen et al. (2014), as we do not provide a comparison of ATETs for the subset of participants overlapping across the three inspected programmes. Instead, we explore ATETs for the programme-specific composition of participants. Based on this approach, we observe how the advantage of the cream-skimming GP or the not-as-selective VAW disappears if provided in the later stages of unemployment. Depending on the tenable level of causal claims present in our empirical strategy, we see two possible interpretations of our results. Assuming we are able to account for the selection into the programme, we show that the same programme might work differently depending on the period of participation and the proportion of motivated/demotivated jobseekers leaving/remaining in registered unemployment. Alternatively, assuming that we are not fully able to account for the selection into the programme and residual confounding drives our results, we describe the mechanism of how this works in the case of a selective programme (GP) and its contrasting example (AW).

A comparable portfolio of ALMP programmes is quite common in the CEE region (Krekó et al. 2023; Hora and Sirovátka 2020). For example, Hora and Sirovátka (2020) studied a Czech ALMP programme that is similar to the GP programme and highlighted its pattern of selecting more employable young JSs. The authors also identified a bias in the composition of participants towards the more employable, i.e., the creaming-off effect. As it was observed in a similar institutional context, the evidence presented herein aligns with the findings of the authors concerning the composition of participants. Additionally, the authors concluded that in the case of the Czech version of the GP programme, the employment effect of participation appears to be higher for long-term-unemployed and medium-skilled participants. Heterogeneity analysis of our ATETs reveals that the ATETs for male or medium-skilled participants are higher for later GP participation. Although we document some potential of the GP programme to yield positive effects on less-educated, less-skilled and, thus, less-employable groups of participants, we also observe that a substantial share of its positive effects is driven by participants with higher education and those who enrol in the programme soon after they become unemployed. Our findings align with those of Hora and Sirovátka (2020), which suggests that better targeting and extending the GP programme to less easily employable participants highlights the potential to yield additional impact of the programme.

If provided after 12 months of unemployment, all three evaluated programmes perform comparably poorly, with only marginal positive effects observable in the long run, i.e., after 2 years, and only for the better educated. In line with Holtmann et al. (2021), we document that programmes that facilitate the collection of workplace experience for unemployed youth perform better for those who have completed their secondary education. A meta-analysis of studies that evaluate youth-targeted ALMP programmes (Caliendo and Schmidl 2016) pointed out that supported employment is usually linked to positive employment effects through the so-called foot-in-the-door effect (Brown 2015). This effect also plays a role in the cases of GP and VAW. In contrast with the efficiency principle, these programmes do not specifically target individuals furthest from the labour market.

In evaluating wage subsidy programmes, it is difficult to empirically disentangle the impact of participating in the programme from a potential substitution or dead-weight effect when employers use subsidised labour instead of hiring new employees. This concern is underlined if the easily employable are selected to participate in the programme, as in the case of GP. Nevertheless, with the relatively lower participation costs of GP, these concerns become less economically significant. Since 2014, the EU-wide Youth Guarantee initiative has changed the proportions of resources flowing to youth activation. Thus, the financial contribution to GP participants has increased substantially; employers’ costs have become eligible for reimbursement, thereby increasing the unit costs of GP. Additionally, alternative, youth-targeting, wage-subsidy programmes have entered the scene. Despite the growth of its unit costs, GP remains the least expensive option in the portfolio of youth-targeting ALMPs. Despite its sub-mediocre impact, this programme presents an attractive option for putting the feet of the cream-skimmed, easy-to-employ young JSs in the doors of their potential future employers.

The effects estimated herein of the public works programme (AW) are predominantly not significantly different from zero. Examples of public works programmes associated with negative employment effects are fairly common (Caliendo and Schmidl 2016; Card et al. 2018). Earlier studies, based on a static evaluation framework, have claimed that AW has a statistically significant negative impact on postparticipation employment (Štefánik et al. 2014a; Institute of Fiscal Policy 2016). We show that when a dynamic evaluation framework is applied to the same data and institutional contexts, the negative effects turn positive, although the results are mostly not statistically significant.

In the case of AW, the foot-in-the-door effect (Brown 2015) is predominantly absent because young participants are only rarely placed in a real work environment with a real option of bridging participation to real employment. Although the overall picture drawn about the impact of participation in AW is not favourable, it shows positive results for more educated participants who reside in smaller villages and municipalities with an increased presence of Roma communities. This might be driven by cases when municipalities manage to organise AW-facilitated works in a manner comparable to the private sector. There are rare cases of municipalities that establish their own businesses by using AW as a source of probationary hiring from the local pool of unemployed people.¹⁷ After 2014, participating in AW became a precondition for eligibility for the social assistance benefit. This approach increased the number of participants and overwhelmed the municipalities with an excess number of community workers. This change made AW a factual social assistance measure and thus reduced its already questionable activation level. In 2023, AW and VAW were jointly abolished.

We observe that JSs’ individual differences in the preparticipation phase translate into the selection of one of the alternative programmes and are further deepened by the differences in the impacts of the programmes on postparticipation absence from the unemployment register. Although the programme that accepts the most employable young registered JSs (GP) improves their chances of absence from registered unemployment, the programme that aims for those furthest from the labour market (AW) is linked to no or only marginal and heterogeneous improvement in unemployment chances during subsequent stages of participants’ careers. Our findings thus contribute to the existing empirical literature by confirming the findings of earlier studies (Sianesi 2008; Caliendo et al. 2011) and meta-analyses (Caliendo and Schmidl 2016; Card et al. 2018; Kluve et al. 2019) in identifying the poorer impact of workplace experience obtained under public or community services on postparticipation chances of absence from registered unemployment. We complement the information generated by the meta-analyses by describing how the difference in the impact of public and private sector workplace experience disappears if provided in the later stages of the unemployment spell (around 12 months).

The causal interpretation of the evidence presented here rests on the assumption that our observed variables (information about the characteristics of JSs) can account for the differences in selection into the programme. While we were creative in capturing potential unobserved drivers of selection into the programmes by an extended list of individual and regional labour market history covariates, we admit that potential violations of the sequential unconfoundedness assumption cannot be fully ruled out. After accounting for the observable differences between the population of participants and eligible nonparticipants, the estimated effects present average values over the population of participants. Apparent between-programme differences in the composition of participants, therefore, limit a straightforward comparison of the estimated impact on the postparticipation absence from the unemployment register. Moreover, not only unobserved differences among individuals (such as motivation or social capital) but also social constructs (such as discrimination against Roma individuals) may play important roles in the decision to participate in one of the programmes; furthermore, we assume that these differences should also be captured in the observed preparticipation labour market outcomes of individuals and the characteristics of particular settlements (municipalities). Crucially, our empirical strategy accounts for JSs’ differences in a rich list of indicators that specifically covers preparticipation labour market outcomes and employs high-granularity regional information.

Finally, we note that our findings are limited by the character of our data, which does not cover a substantial share of Slovak youth not in education, employment or training. Those we observe had to have successfully completed the procedure of registration with COLSAF. Thus, examining those young unemployed who were not able to manage this step might reveal a different behavioural pattern not captured in our data. For example, our data do not capture discouraged young individuals. Such discouragement might result from previous experiences with COLSAF registration or programme participation and, at the same time, impact our outcome variable (absence from the register). However, we believe that these discouraged individuals do not form a substantial group since being out of school with either registered unemployment or employment is costly due to the obligation to pay individual health insurance contributions.

Being aware of all these limitations, we still see the contribution of our results to the existing empirical evidence on the impact of ALMP programmes in at least two points. First, applying a dynamic evaluation framework to compare three programmes that facilitate workplace experience, we document that the disadvantage of the “public works” type of programme disappears with time between the start of unemployment and the start of participation. Although the relatively poorer performance of “public works” type of programmes in comparison to other comparable types of ALMPs (e.g. hiring subsidies, internships or subsidised employment) is well documented in international empirical studies, as well as those from Slovakia, only a few of these studies consider the timing of participation. Adding to this stream of literature, we show that if the timing of participation is considered, then the public works type of programme (AW), although showing no statistically significant average effect, reveals a positive contribution to the reemployment chances for specific subgroups, such as in the case of settlements with a higher share of Roma individuals in the population (see Table A.3 in the empirical appendix). Our second contribution points to the promising performance of voluntary activation works (VAW), which highlights a mixture of activation works supported by the nongovernmental sector. VAW evolved as a spin-off variant of the numerous and widely known AW. It inherited the entire existing set-up while offering the same type and intensity of support through the NGO sector instead of the municipalities. We show that in terms of impact, VAW outperformed both the stigmatising AW and the highly selective internship measure (GP) despite providing support of comparable intensity and length. Thus, our evidence hints at the potentially beneficial cooperation of public employment services with the nongovernmental sector, opening potential pathways for more in-depth research in the future.