Open Access

Can public works programs mitigate the impact of crises in Europe? The case of Latvia

IZA Journal of European Labor Studies20132:10

DOI: 10.1186/2193-9012-2-10

Received: 12 June 2013

Accepted: 26 July 2013

Published: 28 August 2013

Abstract

To mitigate the impact of the 2008–2010 global financial crisis on vulnerable households, the Government of Latvia established Workplaces with Stipends, an emergency public works program that targeted registered unemployed people who were not receiving unemployment benefits. This paper evaluates the targeting performance and welfare impacts of the program. The paper employs a quasi-experimental estimation strategy and analyzes a unique household survey. The authors find that the Latvian public works program was successful at targeting poor people, and leakage of benefits to non-poor households was small. Using propensity score matching, the authors find that the program’s stipend mitigated the impact of job loss and raised participating household incomes by 37 percent relative to similar households not benefiting from the program. The paper also finds that the forgone income for this program was less than forgone incomes estimated in other countries.

JEL codes

I38, J64, J68

Keywords

Crisis Latvia Public works Safety net

1. Introduction

Latvia was one of the hardest hit countries in the world during the 2008–2010 global financial crisis; during 2008–2010, Latvia’s gross domestic product (GDP) contracted by 21 percent. During 2009 alone, GDP contracted by 18 percent (Figure 1a). Poverty rates increased by eight percentage points - from 10.1 percent in 2008 to 18.1 percent in 2009 (Ajwad et al. 2012). Net job creation was negative as layoffs rose sharply. Between 2008 and 2010, 126,000 jobs were lost, equivalent to 11.2 percent of the pre-crisis workforce. In 2008 Q3, unemployment rates began to rise and reached a peak of about 21 percent in 2010 Q1, compared to about six or seven percent in the pre-crisis 2007–08 period (Figure 1b).
https://static-content.springer.com/image/art%3A10.1186%2F2193-9012-2-10/MediaObjects/40174_2013_Article_58_Fig1_HTML.jpg
Figure 1

Economic growth and unemployment rates in Latvia. (a) Real GDP growth rate. (b) Unemployment rate (15-64).

In September 2009, in response to rapidly rising unemployment rates, the Government of Latvia introduced an emergency public works program known as the Workplaces with Stipends (WWS) program, darba praktizēana, simtlatnieku programma, or the 100-Lats-programma.1 The program’s goal was to strengthen Latvia’s relatively weak social safety net in response to the unprecedented drop in economic activity and rapidly rising poverty.2 Specifically, the public works program created temporary labor-intensive employment for people who were unemployed but were ineligible for unemployment benefits.

This paper evaluates the impact of Latvia’s emergency public works program on mitigating the impact of the 2008–2010 global financial crisis on households. We employ a quasi-experimental estimation strategy and analyze a unique household survey administered to 3,000 households during December 2010 - March 2011. This paper contributes to the existing literature because we evaluate a public works program that was deliberately introduced as a social safety net to mitigate the impact of a very sharp decline in economic activity. To our knowledge, the only other evaluations of crisis-inspired public works programs are found in Latin America, in Argentina (Jalan and Ravallion, 2003Galasso and Ravallion 2004) and Peru (Chacaltana, 2003).

The key findings of this paper are threefold. First, the Latvian public works program increased household incomes by 37 percent relative to similar households not benefiting from the program. The increased incomes helped households to cope with the crisis significantly better than identical households that were not in the public works program. Second, the income that households had to forego was low, and some of the income that was foregone is accounted for by a loss of other safety net payments, suggesting that the public works program did not replace other labor market opportunities. The program increased household income by LVL 67 while the actual public works payment was LVL 100 per month. Third, the public works program targeted poorer households very well, meaning that the design parameters in the public works program performed as expected. Two key design parameters were the relatively low stipend and the labor intensive work requirement led to the good targeting performance.

The remainder of this paper is organized as follows: Section Context and program description presents the context in which the Latvian public works program was introduced and describes the program; Section Data describes the data used in this paper; Section Methodology outlines the quasi-experimental estimation strategy employed to evaluate the public works program; Section Results presents results, namely the program impacts on participant household welfare and the targeting performance of the program; and Section Conclusion concludes.

1.1 Context and program description

During 2008–2010, as the global financial crisis unfolded in Latvia, gross domestic product (GDP) contracted by 21 percent. Between 2008 and 2009 alone, Latvia’s GDP contracted by 18 percent (Figure 1a). Labor market conditions also worsened and the unemployment rate reached record highs (Figure 1b). Unemployment rates rose from relative low rates of about 5 percent to about 7.5 percent prior to the crisis to a record high of 21 percent in Q1 2010 (Figure 1b).

Prior to the crisis in Latvia, there were two main social safety net instruments to help people affected by an income shock – the unemployment insurance scheme and the poverty targeted social safety net scheme. However, these social safety nets were inadequate when confronting the broad and deep crisis of 2008–2010. With unemployment insurance, the main inadequacy was that there were about 40 percent of workers laid-off during the crisis who were not eligible for unemployment benefits, primarily because they failed to meet the requirement of having paid nine months of contributions into the unemployment insurance fund in a 12-month period.3 With the poverty targeted social safety net programs, the main inadequacy was that the programs only covered a small fraction of the population (less than 2.5 percent of the total population, and 5.2 percent of households in the bottom quintile were covered).4

The goal of the Latvian public works program was to reduce the severity of the social consequences of the global financial crisis on Latvians by providing temporary labor-intensive jobs to people who expressed an interest in the program by signing up on the waiting list. Central government expenditures on the public works program amounted to about LVL 8.0 (US$16) million in 2009; LVL 27 (US$54) million in 2010; and LVL 20 (US$40) million in 2011 (Government of Latvia, 2011).5 During 2010 and 2011, Government expenditures amounted to about 0.25 percent of GDP, or 2.0 to 2.5 times expenditures on the main poverty-targeted social assistance program, the Guaranteed Minimum Income program (World Bank, 2010). The public works program created almost 123,000 temporary jobs over 2009–11.6

The emergency public works program was open to anyone who was registered unemployed but not receiving unemployment benefits and signed up for the program.7 The program participants were eligible to participate for up to six months with a two week minimum requirement and jobs were provided on a first-come, first-served basis. There was no limit to the number of times a worker could benefit from the public works program, but re-entry into the program required signing up on the waiting list.8

The Latvian public works program was rationed through a self-targeting mechanism with two main components. First, a relatively low stipend was offered to public works participants to ensure that non-poor people would not crowd out poorer people in the program. Participants earned a stipend of about 80 percent of the binding net minimum monthly wage, or LVL 100 per month (about €142 or US$200).9 The stipend of LVL 100 was not subject to taxes or social contributions and all program participants were automatically insured against work-related accidents. Second, public works tasks were labor-intensive, which also dissuaded non-poor people from participating. The program required that the ratio of labor costs to material and tool costs be about 80 percent; therefore, some maintenance activities were viable, but few if any asset-creation activities qualified. Public works opportunities included public infrastructure maintenance, environmental clean-up, social services (through civil society organizations), and municipal and state services (excluding municipal and state enterprises). In some cases, the program also included a small training component that aimed to improve skills to perform public works tasks.

Cost-effective safety net programs transfer most program resources to their intended target group (Grosh et al. 2008). Naturally, they also economize the administrative resources required to implement the program. With the Latvian WWS program, the low stipend together with the requirement that 80 percent of the program budget be dedicated to labor costs means that the program was relatively cost effective. Subbarao et al. (2012) reviews data from 77 public works programs in 62 countries and find that 63 percent of the programs assigned a high labor cost share of 60 percent or higher, while the remaining programs were less labor intensive. This evidence suggests that the Latvia program was amongst the more cost effective public works programs in the world. However, a full cost-effectiveness analysis will require more information on administrative costs, which are notoriously difficult to find in most countries, partly because existing physical and human resources are used to implement the public works program and hence, it is difficult to assign a cost share to the program.

The program was implemented throughout Latvia and administered by all 28 State Employment Affiliates (SEAs). Municipalities supervised the works and in some cases, non-governmental organizations partnered in the activities. The availability of jobs depended on each municipality’s ability to create “new” jobs rather than transfer previously funded functions to the public works program. This requirement proved challenging in Latvia because municipalities outsource several activities and the public works’ central financing structure was an incentive for municipalities to transfer regular municipal functions to the public works budget.10 To ensure that public works positions were newly created, the Government provided municipalities with technical assistance to illustrate tasks that were eligible for program support. In addition, the central Government followed up with inspections to ensure compliance.

2 Data

This paper uses data from a unique household survey, henceforth referred to as the WWS Household Survey, commissioned by the Latvian State Employment Agency and fielded during December 2010-March 2011.11 The sample of WWS Household Survey respondents is representative of all Latvians who were eligible for the public works program, meaning that they were registered as unemployed persons. All five regions of Latvia - Kurzemes, Latgale, Riga, Vidzemes, and Zemgales - were sampled. We focus on data from the registered unemployed population from two strata within each region:

Strata 1: people enrolled in the emergency public works program (Treatment)   Strata 2: people on the waiting list for the emergency public works program (Control) A random sample of 1,166 people was drawn from Strata 1 (Treatment) and a random sample of 1,016 people was drawn from Strata 2 (Control).12

In this paper, we refer to individuals who were originally selected from the list of registered unemployed persons through random sampling as assigned individuals. The survey was conducted through face-to-face interviews and collected information on all members of the assigned individual’s household.

The household survey questionnaire resembles other labor force surveys in Latvia and includes questions on education, employment status of household members, household expenditures, asset ownership, and detailed questions for public works participants. Due to the time lapse between sampling (November, 2010) and the actual interviews (December, 2010 – March, 2011), about 396 assigned persons in the Treatment group ended their participation in the program, and about 222 assigned persons from the Control group began participating in the program. To avoid any contamination bias, we drop those assigned persons who have a different status than actually assigned.13 In addition, about 64 assigned persons in the Treatment group have an additional household member enrolled in the public works program besides the assigned individual. We drop these assigned individuals to avoid possible over estimation of the impact of the program on household welfare. Similarly, we drop 22 assigned workers from the Control group because another household member was enrolled in the program. The final sample size used in the analysis is: Treatment - 721; Control - 769.14

Omitting some of the assigned individuals (from Treatment and Control) to avoid contamination bias raises concerns about selection bias. However, as shown in Table 1, we reject this concern because we do not find significant differences in the characteristics of individuals who are dropped from the Treatment group from characteristics of those remaining in the Treatment group. Similarly, with the Control group, we find significant differences only in 2 out of 28 characteristics between individuals from the Control group who are dropped and those who remain in the Control group. Hence, dropping observations to avoid contamination bias does not introduce a significant selection bias.15
Table 1

Difference between assigned individuals in final sample and dropped assigned individuals

 

Remaining Treatment-Dropped treatment

T-test: Remaining Treatment = Dropped treatment

Remaining Control-Dropped control

T-test: Remaining Control = Dropped Control

 

Difference

T-Stat

Difference

T-Stat

Age 18-24

−0.004

(−0.25)

−0.046

(−1.83)

Age 25-29

0.004

(0.23)

−0.004

(−0.18)

Age 30-39

0.024

(1.00)

0.050

(1.90)

Male

0.047

(1.56)

−0.035

(−0.96)

Household head

0.006

(0.23)

0.022

(0.63)

Spouse of household head

−0.032

(−1.34)

−0.008

(−0.29)

Single

0.025

(1.02)

−0.057

(−1.82)

Married

0.042

(1.45)

−0.035

(−1.05)

Share of members in age 0-5

−0.459

(−0.65)

−0.931

(−0.95)

Share of members in age 6-17

0.525

(0.48)

1.089

(0.77)

Share of members in age 1864

−0.368

(−0.25)

0.966

(0.53)

Household size

0.043

(0.44)

0.013

(0.12)

Unemployed 12 months ago

0.045

(1.48)

−0.066

(−1.82)

Secondary education

−0.004

(−0.15)

−0.011

(−0.34)

Secondary profession

−0.026

(−0.90)

−0.000

(−0.00)

Higher education

0.026

(1.77)

−0.050**

(−2.74)

Own a flat

0.007

(0.25)

−0.023

(−0.64)

Own a house

−0.033

(−1.57)

0.014

(0.66)

House is owned by state

0.011

(0.43)

0.070*

(2.26)

House is owned by private entity

−0.005

(−0.27)

−0.046

(−1.66)

Have other Dwelling in other parts of Latvia

0.002

(0.11)

0.011

(0.56)

House have 1 room

0.008

(0.30)

−0.013

(−0.42)

2 room

−0.018

(−0.61)

0.060

(1.67)

3 room

0.038

(1.46)

−0.030

(−0.95)

Detached/semi-detached House

0.012

(0.39)

−0.005

(−0.14)

Flat in apartment

−0.007

(−0.70)

−0.018

(−1.68)

Wooden wall

0.015

(0.62)

0.105***

(3.46)

Concrete wall

−0.027

(−1.10)

−0.059

(−1.90)

Number of observations

1163

1016

Note: t statistics in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1.

The WWS Household Survey collected household income information through multiple techniques, but our preferred technique is the one that collected income, including the program stipend, through a series of questions posed to households about income components.16 Although the survey collected information on the program stipend, it did not collect information on the duration that a person received the stipend payments.17 Nevertheless, 99 percent of public works recipients report that they received their full pay and hence, we added the LVL 100 for individuals in the public works program (and LVL 0 for households in the Control group) to household monthly income to get total monthly household income. There are marginal differences in average incomes based on the two income definitions (reported in Table 2).
Table 2

Household monthly income, definitions

 

Description

Treatment

Control

Treatment and Control

  

Currently in WWS

On waiting list

Total

Income

Monthly aggregate income calculated adding individual components (LVL)

249

183

219

Income (secondary measure)

Monthly aggregate income reported by household (LVL)

237

202

220

3 Methodology

3.1 Targeting performance

To assess the targeting performance of the public works program, we identify the location of beneficiary households in the welfare distribution of the Latvian population. However, by design, the WWS Household Survey represents only the registered unemployed population and hence, the ranking of WWS Household Survey respondents in the overall population is not immediately known. We therefore combine the WWS Household Survey data with quintile cut-offs from the 2009 Household Budget Survey (HBS). Because the HBS is representative of the entire Latvian population, the quintile cut-offs generated from that household welfare distribution can be used to determine the quintile to which the WWS Household Survey respondents belong. Thereby, we assess the targeting performance of the public works program during the crisis.

3.2 Program impact

This section describes the methodology used to measure the short-term impact of the WWS program on household income. A quasi-experimental design is employed in the evaluation by exploiting the excess demand that prevailed throughout the implementation of the program. This excess demand led to a persistent waiting list of people who were interested in the public works program and not yet benefiting from the program, but crucially who were similar to program participants in their preference for the program. In other words, people on the waiting list had to some extent revealed unobserved factors influencing their choice to participate (Glasasso and Ravallion 2004).

However, latent heterogeneity between WWS participants and those on the WWS waiting list may still bias impact estimates. As WWS enrolment is on a first-come, first-served basis, the possibility remains that those individuals who were more likely to be impacted by any crisis are first to register for the program and the first to participate in the program. To control for observable heterogeneity, propensity score matching is used to construct a counterfactual outcome from the sample of individuals on the waiting list. Following Rosenbaum and Rubin (1983) we use propensity score matching (PSM) to estimate outcomes without the program (i.e., the average outcome for individuals who did not participate in the WWS program) and compare that outcome to the outcome for observationally similar participants in terms of their propensity to participate in the WWS program. Propensity is estimated using Prob(T i  = 1|X i ), i.e., probability of participating (T i  = 1) in the WWS program conditional on observed (pre-determined) covariates, X i .

Although propensity score matching controls for observable differences, it does not rule out the possibility of selection bias due to unobserved differences between participants and even a well-matched control group. We use Rosenbaum bounds (Rosenbaum, 2002) to assess the sensitivity of our results to the selection on unobservables. In the interest of brevity, and because Rosenbaum bounds have become more widely used in econometric analyses of program evaluation, we have omitted the formal details. Instead, we note that the objective of the method is to obtain bounds on the significance level of a one-sided test for no treatment effect under different assumptions concerning the role of unobservables in the treatment selection process. Specifically, we report upper bounds on the p-value of the null of zero average treatment effect on different values of Γ, where Γ reflects the relative odds ratio of two observationally identical persons participating in the WWS program. Thus, Γ is one in a randomized experiment or in non-experimental data free of bias from selection on unobservables; higher values of Γ imply an increasingly important role of unobservables. For example, Γ = 2 implies that observationally identical persons can differ by a factor of two in their relative odds of participating in WWS program.

4 Results

4.1 Targeting performance

One of the crucial goals of the Latvian emergency public works program was to target poor households. We find that 80 percent of beneficiaries are poor, meaning that their per capita household incomes per month are below the “needy line” of LVL 90 (Panel A of Table 3).18,19 Almost 83 percent of public works beneficiaries are in the bottom 20 percent of the welfare distribution and 96 percent of program beneficiaries are in the bottom 40 percent of the welfare distribution. The targeting performance of the Latvian public works program is very good in comparison to other public works programs that have been assessed around the world. Jalan and Ravallion (2003) find that 75–85 percent of participants in the Trabajar program (considered a well-targeted program) in Argentina were poor.
Table 3

Targeting performance of WWS Program

 

Quintile

 

Poorest

Quintile 2

Quintile 3

Quintile 4

Richest

Poor

Non-poor

Total

Panel A: Latvia

        

Treatment-Currently in WWS

83.0

13.0

2.1

1.5

0.4

80.2

19.8

100

Control-On waiting list for WWS

76.6

15.7

3.2

3.4

1.1

73.2

26.8

100

Panel B: Regions

        

Kurzemes

        

Treatment -Currently in WWS

82.9

11.9

2.9

2.4

0.0

80.8

19.3

100

Control-On waiting list for WWS

72.3

19.6

3.8

3.8

0.5

68.8

31.2

100

Latgale

        

Treatment -Currently in WWS

87.7

8.6

2.5

1.2

0.0

87.7

12.4

100

Control-On waiting list for WWS

79.0

14.0

2.6

4.4

0.0

76.5

23.5

100

Riga

        

Treatment -Currently in WWS

80.5

14.3

3.0

1.5

0.8

75.7

24.3

100

Control-On waiting list for WWS

72.3

16.2

4.7

3.7

3.1

66.2

33.9

100

Vidzemes

        

Treatment -Currently in WWS

80.5

16.7

1.2

0.6

1.2

77.8

22.2

100

Control-On waiting list for WWS

81.0

13.9

2.2

2.2

0.7

79.6

20.4

100

Zemgales

        

Treatment -Currently in WWS

87.6

10.7

0.8

0.8

0.0

83.7

16.3

100

Control-On waiting list for WWS

83.7

12.2

1.6

2.4

0.0

81.6

18.4

100

Note: Per capita income is calculated without WWS stipend. A household is considered poor if its per capita income (without WWS stipend) is less than LVL 90 per month. The quintiles cut-offs are derived using the income distribution of Household Budget Survey-2009 (the inflation between 2009 and 2010 has been about zero, hence no adjustment is made in HBS-2009 cut offs).

Conversely, in the Latvia public works program, leakage of benefits to non-poor households is low. About one-fifth of enrolled beneficiaries are non-needy, meaning that their incomes exceed the needy line income of LVL 90 per person per month. Alternately, less than five percent of all program beneficiaries are in the top 40 percent of the welfare distribution.

All five regions of Latvia perform well in terms of targeting, although targeting performance varies across regions (Panel B of Table 3). Importantly, targeting performance is good in poor regions, such as Latgale. The WWS program targeting performance remains robust to alternative indicators of welfare. For example, most WWS beneficiaries are less educated (highest educational attainment is basic or secondary level), and only a few university graduates report participating in WWS.20

4.2 Program impact: the impact of the WWS program on income

As discussed earlier, we believe that workers in the control group are similar to those workers in treatment group. The plausibility of the Conditional Independence Assumption (CIA) in PSM crucially relies on matching treated and control units on the basis of a large and informative set of pre-treatment variables.21 The variables of interest, summarized in Table 4, contain detailed information on demographic characteristics, educational attainment, family background, and the unemployment status twelve months prior to the survey.22 According to Heckman et al. (2009), when evaluating government sponsored training programs, the most important criterion are that outcome variables should be measured in the same way for both participants and non-participants; that members of treatment and control groups should be drawn from the local labor markets; and the data should allow one to control for dynamics of an individual’s labor force status prior to enrollment (Mueser et al. 2007).
Table 4

Difference in ex-ante variables, before matching

 

Mean

 

Treatment = Control

Variable

Treatment

Control

% bias

t

p > t

Age 18-24

0.094

0.149

−17.0

−3.27

0.00

Age 25-29

0.066

0.084

−6.9

−1.34

0.18

Age 30-39

0.178

0.139

10.7

2.07

0.04

Male

0.383

0.468

−17.3

−3.34

0.00

Household head

0.680

0.678

0.5

0.10

0.92

Spouse of household head

0.204

0.153

13.3

2.56

0.01

Single

0.193

0.252

−14.2

−2.74

0.01

Married

0.331

0.313

3.8

0.74

0.46

Share of members in age 0-5

5.040

6.531

−11.7

−2.25

0.02

Share of members in age 6-17

10.799

10.802

0.0

0.00

1.00

Share of members in age 1864

78.453

77.219

5.0

0.96

0.34

Household size

2.884

2.881

0.2

0.04

0.97

Unemployed 12 months ago

0.439

0.466

−5.3

−1.02

0.31

Secondary education

0.307

0.278

6.3

1.22

0.22

Secondary profession

0.390

0.389

0.1

0.02

0.98

Higher education

0.051

0.075

−10.0

−1.92

0.06

Own a flat

0.343

0.364

−4.4

−0.85

0.40

Own a house

0.152

0.096

16.8

3.24

0.00

House is owned by state

0.240

0.222

4.3

0.82

0.41

House is owned by private entity

0.132

0.185

−14.4

−2.77

0.01

Have other Dwelling in other parts of Latvia

0.084

0.075

3.3

0.63

0.53

House have 1 room

0.229

0.232

−0.8

−0.16

0.87

2 room

0.416

0.416

0.1

0.02

0.99

3 room

0.227

0.251

−5.5

−1.06

0.29

Detached/semi-detached House

0.417

0.437

−3.9

−0.75

0.45

Flat in apartment

0.025

0.025

−0.2

−0.03

0.97

Wooden wall

0.198

0.198

0.1

0.02

0.98

concrete wall

0.227

0.252

−5.8

−1.11

0.27

region==Latgale

0.112

0.149

−11.2

−2.14

0.03

region==Riga

0.186

0.256

−16.9

−3.25

0.00

region==Vidzemes

0.242

0.181

15.1

2.92

0.00

region==Zemgales

0.167

0.165

0.5

0.09

0.93

Note: Standardized bias (SB) for each variable is defined as the difference of sample means in the treated and control subsamples as a percentage of the square root of the average of sample variances in both groups.

In our case, the information was collected with the same questionnaire for both the treatment and control groups, which were drawn from the same local labor market. We capture the local labor market conditions by including a dummy variable for the region where the individual lives. However, the only measure available in our data identifying prior employment activity is an indicator of whether the individual was unemployed 12 months prior to the survey. If individuals who sign up sooner for the program are those with less stable work histories, even after measured characteristics (and the unemployment indicator) are held constant, bias could persist.

Table 3 presents summary statistics for our treatment and control groups. Based on 32 ex ante variables, we find statistically significant differences for 11 variables: age, gender, relationship to head of household, share of household members in 0–5 age range, home ownership, or living in a house held by private entity, and each of the five regional dummy variables. These findings suggest that although the assigned individuals in the treatment and control groups have characteristics that are statistically similar, they also differ significantly in some observed characteristics. Therefore, we control for observed differences before comparing outcomes of WWS participants and non-participants.

To control for observable heterogeneity, we adopt a propensity score matching technique to construct a counterfactual outcome from the sample of individuals on the waiting list. First, we estimated a probit model for calibrating the propensity score on the pooled sample of assigned individuals in the treatment and control groups. The complete model is reported in Table 5. The explanatory power of the model is low, suggesting that the two groups of individuals are similar with respect to many observed characteristics. Most explanatory variables have insignificant coefficients; while geography, gender, relationship to household head, share of household members in 0–5 age range, and higher education have significant impacts on WWS participation.23 Propensity score results confirm our expectations from the simple averages reported in Table 4, as there are no differences in simple averages of many characteristics, and they are insignificant in the propensity score model.
Table 5

Probit for calibrating propensity score

 

Coefficient

Standard error

Age 18-24

−0.172

(0.135)

Age 25-29

−0.020

(0.146)

Age 30-39

0.171

(0.105)

Male

−0.198***

(0.071)

Household head

0.268**

(0.120)

Spouse of household head

0.413***

(0.144)

Single

−0.057

(0.101)

Married

−0.077

(0.085)

Share of members in age 0-5

−0.009**

(0.004)

Share of members in age 6-17

−0.004

(0.003)

Share of members in age 1864

0.000

(0.002)

Household size

0.063*

(0.034)

Unemployed 12 months ago

−0.067

(0.068)

Secondary education

0.034

(0.094)

Secondary profession

−0.009

(0.093)

Higher education

−0.291*

(0.155)

Own a flat

−0.024

(0.177)

Own a house

0.287**

(0.143)

House is owned by state

−0.006

(0.133)

House is owned by private entity

−0.193

(0.143)

Have other Dwelling in other parts of Latvia

0.107

(0.124)

House have 1 room

0.056

(0.144)

2 room

0.016

(0.129)

3 room

−0.111

(0.127)

Detached/semi-detached House

−0.003

(0.133)

Flat in apartment

−0.102

(0.237)

Wooden wall

−0.113

(0.093)

concrete wall

−0.031

(0.085)

region==Latgale

−0.269**

(0.115)

region==Riga

−0.266***

(0.100)

region==Vidzemes

0.116

(0.099)

region==Zemgales

−0.121

(0.105)

Constant

−0.119

(0.309)

Number of observations

1,484

 

Pseudo R-Square

0.0417

 

Note: Standard errors in parentheses. *** p < 0.01, ** p < 0.05, * p < 0.1.

As expected, we find considerable overlap in support between the treatment and control groups (Figure 2). Table 6 explores whether the model has balanced all ex ante variables, i.e., we calculate differences between treatment and control groups for each characteristic in the matched sample. Conditioning variables are balanced, as indicated by the t-tests in Table 6, Panel A. Matching balances differences observed in the raw data; in the matched sample, all differences between the treatment and control groups are insignificant. Matching also significantly reduces standardized bias. In most empirical studies, a standardized bias below three percent, or five percent after matching, is acceptable (Caliendo and Kopeinig, 2008). In our case, the standardized bias is below three percent for almost all covariates. We report another test in Table 6, Panel B. Following Sianesi (2004), we re-estimated the propensity score on the matched sample, i.e., only on participants and matched nonparticipants, and compare the pseudo-R2 before and after matching. The pseudo-R2 indicates how well the regressors explain the probability of participation. After matching, no systematic differences should exist in the covariate distribution between the two groups, therefore the pseudo-R2 should be low. In our case, the pseudo-R2 indeed approaches zero after matching.
https://static-content.springer.com/image/art%3A10.1186%2F2193-9012-2-10/MediaObjects/40174_2013_Article_58_Fig2_HTML.jpg
Figure 2

Overlapping support in the distribution of the propensity score. Note: Histogram of propensity score distribution for WWS participants and WWS waiting list; 3 (0.2%) of the participants are off the common support.

Table 6

Balancing tests: difference in ex-ante variables after matching

Panel A

    
 

Mean

 

% reduction

Treatment = Control

Variable

Treated

Control

% SB

|SB|

t

p > t

Age 18-24

0.095

0.104

−3.0

82.2

−0.62

0.53

Age 25-29

0.065

0.071

−2.3

67.5

−0.45

0.66

Age 30-39

0.175

0.169

1.7

83.9

0.32

0.75

Male

0.385

0.392

−1.4

92.2

−0.26

0.80

Household head

0.684

0.675

2.0

−303.8

0.38

0.70

Spouse of household head

0.199

0.196

0.8

94.1

0.14

0.89

Single

0.195

0.201

−1.5

89.6

−0.29

0.77

Married

0.328

0.333

−1.0

74.9

−0.18

0.86

Share of members in age 0-5

5.066

5.322

−2.0

82.8

−0.41

0.68

Share of members in age 6-17

10.637

10.691

−0.3

−1543.1

−0.06

0.96

Share of members in age 1864

78.533

78.316

0.9

82.4

0.17

0.87

Household size

2.873

2.909

−2.4

−1077.0

−0.44

0.66

Unemployed 12 months ago

0.440

0.445

−1.1

78.5

−0.22

0.83

Secondary education

0.309

0.306

0.7

89.4

0.12

0.90

Secondary profession

0.389

0.388

0.4

−184.8

0.07

0.95

Higher education

0.051

0.054

−0.9

91.3

−0.18

0.86

Own a flat

0.344

0.358

−3.0

32.6

−0.57

0.57

Own a house

0.150

0.139

3.4

80.0

0.59

0.55

House is owned by state

0.241

0.233

1.8

58.2

0.34

0.74

House is owned by private entity

0.134

0.136

−0.6

96.1

−0.11

0.91

Have other Dwelling in other parts of Latvia

0.082

0.085

−1.0

69.6

−0.18

0.85

House have 1 room

0.229

0.222

1.8

−113.4

0.34

0.73

3 room

0.416

0.420

−0.9

−952.1

−0.17

0.87

4 room

0.227

0.239

−2.8

49.4

−0.53

0.60

Detached/semi-detached House

0.419

0.431

−2.6

34.3

−0.49

0.63

Flat in apartment

0.025

0.025

−0.3

−57.6

−0.05

0.96

Wooden wall

0.200

0.203

−0.6

−417.3

−0.11

0.91

concrete wall

0.229

0.239

−2.2

62.8

−0.41

0.68

region==Latgale

0.111

0.118

−2.0

82.5

−0.39

0.70

region==Riga

0.188

0.188

−0.1

99.7

−0.01

0.99

region==Vidzemes

0.238

0.232

1.3

91.1

0.25

0.81

region==Zemgales

0.168

0.174

−1.5

−226.8

−0.29

0.77

Panel B

      

Sample

Pseudo R2

LR chi2

p > chi2

   

Unmatched

0.042

85.8

0.0

   

Matched

0.001

2.97

1.0

   

Note: Standardized bias (SB) for each variable is defined as the difference of sample means in the treated and control subsamples as a percentage of the square root of the average of sample variances in both groups. Reduction in bias refers to the percentage reduction in bias after matching.

The diagnostic analysis reveals that matching controls for the small differences observed in the unmatched data. In Table 7, we present the average impact of the WWS program on short-term household incomes.24 We use kernel matching (KM), which is a nonparametric matching estimator that uses weighted averages of (nearly) all - depending on the choice of the kernel function - individuals in the control group to construct the counterfactual outcome.25 Thus, a key advantage is the lower variance, which is achieved because more information is used.
Table 7

Average impact of WWS on incomes

    

Matched

Bootstrap

 

Normal-based

  

E(Y1|T = 1)

E(Y1|T = 0)

Difference

Standard error

P-value

[95% Confidence Interval]

Household income

       

Income

Monthly aggregate income calculated adding individual income components

248.53

181.57

66.96

9.01

0.00

49.31

84.62

Income (secondary measure)

Monthly aggregate income reported by household

236.74

200.31

36.42

8.17

0.00

20.40

52.45

Per capita income

       

Income

Monthly aggregate income calculated adding individual income components

97.97

62.86

35.10

3.29

0.00

28.78

41.43

Income (secondary measure)

Monthly aggregate income reported by household

92.23

73.11

19.13

2.71

0.00

13.82

24.43

Note: The standard errors are derived via bootstrapping with 100 replications.

Households in the treatment group have an average monthly income of LVL 248.5 while households in control group have an average monthly income of LVL 181.5 (Table 7). Hence, the WWS program increases household’s income by about 37 percent. We conclude, therefore, that the WWS program acted as an effective short-run safety net (i.e., during program enrollment).

Participation in the WWS program transferred a nominal amount of LVL 100 each month to the household. However, it is important to look beyond the net transfers from the program (wage or stipend) because public works beneficiaries must forgo some income to participate in the program, which requires a time commitment from participants. The value of forgone income depends on the national context – including the prevailing market conditions, household labor-supply decisions, and access to safety nets.26 For example, using a quasi-experimental approach, Jalan and Ravallion (2003) show that in Argentina’s Trabajar program, the average net gain to public works participants was about half of the gross wage. Glasasso and Ravallion (2004) found that in Argentina’s Trabajar and Plan Jefes y Jefas programs, a large share of participants were women who would not otherwise have participated in the labor force. Therefore, under the Trabajar and Plan Jefes y Jefas programs, about half of the employment gain came from unemployed workers, and the other half arose from inactive workforce participants. Chacaltana (2003) found that the net gain derived from the Trabajar Urbano program in Peru was equal to 24 percent of the nominal transfer. Beneficiaries received a monthly salary of 300 soles, while their control group was able to generate 227 soles on their own, in the absence of the program. In the Empleo in Action program in Colombia, participant monthly employment income increased on average close to 39 percent over what would be earned without a program; but income was much higher for women (90 percent) and for youth between 18 and 25 year old (54 percent) (Departamento Nacional de Planeacion, Colombia, 2004). Datt and Ravallion (1994) found that other family members took up displaced productive activities when someone joined a workfare program in rural India.

In the case of Latvia’s WWS program, the average gain for participant households is about LVL 67 per month, which is about two-thirds of the WWS stipend (LVL 100 per month). Thus, forgone income is only about LVL 33.27 We calculate the average gains in WWS program based on other matching estimators (Table 8), but we find similar gains. Thus, the net gain from the WWS program is not sensitive to the choice of the estimator.
Table 8

Impact of WWS program, alternative matching methods

Income

 

With replacement:

 

Nearest neighbor

70.77

 

(13.39)

5-nearest neighbor

63.42

 

(10.19)

Caliper (б = 0.001)

67.55

 

(11.00)

Caliper (б = 0.01)

64.72

 

(9.57)

Note: The standard errors are in parenthesis, and are derived via bootstrapping with 100 replications.

The forgone income in the WWS program is lower than forgone income estimates for public works programs in other countries. In other words, the control group was unable to generate income, likely because of a lack of labor market opportunities, which was also reflected in record high unemployment rates; and because of the low coverage and benefits of poverty-targeted social assistance programs.28 However, some forgone income can be explained by the loss of other safety net income for participants, e.g., guaranteed minimum income (GMI). For example, households in the treatment group earn LVL 6 less (LVL 4) from GMI (unemployment benefits) compared to households in the control group (Table 9). Moreover, the households that qualify for other safety nets might prefer the WWS program because it offers higher benefits. When households choose WWS, they lose top-up benefits such as those available under GMI. However, municipalities in Latvia might encourage WWS participation because the WWS costs are borne entirely by the central government, whereas municipal governments co-finance GMI benefits. Also, the work requirement makes the safety net politically appealing.
Table 9

Difference in household income components

 

Treatment group

Control group

Matched Difference (ATT)

Standard error

T-stat

Labor income

68.0

94.6

−26.6

7.5

−3.6

Income from WWS

100.0

0.0

100.0

-

-

Income from informal sources

3.5

4.6

−1.1

0.9

−1.2

Income from other sources

7.5

8.3

−0.8

2.3

−0.4

Pension

41.5

35.4

6.0

4.5

1.3

Social transfers

     

 Guaranteed Minimum Income (GMI)

4.2

9.8

−5.6

1.5

−3.8

 Housing Allowance

1.2

2.4

−1.2

0.7

−1.7

 Heating Allowance

3.9

4.1

−0.2

1.1

−0.2

 School Meals

1.8

3.0

−1.3

0.7

−1.7

 Other municipal assistance

1.6

1.4

0.1

1.0

0.2

 Children’s Allowance

1.3

1.1

0.1

0.6

0.2

 Parental/maternal Benefit

0.4

0.4

0.0

0.4

−0.1

 State Family Benefit

5.4

5.2

0.2

0.5

0.5

 Other family state benefit

1.2

2.3

−1.1

0.7

−1.5

 Unemployment Benefit

0.4

4.0

−3.6

0.8

−4.5

 Sickness Benefit

0.5

1.0

−0.5

0.5

−0.9

 Disability Benefit

3.6

2.5

1.1

1.1

0.9

 Other state benefit

2.1

1.2

0.9

0.7

1.3

In addition to looking at the WWS program impact on income gains; we also look at the impact of public works on the type of coping strategies adopted by the households during the crisis (Table 10). In the questionnaire, each household was asked whether they adopted any of the strategies to cope with the recent crisis, and a number of common coping strategies were enumerated with the option of yes or no.29 As households in both treatment and control groups were affected by the crisis, they reported that they adopted multiple coping strategies to mitigate the impact of the crisis. Table 10 presents the impact of the WWS program on the adoption of different coping strategies.30 Interestingly, a household in the treatment group is less likely to adopt any coping strategy compared with a similar household in the control group. The WWS program impact on nutrition and health outcomes is particularly noteworthy: a lower proportion of households participating in WWS reported reducing their food intake (quantity and frequency), or reducing doctor visits (preventive and during illness) than households in the control group.31
Table 10

Coping strategy adopted by Treatment and Control households

 

Difference in adoption rate (Treatment-Control)

T-stat (Treatment = Control)

Reduced consumption of food staple

−7.31

2.70

Skipped meals

−8.12

 

Reduced lighting/heating/water consumption

−5.73

2.13

Reduced entertainment consumption

−2.04

0.78

Bought less clothes

−3.62

1.40

Withdrew preschool kid

−1.08

1.39

Withdrew from university

−0.21

0.24

Withdrew from training classes

−0.97

0.86

Reduced educational expenditures

−0.64

0.39

Reduced doctor’s appointments (preventive)

−6.68

2.61

Reduced doctor’s appointments (when ill)

−3.54

1.36

Stopped buying medicine

−5.14

1.99

Cancelled phone service

−3.21

2.26

Postponed investments in business

−1.13

1.05

Reduced help to friends

−3.36

1.95

Cut TV service

−4.23

3.25

Change transportation mode

−1.49

1.26

Cut internet service

0.28

0.21

Note: The difference is the difference in percentage of the households in Treatment and Control group reporting using the particular strategy as a response to crisis.

4.2.1 Sensitivity of the PSM estimates to the unobservables

Although the PSM estimation controls for selection of observables, any selection of unobservables can bias the results. We assess the sensitivity of our results to the selection of unobservables using Rosenbaum bounds. Table 11 reports the upper bound on the p-value of the null of zero average treatment effect for different values of Γ. If the upper bound on the p-value is less than, say, 0.10 for reasonably large values of Γ, then the treatment effect is said to be robust to hidden bias. We find the income gains to be sensitive to hidden bias only if Γ is larger than 2. While the Rosenbaum bounds do not yield point estimates of the treatment effects once hidden bias is taken into account, they increase confidence in WWS program impacts because the positive impact of the WWS program on income is robust to a large selection bias of unobservables.
Table 11

Sensitivity of PSM estimates with respect to unobservables, Rosenbaum Bounds

Income

Г

1.0

1.2

1.4

1.6

1.8

2.0

2.2

P-value

0.000

0.000

0.000

0.000

0.007

0.092

0.375

Note: Rosenbaum critical p-values for test of the null of zero average treatment effect on treated (ATT). For controls included in the propensity score, see Table 5.

5 Conclusion

Latvia was one of the hardest hit countries during the 2008–2010 global financial crisis. GDP contracted by 18 percent in 2009; poverty rates increased by about 8.0 percentage points in 2009; and in 2010 Q1, the unemployment rate was about three times the pre-crisis unemployment rate. To mitigate the impact of the crisis on poor households, the Government of Latvia introduced an emergency public works program targeting registered unemployed people who were not receiving unemployment benefits. Between Q3 2009 and end-2011, the public works program created almost 123,000 temporary jobs. This paper assesses the household welfare impacts and the targeting performance of the emergency public works program using a unique household survey collected between December 2010 and March 2011.

This paper employs a quasi-experimental design to measure the program impact. More specifically, we exploit the excess demand for the program that prevailed throughout the implementation of the emergency public works program in Latvia to construct a counterfactual group. Using a propensity score matching (PSM) method, we find that the program increased household incomes by 37 percent relative to similar households not benefiting from the program. The program increased household incomes by LVL 67 per month while the actual payment was LVL 100 per month. Thus, participants had to forego about LVL 33 per month to participate in the program; some of this foregone income is due to the loss of other safety net payments such as guaranteed minimum income benefits. Foregone income due to program participation is lower than foregone incomes estimated elsewhere; likely because alternate income-generating options were limited during the crisis because of a lack of labor market opportunities, which was also reflected in record high unemployment rates; and because of the low coverage and benefits of poverty-targeted social assistance programs.

The Latvian public works program experience highlights the usefulness of public works as a safety net during times of labor market crises. For policymakers and development experts interested in public works programs, this evaluation raises some points worth considering when designing a program. First, setting the stipend level below the binding minimum wage and making participants engage in labor-intensive activities increases the chances of good targeting performance. These conditions make the program only attractive to poor people, and non-poor people are unlikely to apply for the program. In effect, the public works program becomes a more effective self-targeting program. Self-targeting is particularly useful when fast scale up is needed during times of crises, and knowledge of household welfare is not well known to policymakers. Second, timing public works programs to coincide with spikes in unemployment (shocks, low seasons) can help people during periods when other employment opportunities are limited and alternative income sources are low. In other words, public works can be a good means of fortifying a country’s existing safety net. Third, public works programs during times of crises can help families maintain nutrition and health, thus preventing longer-term problems that might result from reducing food intake or foregoing medical visits. While the past literature has focused on the monetary benefits of safety nets during crises, a well-designed public works program during a crisis can also help households maintain their nutritional intake and can help them continue health checkups and health care.

Endnotes

1 Lats is the currency of Latvia, and 1 Lat was US$2 and €1.41 on March 31, 2011.

2 Prior to the crisis and relative to its neighbors, Latvia allocated a very small share of social welfare spending to programs designed to target the poorest households. The low spending on poverty-targeted benefits resulted in low coverage indicators---about six percent of households in the poorest quintile were covered by poverty-targeted safety nets (World Bank, 2010).

3 Although the proportion of unemployed workers who were not eligible for unemployment benefits varied over time because of variations in proportions meeting the eligibility requirements, as the crisis dragged on, the proportion of people not meeting the unemployment benefit eligibility requirements rose to more than 50 percent. This occurred as workers became increasingly involved in short-term and part-time employment as the crisis offered few other opportunities (Hazans, 2012). As a result, the requirement that workers complete nine months of unemployment insurance contributions in a 12-month period prior to qualifying for unemployment benefits was increasingly unmet.

4 World Bank (2010) reports that the two main poverty targeted social safety net programs are the Guaranteed Minimum Income and the Housing Benefit programs.

5 On March 31, 2011 the exchange rate was: €1.00: US$1.42: LVL 0.704.

6 There is significant variation in the duration of participation in the program, with some participants completing a few days and others completing six months in the program. Also, some program beneficiaries re-enlisted into the program and hence, only 75,000 people benefited from the emergency public works program while 122,000 jobs were created.

7 Registered unemployed could also chose the program instead of unemployment benefits if unemployment benefits were less than the program stipend.

8 After completing six months in a year, a beneficiary can re-register for the program, but they are placed at the bottom of the waiting list.

9 Despite initial opposition to what was believed to be a low stipend, the public works program was always oversubscribed, and the program waiting list was almost double the number of available positions. No doubt a lower stipend would have resulted in a shorter waiting list, but Government set the rate at LVL 100 to maintain political support for the program. In July 2011, the stipend was reduced to LVL 80 per person per month as the Government started to phase the program out.

10 Municipalities outsourcing is particularly common among the larger and wealthier municipalities.

11 The field work for the survey was carried out by GfK Custom Research Baltic.

12 In addition to Strata 1 and Strata 2, the survey also collected information on 500 individuals who were laid off during August-October 2009, and subsequently became WWS beneficiaries; however, they completed their participation in WWS at least six months prior to the survey (Strata 3). The survey also collected information on 500 individuals (Strata 4) who were laid off during August-October 2009 but who did not register for the WWS program (were not interested in the WWS program). Individuals in Strata 3 and Strata 4 are excluded from our sample because those individuals are neither current beneficiaries nor are they individuals who expressed a desire to join the WWS program at the time of the survey.

13 Contamination bias can occur if individuals in the control group are participants in the program being evaluated (Mueser et al. 2007). Mueser et al. (2007) only choose those in control group (ES participants) who were not enrolled in the program (JTPA) in the program year.

14 We keep only those assigned workers in the Treatment group who have participated in the program for at least one month. This led to 12 assigned individuals being dropped from the Treatment group.

15 The two significant characteristics---higher education and wooden walls in the household---do not affect the probability of participation in the program (Table 5) significantly. Hence, we believe that the statistical difference in those two variables between the observations remaining in the control group and being dropped from control group do not create a significant selection bias. However, by dropping the observations, we are able to eliminate contamination bias which would have been significant.

16 An additional household income variable is available because households were asked: “What is the total monthly income of your household at the moment?”

17 It is also not feasible to find out the duration of payment from the time spent in the program, as the duration of payment will not exactly match the time spent in the program.

18 A household is defined as poor if its per capita income (before the program stipend) is less than LVL 90 per month. Latvia has no official poverty line but the LVL 90 per capita per month is known as the “needy line”.

19 Households reported that the public works program is useful as a safety net and as a program to uplift the local community. Most participants view the program as an important safety net, and 96 percent believe that public works projects are beneficial to the community.

20 The education distribution among program participants is not surprising given that public works jobs were physically demanding. Three-quarters of program participants reported that their public works jobs are physically demanding.

21 The conditional independence assumption (CIA) postulates that given a set of observed characteristics X = x, the (counterfactual) distribution of outcome of individuals in the treatment group is the same as the (observed) outcome distribution of individuals in the waiting list. A weaker version in terms of conditional mean independence actually is sufficient.

22 Technically, these variables were recorded at the time of interview; however, we do not expect these variables to have changed because of participation in short term WWS program.

23 If the treatment was assigned randomly, none of the covariates is expected to significantly affect participation.

24 We use psmatch2 (Leuven and Sianesi, 2003) in STATA to get the PSM estimate.

25 To implement the matching estimator, we used the epanechnikov kernel and a fixed bandwidth of 0.10. Confidence intervals are obtained using 100 bootstrap repetitions.

26 Forgone income is an opportunity cost of participating in the program, i.e., what the household would have earned if the households had not participated in the program. This is unlikely to be zero -- the poor can rarely afford to be idle. As forgone income in not observable once a household starts working in a program, one way is to assess the forgone income by assessing how much a similar household, which is not currently participating in the program, is able to earn.

27 Since some income is forgone, the targeting performance reported in the earlier section (based on zero forgone income) overestimates the WWS program pro-poor finding.

28 About 83 percent of workers who have participated or are enrolled in the WWS program report they had to wait; almost 46 percent of workers report waiting six or more months before participating. Among people who are now waiting, more than three-quarters report waits of three or more months, and about 34 percent have been waiting for six or more months. This suggests that for many of workers, WWS was the only opportunity available.

29 See World Bank (2011, chapter 3), for further details on coping strategies adopted by households in Eastern Europe and Central Asian countries during the recent financial crisis.

30 In this case our outcome variable is whether a household has adopted that particular coping strategy or not.

31 Households reported that the WWS program is useful as a safety net and as a program to uplift the local community. Most participants view the WWS program as an important safety net, and 96 percent believe that WWS projects are beneficial to the community.

Declarations

Acknowledgements

An earlier version of this paper was circulated under the title “Did Latvia’s Public Works Program Mitigate the Impact of the 2008–2010 Crisis?” IZA Discussion Papers 6772, Institute for the Study of Labor (IZA).

The views expressed in this paper are those of the authors and should not be attributed to any organization. This paper has benefited from inputs from staff at the Ministry of Welfare and the State Employment Agency, Latvia, GfK Custom Research Baltic, and participants at a seminar at the University of Latvia. In addition, the authors acknowledge comments and suggestions from Colin Andrews, David Balan, Basab Dasgupta, Valerie Evans, Charles Griffin, Mihails Hazans, Herwig Immervoll, Theresa Jones, Margaret Koziol, Santosh Kumar, David McKenzie, Truman Packard, Ismail Radwan, Emily Sinnott, and Ilze Zvidrina. The authors may be contacted at mazam@okstate.edu or majwad@worldbank.org.

Responsible editor: Sara de la Rica

Authors’ Affiliations

(1)
Oklahoma State University, IZA
(2)
Independent Consultant
(3)
The World Bank

References

  1. Ajwad MI, Haimovich F, Azam M: Simulating the Impact of the 2009 Financial Crisis on Welfare in Latvia. World Bank Working Paper 2012, 5960.Google Scholar
  2. Caliendo M, Kopeinig S: Some Practical Guidance for the Implementation of Propensity Scores. J Econ Surv 2008,22(1):31–72.View ArticleGoogle Scholar
  3. Chacaltana J: Impacto del Programa A Trabajar Urbano: Ganancias de ingreso y utilidad de las obras. Informe Final. Lima, diciembre del: Centro de Estudios para el Desarrollo y la Participación; 2003. http://cajvavariados.com/variedades/a%20trabajar%20urbano.pdfGoogle Scholar
  4. Datt G, Ravallion M: Transfer Benefits from Public-Works Employment: Evidence for Rural India. Econ J 1994,104(427):1346–1369.View ArticleGoogle Scholar
  5. Departamento Nacional de Planeacion, Colombia: Evaluación de Polìticas Públicas No.2: Documento Programa Empleo en Acción, Condiciones inciales de los beneficiarios eimpactos de corto plazo. 2004.Google Scholar
  6. Galasso E, Ravallion M: Social Protection in a Crisis: Argentina’s Plan Jefes y Jefas. World Bank Economic Review 2004,18(3):367–399.View ArticleGoogle Scholar
  7. Government of Latvia: Latvian Labour Market 2010–2011. Riga, Latvia; 2011.Google Scholar
  8. Grosh M, del Ninno C, Tesliuc E, Ouerghi A: For Protection and Promotion: The Design and Implementation of Effective Safety Nets. The World Bank; 2008.View ArticleGoogle Scholar
  9. Hazans M: ‘What Works When the Labour Market Doesn’t? Host Country Discussion Paper presented at “Activation Measures in times of Crisis: the Role of Public Works”. Riga, Latvia; 2012:26–27. April, 2012 April, 2012Google Scholar
  10. Heckman JJ, LaLonde RJ, Smith JA: The Economics and Econometrics of Active Labor Market Programs. In Handbook of Labor Economics. Volume 3. Edited by: Ashenfelter O, Card D. Amsterdam: North Holland; 2009. 1999 1999Google Scholar
  11. Jalan J, Ravallion M: Estimating the Benefit Incidence of an Anti-Poverty Program. J Business Econ Stat 2003,21(1):19–30.View ArticleGoogle Scholar
  12. Latvian Academy of Agricultural and Forestry Sciences: Evaluation of the results of active employment measure: Training for acquiring and maintaining work skills when employer is a municipality, Final Report. 2011. http://www.nva.gov.lv/docs/17_4e1431090ba7a3.03485280.pdfGoogle Scholar
  13. Leuven E, Sianesi B: PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing. 2003. , November, 2010 http://ideas.repec.org/c/boc/bocode/s432001.html , November, 2010Google Scholar
  14. Mueser PR, Troske KR, Gorislavsky A: Using State Administrative Data to Measure Program Performance. Rev Econ Stat 2007,89(4):761–783.View ArticleGoogle Scholar
  15. Rosenbaum P, Rubin D: The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika 1983, 70: 41–55.View ArticleGoogle Scholar
  16. Rosenbaum PR: Observational Studies. 2nd edition. New York: Springer; 2002.View ArticleGoogle Scholar
  17. Sianesi B: An evaluation of the active labour market programmes in Sweden. Rev Econ Stat 2004, 86: 133–155.View ArticleGoogle Scholar
  18. Subbarao K, del Ninno C, Andrews C, Rodríguez-Alas C Directions in Development. In Public Works as Safety Net: Design, Evidence, and Implementation. The World Bank; 2012.View ArticleGoogle Scholar
  19. World Bank: From Exuberance to Prudence: A Public Expenditure Review of Government Administration and the Social Sectors. Washington, DC: World Bank; 2010.Google Scholar
  20. World Bank: The Jobs Crisis: Household and Government Responses to the Great Recession in Eastern Europe and Central Asia. Washington, DC: World Bank; 2011.View ArticleGoogle Scholar

Copyright

© Azam et al.; licensee Springer. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.