Unemployment rate forecasting: LSTM-GRU hybrid approach

Abstract

Unemployment rates provide information on the economic development of countries. Unemployment is not only an economic problem but also a social one. As such, unemployment rates are important for governments and policy makers. Therefore, researchers around the world are constantly developing new forecasting models to successfully predict the unemployment rate. This article presents a new model that combines two deep learning methodologies used for time series forecasting to find the future state of the unemployment rate. The model, created by combining LSTM and GRU layers, has been used to forecast unemployment rates in the United States, United Kingdom, France and Italy. Monthly unemployment rate data was used as the dataset between January 1983 and May 2022. The model’s performance was evaluated using RMSE, MAPE, and MAE values and compared to a stand-alone LSTM and GRU model. Results indicate that the hybrid model performed better for the four countries, except for Italy where the GRU model yielded better results.

1 Introduction

Unemployment is a pervasive issue that affects all countries. While the unemployment rate has worsened with various crises from the past to the present, it has remained more stable in some periods. Although it seems to be a bigger problem for underdeveloped countries, it affects all countries in times of crisis. The economic crises and their social effects in recent years have increased the interest of economists and other social sciences, such as sociology and psychology. Unemployment has been shown to negatively affect individuals’ life satisfaction, access to economic resources and mental health (Pohlan 2019). It also affects food security (Haini et al. 2022) and has been found to increase the risk of social exclusion (Kieselbach 2003; Popîrlan et al. 2021). However, it can bring complex experiences in terms of social class position and cultural context (Ali et al. 2013).

Weak demand in the economy leads to unemployment. Lower demand means less labour is needed, leading to reduced working hours or lay-offs. In countries with high unemployment, one of the most important policies is to ensure that the unemployed find work and receive social support. Failure to do so can lead to social instability, which affects the development of a country in all aspects. An increase in the unemployment rate reduces individual and social income, increases poverty and creates inequity in income distribution. Therefore, if the government can forecast the number and rate of unemployed workers, it can implement policies and measures that will ensure the welfare and development of the country according to these forecast data (Chakraborty et al. 2021). Therefore, accurate estimation of the unemployment rate is important for experts and policymakers in planning a country’s economic policies (Lai et al. 2021). Studies in the literature have drawn attention to the links between unemployment, gross domestic product (GDP), migration and social life (Fendel 2014; Krug et al. 2019; Ochsen and Welsch 2011; Iuga and Cioca 2013; Cazzola et al. 2016).

The accuracy of predictions about the future has a direct impact on the success of the policies and decisions taken. For this reason, not only is the subject of forecasting important, but also the accuracy of the forecast is critical. In addition, the fact that decision-makers can make more successful decisions thanks to accurate forecasts increases the interest in forecasting studies.

The unemployment rate has been a significant concern since the 1970s, when high inflation, high unemployment, and slow economic growth first emerged (Brunner et al. 1980). However, many statistical and econometric techniques have been used in the literature on unemployment estimation. Yet, in recent years, researchers have increasingly focused on using machine learning and deep learning models for this purpose.

Sermpinis et al. (2014) used a hybrid method consisting of genetic algorithms and support vector regression in their study. The results showed better performance than other methods used for comparison in the study. Stasinakis et al. (2016) investigated the effectiveness of Radial Based Function Neural Networks (RBFNN) in predicting unemployment in the US. They also examined the usefulness of the Kalman filter and support vector regression as forecasting combination techniques. The results show that RBFNN statistically outperforms the individual performance of all models. Davidescu et al. (2021a) applied seasonal model autoregressive integrated moving average (SARIMA), self-exciting threshold autoregressive (SETAR), Holt-Winters, ETS (error, trend, seasonal) and NNAR (neural network autoregression) to the unemployment rate in Romania. Lai et al. (2021) forecast unemployment for five developed and five developing countries in Asia. In the study, where ARIMA-ARNN, ARIMA-ANN, and ARIMA-SVM models were used, the hybrid ARIMA-ARNN model was found to perform well among all the hybrid models for the developed countries in Asia.

Davidescu et al. (2021b) used univariate and multivariate methods to forecast the unemployment rate in Romania. The empirical results show that SETAR and VECM provide comparable results in terms of accuracy, accurately reflecting the pre-pandemic period.

Chakraborty et al. (2021), using unemployment rate data from seven countries, found that the hybrid method used produced more meaningful results than the traditional methods. Another study using hybrid methods found that ARIMA-ARNN and ARIMA-SVM methods produced successful results (Ahmad et al. 2021). The study by Katris (2020) found that different methods were successful depending on the length of the forecast period. While fractional ARIMA (FARIMA) was successful for 1-step forecasts, neural network models were successful for 12-step forecasts. Google Trend indices were used by Simionescu and Cifuentes-Faura (2022) to forecast youth unemployment in Spain. Bayesian panel data models and fixed effects models were used to study regional data, while Bayesian vector autoregressive models and vector error correction models were used for national data. The Bayesian forecasting results provided the best forecasts for the nine regions. Gogas et al. (2022) used decision trees, random forests and support vector machines in their study of forecasting unemployment in the euro area. The Random Forest method achieved higher accuracy than other models.

The aim of this study is to investigate how accurately the hybrid method developed using LSTM and GRU methodologies can predict unemployment rates. Hypothesis statements within the scope of the study are presented as follows.

• The hybrid method created with LSTM and GRU methodologies will provide high accuracy in estimating the unemployment rate.

• Different datasets used in the study will affect the unemployment rate estimation accuracy of the hybrid method.

This study presents a new model for forecasting unemployment using univariate time series. Unemployment rate data for the years 1983(1)–2010(6) were used as the training set. The test set was used between 2010 (7) and 2022 (5). Although the unemployment rate has been predicted by various linear and non-linear methods in the literature, the presentation of a new hybrid model is the original aspect of the study. There is still a need for methods that offer better performance in forecasting the unemployment rate.

The paper is structured as follows: In Part 2, the data and techniques are briefly reviewed, and in Part 3, empirical results are summarized. Finally, Chap. 4 concludes with conclusions and recommendations.

2 Method

2.1 Long short-term memory (LSTM)

Long Short Term Memory networks, often known as “LSTM” networks, are specific types of recurrent neural networks (RNN). It was first developed by Hochreiter and Schmidhuber (1997). Because gradients are lost when learning long sequences of data, RNN is often not advantageous. LSTM solves this problem by deciding whether or not to use certain information. The gradients at the first few input points disappear and become zero as the length of the input sequence increases, making it harder to capture the effect of the early stages. The LSTM model, which has an input gate, an output gate and a forget gate, solves the drawbacks of the RNN. The three gates of LSTM facilitate the organisation of long-term memory (Yang et al. 2019). These three gates control all the cells. The LSTM architecture uses hidden units called memory cells when long-term dependencies are needed. It stores the items it needs to remember over a long period of time in these memory units. The LSTM can carefully control structures called gates to extract or add information to the cell state. In LSTM, the first stage is to decide which information from the cell state to discard. This decision is made by the forget gate. The next step is to use the input gate to decide whether the new context information should be stored in the cell state. A sigmoid layer is used to decide what information needs to be updated, and a ℎ layer is used to create a vector of new candidate information. Finally, the output gate determines the output information. The most important features of the LSTM architecture are that the inputs can be stored without forgetting to prevent information loss. The LSTM architecture is presented in Fig. 1.

Fig. 1

LSTM architecture (Olah, 2022)

Given the input array $x=({x}_{1},{ x}_{2},\dots .{x}_{t})$, the network calculates a mapping to the output $y=({y}_{1},y,\dots .{y}_{t})$. The following equations determine the unit activations. σ is the logistic sigmoid function. Gates I, f, o, and c are the input gate, forget gate, output gate, and cell activation vectors, respectively. All these vectors have the same dimension as the hidden vector h. W terms denote weight matrices from cell to gate vectors. Finally, Tanh is the output activation function. Activation functions commonly used in the LSTM network are sigmoid and hyperbolic tangent.

$${f}_{t}=\sigma ({W}_{f}\text{*}\left[{h}_{t-1},{x}_{t}\right]+{b}_{f})$$

(1)

$${i}_{t}=\sigma ({W}_{i}\text{*}\left[{h}_{t-1},{x}_{t}\right]+{b}_{i})$$

(2)

$${\acute{\text{C}}}_{t}=tanh({W}_{C}\text{*}\left[{h}_{t-1},{x}_{t}\right]+{b}_{C})$$

(3)

$${C}_{t}={f}_{t}\text{*}{C}_{t-1}+{i}_{t}\text{*}{\acute{\text{C}}}_{t}$$

(4)

$${o}_{t}=\sigma ({W}_{o}\text{*}\left[{h}_{t-1},{x}_{t}\right]+{b}_{o})$$

(5)

$${h}_{t}={o}_{t}\text{*}\text{t}\text{a}\text{n}\text{h}\left({C}_{t}\right)$$

(6)

The process described above continues by iterating through the LSTM. Weight parameters () and bias parameters () are learned by the model to minimize the difference between the actual training values and LSTM output values.

2.2 Gated recurrent unit (GRU)

GRU is an architecture that addresses the vanishing gradient problem that arises in iterative neural networks such as LSTM. GRU can learn long term dependencies. The GRU structure, a Gated Recurrent Unit, is an RNN-based architecture. Long-term dependencies that occur in RNN-based methods are removed by checking the information that has changed in the hidden state. (Cho et al. 2014). The GRU has a reset gate and an update gate in its structure. The update gate acts similarly to the forget and input gate of an LSTM, deciding what information to keep, discard, and add new information. The reset gate decides how much of the previous information should be forgotten. Because the resulting model is simpler than the traditional LSTM model, the GRU is becoming more popular every day. The organisation of a GRU unit is shown in Fig. 2. Due to its very straightforward structure, GRU outperforms LSTM in terms of training time and prediction accuracy (Jianwei et al. 2019).

Fig. 2

GRU architecture

The following equations perform these operations:

$${r}_{t}=\sigma ({W}_{r}{h}_{t-1}+{U}_{r}{x}_{t})$$

(7)

$$\widehat{{\text{h}}}_{t} = tanh({\text{W}}\left( {r_{t} {\text{*}}h_{{t - 1}} } \right) + {\text{U}}x_{t} )$$

(8)

$${z}_{t}=\sigma ({W}_{z}{h}_{t-1}+{U}_{z}{x}_{t})$$

(9)

$${h}_{t}=\left(1-{z}_{t}\right)*{h}_{t-1}+{z}_{t}\text{*} \widehat{{\text{h}}}_{t}$$

(10)

${h}_{t}$ and ${h}_{t-1}$ represent the output of the current and previous states, respectively. ${r}_{t}$ and ${z}_{t}$ indicate the reset and update gates, respectively. σ is the logistic sigmoid function while ${W}_{r}$ and ${U}_{r}$ are the weight matrices.

2.3 Hybrid method

Many hybrid linear and non-linear models have been used for forecasting in various studies. The hybrid model proposed in this study integrates LSTM and GRU models. The training and test data sets are divided into 70% and 30%. It was normalised between (0 and 1) using the min-max scaler, a simple technique to fit the data within a predefined limit. The first layer contains LSTM with 128 hidden neurons and the second layer contains GRU with 64 hidden neurons. The third layer is a dense layer with one neuron. The output of the LSTM model is fed into the GRU model to produce a single final output as it is, forming a combined, fully connected layer.

LSTM network, which produces the first level prediction. Next, the output of the LSTM layer is passed to the GRU layer to generate the final forecast. The complete network architecture consists of the LSTM layer, the GRU layer, and the dense layer. The structure of the proposed model is shown in Fig. 3.

Fig. 3

Hybrid method

. Serialized unemployment rate data is fed into the LSTM layer in the first step of the model. After receiving the input data, each LSTM neuron along the path generates a weighted value. The data passed through the LSTM layer is transferred to the GRU layer in the next stage. A weighted value is generated along the path from the LSTM layer to the GRU layer. The output neuron receives the data from the GRU layer and produces the weight accordingly. The cost function is then calculated by comparing the output with the initial values. When the minimum point of the cost function is reached, the weights are changed to reflect the difference between the predicted value and the actual value. The weights are then stored for future estimates.

2.4 Forecast evaluation metrics

Various criteria are used to compare the forecasting performance of the models. The most important of these criteria is the accuracy of the forecast. Since the problem in the study was formulated as a regression problem, the mean absolute error (MAE), root mean square error (RMSE) and mean absolute percentage error (MAPE) metrics were used for evaluation.

The MAE measures the difference between two continuous variables. The ideal model has an expected MAE value of zero.

$$MAE=\frac{1}{n}\sum _{i=1}^{n}|{y}_{i}-{\widehat{y}}_{i}|$$

(11)

Mean Square Error (MSE) is the most commonly used regression loss function. MSE is the average frame loss per sample over the entire dataset. Root Mean Square Error (RMSE) is the square root of the MSE. It is more frequently used than MSE since the MSE value occasionally can be too high to be easily compared. As a result, the MSE is computed using the error’s square, simplifying its interpretation.

$$RMSE=\sqrt{\frac{1}{n}\sum _{i=1}^{n}{({y}_{i}-{\widehat{y}}_{i})}^{2}}$$

(12)

One of the most used criteria for assessing forecasting performance is the mean absolute percentage error (MAPE). Where, ${\widehat{y}}_{i}$, is the forecasting value, ${y}_{i}$ stands for the actual value, and n is the time series length. In order to describe the value as a percentage, the formula frequently calls for multiplying the value by 100%.

$$MAPE=\left(\sum _{t=1}^{n}\left|\frac{{y}_{i}-{\widehat{y}}_{i}}{{y}_{i}}\right|\right)\frac{100}{n}$$

(13)

2.5 Data

Within the scope of the study, the unemployment rate data set covering 1983 (1)–2022 (5) of 473 months belonging to the USA, Great Britain, France, and Italy was used. The data sets were obtained from the OECD website (OECD, 2022). While making this choice, two European Union member countries, a country that left the European Union and the United States, which has the world’s largest economy, were chosen. The datasets graphical representation demonstrates the nonlinear nature of the unemployment rate data. The relevant data set is a string for use in the models. The graph of the data represented by years is given in Fig. 4. Table 1 provides a summary of the unemployment rate datasets.

Fig. 4

Monthly unemployment rate data

Table 1 Descriptions of the unemployment rate data sets

In forecasting applications, a data set is split into training and testing sets. The training set is used to train the model, and the testing set is used to evaluate the model’s performance on new data that it has not seen during training. The most important issue in forecasting applications is the amount of training and test data. The largest amount of data should be used when training neural networks. The output of the network when fed with test data is compared to the actual output values. The primary goal is to determine whether the sample is powerful enough to reflect the population. For this study, the hybrid method created by combining LSTM and GRU methodologies, two widely used deep learning algorithms in time series forecasting, was employed. All experiments in this study have been performed using the Python programming language, version 3.6, on a computer running the Microsoft Windows 10 operating system. The hybrid model was implemented using the Keras deep learning library.

3 Results

For each country dataset, training of the model was performed and it was run 50 times for each epoch value. The number of epochs is the number of times all training data is exposed to the network during training. The estimation was then performed on the test data. The performance of the methods was then measured using the MSE, RMSE and MAE metrics. The results of using the GRU model and the LSTM model alone are used to compare the effectiveness of the proposed model. The main objective of the research is to investigate whether the overall performance can be improved and whether one of these models can be outperformed by proposed model. The performance of the models in Table 2 has been calculated by comparing the predicted results with the actual values.

Table 2 Performance results of models

Hybrid models gave the best results for the US, UK and France datasets. For Italy, the GRU model gave the best prediction. The MAE, RMSE and MAPE values of the hybrid model (1500 epochs) for the United States are 0.26, 0.93 and 3.91 respectively. The MAE, RMSE and MAPE of the hybrid model (500 epochs) for the United Kingdom are 0.20, 0.28 and 4.66, respectively. The MAE, RMSE and MAPE of the hybrid model (1000 epochs) for France are 0.0097, 0.16 and 1.10 respectively. The MAE, RMSE and MAPE values of the GRU model (1500 epochs) for Italy are 0.23, 0.33 and 2.27 respectively. It can be seen that the GRU method gives results close to the hybrid method in countries other than Italy. The LSTM method, on the other hand, performs worse than the GRU and hybrid methods. Figure 5 shows the output of the estimation results and the actual values of the models that give the best estimation result for all countries.

Fig. 5

Actual and forecast values

Looking at the hybrid studies on estimating the unemployment rate in the past, it is generally seen that linear and non-linear models are combined. Chakraborty et al. (2021) aimed to improve their estimates by reflecting the asymmetry of the unemployment rate with the hybrid model they created in the ARIMA-ANN structure. Similarly, Ahmad et al. (2021) combined linear and non-linear methods in their study, in which they presented two hybrid methods, ARIMA-ARNN and ARIMA-SVM. The basis of these studies is that combining linear and nonlinear models can reduce the bias and variance of the prediction error of the models. In addition, some studies offer hybrid methods (Ruwali et al. 2020; Sajjad et al. 2020; Sah et al. 2022) using state-of-the-art methods such as LSTM, GRU and CNN. Examining the studies, the hybrid models created using two different deep learning methods are successful. They also help to better represent the incoming sequence data over time. Menculini et al. (2021) found that ARIMA models and LSTM neural networks performed similarly, while a combination of CNNs and LSTMs gave the best results. By separating stationary and non-stationary components, hybrid models can reduce the need for specialized input preprocessing (Lim and Zohren 2021).

4 Conclusion

Unemployment is one of the critical indicators of the economy. As the unemployment rate increases, less total economic production takes place. Unemployment also has adverse social effects on individuals.

Economic indicators such as the unemployment rate and the labour force can be used to identify economic trends and make investment decisions. An accurate forecast of the unemployment rate is essential to protect investors from market risks arising from unexpected changes in economic conditions and monetary policy. As one of the roles of central banks is to create monetary policy to reduce unemployment, accurate forecasts of the unemployment rate will be effective in the decisions that central banks have to make.

Accurately measuring and controlling the unemployment rate, one of the leading indicators of the labor market that fundamentally affects future social policy strategies, is one of the main goals of governments. The unemployment rate is a good indicator of productivity and capacity in an economy. It can also indicate changes in other indicators such as wages, household income and house prices. As a precautionary measure, governments can introduce tax breaks and other subsidies to encourage firms to increase hiring. Similarly, they can increase public spending to boost demand for labour in infrastructure projects or in health and education. Central banks can also help reduce unemployment by lowering interest rates. Low interest rates make borrowing cheaper and saving less rewarding, thereby stimulating demand in an economy.

This study proposes the LSTM-GRU hybrid model to forecast the unemployment rate of four countries (United States, United Kingdom, France and Italy). The empirical part used LSTM, GRU and the hybrid method created by combining these two methods. All models used 70% of the dataset for training and 30% for testing. The best accuracy rates were obtained with the hybrid method for all countries except Italy. The purpose of using different models and hybrids developed with these different models, rather than being limited to a single model, is to determine the model that gives the lowest root mean square (RMSE) and mean absolute error (MAE) values.

By using multiple models and comparing their performance, the study aimed to identify the most accurate method for predicting unemployment rates. Overall, the proposed hybrid model showed promising results and offers a potential area for further research and development testing. In the future, the performance of model will be evaluated by applying it to different time series. Although promising results were obtained by the hybrid method combining LSTM and GRU in forecasting unemployment rates in my study, some limitations may still exist. One of the limitations could be the hyperparameters of the model, which may need to be further optimized to achieve better results. Additionally, the performance of the model may vary for different countries or periods due to economic and social factors that affect the unemployment rate. Therefore, it is important to carefully evaluate the limitations and assumptions of the study when interpreting the results.