Early Warning Indicator Model of Financial Developments Using an Ordered Logit – Assessing the Robustness of the Model

Stability of the Coefficient Estimates
In order to check the stability of our results, we conduct two procedures. Firstly, we examine the time invariance. Secondly, the coefficient stability in respect of the pooled countries is investigated. Turning to the first approach, we start with a sample end in 2000Q4. In the next step the sample size is extended by four quarters. The last sample ended in 2011Q2. The results of this comparison are reported in Table 2. In all samples the signs of the coefficients are time invariant. Moreover, the coefficients are relatively stable. In line with an extended sample size, the standard errors of the coefficients are declining (given in parentheses).
The second approach analyses the stability of the coefficient regarding the used cross section members. Therefore, the pool varies in such a way that each country is excluded of the pool. The remaining countries establish the pool. In Table 3 the results are given. Each column has given a headline. This indicates which country is excluded. The following cell includes the estimate of a coefficient. It is apparent that the estimates are very stable. The exclusion of a country does not change the sign of any coefficient. Moreover, the magnitude of the coefficients is more or less the same.
Testing for the Impact of Different Forecasting Horizons
It can be argued that a forecasting horizon of 1 quarter is too small. Therefore, the forecasting horizon should be extended up to five quarters. The results are reported in Table 4. It can be noticed that the information content of the variables are not stable. The coefficients of credit gaps increase, however its sum becomes smaller. The size of the coefficient of the house price gap is slightly higher. But the coefficient of the house price change is not stable. Increasing the forecasting horizon reduces its coefficient. It becomes insignificant. The same difficulties are apparent for the stock price changes. Their coefficients include short term affects. However, the spread coefficient is stable. As expected, the pseudo R2 declines with increasing forecasting horizon.
Determining the Threshold
In the previous analyses we used as decision rule the maximum of predicted probability to determine the category. This assumes an equal weight of the decision maker for all categories. Having in mind the high costs of financial crisis manifested in form of large output losses, rising unemployment and huge public deficits it is reasonable to assume that decision makers give the crisis a higher weight. Assuming that they are concentrated on crisis they may have a lost function L(T) of the form:
L(T) = 0 PrNS (T) + (1 -0) PrS (T)    (11)
with PrNS (T) as the probability of a missing a crisis and PrS (T) as the probability of issuing the signal that a crisis will occur. 0can be interpreted as the relative cost of missing a crisis or the decision maker’s degree of relative risk-aversion of missing a crisis. The results of losses do not only depend on the risk aversion but also on the threshold T. To show the implications of different thresholds the decision rule is changed. The signal is given regarding the maximum of the predicted probability of booms and busts. If the maximum of these two categories is greater than the threshold a signal for this category is given. If the maximum is lower a signal of a normal period is taken. The results for different T are given in the Table 5. It is apparent that an increase of the threshold value raises the number of correct signals over all categories. However, concentration on crisis a lower threshold is sensible. In contrast an equal weight of all three phases would imply a high threshold. The number of correct signals is at maximum for T = .50. This measure has the drawback that it is dominated by the good results for the normal phase. This is avoided by using a modification relying on the definition of conditional forecast. The CP/s are calculated as the proportion of correct predictions divided by the total of each row. This modifies the measure of predictive ability to discount the influence of the dominant outcome. Only when the predictor is accurate for all categories it will obtain a high CP value. The empirical values are given in last row of Table 5. The best value is obtained for the threshold of .10. The main conclusion that can be drawn from this line compared to the line above is that the outcome is highly dominated by the normal phase.

Table 2. Coefficient estimates of different sample ends

Variable End of sample
2000q4 2001q4 2002q4 2003q4 2004q4 2005q4 2006q4 2007q4 2008q4 2009q4 2010q4 2011q2
D4_ncl_gap_1 .352(.091) .339(.088) .355(.087) .362(.087) .356(.087) .334(.084) .383(.082) .328(.079) .427(.075) .418(.075) .410(.074) .404(.074)
D4_ncl_gap_2 -.293(.089) -.277(.087) -.292(.086) -.298(.086) -.286(.086) -.266(.083) -.288(.081) -.193(.078) -.345(.074) -.331(.074) -.325(.073) -.320(.073)
D4_nhp_gap_1 .108(.017) .106(.016) .103(.016) .103(.016) .102(.016) .108(.016) .104(.016) .077(.015) .086(.014) .085(.014) .087(.014) .087(.014)
D1_nhp_1 .189(.033) .193(.033) .190(.033) .190(033) .189(.033) .180(.032) .168(.032) .181(.031) .210(.029) .203(.029) .202(.029) .200(.029)
D1_nhp_4 -.100(.035) -.094(.034) -.090(.034) -.095(.034) -.092(.034) -.093(.034) -.085(.033) -.063(.032) -.057(.030) -.062(.030) -.060(.030) -.059)(.029)
D4_n_sto_1 .032(.006) .032(.005) .032(.005) .031(.005) .031(.005) .030(.005) .028(.005) .028(.005) .037(.005) .035(.005) .034(.004) .034(.004)
D4_n_sto_2 -.020(.005) -.019(.033) -.018(.005) -.016(.005) -.016(.005) -.016(.005) -.013(.005) -.014(.005) -.020(.005) -.021(.004) -.019(.004) -.019(.004)
Spread_1 .116(.028) .119(.028) .112(.028) .114(.028) .112(.028) .106(.028) .089(.028) .064(.027) .068(.026) .077(.026) .082(.026) .084(.025)
D1_hicp_1 -.292(.061) -.287(.060) -.261(.059) -.256(.058) -.252(.059) -.245(.058) -.279(.057) -.361(.055) -.301(.052) -.327(.051) -.339(.051) -.340(.051)
Limit_2 -3.191 -3.155 -3.070 -3.099 -3.101 -3.109 -3.212 -3.285 -2.825 -2.938 -2.981 -2.986
Limit_3 2.760 2.793 2.876 2.918 2.969 2.952 2.772 2.537 2.721 2.645 2.652 2.646
Pseudo R2 .158 .155 .152 .152 .151 .150 .160 .162 .165 .164 .163 .162

Table 3. Coefficient estimates of reduced cross section dimension

Variable Pooled estimates without specified country
-U2 -au -de -jp -uk -us -ca -ch -dk -es -fr -ie -it -nl -se -no -nz -pt
D4_ncl_g_1 .400 .391 .409 .407 .402 .351 .390 .410 .420 .437 .417 .421 .398 .405 .491 .376 .341 .433
D4_ncl_g_2 -.318 -.304 -.326 -.323 -.322 -.266 -.303 -.328 -.341 -.350 -.326 -.338 -.314 -.323 -.399 -.288 -.252 -.355
D4_nhp_g_1 .083 .092 .091 .093 .096 .086 .088 .088 .091 .074 .079 .090 .092 .081 .074 .092 .089 .087
D1_nhp_1 .204 .196 .191 .183 .192 .197 .199 .210 .204 .212 .207 .218 .190 .208 .208 .197 .192 .197
D1_nhp_4 -.060 -.068 -.071 -.061 -.064 -.062 -.054 -.059 -.068 -.024 -.049 -.055 -.057 -.073 -.050 -.059 -.058 -.060
D4_n_sto_1 .033 .034 .033 .033 .033 .034 .035 .033 .032 .034 .034 .034 .037 .034 .033 .036 .034 .034
D4_n_sto_2 -.018 -.019 -.017 -.020 -.019 -.019 -.020 -.019 -.016 -.020 -.021 -.019 -.022 -.019 -.020 -.020 -.018 -.019
Spread_1 .088 .084 .085 .084 .095 .093 .074 .075 .096 .082 .090 .076 .088 .063 .074 .099 .086 .080
D1_hicp_1 -.335 -.336 -.356 -.335 -.335 -.310 -.341 -.336 -.317 -.315 -.363 -.375 -.360 -.349 -.368 -.345 -.317 -.330
Pseudo R2 .161 .167 .167 .162 .159 .161 .166 .165 .158 .161 .164 .164 .167 .156 .163 .164 .160 .162

Table 4. Coefficients and statistics of the preferred specification for extended forecast horizons

Forecasting horizon
Variable 1 2 3 4 5
D4 ncl gap 1 .404(.074) .586(.074) .722(.074) .800(.075) .770(.074)
D4 ncl gap 2 -.320(.073) -.521(.073) -.678(.073) -.781(.073) -.778(.073)
D4 nhp gap 1 .087(.014) .091(.014) .106(.014) .121(014) .094(.014)
D1 nhp 1 .200(.029) .153(.028) .084(.028) -.031(.029) -.019(.033)
D1 nhp 4 -.059(.029) -.072(.029) -.118(.029) -.128(.029) -.079(.029)
D4 n sto 1 .034(.004) .024(.004) .017(.004) .006(.004) -.005(.004)
D4 n sto 2 -.019(.004) -.019(.004) -.018(.004) -.012(.004) -.003(.004)
Spread 1 .084(.025) .095(.025) .101(.025) .099(.025) .089(.025)
D1 hicp 1 -.340(.051) -.279(.051) -.164(.051) -.038(.051) -.050(.051)
Limit 2 -2.986 -2.981 -3.012 -3.021 -2.881
Limit 3 2.646 2.458 2.281 2.117 2.110
Pseudo R2 .162 .136 .116 .096 .074

Table 5. Results of signals for different thresholds

Category Signal Threshold
.50 .45 .40 .35 .30 .25 .20 .15 .11 .10 .09
Boom Correct 36 40 46 55 79 98 130 169 207 217 224
False 14 23 35 62 95 146 234 371 624 706 812
Normal Correct 2152 2136 2109 2060 1996 1909 1768 1524 1102 971 800
False 441 432 418 403 368 344 284 222 133 106 84
Bust Correct 9 14 21 27 37 41 66 85 126 139 150
False 18 25 41 63 95 132 188 299 478 531 600
Sum Correct 2197 2190 2176 2142 2112 2048 1964 1778 1435 1327 1174
CP .082 .097 .118 .137 .191 .216 .300 .361 .427 .445 .443