Upload
jtnylson
View
220
Download
0
Embed Size (px)
Citation preview
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
1/25
1
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
2/25
Credit Risk Model ValidationSelected Model Validation Challenges
2
Dr Christian Kaiser
Head of Independent Review HSBC Brazil 23 October 2012
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
3/25
Contents
1. Materiality of Defaults2. Rating System Granularity
3. Rating System Dimensionality
3
4. Multicollinearity and Omitted Variable Bias
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
4/25
Materiality of Defaults
4
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
5/25
Materiality of Defaults (1)
FSA default definition
[BIPRU 4.3.56 01/01/2007] A default must be considered to have occurred with regard to a particular obligor when
ac groun Central point for model validation is check if default definition used for modelling is correct
2 criteria according to regulation (1) defaults due to 90 dpd; (2) unlikeliness to pay
Contested question: Should there be a loss materiality threshold to consider defaults (in
application and/or modelling)?
either or both of the two following events has taken place:
the firm considers that the obligor is unlikely to pay its credit obligations to the firm, the parent undertaking
or any of its subsidiary undertakings in full, without recourse by the firm to actions such as realising security
(if held); and
the obligor is past due more than 90 days on any material credit obligation to the firm, the parent
undertaking or any of its subsidiary undertakings
Default definition generally very similar across regulators [aligned with BCBS, 2006, art. 452)]
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
6/25
Materiality of Defaults (2)
FSA (CP06/3): Initial consideration - Specification of prescriptive exposure thresholds
(exclude smaller than 100 for retail, 1000 for non-retail)
After industry feedback (PS06/6) Thresholds can be defined by firms themselves, (CRSG
2008) but must be in relation to total exposures, and not in relation to overdue amounts
In practice threshold generally considered to be optional
Materiality of pre 90 dpd (qualitative) default discussion
Unlikeliness to a extracts of indicators
6
FSA Firm selling credit obligation at a material credit-related economic loss
Firm consenting to a distressed restructuring likely to result in a diminishedfinancial obligation caused by the material forgiveness or postponement ofprincipal, interest or fees.
Bacen A instituo vende, transfere ou renegocia com perda econmica relevante os
direitos de crdito relativos obrigao, devido a deteriorao significativa daqualidade to crdito do tomador ou contraparte
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
7/25
Materiality of Defaults (3)
PD
LGD 10% 20% 30% 50%
10%3,6% 4,5% 4,7% 4,0%
20% 7,3% 9,0% 9,3% 8,1%
30% 10,9% 13,5% 14,0% 12,1%
50% 18,2% 22,5% 23,4% 20,1%
Cherry Picking, if defaults with low materiality are chosen wider default definition
conservative PD, but lower LGD could reduce RWA
Basically possible to choose wider default definition
(increase PD, decrease LGD reduce K/RWA)
K values
LGD 5% 10% 15% 20%20% 5,3% 7,3% 8,4% 9,0%
25% 6,6% 9,1% 10,5% 11,2%
30% 7,9% 10,9% 12,6% 13,5%
35% 9,2% 12,7% 14,7% 15,7%
7
reduce RWA (inverse cherry picking)
Total effect of wide/narrow default definition on RWA
depends on PD/LGD and PD/LGD combination
Likelihood that wide default definition reduces RWA. But effect uncertain (particularly forlow default portfolios the opposite can occur)
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
8/25
Materiality of Defaults (4)
ercept on raz General insecurity of model developers in industry about correct default definition
regarding materiality
Model redevelopments have been/are being undertaken to address issue
Perception - alternative considered in industry: Exclude all or specific restructurings for
(retail) default modelling but no application of quantitative materiality filter
8
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
9/25
Materiality of Defaults (5)
1. Removing all (or specific) types of restructurings in general without individual materiality
check
a. Unclear if this fully answers materiality question analyses suggest that restructurings
are on average associated with losses (average LGD may be lower than for 90dpddefaults, but can still be material)
b. Potential conflict with regulatory views in other countries (e.g. FSA)
c. Model will not be strong in predicting defaults due to restructurings
Recommendations
Questions: Are qualitative defaults unlikeliness to pay cases, are they (in parts) material?
1. Perform studies on significance and materiality: Percentage of restructured companies
that default 90 dpd, calculate LGD and absolute losses)
2. Sensitivity analysis of model parameter estimates and RWA with/without inclusion of
questionable cases9
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
10/25
Materiality of Defaults (6)
.
Case
Unlikeliness to pay e.g. at 60 dpd triggers specific loss recoveries case not
considered for LGD model (as not 90 dpd)
But the same facility defaults at 90 dpd default -- then considered for LGD model
Potential inconsistencies
From which point of time should recoveries be counted?
1. Consider alternative: Use same materiality filter by level of loss for 90 dpd and qualitative
defaults
2. Get specific regulatory guidance on how to treat these cases
10
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
11/25
Materiality of Defaults (7)
.
Recommendations
1. Simple alternative solutions would in principle be sufficient to address RWA (cherry
picking) concern for modelling:
a. Only LGD model filtering (most conservative solution: wide default definition for PD/Score
model; narrow default definition for LGD model )
b. For PD models (if needed): Filtering at calibration stage only
. s or specific regulatory guidance
a. Obtain certainty that applied default definition is acceptable by regulator before
model development/enhancement
b. Desirable to have explicit regulatory guidance communicated to all industry
(defensible to other foreign regulators and equal level playing field)
11
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
12/25
Rating Systems - Granularity
12
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
13/25
Rating Systems Granularity (1)
In Brazil Regulatory Requirement to use a rating system
Need to segment population into buckets of PD model scores and LGD bands:
10% 30% 50% 80% 5% 10% 20% 30% 40% 70% 80% 90%
1 2 3 4 5 6 7 8 9 10 11 12
0
1
2
LGD No Default
Pool 1Pool 5
GRUPO_LGD
GRUPO_PD
LGD Default
4
5
6
7
8
9
S/ Behavior
Default
Pool 4 Pool 8
Pool 9 Pool 10
Pool 2
Pool 7
Pool 3
13
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
14/25
Rating Systems Granularity (2)Re uirements for ratin s stems and Pool 1 p1
Safra 1
p11
Safra 2
p12
Safra 3
p13
Safra 4
p14...
Safra n
p1n
Pools
examples of suitable tests:Pool 2 p2
Pool 3 p3
Pool 4 p4
Pool 5 p5
Pool 6 p6
Pool 7 p7
18PontosdeObservaes
p1
p11
p12
p13p14
p1n
C.V.
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
15/25
Rating Systems Granularity (3)
Same score
Difficulty to prove all criteria in some portfolios
To fulfil all requirements can require a rating system with low granularity
Essential reason for problems: same model score is associated with a different DR
(typical for instance for TTC models, for structural data changes, or insufficient number
of variables,)
Also more likely in low default portfolios (large random volatility)!
12%
300
DR
t
95%
CI
eren s
over time
Avg. DRby buckets,12 months
forward;genericexample
15
0%
2%
4%
6%
8%
10%
200
801
200
803
200
805
200
807
200
809
200
811
200
901
200
903
200
905
200
907
200
909
200
911
201
001
201
003
201
005
201
007
201
009
201
011
IC 1 sup IC 1 inf IC 2 sup IC 2 inf bucket 1 bucket 2
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
16/25
Rating Systems Granularity (4)
RS PD cust A
RS with 3 buckets
PD cust B
RS PD cust B
RS PD cust A
Min Max Midpoint
0 0.33 0.165
0.33 0.67 0.5
0.67 1 0.835
IndividualPD vs RS PD
Case 1: PD stayswithin bucket limits
Significant
smoothing but over-,
or underestimation
Comparison
Case 2: PD crosses
bucket limits
Ra in S m
t
Non granular rating systems can have verynegative consequences! Average RS PD can be far away from
individual best estimate PDs PD and RWA volatility increases in parts
cus
PD cust B
RS PD cust B
(Bucket Mid-Points)
Choice of 2 typical cases
increases volatilityover time
16
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
17/25
Rating Systems Granularity (5)
Required number of buckets according to BCBS (2006, Art. 404):
For corporate, sovereign and bank exposures [] a bank must have a minimum of
seven borrower grades for non-defaulted borrowers and one for those that have
defaulted. [ ] supervisors may require banks, which lend to borrowers of diverse credit quality, to
have a greater number of borrower grades Build model and rating system jointly include stability considerations already in score
model development (choice of model factors, consider pooling models)
If a satisfactory improvement of the model stability is impossible despite all efforts
t
Validate with more tolerance to accept expectations for rating system (?)
Recommend monitoring and frequent re-alignment until better data vailable (?)
disadvantages of low granularity can outweigh advantages of meeting all criteria
RS with few buckets may fulfil regulatory requirement for retail, but should be avoideddue to severe disadvantages
17
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
18/25
Rating Systems - Dimensionality
18
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
19/25
Rating Systems Dimensionality (1)
IRB-A idea to have estimates of PD and LGD, independently of each other, such that the
model allows flexible combinations of PD and LGD
In case of rating system (RS) this implies a bi-
dimensional RS in matrix form
Complexity: It may often be difficult to prove allcriteria for this bi-dimensional rating system
10% 30% 50% 80% 5% 10% 2 0% 30% 4 0% 7 0% 8 0% 9 0%
1 2 3 4 5 6 7 8 9 10 11 12
0
1
2
3
4
5
6
7
8
9
S/ Behavior
Default
Pool 4 Pool 8
Pool 9 Pool 10
LGD No Default
Pool 1Pool 5
Pool 6
Pool 2
Pool 7
Pool 3
GRUPO_LGD
GRUPO_PDLGD Default
Solution: Regulation can be understood that in retail
it is ermitted to have onl a PD RS s stem with an
generic example
t
Art. 75. (Bacen Circular 3581) 3 Para as exposies classificadas na categoria "varejo", ovalor do parmetro LGD deve ser estimado para cada grupo
homogneo de risco, podendo ser obtido a partir das taxasde perdas observadas no longo prazo e do parmetro PD.
19
average LGD attached to each PD bucket RS
becomes one dimensional
1 0,5% 59%2 4% 59%
3 8% 60%
4 15% 59%
5 30% 60%
6 60% 61%7 100% 62%
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
20/25
Rating Systems Dimensionality (2)
Recommendations
1. Test rating systems separately
LGD model granularity is completely lost
RS becomes a PD RS only, with fixed LGD factors multiplied
In case of low or no PD-LGD correlation: LGD will not even differ significantly between
rating classes
Similar to IRB-F
t
Build 2 separate one dimensional rating systems (PD, LGD)
Test PD and LGD rating system requirements separately (but not necessarily for joint bi-
dimensional system together)
2. Request guidance from regulator if LGD RS really needed
Internationally common to have only a PD RS (if at all), but not an LGD RS
PD RS and individual LGD estimates
20
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
21/25
Multicollinearity and Omitted Variable Bias
21
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
22/25
Multicollinearity and Omitted Variable Bias (1)
Multicollinearity is effect that predictor variables are themselves highly correlated.
With full correlation between the predictor variables the OLS estimator cannot be calculated
This is unusual in practice, but a high correlation between the predictors is common
Consequences OLS estimator remains unbiased and is still the best
linear unbiased estimator (BLUE)
R2 statistic is unaffected
Y
Low confidence in the parameter estimate/low power of hypothesis testing
Parameter estimate may not be precise (e.g. in chart 2nd dimension uncertain)
Potential out of sample problems if correlation structure between variables changes
t
However, high variance in the parameter estimate
Effect similar to small sample - having low
variability in regressorsY=X1+aX2
22
X1X2
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
23/25
Multicollinearity and Omitted Variable Bias (2)1 2 3 4
e ec ng u co near y
1. Common to investigate Correlation Matrix: values > 0.8 or 0.9
are considered high in literature
Disadvantage:
only captures bi-variate correlation
Multicollinearity can still be present
2 1 1 04 2 0 3
6 0 4 2
8 3 1 2
2 0 0 1
-
1 2 3 4
1 1 0,6 0,4 0,62 0,6 1 -0,4 0,4
3 0,4 -0,4 1 0,1
4 0,6 0,4 0,1 1
Correl (Sum) 0,9
R2 0,85
VIF 6,8
3. Other alternative: Condition Index:
Square root of the ratio of the largest to the smallest characteristic root of XX
Rule of thumb: Value > 30 is harmful
23
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
24/25
Multicollinearity and Omitted Variable Bias (3)
Omitted Variable Bias occurs when a model is created which incorrectly leaves out one or
more important causal factors
Bias: model compensates for the missing factor by over- or underestimating one of the other
factors
Consequences Presence of omitted variable bias violates OLS assumption: Error and regressors are
correlated
LS ima r bi d and in on i n
t
Direction of the bias depends on estimators and covariance between regressors andomitted variables
Positive estimator and positive covariance OLS estimator will overestimate true
value
Omitted variable bias can be a consequence if variables are too generouslydiscarded for modelling (e.g. due to collinearity considerations)
24
7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser
25/25
Multicollinearity and Omitted Variable Bias (4) Recommendations
1. From a forecast point of view, multicollinearity issue is often overrated
Essential for hypothesis testing, not very concerning for prediction quality (estimator of
dependent variable remains BLUE)
In practice common to exclude variables already if bivariate correlation is > 0.5
Conservative exclusion of variables ok if variance of dependent variable can be explainedwith alternative variables. Otherwise, need to be cautious about automatically discarding
variables by trading this against other problems (too few variables, reduced prediction power,
higher risk of influence of data errors in single variables, use of variables with many missing
values, lower business acceptance)
2. Check multicollinearity measures such as VIF; bivariate correlations alone are insufficient
3. Severe multicollinearity should not be accepted. Recommend to reduce multicollinearity
(variable transformations, alternative variables)
4. More conservative variable exclusion recommended if expectation that correlation structure
changes out of sample/time5. Test alternative models, accepting higher levels of collinearity at least to benchmark
25