valida%e7%e3o de modelos para risco de cr%e9dito - christian kaiser

7/28/2019 Valida%E7%E3o de Modelos Para Risco de Cr%E9dito - Christian Kaiser

1/25

1


2/25

Credit Risk Model ValidationSelected Model Validation Challenges

2

Dr Christian Kaiser

Head of Independent Review HSBC Brazil 23 October 2012


3/25

Contents

1. Materiality of Defaults2. Rating System Granularity

3. Rating System Dimensionality

3

4. Multicollinearity and Omitted Variable Bias


4/25

Materiality of Defaults

4


5/25

Materiality of Defaults (1)

FSA default definition

[BIPRU 4.3.56 01/01/2007] A default must be considered to have occurred with regard to a particular obligor when

ac groun Central point for model validation is check if default definition used for modelling is correct

2 criteria according to regulation (1) defaults due to 90 dpd; (2) unlikeliness to pay

Contested question: Should there be a loss materiality threshold to consider defaults (in

application and/or modelling)?

either or both of the two following events has taken place:

the firm considers that the obligor is unlikely to pay its credit obligations to the firm, the parent undertaking

or any of its subsidiary undertakings in full, without recourse by the firm to actions such as realising security

(if held); and

the obligor is past due more than 90 days on any material credit obligation to the firm, the parent

undertaking or any of its subsidiary undertakings

Default definition generally very similar across regulators [aligned with BCBS, 2006, art. 452)]


6/25


FSA (CP06/3): Initial consideration - Specification of prescriptive exposure thresholds

(exclude smaller than 100 for retail, 1000 for non-retail)

After industry feedback (PS06/6) Thresholds can be defined by firms themselves, (CRSG

2008) but must be in relation to total exposures, and not in relation to overdue amounts

In practice threshold generally considered to be optional

Materiality of pre 90 dpd (qualitative) default discussion

Unlikeliness to a extracts of indicators

6

FSA Firm selling credit obligation at a material credit-related economic loss

Firm consenting to a distressed restructuring likely to result in a diminishedfinancial obligation caused by the material forgiveness or postponement ofprincipal, interest or fees.

Bacen A instituo vende, transfere ou renegocia com perda econmica relevante os

direitos de crdito relativos obrigao, devido a deteriorao significativa daqualidade to crdito do tomador ou contraparte


7/25


PD

LGD 10% 20% 30% 50%

10%3,6% 4,5% 4,7% 4,0%

20% 7,3% 9,0% 9,3% 8,1%

30% 10,9% 13,5% 14,0% 12,1%

50% 18,2% 22,5% 23,4% 20,1%

Cherry Picking, if defaults with low materiality are chosen wider default definition

conservative PD, but lower LGD could reduce RWA

Basically possible to choose wider default definition

(increase PD, decrease LGD reduce K/RWA)

K values

LGD 5% 10% 15% 20%20% 5,3% 7,3% 8,4% 9,0%

25% 6,6% 9,1% 10,5% 11,2%

30% 7,9% 10,9% 12,6% 13,5%

35% 9,2% 12,7% 14,7% 15,7%

7

reduce RWA (inverse cherry picking)

Total effect of wide/narrow default definition on RWA

depends on PD/LGD and PD/LGD combination

Likelihood that wide default definition reduces RWA. But effect uncertain (particularly forlow default portfolios the opposite can occur)


8/25


ercept on raz General insecurity of model developers in industry about correct default definition

regarding materiality

Model redevelopments have been/are being undertaken to address issue

Perception - alternative considered in industry: Exclude all or specific restructurings for

(retail) default modelling but no application of quantitative materiality filter

8


9/25


1. Removing all (or specific) types of restructurings in general without individual materiality

check

a. Unclear if this fully answers materiality question analyses suggest that restructurings

are on average associated with losses (average LGD may be lower than for 90dpddefaults, but can still be material)

b. Potential conflict with regulatory views in other countries (e.g. FSA)

c. Model will not be strong in predicting defaults due to restructurings

Recommendations

Questions: Are qualitative defaults unlikeliness to pay cases, are they (in parts) material?

1. Perform studies on significance and materiality: Percentage of restructured companies

that default 90 dpd, calculate LGD and absolute losses)

2. Sensitivity analysis of model parameter estimates and RWA with/without inclusion of

questionable cases9


10/25


.

Case

Unlikeliness to pay e.g. at 60 dpd triggers specific loss recoveries case not

considered for LGD model (as not 90 dpd)

But the same facility defaults at 90 dpd default -- then considered for LGD model

Potential inconsistencies

From which point of time should recoveries be counted?

1. Consider alternative: Use same materiality filter by level of loss for 90 dpd and qualitative

defaults

2. Get specific regulatory guidance on how to treat these cases

10


11/25


.

Recommendations

1. Simple alternative solutions would in principle be sufficient to address RWA (cherry

picking) concern for modelling:

a. Only LGD model filtering (most conservative solution: wide default definition for PD/Score

model; narrow default definition for LGD model )

b. For PD models (if needed): Filtering at calibration stage only

. s or specific regulatory guidance

a. Obtain certainty that applied default definition is acceptable by regulator before

model development/enhancement

b. Desirable to have explicit regulatory guidance communicated to all industry

(defensible to other foreign regulators and equal level playing field)

11


12/25

Rating Systems - Granularity

12


13/25

Rating Systems Granularity (1)

In Brazil Regulatory Requirement to use a rating system

Need to segment population into buckets of PD model scores and LGD bands:

10% 30% 50% 80% 5% 10% 20% 30% 40% 70% 80% 90%

1 2 3 4 5 6 7 8 9 10 11 12

0

1

2

LGD No Default

Pool 1Pool 5

GRUPO_LGD

GRUPO_PD

LGD Default

4

5

6

7

8

9

S/ Behavior

Default

Pool 4 Pool 8

Pool 9 Pool 10

Pool 2

Pool 7

Pool 3

13


14/25

Rating Systems Granularity (2)Re uirements for ratin s stems and Pool 1 p1

Safra 1

p11

Safra 2

p12

Safra 3

p13

Safra 4

p14...

Safra n

p1n

Pools

examples of suitable tests:Pool 2 p2

Pool 3 p3

Pool 4 p4

Pool 5 p5

Pool 6 p6

Pool 7 p7

18PontosdeObservaes

p1

p11

p12

p13p14

p1n

C.V.


15/25


Same score

Difficulty to prove all criteria in some portfolios

To fulfil all requirements can require a rating system with low granularity

Essential reason for problems: same model score is associated with a different DR

(typical for instance for TTC models, for structural data changes, or insufficient number

of variables,)

Also more likely in low default portfolios (large random volatility)!

12%

300

DR

t

95%

CI

eren s

over time

Avg. DRby buckets,12 months

forward;genericexample

15

0%

2%

4%

6%

8%

10%

200

801

200

803

200

805

200

807

200

809

200

811

200

901

200

903

200

905

200

907

200

909

200

911

201

001

201

003

201

005

201

007

201

009

201

011

IC 1 sup IC 1 inf IC 2 sup IC 2 inf bucket 1 bucket 2


16/25


RS PD cust A

RS with 3 buckets

PD cust B

RS PD cust B

RS PD cust A

Min Max Midpoint

0 0.33 0.165

0.33 0.67 0.5

0.67 1 0.835

IndividualPD vs RS PD

Case 1: PD stayswithin bucket limits

Significant

smoothing but over-,

or underestimation

Comparison

Case 2: PD crosses

bucket limits

Ra in S m

t

Non granular rating systems can have verynegative consequences! Average RS PD can be far away from

individual best estimate PDs PD and RWA volatility increases in parts

cus

PD cust B

RS PD cust B

(Bucket Mid-Points)

Choice of 2 typical cases

increases volatilityover time

16


17/25


Required number of buckets according to BCBS (2006, Art. 404):

For corporate, sovereign and bank exposures [] a bank must have a minimum of

seven borrower grades for non-defaulted borrowers and one for those that have

defaulted. [ ] supervisors may require banks, which lend to borrowers of diverse credit quality, to

have a greater number of borrower grades Build model and rating system jointly include stability considerations already in score

model development (choice of model factors, consider pooling models)

If a satisfactory improvement of the model stability is impossible despite all efforts

t

Validate with more tolerance to accept expectations for rating system (?)

Recommend monitoring and frequent re-alignment until better data vailable (?)

disadvantages of low granularity can outweigh advantages of meeting all criteria

RS with few buckets may fulfil regulatory requirement for retail, but should be avoideddue to severe disadvantages

17


18/25

Rating Systems - Dimensionality

18


19/25

Rating Systems Dimensionality (1)

IRB-A idea to have estimates of PD and LGD, independently of each other, such that the

model allows flexible combinations of PD and LGD

In case of rating system (RS) this implies a bi-

dimensional RS in matrix form

Complexity: It may often be difficult to prove allcriteria for this bi-dimensional rating system

10% 30% 50% 80% 5% 10% 2 0% 30% 4 0% 7 0% 8 0% 9 0%

1 2 3 4 5 6 7 8 9 10 11 12

0

1

2

3

4

5

6

7

8

9

S/ Behavior

Default

Pool 4 Pool 8

Pool 9 Pool 10

LGD No Default

Pool 1Pool 5

Pool 6

Pool 2

Pool 7

Pool 3

GRUPO_LGD

GRUPO_PDLGD Default

Solution: Regulation can be understood that in retail

it is ermitted to have onl a PD RS s stem with an

generic example

t

Art. 75. (Bacen Circular 3581) 3 Para as exposies classificadas na categoria "varejo", ovalor do parmetro LGD deve ser estimado para cada grupo

homogneo de risco, podendo ser obtido a partir das taxasde perdas observadas no longo prazo e do parmetro PD.

19

average LGD attached to each PD bucket RS

becomes one dimensional

1 0,5% 59%2 4% 59%

3 8% 60%

4 15% 59%

5 30% 60%

6 60% 61%7 100% 62%


20/25

Rating Systems Dimensionality (2)

Recommendations

1. Test rating systems separately

LGD model granularity is completely lost

RS becomes a PD RS only, with fixed LGD factors multiplied

In case of low or no PD-LGD correlation: LGD will not even differ significantly between

rating classes

Similar to IRB-F

t

Build 2 separate one dimensional rating systems (PD, LGD)

Test PD and LGD rating system requirements separately (but not necessarily for joint bi-

dimensional system together)

2. Request guidance from regulator if LGD RS really needed

Internationally common to have only a PD RS (if at all), but not an LGD RS

PD RS and individual LGD estimates

20


21/25

Multicollinearity and Omitted Variable Bias

21


22/25

Multicollinearity and Omitted Variable Bias (1)

Multicollinearity is effect that predictor variables are themselves highly correlated.

With full correlation between the predictor variables the OLS estimator cannot be calculated

This is unusual in practice, but a high correlation between the predictors is common

Consequences OLS estimator remains unbiased and is still the best

linear unbiased estimator (BLUE)

R2 statistic is unaffected

Y

Low confidence in the parameter estimate/low power of hypothesis testing

Parameter estimate may not be precise (e.g. in chart 2nd dimension uncertain)

Potential out of sample problems if correlation structure between variables changes

t

However, high variance in the parameter estimate

Effect similar to small sample - having low

variability in regressorsY=X1+aX2

22

X1X2


23/25

Multicollinearity and Omitted Variable Bias (2)1 2 3 4

e ec ng u co near y

1. Common to investigate Correlation Matrix: values > 0.8 or 0.9

are considered high in literature

Disadvantage:

only captures bi-variate correlation

Multicollinearity can still be present

2 1 1 04 2 0 3

6 0 4 2

8 3 1 2

2 0 0 1

-

1 2 3 4

1 1 0,6 0,4 0,62 0,6 1 -0,4 0,4

3 0,4 -0,4 1 0,1

4 0,6 0,4 0,1 1

Correl (Sum) 0,9

R2 0,85

VIF 6,8

3. Other alternative: Condition Index:

Square root of the ratio of the largest to the smallest characteristic root of XX

Rule of thumb: Value > 30 is harmful

23


24/25

Multicollinearity and Omitted Variable Bias (3)

Omitted Variable Bias occurs when a model is created which incorrectly leaves out one or

more important causal factors

Bias: model compensates for the missing factor by over- or underestimating one of the other

factors

Consequences Presence of omitted variable bias violates OLS assumption: Error and regressors are

correlated

LS ima r bi d and in on i n

t

Direction of the bias depends on estimators and covariance between regressors andomitted variables

Positive estimator and positive covariance OLS estimator will overestimate true

value

Omitted variable bias can be a consequence if variables are too generouslydiscarded for modelling (e.g. due to collinearity considerations)

24


25/25

Multicollinearity and Omitted Variable Bias (4) Recommendations

1. From a forecast point of view, multicollinearity issue is often overrated

Essential for hypothesis testing, not very concerning for prediction quality (estimator of

dependent variable remains BLUE)

In practice common to exclude variables already if bivariate correlation is > 0.5

Conservative exclusion of variables ok if variance of dependent variable can be explainedwith alternative variables. Otherwise, need to be cautious about automatically discarding

variables by trading this against other problems (too few variables, reduced prediction power,

higher risk of influence of data errors in single variables, use of variables with many missing

values, lower business acceptance)

2. Check multicollinearity measures such as VIF; bivariate correlations alone are insufficient

3. Severe multicollinearity should not be accepted. Recommend to reduce multicollinearity

(variable transformations, alternative variables)

4. More conservative variable exclusion recommended if expectation that correlation structure

changes out of sample/time5. Test alternative models, accepting higher levels of collinearity at least to benchmark

25

valida%e7%e3o de modelos para risco de cr%e9dito - christian kaiser

Documents