Review of estimates
Institutions should specify internal policies for changes of models and estimates of risk parameters used within a rating system. Such policies should provide that changes in the models should be made as a result of at least the following:
Where material deficiencies are identified as a result of the procedures referred to in paragraph 215 institutions should take appropriate actions depending on the severity of the deficiency and apply a MoC in accordance with section 4.4.3.
For the purpose of regular reviews of estimates, institutions should have a framework in place which includes at least the following elements:
a minimum scope and frequency of analyses to be performed, including predefined metrics chosen by the institution to test data representativeness, model performance, its predictive power and stability;
predefined standards, including predefined thresholds and significance levels for the relevant metrics;
predefined actions to be taken in case of adverse results of the review, depending on the severity of the deficiency.
In their regular reviews of estimates institutions may rely on the results of independent validation where such results are up to date.
The reviews of estimates to be performed at least annually in accordance with Article 179(1)(c) of Regulation (EU) No 575/2013 should be performed taking into account the metrics, standards and thresholds defined by the institution in accordance with paragraph 217. The scope of such reviews should comprise at least the following elements:
an analysis of data representativeness, including all of the following:
an analysis of potential differences between the RDS used to quantify the risk parameter and the application portfolio, including the analysis of any changes in the portfolio or any structural breaks, in the manners of analysing the representativeness described in section 4.2.4;
an analysis of potential differences between the RDS used to develop the model and the application portfolio; for this purpose institutions should:
perform the analysis set out in paragraphs 24, 25, and 26;
consider that data used for model development is sufficiently representative in terms of points (a) and (b) of paragraph 21 if the performance of the model in the sense of paragraph 218(b) is sound;
perform the analysis set out in paragraphs 22 and 23 where the performance of the model in the sense of paragraph 218(b) is deteriorating;
an analysis of the performance of the model and its stability over time, which should have both of the following characteristics:
the analysis should identify any potential deterioration of the model performance, including the model’s discriminatory power, through the comparison of its performance at the time of the development against its performance on each subsequent observation period of the extended data set as well as against the predefined thresholds; this analysis should be performed on relevant subsets, for instance with and without delinquency status in the case of PD estimates, or for various recovery scenarios in the case of LGD estimates;
the analysis should be performed with regard to the whole application portfolio, without any data adjustments or exclusions performed in model development; for comparison purposes, the performance at the time of development should also be obtained for the whole application portfolio, prior to any data adjustments or exclusions;
an analysis of the predictive power of the model, including at least:
an analysis of whether the inclusion of the most recent data in the dataset used to estimate risk parameters leads to materially different risk estimates and in particular:
for PD, whether including the most recent data leads to a significant change in the long-run average default rate; this analysis should take into account the appropriate redefinition of the period of likely range of variability of default rates and of the mix of good and bad years, if necessary;
for LGD, whether including the most recent data leads to a significant change in the long-run average LGD or downturn LGD;
a back-testing analysis, which should include a comparison of the estimates used for the calculation of own funds requirements against observed outcomes for each grade or pool; for this purpose institutions may take into account the results of back-testing performed as part of the internal validation in accordance with Article 185(b) of Regulation (EU) No 575/2013 or they may perform additional tests, for instance with regard to a different timeframe of the dataset.
Institutions should specify conditions under which the analyses referred to in paragraph 218 should be performed more frequently than annually, such as major changes in the risk profile of the institution, credit policies or relevant IT systems. Institutions should perform the review of the PD or LGD model whenever they observe significant change in economic conditions as compared with the economic conditions underlying the dataset used for the purpose of model development.
For the purpose of performing the tasks referred to in Article 190(2) of Regulation (EU) No 575/2013 institutions should define a regular cycle for the full review of the rating systems, taking into consideration their materiality, and covering all aspects of model development, quantification of risk parameters and, where applicable, the estimation of model components. This review should include all of the following:
a review of the existing and potential risk drivers and an assessment of their significance based on the predefined standards of review referred to in paragraph 217;
an assessment of the modelling approach, its conceptual soundness, the fulfilment of the modelling assumptions and alternative approaches.
Where the results of this review recommend changes to model design, appropriate actions should be taken following the results from this analysis.
For the purpose of the review specified in paragraphs 217 to 220 institutions should apply consistent policies for data adjustments and exclusions and ensure that any differences in the policies applied to the relevant datasets are justified and do not distort the results of the review.