PROPOSED ACTUARIAL STANDARD OF PRACTICE

Modeling

 THIRD EXPOSURE DRAFT

TRANSMITTAL MEMORANDUM

June 2016

TO: Members of Actuarial Organizations Governed by the Standards of Practice of the Actuarial Standards Board and Other Persons Interested in Modeling

FROM: Actuarial Standards Board (ASB)

SUBJ: Proposed Actuarial Standard of Practice (ASOP) on Modeling

This document contains a third exposure draft of a proposed ASOP titled Modeling. Please review this exposure draft and give the ASB the benefit of your comments and suggestions. Each written response and each response sent by e-mail to the address below will be acknowledged, and all responses will receive appropriate consideration by the drafting committee in preparing the final document for approval by the ASB.

The ASB accepts comments by either electronic or conventional mail. The preferred form is e-mail, as it eases the task of grouping comments by section. However, please feel free to use either form. If you wish to use e-mail, please send a message to comments@actuary.org. You may include your comments either in the body of the message or as an attachment prepared in any commonly used word processing format. Please do not password protect any attachments. If the attachment is in the form of a PDF, please do not copy protect the PDF. Include the phrase “ASB COMMENTS” in the subject line of your message. Please note: Any message not containing this exact phrase in the subject line will be deleted by our system’s spam filter. Also please indicate in the body of the e-mail if your comments are being submitted on your own behalf or on behalf of a company or organization.

If you wish to use conventional mail, please send comments to the following address:

Modeling (Third Exposure)
Actuarial Standards Board
1850 M Street, Suite 300
Washington, DC 20036

The ASB posts all signed comments received to its website to encourage transparency and dialogue. Unsigned or anonymous comments will not be considered by the ASB nor posted to the website. The comments will not be edited, amended, or truncated in any way. Comments will be posted in the order that they are received. Comments will be removed when final action on a proposed standard is taken. The ASB website is a public website, and all comments will be available to the general public. The ASB disclaims any responsibility for the content of the comments, which are solely the responsibility of those who submit them.

Deadline for receipt of responses in the ASB office: October 31, 2016

Background

The ASB first began work on a standard for modeling in the late 1990s. Motivated primarily to address the role catastrophe modeling of earthquakes and hurricanes played in casualty ratemaking, this work was focused on the use of specialized models where actuaries would have to rely on a model that was developed by professionals other than actuaries. As a result of this work, ASOP No. 38, Using Models Outside the Actuary’s Area of Expertise, was approved by the ASB in June of 2000 with the scope of the standard limited to the Property/Casualty area of practice. Historically, ASOP No. 38 had been the only ASOP that specifically addressed modeling.

Recently, the number and importance of modeling applications in actuarial science has increased, with the results of actuarial models often entering financial statements directly. Recognizing this trend, the ASB asked the Life Committee in 2010 to begin work on an ASOP focused on modeling. The Life Committee formed a task force to address this issue and, in February of 2012, a discussion draft titled Modeling in Life Insurance and Annuities was released and nineteen comment letters were received. The transmittal letter also mentioned that the scope might be expanded to all practice areas and asked for comments on this idea.

Based upon the feedback received, and numerous other discussions on the topic of modeling, in December of 2012 the ASB created two multi-disciplinary task forces under the direction of the General Committee: i) a general Modeling Task Force, charged with developing an ASOP to address modeling applications in all practice areas, and ii) a Catastrophe Modeling Task Force to consider expanding ASOP No. 38 to all practice areas while focusing exclusively on using catastrophe models. The membership of these task forces has experience in all actuarial practice areas, including enterprise risk management.

As the guidance in this proposed modeling ASOP and ASOP No. 38 currently titled Catastrophe Modeling (for All Practice Areas) is intended to be coordinated, the ASB will issue final versions of both ASOPs to be effective concurrently. To facilitate review of this proposed modeling ASOP, a link to the current working draft of ASOP No. 38 is provided here for your information. The working draft of ASOP No. 38 is not being exposed for comment but does reflect guidance that the ASB and General Committee believe works in concert with the guidance in the third exposure draft of this proposed modeling ASOP.

First Exposure Draft

The first exposure draft titled Modeling was released in June 2013 with a comment deadline of September 30, 2013. Forty-eight comment letters were received and considered in making changes that were reflected in the second exposure draft.

Second Exposure Draft

In November 2014, the ASB approved a second exposure draft with a comment deadline of March 1, 2015. Thirty-seven comment letters were received and considered in making changes that were reflected in this third exposure draft. For a summary of issues contained in these comment letters, please see appendix 2.

Key Changes

Changes made to the second exposure draft in response to the comment letters include the following:

1. The scope was narrowed but, within that scope, the guidance is less subject to professional judgement as to its applicability.

2. Distinctions within section 2.6, Intended Purpose, were removed between “intended application” and “project objective.”

3. The definitions were clarified, particularly for “model,” “data,” and “model run.”

4. The guidance with respect to using models designed or built by others was clarified.

5. The guidance with respect to an actuary with a role on a modeling team was clarified.

6. The guidance in section 3.4.5 related to model structure was clarified.

Given the extensive clarifications, the ASB believes it would be appropriate to obtain additional feedback on the proposed Modeling ASOP through the issuance of this third exposure draft. The ASB thanks everyone who took the time to contribute comments and suggestions on the first and second exposure drafts.

Request for Comments

The ASB would appreciate comments on all areas of this proposed standard and would like to draw the reader’s attention in particular to the following questions:

1. Does the proposed standard provide sufficient and appropriate guidance to actuaries working with models? If not, what suggestions do you recommend for improving the guidance?

2. Does the proposed standard provide sufficient and appropriate guidance to actuaries working with all types of models, including financial projection models, predictive models, and statistical models?

3. The scope of the proposed ASOP excludes “simple” models, which are defined in section 2.13. Is this definition appropriate and sufficiently clear?

4. Section 3.2 requires the actuary to make practical efforts to comply with applicable sections of this standard with respect to models designed or built by someone else, such as a vendor or a colleague, when the actuary has a limited ability to obtain information about the model or to understand the underlying workings of the model. Is this guidance appropriate and clear?

5. Does any guidance in this exposure draft conflict with the guidance in the proposed working draft of ASOP No. 38, Catastrophe Modeling (for All Practice Areas)?

The ASB voted in June 2016 to approve this third exposure draft.

Modeling Task Force

Dale S. Hagstrom, Chairperson

                      Maryellen J. Coggins                                              Kenneth R. Kasner

                      Julie H. Fried                                                            Aaron R. Weindling

General Committee

Maria M. Sarli, Chairperson

                      Shawna S. Ackerman                                           John C. Lloyd

                      Ralph S. Blanchard III                                           Margaret Tiller Sherwood

                      Raymond Brouilette                                              Mary Simmons

                      David Driscoll                                                        Thomas D. Snook

                      Dale S. Hagstrom

Actuarial Standards Board

Maryellen J. Coggins, Chairperson

                      Christopher S. Carlson                                           Kathleen A. Riley

                      Beth E. Fitzgerald                                                   Barbara L. Snyder

                      Darrell D. Knapp                                                     Frank Todisco

                      Cande J. Olsen                                                      Ross A. Winkelman

The Actuarial Standards Board (ASB) sets standards for appropriate actuarial practice in the United States through the development and promulgation of Actuarial Standards of Practice (ASOPs). These ASOPs describe the procedures an actuary should follow when performing actuarial services and identify what the actuary should disclose when communicating the results of those services.

Section 1. Purpose, Scope, Cross References, And Effective Date

1.1 Purpose

This actuarial standard of practice (ASOP) provides guidance to actuaries selecting, designing, building, modifying, developing, using, reviewing, or evaluating models when performing actuarial services.

1.2 Scope

This ASOP applies to actuaries in all practice areas performing actuarial services when selecting, designing, building, modifying, developing, using, reviewing, or evaluating all types of models that are not simple models.

If the model results are not heavily relied upon by the intended user, or do not have material financial effect, the requirements of this ASOP are limited to certain disclosure requirements in section 3.1.

If the actuary departs from the guidance set forth in this ASOP in order to comply with applicable law (statutes, regulations, and other legally binding authority), or for any other reason, the actuary should refer to section 4.

1.3 Cross References

When this ASOP refers to the provisions of other documents, the reference includes the referenced documents as they may be amended or restated in the future, and any successor to them, by whatever name called. If any amended or restated document differs materially from the originally referenced document, the actuary should consider the guidance in this ASOP to the extent it is applicable and appropriate.

1.4 Effective Date

This ASOP is effective for work performed on or after nine months after adoption by the Actuarial Standards Board.

Section 2. Definitions

The terms below are defined for use in this actuarial standard of practice.

2.1 Assumptions

A type of input to a model that represents expectations or possibilities based on professional judgment, or that may be prescribed by law or by others.

2.2 Data

Facts or information that are either direct input to a model or inform the selection of input; data may be collected from sources such as records, experience, experiments, surveys, observations, or outputs from other models.

2.3 Granularity

The level of detail built into a model.

2.4 Implementation

An executable form of a model.

2.5 Input

Information such as data, assumptions, or parameters used in a model to produce output.

2.6 Intended Purpose

The planned uses for the model or the specific goal or question addressed or both, depending on the actuary’s role at the time actuarial services are performed to meet the needs of the principal or the actuary. The “planned uses” definition applies if the actuary’s role includes designing, building, or developing the model, or if the actuary’s role includes modifying, reviewing or evaluating the model before being selected or used in a specific project. The “specific goal or question addressed” definition applies if the actuary’s role includes selecting or using the model in a specific project or if the actuary’s role includes modifying, reviewing, or evaluating the model when it is being selected or used in a specific project.

2.7 Model

A simplified representation of relationships among real world variables, entities, or events using statistical, financial, economic, mathematical, or scientific concepts and equations. Models are used to help explain a system, to study the effects of different parts of a system, and to derive estimates and guide decisions. A model consists of three components: an information input component, which delivers assumptions, parameters and data to the model; a processing component, which transforms inputs into estimates; and an output component, which translates the estimates into useful business information. A modelevolves through a life cycle as follows: (1) a specification phase, (2) an implementation phase, and (3) a production phase, that consists of one or more model runs.

2.8 Modeling

Selecting, designing, building, modifying, developing, using, reviewing, or evaluating models.

2.9 Model Risk

The risk of adverse consequences resulting from reliance on a model that does not adequately represent that which is being modeled or that is misused or misinterpreted.

2.10 Model Run

The process and end result of transforming a particular selection of input to a particular set of output in a model.

2.11 Parameters

A type of mathematical, financial, contractual, economic, scientific, or statistical input to models. Examples include pension plan provisions, expected values in mathematical distributions, and coefficients of variables in regression formulas.

2.12 Simple Model

A model wherein, in the actuary’s professional judgment, the model results are transparent and can be predicted without an actual model run or readily obtained from an external source that is not another model.

2.13 Specification

A description of a model that identifies the inputs and the formulas, algorithms, or logic (collectively referred to as “formulas”) to be used to produce output. The specification may be explicit, or it may be implicit in the implementation.

Section 3. Analysis Of Issues And Recommended Practices

3.1 Application Of Asop Guidance

The guidance in this ASOP applies to actuarial practice regarding models that are not simple models when, in the actuary’s professional judgment, intended users of the model rely heavily on the results, and the use of the results of the model has a material financial effect for the intended user. In assessing materiality, the actuary should be guided by ASOP No. 1, Introductory Actuarial Standard of Practice, section 2.6. For example, corporate financial planning, ratemaking, and reserving models would typically require application of the guidance.

In modeling situations where, in the actuary’s professional judgment, the results (1) are not heavily relied upon by the intended users or (2) do not have material financial effect, then application of the guidance in this ASOP is not required except that the actuary should disclose that the model was not deemed subject to the guidance of this ASOP for one or both of these two reasons.

In deciding whether this ASOP applies, the actuary should use professional judgment, considering the extent of reliance by the intended user and the materiality of the financial effect. This judgment should be made within the context of the use of the model results and the needs of the principal, based on facts known by the actuary at the time the actuarial services are performed.

In instances where a deviation from guidance is material, the actuary should disclose that deviation from guidance as addressed in section 4.3.

If an actuary is part of a modeling team, the actuary should identify the scope of his or her responsibilities. This scope could extend to the entire model or just to a small portion of the model. This standard only applies to the extent of the actuary’s responsibility, as identified and disclosed pursuant to section 4.2.1.

3.2 Models Developed By Others

If the actuary uses a model designed or built by someone else, such as a vendor or colleague, and the actuary has a limited ability to obtain information about the model or to understand the underlying workings of the model, the actuary should continue to make a reasonable attempt to have a basic understanding of the model, including the following:

a. the designer’s or builder’s original intended purpose for the model;

b. the general operation of the model;

c. major sensitivities and dependencies within the model; and

d. key strengths and limitations of the model.

In these instances, the actuary should make practical efforts to comply with other applicable sections of this standard.

3.3 Reliance On Another Actuary On A Modeling Team

When the actuary is part of a modeling team, the actuary may reasonably rely on another actuary who has selected, designed, built, modified, developed, reviewed, evaluated, or used the model. However, the relying actuary should be reasonably satisfied that the other actuary’s selecting, designing, building, modifying, developing, reviewing, evaluating, or use of the model was performed in accordance with this ASOP and is appropriate for the intended purpose.

3.4 Model Meeting The Intended Purpose

The actuary should select, design, build, modify, develop, or use a model that reasonably meets the intended purpose. An actuary who is reviewing or evaluating a model should evaluate whether the model reasonably meets the intended purpose.

3.4.1 Designing, Building, Or Developing The Model For The Intended Purpose

The actuary should confirm that the capability of the model is consistent with the intended purpose when the actuary designs, builds, or develops the model. In this confirmation, examples of items that the actuary should consider, if applicable, include but are not limited to granularity, the relationships recognized, and the model’s ability to identify possible volatility around expected values. Further, when appropriate, the actuary may consider whether the model can be easily updated for anticipated changes in data, parameters, or assumptions.

3.4.2 Selecting Or Using The Model For The Intended Purpose

The actuary should select or use the model to meet the intended purpose. In the actuary’s use of the model, efforts to revise the inputs and formulas, documentation, controls, validation, and presentation of results should be consistent with the intended purpose.

3.4.3 Reviewing, Evaluating, Or Modifying The Model

When reviewing or evaluating a model for its planned uses, or when modifying a model to change the planned uses or to improve the model’s ability to meet its planned uses, the actuary should be guided by section 3.4.1. When reviewing or evaluating a model for a specific use, or when modifying a model to improve the inputs, formulas, and outputs to meet the specific goal or question being addressed by the model, the actuary should be guided by section 3.4.2.

3.4.4 Understanding The Model

The actuary’s responsibilities may include expressing an opinion, using or communicating results, or preparing documentation based on or in relation to a model. In these instances, the actuary should understand the following:

a. important aspects of the model being used, including but not limited to, basic operations, important relationships, major sensitivities, strengths and potential weaknesses; and

b. whether, and the extent to which, the model can fulfill its intended purpose, recognizing limited information, time constraints, and other practical considerations.

3.4.5 Model Structure

The actuary should evaluate whether the structure of the model is appropriate for the intended purpose. Where applicable for a particular model structure, the actuary should consider the following:

a. which provisions and risks specific to a business segment, contract, or plan are material and appropriate to reflect in the model;

b. whether the use of the model dictates some level of granularity, such as whether grouping inputs will produce reasonable results;

c. whether deterministic or stochastic results, or both, are appropriate; and

d. whether the model appropriately represents options that are available either to the entity or its counterparties, where the options could have a material effect on the results of the model. Examples of options include call options on fixed income assets, policyholder surrender options, and early retirement options.

3.4.6 Data

The actuary should use data appropriate for the model’s intended purpose and should refer to ASOP No. 23, Data Quality, when selecting, reviewing, or evaluating data used in the model, either directly or as the basis for deriving assumptions and parameters.

3.4.7 Assumptions And Parameters

The actuary should use assumptions and parameters that are appropriate in light of the model’s intended purpose.

a. Experience Reflected in Setting Assumptions and Parameters—When setting assumptions and parameters, the actuary should consider using the following:

1. assumptions and parameters based on actual experience, to the extent it is available, relevant, and sufficiently reliable;

2. other relevant and sufficiently reliable experience, such as industry experience that is properly modified to reflect the circumstances being modeled, if actual experience is not available or relevant, or is not sufficiently reliable; and

3. professional judgment to modify other available sources of information.

b. Experience Reflected in Setting Assumptions and Parameters—When setting assumptions and parameters, the actuary should consider using the following:Margins—The actuary may consider whether adjusting the assumption or parameter to include a margin having a material effect would be appropriate, given the model’s intended purpose. Possible reasons for a margin include a) experience data that are not fully reliable, b) conservatism, c) an adjustment for the cost of bearing risk, or d) future unpredictability.

c. Range of Assumptions and Parameters—The actuary should consider whether the range of assumptions and parameters used and the number of model runs analyzed reflect a range of conditions consistent with the intended purpose.

d. Consistency—Where appropriate, the actuary should use assumptions and parameters for the modelthat are reasonably consistent with one another for a given model run. For example, where appropriate, the actuary should use assumptions and parameters consistent with the underlying economic scenario(s) assumed in the model.

If the actuary is aware of any material inconsistencies among assumptions and parameters used by the actuary in the model, the actuary should disclose the inconsistencies and the reasons for the inconsistencies in accordance with section 4.1.3. This disclosure applies whether the inconsistencies are (i) required by legal constraints or by the principal, (ii) the result of intentional redundancy such as added conservatism, or (iii) for any other reason. However, in the case of assumptions and parameters prescribed by applicable law (statutes, regulation, or other legally binding authority), the actuary’s disclosure may be limited to identifying the possibility of an inconsistency with other assumptions and parameters.

e. Appropriateness of Input in Current Model Run—Where practical and appropriate, the actuary reusing an existing model should evaluate whether the input is still appropriate for use in the current model run. For example, models used in financial reporting may offer opportunities to compare assumptions and parameters to emerging experience in the aggregate.

3.5 Mitigation Of Model Risk

The actuary should examine the potential for model risk and undertake reasonable and appropriate steps to mitigate such risk, using validation, governance, and controls, as appropriate to the intended purpose. When deciding on what steps to undertake that would be reasonable and appropriate, the actuary may consider the balance between the cost of the mitigation efforts and the reduction in potential for model risk.

3.5.1 Validation

The nature and degree of validation selected by the actuary should be consistent with the complexity of the model and the intended purpose.

a. Model Integrity—For each model run or set of model runs generated at one time or over time, that is to be relied upon by the intended user, the actuary should validate that the model reasonably represents that which is being modeled. Examples of validation of the model may include the following:

1. reconciling relevant input values to actual information, addressing and documenting the differences appearing in the reconciliation, if material;

2. checking formulas, logic, and table references; and

3. testing, where applicable, model projection results against historical actual results to verify that modeled results would bear a reasonable relationship to actual results over a given time period if inputs to the model were set to be consistent with the conditions prevailing during such period. However, such a test may not be applicable, even for a projection model, if the model is concerned with certain catastrophic scenarios or if the historical results are from a period with infrequent events.

The degree of reconciliation, checking and testing that is appropriate will depend on the intended purpose, the context and nature of the model, the operating environment and controls related to the model, whether there have been any changes to the modelor the model environment, and the residual risk that may remain after the model integrity efforts.

b. Analyzing the Output—The actuary should take appropriate steps to evaluate whether the model results are reasonable. Depending on the intended purpose, the actuary should consider the following:

1. performing analytical tests on model results to assess their reasonableness;

2. reconciling the results of a model run to prior model runs, given any changes in assumptions and parameters, data, formulas, or other aspects of the model since the prior model run;

3. running tests of variations on key assumptions and parameters to test that changes in the results are consistent with the changes in those assumptions and parameters; and

4. comparing model results to those of alternative model(s), where appropriate.

c. Peer Review—The actuary should consider obtaining a peer review, where appropriate, depending on the intended purpose and the actuary’s role. Such peer review, if obtained, may include items such as review of the reasonableness of the input to the model, the implementation of the model, and the model results.

3.5.2 Appropriate Governance And Controls

The actuary should use or, if appropriate, rely on others to use appropriate governance and controls to minimize model risk, to maintain the integrity of the model, and to avoid the introduction or use of unintentional or untested changes.

3.6 Presentation Of Results

When the actuary presents results of the model, the actuary should explain methodology, key assumptions and parameters, and possible model limitations, consistent with the guidance in ASOP No. 41, Actuarial Communications. If appropriate, the actuary should consider describing any material changes in methodology, key assumptions and parameters, and possible model limitations affecting results since the prior communication.

3.6.1 Explanation Of Limitations Of Models

In actuarial reports that include information derived from models, the actuary should include explanations of the following, if applicable:

a. the extent to which a model fails to fulfill its intended purpose, due to limited information, time constraints, or other practical considerations; and

b. any other known material limitations of the models that have been used and the implications of those limitations.

If there is anything to explain pursuant to (a) or (b), then the actuary should refer to section 4.1.

3.6.2 Discussion Of Models

In actuarial reports that include information derived from models, the actuary should consider including explanations of the following:

a. the intended purpose of the models and how the intended users’ needs are addressed by those models; and

b. any significant uncertainty in the model results.

3.6.3 Comparison To Prior Reports

The actuary should consider including in the actuarial report a comparison to corresponding items in a prior actuarial report, as applicable depending on the type of the model. Such a comparison, if any and where reasonably possible, should include an explanation of assumptions and parameters or methods that have changed materially from that prior actuarial report.

3.6.4 Description Of Conservatism Or Optimism

The actuary should consider including a description of the conservatism or optimism inherent in the inputs and methodology selected in relation to anticipated future experience, as applicable depending on the type of the model. Terminology may include language such as “conservative,” “most likely,” “reflecting asymmetric outcomes,” or “optimistic,” along with a description of the relationship to anticipated future experience by appropriate quantitative, qualitative, or directional language.

If applicable law specifies some or all of the inputs or methodology, then this section 3.6.4 does not apply with respect to the inputs or methodology so specified.

3.7 Reliance On Data Or Other Information Supplied By Others

When relying on data or other information supplied by others, the actuary should refer to ASOP Nos. 23 and 41 for guidance. When relying on outputs from other models supplied by others, the actuary should refer to ASOP No. 23, deeming such outputs from other models as data covered by that standard. Similarly, the actuary should refer to ASOP No. 41 with respect to the disclosure of responsibility for data, assumptions, parameters, and methods.

3.8 Documentation

For model results used in actuarial communications, the actuary should document the nature of the  data used, and material  assumptions and  parameters used in the model and, in the case of an actuarial report, the actuary should follow the guidance of ASOP No. 41, including section 3.2 in ASOP No. 41.

If no actuarial report is created, the actuary should consider documenting the items mentioned in sections 3.6.1 and 3.6.2 of this standard, and the actuary may consider documenting other items mentioned in sections 3.1-3.7 that the actuary believes may be helpful to subsequent users.

3.9 Relation To Other Asops

Other ASOPs provide specific requirements for actuarial services that often use modeling, including guidance on selecting assumptions, parameters, and data (see ASOP No. 23) and providing disclosures (see ASOP No. 41). If such specific guidance from an applicable ASOP is inconsistent with the guidance of this ASOP, the guidance of such other ASOP governs.

Section 4. Communications And Disclosures

4.1 Actuarial Communications

In any actuarial communications that use results of work subject to this ASOP, the actuary should note the guidance in sections 3.1 and 3.6-3.9.

4.2 Actuarial Report

In any actuarial report that uses the results of work subject to this ASOP, the actuary should disclose the following, as applicable:

4.2.1 Scope Of Actuary’s Responsibility

Where the actuary is responsible for only a portion of the overall model, the actuary should disclose the extent of the responsibility (as discussed in section 3.1).

4.2.2 Failure To Meet Intended Purpose

Any reasons that prevent the model from meeting its  intended purpose, as discussed in sections 3.4.4 and 3.6.1. In this situation, the actuary should disclose the intended purpose of the model.

4.2.3 Inconsistent Assumptions And Parameters

Any material inconsistencies among assumptions and parameters and the reasons for such inconsistencies, as discussed in section 3.4.7(d).

4.3 Deviation From Guidance In The Standard

In any actuarial communication that uses the results of work subject to this ASOP, the actuary should refer to ASOP No. 41 and should include the following where applicable

a. the disclosure in ASOP No. 41, section 4.2, if any material assumption , parameter, or method was prescribed by applicable law (statutes, regulations, and other legally binding authority);

b. the disclosure in ASOP No. 41, section 4.3, if the actuary states reliance on other sources and thereby disclaims responsibility for any material assumption, parameter, or method selected by a party other than the actuary; and

c. the disclosure in ASOP No. 41, section 4.4, if, in the actuary’s professional judgment, the actuary has otherwise deviated materially from the guidance of this ASOP.

Appendix 1

Note: This appendix is provided for informational purposes but is not part of the standard of practice.

Current Practices

Models are used to help explain a system, to study the effects of different components, and to derive estimates and guide decisions. Models have always played a fundamental role in actuarial work, with every discipline relying on a broad range of modeling applications, ranging from simple spreadsheets to complex capital models. The number and importance of modeling applications in actuarial science have continued to increase, with the results of actuarial models often entering financial statements directly.

Actuaries often develop and use models when analyzing uncertain outcomes. Even a model that is prudently developed and carefully used does not eliminate inherent uncertainty and variability and actual experience may differ, sometimes significantly, from the estimates derived from the model results. A model is only an approximation of reality, not the reality itself, and the differences between the model and actual experience, by themselves, do not indicate a flawed model or noncompliance with standards.

When a model will be used repeatedly, it is common that the model will be subject to appropriate governance and controls. Examples of model governance and controls include the following:

• limitations on access to use and modify the model (that is, restricting access to model inputs, model code and calculations, and model outputs);
• confirmation that model results are reproducible upon rerun (if the model allows for such reproducibility);
• implementing a model change management process;
• specification, documentation, and programming standards for the implementation;
• procedures for secure back-up of the media storing the implementation and data;
• appropriate staff training or cross-training for continuity of use;
• plans for periodic consideration of the organization’s continued ability to access and maintain the model, including data, software, staff, hardware, and vendor relationships;
• plans for periodic updating of model input; and
• plans for periodic review of the assumptions, parameters, functionality, and methodology.

Appendix 2

Comments and Exposure Draft Responses

The second exposure draft of this ASOP, Modeling, was approved by the ASB in November 2014 with a comment deadline of March 1, 2015. Thirty-seven comment letters were received, some of which were submitted on behalf of multiple commentators, such as by firms or committees. For purposes of this appendix, the term “commentator” may refer to more than one person associated with a particular comment letter. Where similar opinions are expressed, even in the case of essentially identical letters submitted by different commentators, this appendix refers to “several commentators.” The Modeling Task Force carefully considered all comments received, reviewed the exposure draft, and proposed changes. The General Committee and the ASB reviewed the proposed changes and made modifications where appropriate.

Summarized below are the significant issues and questions contained in the comment letters and responses.

The term “reviewers” in appendix 2 includes the Modeling Task Force, the General Committee, and the ASB. Also, unless otherwise noted, the section numbers and titles used in appendix 2 refer to those in the second exposure draft.

Click here to view Appendix 2 in its entirety.

Comments

Click here to view comments in their entirety.