<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Applying Deterministic Methods
Getting Started
This tutorial focuses on using deterministic methods to square a triangle. 
Note that a lot of the examples shown here might not be applicable in a real world scenario, and is only meant to demonstrate some of the functionalities included in the package. The user should always exercise their best actuarial judgement, and follow any applicable laws, the Code of Professional Conduct, and applicable Actuarial Standards of Practice.
Be sure to make sure your packages are updated. For more info on how to update your pakages, visit Keeping Packages Updated.
Step1:  The Chainladder Method
The basic chainladder method is entirely specified by its development pattern selections. For this reason, the Chainladder estimator takes no additional assumptions, i.e. no additional arguments. Let's start by loading an example dataset and creating an Triangle with Development patterns and a TailCurve.  Recall, we can bundle these two estimators into a single Pipeline if we wish.
Step2:  We can now use the basic Chainladder estimator to estimate ultimate_ values of our Triangle.
Step3:  We can also view the ibnr_.  Techincally the term IBNR is reserved for Incurred but not Reported, but the chainladder models use it to describe the difference between the ultimate and the latest evaluation period.
Step4:  It is often useful to see the completed Triangle and this can be accomplished by inspecting the full_triangle_.  As with most other estimator properties, the full_triangle_ is itself a Triangle and can be manipulated as such.
Step5:  Notice the calendar year of our ultimates.  While ultimates will generally be realized before this date, the chainladder package picks the highest allowable date available for its ultimate_ valuation.
Step6:  We can further manipulate the "triangle", such as applying cum_to_incr().
Step7:  Another useful property is full_expectation_. Similar to the full_triangle, it "squares" the Triangle, but replaces the known data with expected values implied by the model and development pattern.
Step8:  With some clever arithmetic, we can use these objects to give us other useful information.  For example, we can retrospectively review the actual Triangle against its modeled expectation.
Step9:  We can also filter out the lower right part of the triangle with [genins_model.full_triangle_.valuation &lt;= genins.valuation_date].
Step10:  Getting comfortable with manipulating Triangles will greatly improve our ability to extract value out of the chainladder package. Here is another way of getting the same answer.
Step11:  We can also filter out the lower right part of the triangle with [genins_model.full_triangle_.valuation &lt;= genins.valuation_date] before applying the heatmap().
Step12:  Can you figure out how to get the expected IBNR runoff in the upcoming year?
Step13:  The Bornhuetter-Ferguson Method
The BornhuetterFerguson estimator is another deterministic method having many of the same attributes as the Chainladder estimator. It comes with one input assumption, the a priori (apriori). This is a scalar multiplier that will be applied to an exposure vector, which will produce an a priori ultimate estimate vector that we can use for the model.
The BornhuetterFerguson method
The BornhuetterFerguson estimator is another deterministic method having many of the same attributes as the Chainladder estimator.  It comes with one assumption, the apriori.  This is a scalar multiplier that is to be applied to an exposure vector to determine an apriori ultimate estimate of our model.
Since the CAS Loss Reserve Database has premium, we will use it as an example.  Let's grab the paid loss and net earned premium  for the commercial auto line of business.
Remember that apriori is a scaler, which we need to apply it to a vector of exposures. Let's assume that the a priori is 0.75, for 75% loss ratio.
Let's set an apriori Loss Ratio estimate of 75%
The BornhuetterFerguson method along with all other expected loss methods like CapeCod and Benktander (discussed later), need to take in an exposure vector. The exposure vector has to be a Triangle itself. Remember that the Triangle class supports single exposure vectors.
Step14:  Having an apriori that takes on only a constant for all origins can be limiting; however, we can apply a varying vector on the exposure vector to get a varying apriori.
Step15:  Having an apriori that takes on only a constant for all origins can be limiting.  This shouldn't stop the practitioner from exploiting the fact that the apriori can be embedded directly in the exposure vector itself allowing full cusomization of the apriori.
Step16:  If we need to create a new colume, such as AdjEarnedPrmNet with varying implied loss ratios. It is recommend that we perform any data modification in pandas instead of Triangle forms.
Let's perform the estimate using Chainladder and compare the results.
Step17:  The Benktander Method
The Benktander method is similar to the BornhuetterFerguson method, but allows for the specification of one additional assumption, n_iters, the number of iterations to recalculate the ultimates. The Benktander method generalizes both the BornhuetterFerguson and the Chainladder estimator through this assumption.
When n_iters = 1, the result is equivalent to the BornhuetterFerguson estimator.
When n_iters is sufficiently large, the result converges to the Chainladder estimator.
Step18:  Fitting the Benktander method looks identical to the other methods.
Step19:  The Cape Cod Method
The CapeCod method is similar to the BornhuetterFerguson method, except its apriori is computed from the Triangle itself. Instead of specifying an apriori, decay and trend need to be specified.  
decay is the rate that gives weights to earlier origin periods, this parameter is required by the Generalized Cape Cod Method, as discussed in Using Best Practices to Determine a Best Reserve Estimate by Struzzieri and Hussian. As the decay factor approaches 1 (the default value), the result approaches the traditional Cape Cod method. As the decay factor approaches 0, the result approaches the Chainladder method. 
trend is the trend rate along the origin axis to reflect systematic inflationary impacts on the a priori.
Step20:  When we fit a CapeCod method, we can see the apriori it computes with the given decay and trend assumptions. Since it is an array of estimated parameters, this CapeCod attribute is called the apriori_, with a trailing underscore.
Step21:  With decay=1, each origin period gets the same apriori_ (this is the traditional Cape Cod). The apriori_ is calculated using the latest diagonal over the used-up exposure, where the used-up exposure is the exposure vector / CDF. Let's validate the calculation of the a priori.
Step22:  With decay=0, the apriori_ for each origin period stands on its own.
Step23:  Doing the same on our manually calculated apriori_ yields the same result.
Step24:  Let's verify the result of this Cape Cod model's result with the Chainladder's.
Step25:  We can examine the apriori_s to see whether there exhibit any trends over time.
Step26:  Looks like there is a small positive trend, let's judgementally select the trend as 1%.
Step27:  We can of course utilize both the trend and the decay parameters together. Adding trend to the CapeCod method is intended to adjust the apriori_s to a common level. Once at a common level, the apriori_ can be estimated from multiple origin periods using the decay factor.
Step28:  Once estimated, it is necessary to detrend our apriori_s back to their untrended levels and these are contained in detrended_apriori_. It is the detrended_apriori_ that gets used in the calculation of ultimate_ losses.
Step29:  The detrended_apriori_ is a much smoother estimate of the initial expected ultimate_.  With the detrended_apriori_ in hand, the CapeCod method estimator behaves exactly like our the BornhuetterFerguson model.
Step30:  Recap
All the deterministic estimators have ultimate_, ibnr_, full_expecation_ and full_triangle_ attributes that are themselves Triangles.  These can be manipulated in a variety of ways to gain additional insights from our model. The expected loss methods take in an exposure vector, which itself is a Triangle through the sample_weight argument of the fit method. The CapeCod method has the additional attributes apriori_ and detrended_apriori_ to accommodate the selection of its trend and decay assumptions.
Finally, these estimators work very well with the transformers discussed in previous tutorials. Let's demonstrate the compositional nature of these estimators.
Step31:  Let's calculate the age-to-age factors
<ASSISTANT_TASK:>
Python Code:
# Black linter, optional
%load_ext lab_black
import pandas as pd
import numpy as np
import chainladder as cl
import matplotlib.pyplot as plt
import os
%matplotlib inline
print("pandas: " + pd.__version__)
print("numpy: " + np.__version__)
print("chainladder: " + cl.__version__)
Explanation: Applying Deterministic Methods
Getting Started
This tutorial focuses on using deterministic methods to square a triangle. 
Note that a lot of the examples shown here might not be applicable in a real world scenario, and is only meant to demonstrate some of the functionalities included in the package. The user should always exercise their best actuarial judgement, and follow any applicable laws, the Code of Professional Conduct, and applicable Actuarial Standards of Practice.
Be sure to make sure your packages are updated. For more info on how to update your pakages, visit Keeping Packages Updated.
End of explanation
genins = cl.load_sample("genins")
genins_dev = cl.Pipeline(
    [("dev", cl.Development()), ("tail", cl.TailCurve())]
).fit_transform(genins)
Explanation: The Chainladder Method
The basic chainladder method is entirely specified by its development pattern selections. For this reason, the Chainladder estimator takes no additional assumptions, i.e. no additional arguments. Let's start by loading an example dataset and creating an Triangle with Development patterns and a TailCurve.  Recall, we can bundle these two estimators into a single Pipeline if we wish.
End of explanation
genins_model = cl.Chainladder().fit(genins_dev)
genins_model.ultimate_
Explanation: We can now use the basic Chainladder estimator to estimate ultimate_ values of our Triangle.
End of explanation
genins_model.ibnr_
Explanation: We can also view the ibnr_.  Techincally the term IBNR is reserved for Incurred but not Reported, but the chainladder models use it to describe the difference between the ultimate and the latest evaluation period.
End of explanation
genins
genins_model.full_triangle_
genins_model.full_triangle_.dev_to_val()
Explanation: It is often useful to see the completed Triangle and this can be accomplished by inspecting the full_triangle_.  As with most other estimator properties, the full_triangle_ is itself a Triangle and can be manipulated as such.
End of explanation
genins_model.full_triangle_.valuation_date
Explanation: Notice the calendar year of our ultimates.  While ultimates will generally be realized before this date, the chainladder package picks the highest allowable date available for its ultimate_ valuation.
End of explanation
genins_model.full_triangle_.dev_to_val().cum_to_incr()
Explanation: We can further manipulate the "triangle", such as applying cum_to_incr().
End of explanation
genins_model.full_expectation_
Explanation: Another useful property is full_expectation_. Similar to the full_triangle, it "squares" the Triangle, but replaces the known data with expected values implied by the model and development pattern.
End of explanation
genins_model.full_triangle_ - genins_model.full_expectation_
Explanation: With some clever arithmetic, we can use these objects to give us other useful information.  For example, we can retrospectively review the actual Triangle against its modeled expectation.
End of explanation
genins_model.full_triangle_[
    genins_model.full_triangle_.valuation <= genins.valuation_date
] - genins_model.full_expectation_[
    genins_model.full_triangle_.valuation <= genins.valuation_date
]
Explanation: We can also filter out the lower right part of the triangle with [genins_model.full_triangle_.valuation &lt;= genins.valuation_date].
End of explanation
genins_AvE = genins - genins_model.full_expectation_
genins_AvE[genins_AvE.valuation <= genins.valuation_date]
Explanation: Getting comfortable with manipulating Triangles will greatly improve our ability to extract value out of the chainladder package. Here is another way of getting the same answer.
End of explanation
genins_AvE[genins_AvE.valuation <= genins.valuation_date].heatmap()
Explanation: We can also filter out the lower right part of the triangle with [genins_model.full_triangle_.valuation &lt;= genins.valuation_date] before applying the heatmap().
End of explanation
cal_yr_ibnr = genins_model.full_triangle_.dev_to_val().cum_to_incr()
cal_yr_ibnr[cal_yr_ibnr.valuation.year == 2011]
Explanation: Can you figure out how to get the expected IBNR runoff in the upcoming year?
End of explanation
bf_model.fit(
    comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
bf_model.ultimate_
Explanation: The Bornhuetter-Ferguson Method
The BornhuetterFerguson estimator is another deterministic method having many of the same attributes as the Chainladder estimator. It comes with one input assumption, the a priori (apriori). This is a scalar multiplier that will be applied to an exposure vector, which will produce an a priori ultimate estimate vector that we can use for the model.
The BornhuetterFerguson method
The BornhuetterFerguson estimator is another deterministic method having many of the same attributes as the Chainladder estimator.  It comes with one assumption, the apriori.  This is a scalar multiplier that is to be applied to an exposure vector to determine an apriori ultimate estimate of our model.
Since the CAS Loss Reserve Database has premium, we will use it as an example.  Let's grab the paid loss and net earned premium  for the commercial auto line of business.
Remember that apriori is a scaler, which we need to apply it to a vector of exposures. Let's assume that the a priori is 0.75, for 75% loss ratio.
Let's set an apriori Loss Ratio estimate of 75%
The BornhuetterFerguson method along with all other expected loss methods like CapeCod and Benktander (discussed later), need to take in an exposure vector. The exposure vector has to be a Triangle itself. Remember that the Triangle class supports single exposure vectors.
End of explanation
bf_model.fit(
    comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
Explanation: Having an apriori that takes on only a constant for all origins can be limiting; however, we can apply a varying vector on the exposure vector to get a varying apriori.
End of explanation
b1 = cl.BornhuetterFerguson(apriori=0.75).fit(
    comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
b2 = cl.BornhuetterFerguson(apriori=1.00).fit(
    comauto["CumPaidLoss"],
    sample_weight=0.75 * comauto["EarnedPremNet"].latest_diagonal,
)
b1.ultimate_ == b2.ultimate_
Explanation: Having an apriori that takes on only a constant for all origins can be limiting.  This shouldn't stop the practitioner from exploiting the fact that the apriori can be embedded directly in the exposure vector itself allowing full cusomization of the apriori.
End of explanation
cl_model = cl.Chainladder().fit(comauto["CumPaidLoss"])
plt.plot(
    bf_model.ultimate_.to_frame().index.year, bf_model.ultimate_.to_frame(), label="BF",
)
plt.plot(
    cl_model.ultimate_.to_frame().index.year, cl_model.ultimate_.to_frame(), label="CL",
)
plt.legend(loc="upper left")
Explanation: If we need to create a new colume, such as AdjEarnedPrmNet with varying implied loss ratios. It is recommend that we perform any data modification in pandas instead of Triangle forms.
Let's perform the estimate using Chainladder and compare the results.
End of explanation
bk_model = cl.Benktander(apriori=0.75, n_iters=2)
Explanation: The Benktander Method
The Benktander method is similar to the BornhuetterFerguson method, but allows for the specification of one additional assumption, n_iters, the number of iterations to recalculate the ultimates. The Benktander method generalizes both the BornhuetterFerguson and the Chainladder estimator through this assumption.
When n_iters = 1, the result is equivalent to the BornhuetterFerguson estimator.
When n_iters is sufficiently large, the result converges to the Chainladder estimator.
End of explanation
bk_model.fit(
    X=comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
plt.plot(
    bf_model.ultimate_.to_frame().index.year, bf_model.ultimate_.to_frame(), label="BF"
)
plt.plot(
    cl_model.ultimate_.to_frame().index.year, cl_model.ultimate_.to_frame(), label="CL"
)
plt.plot(
    bk_model.ultimate_.to_frame().index.year, bk_model.ultimate_.to_frame(), label="BK"
)
plt.legend(loc="upper left")
Explanation: Fitting the Benktander method looks identical to the other methods.
End of explanation
cc_model = cl.CapeCod(decay=1, trend=0).fit(
    X=comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
Explanation: The Cape Cod Method
The CapeCod method is similar to the BornhuetterFerguson method, except its apriori is computed from the Triangle itself. Instead of specifying an apriori, decay and trend need to be specified.  
decay is the rate that gives weights to earlier origin periods, this parameter is required by the Generalized Cape Cod Method, as discussed in Using Best Practices to Determine a Best Reserve Estimate by Struzzieri and Hussian. As the decay factor approaches 1 (the default value), the result approaches the traditional Cape Cod method. As the decay factor approaches 0, the result approaches the Chainladder method. 
trend is the trend rate along the origin axis to reflect systematic inflationary impacts on the a priori.
End of explanation
cc_model.apriori_
Explanation: When we fit a CapeCod method, we can see the apriori it computes with the given decay and trend assumptions. Since it is an array of estimated parameters, this CapeCod attribute is called the apriori_, with a trailing underscore.
End of explanation
latest_diagonal = comauto["CumPaidLoss"].latest_diagonal
cdf_as_origin_vector = (
    cl.Chainladder().fit(comauto["CumPaidLoss"]).ultimate_
    / comauto["CumPaidLoss"].latest_diagonal
)
latest_diagonal.sum() / (
    comauto["EarnedPremNet"].latest_diagonal / cdf_as_origin_vector
).sum()
Explanation: With decay=1, each origin period gets the same apriori_ (this is the traditional Cape Cod). The apriori_ is calculated using the latest diagonal over the used-up exposure, where the used-up exposure is the exposure vector / CDF. Let's validate the calculation of the a priori.
End of explanation
cc_model = cl.CapeCod(decay=0, trend=0).fit(
    X=comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
cc_model.apriori_
Explanation: With decay=0, the apriori_ for each origin period stands on its own.
End of explanation
latest_diagonal / (comauto["EarnedPremNet"].latest_diagonal / cdf_as_origin_vector)
Explanation: Doing the same on our manually calculated apriori_ yields the same result.
End of explanation
cc_model.ultimate_ - cl_model.ultimate_
Explanation: Let's verify the result of this Cape Cod model's result with the Chainladder's.
End of explanation
plt.plot(cc_model.apriori_.to_frame().index.year, cc_model.apriori_.to_frame())
Explanation: We can examine the apriori_s to see whether there exhibit any trends over time.
End of explanation
trended_cc_model = cl.CapeCod(decay=0, trend=0.01).fit(
    X=comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
plt.plot(
    cc_model.apriori_.to_frame().index.year,
    cc_model.apriori_.to_frame(),
    label="Untrended",
)
plt.plot(
    trended_cc_model.apriori_.to_frame().index.year,
    trended_cc_model.apriori_.to_frame(),
    label="Trended",
)
plt.legend(loc="lower right")
Explanation: Looks like there is a small positive trend, let's judgementally select the trend as 1%.
End of explanation
trended_cc_model = cl.CapeCod(decay=0, trend=0.01).fit(
    X=comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
trended_decayed_cc_model = cl.CapeCod(decay=0.75, trend=0.01).fit(
    X=comauto["CumPaidLoss"], sample_weight=comauto["EarnedPremNet"].latest_diagonal
)
plt.plot(
    cc_model.apriori_.to_frame().index.year,
    cc_model.apriori_.to_frame(),
    label="Untrended",
)
plt.plot(
    trended_cc_model.apriori_.to_frame().index.year,
    trended_cc_model.apriori_.to_frame(),
    label="Trended",
)
plt.plot(
    trended_decayed_cc_model.apriori_.to_frame().index.year,
    trended_decayed_cc_model.apriori_.to_frame(),
    label="Trended and Decayed",
)
plt.legend(loc="lower right")
Explanation: We can of course utilize both the trend and the decay parameters together. Adding trend to the CapeCod method is intended to adjust the apriori_s to a common level. Once at a common level, the apriori_ can be estimated from multiple origin periods using the decay factor.
End of explanation
plt.plot(
    trended_cc_model.apriori_.to_frame().index.year,
    trended_cc_model.apriori_.to_frame(),
    label="Trended",
)
plt.plot(
    trended_cc_model.detrended_apriori_.to_frame().index.year,
    trended_cc_model.detrended_apriori_.to_frame(),
    label="Detended to Original",
)
plt.legend(loc="lower right")
Explanation: Once estimated, it is necessary to detrend our apriori_s back to their untrended levels and these are contained in detrended_apriori_. It is the detrended_apriori_ that gets used in the calculation of ultimate_ losses.
End of explanation
bf_model = cl.BornhuetterFerguson().fit(
    X=comauto["CumPaidLoss"],
    sample_weight=trended_cc_model.detrended_apriori_
    * comauto["EarnedPremNet"].latest_diagonal,
)
bf_model.ultimate_.sum() - trended_cc_model.ultimate_.sum()
Explanation: The detrended_apriori_ is a much smoother estimate of the initial expected ultimate_.  With the detrended_apriori_ in hand, the CapeCod method estimator behaves exactly like our the BornhuetterFerguson model.
End of explanation
wkcomp = (
    cl.load_sample("clrd")
    .groupby("LOB")
    .sum()
    .loc["wkcomp"][["CumPaidLoss", "EarnedPremNet"]]
)
wkcomp
Explanation: Recap
All the deterministic estimators have ultimate_, ibnr_, full_expecation_ and full_triangle_ attributes that are themselves Triangles.  These can be manipulated in a variety of ways to gain additional insights from our model. The expected loss methods take in an exposure vector, which itself is a Triangle through the sample_weight argument of the fit method. The CapeCod method has the additional attributes apriori_ and detrended_apriori_ to accommodate the selection of its trend and decay assumptions.
Finally, these estimators work very well with the transformers discussed in previous tutorials. Let's demonstrate the compositional nature of these estimators.
End of explanation
patterns = cl.Pipeline(
    [
        (
            "dev",
            cl.Development(
                average=["volume"] * 5 + ["simple"] * 4,
                n_periods=7,
                drop_valuation="1995",
            ),
        ),
        ("tail", cl.TailCurve(curve="inverse_power", extrap_periods=80)),
    ]
)
cc = cl.CapeCod(decay=0.8, trend=0.02).fit(
    X=patterns.fit_transform(wkcomp["CumPaidLoss"]),
    sample_weight=wkcomp["EarnedPremNet"].latest_diagonal,
)
cc.ultimate_
plt.bar(cc.ultimate_.to_frame().index.year, cc.ultimate_.to_frame()["2261"])
Explanation: Let's calculate the age-to-age factors:
- Without the the 1995 valuation period
- Using volume weighted for the first 5 factors, and  simple average for the next 4 factors (for a total of 9 age-to-age factors)
- Using no more than 7 periods (with n_periods)
End of explanation
<END_TASK>