Re-calibration

The Need for Re-calibration

Re-calibration is performed to ensure PBL algorithms are accurate for their intended population. We recommend re-calibration whenever possible.

Re-calibration is performed by including a calibration object that represents the user's population. For example, a user from the United States should have the MPoRT algorithm re-calibrated to the offical age and sex-specific mortality risk for the United States.

Background

An algorithm is accurate, or well-calibrated, if the predicted risk closely approximates observed risk. For example, the PBL calculator engine would be considered well-calibrated if the calculator engine, when used by a representative population like the United States, resulted in an average life expectancy that was the same as the official life expectancy estimates as reported by the National Vital Statistics Reports.

Health and medical algorithms typically require re-calibration because health outcomes such as death and disease varies between populations and over time due to a variety of reasons that may not be fully represented in an algorithms predictors. Re-calibration is performed by adjusting the algorithm's baseline risk to the average risk of a new, target population.

Calibration object

This is what a calibration object looks like:

{
  age: number;
  baseline: number;
}

When calling the addCalibration method, depending on the model you will need to pass in one of the following arguments:

• If the model is not gender specific (i.e. it does not contain a male and female algorithm) you just need to pass in an array of calibration objects for the age range for that model.
• If the model is gender specific then the argument looks like below:

{
    male: Array<CalibrationObject>; // The calibration objects for the male algorithm
    female: Array<CalibrationObject>; // The calibration objects for the female algorithm
}

The following example shows how to add a calibration object to the MPoRT model

/* Initialize the MPoRT model. Look at the Building Models section to see how
to do this*/
const mportModel;

/* The allowable age range for the MPoRT model. 20 is the min and 79 is the max */
const mportAgeRange = [20, 79];
/* This is where we store the calibration objects. MPoRT is a gender specific
model. So, there are calibration objects for the male and the female
algorithms */
const calibrationObjects = { male: [], female: [] };
/* Generate random calibration objects for the ages from 20 to 79 and add them
to the calibrationObjects object */
for(let i=mportAgeRange[0]; i<=mportAgeRange[1]; i++) {
    calibrationObjects.male.push({
        age: i,
        baseline: Math.random()
    });
    calibrationObjects.female.push({
        age: i,
        baseline: Math.random()
    });
}

/* Create a new MPoRT model using the above calibration objects and store it
in the variable calibratedMportModel. Do this since the old model is
immutably updated */
const calibratedMportModel = mportModel.addCalibration(calibrationObjects);

/* Data with which we will calculate risk. For more information on this array go
to the Building Your Data section */
const data = [
    { name: 'age', coefficent: 21 },
    { name: 'sex', coefficent: 'female' }
];

/* Calculate risk using the original MPoRT model */
const oldMportModelRisk = mportModel.getRiskToTime(data);
/* Calculate risk using the new model */
const calibratedMportModelRisk = calibratedMportModel.getRiskToTime(data);

/* The 2 risks show not be equal since they are different models now */
console.log(oldMportModelRisk === calibratedMportModelRisk)