A new prediction hack delivers results that are shockingly close to reality

The researchers call their technique the maximum agreement linear predictor, or MALP. Its primary goal is to enhance the extent to which predicted values ​​match observed values. MALP does this by maximizing the harmonic correlation coefficient, or CCC. This statistical measure evaluates how pairs of numbers fall along a 45-degree line in a scatter plot, reflecting precision (how tight a group of dots is) and accuracy (how close they are to that line). Traditional methods, including the widely used least squares method, usually attempt to minimize the mean error. Although effective in many situations, these approaches can miss the mark when the main goal is to ensure a strong fit between expectations and actual values, says Kim, the assistant professor of mathematics.

“Sometimes, we don’t just want our predictions to be close, we want them to have the highest degree of agreement with true values,” Kim explains. “The issue is, how can we determine the agreement of two things in a scientifically meaningful way? One way we can visualize this is how closely the points align with the 45-degree line on a scatter plot between the predicted value and the actual values. So, if the scatter plot of those shows strong alignment with the 45-degree line, then we can say that there is a good level of agreement between those two.”

Why agreement matters more than simple association

According to Kim, people often first think of the Pearson correlation coefficient when they hear the word “agreement,” since it was introduced early in statistics education and remains an essential tool. The Pearson method measures the strength of a linear relationship between two variables, but it does not specifically check whether the relationship conforms to the 45 degree line. For example, it can detect strong correlations for lines inclined at 50 degrees or 75 degrees, as long as the data points lie near a straight line, Kim says.

“In our case, we are particularly interested in alignment with the 45 degree line. For this we use a different measure: the fit correlation coefficient, which was introduced by Lin in 1989. This measure focuses specifically on how well the data align with the 45 degree line. What we have developed is a predictor designed to maximize the fit correlation between predicted values ​​and actual values.”

MALP testing through eye examination and body measurements

To evaluate how well the MALP performed, the team conducted tests using both simulated data and real measurements, including eye exams and body fat assessments. One study applied MALP to data from an ophthalmology project comparing two types of optical coherence tomography (OCT) devices: the older Stratus OCT and the newer Cirrus OCT. As medical centers move to the Cirrus system, doctors need a reliable way to translate measurements so they can compare results over time. Using high-quality images from 26 left eyes and 30 right eyes, the researchers examined how accurately MALP predicted Stratus OCT readings from Cirrus OCT measurements and compared its performance with the least squares method. MALP produced predictions that more closely matched true Stratos values, while least squares slightly outperformed MALP in reducing mean error, highlighting the trade-off between agreement and error reduction.

The team also looked at a body fat dataset from 252 adults, which included weight, abdominal size and other body measurements. Direct measures of body fat percentage, such as underwater weighing, are reliable but expensive, so easier measurements are often replaced. MALP was used to estimate body fat percentage and was evaluated according to the least squares method. The results were similar to the eye scanning study: MALP provided predictions that closely matched the true values, while least squares again had slightly lower average errors. This recurring pattern emphasized the constant balance between agreement and error minimization.

Choose the right tool for the right job

Kim and colleagues note that MALP often provides predictions that match actual data more effectively than standard techniques. However, they note that researchers must choose between MALP and more traditional methods based on their specific priorities. When minimizing overall error is the primary goal, the methods still perform well. When the focus is on forecasts that correspond as closely as possible to real results, MALP is often the stronger choice.

The potential impact of this work reaches many scientific fields. Improved forecasting tools can benefit medicine, public health, economics, and engineering. For researchers who rely on prediction, MALP offers a promising alternative, especially when achieving close agreement with real-world results is more important than simply narrowing the mean gap between predicted and observed values.

“We need to investigate further,” Kim says. “Currently, our setup falls into the category of linear predictors. This group is large enough to be used practically in different fields, but it is still mathematically constrained. Therefore, we would like to extend this to the general category so that our goal is to remove the linear part and thus become the maximum agreement predictor.”