Your health comes first. This educational article provides general information. Discuss your individual circumstances, pregnancy, illness or symptoms with your clinician; the article cannot replace an individual assessment.

Look for the question behind a claim of “high accuracy”

You read that a sports watch showed a very high correlation with a reference device. At first, this sounds as though every reading differed only slightly from the truth. Correlation, however, answers a different question from agreement. Two devices’ values can rise and fall together even if they differ consistently. Start by finding what the study measured, what it used for comparison and what size of difference would have been considered usable.

Bland and Altman’s classic methodological work specifically warns that correlation alone can mislead when comparing two measurement methods. Assessing agreement also requires the size of the differences between measurements. This is an established statistical principle, rather than an objection to a present-day brand. Source: Bland and Altman, agreement between measurement methods. Your task as a reader is to check whether the paper actually answered that question.

The aim is not to recalculate every validation study as a statistician. A few well-chosen questions already help considerably. Alongside a high r value, look for the mean difference, variability, study environment and participants. If these are absent from the abstract, you may need the full methods description. A striking conclusion only makes sense when tied to the measured quantity and the particular use situation.

Perfect association, yet different numbers

In a hypothetical example of our own, three reference values are 200, 300 and 400 units. The device under investigation gives 240, 360 and 480 respectively. The two series change together perfectly: one is always 1.2 times the other. Yet agreement is not perfect; the differences are 40, 60 and 80. This is not a measurement of a particular sports watch, and the unit is deliberately generic. It shows why high correlation cannot simply be translated into small error.

In our second example, the device adds 15 to every reference value. The series again have the same shape, but the difference is constant. A systematic offset and a proportional difference are two distinct patterns. It is therefore worth asking whether the error depends on the size of the value. Compressing all points into one association statistic can easily hide whether the higher-intensity or higher-value range behaves differently.

The line of agreement on a graph is not the same as the best-fitting regression line either. If every point showed exact agreement, the two devices would give the same number. A close linear relationship that differs from this indicates only a consistent association. When reading, therefore, notice which line has been drawn and what claim accompanies it. An attractive, narrow cloud of points does not itself establish interchangeability.

A runner undergoes treadmill measurement in a sports physiology laboratory, wearing a respiratory gas-analysis mask under professional observation.Enlarge image
AI-generated editorial illustration of laboratory assessment of running economy; not a documentary photograph.

Opposing differences can cancel out behind a mean error

In our example, let five differences be minus 20, minus 10, zero, plus 10 and plus 20 units. Their mean is zero. This does not mean every measurement is error-free, only that the signed differences cancel out. The mean absolute difference is 12 units. The two numbers summarise different things. A small mean bias can coexist with individual differences that matter for the particular use.

If the paper uses “mean bias”, find how the sign is defined: which device’s value was subtracted from which? The direction of a positive difference depends on this. Mean absolute error instead ignores signs, while percentage error uses a reference value. These should not be interchanged as a single generic “accuracy percentage”. A brief check of the unit and formula prevents many misunderstandings.

The relevance of an error measure also depends on your task. A small mean difference may have different significance during a long steady observation than in a short, rapidly changing section. The study defines what its measurement concerned, but you need to match your use question to it. If the paper studied energy expenditure, an identical result for instantaneous heart rate does not follow. Different metrics from a device require separate validation.

Limits of agreement are more than reassuring decoration around the bias

The Bland–Altman approach examines the mean and variability of the differences together. Under appropriate conditions, the usual limits of agreement extend roughly 1.96 standard deviations on either side of the mean difference. This is not the same as a confidence interval for the mean error. The conditions for applying the method and uncertainty in the estimates also need checking. Source: the limits-of-agreement method.

As a reader, the practical question is how large a difference may occur in an individual comparison under the study conditions. Even a very precisely estimated mean difference can coexist with much wider variation among individual pairs. Do not confuse these two kinds of uncertainty. Limits of agreement from a small study are estimates too, rather than hard guarantees for every future person and measurement. Both the limits and their uncertainty are valuable information.

Do not turn our five invented differences into a device certificate or a general 95 per cent error bound. This illustrative series is small and artificial, serving only to show the distinction between a mean and individual differences. A real study requires an appropriate sample, design and statistical analysis. A teaching example is useful when it illuminates the basic concept without presenting itself as genuine evidence of performance.

The points are not necessarily independent people

A study may contain several thousand seconds of data from a few dozen participants. The number of sample points and the number of people provide different information. Consecutive data from the same person may be related, and the analysis needs to handle repeated measurements appropriately. A large record count alone does not establish broad generalisability across people. Find how many participants and separate measurement occasions were included.

Plotting the relationship between a difference and measurement magnitude is also a methodological decision. Bland and Altman’s 1995 paper shows that plotting a difference against one method alone can create a misleading relationship; plotting it against the mean of the two measurements has different properties. This is a useful reading guide because the meaning of the horizontal axis is not a minor detail. Source: plotting differences and measurement magnitude.

Keep your practical check simple: what does one point represent, how do points belong to participants, and which conditions produced them? If the same few people provide many observations, the study can still be thorough for its own question. Just do not describe it as validation in thousands of people. Precise wording preserves the result’s value without adding evidence absent from the method.

The reference and protocol determine what the result covers

Johansson and colleagues’ 2026 HRV study compared a separate ECG, chest strap and phone-camera app in 37 trained participants. It involved two short recordings on the same day in a controlled environment. Several measures of reliability and agreement were used. This is a well-defined question, rather than a complete investigation of every overnight or running use of a device. Source: Johansson and colleagues, original study.

For your own running task, therefore, ask separately whether the study used the same quantity, similar intensity, posture and movement conditions. Resting HRV and heart-rate tracking during running are not the same investigation. A short, controlled laboratory recording does not automatically establish behaviour throughout a long, sweaty trail run. That distinction does not make the research worthless; it means the actual protocol defines the result’s scope.

If the claim concerns actual measurement accuracy, the reference cannot simply be another familiar consumer display. Two devices may be enough to examine their difference from each other, but relating readings to truth requires an appropriate reference and method. If the paper regards the reference itself as having limited accuracy, include that in your interpretation. Read “gold standard” in relation to the particular measured quantity and setup, rather than as a general brand name.

While reviewing training-log data, a runner writes a separate note about an uncertain measurement.Enlarge image
AI-generated editorial illustration of clearly marking suspicious sensor data; not a documentary photograph.

Model, firmware and participant group belong in the description too

Research on an old model does not automatically validate every metric from a new generation. Sensors, processing and software can change. Even on the same hardware, the firmware in use during testing may matter. If the paper gives it, keep it in your notes. If it is missing, do not infer it from the version available today. The scientific result belongs to a particular tested system.

Participants’ age, training status and selection may also limit generalisation. Results from a young, regularly exercising group do not investigate every age and health condition. Look for the sample composition and exclusion criteria. Similarity to your situation can help assess relevance, but a difference does not automatically predict a poor result either. It simply identifies a question without a direct answer yet.

Read the number of missing or unevaluable recordings too. If agreement was calculated only from measurements that could be classified appropriately, the result may not describe how often you obtain usable data at all. Accuracy and data availability are separate properties. In your own use, the value of a device that performs well on suitable recordings also depends on how many successful recordings it produces. Keep these two outcomes separate.

Read with five questions rather than one ranking

First, identify the metric studied and the reference method. Second, examine participants and protocol. Third, distinguish association, mean difference and individual variability. Fourth, look for missing data and the system version. Fifth, match the result to your own task. This sequence is clearer than building a complete product ranking from an abstract’s final sentence.

Your own conclusion can be narrow: this device showed this agreement for this metric under this protocol. You do not need to derive a general purchasing or health decision from every publication. If your important use situation was not studied, it remains a further question. A carefully bounded answer is useful even when it does not cover everything. Precise limits are what allow the value of research evidence to be assessed.

Correlation therefore concerns the strength of a relationship, while agreement also concerns differences between measurements. Usefulness additionally involves purpose, environment and uncertainty. Reading these together turns sports-watch accuracy studies into more than marketing proof or apparently contradictory numbers. They offer method-dependent answers to specific questions, providing a more realistic basis for your own running-technology decisions.

Sources

Helpful? Pass it on.