Data Logger Validation: 5 Checks Before Sensor Data Is Transmitted

A pH sensor reads 28.4. The value passes through the data logger, travels to the server, appears on a dashboard, and someone makes a decision based on a number that is physically impossible. Or a dissolved oxygen sensor reports 8.2 mg/L for 72 hours straight – not because conditions are stable, but because the sensor is stuck. These are not hypothetical scenarios. They are the kind of errors that reach compliance reports and trigger unnecessary maintenance dispatches when data leaves the station unchecked.

The WMO Guide to Instruments and Methods of Observation states that data leaving a monitoring station should have been quality controlled (WMO-No. 8, Vol. V). That expectation applies at the point of acquisition – the data logger – not only at a central server hours later.

Five deterministic validation checks, drawn from the IOOS QARTOD framework and WMO hydrological practices, can run inside most data loggers before data is transmitted. They do not require internet connectivity, historical baselines, or machine learning models. They require configuration.

What Data Logger Validation Actually Means

Validation at the data logger means running automated checks on each sensor reading before transmission to a server or dashboard. It is not signal conditioning, and it is not alarm threshold crossing – two processes it is regularly confused with.

Signal conditioning – filtering, averaging, damping – transforms the raw signal to reduce noise. It changes the value. Validation does not change the value. Validation judges whether a value, filtered or not, is plausible as a measurement.

Alarm threshold configuration checks whether a value crosses an operational limit and triggers a response. Validation checks whether the value is trustworthy enough to act on in the first place.

The data logger runs these checks on every record before transmission. The result is not deletion – it is flagging. Each reading receives a quality flag indicating its status.

Five Standard Checks a Data Logger Can Run

The five checks below come from the QARTOD (Quality Assurance/Quality Control of Real-Time Oceanographic Data) framework published by U.S. IOOS/NOAA and widely adopted for real-time environmental monitoring. WMO Guide No. 168 also recognizes range checks, step/jump checks, and plausibility checks as standard automated QC for hydrological data loggers.

Gross Range Test

QARTOD Test 4 – priority: Required.

The gross range test checks whether a reading falls within two tiers of limits. The first tier is the sensor range: the physical measurement boundaries of the hardware. The second tier is the user range: site-specific limits that reflect plausible conditions at a given location.

Example for pH: sensor range 0-14, user range 5.5-9.5 for a particular river monitoring station. A reading of 28.4 fails at the sensor tier – it is outside what the hardware can physically produce. A reading of 4.2 passes the sensor tier but fails the user tier – physically possible, but implausible at that site.

Rate of Change Test

QARTOD Test 7 – priority: Strongly Recommended.

This test flags readings that change faster than is physically plausible. QARTOD defines the parameter N_DEV (number of standard deviations) calculated over a moving time window TIM_DEV. A reading that exceeds N_DEV standard deviations from the mean within that window is flagged as suspect.

An important note from QARTOD: the rate-of-change check does not apply to zero-valued dissolved oxygen observations. Under anoxic conditions, DO can legitimately drop to zero and remain there – that is not an anomaly.

Flat Line Test (Stuck Sensor)

QARTOD Test 8 – priority: Strongly Recommended.

This test detects sensors that report the same value repeatedly. QARTOD uses the parameters REP_CNT (repetition count) and a tolerance EPS. If a value does not change beyond EPS for REP_CNT consecutive readings, the data is flagged as suspect or fail depending on the count.

A closely related check is the Attenuated Signal Test (QARTOD Test 10), which detects nearly-flat signals – for example, when sediment or debris covers a sensor head, dampening natural variation. This test uses standard deviation or range falling below MIN_VAR over a period TST_TIM.

Spike Test

QARTOD Test 6 – priority: Strongly Recommended.

The spike test identifies single-point anomalies: one reading that deviates sharply from the points before and after it, but the next point returns to normal. This distinguishes a spike from a genuine level shift. Momentary electromagnetic interference or communication errors typically produce this pattern.

Cross-Parameter Test

QARTOD Test 9 – priority: Suggested.

This test checks consistency between parameters that are physically related. For example, dissolved oxygen changes sharply but water temperature does not change accordingly. Since oxygen solubility is governed by temperature, a large DO shift without a corresponding temperature change indicates a possible sensor problem, not a real environmental change.

This check requires a data logger reading more than one parameter simultaneously. A multi-channel data logger with multiple sensor inputs can run this test because it has access to several parameter readings within the same acquisition cycle.

Example Thresholds by Parameter

The table below presents illustrative threshold values based on QARTOD and USGS references. These are not universal standards – every site requires threshold calibration based on local conditions.

Parameter	Sensor Range (Fail)	User Range (Suspect)	Rate-of-Change Flag	Flat Line Suspect
pH	0–14	Site-adjusted, e.g. 5.5–9.5	>1 unit/hour	>12 identical readings
DO (mg/L)	0–20	0–15	N_DEV = 3, TIM_DEV = 25	>12 identical readings
Turbidity (NTU)	0–4,000	0–1,000	Site-specific	>8 identical readings
Conductivity (µS/cm)	0–100,000	100–2,000 (freshwater)	>500 µS/cm per hour	>12 identical readings
Water Level (m)	Sensor spec	Site-adjusted	>0.5 m/6 min	>20 identical readings

Note: Values are illustrative starting points derived from QARTOD and USGS guidance. Actual thresholds must be calibrated to site conditions, seasonal baselines, and parameter-specific behavior.

What Should Not Run at the Edge

Not every quality check belongs on the data logger. Some require data that is not available at a single station.

Neighbor checks compare a station’s readings against nearby stations – that requires multi-station data only available on a server. Climatological analysis compares data against long-term historical records that are also stored server-side. ML-based anomaly detection typically requires more computational resources and training data than available on typical environmental data loggers.

The division is practical: the data logger runs deterministic checks that need only local sensor data and configuration parameters. The server runs analyses that need historical archives, data from other stations, or complex statistical models.

What Happens When a Reading Is Flagged

The governing principle is flag and forward, not suppress and discard. Data that fails a check is still transmitted – with its quality flag attached. Raw data must be retained. Both QARTOD and WMO emphasize that data must not be deleted. A reading that appears anomalous may turn out to capture a real event that can only be confirmed through later analysis.

QARTOD uses a 5-level quality flag scheme: 1 (Pass), 2 (Not Evaluated), 3 (Suspect), 4 (Fail), 9 (Missing Data). This scheme can be applied at the data logger level for initial flagging and then updated on the server after additional checks. How these flags appear on a monitoring dashboard and feed into operational decisions is covered in detail in the companion article on sensor data quality visibility.

When a flag shows suspect or fail, the next step is diagnosis – not deletion. For issues related to electrical signals, the 4-20 mA troubleshooting guide can help identify whether the problem lies in the sensor, the cabling, or the configuration.

Edge validation is the first layer. The server adds a second layer. The dashboard displays the results alongside every value. A monitoring system that includes quality context at each stage provides a stronger foundation for operational decisions and reporting compliance.