Historian Data Compression: Deadband, Swinging Door and Exception Reporting

A process historian that stores every scan of every tag would need an absurd amount of disk, yet a plant that compresses carelessly quietly rewrites its own history. A temperature tag scanned once a second produces 86,400 values a day; a mid-size site has hundreds of thousands of tags. For decades the answer has been historian data compression: a two-stage filter that throws away values a straight line could have recreated, and keeps the ones that carry information.

The trade is not free. The same filter that cuts storage by an order of magnitude also changes the variance, the mean and the apparent linearity of your signals, which is exactly what data-science teams and digital twins consume years later. A setting chosen for a trend screen in 1998 is now feeding a machine-learning model nobody told the control engineer about.

You will leave with a precise mental model of exception reporting (deadband) and swinging door compression, a working Python implementation you can run against your own exports, tuning rules grounded in vendor documentation, and a clear view of when compression helps, when it silently distorts analytics, and how to stage it for twin and ML use.

What this covers: the two-stage pipeline, how swinging door really works, a runnable implementation and experiment, how OPC UA and PI-style settings map onto each other, tuning by tag class, failure modes, and a rollout checklist.

Context and Background

Industrial plants are noisy and slow at the same time. Most process variables, such as tank level, line temperature or header pressure, drift smoothly and carry sensor noise on top. Sampling at one second gives a faithful record, but the signal between two stored samples is usually well described by a straight line plus noise. Storing the line’s endpoints instead of every sample is the central idea of every industrial compression scheme.

The method most people mean by “historian compression” is the swinging door algorithm. Edgar H. Bristol, working at Foxboro, filed US patent 4,669,097, “Data compression for display and storage”, in October 1985; it was granted in May 1987 and now sits with Schneider Electric. The patent describes an error corridor around a starting point whose “doors” pivot inward as points arrive until the corridor can no longer contain a point, at which point the corridor endpoints replace the raw data. Vendors including AVEVA (the OSIsoft PI lineage), Weatherford’s CygNet and others document their own variants.

Compression is only the second half of the story. Before data reaches the archive, most collection stacks apply a simple deadband called exception reporting. AVEVA’s training material on PI System performance describes the split precisely: exception is performed by PI interfaces and conserves network bandwidth, while compression runs in the snapshot subsystem or the PI Buffer Subsystem and reduces what is stored. The 2023 AVEVA user conference deck I used as the primary source also notes that adapters and connectors do not behave identically here: AVEVA Adapters can deadband but ignore tag exception settings, and PI Connectors do not run exception tests. That matters when you migrate from interfaces to newer collectors, because the effective filtering can change without anyone editing a tag.

If you are designing the wider flow, read our walk-through of OPC UA historian to digital twin time-series ingestion architecture, and for the storage side the survey of industrial IoT time series database platform architecture. This post sits between them: it explains what the historian does to the data before either pipeline sees it.

Two further framings keep the discussion honest. First, compression is lossy and signal-dependent: the same setting gives a 3:1 ratio on a noisy flow and a 500:1 ratio on a quiet level. Second, there is a second, unrelated family of lossless codecs, such as Gorilla-style delta-of-delta and XOR encoding, used by modern time-series databases. Those operate on stored values, not on which values to keep, and we cover them in time series compression with Gorilla, delta-of-delta and XOR. A well-tuned stack uses both: lossy point selection first, lossless encoding of what survives.

The Two-Stage Pipeline: Exception Then Compression

Direct answer: Historian data compression is usually two filters in series. Exception reporting applies a deadband at the source and drops values that barely changed; swinging door compression then runs at the archive and keeps only the points needed to reconstruct the signal within a set tolerance by linear interpolation.

Figure 1: Where each filter sits. Exception reduces what crosses the network; compression reduces what is stored.

Figure 1 shows the flow for a PI-style stack, but the same shape appears in most historians. A scan produces a value; the interface tests it against the last value it sent; survivors enter the buffer and the snapshot; the snapshot test decides whether the previous held value becomes a permanent archive event. Queries then interpolate or aggregate across the sparse archive. The digital twin, the trend and the notebook all see that reconstructed view.

Stage one: exception reporting (deadband)

Exception reporting is a plain deadband. In the AVEVA parameter set it has three controls: ExcDev, the deadband size in the tag’s engineering units; ExcDevPercent, the same expressed as a percentage of the tag’s span; and ExcMax, the longest time in seconds allowed between events. A new value is sent only if it differs from the last sent value by more than ExcDev, or if ExcMax seconds have elapsed. When a value passes, AVEVA documents that both the passing value and the previous value are sent, so the receiving end sees the shoulder of a change and not only its peak. Each sent value becomes the new reference.

The ExcMax timer is the underrated parameter. Without it, a flat-lined tag is indistinguishable from a dead interface: no events, no errors. With ExcMax of ten minutes, a healthy but steady signal still produces a heartbeat, which lets downstream systems tell “unchanged” from “disconnected”. We will return to this in the failure modes section.

The same idea exists in OPC UA. A client that creates a monitored item can attach a DataChangeFilter with a deadband type of None, Absolute or Percent. Per the OPC UA specification, Part 4, an Absolute deadband generates a notification when the absolute difference between the last cached value and the current value exceeds the AbsoluteDeadband, where the last cached value is the last value pushed to the queue. It applies only to variables of a numeric type, and the deadband is ignored when the item’s status code changes, so a transition to Bad quality is never swallowed. For arrays, the whole array is returned if any element exceeds the deadband or the array dimensions change. Percent deadband is defined in Part 8; as I understand it the percentage is applied to the variable’s EURange, but I did not re-verify the exact formula for this post, so check Part 8 before relying on it.

Stage two: swinging door compression

Compression keeps a point only when removing it would make the reconstructed line leave a tolerance band. In AVEVA’s description, compression uses the swinging door algorithm to remove data that can be recreated by interpolation, with the explicit aim of filtering instrument and process noise while still recording significant changes. Its controls mirror the exception ones: Compressing as an on/off switch, CompDev for the tolerance, CompDevPercent as a percentage of span, and CompMax for the maximum time between archived events. The documented relationship is CompDev = CompDevPercent multiplied by Span divided by 100, and if both are supplied together, the percent form wins.

Two details from the same source are worth memorising. Out-of-order data bypasses compression, which has consequences for backfills and late-arriving buffered data. And default values are not zero: setting Compressing to 0 is not the same as Compressing equal to 1 with CompDev equal to 0. The second configuration still removes exactly collinear points and stores digital tags efficiently, while the first stores everything.

Why two stages and not one

It is reasonable to ask why exception exists at all if compression is more sophisticated. The reasons are cost location and a different failure surface. Exception runs on the collector, so it protects the network and the server from raw scan-rate traffic; compression needs a corridor state per tag and runs where the data is committed. AVEVA’s example makes the scale concrete. For a 0/1 status tag scanned each second, with a CompMax of 28,800 seconds, an ExcMax of 600 seconds and a span of 100, the document reports 172,800 events over 48 hours with no compression, and 15 with minimal compression. That is a ratio of roughly 11,500 to one, with no information loss at all, because a status tag between transitions has nothing to say.

Running both also compounds tolerance. A value can be off by up to the exception deadband before it is even sent, and then be interpolated away within the compression corridor. AVEVA’s guidance is to set ExcDev to half of CompDev so the two do not fight, and to set CompDev at or below instrument precision. We quantify that stacking in the experiment later in this post.

How the doors actually swing

Figure 2 traces the loop. The algorithm holds an anchor, the last archived point A. It imagines two pivots, one at A plus CompDev and one at A minus CompDev. Each new point defines two slopes: the slope of the line from the upper pivot to the point, and the slope of the line from the lower pivot to the point. The upper slope bound can only rise and the lower bound can only fall, which is why AVEVA’s diagrams note that the old extreme slopes are reused, because the slopes must constantly narrow.

Figure 2: The corridor narrows as points arrive. When the bounds cross, the previous point is archived and becomes the new anchor.

While the upper bound stays at or below the lower bound, there exists at least one straight line from the anchor that passes within CompDev of every held point. The moment the bounds cross, no such line exists. The algorithm then archives the point before the one that broke the corridor, makes it the new anchor and restarts. That detail, archiving the previous point rather than the offender, is why a step change shows up as a pair of closely spaced events: one at the last point before the step, one at its first point after.

CompMax adds a time bound. Even if the corridor never collapses, as with a perfectly flat tag, a point is archived when CompMax elapses. This produces the periodic stored value that makes flat lines verifiable, and it bounds interpolation error from missed events.

Two properties that surprise people

The first is that the error guarantee is on the filtered stream, not on the raw one. The corridor keeps the interpolated line within CompDev of every point that the compression stage saw. If exception reporting already discarded values, those never enter the corridor. The total error against the raw signal can therefore exceed CompDev, and in my experiment below it did.

The second is that compression ratio is a property of the signal, not the setting. A tag with high-frequency noise much larger than CompDev defeats the algorithm, since the corridor collapses almost every scan. A tag whose noise sits well inside CompDev compresses very hard. This is why teams that tune on a quiet test period get a nasty surprise when a pump starts cavitating and the archive rate for that tag jumps by two orders of magnitude.

A Runnable Implementation and Experiment

Reading the algorithm is not the same as feeling it behave. The following Python module implements a deadband stage that, like the AVEVA description, forwards the previous value along with the passing one, a swinging door stage that supports CompMax, and a reconstruction function that interpolates linearly as a historian does at query time. It uses only the standard library.

import math, random

def deadband(points, dev, max_gap=None):
    """Exception reporting: forward a value only if it moves more than dev
    from the last forwarded value, or max_gap seconds have elapsed."""
    out = [points[0]]
    last_t, last_v = points[0]
    prev = points[0]
    for t, v in points[1:]:
        if abs(v - last_v) > dev or (max_gap is not None and t - last_t >= max_gap):
            if prev != out[-1]:
                out.append(prev)      # also forward the previous value
            out.append((t, v)); last_t, last_v = t, v
        prev = (t, v)
    return out

def swinging_door(points, comp_dev, comp_max=None):
    """Swinging door compression. Returns the archived subset of points."""
    archived = [points[0]]
    a_t, a_v = points[0]
    s_max, s_min = -math.inf, math.inf   # upper and lower slope bounds
    prev = points[0]
    for t, v in points[1:]:
        dt = t - a_t
        s_max = max(s_max, (v - (a_v + comp_dev)) / dt)   # bounds only narrow
        s_min = min(s_min, (v - (a_v - comp_dev)) / dt)
        too_old = comp_max is not None and dt >= comp_max
        if s_max > s_min or too_old:                      # corridor collapsed
            archived.append(prev)
            a_t, a_v = prev                               # new anchor
            dt = t - a_t
            s_max = (v - (a_v + comp_dev)) / dt
            s_min = (v - (a_v - comp_dev)) / dt
        prev = (t, v)
    archived.append(prev)
    return archived

def reconstruct(archived, times):
    out, j = [], 0
    for t in times:
        while j < len(archived) - 2 and archived[j + 1][0] < t:
            j += 1
        (t0, v0), (t1, v1) = archived[j], archived[j + 1]
        out.append(v0 + (v1 - v0) * (t - t0) / (t1 - t0) if t1 != t0 else v0)
    return out

if __name__ == "__main__":
    random.seed(7)
    n = 86400                                             # one day at 1 Hz
    pts = [(t, 50 + 8 * math.sin(t / 3600) + random.gauss(0, 0.15)) for t in range(n)]
    for dev in (0.1, 0.3, 0.5, 1.0):
        ex = deadband(pts, dev / 2, 600)                  # ExcDev = CompDev / 2
        sd = swinging_door(ex, dev, 28800)
        rec = reconstruct(sd, [t for t, _ in pts])
        err = max(abs(r - v) for r, (_, v) in zip(rec, pts))
        print(f"CompDev={dev:<4} raw={n} exc={len(ex)} arch={len(sd)} "
              f"CF={n / len(sd):.1f} maxerr={err:.2f}")

I ran this once with the seed shown. The signal is synthetic and illustrative: a slow sine of amplitude 8 around 50 with Gaussian noise of standard deviation 0.15, one day at 1 Hz. It is not a benchmark of any product, and your plant’s signals will behave differently.

CompDev	After exception	Archived points	Compression factor	Max error vs raw
0.1	83,018	45,535	1.9	0.20
0.3	60,752	5,015	17.2	0.60
0.5	34,013	145	595.9	0.81
1.0	4,283	17	5,082.4	1.73

Table 1: Illustrative output of the script above on synthetic data. The cliff between CompDev 0.1 and 0.5 is the point at which tolerance overtakes the noise.

Three lessons come out of the table. When CompDev is below the noise amplitude, the compression factor is about 2 and you are archiving noise. When CompDev passes the noise band, the factor leaps by two orders of magnitude within a narrow range of settings; the curve is a cliff, not a slope. And the maximum error against the raw signal is larger than the filter’s nominal tolerance at the aggressive end, 1.73 against a CompDev of 1.0, because exception and compression errors stack and because the slow sine is cut by chords. That is why measured error on your own data beats any rule of thumb.

Figure 3: The evaluation loop used in the experiment. Always compare reconstruction against the raw signal, not against the filtered stream.

Figure 3 is the loop worth building around any compression change. Capture a representative window of raw data at scan rate, apply candidate settings offline, reconstruct on the original timestamps, and compute errors and statistics, not only the compression factor. AVEVA recommends the same discipline: test stronger deviation values with parallel test tags, partner products or spreadsheet tools before changing production tags.

What Compression Does to Analytics and Digital Twins

Compression is invisible to the operator watching a trend, because linear interpolation looks smooth. The cost lands on whoever computes with the data later.

The evidence from process-control research

The most cited study is Thornhill, Choudhury and Shah, “The impact of compression on data-driven process analyses”, in the Journal of Process Control, volume 14, 2004. They examined piecewise linear compression such as swinging door and reported several effects. Compression changes the mean and variance of reconstructed signals: mean shifts are small but could be mistaken for real events such as leaks, and the reconstructed variance is smaller than the original. The reconstruction error is correlated with the compressed signal, so familiar variance decompositions stop adding up.

More worrying, two of their three data sets were linear when uncompressed but looked non-linear at a compression factor of 4, which could send a team to inspect a valve that is working normally. The Harris index, a minimum-variance benchmark used to rank control loops, rose with compression for all data sets, which could trigger maintenance on loops that perform well. The authors suggested that data with a compression factor of 3 or less may be used with caution, and “strongly recommend caution” about compression in process archives. In their industrial historian data, compression factors reached 93.8 and only five tags had a factor of 3 or less. Their detection method estimates the factor by counting zero second differences in data reconstructed at the original sample interval.

That paper is twenty years old, but the mechanism has not changed, and modern machine learning inherits it. Anomaly detectors learn variance, spectral content and autocorrelation. A compressed history has deflated variance, missing high-frequency content and artificially straight segments. A model trained on that history and then served raw data at scan rate sees a distribution shift on day one.

Interpolation is a modelling choice

When you ask a historian for the value at 10:00:00.5 it computes a linear interpolation between two archived points. For a continuous process variable that is a defensible default. For a valve position that stepped, a mode flag, or a counter, it can be wrong. Stepped tags should be configured so retrieval holds the previous value, and the choice must travel with the data into the twin. If your twin consumes an hourly average, remember that an average over a sparse archive is a time-weighted average of the interpolated line, not an average of samples.

Aggregates are safer than raw replays

Downstream consumers should ask the historian for aggregates, such as time-weighted average, minimum, maximum and count, rather than pulling raw archive events and treating them as uniform samples. The archive’s events are non-uniform by design; resampling them naively onto a regular grid with last-value-fill creates staircases that no sensor produced. Our ingestion article on OPC UA historian to digital twin time-series ingestion covers the aggregate read pattern, and the storage trade-offs for the target store are in the survey of Azure time series databases, Data Explorer and Cosmos DB.

Mapping Settings Across Historians and Protocols

Terminology differs by vendor, but the concepts line up. The table maps the PI-style names to the OPC UA monitored-item model and to the generic idea. I only state documented behaviour for the AVEVA and OPC UA columns; for other products, consult their own documentation, since defaults and semantics vary.

Concept	PI-style tag attribute	OPC UA equivalent	Notes
Deadband size, engineering units	ExcDev	AbsoluteDeadband in DataChangeFilter	Applies at the source or subscription
Deadband as percent	ExcDevPercent	Percent deadband (Part 8)	Percent of span or EURange
Heartbeat for flat signals	ExcMax	Sampling plus keep-alive, not identical	OPC UA keep-alive is a connection check, not a value refresh
Compression on or off	Compressing	No equivalent in the protocol	Archive-side feature
Compression tolerance	CompDev, CompDevPercent	No equivalent	Swinging door at the historian
Forced archive interval	CompMax	No equivalent	Bounds interpolation gap

Table 2: Mapping of settings. Exception has a protocol-level cousin; compression belongs to the historian.

A practical consequence follows. If a plant moves from a classic interface to OPC UA-based collection and keeps the same historian, the exception filter may now live in the subscription’s DataChangeFilter, or in a connector that does not run exception at all. Compare event rates per tag before and after; a sudden tenfold rise in snapshot events with unchanged compression settings is the telltale sign that exception disappeared.

Figure 4: Choosing settings by tag class. Critical and regulated tags get minimal compression; noisy analogs are filtered at source first.

Tuning Rules by Tag Class

Tuning is a classification problem before it is a numeric one. The AVEVA material reports a pilot in which tags were categorised as critical or non-critical, with about 1.3 percent critical and 98.7 percent non-critical, and as analog or digital, 37 percent and 63 percent respectively. Those proportions come from one oil and gas pilot of 70 tags and should not be generalised, but the lesson transfers: most tags are boring and deserve a default policy, a few are special and deserve a hand-tuned one.

A default policy that is hard to regret

The documented baseline for non-critical tags is to use at least minimal settings: Compressing set to 1 with CompDev of 0, and ExcDev of 0 with ExcMax larger than the scan class period. Minimal compression removes only values that lie exactly on a straight line. For discrete and status tags, which are most of any plant, that already removes nearly everything redundant. For analog tags it is a safe starting point that still rejects perfectly flat repeats.

The next step is to set CompDev at or below the instrument’s precision, with ExcDev at half of CompDev. The reasoning is physical: a deviation smaller than the sensor’s resolution cannot represent a real change, so discarding it removes noise rather than information. AVEVA’s sample master table for a Celsius temperature tag uses CompDev of 1, CompMax of 28,800 seconds, ExcDev of 0.5, ExcMax of 600 seconds and a 10-second scan, and for a digital status tag CompDev of 0, CompMax of 86,400, ExcDev of 0, ExcMax of 28,800 and a one-second scan. Treat these as worked examples from the vendor, not universal recommendations: a one-degree tolerance is wrong for a reactor jacket controlled to half a degree.

Reported results from the field

The same deck reports two case results worth knowing. In a downstream oil and gas study, compression at 11 percent still cut analysis backfill time by roughly 20 percent, half the tags met or beat a 3:1 compression ratio, and all maximum deltas stayed below measurement precision. In an offshore pilot, roughly 87 percent of tags had no exception or compression before the change and about 98 percent used tuned settings afterwards; exception alone cut data sent over the wire by about 70 percent, and exception plus compression cut onshore archive storage by about 94 percent. These are vendor-presented, site-specific results, so use them as order-of-magnitude expectations and not as promises.

Percent-of-span versus absolute settings

Percent settings are attractive because they scale with the instrument range, so a single policy can cover many units. They also hide a trap: if someone edits the zero or span of a tag, a percent-derived deviation silently changes in engineering units. AVEVA documents that changing CompDevPercent recalculates CompDev from Span, and vice versa. In regulated or safety-relevant applications prefer an explicit engineering-unit value that you can justify against instrument precision, and put span edits under change control.

A tuning workflow that survives audit

Pull a week of raw data at scan rate for a representative sample of tags, covering startups, upsets and steady state.
Estimate noise per tag, for example the standard deviation of the first difference, and compare it with instrument precision from the datasheet.
Replay candidate CompDev values offline with a script like the one above, and record compression factor, maximum error and changes in mean and variance.
Check the downstream consumers: do not compress a tag harder than its most demanding user can tolerate, whether that is a loop-performance study, an alarm rationalisation or a twin.
Roll out on parallel test tags first, compare, then switch the production tags.
Document the setting, its rationale and the date, because six years later someone will ask why the data looks smoothed.

Trade-offs, Gotchas, and What Goes Wrong

Compression is a loan against future questions. Most failure modes are not bugs in the algorithm; they are mismatches between the setting’s original purpose and a later use.

Compression ratio instability. Because the factor depends on signal behaviour, a tag’s archive rate can jump when the process changes. A sticking valve that begins to oscillate, or a transmitter with a failing shield that starts to pick up noise, will defeat the corridor and raise storage and event load. Monitor archived events per tag per day and alert on a sudden multiple; it is a cheap, early indicator of instrument degradation, and a nice side benefit of a compressed system.

Silent flat lines. A steady signal with a long CompMax produces almost no events. That is correct, but a disconnected collector looks identical. Use ExcMax and CompMax deliberately as heartbeats, and add quality or connection-status tags that are compressed minimally. The rule: no safety-relevant tag should be allowed to be silent for longer than its worst acceptable detection time.

Out-of-order and backfilled data. AVEVA documents that out-of-order data bypasses compression. A site that backfills a day of buffered values after an outage can therefore store far more events than the same day’s live data, and the stored detail differs between the live and recovered periods. Compare compression factors across outages when you build models, or you will find a mysterious change in variance exactly where the network failed.

Double compression across tiers. Data are often compressed in the historian, exported to a cloud store through a connector, sampled onto a regular grid and then compressed again by a lossless or lossy codec. Errors accumulate and nobody owns the sum. Decide which tier owns the lossy step, and make downstream tiers lossless. Where practical, publish raw or lightly filtered values on the cloud path, and use the historian purely for operations.

Alarms and event triggers. If a threshold alarm is evaluated on archived data, a short excursion between two stored points may never appear. Alarm logic belongs on live values at scan rate, not on the compressed archive. Likewise, event frames built from archived tags inherit the interpolation.

Timestamps and quality. Compression deals with values and timestamps but must never discard a quality change. The OPC UA rule that a deadband is ignored when the status changes is the right design; check your collectors for the same behaviour, otherwise a transition to a bad-quality state can be swallowed by the filter and the twin continues to trust a stale number.

Regulatory and contractual contexts. Some applications, such as custody transfer, emissions reporting and batch records, require original resolution or defined accuracy. Compression there is a compliance question as much as an engineering one. This post is engineering analysis, not compliance advice; take requirements from your applicable standards and regulators.

Not a substitute for filtering. Swinging door removes noise only to the extent that the noise stays inside the corridor. It does not low-pass the signal. If you need a smoothed signal, filter deliberately and say so; do not rely on compression to do it implicitly and unpredictably.

Practical Recommendations

The core advice is to treat compression settings as part of the data contract, not as a storage knob. Every tag has a stated tolerance, a stated reason and a named owner, and every downstream consumer is told what was done to the data before it reached them.

For a plant starting today, begin with minimal compression everywhere, which is cheap and almost lossless. Tighten selectively for storage or network pressure on tags with proven high event rates, and loosen nothing that feeds a regulatory report or control-loop analytics without evidence. For a twin or machine-learning programme, keep a separate raw or near-raw feed for the signals that matter most, even if it covers only a few weeks, so models can be validated against the unfiltered truth. Where the twin needs uniform sampling, resample from historian aggregates and record the method.

If you are migrating collectors, compare per-tag event rates before and after. If you are consolidating historians, audit the compression policy of each source, since combining a heavily compressed legacy archive with a lightly compressed new one will give a time series whose statistics change at the migration date. For architectures with a unified namespace and brokered telemetry, decide explicitly whether deadband is applied at the edge publisher; the MQTT side of that design is covered in MQTT store-and-forward at the edge gateway.

Checklist:

[ ] Classify tags as critical or non-critical, and analog or digital.
[ ] Set Compressing on with minimal CompDev for all non-critical tags as the floor.
[ ] Set CompDev at or below instrument precision; ExcDev at half of CompDev.
[ ] Set ExcMax and CompMax as heartbeats and document the detection-time reasoning.
[ ] Replay candidate settings offline and record compression factor, error, mean and variance shifts.
[ ] Keep percent-of-span settings under change control, or use engineering units.
[ ] Verify collectors preserve quality changes and out-of-order handling.
[ ] Monitor archived events per tag and alert on large multiples.
[ ] Give twins and ML pipelines aggregate reads and a short-window raw feed for validation.
[ ] Record the policy, owner and date, and review it annually.

Frequently Asked Questions

What is historian data compression?

Historian data compression is the lossy reduction of industrial time-series data before or while it is archived. Typically an exception deadband filters values at the collector, then an algorithm such as swinging door stores only the points needed to reconstruct the signal by linear interpolation within a tolerance. It reduces storage and network load, but changes statistics like variance, so settings should match how the data will later be used.

How does swinging door compression work?

The algorithm anchors on the last archived point and sets a tolerance band of plus or minus CompDev around it. Each new point narrows two slope bounds. While a single straight line can pass within tolerance of every held point, nothing is stored. When the bounds cross, the previous point is archived and becomes the new anchor. CompMax additionally forces a stored point after a maximum interval.

What is the difference between exception and compression?

Exception is a simple deadband run by the collector: a value is sent only if it moves more than ExcDev from the last sent value or ExcMax seconds pass. Compression runs at the archive using swinging door and decides which snapshot values become permanent events. Exception saves bandwidth; compression saves storage. AVEVA suggests setting ExcDev to half of CompDev so the stages stay consistent.

Does compression hurt machine learning and digital twin accuracy?

It can. Research by Thornhill, Choudhury and Shah found that compression shifts means, lowers reconstructed variance and can make linear data appear non-linear at a compression factor of about 4. They suggested caution above a factor of 3. For models, keep a raw validation feed, use time-weighted aggregates, and avoid training on heavily compressed data without checking that the live feed has the same character.

What compression deviation should I use?

Use a value at or below the instrument’s precision, with exception set to half of it, as AVEVA recommends for non-critical tags. For status tags, zero deviation with a long maximum interval already compresses extremely well. For critical or regulated tags, start with minimal compression. Confirm any stronger value by replaying raw data offline and checking error and statistics, because the right number depends on the noise of each signal.

Is deadband in OPC UA the same as historian exception reporting?

They are the same idea at different layers. In OPC UA, a DataChangeFilter on a monitored item can apply an absolute or percent deadband relative to the last value queued, and it is ignored when status changes. Historian exception reporting is configured per tag on the collector. Neither replaces archive-side compression, and moving collectors can change which filter actually runs, so measure event rates after any migration.

Historian Data Compression: Deadband, Swinging Door and Exception Reporting for Industrial Time Series (2026)