Melbourne, Australia 0428 625 394 tasman.murray@holisticanalytics.com.au

Home  /  Insights  /  Measurement

What signal loss actually changed, and what it didn't

Signal loss was treated as a tracking problem with a tracking solution. Framed that way, the fixes are all temporary, because the constraint is regulatory and cultural, not technical.

Tasman Murray  ·  Managing Partner  ·  14 June 2023

Signal loss removed the ability to observe individual journeys across sites and apps. It did not remove the ability to measure marketing effectiveness. That only depended on individual tracking because the industry chose methods that did. Aggregate modelling techniques that predate digital advertising were never affected.

The distinction matters because it determines whether you spend the next few years patching or rebuilding.

The three responses, and their shelf life

Rebuild the tracking. Server-side tagging, first-party identifiers, hashed email matching, conversion APIs. These work, and they are worth doing. They are also a moving target: each depends on a permission environment that keeps tightening, and each requires ongoing engineering to maintain. Useful, but not a foundation.

Accept modelled conversions. Platforms now fill observation gaps with their own estimates. This is a reasonable statistical response to a real problem. It is also an obvious conflict of interest: the party estimating your results is the party selling you the media. We do not tell clients to ignore these numbers, but we do tell them not to use them as the independent measure of anything.

Move measurement up a level of aggregation. Model outcomes against inputs at the market or region level, where privacy constraints do not apply because no individual is being observed. This is what marketing mix modelling has always done. It was unfashionable during the decade when individual tracking looked complete, and it is durable for precisely the reason it was unfashionable.

What actually got better

Two things improved, and they rarely get acknowledged.

First, the illusion of completeness is gone. Digital measurement was never comprehensive. It was comprehensive about a subset and silent about everything else, which is more dangerous than being visibly incomplete. Teams now know they are working with partial observation, and that changes how they treat a number.

Second, offline channels stopped being second-class. When the comparison was tracked digital against untracked television, digital won by default. When everything has to be inferred from aggregate response, channels compete on measured contribution instead of measurability. In several engagements that shift alone has been enough to reopen a media mix that had been closed for years.

A workable structure

  • Aggregate modelling as the source of truth for what each channel contributes. Privacy-durable, includes offline, includes competitors and price.
  • Geographic and time-based experiments to validate the model against reality. Holding out a region is unglamorous and remains the cleanest causal evidence available.
  • Platform data as an operational signal, for in-flight optimisation, not for reporting contribution to the board.
  • Lead metrics for in-flight tracking, sitting between brand and sales, so a long campaign can be diagnosed months before the sales data confirms it.

The organisations that handled signal loss best were not the ones with the best tag management. They were the ones that already had a measurement framework which did not depend on watching individuals.

Observed across engagements, 2021 to 2023

The question worth asking internally

If a regulator or a platform removed another observation method next year, which of our reported numbers would stop working?

Every number that fails that test is a dependency, not a measurement. It is a useful audit, it takes an afternoon, and the results are usually uncomfortable in a productive way.

Want this applied to your business?

Tell us the decision you are trying to make. If we are not the right people, we will say so and point you somewhere better.

Start a conversation