The False Precision Problem in Spatial Data

The False Precision Problem in Spatial Data

E
By Etzal Earth
12 min read

A building footprint has an area of 412.7364 square metres. A point of interest sits at 106.8271943, minus 6.1753621. A pipeline reports a road segment as 1,847.23 metres long. Every one of those figures is stated to a precision that its underlying method cannot support, and none of them looks suspicious.

False precision is not a rounding preference. It is a claim, made implicitly by the number of digits, about how well something is known. When a figure carries more digits than its method justifies, the surplus digits are not neutral decoration. They are an assertion, and they are read as one, particularly by readers who have no other signal about quality.

The damage is downstream. A number that looks exact gets treated as exact, and it survives into comparisons, thresholds, and decisions that a properly hedged figure would never have been allowed to touch.

Precision and accuracy are separate properties

Precision describes the resolution of a measurement: how finely it is expressed, and how consistently a repeated measurement lands in the same place. Accuracy describes how close it is to the truth. A system can have either without the other, and spatial data manages to produce every combination.

A consumer GNSS receiver reporting to seven decimal places is precise to a fraction of a millimetre and accurate to several metres. Repeat the reading and it moves, but each reading is stated with the same false confidence. A hand digitised building outline traced from imagery has vertices at exact coordinates, but the outline as a whole may sit metres away from the structure, because the imagery was not orthorectified perfectly and the tracer was working at a zoom level where a pixel covered a metre of ground.

The two properties fail differently, which is why conflating them is expensive. Imprecision is visible and self limiting: nobody over reads a figure given to the nearest hundred metres. Inaccuracy is invisible, and precision actively conceals it, because a long decimal expansion signals care.

The practical formulation: digits should express what the method resolves, not what the float type stores. A double precision floating point number will happily hold fifteen significant digits regardless of whether the measurement contained one.

Coordinates with more decimals than anybody earned

Decimal degrees make this easy to reason about, because the scale is fixed and well known. A degree of latitude is roughly 111 kilometres anywhere on Earth. That makes the fourth decimal place worth roughly ten metres, the fifth roughly one metre, the sixth roughly ten centimetres, and the seventh roughly a centimetre. Longitude spacing shrinks with the cosine of latitude, so the same digits buy less distance near the equator than they do near the poles, but the order of magnitude holds.

Set that against how spatial points are actually produced. A geocoded address is usually interpolated along a street segment or snapped to a parcel centroid, so its true uncertainty is tens of metres in dense areas and can be much worse where address ranges are sparse. A point digitised from imagery inherits the georeferencing error of that imagery. A crowd sourced point captured on a phone inherits the receiver error, plus whatever the contributor did while standing near, but not exactly at, the thing being recorded. A centroid of an administrative polygon is not a location at all, it is a summary statistic of a region.

Almost none of those methods justify a sixth decimal place, and many do not justify a fourth. Yet the values propagate at full width because nothing in the pipeline ever truncates them. The float came out of a database, went into JSON, and arrived at a client that rendered it verbatim.

The problem is not aesthetic. Excess coordinate precision defeats deduplication, because two records for the same object never compare equal and matching has to fall back to distance thresholds. It creates the illusion that two nearby points are distinguishable when they are within each other's error. And in an interface that shows coordinates to a user, it invites the user to assume a survey grade fix that nobody ever performed.

Areas computed to the square metre from outlines accurate to several metres

Polygon area is where precision inflation becomes most obviously indefensible, because the arithmetic is exact and the inputs are not.

Take a building footprint traced at a scale where the operator could place a vertex to within about two metres. The polygon that results has perfectly defined vertices, and the shoelace formula will return its area to as many digits as anybody wants. But a systematic two metre offset on each edge of a modest rectangular building changes the true area by a substantial fraction, and the computed figure gives no hint of it.

The general shape of the problem: for a compact polygon, a uniform positional error on the boundary produces an area error that scales with the perimeter times that error. Small buildings are hit proportionally hardest, because their perimeter to area ratio is highest. So a dataset of footprint areas is not uniformly uncertain. It is roughly acceptable for large structures and badly uncertain for small ones, and the reported digits are identical in both cases.

The same argument applies to lengths derived from digitised lines, volumes derived from areas and modelled heights, and any count normalised by one of those quantities. Density per square kilometre inherits everything wrong with the square kilometre.

A reasonable convention is to round derived quantities to a precision consistent with the input error, and to state that precision. An area good to within roughly ten per cent should be reported to two significant figures, not to four decimal places. It will look less impressive. It will also stop a downstream user from computing a difference between two such areas and believing the result.

Unit conversion manufactures digits out of nothing

Conversion is the quietest source of fake precision, because it is arithmetically correct at every step.

A height recorded as 30 feet, which in context clearly means roughly thirty, converts to 9.144 metres. Four significant figures now exist where two did before, and the fractional millimetre is pure artefact. A distance stated as five miles becomes 8.04672 kilometres. A survey conducted in whole metres becomes a set of centimetre valued numbers after a projection change and a reprojection back.

The rule from measurement practice is that a converted value carries the significant figures of the original, not the significant figures the conversion factor happens to produce. Thirty feet is 9.1 metres. Five miles is about 8 kilometres. Pipelines almost never implement this, because the conversion happens in a utility function that has no idea how many figures the input actually had, and because the input arrived as a float that lost that information at parse time.

The same erasure happens with rounded source data. Values published to the nearest hundred, or binned into ranges and then represented by a midpoint, arrive as exact looking numbers. A midpoint of a range is a choice, not a measurement, and after two or three transformations nobody remembers that the value was ever a bin.

The defence is to carry the source precision as a field rather than trying to infer it later. If a record knows it came in as whole feet, or as a bucket midpoint, or as a value rounded to the nearest fifty, every downstream consumer can decide correctly. If it does not, every downstream consumer will guess, and they will guess in favour of the digits in front of them.

Aggregation looks like it cleans up error, and mostly does not

There is a comfortable belief that summing or averaging many uncertain values produces a reliable result, because errors cancel. That belief is true for one kind of error and false for the kind that dominates spatial data.

Random error does shrink under aggregation. If positional errors are independent and centred on zero, averaging many of them reduces the spread of the mean. Systematic error does not shrink at all. If every footprint in a dataset was digitised from imagery with the same georeferencing offset, or every polygon was traced by an operator who consistently drew outside the roof line, or every area was computed in a projection that inflates at that latitude, then summing a million of them produces a total that is wrong by the same proportion as each part. Aggregation makes it smoother and more confident, not more correct.

Spatial data is unusually rich in systematic error, because records are produced in batches by shared processes. One imagery acquisition, one operator, one model version, one vendor, one projection. Errors within a batch are correlated by construction. The independence assumption that makes cancellation work is exactly what batch production destroys.

The practical implication is a habit rather than a formula: before trusting an aggregate, ask what the records have in common. If they share a producer, a sensor, a model, or a coordinate reference system, treat the shared component of their error as surviving the aggregation intact.

Projection choice silently inflates measurements

Computing area or length in geographic coordinates, or in a projection that does not preserve the quantity being measured, produces figures that are wrong by a factor that varies with location. The result is precise, repeatable, and wrong.

Web Mercator is the standard offender, because it is the coordinate system that map tiles arrive in, and it is therefore the one that data ends up sitting in by default. It is conformal, meaning it preserves local angles and shapes, and it does that by stretching scale as latitude increases. The linear scale factor grows as the secant of the latitude, so area grows as roughly the square of that. Near the equator the effect is negligible. At mid latitudes it is large. At high latitudes it is enormous. An area computed in Web Mercator is not approximately right, it is right only along the equator.

The failure is hard to catch because it is monotonic and smooth. Nothing looks broken. Values are consistent with each other within a city, so internal comparisons still rank correctly, and the bug only surfaces when someone compares across latitudes or checks a figure against a known ground truth.

The correct approach is unglamorous: compute area and distance either on the ellipsoid using geodesic methods, or in a projection chosen for the property being measured and the region being measured in, such as an equal area projection for areas or an appropriate local system for distances. Then record which was used. A figure reported without its coordinate reference system is not a measurement, it is a number that happens to have units attached.

Typography that destroys the reader's ability to weigh a number

An interface renders a measured building area and a modelled population estimate in the same font, at the same size, in the same colour, in adjacent cells of the same table. One of them came from a survey. The other came from a model with wide uncertainty, built on inputs that themselves came from another model. The reader has no way to tell, and will not go looking.

This is a design failure with data consequences, and it is worth naming as such, because it is usually invisible to both the data team and the design team. The data team knows the difference and assumes the interface conveys it. The design team optimises for a clean, consistent table, and consistency is precisely what erases the distinction.

Interfaces that handle this well do a small number of things. They vary the precision itself, so that a modelled figure appears rounded and a measured one does not, which communicates uncertainty through the number rather than around it. They mark derived values, with a lightweight visual convention applied consistently rather than a footnote nobody reads. They show a range instead of a point estimate when a range is what the method produced. And they make provenance reachable in one interaction, so that a reader who wants to know where a figure came from can find out without leaving the context.

The same reasoning applies to API responses, and more strongly, since a machine reader has no visual channel at all. If a response mixes measured and modelled values in one flat object with identical types, the caller cannot weigh them, and any agent consuming that response will present both with equal confidence.

Reporting to the precision the method supports

The rule is simple to state and consistently unpopular to apply: report to the precision the method supports, and state what that precision is.

That means truncating coordinates to the resolution the source justifies rather than to the width of the float. It means rounding derived quantities to significant figures consistent with their inputs. It means preserving source precision as metadata at ingestion, because it cannot be recovered afterwards. It means recording the coordinate reference system used for every computed measurement. And it means shipping an uncertainty estimate alongside a figure whenever one can be produced, even a crude one, because a crude bound is more useful than an implied bound of zero.

The objection is always the same, and it is honest: a rounded figure looks less capable. A table of areas to the nearest ten square metres reads like a weaker product than a table to four decimal places, and a competitor who does not round will look more precise to anyone comparing screenshots.

That comparison is worth losing. The customer who eventually checks a figure against ground truth is the one who decides whether the product is trusted, and a number that claimed a centimetre and delivered a metre fails that check permanently. A number that claimed a metre and delivered a metre passes it. The digits that were dropped were never information, and the credibility they cost was real.