The question arrives late, and it always arrives from the same place. A customer's legal team asks which licences apply to the data behind a product, whether any of them require the customer to open their own database, and whether anything in the pipeline prohibits commercial use.
At that point the pipeline either has an answer or it does not. If licence information was recorded per record at ingestion and carried through every transformation, the answer is a query. If it was not, it is an archaeology project across ETL scripts, storage buckets, caches, and the memories of whoever built each stage.
Licence cleanliness is a structural property of a pipeline. It is cheap to build in and expensive to retrofit, and the cost rises with every join, every cache, and every derived table.
What licence clean actually means
The phrase is used loosely, so it is worth defining precisely. A licence clean pipeline is one where, for any output, the following can be answered without inspecting code or asking a person: which sources contributed to it, individually rather than as a general list; under what terms each was obtained; what obligations those terms impose on this specific output, which depends on how the data was used and not only on which licence applies; whether those obligations have been met in every channel through which the output is delivered; and what the recipient may lawfully do with it.
Note the shape of that definition. It says nothing about only using permissive data. Plenty of good pipelines contain share alike sources, non commercial sources, and sources with unusual restrictions. Clean means known and separable, not restriction free. The failure mode is not using a share alike source. The failure mode is not knowing that a share alike source is in the output.
Why retrofitting is so much harder than building it in
Adding licence tracking at ingestion is a schema field and a small amount of discipline. Reconstructing it afterwards is a research project, for reasons that compound.
Provenance is destroyed by ordinary operations. A join produces rows whose ancestry is implicit in the code that produced them. An aggregation collapses many rows into one and typically keeps none of their sources. A deduplication step picks a winner and discards the losers, along with the record of which source the winner came from. None of these operations preserve lineage unless someone deliberately makes them.
Caches and derived artefacts multiply the surface. Tile sets, precomputed indices, search documents, materialised views, and model training sets are all copies with no lineage attached.
Institutional memory expires, and sources change terms. The person who added a source two years ago and knew what it permitted has moved on, the terms page may have been rewritten, and without a record of when the data was obtained, even the question of which version applied becomes unanswerable.
The asymmetry is the practical argument. Recording licence at ingestion costs a field and a policy. Recovering it later costs a team, and the recovery is never complete, because some of the information no longer exists anywhere.
Record licence per record, and never flatten it
The core rule is that licence lives at the finest granularity that ingestion can support, which is usually the record.
Dataset level tracking looks sufficient and fails immediately, because real pipelines mix. A points of interest table gets rows from several sources. A building layer combines an official release with contributed edits and model inferences. Once rows from different sources sit in one table, a dataset level licence field describes the table's origin story rather than any individual row, and any query that filters, joins, or samples produces a result whose licence composition is unknown.
A workable minimum per record: a source identifier pointing into a source registry, the licence identifier as it stood at retrieval, the retrieval timestamp, and where available a stable upstream identifier for the original object. The registry holds what does not vary per record, including the required attribution string, the terms link, whether share alike applies, whether commercial use is permitted, and whether modification notice is required.
Then the harder discipline: never flatten it. Every transformation carries licence forward, and the rules differ by operation. A filter preserves the source of each surviving row. A join produces rows subject to the union of both inputs' obligations, which is the case people most often get wrong, since a permissive row joined to a share alike row is not permissive. An aggregation is subject to the union of everything that contributed. A deduplication that picks one record's geometry and another's attributes has produced a record derived from both.
The array of contributing sources on a derived row is not elegant. It is what makes every later question answerable.
Keeping share alike and permissive data separable
Share alike terms, of which the Open Database Licence is the most familiar in the geospatial world, require that a derived database be released under the same terms. Whether that applies to a given output depends on how the data was used, which is exactly why separability matters: it preserves the ability to make that determination, and to produce outputs free of share alike material when that is required.
Separation has to hold at four levels, and a leak at any one of them defeats the other three.
Storage. Keep share alike sources in identifiable tables, schemas, or namespaces rather than merged into a general pool. The point is not physical isolation, it is being able to answer which of these rows are share alike without inspecting each one.
Joins and derivations. This is where separation is usually lost, and it is lost silently. A join that enriches permissive records with a share alike attribute has produced a result that is no longer separable. The remedy is not to forbid such joins, it is to mark their outputs correctly and to keep a parallel path that does not include them.
Caches and precomputed artefacts. A cache key that ignores licence composition will happily serve a share alike derived value to a request that was supposed to receive permissive data only. Tiles, search indices, and materialised aggregates all share this failure mode. Licence composition belongs in the cache key.
API responses. If an endpoint can return a mixture, the response must state its composition, and any filtering the caller requested must be enforceable. An endpoint that promises permissive only output over a mixed store, with no mechanism to enforce the split, is making a promise the pipeline cannot keep.
The payoff is the ability to serve customers with genuinely different constraints from one system. Some cannot accept share alike material in their products. Others do not care. A separable pipeline serves both. A merged one serves the least constrained customer only, and does not find out until someone asks.
Derived database versus produced work
The distinction that decides what obligations travel is between a derived database and a produced work, and it recurs across share alike data licences even where the terminology differs.
A derived database is a database made from another database: filtered, corrected, joined, enriched, restructured, translated into another format. Share alike obligations typically attach here.
A produced work is something made using the database that is not itself a database: a rendered map image, a chart, a report, an analysis, an application screen. Share alike obligations typically do not force such a work to be released under the data licence, though attribution and notice requirements still apply.
The distinction matters more for engineering than the terminology suggests, because products cross it constantly and the crossings are not obvious. A vector tile set is closer to a database than to an image, since it is structured data intended to be queried and re rendered, while a raster tile is closer to a produced work. An API returning geometries and attributes is distributing data. An API returning a single computed score may not be, depending on how much of the underlying data a caller can reconstruct by asking repeatedly: an endpoint returning a boolean per coordinate reveals an entire layer to anyone querying a fine grid.
Nothing here is legal advice, and a specific product needs specific review. The engineering point is prior to the legal one: the pipeline must record enough to make the determination possible. A system that knows which outputs contain redistributable data, which contain only computed summaries, and which sources contributed to each is a system where a lawyer can give a clear answer. A system that does not know is one where the answer is expensive regardless of what it turns out to be.
Imagery terms are not data terms
Imagery is the most common place an otherwise careful pipeline acquires a problem, because it arrives through different channels than vector data and its terms follow different conventions.
Basemap tiles, satellite scenes, aerial photography, and street level imagery each carry their own restrictions, and several patterns recur. Viewing permissions and derivation permissions are frequently distinct: a licence may permit displaying imagery in an application while prohibiting extracting features from it. Caching and retention limits are common. Attribution requirements are usually stricter than for vector data, and often specify placement. Restrictions on training machine learning models on the imagery are easy to miss, because they were not present in older versions of the same terms.
The dangerous move is deriving vector data from imagery whose terms do not permit it. Extracted footprints inherit the imagery's restrictions, so a layer traced or inferred from restricted imagery is not free of them merely because the output is a polygon. Records of this kind end up deep inside a pipeline with no indication of where they came from, particularly when the extraction happened in a one off project years earlier.
The corresponding rule: treat imagery as a source in the registry, exactly like a vector source, and mark every derived feature with the imagery it was extracted from. The imagery may be deleted after processing. The obligation attached to what was extracted from it does not go away with the file.
Auditing a pipeline you inherited
Most engineers meeting this problem are not building from scratch. They have a system of unknown licence composition and a reason to find out. A workable order of operations, from cheapest to most expensive.
Start with what leaves the building: endpoints, exports, downloads, tiles, reports, embedded widgets, and anything shipped to a customer. Outputs are finite and visible in a way that internal tables are not, and they define the actual exposure. Trace each one backwards, a hop at a time, to the tables and jobs that produce it. This is tedious and it is the only reliable method.
Enumerate the sources found, and for each establish the terms as they were at acquisition where that can be determined, and as they stand now. Classify by obligation rather than by name: attribution only, share alike, non commercial, restricted derivation, unclear. Unclear is a real category and belongs on the list rather than being resolved optimistically.
Identify the crossings, meaning every point where a share alike or restricted source enters a path that produces an output intended to be unrestricted. These are the findings, and there are usually fewer of them than the size of the audit suggests, because most restricted sources feed a small number of paths. Each is fixed by removal and accepting the gap, by replacement with an equivalently useful source, or by accepting the obligation and meeting it. The last is often the right answer and is treated as a last resort more often than it deserves.
Finally, add the tracking that would have prevented the audit, because an audit without instrumentation has to be repeated the next time anyone asks.
Refusing a source with unclear terms
The hardest discipline in a licence clean pipeline is turning down data that would be genuinely useful because its terms cannot be established.
Unclear terms take recognisable forms. A portal publishing datasets with no licence statement at all. A page saying the data is free to use without defining what use means. Terms in a language the team cannot read reliably, attached to a source nobody can contact. A licence naming an entity that no longer exists. An aggregator that redistributes data without stating what its own rights to redistribute were, which catches the most people, because the aggregator looks authoritative.
The temptation is to proceed. The data is public, the source is a government body, nobody has ever complained, and the alternative is a hole in coverage. Each of those observations is true and none is a licence. Proceeding creates a liability that grows silently: it propagates into derived tables, becomes load bearing in a product, gets sold to customers who now have exposure too, and surfaces during a customer's diligence rather than an internal review.
The workable policy is to attempt contact and record the attempt, to keep the source on a candidate list rather than in the pipeline while the question is open, and to state in the coverage documentation that a gap exists because a source could not be cleared. A documented gap is a professional position. An undocumented liability is not.
The commercial argument, which is the real one
Everything above reads as risk management, which is a weak motivator for engineering teams under delivery pressure. The stronger argument is commercial.
A clean pipeline is what allows a team to say yes quickly.
Serious customers ask hard questions before they buy. Their legal or procurement function wants to know what is in the data, whether anything obliges them to open their own systems, whether any source prohibits their use, and whether they can redistribute derived material to their own customers. These questions arrive at the end of a sales process, when the technical evaluation has already gone well.
A team with per record licence tracking answers in a day, with a document listing every source, its terms, and its role in the product. A team without it answers in weeks, hedges, or discovers a problem that kills the deal. The discovery is worse than the delay, because the customer now knows something about the vendor's engineering practice that no amount of product quality will offset.
The rule that follows is short. No record enters the pipeline without a licence field, and no transformation may drop it. Everything else here elaborates those two sentences, and a team that enforces them from the first ingestion job never needs the audit described above.