What a sanctions-screening project taught us about matching civil contractors

Ask a platform "how many hours did the big Cat do on the Bruce last month" and the arithmetic is the easy part.

"The big Cat" is a machine that appears as EX-014 in the scheduling system, FA00231 in the ERP fixed-asset register and serial 7Q4-118 in the tool-tracking app. "The Bruce" is a job that is a five-digit cost centre in the ERP, a tender in the estimating system, and a plain name in the scheduler. "Last month" is a pay period in one system and a calendar month in the other. And the plant hire invoice from "J&K Civil Pty Ltd" has to be the same company as the AusTender award to "J & K CIVIL".

That is entity resolution, and there is a whole industry selling it. Before we built our own we owed ourselves an honest look at buying it. This is that look, written after the decision rather than before it, and it ends somewhere we did not expect: a Berlin open-data project that matches names on sanctions lists.

---

Four problems, not one

Vendors describe entity resolution as a single capability. In civil construction it is at least four, and they do not share a solution.

People. A worker appears in payroll with a surname, a date of birth, a personal email and a mobile. The same worker appears in the field app with a full name, a work email and, if someone remembered, a payroll employee id. Matching them is a privacy decision as much as a data one. Under Australian privacy law a wrong merge is a disclosure.

Plant and equipment. A machine has a serial number, a registration plate, and a label a scheduler typed. Serials disagree between systems more often than they agree. Plates get corrected after the fact. Generic labels like "Hammer Drill" match six other hammer drills. No name engine was built for this, because it is not a name problem.

Projects and opportunities. This is the one the vendors do not have a category for, and it is the one a contractor's money runs through. A job is a cost centre. The ERP keys it on a job number, the estimating system on a tender reference, the scheduler on a name, and the document library on a folder path. The same job number gets written three ways across those systems: bare, prefixed with the system name, and with a letter suffix for a sub-job. A tender-stage opportunity and a delivery-stage project for the same work are different things with different facts, and a question about one must not be answered from the other. Then there are the overhead buckets: "Yard", "Training", "Leave" exist in the scheduler and nowhere else, and they are correctly nobody's job.

We learned the sharpest lesson here the hard way. A plain-name reference that fuzzy-matched exactly one sub-job looked clean and was wrong: it silently excluded the main job and its four siblings, whose labels did not contain the literal string. A single fuzzy candidate is more dangerous than several, because several at least tell you to ask.

Companies. Federal awards carry an ABN. Most state disclosure feeds carry a name and nothing else. Trading names, "T/A", trustee structures and a supplier cell that sometimes contains forty companies in one field are normal. Two genuinely different ABN-registered companies can share a normalised name; two companies that both collapse to "vocus" each have their own award history, and a merge on name alone would have added them together.

---

And a fifth thing nobody sells: vocabulary

Australian crews do not use the names in the systems. The excavator is "the big Cat" or whatever the operator called it in his first week. The highway upgrade is "the Bruce". The client is "TMR", never its full name. A question in the site office is asked in that language, and a platform that answers only to FA00231 is not going to be used.

So the spine carries a vocabulary layer. A nickname taught in chat ("the big Cat is the 20-tonner") becomes an alias on the real entity, human-vouched, resolved at the same confidence as a system key. A road name or a suburb resolves to the job. A term lexicon expands field slang before search runs, so "roo" finds the site diary entry that says "kangaroo". Generated aliases carry their own source label so they are separable from ones a person taught, and revocable on their own.

None of the products below have this, because none of them serve a crib hut.

---

We went shopping

Everything that turns up when you search for entity resolution in 2026 is below. Pricing is from the vendors' own pages where it exists; most of this market is contact-sales.

Senzing is the one we took most seriously, and it is genuinely good: an embedded library that resolves each record against whole entities in real time, learns which values are too common to trust, un-merges automatically when new evidence arrives, and answers "why are these two records one entity" with a full construction history. Three properties broke it for us. A worker who moves between two of our customers is the same person, and Senzing would merge them unless we ran one engine per customer, which its licence does not allow on one instance. Its entity ids are explicitly unstable across re-resolution, and our facts anchor on stable ids. And it stores the values it compares in its own tables, which for a payroll register means date of birth, phone and email in a second schema. It also has no Australian presence and no ABN or ACN handling in its documentation.

Splink is what we would reach for if a customer's data ever needed a statistical scorer. It is free, maintained by a government team under the same auditability pressures we have, and its output is a calibrated probability with a per-field waterfall a person can read. It has no name standardisation of its own.

The cloud services want the data in S3 and bill per run. The MDM suites solve a real problem for banks and manufacturers with millions of records and a stewardship team; a civil contractor has ten to fifty thousand entities and nobody whose job is data stewardship.

At this point we had a list of things we could not buy. Then we found the one we could.

---

What OpenSanctions got right

OpenSanctions maintains a consolidated database of sanctioned people and companies, politically exposed persons and criminal watchlists, and screens names against it for banks, journalists and compliance teams. Their problem is ours with the stakes turned up: is "Vocus Pty Ltd" the same company as "VOCUS PTY. LIMITED", and is either the one on the list?

Three things about how they solved it changed our plan.

The matcher is open. Their scoring library, nomenklatura, and its reference data, rigour, are MIT-licensed Python. The scorer is deterministic and weighted: name features carry full weight, weak aliases less, registration and tax identifiers nearly as much as a name, and specific penalties fire when the numbers inside two company names disagree. Every match comes back with its per-feature scores, so a steward can see exactly which feature carried it. The legal-form data covers "Pty Ltd" and its spellings across languages, and the component-equivalence tables know that "Holdings" and "Group" are the noise, not the signal. That is a better organisation-name layer than our regular expression, and it is free.

They tested the alternatives and published the result. In February 2026 they released OpenSanctions Pairs, a dataset of 755,540 expert-labelled entity pairs, and ran their rules against large language models on it. Their reported numbers: the rule-based matcher scored 91.3 percent F1, GPT-4o 99.0 percent, and a distilled 14-billion-parameter open model 98.2 percent. Their reading of it, in their name-matching write-up, was that rules over-match and models struggle with names across scripts.

They kept rules in production anyway. Not because the models were worse, but because a compliance decision has to be explainable and repeatable, and a model's answer is neither in the way an auditor means. Models are useful as judges over a candidate set, not as the engine that decides identity.

That is the conclusion we had reached from the other direction, for a different regulator. A merge that puts one worker's hours on another worker's timesheet, or one company's award on another company's record, has to be explainable to the person it affects. We would rather be right and show why than be slightly more right and unable to.

---

What a civil platform still has to build

OpenSanctions matches people and organisations. It does not match excavators, or cost centres, or "the big Cat". So here is what we built around it, and the rules it runs on.

Keys at write time. When a record is harvested it gets a canonical key from a declared field: ABN for a company, registration or serial for a machine, job number for a project, employee id or email for a worker. Two sources that compute the same key land on the same entity without anything comparing them. Every normaliser rejects the placeholders real systems contain (TBA, N/A, 0), and an ABN that is not exactly eleven digits is refused rather than guessed.

Links, not merges. A cross-source match is a link between two entities, traversed at read time. Neither entity is deleted and no fact moves. A link can be retracted with a reason, and a retraction is a durable negative, so the same rule cannot silently re-propose the same pair next week.

Automatic for trusted identifiers, a person for the outliers. A match on serial, registration, email, employee id, or surname plus date of birth is accepted without anyone clicking. That is where adoption lives: a platform that asks a scheduler to approve hundreds of obvious matches on day one does not get used on day two. A person sees only what a rule cannot settle: a name-only company match, a value shared by several records in one register, a link that would cross a legal entity or an environment. Those land in the same approval queue that governs every other write the platform makes, with the evidence on the row.

Evidence, not values. For people, the comparison happens in memory. What gets stored is which fields agreed, and a hash. Date of birth and phone never land in a second table.

Scoped to one customer, one legal entity. Every entity, alias and link carries the customer and the legal entity it belongs to, and the linker refuses to write across that boundary. A person who works for two of our customers is two workers, on purpose.

Projects get their own treatment. A job number is normalised across its bare, prefixed and suffixed spellings, and the plain-name spelling gets a generated alias so a delivery-stage question lands on the project rather than the tender that preceded it.

---

The measurement that kept the scorer on the shelf

The case for a statistical engine is the fuzzy band: the pairs only a scorer can reach. We measured it before buying anything to close it.

We drafted a Fellegi-Sunter scorer twice, the same model Splink uses, and ran it on a production tenant with four connected registers. Then we split every candidate pair by whether exact agreement on a declared identifier already reached it.

For workers the residual was zero. Every pair the scorer liked was reachable by surname plus date of birth, or surname plus full first name, once someone wrote the field map between the two systems. The "fuzzy" cases were a work email in one system and a personal one in the other.

For equipment the residual was a handful of pairs, all the kind a person recognises instantly ("Laser Level - QLD" against "Topcon Laser Level (QLD)"). The noise the scorer surfaced instead was generic labels matching several machines at once, which a one-per-source rule rejects for free.

That is why the probabilistic layer sits designed and unbuilt behind a gate that re-measures the residual per customer. Buying an engine to close a handful of pairs is machinery serving nothing.

---

Where the incumbents still beat us

Saying this honestly is the point of the post.

Organisation names, until the rigour swap lands. Our normaliser is a regular expression. It once missed the t/as spelling and split forty entities, which misreported one contractor's award total until we found it. It strips a bare "co", which is wrong for "Smith & Co Civil". Senzing's name engine and OpenSanctions' legal-form data are both better than this today.

Un-merge. A provisional identity minted for a name-only supplier keeps that identity for life. When an ABN later arrives for the same name, nothing sweeps the register to reconcile them, and a normalisation fix reproduces the split rather than healing it. Senzing re-resolves on every write.

Generic values. Our guard against a value too common to be an identifier is a frequency cap within one register's batch. Senzing learns frequency across the whole store.

Scale. Our cross-source proposer is an unblocked nested loop. Against a large tool register it is millions of comparisons and had to be moved off the request path. Senzing customers run hundreds of millions of records. We are nowhere near needing that, but it is true.

---

What we are changing

1. Normalise the ABN inside the resolver, not upstream. Today a spaced ABN from one publisher can mint a second entity beside the digits-only ABN from another.
2. Adopt rigour's legal-form data in place of our regex, and stop stripping bare "co".
3. Add a company-name tier shaped like nomenklatura's scorer. Weighted features, per-feature evidence on the row, and a proposal to a person only when the score is in the band a rule cannot settle.
4. Resolve names to ABNs through the Australian Business Register. The bulk extract is free and authoritative, and it turns most of the name-only problem into a key problem.
5. Re-measure the fuzzy residual automatically when a customer connects a new source, so the gate that decides whether Splink gets built is read by the event that changes it.

---

Why this matters to a contractor

You do not buy identity resolution. You buy an answer to "what did that machine cost us on that job" that is right, and a competitor lens that does not count one company as three. But the shape of the engine underneath decides whether your payroll data ends up in a vendor's side table, whether a wrong merge can be undone, and whether the platform understands "the big Cat".

We chose to build the shape that keeps those decisions with you, and to take the best open component we could find for the part that is genuinely hard. The incumbents are better at some things, and we have said which. If you work on OpenSanctions and any of this is wrong, tell us and we will fix it.

---

Sources

• OpenSanctions: name matching, matcher documentation, nomenklatura, rigour, OpenSanctions Pairs paper
• Senzing: pricing, entity specification, how an entity id behaves, end user licence agreement
• Splink: repository
• AWS Entity Resolution: pricing
• Australian Business Register: web service methods
• Gartner Magic Quadrant for MDM 2026 as reported by Reltio and Semarchy