Drafted with AI assistance, then checked and revised by the author. The judgements and the errors are the author’s. How this site is written sets out what is declared on every piece, who checks it, and where the per-piece record lives.
It might be. The defence is not our good intentions, it is that the instrument is constructed to be argued with.
Every test states its pass condition and its evidence requirement before the finding. You can apply it yourself and reach a different result, and if you do, the disagreement will be about a specific piece of evidence rather than about whether we were fair. Each finding carries a marker saying whether it is evidenced or rests on something we searched for and did not find, so the weak points are labelled rather than buried.
The stronger objection is not that the tests are rigged but that they were chosen — that a different five or six questions would produce a different score. That is true and it is worth saying. We think these are the questions that matter because each one, failed, removes an organisation’s ability to decide something about itself. If you think a test is missing or one of ours does not belong, that is a useful argument and we would rather have it in public.
Because the alternative was to publish the finding without saying where it came from, or to hold the whole instrument while one strand is chased.
We did the third-best thing and marked it: the reporting is described as reporting, the primary documents are named, and the fact that we have not obtained them is stated in the finding itself and again in the limits section. A reader can discount that strand entirely and Test 0 still fails on the procurement search, which is first-hand.
It is a real weakness. It is item 7 on our own outstanding list. If the primaries change the picture, the instrument changes.
Both, and the distinction is the point of the marker system.
A test can fail because the evidence shows failure, or because the question cannot be answered. Those are different conditions and we mark them differently. But an organisation that cannot answer a question about itself is not in a neutral position — it is in the position the question was designed to detect. That is why Test 0 is numbered zero: without it the others are not failed, they are unanswerable, which is worse.
What we will not do is present an absence as a conclusion. The form is always: this is what we searched, this is what we did not find. If you have access we lack — inside an agency, inside a vendor relationship — you can close one of these and we will publish the correction.
It advantages nobody and it disadvantages everybody, including any organisation we might ever be associated with.
The reasoning is that these are not independent virtues to be traded off. An organisation satisfying five tests and failing the sixth is captive by way of the sixth, and the five do not compensate. If you can be refused compute, it does not matter that your data is well governed. If you cannot leave, everything else is held at someone else’s discretion.
An average would let a provider offset a hard failure with easy passes, which is exactly how assurance scoring usually goes wrong.
Said plainly: yes.
The author of this series is developing infrastructure that would qualify under these tests. That is a conflict and it should affect how you read the instrument. Three things reduce it and none removes it.
The instrument names no supplier as preferred and confers no status on anyone. Every test is written as a property with an evidence requirement, so more than one design can satisfy it and none is described in a way only one product could meet. And it is published under CC BY 4.0 for anyone to adapt or contest — a corrected version by someone else is a better outcome for us than a defended version.
Our view is that the instrument should eventually be held by an independent standards body rather than by its author. Until that is resolved, we make no claim to authority over it.
No, and the framing is the wrong axis.
The test is about the structure of an arrangement, not the nationality of a party. A foreign-owned operator that restructured to meet it would meet it. A New Zealand company with a foreign parent would not.
And the exposure this series is most concerned about in the medium term runs the other way: the open-weights ecosystem most often proposed as the escape route is substantially maintained outside the countries usually named in this debate. That is a dependency too, and Test 4 catches it.
At the hardware layer, correct — no provider anywhere passes, which is why the tests are scoped to the operator and the data path and say so before the findings rather than in an appendix.
Drawing the boundary there is a judgement, and a competent reviewer should notice that we drew it exactly where we would otherwise fail. We would rather say that ourselves than have it found. A hardware-inclusive test would exclude every operator on earth and would therefore measure nothing.
Often, which is why the instrument avoids arguing about the word and asks six answerable questions instead.
Nothing here requires you to accept a definition. If you dislike the term, apply the questions and ignore the label. The findings do not depend on it.
Specifically, any of these:
Five specific facts. Any one of them, evidenced, and we revise.
That is what it is for. Score each test against the evidence requirement rather than against what a supplier tells you, and take the lowest score as the finding.
Two cautions. Applying it does not imply that we endorse your result, and nobody may represent a self-assessment as approved or certified by us — there is no certifying body and who should hold that role is unresolved. And Test 0 first: until you know which of your systems perform inference and where, the other five cannot be answered at all.
What this publication does not claim, and what is outstanding against it in the register.
A question this rests on is parked: What do we do with an application that refuses to run inside the boundary?
We do not claim that a member's existing applications can be made to run inside the boundary. We claim only that the boundary reveals which ones cannot.
Alongside: the publication · glossary · sources and provenance