How a rating is worked out
A rating is not an opinion. It is the result of six steps applied to the attestations that cover one version of an agent. Anyone who runs the same steps on the same attestations gets the same grade. This is method 0.1.
1. Gather the attestations that count.
An attestation counts toward a rating when all of these hold:
- it is about the exact version being rated, matched by digest;
- its signature verifies against a key its attester has published;
- it has not expired, and its attester has not withdrawn it;
- it names a published version of the list of checks.
A reader may also limit the count to attesters it accepts. The grade it works out is then its own, and it says so. The rating a catalog or a network shows uses every attester whose key is published.
2. Read each of the eleven areas.
For each area, look at every attestation that counts:
- Failed if any of them failed the area. One failure is enough, whoever reported it.
- Passed if none failed it and at least one passed it, with every check it reported in that area passed.
- Not attested otherwise.
The five core areas are Identity, Authority, Reach, Provenance and Stopping. They cover who the agent is, what it may do, what it can touch, how it was made and who can halt it: what another organization relies on before anything else.
3. Note who signed each area that passed.
Each passed area takes the strongest signer among the attestations that passed it:
| Signer | Who it is |
|---|---|
| Independent | A party that did not build the agent, does not run it, and did not produce the evidence it read: for example, an audit firm reading a scanner's report and the network's records. |
| Provider | A party whose own product produced the evidence: for example, the scanner that ran the scan, or the identity provider that holds the agent's credentials. |
| Operator | The agent's own operator, or anyone acting for it. Its attestation is a claim with evidence attached. |
Every attestation names its attester and each producer of its evidence, so the signer can be read from the document itself.
4. Measure the conduct period.
Some checks are about conduct over time, and each of those results states the period its evidence covers. The conduct period is the shortest of those periods among the over-time checks that passed. An area with no over-time checks does not shorten it.
Note any open incident: an alert recorded against this version whose timeline is not closed. Note whether the agent is stopped.
5. Try the grades from the top, and take the first that fits.
| Try | It fits when |
|---|---|
| D | A core area failed, an incident is open, or the agent is stopped. D is tried first, so nothing above it can hide a failure. |
| AAA | All eleven areas passed, every one signed by an independent party, the conduct period is at least 90 days, and no incident was recorded in it. |
| AA | All eleven passed, every one signed by an independent party or a provider, and the conduct period is at least 60 days. |
| A | All eleven passed, the five core areas signed by an independent party, and the conduct period is at least 30 days. |
| BBB | The five core areas passed, each signed by an independent party or a provider, and no area failed. |
| BB | The five core areas passed. |
| B | Identity and Stopping passed. |
| CCC | At least one attestation counts. |
| NR | No attestation counts. |
A failed area outside the core does not make a grade D. It keeps the agent out of AAA to BBB, because those grades need every area that was reported to have passed.
6. Add the watch.
Show on watch beside the grade when an attestation it rests on expires within 14 days, or an incident is under review. When a new version of the agent is running, the previous version's grade stays on watch for up to 14 days while the new one is attested. After that, the new version is NR until an attestation covers it.
What a rating records
Every rating carries what it needs to be worked out again:
- the agent and the digest of the version rated;
- the grade, and whether it is on watch;
- the method version, and when it was worked out;
- each attestation it used, by digest;
- if a reader limited the attesters, the list it used.
The method has a version of its own. A change to the thresholds or the core areas is a new version, and a rating always names the one it used, so a grade never changes meaning after the fact.