The reader becomes a runtime¶

The applications office has the cork board. The card that said WHERE IS THE ASSET REGISTER came down in the summer, and a link to the register went up in its place, which the IT manager regards as the single largest achievement of his tenure and the business analyst regards as a first draft. The consultant spent the second day there, with the two of them and the register, working out what the reader could reach.

Access control at the Home was designed, as it is everywhere, for a human opening one file at a time. Each grant was priced on that assumption: this person, this folder, tolerable. Nobody priced the aggregate, because no reader could realise the aggregate. The new one can. The mosaic problem arrives immediately: every document individually authorised, and a synthesis across four thousand of them crossing a line that no single grant crossed. Salary bands assembled from expense claims. The Consortium negotiation inferred from travel bookings and the names of files. A reorganisation visible in the drift of chat-room membership. And the one the Home does not discuss: thirty chapters’ worth of field notes, each an ordinary welfare record, which read together are the most natural human-terrain source on the Disc, and have been read that way for years by a desk that never needed a login. Compartmentalisation by chaos was never a policy, but it functioned as one, and it is gone the day the index finishes building.

Deployment tends to discover this before any attacker does, at a price of its own. The permissions mess was tolerable because humans tolerate mess. The move off the tenant had carried the mess across in boxes, and boxes do not audit themselves: thousands of inherited grants nobody can explain, former employees still attached to groups, project folders shared permanently just in case, the photograph from the Covenant onboarding with a webhook secret readable in it, the salary spreadsheet in a folder nobody has reviewed since it was created, night-shift channels whose membership Nikolaj set up at half past eleven one evening and never revisited. None of it was a crisis while no reader could realise the aggregate. The reader makes the aggregate real, so the mess becomes a blocking dependency, and the AI project turns out to contain an identity governance programme that appeared on no slide of the original deck. Mid-rollout, before the estate can be safely read by one principal, its permissions have to be understood for the first time, and understanding a legacy permission structure is archaeology, priced by the layer. The card on the cork board that says “enchanted ledger, do not touch, Gerald knows why” is one of the layers.[1]

The Home’s security also rests on a set of separations old enough to be invisible: data is data, code is code, instructions are instructions, and the controls are organised accordingly. Bestiary’s server gets patched and kept in a box with a label on it; configuration gets change control, now, after the move; documents get, at most, malware scanning and a sensitivity label, because a document is inert. The reader retires the assumption. A PDF is data until the reader reads it, at which point it may be instruction. A page in the Burrow is documentation until it becomes a control input to a rota. A field worker’s note is a record until it is retrieved into a transfer dossier. The security function inherits a category its whole control estate was not built for: content that behaves like code without looking like code, arriving through channels no software supply chain policy was written to treat as a supply chain, authored by anyone with write access to a shared folder, which across thirty chapters means everyone the Home has ever trusted and some it has not.

The cost is the re-derivation of the access model. Least privilege loses its unit of account. The old question, can this person open this document, has a checkable answer, and every existing access control answers it correctly. The operative question becomes whether this person may ask a question whose answer requires combining eight hundred documents, and that question has no place in the model as it stands: each of the eight hundred grants is individually valid, and the exposure is the combination, constructed at query time, differently for every query. An access matrix can be technically correct in every row and still inadequate, because it was drawn for readers who could not aggregate. Per-query reasoning about the sensitivity of a synthesis would answer the operative question, and that is an unsolved problem currently being carried in production, at the Home and elsewhere.

Aggregation reshapes the attacker’s economics as well as the defender’s. A compromised credential used to buy months of folder-crawling and guesswork; it now buys a natural-language exfiltration interface over the whole estate, with the reconnaissance phase reduced to a well-phrased question, such as which of the Home’s people in Klatch talk to whom. And the estate itself becomes an instruction channel. A reader that reads documents and acts on what it reads treats every file as potential input, which means a poisoned document planted in a chapter’s shared drive, in a capital where the arrangement’s writ has never run, is a command waiting to be retrieved in Ankh-Morpork. Mixing instructions and data in one channel is a problem software engineering learned to recognise, classify and constrain, reintroduced here in an estate that was never built around that assumption.

There is a shorter route still, and it is the cheapest on the menu. Much of what the reader can do hangs from what it treats as instruction: the system prompt, the configuration, the standing orders that tell it what it is and what it may touch. At the Home, during the pilot, those lived in a document in the IT folder, two entries down from the onboarding photographs. That text is a single point where words become estate-wide action. An attacker who reaches it needs no malware, no exploit chain, no lateral movement through a dozen systems; the reach that once required compromising many accounts and hosts now sits behind one writable surface, and the currency of the attack is prose. This is the flip side of the amplification being sold. Capability was concentrated, so that one reader could do the work of many, and concentration does not check who is speaking.

Persistence changes character along with the entry: a poisoned instruction crashes nothing, trips no signature and leaves no binary to find, and the audit target becomes natural language, where the usual parser, checksum and invariant are no longer enough to establish what the instruction did. Detection tooling grew up around code behaving strangely; the Home’s shared SOC is good at that. Instructions arriving by prompt are a younger problem, and the tooling for them is mostly not on the shelf.

The reader also breaks the review calendar. Security review at the Home has moments: before deployment, after major change, at audit, and the incident response plan has been tested at two in the morning by people who had rehearsed it in daylight. The moments exist because the reviewed object holds still between them. Nothing in this deployment holds still. Documents change daily, which changes what the reader can say; permissions change, which changes what it can reach; connectors are added, thirty chapters of them in phase two, which changes where its words can land; models update, which changes how it reads; prompts get edited, which changes what it is; and users keep discovering uses nobody threat-modelled, which changes what the system is for. A review conducted at the project gate describes a system that no longer exists by the first quarterly meeting. Security review of a reader over a living estate is an operating activity, staffed and recurring, and the board paper had budgeted it as a one-off assessment, with the consultant’s fortnight as the assessment.

The audit trail thins. Human access left per-file traces: who opened what, when. A synthesised answer leaves a different kind of entry, that the reader read widely and said something, and the full price of that entry surfaces later, when reconstruction is attempted. The Home has always been able to say, truthfully, that nothing was tasked and nothing was signed. An entry that says the reader read widely is more than the nothing the Home’s denial was built on.