Cohere wants companies to stop taking a cloud provider's word for it. On October 22 the Canadian enterprise AI company opened a public beta of Confidential Computing on Model Vault, its dedicated and fully managed inference service. The feature is aimed at data many firms have so far kept away from hosted models, such as financial records, clinical files, privileged legal material, proprietary code and engineering designs.
The problem it targets is the third state of data. Encryption normally covers information sitting on disk and moving over a network, but prompts, documents and model outputs are exposed in memory while a model is working on them. In Cohere's setup, the serving stack runs inside a confidential virtual machine whose memory is isolated by hardware from the host, including privileged software and operators. NVIDIA Confidential Computing extends that boundary to the GPU, protecting its memory and encrypting traffic between processor and accelerator. The beta runs on NVIDIA H100 chips in Microsoft Azure, which supplies the underlying confidential infrastructure.
The second piece is verification. Before a request goes out, software on the customer's side checks signed evidence describing the remote environment's hardware, firmware and software against an approved policy. If the evidence does not match, the client simply does not send the data. Cohere sums up the change as moving from trust us to verify it yourself. Its chief technology officer, Phil Blunsom, says customers can confirm their data is shielded from unauthorized access, including access by Cohere.
The third is the path in between. Ordinary TLS secures a connection but can leave payloads readable to intermediaries that run a managed service. Model Vault instead uses Oblivious HTTP to encrypt each request to a key held only by a gateway inside the attested environment, so routing layers see ciphertext and responses are protected on the way back. Developers keep using the same Cohere API and request formats, since the extra protection sits in the client and infrastructure.
Executives from Microsoft's Azure confidential computing group and NVIDIA's enterprise software unit endorsed the launch in Cohere's announcement. Cohere also published a technical companion post that walks through the trust boundary and threat model. It describes the serving software as locked down by policy, so that only approved code can run and operators have no interactive way into the workload.
The company is careful about the limits. Confidential computing does not remove all risk, it says, and does not replace application security, compliance, contracts or data residency rules. The beta is restricted to one GPU generation on one cloud, and Cohere did not publish pricing, latency overhead or an outside security audit. Still, the release puts a frontier model vendor in the same race as hyperscalers to make verifiable privacy a selling point for regulated customers who have so far held back from hosted AI.