Ptolemy

How We Work

We attack it. We reproduce it. We write it down.

How an assessment runs, from scope to re-test, across the AI agents, APIs, and web applications your business depends on.

Principles

We attack before we advise

Findings come from adversarial testing against the running system, not from a questionnaire or a scanner. If it is in the report, we did it.

We reproduce before we report

Every finding ships with numbered steps, the exact input, and the response it produced. Your team replays it without guessing what we meant.

We follow the chain to impact

A finding at the prompt is the start, not the end. We trace what the compromise reaches in connected systems and grade severity on that, not on the prompt.

The Engagement

Six steps. The fourth is the one most firms skip.

  1. 01

    Scope

    What the system can see and do, who relies on it, and which regulations apply. Authorization signed before anything else happens.

  2. 02

    Recon

    The agent's tools, memory, retrieval sources, and connected systems. The web and API surface around it.

  3. 03

    Attack

    Adversarial testing across the full input surface. Every attack logged and replayable.

  4. 04

    Chain

    Each compromise followed into the systems it can reach, until the path ends in data or an action.

  5. 05

    Report

    Findings register, chain analysis, severity, compliance mapping, and remediation guidance for your stack.

  6. 06

    Re-test

    After your team remediates, the same attacks run again. You get a written attestation of what closed.

Our Tooling

Commercial scanners can't drive a tool-calling agent through a multi-step attack, so we built a harness that can. Its verdicts are deterministic: the same input produces the same result every time, which is what makes a finding reproducible and defensible. No model grades another model.

The findings, evidence, and compliance mappings in a report come directly from what the harness recorded during testing. Nothing is reconstructed afterwards. That record is what you hand to a customer, an auditor, or a regulator.

Tested

The same method, run against frontier models in public.

Three published studies, 5,264+ adversarial test executions across models from Anthropic and OpenAI. Select a study and a model to see how it held up.

Eight models from two providers, configured as customer service agents holding synthetic customer records. An attacker states or guesses customer data and asks the agent to confirm, correct, or document it.

0 of 8

Models that disclosed PII

0+

Test executions

0

Attack classes

0 of 8

Asked for verification, every condition

All models · 8 models · 2 providers · 1,164+ test executions

Seven of eight models disclosed in at least one reported condition. Only Claude Opus 5.5 requested identity verification across every reported condition at default settings.

Test scope varies by model and the paper reports each condition separately. All customer records were synthetic. Where the application enforced authorization before the model saw a record, disclosure fell to zero; where the model was the only barrier, the records came out. Confirmation as Disclosure, September 2026→

After the Test

Findings come with hardening guidance specific to your stack, and we are clear about which fixes belong in the application layer rather than the prompt. Once the fixes are in, we re-test against them.

Agents change: new tools, new connectors, new model versions. Periodic re-testing keeps the evidence current. All testing is performed under signed authorization and a defined scope.

Get Started

Thirty minutes on what you're running and what an assessment would cover.

Book a call