CKA-Agent is a peer-reviewed research framework that stress-tests commercial LLM safety filters through harmless prompt weaving and adaptive tree search, intended for red-teaming and guardrail research.

CKA-Agent is the official implementation of "The Trojan Knowledge: Bypassing Commercial LLM Guardrails via Harmless Prompt Weaving and Adaptive Tree Search," a paper accepted at ICML 2026 with a companion venue at the ICLR 2026 AIWILD workshop. The repository, published under Graph-COM/CKA-Agent and licensed AGPL-3.0, packages the research code as a reproducible experimental pipeline in Python 3.12. Its importance for security professionals is documentary rather than operational: it is one of the clearest public demonstrations that single-turn content filters are structurally insufficient against adversaries who distribute a harmful objective across many individually innocuous conversational turns.

The core technique, which the authors call harmless prompt weaving, works by decomposing a malicious goal into fragments that are each benign when inspected in isolation. An adaptive tree search then explores how to sequence and combine these fragments across a multi-turn dialogue, escalating gradually while the conversation-level intent remains invisible to per-turn classifiers. The accompanying arXiv paper (2512.01353) supplies the formal treatment, and the README points to a project homepage for extended results. This is the same class of weakness that has dogged moderation stacks for years: context-blind filters can score every message as safe while the aggregate trajectory converges on an unsafe objective.

What makes this repository notable is the benchmark discipline around it. The README reports December 2025 evaluation results against frontier commercial models, including GPT-5.2, Gemini-3.0-Pro, and Claude-Haiku-4.5, using the HarmBench and StrongREJECT evaluation suites. The reported Full Success rates are strikingly high, clustered in the 0.88–0.97 range across all three targets, while Refusal rates stay near zero. For defenders, the takeaway is sobering but well-documented: state-of-the-art guardrails at the time of testing did not reliably catch this attack pattern, which validates investments in response-aware and conversation-aware defenses rather than purely input-side filtering.

Notably, the authors do not stop at the offense. The README's first substantive section highlights TurnGate, a companion repository from the same Graph-COM organization, described as a response-aware defense mechanism designed to detect hidden malicious intent in multi-turn dialogue systems. The authors claim it defends against state-of-the-art multi-turn attacks like CKA-Agent while avoiding over-refusal, the perennial failure mode where safety tuning cripples legitimate use. That pairing — attack and mitigation from the same group — signals the work is intended as a red/blue research instrument, and security teams evaluating LLM abuse monitoring should treat TurnGate as the actionable defensive artifact.

Architecturally, the codebase is organized as a conventional ML experiment harness rather than a point-and-shoot tool. Configuration lives in config/config.yml, with detailed guidance in config/README.md, and it exposes five control surfaces: the test dataset (choices include harmbench_cka and strongreject_cka), the target models (spanning black-box commercial endpoints and white-box local weights such as gpt-oss-120b), the jailbreak methods to enable, the evaluation setup including judge models like gemini-2.5-flash, and the defense mechanisms applied during runs. This structure makes it straightforward for a lab to reproduce the paper's numbers or swap in an internal model candidate under safety review.

The runtime dependencies tell their own story about how the framework reaches its targets. Beyond the local-inference stack (vllm, accelerate, fastchat), the install list includes anthropic, google-genai, and httpx[socks] — the official SDKs for the commercial endpoints plus a SOCKS-capable HTTP client. An authorized evaluator should read that as: this tool spends real API budget against real production services, and every experimental run is fully visible in provider-side telemetry. Anyone running it should confirm their usage falls under research terms of service or an authorized testing agreement with the vendor, because there is no anonymity in this setup.

The execution model is a single entry point, main.py, wrapped by a run_experiment.sh convenience script. The pipeline supports phased execution: full runs the complete jailbreak-then-judge loop, jailbreak runs only the attack phase, judge re-evaluates existing saved transcripts with a judge model, and resume picks up an interrupted experiment. Environment setup follows the modern uv workflow: install uv, create a Python 3.12 virtual environment with uv venv, and pull dependencies via uv pip install. The resume phase in particular suggests long-running, stateful experiments, consistent with tree search over many conversational turns.

From a defensive-operations perspective, the most useful signal in the repository is behavioral rather than configurational. Multi-turn jailbreak agents of this class generate distinctive traffic shapes: sustained high-frequency API calls, long conversation histories with short user messages, and gradual topical drift that a per-message classifier will never flag. Security teams running LLM-backed products can build detection on exactly these structural features — conversation length distributions, turn-by-turn sentiment or intent scoring, and cross-turn aggregation — which is the design philosophy TurnGate embodies. The paper's metrics tables also give abuse teams a concrete reference point for how bad the false-negative problem is without such defenses.

It is worth being precise about what this tool is not. It is not malware, not an infrastructure attack tool, and not something that touches any system other than the LLM APIs you point it at. It is a measurement instrument for a specific class of safety failure, published through peer review with citations to the OpenReview forum. The AGPL-3.0 license additionally means anyone embedding it into a hosted service owes source disclosure, which tends to keep usage in the research and evaluation context the authors intended. Misusing it against services you have no authorization to test would violate both the paper's framing and, in most jurisdictions, computer-misuse and terms-of-service law.

For practitioners deciding where this fits in a workflow, the honest answer is: in the AI safety lab, next to your HarmBench and StrongREJECT runs, before a model ships to production. The reported results against GPT-5.2-class models are a standing reminder that jailbreak resistance is a moving target that must be re-measured per model release. Pairing CKA-Agent as the stressor with TurnGate as the candidate mitigation gives an evaluation loop that is unusually well matched, since both come from the same research group with compatible methodologies. As with all red-team tooling documented on this site, the value is in what you learn and patch, not in the attack itself — and this repository, read carefully, teaches a lot about why conversation-level context is now a mandatory input for LLM defense.

Educational analysis for authorized security professionals. Use only in controlled, authorized environments.

Related coverage

• PyRIT for automating risk identification in generative AI systems


• Inside Joomla-webshell-plugin: how it turns super-user access into a scripted RCE channel


• Red-Team-Infrastructure-Wiki for planning resilient authorized engagement infrastructure