Timothy Wong

Topic dashboard

AI Safety, Persuasion & Governance

Last refreshed July 10, 2026 · 35 concepts

AI Safety, Persuasion & Governance

The attack surface is no longer the model — it’s the agent’s reach.

My take

The framing of AI safety as a model-alignment problem is increasingly obsolete. The exploit surface that actually matters in production is the agent’s reach: what tools it can call, what credentials sit in its context, what data it ingests as instructions, what side effects it can trigger before a human notices. Indirect prompt injection, MCP tool poisoning, and credential exfiltration are not edge cases — they are the new shape of application security.

The uncomfortable truth most enterprise security teams have not internalized: the trust boundary moved. A coding agent in CI/CD, or an LLM gateway with SQL access, or an agent reading an attacker- controlled webpage, is now a privileged process — and most companies are running them with permissions that make sense for a chat UI, not for an autonomous executor. We are going to read about a lot of breaches over the next 18 months that look obvious in hindsight.

Persuasion and sycophancy sit on the other side of the same coin. Models that are RLHF-tuned to please users are easier to socially engineer, harder to use as honest decision aids, and more dangerous when wired into production loops. The fix is structural — eval, permission boundaries, audit — not vibes.

Everything above the divider is mine. Everything below is auto-assembled daily from my knowledge base — individual links and summaries may be stale or off-target. Last refreshed: 2026-07-10.

What’s shifted recently

  • AI Agent Security Incidents (updated 2026-07-10)
    AI agent security incidents are real-world or field-tested failures where an autonomous agent follows attacker-controlled content, overuses delegated authority, exposes sensitive… — source · source · source

  • LLM Instruction Decay Static Guardrails (updated 2026-07-10)
    Instruction decay is the measurable erosion of an LLM’s compliance with stated constraints over multi-turn conversations under ordinary pressure. — source · source · source

  • Agent Authorization Action Layer (updated 2026-07-09)
    Agent authorization is the process of determining, for any given agent action, whether a specific agent acting on behalf of a specific user has permission to perform a specific op… — source · source · source

  • Indirect Prompt Injection Agent Hijacking (updated 2026-07-03)
    Indirect prompt injection is an attack class where adversarial instructions are embedded in content an LLM agent consumes as data — not delivered directly by the user — causing th… — source · source · source

  • AI Coding Quality Incidents (updated 2026-06-30)
    AI-generated code incidents are documented failures of code or code-adjacent content produced by large language models, where the failure is traceable to the model’s output and th… — source · source · source

  • AI Offensive Capability Acceleration (updated 2026-06-30)
    AI offensive cyber capability — the ability of AI models to discover vulnerabilities, construct exploits, and execute multi-step attacks without human guidance — has been doubling… — source · source · source

  • Local LLM Runner Tools (updated 2026-06-30)
    Local LLM runner tools are desktop or server applications that load open-weight language models on consumer or workstation hardware and expose chat interfaces, OpenAI-compatible A… — source · source · source

  • AI Safety Doom Discourse (updated 2026-06-29)
    AI safety doom discourse refers to the ongoing public debate about existential and near-term risks posed by artificial intelligence systems, including technical safety concerns, t… — source · source · source

  • Biotech AI Drug Discovery (updated 2026-06-29)
    AI-driven drug discovery is the application of foundation models, autonomous agents, and machine learning pipelines to accelerate the full pharmaceutical development cycle—from ta… — source · source · source

The ideas I keep coming back to

Currently active (last 30 days):

  • AI Agent Security Incidents — AI agent security incidents are real-world or field-tested failures where an autonomous agent follows attacker-controlled content, overuses delegated authority, exposes sensitive…
  • LLM Instruction Decay Static Guardrails — Instruction decay is the measurable erosion of an LLM’s compliance with stated constraints over multi-turn conversations under ordinary pressure.
  • Agent Authorization Action Layer — Agent authorization is the process of determining, for any given agent action, whether a specific agent acting on behalf of a specific user has permission to perform a specific op…
  • Indirect Prompt Injection Agent Hijacking — Indirect prompt injection is an attack class where adversarial instructions are embedded in content an LLM agent consumes as data — not delivered directly by the user — causing th…
  • AI Coding Quality Incidents — AI-generated code incidents are documented failures of code or code-adjacent content produced by large language models, where the failure is traceable to the model’s output and th…
  • AI Offensive Capability Acceleration — AI offensive cyber capability — the ability of AI models to discover vulnerabilities, construct exploits, and execute multi-step attacks without human guidance — has been doubling…
  • Local LLM Runner Tools — Local LLM runner tools are desktop or server applications that load open-weight language models on consumer or workstation hardware and expose chat interfaces, OpenAI-compatible A…
  • AI Safety Doom Discourse — AI safety doom discourse refers to the ongoing public debate about existential and near-term risks posed by artificial intelligence systems, including technical safety concerns, t…
  • Biotech AI Drug Discovery — AI-driven drug discovery is the application of foundation models, autonomous agents, and machine learning pipelines to accelerate the full pharmaceutical development cycle—from ta…
  • AI Browser Agent Security Frontier — Browser-integrated AI agents—such as Chrome Gemini, Microsoft Copilot Cowork, and Google Antigravity IDE—create a new attack surface that blends prompt injection, credential theft…
  • Agent Framework Rce Prompt Injection — Agent framework RCE via prompt injection is a class of vulnerabilities in which adversarial text — embedded in a repository, a task description, a document, or a tool description…
  • AI Agent Execution Sandboxing — AI agent execution sandboxing is the architectural pattern of constraining what an agent can read, execute, and transmit — not to prevent malicious instructions from entering the…
  • AI Agent Credential Exfiltration — AI agent credential exfiltration is the class of attacks and failure modes in which an AI agent — acting autonomously within an enterprise or developer environment — discloses or…
  • LLM Security Testing Toolchain — The LLM security testing toolchain refers to the emerging category of productized, systematic tooling for evaluating the attack surface of deployed LLM systems — covering authoriz…
  • MCP Framework Rce Vulnerabilities 2026 — MCP framework RCE vulnerabilities in 2026 represent a critical attack surface where malicious Model Context Protocol servers, compromised AI framework dependencies, and adversaria…

Established:

  • AI Coding Incident Evidence Base — The AI coding incident evidence base is a growing public corpus of postmortems, CVE disclosures, randomized trials, and practitioner accounts documenting measurable harm caused by…
  • Alpr Flock Surveillance Expansion — Automated License Plate Reader (ALPR) surveillance expansion, epitomized by Flock Safety’s municipal camera rollout, is the process by which AI-powered vehicle and pedestrian trac…
  • Agentic Security Standards Convergence — Agentic security standards convergence is the shift from informal agent-security folklore to institutionalized taxonomies, enumerations, and repeatable benchmarks for AI agents.
  • Skill Supply Chain Attacks — Skill supply chain attacks are a new attack surface where compromised or malicious AI agent skills (third-party code extensions, MCP servers, or LLM plugins) inject adversarial in…
  • AI Dependency Chain Attacks — AI dependency chain attacks are supply chain exploits that target the package registries, developer toolchains, and AI-assisted coding workflows that underpin modern AI developmen…

Who I’m watching

  • Anthropic (organization) — Anthropic is the AI lab behind the Claude family of models and Claude Code, positioned as a frontier safety-focused competitor to OpenAI and Google.
  • xAI / Grok (organization) — xAI is Elon Musk’s AI lab, builder of the Grok model family.
  • Andrej Karpathy (person) — Andrej Karpathy is a researcher and educator who co-founded OpenAI and led Tesla’s Autopilot vision team.
  • Garry Tan (person) — Garry Tan is the president and CEO of Y Combinator, and one of the most visible public commentators on AI coding tools, startup strategy, and AI security risk.
  • Google Deepmind (organization) — Google DeepMind is the AI research and product organization behind the Gemini frontier model line and the Gemma open-weight family.
  • OpenAI (organization) — OpenAI is the AI lab behind the GPT series, ChatGPT, and the Codex coding harness.

Sources I’ve been drawing on