Agent risk scoring

Every agent, scored like an identity

An agent with tenant-wide Graph access and a stale credential is a standing risk. Sentinel scores each custom agent from six factors so the dangerous agents can't hide behind the useful ones.

  • Reach & usage - how many users and channels an agent touches, and how heavily it is used.
  • Data access & tooling - Graph permissions, connectors, and external tools - escalation-level grants weigh most.
  • Safety threat exposure - jailbreak and injection attempts against the agent, and unsafe output from it.
  • Failures & hygiene - error rates, failing tools, slow responses, expired credentials, and missing registration.
app.princetonsentinel.com/dashboard/agent-risk
Agent Risk page with risk-level counts, agent risk gauge, factor distribution, and AI risk summary
Agent detail

Watch the trend, not just the number

Open any agent to see which factors are maxed out, how its score has moved day by day, and what to change first.

  • Factor breakdown - each of the six factors against its cap, with the inputs behind it.
  • Daily score history - see whether a fix actually brought the score down, with the level bands marked.
  • Confidence shown - missing telemetry or permission data lowers confidence instead of hiding risk.
app.princetonsentinel.com/dashboard/agent-risk/sales-proposal-assistant
Agent risk detail for one agent with exposure gauge, six-factor breakdown, and daily score history
Telemetry

See what your agents are actually doing

Governance without visibility is a policy document. Sentinel pulls Copilot Studio telemetry from Application Insights so you can see usage, outcomes, and failures - and spot the agent that's suddenly everywhere.

  • Conversations & users - daily conversations, unique users, new versus returning users, and average duration.
  • Outcomes - resolved, escalated, and abandoned conversations, per agent.
  • Topics, tools & latency - what agents are asked, which tools they call, and how fast they answer.
  • Transcripts - drill from a spike to the individual conversations behind it.
app.princetonsentinel.com/dashboard/agents
Custom Agents telemetry with conversation KPIs, daily conversation trend, and conversation outcomes
AI Safety

Every prompt, classified

Copilot Studio blocks some attacks; others get through. Sentinel collects the platform's safety events and runs its own classifier over every user prompt, so you see the attempts the platform stopped and the ones it didn't.

  • Safety events - content-filter, jailbreak, injection, DLP, and connector blocks, by agent, channel, and month.
  • Conversation scan - an hourly verdict on every prompt - including ones the platform let through.
  • Block-worthy & review queues - prompts rated high-risk and prompts flagged for a human, counted up front.
  • Production vs test - test-canvas traffic is separated so makers' experiments don't drown real incidents.
app.princetonsentinel.com/dashboard/agents/safety
AI Safety dashboard with safety event KPIs, events by month and by agent, RAI type distribution, and prompt classification counts
Prompt verdicts

Evidence, not anecdotes

Each flagged conversation opens to the verdict and the exchange behind it, so a reviewer can decide in seconds whether it was an attack, a curious user, or a false alarm.

  • Verdict - attack label, threat category, severity, confidence, and recommended action.
  • Admin review flag - verdicts that need a human are routed to a review queue.
  • The exchange itself - the prompt and the platform's safety event, side by side - visible to admins only.
app.princetonsentinel.com/dashboard/agents/safety/session
AI Safety conversation detail with a Jailbreak verdict, critical severity, block recommendation, and the safety event log
Access control

Contain a risky agent in minutes

When an agent crosses a line, you shouldn't have to file a ticket and wait. Sentinel gives admins direct, audited controls - and keeps the record of who did what.

  • Block & unblock users - cut a specific user off from a specific agent through Dataverse, and reverse it.
  • Quarantine - take an agent out of service through the Power Platform pending review, then release it.
  • Disable the agent's identity - switch off the agent's Entra service principal to stop its data access outright.
  • Logged & exportable - every action lands in the Agent Access or Agent Quarantine log, with CSV export.
app.princetonsentinel.com/admin/agent-quarantine
Agent Quarantine log listing quarantine and release actions with admin, agent, reason, and status
FAQ

AI agent security questions

Which agents does Sentinel cover?

Custom agents built in Copilot Studio across your tenant - including registered agents that have gone quiet and agents seen in telemetry that nobody registered. Microsoft 365 Copilot itself is covered separately, on the Copilot Utilization pages, as adoption and usage analytics rather than a risk score.

What drives an agent's risk score?

Six factors that add up to 100: Reach (users and channels, 27), Data access and tooling (Graph permissions, connectors, and tools, 24), Safety threat exposure (attacks aimed at the agent and unsafe output from it, 15), Operational failures (errors, failed tools, and slow responses, 14), Usage intensity (10), and Control & hygiene (registration, credentials, and blocks, 10). Permissions that let an agent grant itself more rights weigh far more than broad read access. The score is deterministic and the full breakdown is shown.

What can Sentinel actually do about a risky agent?

Admins can block or unblock a specific user from a specific agent, quarantine or release an agent through the Power Platform, and disable or re-enable the agent's own Entra identity. Every action is admin-initiated and written to the Agent Access and Agent Quarantine logs - Sentinel never acts on an agent on its own.

How does AI Safety work?

Sentinel collects the responsible-AI events Copilot Studio records - content-filter hits, jailbreak and injection blocks, DLP and connector blocks - and runs its own classifier over every user prompt each hour, including prompts the platform let through. Each prompt gets a verdict: attack type, severity, a recommended action, and whether an admin should review it. It is an after-the-fact review, not an inline filter, so conversations are never slowed down.

Who can see prompt text, and how long is it kept?

Prompt text is visible to admins only. Copilot Studio message content is removed after 90 days by default while the verdicts and counts are kept. For Microsoft 365 Copilot, Sentinel stores prompt counts only - never prompt or response text.

Bring your riskiest agent to the demo

We'll score it live, walk its telemetry and safety verdicts, and show the exact quarantine path - all inside a tenant that looks like yours.

Request a demo