IMDA agentic testing starter kit

IMDA's Starter Kit for Testing Agentic Systems for Safety and Reliability is version 1.0, dated October 2026. It is aimed at organisations that deploy AI agents able to use tools, reach sensitive data and carry out tasks for a user. The kit sets out four steps, summarised here:

1. Identify the relevant risks for testing.
2. Design realistic tests.
3. Run the tests and analyse the logs.
4. Monitor continuously and re-test.

The kit says that all AI systems can drift over time, and that agentic systems need particular attention to that. It also says that trajectory analysis at scale remains challenging.

In practice

The first step is a risk list written during design, not after build. The second step is a test set built from that list. The third step records each run in enough detail to reconstruct what the agent did, step by step, including the tools called. The fourth step is a schedule for re-running the tests after every change, and a monitoring plan after launch.

Example

[Composite] A bank's agent for card disputes passed its output tests but failed one trajectory test, where it had opened a case before reading the customer's record. The fix went into the test set, and the monitoring plan checked the sequence of steps after launch.

Common mistakes

• Testing only the final answer and not the steps the agent took.
• Running the tests once before launch and never again.
• Treating the kit as a complete standard. It is version 1.0, and it says that some thresholds are still to come.