Calvin Simulate: from a mission statement to agent simulations

Turn a mission into commitments you can check#

A mission such as “Help patients book the right appointment, easily” describes the desired result. Simulate turns that mission into scenarios and commitments you can review. For a booking agent, commitments might include creating the appointment before confirming it, collecting required new-patient details, and routing after-hours requests to the front desk.

Simulate handles scenario creation. Your team reviews the commitments against its business rules, so the checks reflect the work you actually need done. The review should cover:

• State the behavior or outcome that must hold.
• Name the evidence source and the fields needed to make a reliable comparison.
• Set the acceptance threshold and timing for the workflow.
• Assign an owner for failures and ambiguous rules.
• Define what happens when the evidence cannot be obtained.

Exercise ordinary requests and difficult cases#

Simulate creates scenarios covering everyday requests and difficult edge cases. For a booking mission, these can include an available appointment, a conflicting request, and an after-hours handoff. Evaluate the resulting record or received handoff as well as the agent’s reply.

Test unavailable evidence separately. If calendar access is missing, the outcome should stay unverified. That is a different result from checking the calendar successfully and finding no matching appointment.

Make the acceptance report actionable#

Record the commitment, result, supporting evidence, and responsible owner. Preserve unresolved or unverified results so the release decision reflects what was actually tested. Do not hide evidence gaps inside an overall success score.

Simulate gives each commitment a clear acceptance result. Review the evidence, fix issues, and rerun the checks. Your team retains control over changes and the release decision.

Reuse the checks when the agent changes#

Rerun Simulate checks for new agent versions and changed business rules. The same commitments give your team a consistent basis for reviewing the result.

After launch, integrate your agent with Observe in 5 minutes. Observe checks real outcomes, groups recurring problems, and routes evidence to a named owner. Turn those production examples into future Simulate checks so each release benefits from what happened in real work.

Start with the job your agent needs to do#

Write a simple mission statement, then review the scenarios and commitments Simulate creates. You bring the business context; Calvin handles the simulation work.

Use Observe after launch to keep checking outcomes and send new issues back into the simulation cycle. Simulate, Observe, and Improve are the three products. Define the mission, test the agent, observe real work, then use Improve to recommend prompt, configuration, or tool changes based on observed misalignment.