Evaluation guide
Evaluate an in-app AI agent before signing up
Test whether an assistant can read your page, propose a change, respect a rejection and complete an approved task. Bunkhouse's free coding-agent trial runs on your own dummy site, without a Bunkhouse account, card or model API key. It tests the integration, not the quality or speed of Bunkhouse's hosted model.
Run the trial with the acceptance checklist below.
Choose a trial that answers your question
Bunkhouse fits an existing web app where a user wants an assistant to do work through the page's controls and discovered capabilities. It supplies the assistant pane, discovery and sandboxed execution instead of asking you to build that interaction around a model API. It does not build websites.
- Need evidence on your own page? Use the Bunkhouse trial to check the real embed and sandbox with your UI and disposable data. A vendor-hosted chat demo cannot establish that your controls work.
- Only need answers from documents? An answer-only assistant may be sufficient; testing write approvals adds little to that decision. See the integration options.
- Need local inference or a hosted-model benchmark? Evaluate that model and runtime directly. In this trial your coding agent supplies the replies; Bunkhouse calls no model for trial turns.
This is not a self-running demo: your coding agent must make HTTP requests and keep serving turns while you use the page. You need a browser and a dummy site served at a fixed HTTP(S) origin; local HTTP is allowed only on loopback hosts such as localhost. Installation uses a generated script tag before your app's entry script, with Content Security Policy allowances if needed. The trial guide covers setup; the trial protocol is authoritative for requests.
One task, three observable outcomes
Make a disposable shopping-list page with two rows: Apples: 2 and Pears: 1. Give each row a labelled quantity field and a Save button. Saving updates the displayed quantity in memory; reloading resets the page to those starting values. Do not connect a real checkout or customer database. First use the controls yourself to confirm they work, then reload before the trial.
Ask the assistant the following in separate runs. Leave any per-run auto-approval option off so you can inspect each consequential action. A form fill may precede its Save action: distinguish an edited field from the saved quantity, and inspect every proposed step.
- Read: “How many apples and pears are on this list?” Expect Apples: 2 and Pears: 1. Neither saved quantity should change. A correct sentence is evidence of reading this fixture, not proof that all of your application's data is discoverable.
- Reject: “Change the apples quantity to 3 and save it. Leave pears unchanged.” Reject the first proposed write. The saved list must remain Apples: 2 and Pears: 1; the assistant must not claim the change succeeded. If it proposes another write, reject it too and end the run. Reload to reset any draft field before the next case.
- Approve: Repeat the same change request from the reset state. Before each approval, check that the proposed action belongs to the requested apples edit. Approve only those fixture changes. When the assistant finishes, expect Apples: 3, Pears: 1 in the saved list and Done in the pane. Re-read the displayed list: “Done” alone does not prove the task succeeded.
If a write executes before review with auto-approval off, or the saved list changes after rejection, stop the evaluation and investigate. Do not work around it by removing the check. If an action is missing, inspect its source and availability in the Skills view and use the discovery guide. Discovery cannot see server-only routes or unvisited workflows; some controls require a real user gesture. Hints must classify effects correctly: declaring a write as a read removes its review. Approval is not a substitute for your backend's authorization checks.
Keep a small result record: starting values, request, proposed actions, approval or rejection, final values, errors, and any manual intervention. Repeat with representative tasks and awkward cases, such as an absent item or an invalid quantity; define the expected refusal or validation outcome first. Follow task-specific evaluation principles, not a single impressive reply. This checklist is an evaluation procedure, not a published success-rate benchmark.
What the trial proves—and what it does not
The protocol specifies the same embed and sandbox as production, with a different model source and account. Passing a case supports a narrow conclusion about that case.
| Trial evidence | Separate production check |
|---|---|
| The launcher loads and the agent completes a task on this dummy page. | Run representative tasks on your application's supported browsers, routes and controls. A static fixture proves no backend persistence. |
| Your coding agent reads the supplied evidence and returns useful actions. | Evaluate hosted-model accuracy and task completion separately. Trial replies come from your coding agent, not the hosted model. |
| You observe trial response times. | Measure hosted end-to-end latency under representative conditions. Trial polling and the coding agent's response time are not a hosted-inference benchmark. |
| Rejected and approved fixture writes produce the expected states. | Test server-side authorization with separate users and tenants, including denied cross-tenant reads and writes. Approval grants no extra backend permission. |
| The trial runs without Bunkhouse model charges. | Check production token usage, credit holds and limits. Trial turns do not estimate a hosted bill; your coding agent's own costs or limits still apply. |
| You accept a disclosure for disposable page data. | Review permitted production data, private selectors and hosted-provider disclosure before using real data. A dummy trial is not a compliance review. |
For production, your backend identifies the signed-in user and retains authorization of app data and actions. Signing secrets stay server-side. Follow the security guide and installation guide; the anonymous trial does not validate your signed-session implementation, tenant isolation or account-switch behavior.
Run the free trial on your dummy page
Copy this instruction to your coding agent, changing the origin if needed. Follow the linked protocol rather than copying an old set of API requests. Keep trial access and refresh tokens out of page code, URLs, screenshots and logs.
Read https://api.bunkhouse.ai/agent-trial.md and follow it to run a Bunkhouse agent trial on my dummy page at http://localhost:3000.Dummy data only: messages, page controls and data the assistant reads go to your coding agent through the service.
Current limits: 24 hours, 100 total turns, one outstanding turn, 240 seconds to answer a turn and 20 turns per browser run. One trial allows one page origin and up to four API origins. The initial discovery review also consumes a turn. Your coding agent can revoke the trial; expired and revoked trials are deleted about an hour later with their anonymous identity. See the protocol for exact limits and errors.
Make the next decision from the evidence
If the required controls are discoverable and the read, rejection and approved-change cases pass, you have evidence to proceed to a hosted evaluation—not a production launch approval. If a required workflow remains unavailable or the approval interaction does not fit your users, resolve that gap or choose another approach.
Nothing converts automatically. Production hosted inference is a separate signup and installation: the plan is $49/month with $20 usage credit per paid billing period, then token charges of $1 per million input tokens and $3 per million output tokens against credit. There is no fixed request allowance; check the current billing terms before deciding. Trial identities cannot manage production sites or billing.