Human-in-the-loop AI: an API for real human judgement

Once live answering opens, use this human in the loop platform when a decision needs a first reaction rather than a calculation. Suppose an agent has drafted two replies to a customer. Before it sends one, it could ask five people which is clearer and what tripped them up. The same pattern could work for a delicate email or two product images. Keep the question neutral and put the result in front of the person who owns the decision. The quickstart shows the sandbox REST flow.

A small pool may give you a direction and reasons to inspect. It does not establish what a whole market would do. Read the comments alongside the split, especially when the choices are close.

Sandbox only during early access. Its synthetic test members let you test the integration; they provide no customer evidence. Live requests, paid answering and targeted panels are not open. A question goes to 5 to 50 people. If you need thousands of labels, or a decision before the next API call returns, this is the wrong tool.

Start with GET /v1/question-types. You can request a launch-example quote, then send the question with a unique idempotency key. This example uses a sandbox key and fictional values:

POST /v1/quotes
Authorization: Bearer pfy_test_...
Content-Type: application/json

{"type":"quick_pick","prompt":"Which reply sounds clearer?","options":["Reply A","Reply B"],"pool":5}

POST /v1/questions
Idempotency-Key: example-unique-request-1

{"type":"quick_pick","prompt":"Which reply sounds clearer?","options":["Reply A","Reply B"],"pool":5,"quote_id":"example_quote_id"}

POST /v1/questions returns a question ID and cursor. Pass both to GET /v1/questions/{id}?cursor=…&wait_seconds=25 to wait for a change. The result contains the count or rating, individual reasons and a short summary. In the sandbox, every answer is marked synthetic. One quick judgement call is not a substitute for a full user study.

Each answer has a signed ES256 JWS receipt. It binds the question, the exact asset shown, the answer and its status without naming the member. Verify a live receipt against /.well-known/jwks.json. Sandbox receipts use /v1/sandbox/jwks.json and explicitly say they are synthetic; they are not proof that a person responded. The receipt guide has verification steps and the claim fields.

Nothing is charged for synthetic sandbox answers. Asking is free during early access. At paid launch, a quick take is planned from 30p an answer plus 50p a question (launch example); 20 quick takes come to about £6.50. These are asker prices, and the paid launch has no date. A sandbox key is issued by invitation. See developer pricing and current request limits before building around a particular rate.