Sandbox and limits

By PennyForYours

What the sandbox does, how fast you can call it, and what is not built yet.

Last checked against the rate-limit and question settings in the code.

On this page

How the sandbox works

  • Keys starting pfy_test_ are sandbox keys. They are the only keys issued in early access.
  • Sandbox questions skip human screening, because they never reach real members. Synthetic test members answer them through the same answer path real members use, a few at a time, usually within a few minutes.
  • Answers carry synthetic: true, questions sandbox: true, and receipts sbx: true and chk: ["synthetic"]. Sandbox receipts are signed with a separate sandbox key (published at /v1/sandbox/jwks.json, never in the production key set), so you can test verification end to end, but the verify endpoint always says valid: false and sandbox: true for them.
  • Nothing is charged. Quotes show the launch example price alongside charge_minor: 0.
  • Every sandbox record is flagged as sandbox. Sandbox questions are never shown to real members and never affect their Standing.

Testing tips

  • Use pools of 5 while testing: they fill fastest.
  • Synthetic answers are canned lines, so do not read anything into them.
  • Test your error handling: a link in the prompt returns 422 url-questions-disabled; cancelling after the first answer returns 409 invalid-state.
  • Rate limits apply in the sandbox exactly as they will in live use.

Question limits

Limits on a question
ItemLimit
Pool size5 to 50 people
Question text3 to 280 characters, no links
Options (quick pick)2 or 3, each up to 80 characters, all different
ImagePNG, JPEG or WebP. Up to 4 MB, which keeps requests under the hosting platform's body limits (about 4.5 MB of binary, 6 MB of JSON including base64)
Deadline1 to 168 hours (optional)
Long-poll waitUp to 25 seconds per call
Event streamUp to 50 seconds per connection, then reconnect with Last-Event-ID
Idempotency key8 to 100 characters, required to ask

Rate limits

Limits are token buckets per key, shared between REST and MCP: a burst up to the bucket size, then a steady refill. Over the limit you get 429 rate-limited with Retry-After (over MCP, a tool error with retry_after).

Per-key rate limits
WhatBurstSustained
Quotes and receipt verification6060 a minute
Asking questions1010 a minute
Reading results, receipts and streams120120 a minute
Image uploads10120 an hour
Opening event streams5About 5 open at once
Long-polls (wait_seconds above 0)512 a minute, so about 5 waiting at once
Ratings, cancelling, balance, usage and question types120120 a minute
MCP requests (any method)120120 a minute

Not available yet

  • Live keys (pfy_live_), payments and credit packs.
  • Webhooks. Use long-polling or the event stream for now.
  • URL questions (a page or link shown to members). Upload a screenshot instead.
  • Short answer, considered feedback and expert view types.
  • OAuth sign-in for MCP clients. Use an API key header.
  • Keyless pay-per-call (x402). Every call needs an operator key.
  • SDKs. Generate a client from the OpenAPI document.
  • An uptime commitment. Design for retries: send an Idempotency-Key and resume streams with Last-Event-ID.

Questions developers ask

How many sandbox keys can I have?

Keys are issued by invitation. Ask if you need more than one, for example one per environment.

Do sandbox questions go through human screening?

No, because they never reach real members. Live questions are screened, and during early access a person reviews each one.

Why are synthetic answers so bland?

They're canned lines, so you can test your flow. Don't read anything into them.

Do rate limits differ between sandbox and live?

No. The sandbox applies the same limits, so what works in testing works live.

Is MCP rate-limited separately from REST?

They share a budget per key: a tool call spends from the same bucket as its REST endpoint. MCP also has its own cap of 120 requests a minute.

What happens if I hit a limit?

You get 429 rate-limited with Retry-After (over MCP, a tool error with retry_after). Wait that long and retry.

Stuck?

Use the developer route on the contact page and include the requestId from the error response, so we can find the call. A real person replies, usually within 2 working days.

Contact the developer route