Answer randomisation

Answer randomisation requires @userreason/js 0.4.0 or later. It is optional and off by default. See Writing useful surveys for choosing when it helps.

Supported questions

Question	What changes
Single or multiple choice	Option order, including supported dropdown and image-choice presentations.
Matrix	Row order. The numeric rating columns keep their order.
Ranking	The initial option order. The respondent still supplies the final ranking.
Rating and other question types	Answer randomisation is not supported. Rating scales keep their meaningful order.

In the dashboard question editor, select Shuffle above the options or rows. Use Keep at end on items that should remain last. An explicit own-words option starts pinned when shuffling is enabled; review the pins when editing the choices.

JSON configuration

Add answer_order to a supported question in a complete survey definition:

json

{
  "id": "reason",
  "kind": "single_choice",
  "prompt": "What would you most like us to improve?",
  "required": false,
  "options": [
    { "id": "setup", "label": "Setup" },
    { "id": "reports", "label": "Reporting" },
    { "id": "other", "label": "Something else", "other": true }
  ],
  "answer_order": { "mode": "randomized", "fixed_last_ids": ["other"] }
}

fixed_last_ids contains unique IDs belonging to this question. At least two items must remain movable. Fixed items stay last in their original authored order. For a matrix, use row IDs. Omit answer_order for the authored order. Saving a draft can preserve an unfinished configuration; publishing rejects invalid pins or too few movable items.

Keep option IDs stable when changing labels. Answers and routing use IDs, not display positions. For rankings, shuffling the initial list does not submit a ranking, and an untouched list does not count as an answer.

SDK support and saved order

The compatible SDK requests presentation=v1. The API withholds surveys requiring this capability from older installations. In draft validation, declare client_capabilities: ["presentation:v1"] only after verifying that the installed SDK supports it. This declaration checks configuration; it does not upgrade the SDK.

The server saves a fresh order for each admitted attempt. Going back, retrying the same request or continuing that attempt preserves its order. A later attempt can receive a different order even for the same person. Experiment variant assignment, by contrast, stays stable within a run.

Reading the results

On the dashboard Results tab, shuffled questions have Totals and By position views. Position data also appears in the experiment report, including for surveys without an experiment node.

Choice cells show selections divided by presentations at that position.
Ranking and matrix rows include mean rank or rating among answers.
Pinned items have no movable positions to compare.
Position statistics cover the pinned survey version. By default each person's first recorded presentation of each question counts; all-attempt and reported-only filters are also available through the report API.

Response detail's As presented view shows the saved item order. An assigned order alone does not establish that the person saw it. A reported question means the browser sent exposure evidence; an order-confirmed question additionally means its order digest matched the saved order. Exposure can also be inferred from accepted answers. The report keeps these counts separate.

Differences between positions are descriptive evidence. They do not establish that shuffling removed all bias or that position caused the difference.