# Experiments

Deterministic A/B experimentation — sticky variant assignment and deduped exposure tracking, as a Bijection component

- Package: @bijectionhq/experiments
- Category: Database & Backend
- Status: Included
- License: (MIT AND Apache-2.0)
- Bijection source (repository access required): https://github.com/bijectionhq/experiments/tree/f9e797855da70dbc3572377a23fe1ac794fb0a32
- Upstream: https://github.com/vllnt/convex-experiments/tree/5cba3038154ba9989ccb70ea3ca560fb85263808
- Documentation: https://docs.bijection.com/components/using
- Directory: https://bijection.com/components/experiments

## Get the source

Bijection repository access is required. Request access at https://bijection.com/contact.

```sh
git clone https://github.com/bijectionhq/experiments.git
cd experiments
git checkout f9e797855da70dbc3572377a23fe1ac794fb0a32
```

## Before using this component

Read the package README at the recorded revision for setup, dependencies, and API examples. Upstream documentation describes the original package and may differ from the Bijection adaptation. This source is included in Bijection's workspace. Catalog inclusion does not establish npm availability or runtime qualification. Check compatibility with your SDK and deployment before adoption. External providers run their own compute.

## Upstream package documentation

Recorded 2026-10-23. This describes @vllnt/convex-experiments; check the Bijection adaptation separately.

@vllnt/convex-experiments assigns subjects to weighted variants and records which variants they encounter. It supplies the assignment and exposure layer of an A/B test while your application measures outcomes.

Define weighted variants, start enrollment, and use logExposure to assign and record a subject. peek resolves a variant without writing, and results returns assignment and exposure tallies. Stopping an experiment preserves its recorded data. Scopes separate experiments across surfaces or tenants.

The host authenticates subjects, authorizes experiment management, and records conversions or quality scores. Exposure tallies are not conversion analysis or statistical significance. Use a new experiment key when changing an established allocation.

Maintained by [bntvllnt](https://bntvllnt.com) at [VLLNT](https://vllnt.com). Find the author on [GitHub](https://github.com/bntvllnt) and [X](https://x.com/bntvllnt).

### Use cases

- Mobile onboarding: keep each person on the same introduction flow across visits.
- SaaS conversion tests: compare checkout copy, upgrade screens, or feature presentations.
- AI applications: assign prompt or model configurations and record outcomes in your own evaluation data.
- Games and learning: compare challenge presentations or tutorial variants with stable enrollment.

### How it works

Define weighted variants, start enrollment, and use logExposure to assign and record a subject. peek resolves a variant without writing, and results returns assignment and exposure tallies. Stopping an experiment preserves its recorded data. Scopes separate experiments across surfaces or tenants.

The host authenticates subjects, authorizes experiment management, and records conversions or quality scores. Exposure tallies are not conversion analysis or statistical significance. Use a new experiment key when changing an established allocation.

### Upstream README

# @vllnt/convex-experiments

Deterministic A/B experimentation — sticky variant assignment and deduped exposure tracking, as a
Convex component.

```ts
const experiments = new Experiments(components.experiments);
await experiments.define(ctx, "checkout_button", {
  variants: [
    { key: "control", weight: 1 },
    { key: "treatment", weight: 1 },
  ],
});
const { variant } = await experiments.logExposure(ctx, "checkout_button", userId);
if (variant === "treatment") showNewButton();
```

Define an experiment with weighted variants; enroll subjects into a **sticky** variant chosen
deterministically from `(salt, subjectRef)`; record exposures and read per-variant tallies. Pair it
with a feature-flag system for kill-switches and measure conversions in your own tables — this
component handles assignment and exposure. Domain-neutral: any subject, any surface.

## Features

- **Deterministic + sticky** — the same subject always lands in the same weighted variant, even before anything is stored.
- **Weighted variants** — split traffic by relative weights (`1:1`, `90:10`, N-way); equal weights split evenly.
- **Enrollment lifecycle** — `draft → running → stopped`; only `running` enrolls, and stopping preserves recorded data.
- **Deduped exposures + O(variants) results** — one tallied row per subject; `results` reads maintained per-variant tallies (`assigned`/`subjects`/`exposures`/`weight`), never scanning the exposure table.
- **Sample-ratio-ready** — `assigned` + `weight` per variant let you check observed vs expected split (SRM).
- **`peek`** — a read-only deterministic query returns a subject's sticky variant without writing (SSR / flicker-free first paint).
- **Immutable once assigned** — `variants`/`salt` are fixed after enrollment (changing them throws); define a new key to re-randomize.
- **Lifecycle + GDPR** — `listExperiments` to discover, `forgetSubject` to erase one subject, `deleteExperiment` to cascade-delete.
- **Scopes** — global by default, or namespace per tenant / surface (folded into the hash for independent bucketing).
- **Fully typed** — variant keys, weights, and outcomes are concrete types end to end; no `any`.
- **Server-sourced time** — assignment timestamps come from the server, never the caller.

## Installation

```bash
pnpm add @vllnt/convex-experiments
```

Peer dependency: `convex@^1.41.0`.

## Usage

```ts
// convex/convex.config.ts
import { defineApp } from "convex/server";
import experiments from "@vllnt/convex-experiments/convex.config";

const app = defineApp();
app.use(experiments);
export default app;
```

```ts
// convex/checkout.ts — host owns auth; pass an opaque subjectRef in.
import { components } from "./_generated/api";
import { mutation } from "./_generated/server";
import { v } from "convex/values";
import { Experiments } from "@vllnt/convex-experiments";

const experiments = new Experiments(components.experiments);

export const view = mutation({
  args: { userId: v.string() },
  handler: async (ctx, { userId }) => {
    const { variant } = await experiments.logExposure(ctx, "checkout_button", userId);
    return { variant }; // "control" | "treatment" | null (not enrolled)
  },
});
```

Measure the outcome in your own table, joined on the assigned variant — then read tallies with
`results(ctx, "checkout_button")`. See [`example/convex/example.ts`](example/convex/example.ts) for a
host-side conversion recorder.

## API Reference

| Method | Kind | Result |
|--------|------|--------|
| `define(ctx, key, { variants, scope?, salt?, status? })` | mutation | `{ created: boolean }` |
| `setStatus(ctx, key, status, scope?)` | mutation | `boolean` |
| `assign(ctx, key, subjectRef, scope?)` | mutation | `{ variant: null } \| { variant: string; isNew: boolean }` |
| `logExposure(ctx, key, subjectRef, scope?)` | mutation | `{ variant: string \| null }` |
| `forgetSubject(ctx, key, subjectRef, scope?)` | mutation | `boolean` (GDPR erasure) |
| `deleteExperiment(ctx, key, { scope?, batch? })` | mutation | `number` (cascade delete) |
| `getExperiment(ctx, key, scope?)` | query | `ExperimentDefinition \| null` |
| `listExperiments(ctx, { scope?, status? })` | query | `ExperimentDefinition[]` |
| `getAssignment(ctx, key, subjectRef, scope?)` | query | `{ variant, assignedAt } \| null` |
| `peek(ctx, key, subjectRef, scope?)` | query | `{ variant: string \| null }` (no write) |
| `results(ctx, key, scope?)` | query | `{ variant, assigned, subjects, exposures, weight }[]` |

Full reference: [docs/API.md](docs/API.md).

## React

Optional, tree-shakeable hooks via `@vllnt/convex-experiments/react` (`react` is an optional peer
dep — a backend-only consumer pulls none of it). Each hook wraps `useQuery` over a query reference
**you re-export** from your app, so the component never owns your `api`.

```tsx
// convex/experiments.ts — re-export the host-side wrappers (auth gated)
// export const myVariant = query({ args: { userId: v.string() },
//   handler: (ctx, { userId }) => experiments.peek(ctx, "checkout_button", userId) });

import { useVariant } from "@vllnt/convex-experiments/react";
import { api } from "@/convex/_generated/api";

function CheckoutButton({ userId }: { userId: string }) {
  // deterministic first paint via peek → no flash-of-control
  const variant = useVariant(api.experiments.myVariant, { key: "checkout_button", subjectRef: userId });
  return variant === "treatment" ? <OneClick /> : <Classic />;
}
```

| Hook | Wraps | Returns |
|------|-------|---------|
| `useVariant(peekRef, args)` | `peek` | `string \| null \| undefined` (variant / not-enrolled / loading) |
| `useAssignment(getAssignmentRef, args)` | `getAssignment` | `Assignment \| null \| undefined` |
| `useExperimentResults(resultsRef, args)` | `results` | `VariantResult[] \| undefined` |

## Security

- Auth-agnostic — the host resolves identity and decides who may define, start, stop, or enroll.
- Tables sandboxed — reached only through the exported functions; `subjectRef`, variant keys, and `scope` stay opaque.
- Server-sourced timestamps — a caller cannot supply assignment time.

See [docs/API.md](docs/API.md).

## Testing

```bash
pnpm test           # single run
pnpm test:coverage  # enforced 100% on covered files
```

Tests run against the real component runtime via `convex-test` (`@edge-runtime/vm`), not mocks.

## Contributing

See [CONTRIBUTING.md](CONTRIBUTING.md).

## Author

Built by [bntvllnt](https://github.com/bntvllnt) · [bntvllnt.com](https://bntvllnt.com) · [X @bntvllnt](https://x.com/bntvllnt)

Part of the [@vllnt](https://github.com/vllnt) Convex component fleet — [vllnt.com](https://vllnt.com)

If this is useful, [sponsor the work](https://github.com/sponsors/bntvllnt).

## License

MIT — see [LICENSE](LICENSE).