DynamoDB Conformance Suite

About the conformance suite

AWS doesn't publish a conformance suite for DynamoDB. So every emulator author ends up guessing at how the real thing behaves and testing against their own assumptions. That's how subtle differences slip through and surface later as production bugs you can't reproduce locally.

The suite at dynamodb-conformance.org closes that gap. It runs every test against real DynamoDB on AWS first, records what passes, and treats that as the baseline. An emulator only passes a test if it gives the same answer DynamoDB does. Real DynamoDB sits at the top of every table at a flat 100%, because it's the thing everything else is measured against.

Behaviour, not "works with the SDK"

Plenty of emulators will take your requests and hand something back. The question that matters is whether they hand back the same something. A subtly different error, a validation check that fires in the wrong order, a filter that returns one extra item: none of it breaks the happy path, all of it bites you in production.

So the suite tests observable behaviour and nothing else. It drives the standard AWS SDK against an HTTP endpoint and asserts on what comes back: the response shape, the exact error, the order things are validated in. No internal hooks, no privileged access. If your application talks to the target through the SDK, that's exactly what gets checked.

Three tiers, because one number hides too much

A single "92%" tells you almost nothing. Ninety-two percent of what? So the suite splits its tests into three tiers. Core covers the CRUD, queries, scans and batch operations most people rely on. Complete adds the documented but less common features: transactions, PartiQL, TTL, streams. Strict checks validation ordering, exact error wording, limits and legacy API shapes. A gap in Core bites everyday code; a gap in Strict mostly bites your CI when it asserts on error messages.

Why a whole site for it

A README can show a single snapshot. The site shows everything else: how each emulator's score has shifted over time, why it moved, and which run-over-run changes were noise versus signal.

Every figure on the site is built from the suite's own results at build time, never typed in by hand. The moment you copy a number into a second place the two start drifting apart.

Built alongside Dynoxide

I built the suite while building Dynoxide, my own DynamoDB emulator, because I needed a way to prove it behaves like the real thing rather than just hoping it did. It runs against every emulator, not only mine, so the comparison stays honest.

Repository

The suite is Apache-2.0 licensed and every test is in the repo. Read them, run them against your own target, or send a PR. That's how the comparison stays honest.