The Dynoxide hexagonal logo centred on a near-black background

Dynoxide 1.3.0: security fixes and fussier pagination

Dynoxide 1.3.0 ended up a bit bigger than I'd planned. Working through the compatibility fixes turned up some DynamoDB behaviour I hadn't expected, and I also fixed two security issues that needed advisories.

There's a lot in the release notes, so I've picked out a few highlights below, including a contribution that makes test runs quieter and a few things to check before upgrading.

Two security fixes

One of these was in the code that explains why an expression is invalid. A request containing certain non-ASCII characters could make Dynoxide panic while building the syntax error's near: text. It cut the string in the middle of a UTF-8 character.

Release binaries abort on panic, so an emoji in a malformed expression could stop the server whilst it was trying to reject the request.

  • Expression parsing (rated High, 7.5): affected product versions are 0.9.10 through 1.2.1. An invalid character now gets a syntax error instead. Anyone who can reach the HTTP server could trigger the old behaviour without credentials, because it doesn't check request signatures. The default loopback binding limits that exposure, but a server on a shared network needs the upgrade. An in-memory database goes with the process; a file-backed database retains its last completed write. If you embed dynoxide-rs, the same call panics rather than aborting the process, unless you build with panic = "abort". Details in GHSA-gwhp-7jch-7xj6.
  • Tokens in help output (rated Low, 2.8): from 0.10.0 through 1.2.1, subcommand help printed the value of DYNOXIDE_MCP_AUTH_TOKEN if it was set. That's an unfortunate thing to put in a CI log or paste into a bug report. Help now names the variable without showing its value. If you've shared output containing the token, rotate it. Details in GHSA-3rv6-mrj3-vf9j.

Both are fixed in product 1.3.0 and Rust crate 3.0.0. The token fix also has a regression test checking which environment-backed flags are allowed to show their values.

There's a separate aws-smithy-json dependency update in the release notes. That dependency only reaches Dynoxide through the AWS SDK used by tests and benchmarks, and isn't shipped in the released artefacts.

The pagination rules are fussier than I'd expected

DynamoDB will give you a PartiQL pagination token, then refuse it if you add a space to the statement on the next request. Send the same list parameter again and it can refuse that too.

Dynoxide used to accept a pagination token when the statement parsed the same way. DynamoDB binds it to the text as sent, with some rather particular rules for parameters. 1.3.0 follows those rules:

  • Statement text must stay identical. Whitespace, keyword case and even the order of keys in a map literal matter. You can change Limit between pages.
  • Number parameters must match in value and scale. 1E2 and 1e+2 resume, as do 1.5E1 and 15, but 100 and 1E2 don't, and neither do 0 and 0.0.
  • Set order doesn't matter. Reordering the members still lets you resume.
  • List and map parameters don't resume at all. DynamoDB hands out a token on the first page, then rejects it even if you send the same parameter again. This was captured against both eu-west-2 and us-east-1.

If your application rebuilds or reformats its statement between pages, a local test should catch that. Previously Dynoxide let it through.

Requests that used to do the wrong thing

The fixes I care most about are the ones where a request succeeded and quietly did something else:

  • PartiQL comments or a table alias could lose the filter. A -- comment before WHERE could turn a SELECT into a read of every row, and so could a table alias such as FROM "T" t. Comments are now read as whitespace, and an alias is refused, as DynamoDB refuses it.
  • An unfilled ? could match items. flag <> ? with no parameters held for every item without flag, so a DELETE or UPDATE short of its parameters could delete or change them. A statement with more placeholders than parameters is now refused.
  • ORDER BY was accepted and ignored. Descending reads now actually descend, with DynamoDB's restrictions on which key combinations can be ordered.
  • Malformed legacy conditions could disappear. An invalid Expected condition could leave a write unguarded, and an invalid QueryFilter or ScanFilter could leave a read unfiltered. These requests are now refused.
  • A list write past the end padded the gap with NULLs. DynamoDB appends the value. SET tags[20] = :v on a two-element list now adds a third element.
  • A storage error could break transaction atomicity. SQLite can roll back a transaction itself on a full disk or an I/O error. Dynoxide used to carry on, allowing later actions to commit individually. Both transaction APIs now stop on the storage failure.

There's plenty of error-message work too, but these change what your application reads or writes.

More useful figures for tables and vectors

This release fixes how Dynoxide calculates vector write capacity, fills in missing table and index statistics, and makes capacity figures available through MCP:

  • Vector writes use four bytes per dimension. Previously Dynoxide sized the embedding as a list of numbers, which could add two or three write units for a 1,024-dimension vector. Reads and the 400 KB item limit still measure its numbers, as DynamoDB does.
  • Table and index sizes are populated. DescribeTable now reports table sizes and secondary-index counts and sizes that previously came back as zero. Dynoxide reports these live; AWS refreshes them roughly every six hours.
  • MCP tools can return consumed capacity. Ask for return_consumed_capacity: "INDEXES" to get the API's capacity breakdown through the tool response.

Global secondary indexes with multiple partition or sort key attributes also work properly now: sorting uses each sort key attribute in turn, queries require every partition key attribute, and items missing any key attribute stay out of the index. PartiQL still can't query these indexes, matching DynamoDB's restriction.

Quieter test runs, thanks to a contribution

Thanks to Anvilly Huang (@zhua633) for contributing --log quiet in #216. If your tests start and stop Dynoxide, it keeps the informational startup and shutdown messages out of their output:

dynoxide serve --log quiet

Warnings and errors still print. I've extended the option to MCP and import-and-serve too; the first-run MCP token message and the import report still appear.

Before upgrading

  • Keep a copy if you may roll back. Opening a persistent database upgrades it to schema version 11, and there's no way back. An older version only misreads the file if it has a multi-attribute GSI, whose entries are rebuilt, or a vector index, whose item sizes are recomputed. From 1.3.0, a file written by a newer Dynoxide is refused rather than opened. This applies to browser databases too.
  • Check Rust dependency versions. The product is 1.3.0, but dynoxide-rs is 3.0.0 because the Rust API has breaking changes. The parser modules are private and some PartiQL request and error structs have new fields. The crate still supports Rust 1.88; building its tests needs 1.94.1.
  • Check updates near the item-size limit. UpdateItem now measures the request's update charge alongside the finished item's size. Some previously refused updates succeed, and some previously accepted ones fail, matching the behaviour captured from DynamoDB.

The 1.3.0 release notes have the full list. For an npm project, npm install --save-dev dynoxide@1.3.0 pins this release.