The Polyglot's Dilemma: Conformance Testing a Dozen Specs in as Many Languages

A closer look at The Polyglot's Dilemma: Conformance Testing a Dozen Specs in as Many Languages

Why MongoDB needed a shared test language

MongoDB implements client drivers natively in many programming languages, with most drivers rewritten in their target language rather than being thin wrappers. That created a challenge: ensure behaviors and APIs line up across languages while allowing language-appropriate idioms.

Early proposals like Cucumber were rejected; engineers wanted something different. The team settled on a custom test language built on YAML to specify and run the same tests against each driver implementation.

What the Unified Test Format (YAML tests) actually does

The project uses a Unified Test Format based on YAML to express driver conformance tests. Each YAML file describes schemaVersion, runOnRequirements, createEntities, initialData, operations, and expected outcomes.

Test runners for each driver interpret the YAML, map operations to the language-specific API (for example updateOne -> update_one in Python), and use an internal MongoClient to perform server-side actions like failPoint. JSON Schema validates the YAML syntax for its declared schemaVersion.

Who benefits and what issues it catches

Driver engineers working on different language implementations benefit: the same tests can prove consistent behavior, reveal hidden behaviors, and enforce spec changes across drivers.

The tests can exercise things ordinary integration tests can’t observe (for example forcing the server to close a connection with failPoint to test retry behavior), verify event sequences, and confirm final database state via outcome checks.

Questions people ask about The Polyglot's Dilemma: Conformance Testing a Dozen Specs in as Many Languages

Can I see an example test file and how it maps to driver APIs?

The page shows an example Unified Test Format file that uses fields like failPoint, updateOne, expectResult, waitForEvent, expectEvents, and outcome, and explains how runners translate those to each driver's API.

How do test runners handle incompatible server deployments?

YAML files include runOnRequirements so a test runner skips files unless the MongoDB server deployment matches the stated requirements.

How is the YAML syntax kept consistent across versions?

schemaVersion tells the test runner which features the file uses, and JSON Schema is used to check that each YAML test has the proper syntax for its version.

The Polyglot's Dilemma: Conformance Testing a Dozen Specs in as Many Languages was shared on NexURLs by Nexis on October 25, 2026, in Learning & Books. So far it has 1 like from members.