schemathesis
Property-based testing framework for Open API and GraphQL based apps
Description
<p align="center"> <a href="https://github.com/schemathesis/schemathesis/actions" target="_blank"> <img src="https://github.com/schemathesis/schemathesis/actions/workflows/build.yml/badge.svg" alt="Build"> </a> <a href="https://codecov.io/gh/schemathesis/schemathesis/branch/master" target="_blank"> <img src="https://codecov.io/gh/schemathesis/schemathesis/branch/master/graph/badge.svg" alt="Coverage"> </a> <a href="https://pypi.org/project/schemathesis/" target="_blank"> <img src="https://img.shields.io/pypi/v/schemathesis.svg" alt="Version"> </a> <a href="https://pypi.org/project/schemathesis/" target="_blank"> <img src="https://img.shields.io/pypi/pyversions/schemathesis.svg" alt="Python versions"> </a> <a href="https://discord.gg/R9ASRAmHnA" target="_blank"> <img src="https://img.shields.io/discord/938139740912369755" alt="Discord"> </a> <a href="https://opensource.org/licenses/MIT" target="_blank"> <img src="https://img.shields.io/pypi/l/schemathesis.svg" alt="License"> </a> </p> ## Schemathesis > **Catch API bugs before your users do.** Schemathesis automatically generates thousands of test cases from your OpenAPI or GraphQL schema and finds edge cases that break your API. <p align="center"> <img src="https://raw.githubusercontent.com/schemathesis/schemathesis/master/img/demo.gif" alt="Schemathesis automatically finding a server error"/> <br> <i>Finding bugs that manual testing missed</i> </p> ## Try it now ```console # Test a demo API - finds real bugs in 30 seconds uvx schemathesis run https://example.schemathesis.io/openapi.json # Test your own API uvx schemathesis run https://your-api.com/openapi.json ``` ## What problems does it solve? - š„ **500 errors** that crash your API on edge case inputs - š **Schema violations** where your API returns different data than documented - šŖ **Validation bypasses** where invalid data gets accepted - š **Integration failures** when responses don't match client expectations - š **Stateful bugs** where operations work individually but fail in realistic workflows > ā ļø **Upgrading from older versions?** Check our [Migration Guide](https://github.com/schemathesis/schemathesis/blob/master/MIGRATION.md) for key changes. # Installation & Usage **Command Line:** ```console uv pip install schemathesis schemathesis run https://your-api.com/openapi.json ``` **Python Tests:** ```python import schemathesis schema = schemathesis.openapi.from_url("https://your-api.com/openapi.json") @schema.parametrize() def test_api(case): # Tests with random data, edge cases, and invalid inputs case.call_and_validate() # Stateful testing: Tests workflows like: create user -> get user -> delete user APIWorkflow = schema.as_state_machine() # Creates a test class for pytest/unittest TestAPI = APIWorkflow.TestCase ``` **CI/CD:** ```yaml - uses: schemathesis/action@v3 with: schema: "https://your-api.com/openapi.json" ``` ## Who uses it Used by teams at **[Spotify](https://github.com/backstage/backstage)**, **[WordPress](https://github.com/WordPress/openverse)**, **JetBrains**, **Red Hat**, and dozens of other companies. > "_Schemathesis is the best tool for fuzz testing of REST APIs on the market. We at Red Hat use it for examining our applications in functional and integration testing levels._" - Dmitry Misharov, RedHat ## See it in action š¬ **[Live Benchmarks](https://workbench.schemathesis.io)** showing continuous testing results from real-world APIs: - Code & API schema coverage achieved - Issues found with detailed categorization - Performance across different fuzzing strategies ## Reporting - **[Allure](https://schemathesis.readthedocs.io/en/stable/guides/allure/)** ā Rich visual reports with per-operation results, failure steps, and curl reproduction commands - **[JUnit XML](https://schemathesis.readthedocs.io/en/stable/guides/cicd/)** ā For GitHub Actions, GitLab CI, Jenkins, and any CI tool that consumes JUnit ## Documentation š **[Documentation](https://schemathesis.readthedocs.io/en/stable/)** with guides, examples, and API reference. ## Get Help - š¬ [Discord community](https://discord.gg/R9ASRAmHnA) - š [GitHub issues](https://github.com/schemathesis/schemathesis/issues) ## Contributing We welcome contributions! See our [contributing guidelines](CONTRIBUTING.md) and join discussions in [issues](https://github.com/schemathesis/schemathesis/issues) or [Discord](https://discord.gg/R9ASRAmHnA). ## Acknowledgements Schemathesis is built on top of <a href="https://hypothesis.works/" target="_blank">Hypothesis</a>, a powerful property-based testing library for Python. ## License This project is licensed under the terms of the [MIT license](https://opensource.org/licenses/MIT).
Release History
| Version | Changes | Urgency | Date |
|---|---|---|---|
| 4.15.2 | Imported from PyPI (4.15.2) | Low | 4/21/2026 |
| v4.15.2 | ### :wrench: Changed - Show compact path for body property violations instead of a verbose chain. ### :bug: Fixed - False positive `negative_data_rejection` for integer query parameters with type mutations. [#3712](https://github.com/schemathesis/schemathesis/issues/3712) - False positive `positive_data_acceptance` for arrays with `uniqueItems` and enum items. | Medium | 4/14/2026 |
| v4.15.1 | ### :bug: Fixed - False positive `negative_data_rejection` for `type: number` body fields in fuzzing. [#3697](https://github.com/schemathesis/schemathesis/issues/3697) - False positive `negative_data_rejection` for `type: integer` query parameters mutated to array. [#3697](https://github.com/schemathesis/schemathesis/issues/3697) - Double JSON encoding for `content: application/json` query parameters in coverage phase. [#3701](https://github.com/schemathesis/schemathesis/issues/3701) - Sibling | Medium | 4/10/2026 |
| v4.15.0 | ### :rocket: Added - `st fuzz` command for continuous multi-step API fuzzing across operation sequences. - Structured generation for `If-Match`, `If-None-Match`, `If-Modified-Since`, `If-Unmodified-Since`, and `Range` headers. - Apply `filter_case` and `map_case` hooks in the coverage phase. [#3675](https://github.com/schemathesis/schemathesis/discussions/3675) - `schemathesis.pytest.parametrize()` for testing multiple named schemas in a single test function. [#1409](https://github.com/schemat | Medium | 4/5/2026 |
| v4.14.3 | ### :bug: Fixed - `deadline=None` lost when `@settings` applied after `@lazy_schema.parametrize()`. - `format: uuid` now generates RFC 4122 compliant UUIDs. [#2909](https://github.com/schemathesis/schemathesis/issues/2909) | Medium | 3/31/2026 |
| v4.14.2 | ### :bug: Fixed - `AttributeError` on `DoctestItem` when schemathesis plugin is loaded. [#3663](https://github.com/schemathesis/schemathesis/issues/3663) - Make dynamic auth (`[auth.dynamic.openapi.<scheme>]`) work with WSGI & ASGI transports. [#3664](https://github.com/schemathesis/schemathesis/issues/3664) | Medium | 3/28/2026 |
| v4.14.1 | ### :bug: Fixed - `ModuleNotFoundError` on `xdist` session end when the allure extra is not installed. | Medium | 3/27/2026 |
| v4.14.0 | ### :rocket: Added - Allure report integration. [#2756](https://github.com/schemathesis/schemathesis/issues/2756) - Support for recording VCR, HAR, and JUnit XML reports from the `pytest` plugin via `schema.config.reports`. [#701](https://github.com/schemathesis/schemathesis/issues/701) - `after_validate` hook fires after all checks run on a response, carrying a list of `CheckResult` objects. - Examples phase generates optional properties and one disjoint set per `oneOf`/`anyOf` branch. [#2371 | Medium | 3/27/2026 |
| v4.13.0 | ### :rocket: Added - `[auth.dynamic.openapi.<scheme>]` config block for token fetch authentication that allows for dynamic authentication without writing Python code. [#3620](https://github.com/schemathesis/schemathesis/issues/3620) - Use captured response data in the examples phase. [#638](https://github.com/schemathesis/schemathesis/issues/638) - `--request-retries` to retry requests on network failures with exponential back-off. ### :wrench: Changed - Strip API version prefixes during dep | Medium | 3/22/2026 |
| v4.12.2 | ### :bug: Fixed - False positive `positive_data_acceptance` in the coverage phase for path parameters with `minLength` greater than 1. | Low | 3/19/2026 |
| v4.12.1 | ### :bug: Fixed - Malformed request body media types (e.g. `application.json` instead of `application/json`) now report a clean "Schema Error" with the location in the schema instead of a raw Python traceback. [#3615](https://github.com/schemathesis/schemathesis/issues/3615) | Low | 3/14/2026 |
| v4.12.0 | ### :rocket: Added - Docker images now ship with [TraceCov](https://docs.tracecov.sh) pre-installed and enabled by default, providing schema coverage reports out of the box. ### :bug: Fixed - False negative `missing_required_header` during the coverage phase when negative mode was active - the base test case used an invalid body, preventing header validation from being reached. - False positive `negative_data_rejection` for path parameters with `format: uuid` and a `pattern` constraint in th | Low | 3/11/2026 |
| v4.11.2 | ### :rocket: Added - `positive_data_acceptance` failures now include a hint when Hypothesis-generated extra properties are the likely cause of rejection, suggesting `additionalProperties: false`. [#3583](https://github.com/schemathesis/schemathesis/issues/3583) ### :bug: Fixed - Server-Sent Events: Parse `retry` field as integer. - `ensure_resource_availability` no longer fires a false positive when a successful DELETE on the same resource ID exists in a different root's subtree. [#3588](htt | Low | 3/6/2026 |
| v4.11.1 | ### :rocket: Added - Support filesystem paths for `SCHEMATHESIS_HOOKS`. ### :memo: Documentation - "Triaging Failures" guide. - "Docker" guide. ### :bug: Fixed - Server-Sent Events: Ignore invalid `retry` field values. - Server-Sent Events: Treat empty `event` fields as `message`. - `[auth.openapi.*]` HTTP Basic authentication now correctly applied when using WSGI or ASGI apps in pytest mode. [#3575](https://github.com/schemathesis/schemathesis/issues/3575) - Misconfigured `[auth.openapi.* | Low | 3/5/2026 |
| v4.11.0 | ### :rocket: Added - OpenAPI 3.2 support for the `QUERY` HTTP method. - OpenAPI 3.2 support for `in: querystring` parameters. - OpenAPI 3.2 support for Server-Sent Events (`text/event-stream`) response validation. [#3064](https://github.com/schemathesis/schemathesis/issues/3064) ### :bug: Fixed - Legacy Draft 4 exclusive bounds in OpenAPI 3.1 schemas (`exclusiveMinimum: true`) no longer silently ignored. - Coverage-phase crash: `ValueError: Unsupported type: 'Binary'` when a `oneOf`/`anyOf` | Low | 3/2/2026 |
| v4.10.2 | ### :bug: Fixed - Connection failures when using multiple workers on free-threaded Python due to shared `requests.Session` across threads. [#3507](https://github.com/schemathesis/schemathesis/issues/3507) | Low | 2/14/2026 |
| v4.10.1 | ### :bug: Fixed - False positive `negative_data_rejection` for integer/number path parameters in the coverage phase. | Low | 2/10/2026 |
| v4.10.0 | ### :rocket: Added - Negative testing support for GraphQL schemas. [#2355](https://github.com/schemathesis/schemathesis/issues/2355) ### :bug: Fixed - Misleading "Schema Error: Cannot generate test data" when `filter_case` hook rejects all generated cases. Now reports "Hook Error" with actionable guidance. - Incorrect HTTP method shown in failure headers during coverage phase. [#3322](https://github.com/schemathesis/schemathesis/issues/3322) - False positive `negative_data_rejection` | Low | 2/7/2026 |
| v4.9.5 | ### :bug: Fixed - Examples phase crashing with `RecursionError` when multiple `allOf` items reference the same schema target. - Coverage phase generating "missing Content-Type header" test with Content-Type header present due to automatic JSON body serialization. | Low | 1/29/2026 |
| v4.9.4 | ### :rocket: Added - Detect foreign key fields (e.g., `customer_id`, `order_ids`) in responses and request bodies to generate additional stateful links. ### :bug: Fixed - `map_*`, `filter_*`, and `flatmap_*` hooks receiving `GeneratedValue` wrappers instead of raw `dict` values in negative generation mode. [#3471](https://github.com/schemathesis/schemathesis/issues/3471) | Low | 1/21/2026 |
| v4.9.3 | ### :bug: Fixed - Coverage phase hanging on endpoints with many optional parameters due to combinatorial explosion. [#3046](https://github.com/schemathesis/schemathesis/issues/3046) - Coverage phase crashing with `TypeError` when schema contains non-string `pattern` values (e.g., `"pattern": 0.0`). | Low | 1/20/2026 |
| v4.9.2 | ### :bug: Fixed - Custom media type strategies registered via `schemathesis.openapi.media_type()` not being applied in the coverage phase when using `@schema.parametrize()`. [#3345](https://github.com/schemathesis/schemathesis/issues/3345) - `positive_data_acceptance` failures caused by RFC-invalid control characters in generated headers. Positive mode now generates only valid ASCII headers. [#3462](https://github.com/schemathesis/schemathesis/issues/3462) | Low | 1/18/2026 |
| v4.9.1 | ### :bug: Fixed - Duplicate entries in `required` array causing "non-unique elements" schema validation errors. [#3460](https://github.com/schemathesis/schemathesis/issues/3460) | Low | 1/15/2026 |
| v4.9.0 | ### :rocket: Added - OpenAPI 3.2 support. - Mix schema examples into fuzzing and stateful data generation. - Syntax-level fuzzing for JSON bodies in negative mode. - NDJSON report format (`--report=ndjson`) for exporting all engine events as newline-delimited JSON. - Capture primitive identifiers from POST/PUT responses returning bare strings or integers. - Normalize schema names by stripping common suffixes (`-Output`, `-Input`, `Out`, `In`, `DTO`). - Discover sub-resources inside arra | Low | 1/13/2026 |
| v4.8.0 | This release focuses on test budget efficiency - getting more value from every test case. Benchmarks against a real-world API show success rate improved from **5%** to **48%**, meaning tests now reach deeper application logic instead of being rejected at input validation. ### :rocket: Added - Automatic success rate targeting to guide test generation toward valid API inputs. - Translate PCRE Unicode property escapes (`\p{L}`, `\p{N}`) to Python regex equivalents. - Propagate shared pat | Low | 1/5/2026 |
| v4.7.9 | ### :rocket: Added - Pre-populate resource pool with values from response examples. ### :bug: Fixed - Unhandled `PermissionError` when loading config file. | Low | 12/30/2025 |
| v4.7.8 | ### :wrench: Changed - Improve dependency inference. ### :bug: Fixed - Resources not captured when server returns different 2xx status code than documented. | Low | 12/28/2025 |
| v4.7.7 | ### :memo: Documentation - Fix code snippets config. | Low | 12/21/2025 |
| v4.7.6 | ### :wrench: Changed - Hook runtime errors are now reported as "Hook Error" instead of misleading "Schema Error". | Low | 12/17/2025 |
| v4.7.5 | ### :bug: Fixed - Crash when parameter has non-string `in` value. - Crash during examples phase when schema contains non-string `pattern` value. - Examples phase generating invalid data for schemas with `anyOf`/`oneOf` containing only `required` constraints. [#3404](https://github.com/schemathesis/schemathesis/issues/3404) - Fuzzing phase failing with `unhashable type: 'GeneratedValue'` for `multipart/form-data` with custom encoding. | Low | 12/10/2025 |
| v4.7.4 | ### :bug: Fixed - `Binary` class causing JSON serialization errors in Hypofuzz due to `dataclasses.asdict()` exposing raw bytes. | Low | 12/10/2025 |
| v4.7.3 | ### :wrench: Changed - Add `409` to `positive_data_acceptance` check's default expected statuses to handle conflict responses (e.g., duplicate entries). ### :bug: Fixed - Response schema validation errors displaying internal bundled refs like `#/x-bundled/schema1` instead of original refs like `#/components/schemas/Item`. | Low | 12/8/2025 |
| v4.7.2 | ### :bug: Fixed - Hypothesis `database` setting from `@settings` decorator or loaded profiles being ignored with `@schema.parametrize()`. - `AttributeError` when using Schemathesis with HypoFuzz due to uninitialized `_override` attribute in `ProjectsConfig`. ### :wrench: Changed - Bump `pytest-subtest` from <0.15.0 to <0.16.0. | Low | 12/8/2025 |
| v4.7.1 | ### :bug: Fixed - Stateful testing crash on links with nested `$ref`s. [#3394](https://github.com/schemathesis/schemathesis/issues/3394) | Low | 12/7/2025 |
| v4.7.0 | ### :rocket: Added - Coverage phase now generates test cases for `additionalProperties` with schema constraints. - Coverage phase now generates test cases for `maxProperties` and `minProperties` constraints. - Coverage phase now generates negative test cases for `prefixItems`. ### :bug: Fixed - Coverage phase crash when `anyOf`/`oneOf` contains array schemas with tuple validation (`items` as list). - Missing type violations in coverage phase for properties with `format: binary` in JS | Low | 12/6/2025 |
| v4.6.8 | ### :bug: Fixed - False positive in `negative_data_rejection` check when numeric strings are generated for integer/number path parameters. | Low | 12/4/2025 |
| v4.6.7 | ### :wrench: Changed - Use Python 3.14 in Docker images. - Unsupported regex patterns are now removed with a warning instead of causing schema errors. | Low | 12/4/2025 |
| v4.6.6 | ### :bug: Fixed - False positive in `negative_data_rejection` check when multiple mutations conflict. [#3367](https://github.com/schemathesis/schemathesis/issues/3367) - Path parameters containing `/` in their string representation routing requests to the wrong endpoints during negative testing. | Low | 12/3/2025 |
| v4.6.5 | ### :bug: Fixed - False positive in `negative_data_rejection` check when single-element arrays serialize to scalar values for query/header/cookie parameters. [#3375](https://github.com/schemathesis/schemathesis/pull/3375) - Support `$ref` inside link definitions in Open API 3.1. [#3377](https://github.com/schemathesis/schemathesis/pull/3377) | Low | 12/3/2025 |
| v4.6.4 | ### :bug: Fixed - Crash when handling connection errors with empty header values. - Fuzzing phase not generating format-violating values for custom string formats. - Fuzzing phase generating non-byte values for custom binary media types like `application/x-tar`. | Low | 11/28/2025 |
| v4.6.3 | ### :bug: Fixed - Incomplete fix for false positive `API rejected schema-compliant request` for `application/x-www-form-urlencoded` with required body. [#3360](https://github.com/schemathesis/schemathesis/issues/3360) - Fuzzing phase not generating format-violating values for `format` constraints like `uuid`. | Low | 11/26/2025 |
| v4.6.2 | ### :bug: Fixed - Nested external `$ref` links in multi-file schemas not resolving relative to their containing file. [#3361](https://github.com/schemathesis/schemathesis/issues/3361) - False positive `API rejected schema-compliant request` for `application/x-www-form-urlencoded` with required body but no required properties. [#3360](https://github.com/schemathesis/schemathesis/issues/3360) | Low | 11/25/2025 |
| v4.6.1 | ### :bug: Fixed - False positive in `negative_data_rejection` check for `text/plain` and `application/octet-stream` endpoints where non-string values become valid after serialization. - Authentication headers are no longer reapplied during negative testing when intentionally removed. [#3355](https://github.com/schemathesis/schemathesis/issues/3355) | Low | 11/23/2025 |
| v4.6.0 | :tada: This release extends dependency analysis to non-stateful phases, so they can reuse successful API responses too. ### :rocket: Added - **CLI**: Dependency-based operation ordering in non-stateful tests. - **CLI**: Capture and reuse of successful API responses in the fuzzing phase. ### :racing_car: Performance - Response deserialization caching for YAML and custom formats. ### :bug: Fixed - **pytest**: Reproduction code is no longer shown for configuration errors like mis | Low | 11/22/2025 |
| v4.5.4 | ### :bug: Fixed - Invalid examples extracted from schemas with `allOf` and required fields. [#3333](https://github.com/schemathesis/schemathesis/issues/3333) | Low | 11/18/2025 |
| v4.5.3 | ### :bug: Fixed - Positive test cases generated without required body. [#3327](https://github.com/schemathesis/schemathesis/issues/3327) - Incorrect error message when combining `@schema.given()` with schema examples. [#3328](https://github.com/schemathesis/schemathesis/issues/3328) | Low | 11/17/2025 |
| v4.5.2 | ### :bug: Fixed - `schema.given` now works with fixture-backed schemas regardless of decorator order. [#3320](https://github.com/schemathesis/schemathesis/discussions/3320) - Internal error during OpenAPI operation lookup caused by vendor-specific string entries under `paths`. | Low | 11/15/2025 |
| v4.5.1 | ### :bug: Fixed - Fix propagation of `base_url` / session data from `Case.call_and_validate`, so auth checks no longer crash when schemas are loaded from files. [#3318](https://github.com/schemathesis/schemathesis/issues/3318) | Low | 11/12/2025 |
| v4.5.0 | ### :rocket: Added - Support for `pytest>=9.0`. [#3312](https://github.com/schemathesis/schemathesis/issues/3312) - OpenAPI-aware authentication via `[auth.openapi.<scheme>]` config. Automatically aligns with schema security definitions and warns about unused/misnamed schemes. [#1710](https://github.com/schemathesis/schemathesis/issues/1710) - Display configuration file path in CLI output ### :racing_car: Performance - ~20% faster VCR cassette recording through output sanitization ca | Low | 11/12/2025 |
| v4.4.4 | ### :racing_car: Performance - 10-50% faster schema iteration through parameter bundling cache. ### :wrench: Changed - Replace archived `backoff` with `tenacity`. [#3286](https://github.com/schemathesis/schemathesis/issues/3286) - Suppress Hypothesis reproduction blocks in Schemathesis pytest failures. - Improved `missing_required_header` check message to show which header was missing instead of repeating the check title. ### :memo: Documentation - Improved checks reference docu | Low | 11/8/2025 |
| v4.4.3 | ### :bug: Fixed - Empty `$ref` references in schemas are now gracefully skipped during bundling. - Custom format strategies returning binary data. [#3289](https://github.com/schemathesis/schemathesis/issues/3289) - Duplicate exception reporting in pytest output. | Low | 11/7/2025 |