Introduction

This project is an attempt at providing API Validation for any service that uses an OpenAPI specification. Specifically, this project is designed to proxy requests and responses and verify that they conform to your API specification.

How it works

openapi-validator-proxy works by creating a server that listens for requests and then sends them to an upstream server. For each request it receives, it validates the request and response and then saves the result as a testcase. Each testcase includes information about the path, method, parameters, and response code. If the response fails validation, the testcase will include an error type and message that describes the failure.

High Level Goals

API Validation MUST be agnostic to the clients and servers.
API Validation MUST NOT interfere with existing workflows for building APIs.
API Validation SHOULD be easy!
- It MUST be "one thing". You run one extra command.
- It MUST be invoked locally the same way as in CI.

This tool accomplishes these goals by providing the following:

It is implemented as a proxy that does not interfere with the request or response. This makes the tool usable by any client or server combination.
Any tests that you are currently running can be run through this proxy, effectively gaining API validation for free.
The proxy is built as a single binary compiled for your platform. This makes it easy to run locally or in CI.
The proxy generates a report that can be used as a CI artifact.

Install

GitHub Releases

Check the latest release on the releases page!

Quickstart

A CLI application to validate OpenAPI specification requests and responses.

Usage: openapi-validator-proxy <COMMAND>

Commands:
  proxy  Starts the proxy server with the given file as input
  help   Print this message or the help of the given subcommand(s)

Options:
  -h, --help     Print help
  -V, --version  Print version

The main subcommand is the proxy command. This starts the proxy server that will validate requests and responses against the OpenAPI specification. The command takes two arguments: the path to the OpenAPI file and the URL of the upstream server. Here's an example:

openapi-validator-proxy proxy petstore.yaml http://localhost:8080

This will start the proxy server and read the OpenAPI file petstore.yaml. It will then proxy requests to http://localhost:8080. If you have a server mounted at a different path, you can include that in the URL. For example, if your server is mounted at /api/v1 you can run:

openapi-validator-proxy proxy petstore.yaml http://localhost:8080/api/v1

Then make a GET request to the pets collection:

GET http://localhost:3000/api/v1/pets

You can see the testcase created for this request by making an additional request to the JUnit report endpoint:

GET http://localhost:3000/_ovp/junit

Which should return a JUnit report that resembles the following:

<testsuites tests="1" failures="0">
    <testcase name="e73ac0a9-a28e-446c-aa21-aaad827a489d" time="0.26">
        <system-out>
[[PROPERTY|correlationId=e73ac0a9-a28e-446c-aa21-aaad827a489d]]
[[PROPERTY|method=GET]]
[[PROPERTY|operationId=getPets]]
[[PROPERTY|path=/pets]]
[[PROPERTY|responseContentType=application/json]]
[[PROPERTY|statusCode=200]]
        </system-out>
    </testcase>
</testsuites>

The JUnit report is currently the only report but others will be added in the future.

Validation Failures

name	docs
InvalidHTTPMethod	The HTTP method used in the request is not one of the expected values: DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT, or TRACE.
InvalidStatusCode	The status code returned by the upstream server does not have a matching response in the OpenAPI spec.
MissingResponseDefinition	The OpenAPI spec contained a missing inline response definition or referenced a response that did not exist.
MissingSchemaDefinition	The requested path was not found in the OpenAPI spec. This response was not validated and may be missing relevant testcase properties.The OpenAPI spec contained a missing inline schema definition or referenced a schema that did not exist.
PathNotFound	The requested path was not found in the OpenAPI spec.
RequestFailedJSONDeserialization	The request body could not be deserialized as JSON.
RequestFailedValidationUnexpectedBoolean	The request body contained a boolean value when the OpenAPI spec expected a different type.
RequestFailedValidationUnexpectedNull	The request body contains a null value when the OpenAPI spec did not allow null values.
RequestFailedValidationUnexpectedNumber	The request body contained a number value when the OpenAPI spec expected a different type.
RequestFailedValidationUnexpectedProperty	The request body contained a property that was not defined in the OpenAPI spec.
RequestFailedValidationUnexpectedString	The request body contained a string value when the OpenAPI spec expected a different type.
RequestFailedValidationUnsupportedSchemaKind	The OpenAPI spec contained a schema with an unsupported kind, such as anyOf, oneOf, or not.
RequestMismatchNonEmptyBody	The client included a non-empty body when the OpenAPI spec expected an empty body.
RequestMismatchedContentTypeHeader	The client included a Content-Type header in the request that does not match any content types defined in the OpenAPI spec.
RequestMissingContentTypeHeader	The client did not include a Content-Type header in the request. This is only an issue when the response body is not empty.
ResponseFailedJSONDeserialization	The response body could not be deserialized as JSON.
ResponseFailedValidationUnexpectedBoolean	The response body contained a boolean value when the OpenAPI spec expected a different type.
ResponseFailedValidationUnexpectedNull	The response body contains a null value when the OpenAPI spec did not allow null values.
ResponseFailedValidationUnexpectedNumber	The response body contained a number value when the OpenAPI spec expected a different type.
ResponseFailedValidationUnexpectedProperty	The response body contained a property that was not defined in the OpenAPI spec.
ResponseFailedValidationUnexpectedString	The response body contained a string value when the OpenAPI spec expected a different type.
ResponseFailedValidationUnsupportedSchemaKind	The OpenAPI spec contained a schema with an unsupported kind, such as anyOf, oneOf, or not.
ResponseMismatchNonEmptyBody	The upstream server included a non-empty response body when the OpenAPI spec expected an empty body.
ResponseMismatchedContentTypeHeader	The upstream server included a Content-Type header in the response that does not match any content types defined in the OpenAPI spec.
ResponseMissingContentTypeHeader	The upstream server did not include a Content-Type header in the response. This is only an issue when the response body is not empty.

Custom Headers

openapi-validator-proxy will forward all headers it recieves from the client to the upstream server. However, there are a couple headers that have special meaning to the proxy.

OVP-Correlation-Id

This header controls the name of the testcase when it is captured in the report. Specifying this header is completely optional. If it is not specified, the proxy will generate a UUID v4 to use as the testcase name. This can be useful if you want requests to be correlated with a specific name or identifier.

Example: Setting OVP-Correlation-Id

GET http://localhost:3000/pets
OVP-Correlation-Id: get-pets

OVP-Fused-Correlation-Headers

This header allows you to specify additional headers which should be set with the same value as the OVP-Correlation-Id. This is useful if you want to specify a list of headers instead of headers with redundant values, OR if you want to rely on the uuid generated by the proxy.

Example: Setting OVP-Correlation-Id and OVP-Fused-Correlation-Headers

GET http://localhost:3000/pets
OVP-Correlation-Id: get-pets
OVP-Fused-Correlation-Headers: X-Request-Id, X-Traceid

This is equivalent to:

GET http://localhost:3000/pets
OVP-Correlation-Id: get-pets
X-Request-Id: get-pets
X-Traceid: get-pets

Example: Fusing headers with the generated UUID

GET http://localhost:3000/pets
OVP-Fused-Correlation-Headers: X-Request-Id, X-Traceid

When this request is received, the proxy will generate a UUID and set the headers for OVP-Correlation-Id, X-Request-Id, and X-Traceid to the same value.

Reports

Reports are generated while the proxy is running. The proxy does not persist these reports, you will need to make a request to download them before killing the proxy. Below you can see the list of reports that are available:

JUnit

JUnit

The JUnit report is a standard XML report that can be used with any CI/CD tool that supports JUnit reports. You can download this report by making a GET request to the proxy like this:

GET http://localhost:3000/_ovp/junit

If you're using curl you could save this output to a file like:

curl -o junit.xml http://localhost:3000/_ovp/junit

References:

Contributing

Testing

This project uses cargo-insta to create snapshots of the output to test against. Insta provides a tool that makes running these tests and reviewing their output easier. To install it run cargo install cargo-insta. Once this is installed, changes can be reviewed with cargo insta test --review.

If you're just trying to run the tests you can run cargo test.

Releasing

This project uses cargo-dist and cargo-release for the release process.

The release process looks like this:

Checkout master
Create commit that updates RELEASES.md with notes for the new release and push commit
Run cargo release patch (or minor or major) and verify the release looks correct
Run cargo release patch --execute --no-publish to create the tag and push it to GitHub
The GitHub Action should start immediately for the tag

If you are updating cargo-dist you should also run cargo dist init to capture changes to the action.

Building the Book

This project uses oranda (in conjunction with mdbook) to build a documentation site for the project. To start a development server run oranda dev and navigate to the page. Building the site for production is done within GitHub Actions and committed to the gh-pages branch.