Using oasdiff to Detect Breaking Changes in APIs Posted in Design J Simpson March 4, 2024 API versioning is increasingly common, and not just for massively popular or commercial APIs. Even hobby and special interest APIs like Sportradar feature multiple versions. This can create problems for developers and users alike. oasdiff: A command-line and Go package to compare and detect breaking changes in OpenAPI specs. Versioning has a tendency to break things. Existing code might not get updated along with the API. oasdiff, an open-source tool for detecting changes in OpenAPI specifications, was created to prevent drift from causing issues for your developers and customers. oasdiff can be run as either a Golang package or a command-line utility. The tool compares OpenAPI specifications, usually in either JSON or YAML format, and returns a report highlighting the differences. oasdiff analyzes everything from endpoints to request/response parameters for updates and revisions, particularly looking for changes that can lead to breaking integrations. It’s an invaluable tool for CI/CD pipelines. Below, we’ll show you how to get started with oasdiff to prevent any breakage or service outages when you update your API. 1. Install oasdiff Numerous options for installing oasdiff are mentioned in the oasdiff repository, ranging from macOS, Windows, and Linux installers to loading with Go. For this tutorial, we’ll use the Go method, as the Go package allows easy integration into a CI/CD pipeline. First, you’ll need to install Go if you haven’t already. Once Go is installed, input the following command into your Terminal. go install github.com/tufin/oasdiff@latest Note that if you’ve just installed Go and you get an error when you run that command, open a fresh instance of Terminal, which should fix the problem. If you’re using macOS, you can also install oasdiff using Brew with the following command: brew tap tufin/homebrew-tufin brew install oasdiff oasdiff Wrappers oasdiff has several wrappers, too, if you want to try it without installing anything. GitHub Action Cloud Service OpenAPI Sync 2. Try oasdiff Once oasdiff is installed, you can test it to see how it works. First, make sure you’ve cloned the GitHub Repository. Once you have, you can run oasdiff diff to see the difference between two local YAML files. Input the following command. oasdiff diff data/openapi-test1.yaml data/openapi-test2.yaml When you do, you should get the following output. info: contact: added: true version: from: 1.0.0 to: 1.0.1 paths: deleted: - /subscribe - /api/{domain}/{project}/install-command - /register modified: /api/{domain}/{project}/badges/security-score: operations: added: - POST modified: GET: tags: deleted: - security operationID: from: GetSecurityScores to: "" parameters: deleted: cookie: - test header: - user - X-Auth-Name modified: path: domain: schema: type: from: string to: integer format: from: hyphen-separated list to: non-negative integer description: from: Hyphen-separated list of lowercase string to: Non-negative integers (including zero) example: from: generic-bank to: "100" min: from: null to: 7 pattern: from: ^(?:([a-z]+-)*([a-z]+)?)$ to: ^(?:\d+)$ query: filter: content: mediaTypeModified: application/json: schema: properties: modified: color: type: from: string to: number image: explode: from: null to: true schema: description: from: alphanumeric to: alphanumeric with underscore, dash, period, slash and colon examples: deleted: - "0" token: schema: anyOf: added: - RevisionSchema[0] - RevisionSchema[1] type: from: string to: "" format: from: uuid to: "" description: from: RFC 4122 UUID to: "" example: from: 26734565-dbcc-449a-a370-0beaaf04b0e8 to: null maxLength: from: 29 to: null pattern: from: ^(?:[0-9a-f]{8}-[0-9a-f]{4}-[0-5][0-9a-f]{3}-[089ab][0-9a-f]{3}-[0-9a-f]{12})$ to: "" responses: added: - default deleted: - "200" - "201" - "400" parameters: deleted: path: - domain endpoints: added: - method: POST path: /api/{domain}/{project}/badges/security-score deleted: - method: POST path: /register - method: POST path: /subscribe - method: GET path: /api/{domain}/{project}/install-command modified: ? method: GET path: /api/{domain}/{project}/badges/security-score : tags: deleted: - security operationID: from: GetSecurityScores to: "" parameters: deleted: cookie: - test header: - user - X-Auth-Name modified: path: domain: schema: type: from: string to: integer format: from: hyphen-separated list to: non-negative integer description: from: Hyphen-separated list of lowercase string to: Non-negative integers (including zero) example: from: generic-bank to: "100" min: from: null to: 7 pattern: from: ^(?:([a-z]+-)*([a-z]+)?)$ to: ^(?:\d+)$ query: filter: content: mediaTypeModified: application/json: schema: properties: modified: color: type: from: string to: number image: explode: from: null to: true schema: description: from: alphanumeric to: alphanumeric with underscore, dash, period, slash and colon examples: deleted: - "0" token: schema: anyOf: added: - RevisionSchema[0] - RevisionSchema[1] type: from: string to: "" format: from: uuid to: "" description: from: RFC 4122 UUID to: "" example: from: 26734565-dbcc-449a-a370-0beaaf04b0e8 to: null maxLength: from: 29 to: null pattern: from: ^(?:[0-9a-f]{8}-[0-9a-f]{4}-[0-5][0-9a-f]{3}-[089ab][0-9a-f]{3}-[0-9a-f]{12})$ to: "" responses: added: - default deleted: - "200" - "201" - "400" security: deleted: - bearerAuth servers: deleted: - tufin.com tags: deleted: - security - reuven externalDocs: deleted: true components: schemas: deleted: - network-policies - rules parameters: deleted: - network-policies headers: deleted: - testc - new - test requestBodies: deleted: - reuven responses: deleted: - OK securitySchemes: deleted: - AccessToken - OAuth - bearerAuth As you can see, oasdiff’s output goes into incredible detail about all of the revisions that have taken place between the two versions. For instance, the following endpoints have been deleted in this example: /subscribe, /api/{domain}/{project}/install-command, and /register. Under the modified section, you can see that the /api/{domain}/{project}/badges/security-score: endpoint has added a POST command. A number of features have been deprecated, as well, which are also detailed. You can also return the results as HTML if you want a neater list. Instead of using the -f text command at the end of the string, use -f html instead, like so. oasdiff diff data/openapi-test1.yaml data/openapi-test2.yaml -f html Oasdiff doesn’t only work on local files, either. You can just as easily see the difference between remote APIs using HTTP/s. Start by inputting the following: oasdiff diff https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test1.yaml https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test3.yaml -f text The results show that four endpoints have been modified: security-score, install-command, register, and subscribe. If you want to see any breaking changes between the two versions, try the following: oasdiff breaking https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test1.yaml https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test3.yaml These results show that exchanging the success status for a 200 or 201 in the security-score endpoint results in breaking. There’s even a dedicated command for assessing endpoints with /API in the path. oasdiff diff https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test1.yaml https://raw.githubusercontent.com/Tufin/oasdiff/main/data/openapi-test3.yaml -f text -p "/api" You can exclude endpoints, as well. You can filter out path names using the –match-path command to filter out paths that don’t match a particular expression. You can also filter out specific extensions using the –filter-extension command. 3. Integrate oasdiff Into Go Projects One of the biggest reasons to use oasdiff is integration into an automated workflow. If you’re developing in Go, you can use oasdiff directly inside your code. Simply use the following command: diff.Get(&diff.Config{}, spec1, spec2) Here’s an example of oasdiff in a Go program. loader := openapi3.NewLoader() loader.IsExternalRefsAllowed = true s1, err := loader.LoadFromFile("../data/simple1.yaml") if err != nil { fmt.Fprintf(os.Stderr, "failed to load spec with %v", err) return } s2, err := loader.LoadFromFile("../data/simple2.yaml") if err != nil { fmt.Fprintf(os.Stderr, "failed to load spec with %v", err) return } diffReport, err := diff.Get(diff.NewConfig(), s1, s2) if err != nil { fmt.Fprintf(os.Stderr, "diff failed with %v", err) return } bytes, err := yaml.Marshal(diffReport) if err != nil { fmt.Fprintf(os.Stderr, "failed to marshal result with %v", err) return } fmt.Printf("%s\n", bytes) Which returns the following output: aths: modified: /api/test: operations: added: - POST deleted: - GET endpoints: added: - method: POST path: /api/test deleted: - method: GET path: /api/test `` You can also use oasdiff to detect breaking changes inside of your code. package main import ( "fmt" "os" "strings" "github.com/getkin/kin-openapi/openapi3" "github.com/tufin/oasdiff/checker" "github.com/tufin/oasdiff/diff" "github.com/tufin/oasdiff/load" ) func main() { loader := openapi3.NewLoader() loader.IsExternalRefsAllowed = true s1, err := load.LoadSpecInfo(loader, load.NewSource("../data/openapi-test1.yaml")) if err != nil { fmt.Fprintf(os.Stderr, "failed to load spec with %v", err) return } s2, err := load.LoadSpecInfo(loader, load.NewSource("../data/openapi-test3.yaml")) if err != nil { fmt.Fprintf(os.Stderr, "failed to load spec with %v", err) return } diffConfig := diff.NewConfig().WithCheckBreaking() diffRes, operationsSources, err := diff.GetPathsDiff(diffConfig, []*load.SpecInfo{s1}, []*load.SpecInfo{s2}, ) if err != nil { fmt.Fprintf(os.Stderr, "diff failed with %v", err) return } errs := checker.CheckBackwardCompatibility(checker.GetDefaultChecks(), diffRes, operationsSources) // process configuration file for ignoring errors errs, err = checker.ProcessIgnoredBackwardCompatibilityErrors(checker.ERR, errs, "../data/ignore-err-example.txt", checker.NewDefaultLocalizer()) if err != nil { fmt.Fprintf(os.Stderr, "ignore errors failed with %v", err) return } // process configuration file for ignoring warnings errs, err = checker.ProcessIgnoredBackwardCompatibilityErrors(checker.WARN, errs, "../data/ignore-warn-example.txt", checker.NewDefaultLocalizer()) if err != nil { fmt.Fprintf(os.Stderr, "ignore warnings failed with %v", err) return } // pretty print breaking changes errors if len(errs) > 0 { localizer := checker.NewDefaultLocalizer() count := errs.GetLevelCount() fmt.Print(localizer("total-errors", len(errs), count[checker.ERR], "error", count[checker.WARN], "warning")) for _, bcerr := range errs { fmt.Printf("%s\n\n", strings.TrimRight(bcerr.SingleLineError(localizer, checker.ColorNever), " ")) } } } Which returns: 4 breaking changes: 1 error, 3 warning error at ../data/openapi-test3.yaml, in API GET /api/{domain}/{project}/badges/security-score removed the success response with the status '201' [response-success-status-removed]. warning at ../data/openapi-test3.yaml, in API GET /api/{domain}/{project}/badges/security-score deleted the 'cookie' request parameter 'test' [request-parameter-removed]. warning at ../data/openapi-test3.yaml, in API GET /api/{domain}/{project}/badges/security-score deleted the 'header' request parameter 'user' [request-parameter-removed]. warning at ../data/openapi-test3.yaml, in API GET /api/{domain}/{project}/badges/security-score deleted the 'query' request parameter 'filter' [request-parameter-removed]. Also read: Everything You Need to Know About API Versioning Final Thoughts on oasdiff API versioning will only become increasingly common the longer APIs fuel the software industry. Telling the difference between two versions is just one of the potential uses of oasdiff, anyway. The ability to detect breaking changes, especially before they happen, is even more of a reason to integrate oasdiff into your workflow. A few lines of code can prevent service outages and unexpected downtime for you and your customers. For more context, watch oasdiff contributors Reuven Harrison and Bianca Lisle present at Platform Summit 2023. The latest API insights straight to your inbox
