Introducing Core Libraries in SDKs

Core libraries improve your SDK code quality, reduce code size, and provide better test coverage. New SDKs contain only the code specific to the API for which the SDK was generated

We’re APIMatic.io. We generate strongly typed SDKs along with a complete API reference from OpenAPI definitions. If you’re looking to create an SDK for your REST API, check out how to generate one in under a minute. This post covers a major enhancement to the design of our generated SDKs, aligned with our mission to provide zero-maintenance SDKs to API providers.

In this release, we have majorly revamped SDKs to improve the code quality, reduce code size, and provide better test coverage than ever. At the center of these changes are the new Core Libraries which now help power the SDKs and provide many new quality of life improvements for SDK users. Despite these improvements, we have taken care to make sure that the SDKs have no breaking changes with this release.

Why did we redesign our SDKs?

Every SDK that we generated in the past contained some common utility code. This utility code which we often call the SDK Internals provided functionality like the ability to create HTTP requests, handle responses, facilitate authentication, convert API responses back to class instances, and validate user and server data.

So even though the most interesting part of the SDK is the endpoint methods and the models (a.k.a. data classes) which are part of the SDK’s Interface, these work by building on top of the SDK Internals which do all the heavy lifting needed to make the SDKs work.

Not only do SDK Internals themselves contain significant logic for making API calls happen, they also pull in dependencies to help power the SDK functionality. For example, they can pull in an HTTP client library to help make the HTTP calls for the API, or perhaps a JSON serializer library to help convert Data Objects to and from JSON.

This makes SDK Internals a crucial part of the SDKs on which the rest of the SDK’s work depends. To ensure that the SDK keeps working, we need to provide updates for the SDK Internals, the kind of updates that fix bugs, patch vulnerabilities and provide other improvements to the SDK.

However, that’s where we run into a limitation with our previous design.

The SDK Internals were part of the SDKs. This meant that fixing them required generating a new SDK as well. Even when an API provider did that, this would only get them an updated SDK for the latest version of their API. Meanwhile, all the old SDK versions that are still used by the API users contain SDK Internals that might need to be updated for vulnerabilities or bug fixes.

Generating new SDKs every time a small bug fix or dependency update needs to happen multiplied for every SDK version is definitely impractical for API providers! So we set about to solve this problem with a redesign.

What are Core Libraries?

That is where Core libraries enter the game.

The solution to our code duplication problem was simple. We redesigned the SDKs so that all SDK Internals that were previously part of the SDK itself are now externalized into a library, the Core library, which is installed into the SDK just like any dependency (more on that later).

This means most of the code inside the SDKs now consists of code that is specific to the API. This usually consists of the endpoint methods and data models that are generated by the APIMatic Code Generator using the OpenAPI definition. The SDK Internals was code that stayed mostly unchanged from SDK to SDK and is no longer part of the SDK code base.

This dramatically reduces the amount of code that is present in the generated SDKs. This helps us provide even better care of the generated SDKs by taking charge of when and how Core libraries are updated. Let's see how that happens.

Core Libraries keep your SDKs fresh

With SDK Internals code out of the SDK and into a Core Library, you must be wondering who is responsible for maintaining the Core Library and updating it. APIMatic’s team, of course!

All SDKs use the latest Core Libraries by installing them via the package repository. A package repository is a public database where library authors publish their libraries for anyone to use. For example, npm for Node.js or Pypi for Python are the de facto package repositories for their respective languages. APIMatic maintains the Core Libraries and publishes new versions of them to the package repositories whenever there is an update. These updates then make it into the SDKs by virtue of the SDKs depending on the latest version of the Core Library.

This solves a big problem in the maintainability of the SDKs: how to update both new and old SDKs.

Whenever APIMatic fixes a vulnerable dependency or makes some performance improvement, these automatically make it into all the SDKs without the need for anyone to generate a new SDK from our Code Generator. This means that we are able to push updates and fixes faster than ever before without any additional effort from the API providers.

Core libraries help deliver updates to the SDK

Let’s dive a little deeper into the kind of updates Core libraries receive:

1. New features: These include new functionality in the SDK such as API call retries or HTTP caching. Often functionality like this also requires a new SDK to be generated.
2. Improvements: Smaller changes like improved error messages were made available to the users without the need to update their SDKs. Improvements to HTTP calls, JSON serialization, and validation happen frequently.
3. Language updates: Most languages see new version releases every year. To help make sure that your SDK continues to work on a new language runtime, we provide compatibility updates to our Core libraries as well as our SDKs when needed.
4. Bug fixes: These include patches to issues reported by our customers.
5. Dependency updates: Libraries used by the Core library to make HTTP requests or serialize JSON see frequent updates. These updates can help introduce new features or performance improvements, for example, updates to the HTTP client to introduce HTTP/3 support.
6. 6. Vulnerability fixes: Third-party dependencies require patches from time to time. We track such vulnerabilities using a vulnerability scanner and alert tool and provide updates when available.

Core Libraries improve SDK maintainability

Core libraries help push updates and improvements to old and new SDK versions automatically, improving the maintainability of the SDKs. However, while working on Core Libraries, we used this opportunity to improve the code that goes inside the SDK as well.

As we designed the Core libraries, we thought about what a clean interface between the SDK and the Core library would look like. For us, a clean interface meant that the SDK contained only code that was necessary for the SDK to function and specific to the API for which the SDK was generated. All other code should be delegated to the Core libraries which should provide the implementation details for the SDK.

With this goal in mind, we were able to majorly revamp our implementation code for API calls. Checkout a sample of what an endpoint method (the method that makes an API call) looks like in a generated Java SDK before and after our redesign:

That’s almost half the code! This coupled with redesigning the SDKs so that most of it consists of declarative code that is implemented using Core Libraries has had a positive effect on the testability of the SDK as well.

These test suites are part of the Core Libraries’ codebases and can be viewed on GitHub. For example, checkout the test coverage numbers in PHP’s Core library:

Core Libraries provide an Open-Source core for the SDKs

Having the source for the Core libraries published on GitHub allows us to engage with a wider community of developers to gather bug reports and to get pull requests for fixes and new features. It also provides visibility to the public into the test suites that we run to maintain our projects, something that was not previously available outside of APIMatic. That means the SDK users can now refer back to the code that powers their SDK and submit improvements for it.

We follow all the best practices expected of an Open Source library such as:

• Continuous testing via our test suites and GitHub Actions

• Checking code quality through linting and static analysis checks

• Tracking maintainability through Code Climate

• Vulnerability checks and alerts via Dependabot

• Open collaboration via a permissive MIT license

We are also excited about the possibilities of having alternative implementations for some of the functionalities provided by the Core library, thanks to the modularization of the SDK through Core libraries and the fact that this code is now open source.

This creates opportunities for API providers and SDK users to customize SDKs for their specific needs to meet their business requirements. Think HTTP clients that integrate neatly with their web framework of choice or serialization libraries that integrate directly with the Dependency Injection framework they use.

Summary

This release represents one of the biggest refactoring of the SDKs generated by APIMatic. We have redesigned the SDKs for all of the currently supported languages in our Code Generator. This means when you generate the SDKs now, they will be using the new design and will depend on the Core Libraries to provide their functionality.

Core libraries have not only improved the SDK quality story but will help us iterate even faster in the future. If you haven’t tried out APIMatic’s Code Generator, do try it out and let us know what you think.