From OpenAPI to source code with CI/CD

Introduction

In the era of Microservices, you are probably designing or consuming one or more Web Services. As a result, you might be aware of the importance of having a good definition of your web services, since that is what clients and developers will use to know how a web service can be used.

For that reason, you should have a clear and easy way to define and understand your Web API. In order to help you to make a good definition we have REST (Representational state transfer), which is a software architectural style that defines a set of rules to create Web Services. I'm not going to go into details about RESTful, because there is enough material for a full post and it is out of the scope of this post. But I might write another post about REST, since even nowadays I see some crazy stuff when it comes to Web APIs design.

Well, imagine we already have a really awesome definition for the resources of our Web API, and your clients are going to be really excited to use it, but...how do I let them know which are my endpoints, models, message errors, etc?

In order to provide a way to define our REST APIs, in 2011 Swagger created the first version for RESTful APIs specification. Over the subsequent years, Swagger specification has been evolving. In 2017, the Open APIInitiative, under the sponsorship of the Linux Foundation, published the OpenAPI Specification v3.0.0, which is the consequence of the evolution of RESTful API definition and it is considered the standard to define REST APIs nowadays.

In this article we will see the benefits of having an OpenAPI contract and all the possibilities that it can offer you, such as generating source code automatically form your definition and distributing it among your clients.

Generating your OpenAPI contract

If you are working with Web Services, I strongly recommend that you follow a contract-first approach. No matter if your consumers are external to your organisation, if you define the contract first, you will have several benefits:
  • Separation of concerns.
  • Avoid blocks between teams (frontend and backend).
  • Good and consistent communication between teams.
  • A clear definition of your API from the beginning. 
  • Easy documentation.
  • Consistent API models.
  • Reusability and code generation.
  • Easy versioning control.
In order to define your OpenAPI contract, you can use multiple tools. However, I would like to recommend two of them, one to work locally and another to work in the cloud.

Generating your contract from VS Code

If you prefer working offline, there is a nice extension for VS Code to edit your API definition called OpenAPI (Swagger) Editor (https://github.com/42Crunch/vscode-openapi).

By using this editor, you will have a GUI to edit your contract and you will have your contract edited in real time.


By using VS commands, you will have also a list of commands that can help you to add new elements into your OpenAPI contract:


Finally, one cool feature added recently is the security audit, which can generate a report from your contract, where all the security issues detected are reported for you:

Generating your contract on the cloud

If you prefer working on the cloud, the tool that I like the most is apicurio (https://studio.apicur.io), which is also open source https://github.com/Apicurio/apicurio-studio. With apicurio, you will be able to define and modify your OpenAPI contract from your favourite browser. In addition, you can invite collaborators and have a history with the changes that are made to your contract.


Apart from helping you with your API definition, apicurio can generate documentation from your OpenAPI contract, which can be especially interesting if you are working with third party clients or business people.


If you want to know more about tools for editing your OpenAPI contracts, I suggest having a look at this URL: https://openapi.tools/#gui-editors

Generating Code from your contract

Apart from having a clear definition of what your clients can expect from your API, OpenAPI contracts offer a lot of additional advantages. One advantage that I found extremely useful is generating code from your contract.

During the latest months, I've been working in a project based on React and at the beginning the client code to consume APIs was implemented manually. As a consequence, when there was change in a service, you had to find out what part of the code you had to change to update to a newer version of the contract.

To make things worse, the project was implemented in JavaScript. In other words, mistakes during manual updates of your API code could lead to run-time errors. Considering the fact that our client application was going to consume multiple services in constant evolution, the manual coding approach didn't seem a good option.

One of the best contribution that we had was adding Typescript support. By using Typescript (as I analysed in my previous post Benefits of using TypeScript in your project), one of the greatest advantages is having errors in compilation time. As a result, in spite of having manual updates for contracts, our changes were consistent through the application and real-time errors were reduced to a large extent.

With this scenario, another improvement that I introduced is generating our code from OpenAPI contract. By doing so, we are saving a large amount of time, as no manual changes in our code are required anymore. In addition, when we update a contract for any API, we will have feedback about compilation errors if we have breaking changes, which leads to a smooth and safer maintenance of our client code.

In order to generate your code from an OpenAPI contract, there are multiple alternatives. Particularly, we are using openapi-generator (https://github.com/OpenAPITools/openapi-generator), due to its multiple possibilities to be used by almost any existing framework or language.

I personally use openapi-generator as an npm package, so the first step is to install the package globally:
npm install @openapitools/openapi-generator-cli -g

Next, I can configure the generator to generate typescript client code with the following command:
openapi-generator generate -i MyOpenApiContract.json -g typescript-fetch

Distributing your API definition with CI/CD

At this point you already know how you can generate code from your OpenAPI contract. However, in order to use that code for multiple projects, as it is likely to happen for Web APIs, you might want to pack the generated code and publish it, so all your clients will be able to get the latest version by using your packages.

In order to make this process really easy, I came up with the idea of creating and publishing an NPM package from the source code generated by openapi-generator. Furthermore, I included CI/CD, so I will publish automatically a new version of my package each time I modify my contract.

In my case, I'm using Azure DevOps, so I have a repository where I store my contract and my build will be triggered always that a new change is detected. During the build process, I will execute some commands that I've defined in my file package.json:

{
    "name": "my-api",
    "version": "1.0.0",
    "private": false,
    "repository": {
        "type": "git"
},
    "scripts": {
        "api-gen": "openapi-generator generate -i myapi.contract.json -g typescript-fetch",
        "postapi-gen": "replace-in-file GlobalFetch WindowOrWorkerGlobalScope src/**",
        "tsc": "rimraf ./dist/ && tsc --project tsconfig.json",
        "posttsc": "copyfiles package.json dist"
    },
    "devDependencies": {
        "@openapitools/openapi-generator-cli": "^0.0.21-4.1.2",
        "replace-in-file": "^4.1.3",
        "rimraf": "^2.7.1",
        "tslint": "^5.20.0",
        "typescript": "^3.5.3",
        "copyfiles": "^2.1.0"
    }
}

If we have a look at my build pipeline, we can see the first two tasks, which are intended to read the version from the contract definition file and write it into my package.

In the third step, I will install all the packages that I need to execute the following steps.

Next, I generate my code by execute the command "npm run api-gen". As you can see in the package.json definition, this step is invoking the library openapi-generator and subsequently, it is doing a replacement. This replacement is done because we are working with a more recent version of Typescript.

Next, during the step "Transpile" I execute the command "npm run tsc", which transpiles the typescript code into javascript + ts definitions. Finally, I publish the results of the transpilation as a build artifact.



After a successful build, my release definition basically reads version of the package, concatenates the number of the build, and publishes the package as an NPM package.

Conclusions

In this article, we introduced OpenAPI and the importance of using contracts to define your Web APIs.

Subsequently, we have saw how you can generate your typescript client source code from an OpenAPI contract, which can be extremely useful and can save you time developing and maintaining that code manually.

Finally, we analysed the idea of publishing packages from your OpenAPI contract automatically, by using CI/CD. As a result, our clients will be able to get new versions of your Web API and consume it easily.

This process can be extended, since openapi-generator supports multiple languages and server code generation, so there are a lot of scenarios that this process might cover when it comes to automatise source code generation. For example, here I showed how we can generate an NPM package, but I'm also using this process to generate and publish a dotnet Nuget package.


Comments

Post a Comment

Popular posts from this blog

Building Micro-Frontends with Single-Spa

Image
Introduction Micro-services architecture is becoming trendy since it is a suitable option many projects, specially those that are complex and we want to be able spilt our business logic into different services that can be built and deliver independently. In spite of the fact that micro-services architectures are more complex, they increase the resilience of your system and increase the time-to-market delivery, which can be crucial for many companies. But it is not all about services and backend. With the introduction of single page applications (SPAs), we can build fully-functional user interfaces that can work independently in the browser. And here we should ask ourselves the same questions as for backend applications: Is my SPA becoming a huge monolith? Would I like to deliver functional modules independently? If you using micro-services, is there a relation between my services and your UI? Do we want to have functional teams that can deliver new features (frontend+backend) independe...
Read more

AWS assuming role + MFA with pulumi (and other applications)

Image
Introduction In AWS, managing your IAM in a centralized account is a good practice, so then you can assume a role to access to other accounts without need of duplicating users. You can find more info about this approach in this article . In addition, for security reasons, it is highly recommended to enable MFA (Multi-factor Authentication), so you need a device to generate a temporary code to get access. In the image below it is showed how assume role with MFA works, but you also have an article about MFA in this link . This approach is really useful and most of the configuration can be automated with pulumi (or other providers like terraform or CloudFormation). And once it is configured, you can switch your role and get access to other accounts with a few clicks.  However, when it comes to external software such us pulumi, getting access assuming a role and using MFA might not be too straightforward. In this article we will see how this can be configured and how we can easily refr...
Read more

Managing snapshots for Amazon ElasticSearch with Dotnet Core Lambdas

Image
Introduction It is awesome to have some useful services like ElasticSearch managed by AWS , so you don't have to care about patching, monitoring, etc.  When it comes to backup management for indexes, Amazon ElasticSearch includes automated snapshots. Automated snapshots are only for cluster recovery. You can use them to restore your domain in the event of red cluster status or other data loss. Amazon ES stores automated snapshots in a preconfigured Amazon S3 bucket at no additional charge. In particular, if you are using a higher version than 5.3, Amazon ElasticSearch takes hourly automated snapshots and retains up to 336 of them for 14 days. However, this policy might not be enough if you need to keep your snapshots for a longer 15 days, or you need to take a snapshot to create a new cluster in another region, subnet, etc. If you have different requirements to manage your snapshots, you will have to work with manual snapshots. For this part, AWS does not offer a good managed exper...
Read more