Say “Hello” to the Revolutionized Git Integration with APICTL 4.0.0
WSO2 API Controller 4.0.0 + WSO2 API Manager 4.0.0
APIs are now heavily used by businesses to execute business-critical functions. Without some sort of Continuous Integration and Continuous Delivery/Deployment (CI/CD), it is difficult to envisage a reliable and effective API-driven business. The facility to combine the CI/CD pipelines with a widely used Version Control System (VCS) such as Git has been introduced last year with the WSO2 API Controller (apictl) 3.2.0. But it had the ability only to control and maintain one repository with your APIs, API Products and Applications.
With the release of apictl 4.0.0, the above-stated functionality has been enhanced to support two (2) repositories at once, namely: the Source Repository and the Deployment Repository.
With this enhancement, the Git integration support has been widened too by minimizing the DevOps burden now more than ever, thus adding further value to your business needs.
This article will explain “How the Git integration has been improved to cater CI/CD pipelines of your businesses with the new Source/Deployment structure?”
What are these Source and Deployment repositories?
Before moving onto the Git integration with apictl 4.0.0, let me give you a brief introduction to the new Source and Deployment repository structure.
By looking at the above diagram you can see two (2) sections as the Source Repository and the Deployment Repository. Let me explain those two separately.
Source Repository
Recall the basic functionality of apictl. When you export an API/API Product/Application from an environment via apictl you will get an archive (a .zip file). You can maintain these exported archives in a separate repository (Make sure to unzip those and keep them as directories). This separate repository is called the Source Repository.
Source
├── admin-TestApp
│ ├── application_meta.yaml
│ └── application.yaml
├── MyProduct-1.0.0
│ ├── api_product_meta.yaml
│ ├── api_product.yaml
│ ├── APIs
│ │ ├── PizzaShackAPI-1.0.0
│ │ │ ├── api.yaml
│ │ │ └── Definitions
│ │ │ └── swagger.yaml
│ │ └── SwaggerPetstore-1.0.5
│ │ ├── api.yaml
│ │ └── Definitions
│ │ └── swagger.yaml
│ └── Definitions
│ └── swagger.yaml
├── PizzaShackAPI-1.0.0
│ ├── api_meta.yaml
│ ├── api.yaml
│ ├── Definitions
│ │ └── swagger.yaml
│ └── deployment_environments.yaml
└── SwaggerPetstore-1.0.5
├── api_meta.yaml
├── api.yaml
├── Definitions
│ └── swagger.yaml
└── deployment_environments.yamlNote:- Apart from the exported archives, you can keep the API Project directories created by the DevFirst Approach in the Source Repository. But make sure their names are in the form of <API_Name>-<API_Version>. And it is advised to maintain this naming convention for Git integration to be worked smoothly.
Deployment Repository
In the previous releases of apictl, you had the ability to use an api_params.yaml file to override the environment-specific details for APIs. With apictl 4.0.0, this has been widened by giving the chance for you to maintain a directory per API or API Product. This directory may contain not only the params of an API/API Product but also the certificates as well. (See Configuring Environment Specific Parameters for more information) All these directories for each API/API Product can be maintained in one single directory and it is called the Deployment Repository.
Deployment
├── DeploymentArtifacts_MyProduct-1.0.0
│ ├── api_product_meta.yaml
│ ├── certificates
│ └── params.yaml
└── DeploymentArtifacts_PizzaShackAPI-1.0.0
├── api_meta.yaml
├── certificates
│ ├── dev.crt
│ ├── prod.crt
│ └── test.crt
└── params.yamlNote:- You can generate deployment directory per API/API Product using the newly introduced apictl gen deployment-dir command. The name of the deployment directory for an API/API Product will have the form DeploymentArtifacts_<API_Name>-<API_Version>. And it is advised to maintain this naming convention for Git integration to be worked smoothly.
Git Branches
As shown in the diagram named Source and Deployment Repository Structures, both the Source and Deployment Repositories should be Git repositories. You can maintain Git branches in the Source Repository with APIs/API Products/Applications corresponding to multiple WSO2 API Manager environments you wish.
But you do not need to maintain separate branches in the Deployment Repository. The reason for this is, if we take one deployment repo that belongs to an API/API Product, the params file inside it has the environment-specific details for each environment as a list. Since we have the environment-specific details listed per environment we do not need to keep branches per environment. For example, have a look at the below example params file for an API.
environments:
- name: dev
configs:
endpoints:
production:
url: 'https://dev.wso2.com'
certs:
- hostName: 'https://dev.wso2.com'
alias: Dev
path: dev.crt
policies:
- Unlimited
- name: test
configs:
endpoints:
production:
url: 'https://test.wso2.com'
certs:
- hostName: 'https://test.wso2.com'
alias: Test
path: test.crt
security:
production:
enabled: true
type: digest
username: admin
password: admin
sandbox:
enabled: true
type: basic
username: admin
password: admin
- name: prod
configs:
endpoints:
production:
url: 'https://prod.wso2.com'
certs:
- hostName: 'https://prod.wso2.com'
alias: Prod
path: prod.crt
policies:
- Gold
- SilverIn the next section, I will discuss the way to incorporate Git integration using apictl 4.0.0.
First and Foremost… Prerequisites
Under the assumption that you have already installed Oracle Java SE Development Kit (JDK) version 11.\* and set the JAVA_HOME environment variable (For more information on setting the JAVA_HOME environment variable for different operating systems, see Setup and Install.) you need to satisfy the below requirements.
- Download WSO2 API-M 4.0.0
- Download and initialize apictl 4.0.x
- Install and setup Git in your system
- Make sure both the Source and the Deployment Repositories are Git repos (You may use “git init” to initialize them as Git repos)
Git Integration aka “apictl vcs”
I will show you the basic steps to be performed when
Step 1 — Set the Source and the Deployment Repositories
It is mandatory to set the Source Repository path before proceeding further. You can execute apictl set --vcs-source-repo-path <path> for this task as shown below.
Optionally, you can set the Deployment Repository path as well. You can execute the apictl set --vcs-deployment-repo-path <path> for this task as shown below.
If you have not set the Deployment Repository path, while executing the apictl vcs deploy, only the artifacts inside the Source Repository will be considered when pushing to the environment.
Step 2— Commit the changes in both the Source and the Deployment Repositories
You need to checkout to the relevant branch in the Source Repository and make sure that you have committed all the required changes to be pushed to the environment. Since the Deployment Repository does not contain branches you can directly commit the changes to the existing branch.
Further, you need to execute apictl vcs init in both the Source and the Deployment Repositories separately. This command will generate a file named vcs.yaml that makes the particular directory “an apictl vcs supported Git repo”.
vcs.yamlwill contain a unique identifier for the repository which is used to store deployed commits related meta-information. It is adviced to commit this file as well.
Step 3— Push the changes to the WSO2 API-M instance
First, you need to make sure the WSO2 API-M 4.0.0 server that you need to push the artifact, is up and running. Next, make sure to add your WSO2 API-M instance as an environment using the apictl 4.0.0. (To learn more see Add an environment) For example; I will be adding the WSO2 API-M running on port 9443 as the prod environment.
apictl add env prod \
--apim https://localhost:9443You can now execute the apictl vcs deploy command from anywhere (in your file system), to push the artifacts to the relevant environment.
In the above screenshot, you can see that I have deployed two (2) APIs, one (1) API Product and one (1) Application to the prod environment. If you check your instance you may see that those have been imported.
- Check for the APIs. (Visit the Publisher Portal of your WSO2 API-M)
- If you check the endpoint details of the PizzaShackAPI, you can notice that the endpoint value is the one that we specified in the params example earlier and those changes have been pushed correctly.
- You can check for the API Products as well.
- Check for the Application. (Visit the Developer Portal of your WSO2 API-M)
What if you like “old school”?
Let’s go back to the apictl 3.2.x release. apictl 3.2.x had the ability to do Git integration only with one repository. If you still prefer that way you can do it with apictl 4.0.0 as well. The only difference between this approach and the previously explained approach (with both the Source and Deployment Repositories) is in Step 1, you only need to set the Source Repository, no need to set the Deployment Repository.
But doing this method has a pitfall as well. In apictl 3.2.x, the api_params.yaml was included in the API project repository. So that, even though you had only the Source Repository in 3.2.x, you were able to edit the api_params.yaml inside the API project if needed and those changes will be automatically pushed via the “apictl vcs deploy” command. But, in apictl 4.0.0, the params file will not be included inside the API/API Product project repository. Hence, if you tend to use only the Source repository, you will not be able to push the changes in the params file.
Wrapping things up…
APIs are playing a huge role in businesses becoming the digital reflection of organizations. Developing, testing and maintenance tasks of APIs can be done seamlessly if you use CI/CD pipelines. To meet this need, WSO2 API-M introduced Git integration last year with apictl 3.2.0.
With the introduction of apictl 4.0.0 last month, Git integration has been more widened and it supports a new repository structure named Source/Deployment Repos. This article showed you a glimpse of these new improvements and you are welcome to explore further with WSO2 API-M 4.0.0 and apictl 4.0.x.
Goodbye until next time…. Stay safe everyone !!
