Skip to main content

Overview

We use the next primary technologies for deployment:

  • Docker for delivering applications inside containers;
  • Kubernetes for containers orchestration;
  • Helm for managing Kubernetes applications;
  • GitHub Actions for CI/CD deployment;

To use this guide we highly recommend you to check their documentation and be familiar with basics.

Deployed application is multiple services, wrapped in Docker containers and run inside the Kubernetes cluster.
Ship consists of 4 services by default: Web, API, Scheduler and Migrator.

Deployment can be done manually from your local machine or via a CI/CD pipeline. We have templates of deployment scripts for Digital Ocean and AWS, but you can use another cloud providers with minor changes.

External services you need

Deployment schema

Deployment flow

The deployment flow is pretty simple. Once you make changes in any service, the script builds a new Docker image for it, adds image tag to it, and pushes it to the Container Registry.

Script passes the image tag to deployment command, so Kubernetes knows which image needs to be downloaded from the registry. Image tag consists of repo branch and commits hash.

imageTag = `${branch}.${commitSHA}`;

Then script grabs all resources templates(Ingress, Service, etc) from the templates folder for services that are deploying, packages them as Helm Chart, and creates Helm Release that installs all that resources in the cluster. During the release, Kubernetes will download a new Docker image from registry and use it to create a new version of the service in the cluster.

tip

We use Blue-Green deployment through Rolling Update.

This is the main part of the deployment script.

deploy/script/src/index.js
const buildAndPushImage = async ({ dockerFilePath, dockerRepo, dockerContextDir, imageTag, environment }) => {
await execCommand(`docker build \
--build-arg APP_ENV=${environment} \
-f ${dockerFilePath} \
-t ${dockerRepo} \
${dockerContextDir}`);
await execCommand(`docker tag ${dockerRepo} ${imageTag}`);
await execCommand(`docker push ${imageTag}`);
}

const pushToKubernetes = async ({ imageTag, appName, deployConfig }) => {
await execCommand(`
helm upgrade --install apps-${config.environment}-${appName} ${deployDir} \
--namespace ${config.namespace} --create-namespace \
--set appname=${appName} \
--set imagesVersion=${imageTag} \
--set nodePool=${config.nodePool} \
--set containerRegistry=${config.dockerRegistry.name} \
-f ${deployDir}/${config.environment}.yaml \
--timeout 35m \
`);
}

// build web image and push it to registry
await buildAndPushImage({
...deployConfig,
imageTag: `${deployConfig.dockerRepo}:${imageTag}`,
environment: config.environment
});

// deploy web to kubernetes
await pushToKubernetes({
imageTag,
appName: 'web',
deployConfig
});

We have 2 separate GitHub Actions workflows for services:

If the Migrator fails, API and Scheduler will be not deployed. This approach guarantees us that the API and Scheduler always work with the appropriate database schema.

tip

We have separate GitHub Actions workflows for different environments.

Environment variables

Services include environment folders that are responsible for different environment variables. We store them in a more convenient JSON format. APP_ENV environment variable is responsible for the config file from environment folder that will be taken.

APP_ENVFile
developmentdevelopment.json
development-dockerdevelopment-docker.json
stagingstaging.json
productionproduction.json
caution

These values are public.
Don't upload your source code to a public repository or configure .env and Kubernetes Secrets.

Database setup

We recommend avoiding self-managed database solutions and use cloud service like MongoDB Atlas that provides managed database. It handles many quite complex things: database deployment, backups, scaling, and security.

After you create the database you will need to add a connection string in the config.

config/environment/production.json
  {
"mongo": {
"connection": "mongodb+srv://<username>:<password>@<db>/api-production?retryWrites=true&w=majority",
"dbName": "api-production"
}
}

SSL

To make your application work in modern browsers and be secure, you need to configure SSL certificates. The easiest way is to use CloudFlare, it allows you to set up SSL in most simple way by proxying all traffic through CloudFlare. Use this guide.

Cloudflare can be used as DNS nameservers for your DNS registrar, such as GoDaddy. Also, you can buy and register a domain in Cloudflare itself.

If you are deploying in AWS you can use AWS Certificate Manager for SSL.

Services

Services are parts of your application packaged as Helm Charts.

ServiceDescriptionKubernetes Resource
WebNext.js server that serves static files and API endpointsPod
APIBackend serverPod
SchedulerService that runs cron jobsPod
MigratorService that migrates database schema. It deploys before api through Helm pre-upgrade hookJob

To deploy services in the cluster manually you need to set cluster authorization credentials inside config and run deployment script.

deploy/script/src
node index

? What service to deploy? (Use arrow keys)
api
web

When you will configure GitHub Secrets in your repo, GitHub Actions will automatically deploy your services on push in the repo. You can check the required secrets inside workflow files.

tip

If you are adding new service, you need to configure it in app and script folders. You can do it following the example from neighboring services.

Dependencies

Dependencies are third-party services packaged as Helm Charts and bash scripts that install configured resources in the cluster.

DependencyDescription
ingress-nginxIngress controller for Kubernetes using Nginx as a reverse proxy and load balancer
redisOpen source, advanced key-value store
regcredBash script for creating Kubernetes Secret. Secret needs for authorizing in Container Registry when pulling images from cluster. Required only for Digital Ocean clusters

To deploy dependencies in cluster, you need to run deploy-dependencies.sh script.

deploy/bin
bash deploy-dependencies.sh
tip

If you are adding new dependency, you need to create separate folder inside dependencies folder and configure new Chart. Also, you need to add new dependency in deploy-dependencies.sh script. You can do it following the example from neighboring dependencies.

Deploy scripts structure

FolderDescription
.githubGitHub Actions CI/CD pipelines for automated deployment on push in repo
appHelm charts for services
binUtils scripts
dependenciesHelm charts for dependencies
scriptDeployment script