Overview
We use the next primary technologies for deployment:
- Docker for delivering applications inside containers;
- Kubernetes for containers orchestration;
- Helm for managing Kubernetes applications;
- GitHub Actions for CI/CD deployment;
To use this guide we highly recommend you to check their documentation and be familiar with basics.
Deployed application is multiple services, wrapped in Docker containers and run inside the Kubernetes cluster. Ship consists of 4 services by default: Web, API, Scheduler and Migrator.
Deployment can be done manually from your local machine or via a CI/CD pipeline. We have templates of deployment scripts for Digital Ocean and AWS, but you can use another cloud providers with minor changes.
External services you need
- Kubernetes cluster, you can create DO Managed Kubernetes or AWS EKS;
- Container registry for your Docker images, mostly cloud providers has it own: DO Container Registry, AWS ECR;
- DNS provider account. We recommend CloudFlare, for AWS you can use Route 53;
- Managed MongoDB. We recommend MongoDB Atlas or DO Managed MongoDB;
Deployment schema
Deployment flow
The deployment flow is pretty simple. Once you make changes in any service, the script builds a new Docker image for it, adds image tag to it, and pushes it to the Container Registry.
Script passes the image tag to deployment command, so Kubernetes knows which image needs to be downloaded from the registry. Image tag consists of repo branch and commits hash.
imageTag = `${branch}.${commitSHA}`;
Then script grabs all resources templates(Ingress, Service, etc) from the templates folder for services that are deploying, packages them as Helm Chart, and creates Helm Release that installs all that resources in the cluster. During the release, Kubernetes will download a new Docker image from registry and use it to create a new version of the service in the cluster.
We use Blue-Green deployment through Rolling Update.
This is the main part of the deployment script.
const buildAndPushImage = async ({ dockerFilePath, dockerRepo, dockerContextDir, imageTag, environment }) => {
await execCommand(`docker build \
--build-arg APP_ENV=${environment} \
-f ${dockerFilePath} \
-t ${dockerRepo} \
${dockerContextDir}`);
await execCommand(`docker tag ${dockerRepo} ${imageTag}`);
await execCommand(`docker push ${imageTag}`);
}
const pushToKubernetes = async ({ imageTag, appName, deployConfig }) => {
await execCommand(`
helm upgrade --install apps-${config.environment}-${appName} ${deployDir} \
--namespace ${config.namespace} --create-namespace \
--set appname=${appName} \
--set imagesVersion=${imageTag} \
--set nodePool=${config.nodePool} \
--set containerRegistry=${config.dockerRegistry.name} \
-f ${deployDir}/${config.environment}.yaml \
--timeout 35m \
`);
}
// build web image and push it to registry
await buildAndPushImage({
...deployConfig,
imageTag: `${deployConfig.dockerRepo}:${imageTag}`,
environment: config.environment
});
// deploy web to kubernetes
await pushToKubernetes({
imageTag,
appName: 'web',
deployConfig
});
We have 2 separate GitHub Actions workflows for services:
If the Migrator fails, API and Scheduler will be not deployed. This approach guarantees us that the API and Scheduler always work with the appropriate database schema.
We have separate GitHub Actions workflows for different environments.
Environment variables
When deploying “Ship” to Kubernetes, it’s essential to consider the configuration needs of different environments. By separating the environment-specific settings into dedicated files, you can easily manage and deploy the application across environments.
The APP_ENV
environment variable is typically set based on the environment in which the application is running.
Its value corresponds to the specific environment, such as “development”, “staging” or “production”.
This variable helps the application identify its current environment and load the corresponding configuration.
For the web application, by setting the environment variable APP_ENV
,
the application can determine the environment in which it is running and download the appropriate configuration file:
APP_ENV | File |
---|---|
development | .env.development |
staging | .env.staging |
production | .env.production |
These files should contain specific configuration variables required for each environment.
In contrast, the API utilizes a single .env
file that houses its environment-specific configuration.
This file typically contains variables like API keys, secrets, or other sensitive information.
To ensure security, it’s crucial to add the .env
file to the .gitignore
file,
preventing it from being tracked and committed to the repository.
When deploying to Kubernetes, you’ll need to include the appropriate environment-specific configuration files in your deployment manifests. Kubernetes offers ConfigMaps and Secrets for managing such configurations.
ConfigMaps are suitable for non-sensitive data, while Secrets are recommended for sensitive information like API keys or database connection string. Ensure that you create ConfigMaps or Secrets in your Kubernetes cluster corresponding to the environment-specific files mentioned earlier.
Database setup
We recommend avoiding self-managed database solutions and use cloud service like MongoDB Atlas that provides managed database. It handles many quite complex things: database deployment, backups, scaling, and security.
SSL
To make your application work in modern browsers and be secure, you need to configure SSL certificates. The easiest way is to use CloudFlare, it allows you to set up SSL in most simple way by proxying all traffic through CloudFlare. Use this guide.
Cloudflare can be used as DNS nameservers for your DNS registrar, such as GoDaddy. Also, you can buy and register a domain in Cloudflare itself.
If you are deploying in AWS you can use AWS Certificate Manager for SSL.
Services
Services are parts of your application packaged as Helm Charts.
Service | Description | Kubernetes Resource |
---|---|---|
Web | Next.js server that serves static files and API endpoints | Pod |
API | Backend server | Pod |
Scheduler | Service that runs cron jobs | Pod |
Migrator | Service that migrates database schema. It deploys before api through Helm pre-upgrade hook | Job |
To deploy services in the cluster manually you need to set cluster authorization credentials inside config and run deployment script.
node index
? What service to deploy? (Use arrow keys)
api
web
When you will configure GitHub Secrets in your repo, GitHub Actions will automatically deploy your services on push in the repo. You can check the required secrets inside workflow files.
Dependencies
Dependencies are third-party services packaged as Helm Charts and bash scripts that install configured resources in the cluster.
Dependency | Description |
---|---|
ingress-nginx | Ingress controller for Kubernetes using Nginx as a reverse proxy and load balancer |
redis | Open source, advanced key-value store |
regcred | Bash script for creating Kubernetes Secret. Secret needs for authorizing in Container Registry when pulling images from cluster. Required only for Digital Ocean clusters |
To deploy dependencies in cluster, you need to run deploy-dependencies.sh script.
bash deploy-dependencies.sh
If you are adding new dependency, you need to create separate folder inside dependencies folder and configure new Chart. Also, you need to add new dependency in deploy-dependencies.sh script. You can do it following the example from neighboring dependencies.
Deploy scripts structure
Folder | Description |
---|---|
.github | GitHub Actions CI/CD pipelines for automated deployment on push in repo |
app | Helm charts for services |
bin | Utils scripts |
dependencies | Helm charts for dependencies |
script | Deployment script |