AWS
It’s a step-by-step Ship deployment guide. We will use Amazon Elastic Kubernetes Service (EKS), Mongo Atlas, Amazon Elastic Container Registry (ECR), GitHub Actions for automated deployment, and CloudFlare for DNS and SSL configuration.
You need to create GitHub, AWS, MongoDB Atlas and CloudFlare accounts and install the next tools on your machine before starting:
- kubectl - CLI tool for accessing Kubernetes cluster;
- kubectx - CLI tool for easier switching between Kubernetes contexts;
- helm - CLI tool for managing Kubernetes deployments;
- aws-cli - CLI tool for managing AWS resources;
- eksctl - CLI tool for managing EKS clusters;
- k8sec - CLI tool for managing Kubernetes Secrets easily;
Try the next commands to ensure that everything is installed correctly:
Also, you need git and Node.js if you already haven’t.
Setup project
First, initialize your project. Type npx create-ship-app@latest
in the terminal then choose AWS EKS deployment type.
You will have the next project structure.
AWS Regions
AWS Regions are physical locations of AWS cluster data centers. Each group of logical data centers calls Availability Zone (AZ). AZs allow the operation of production applications and databases that are more highly available, fault-tolerant and scalable.
Now you need to select an AWS region for future use of the services. You can read more about region selection for your workloads here: What to Consider when Selecting a Region for your Workloads.
For this deployment guide, we will use the us-east-1 region.
Usually, you have to create AWS resources in a single region. If you don’t see created resources, you may need to switch to the appropriate AWS region.
Container registry
You need to create private repositories for storing Docker images. The deployment script will upload images to Container Registry during the build step, and Kubernetes will automatically pull these images from Container Registry to run a new version of the service during the deployment step.
Now we should create a repository for each service.
For Ship, we need to create repositories for the next services:
You should create a private repository for each service manually.
After creation, you should have the following 4 services in ECR
Docker images for each service are stored in a separate repository. During the deployment process script will automatically create paths to repositories in the next format:
- API - 276472736030.dkr.ecr.us-east-1.amazonaws.com/api;
- Migrator - 276472736030.dkr.ecr.us-east-1.amazonaws.com/migrator;
- Scheduler - 276472736030.dkr.ecr.us-east-1.amazonaws.com/scheduler;
- Web - 276472736030.dkr.ecr.us-east-1.amazonaws.com/web;
Repository name276472736030.dkr.ecr.us-east-1.amazonaws.com/api
consists of 5 values:
276472736030
- AWS account ID;us-east-1
- AWS region.dkr.ecr
- AWS service.amazonaws.com
- AWS domain.api
- service name.
Images for all environments will be uploaded to the same repository for each service.
Kubernetes Cluster
Now let’s create EKS cluster.
Select Custom Configuration
Navigate to the cluster creation page and choose Custom configuration
Make sure to disable EKS Auto Mode
Name Your Cluster
Enter a name for your cluster. It’s recommended to use your project name.
For multi-environment setups, append the environment name to your cluster:
my-ship-app-staging
my-ship-app-production
Configure Cluster IAM Role
For the Cluster IAM role
:
- Click the
Create recommended role
button - AWS will automatically create IAM roles with necessary EKS cluster permissions
- Return to cluster creation page and select the created policy
Set Authentication Mode
In the Cluster access
section:
- Set
Cluster authentication mode
toEKS API and ConfigMap
Configure Add-ons
Navigate to ‘Select add-ons’ and verify these required add-ons are selected:
- CoreDNS
- kube-proxy
- Amazon VPC CNI
- Node monitoring agent
Review and Create
Move to the review section and verify all configuration parameters are correct before creating the cluster.
Default values for other configuration parameters are suitable unless you have specific requirements.
After creation, you need to wait a few minutes until the cluster status becomes Active.
After cluster creation, you should attach EC2 instances to the cluster. You can do it by clicking on the Add Node Group button on the Compute tab.
Set the node group name as pool-app
and select the relevant Node IAM role from the list.
If you don’t have any IAM roles here, click the Create recommended role
button. You will be prompted to create properly configured IAM roles with all necessary permissions.
AWS recommends creating at least 2 nodes t3.medium instance type for the production environment.
Default values for other configuration parameters are suitable unless you have specific requirements.
Accessing a cluster from a local machine
Before proceeding, ensure you have configured the AWS CLI.
Update kubeconfig
Run the following command to configure cluster access:
Replace us-east-1
with your cluster’s region and my-ship-app
with your cluster name.
Switch to cluster context
Execute kubectx
in your terminal and select your cluster from the list.
Verify cluster access
Check the installed pods by running:
You should see a list of system pods in your cluster:
Ingress NGINX Controller
ingress-nginx is an Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer.
Learn more about ingress-nginx functionality in the official documentation.
Navigate to dependencies directory
Change to the deploy/dependencies
directory in your terminal.
Configure Helm Values (Optional)
This step is required only if you specified a custom node group name in your EKS cluster.
If you did, update the eks.amazonaws.com/nodegroup
value in values.yaml.gotmpl
:
Install dependencies
Install helm dependencies using helmfile:
Review and apply changes
Preview the changes first:
If the preview looks correct, apply the configuration:
DNS and SSL
Get Load Balancer Address
After deploying ingress-nginx, retrieve the Load Balancer’s external hostname:
If you have trouble running the above command, you can alternatively use:
And copy the value from the EXTERNAL-IP
column.
Domain Naming Convention
You can follow this recommended naming pattern for different environments:
Environment | API Domain | Web Domain |
---|---|---|
Production | api.ship.com | app.ship.com |
Staging | api.staging.ship.com | app.staging.ship.com |
Configure DNS in Cloudflare
- First, ensure you have a domain in Cloudflare. You can either:
- In the Cloudflare DNS tab, create 2
CNAME
records:
- One for Web interface
- One for API endpoint
Both should point to your Load Balancer’s external hostname.
Enable the Proxied option to:
- Route traffic through Cloudflare
- Generate SSL certificates automatically
Cloudflare’s free Universal SSL certificates only cover the apex domain and one subdomain level. For multiple subdomain levels, you’ll need an Advanced Certificate.
Update Configuration Files
Update your domain settings in the appropriate environment configuration files:
For API service:
For Web service:
MongoDB Atlas
MongoDB Atlas is a fully managed cloud database service that provides automated backups, scaling, and security features. It offers 99.995% availability with global deployment options and seamless integration with AWS infrastructure.
Cluster Creation
Access MongoDB Atlas
Sign in to your MongoDB Atlas account and create a new project if needed.
Deploy New Cluster
Click Create to start cluster deployment.
Cluster Tier Selection:
- Staging:
M0
(Free tier) - Suitable for development and testing - Production:
M10
or higher - Includes automated backups and advanced features
Provider & Region:
- Select AWS as your cloud provider
- Choose the same region as your EKS cluster for optimal performance
Configure Cluster Name
Enter a descriptive cluster name (e.g., ship-production-cluster
, ship-staging-cluster
)
Security Configuration
Create Database User
Navigate to Database Access → Add New Database User
- Authentication Method: Password
- Username: Use environment-specific names (e.g.,
api-production
,api-staging
) - Password: Generate a strong password
- Database User Privileges: Read and write to any database
Password Requirements: Ensure the password starts with a letter and contains only alphanumeric characters and common symbols. Special characters at the beginning can cause URI parsing issues.
Configure Network Access
Navigate to Network Access → Add IP Address
- Click Allow access from anywhere to allow connections from any IP with valid credentials
- For production, consider restricting to specific IP ranges for enhanced security
Get Connection String
Access Connection Details
Go to your cluster dashboard and click the Connect button.
Copy Connection String
- Select Drivers in the “Connect your application” section
- Choose Node.js driver and latest version
- Copy the connection string and replace
<db_password>
with your actual password
Example Connection String:
Save Connection Details
Store the connection string securely - you’ll need it for environment configuration later
Before deploying to production, configure automated backups in the Atlas console to ensure data recovery capabilities.
Environment variables
Kubernetes applications require proper environment variable configuration for both API and Web components. This section covers how to set up and manage environment variables securely using Kubernetes secrets and configuration files.
API Environment Variables
For the API deployment, you need to set up environment variables using Kubernetes secrets to securely manage sensitive configuration data.
Secrets in Kubernetes are used to store sensitive information, such as passwords, API tokens, and keys. They are encoded in Base64 format to provide a level of security. These can be mounted into containers as data volumes or used as environment variables.
Before deploying the app, make sure all necessary variables from the API config are exist. Here are the minimal set of required variables:
Name | Description | Example value |
---|---|---|
APP_ENV | Application environment | production |
MONGO_URI | Database connection string | mongodb://<username>:<password>@ship.mongodb.net |
MONGO_DB_NAME | Database name | api-production |
API_URL | API domain URL | https://api.my-ship-app.paralect.com |
WEB_URL | Web app domain URL | https://my-ship-app.paralect.com |
JWT_SECRET | JWT signing key | Vz2Ol8HKBO0/38i1IBm2uJ7JnVabWGm0RiRVY5w1sNY= |
Environment Variable Details
Setting up Kubernetes Secrets
Create namespaces and secret objects
Create Kubernetes namespaces and secret objects for staging and production environments:
Initialize secret storage
First, create an APP_ENV
variable to initialize secret storage for k8sec:
Verify secret creation
Run the following command to check the created secret:
Prepare environment file
Create a .env.production
file with all required variables:
Replace all example values with your actual configuration. Never use production secrets in documentation or version control.
Import secrets to Kubernetes
Import secrets from the .env file to Kubernetes secret using k8sec:
After updating environment variables, you must initiate a new deployment for changes to take effect. Kubernetes pods cache variable values during startup, requiring a pod restart or rolling update to apply changes.
Web Environment Variables
The web application uses Next.js environment variables that are embedded at build time and made available in the browser. Unlike API secrets, these variables are stored directly in the GitHub repository.
Why Web Environment Variables Are Safe in Git: Web environment variables (prefixed with NEXT_PUBLIC_
) contain only public configuration like URLs and API endpoints. They don’t include sensitive data like passwords or API keys, making them safe to store in version control. These values are already exposed to users in the browser, so repository storage doesn’t create additional security risks.
Security Notice: Never store sensitive information (passwords, API keys, secrets) in web environment files as they will be accessible on the client side. Only use public configuration values that are safe to expose to end users.
Configuration Files
Web environment variables are stored in separate files for each deployment environment:
Environment Variables Reference
Variable | Description | Example |
---|---|---|
NEXT_PUBLIC_API_URL | Base URL for API requests | https://api.my-ship-app.paralect.com |
NEXT_PUBLIC_WS_URL | WebSocket server URL for real-time | https://api.my-ship-app.paralect.com |
NEXT_PUBLIC_WEB_URL | App’s own URL for redirects/metadata | https://my-ship-app.paralect.com |
Best Practice: Keep web environment files in your repository and ensure all values are non-sensitive. If you need to reference sensitive data from the frontend, create a secure API endpoint that returns the necessary information after proper authentication.
Setting up GitHub Actions CI/CD
Creating IAM user in AWS
To set up CI/CD with GitHub Actions securely, we need to create a dedicated IAM user in AWS with specific permissions.
This separate user will be used exclusively for CI/CD operations, following the principle of least privilege and keeping deployment credentials isolated from other system users.
Create IAM Policy
- Go to AWS IAM Policies
- Click Create policy
- Select JSON tab and add the policy:
- (Optional) Add tags
- Give the policy a name (e.g.
GitHubActionsDeployPolicy
) and create it
Create IAM User
- Navigate to Users in IAM console
- Click Create user
- Give the user a name (e.g.
github-actions
)
- Attach the policy you created by selecting:
- Attach existing policies directly
- Choose the CI/CD policy created in previous step
- (Optional) Add user tags
- Review and create user
Generate Access Keys
- Find your new user in the users list and open user’s page
- Click Create access key
- Select use case: Third-party service
- Save the Access Key ID and Secret Access Key securely
The Secret Access Key will only be shown once - make sure to save it immediately!
Configure EKS Access
- Copy your user’s ARN from the IAM dashboard
- Run the following command to grant Kubernetes access:
Replace YOUR_USER_ARN
with the actual ARN copied earlier.
These permissions enable CI/CD workflows while following security best practices:
- Minimal required permissions for ECR operations
- Limited EKS access for cluster management
- Dedicated CI/CD user separate from other IAM users
Configuring GitHub Actions secrets and variables
Before starting, make sure you have created a GitHub repository for your project.
GitHub Secrets and variables allow you to manage reusable configuration data.
Secrets are encrypted and are used for sensitive data. Learn more about encrypted secrets.
Variables are shown as plain text and are used for non-sensitive data. Learn more about variables.
The deployment will be triggered on each commit:
- Commits to main branch → deploy to staging environment
- Commits to production branch → deploy to production environment
Configure the following secrets and variables in your GitHub repository:
Name | Type | Description |
---|---|---|
AWS_SECRET_ACCESS_KEY | secret | The secret access key from the AWS IAM user created for CI/CD. This allows GitHub Actions to authenticate with AWS services |
AWS_ACCESS_KEY_ID | variable | The access key ID from the AWS IAM user. Used in conjunction with the secret key for AWS authentication |
AWS_REGION | variable | The AWS region where your EKS cluster and ECR registry are located (e.g. us-east-1 ) |
CLUSTER_NODE_GROUP | variable | The name of the EKS node group where your application pods will be scheduled (e.g. pool-app ) |
CLUSTER_NAME_PRODUCTION | variable | The name of your production EKS cluster. Required when deploying to the production environment |
CLUSTER_NAME_STAGING | variable | The name of your staging EKS cluster. Required when deploying to the staging environment |
Never commit sensitive credentials directly to your repository.
Always use GitHub Secrets for sensitive information like AWS keys.
Variables (unlike secrets) are visible in logs and can be used for non-sensitive configuration values that may need to be referenced or modified.
Now commit all changes to GitHub that will trigger deployment, or you can run a workflow manually
Done! Application deployed and can be accessed by the provided domain.
If something went wrong you can check the workflows logs on GitHub and use kubectl logs, kubectl describe commands.
Setting up Upstash Redis database (recommended)
Upstash Redis Integration
Upstash Redis is a highly available, infinitely scalable Redis-compatible database that provides enterprise-grade features without the operational complexity.
How Ship Uses Redis
Ship leverages Redis for several critical functionalities:
Use Case | Description | Implementation |
---|---|---|
Real-time Communication | Pub/Sub mechanism for WebSocket functionality | Socket.io Redis Adapter |
Rate Limiting | API request throttling and abuse prevention | Redis counters with TTL |
Caching | Application data caching for improved performance | Key-value storage with expiration |
Redis as a Message Broker: When scaling to multiple server instances, Redis acts as a message broker between Socket.io servers, ensuring real-time messages reach all connected clients regardless of which server they’re connected to.
Setting Up Upstash Redis
Create Your Database
Access Upstash Console
Log in to your Upstash account and navigate to the Redis section.
Create New Database
Click Create Database in the upper right corner to open the configuration dialog.
Configure Database Settings
Database Name: Choose a descriptive name for your database (e.g., my-ship-app-production
)
Primary Region: Select the region closest to your main application deployment for optimal write performance.
Read Regions: Choose additional regions where you expect high read traffic for better global performance.
Select Plan & Deploy
Choose your pricing plan based on expected usage and click Create to deploy your database.
Region Selection: For Kubernetes deployments on AWS, choose the same AWS region as your EKS cluster to minimize latency and data transfer costs.
Get Connection Details
Once your database is created, you’ll need the connection string for your application:
Navigate to Connection Info
Go to your database dashboard and find the Connect to your database section.
Copy Connection String
- Select the Node tab for the appropriate connection string format
- Click Reveal to show the hidden password
- Copy the complete Redis URI (format:
rediss://username:password@host:port
)
Add to Environment Variables through k8sec
Using k8sec
, add the Redis connection string to your environment configuration:
After updating environment variables, restart your API pod using:
This will trigger Kubernetes to create a new pod with the updated environment variables.
Verify Connection with Redis Insight
Redis Insight is a powerful GUI tool for managing and debugging Redis databases.
Install Redis Insight
Download and install Redis Insight on your local machine.
Add Database Connection
- Open Redis Insight
- Click Add Database
- Paste your Upstash Redis connection string in the Connection URL field
- Click Add Database
Explore Your Database
Once connected, you can use Upstash Redis Console to:
- Browse keys and data structures
- Execute Redis commands directly
- Monitor real-time performance metrics
- Debug application data storage
Real-time Monitoring: Upstash Redis updates database metrics automatically every 10 seconds, giving you near real-time visibility into your Redis performance and usage.