My HNG Journey. Stage Two: Containerization and Deployment of a Three tier application Using Docker and Nginx Proxy Manager

Ravencodess - Jul 8 - - Dev Community

Introduction

This stage brought on a task that at first glance seems easy and straightforward, but when the added requirements were introduced, the complexity grew and the challenge became harder. The task instructs us to containerize a three tier application on a single server and use a proxy manager like nginx to configure reverse proxying to ensure the frontend and backend can be served from the same port. That's not all. It gets more complex.

Here are the full requirements for completing this tasks:

  • Ensure the application runs locally before writing Dockerfiles
  • Configure the Frontend and Backend to listen on port 80
  • Obtain a domain name for the project
  • Write Dockerfiles to containerize the frontend and backend
  • Install adminer to enable database manager through its GUI
  • Configure Nginx proxy manager to handle reverse proxying and setup SSL certificates

Let's get started

Prerequisites

  • A virtual machine running Ubuntu
  • Basic Level Understanding of the Linux CLI

Step 1

Clone the repo
First we have to clone the repository from Github

git clone https://github.com/hngprojects/devops-stage-2
cd devops-stage-2
Enter fullscreen mode Exit fullscreen mode

Step 2

Configure the backend
The frontend of this application depends on the backend for full functionality so we will begin by configuring the backend.

cd backend
Enter fullscreen mode Exit fullscreen mode

Dependencies
The backend depends on a postgresQL database, It would also require poetry to be installed before starting up

Installing Poetry

To install Poetry, follow these steps:

curl -sSL https://install.python-poetry.org | python3 -
Enter fullscreen mode Exit fullscreen mode

Poetry Installation
Add Poetry to your PATH if it's not automatically added:

# Example for Bash shell
export PATH="$HOME/.poetry/bin:$PATH" >> ~/.bashrc
source ~./bashrc
poetry --version
Enter fullscreen mode Exit fullscreen mode

Replace $HOME/.poetry/bin with the appropriate path where Poetry binaries are installed if different on your system. This ensures you can run Poetry commands from any directory in your terminal session.

Adding poetry to Path

Install dependencies using Poetry:

poetry install
Enter fullscreen mode Exit fullscreen mode

poetry install depedencies
Setup PostgreSQL:
Follow these steps to install PostgreSQL on Linux and configure a user named app with password my_password and a database named app. Give all permissions of the app database to the app user.

Install PostgreSQL on Linux (example for Ubuntu):

sudo apt update
sudo apt install postgresql postgresql-contrib
Enter fullscreen mode Exit fullscreen mode

Switch to the PostgreSQL user and access the PostgreSQL

sudo -i -u postgres
psql
Enter fullscreen mode Exit fullscreen mode

Create a user app with password my_password:

CREATE USER app WITH PASSWORD 'my_password';
Enter fullscreen mode Exit fullscreen mode

Create a database named app and grant all privileges to the app user:

CREATE DATABASE app;
\c app
GRANT ALL PRIVILEGES ON DATABASE app TO app;
GRANT ALL PRIVILEGES ON SCHEMA public TO app;
Enter fullscreen mode Exit fullscreen mode

Exit the PostgreSQL shell and switch back to your regular user.

\q
exit
Enter fullscreen mode Exit fullscreen mode

postgresQL setup

Set database credentials
Edit the PostgreSQL environment variables located in the .env file. Make sure the credentials match the database credentials you just created.

---
POSTGRES_SERVER=localhost
POSTGRES_PORT=5432
POSTGRES_DB=app
POSTGRES_USER=app
POSTGRES_PASSWORD=my_password
Enter fullscreen mode Exit fullscreen mode

Set up the database with the necessary tables:

poetry run bash ./prestart.sh
Enter fullscreen mode Exit fullscreen mode

Setup script

Run the backend server and make it accessible on all network interfaces:

poetry run uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
Enter fullscreen mode Exit fullscreen mode

Backend start

Step 3

Configure the frontend
Open up a new terminal.
P.S. We can split the terminal session using tmux or run it as a system service, but to keep things fairly simple, we would leave the backend running in one terminal and open another terminal for the frontend.

cd devops-stage-2/frontend
Enter fullscreen mode Exit fullscreen mode

Dependencies
The frontend was built with Nodejs and npm for dependency management.

sudo apt update
sudo apt install nodejs npm
Enter fullscreen mode Exit fullscreen mode

Install dependencies:

npm install
Enter fullscreen mode Exit fullscreen mode

npm Install
Run the fronted server and make it accessible from all network interfaces:

npm run dev -- --host
Enter fullscreen mode Exit fullscreen mode

Frontend start
Accessing the application using curl:

curl localhost:5173
Enter fullscreen mode Exit fullscreen mode

Step 4

Accessing the UI

Open your browser and navigate to:

http://<your_server_IP>:5173
Enter fullscreen mode Exit fullscreen mode

frontend UI
Enable login access from the UI:
The login credentials can be found in the .env located in the backend folder

---
FIRST_SUPERUSER=devops@hng.tech
FIRST_SUPERUSER_PASSWORD=devops#HNG11
Enter fullscreen mode Exit fullscreen mode

If we try login in now we would be met with a network error.

frontend UI
Looking through the developer tools we can see that connecting to the backend on http://localhost:8000 was refused. This is because we are using a remote server and localhost in our browser's context means our personal computer. So to properly route the browser to the remote server running the application. we will have to Change the VITE_API_URL variable in the frontend .env file:

VITE_API_URL=http://<your_server_IP>:8000
Enter fullscreen mode Exit fullscreen mode

If we try to login now we are met with a new error called CORS which stands for Cross-origin resource sharing.

frontend UI
Basically, our backend doesn't recognise the origin of the request which is coming from our server's IP, so we need to tell our backend to accept request coming from that particular IP address.

In our backed .env file we need to add http://<your_server_IP>:5173 to the end of the string of allowed IPs

BACKEND_CORS_ORIGINS="http://localhost,http://localhost:5173,https://localhost,https://localhost:5173,http://<your_server_IP>:5173"
Enter fullscreen mode Exit fullscreen mode

Now If we try one more time to login.

frontend UI

We successfully setup the application locally.
We can also access the swagger API as well as the documentation paths using http://<your_server_IP>:8000/doc and http://<your_server_IP>:8000/redoc respectively.

Swagger
Redoc

Step 5

Containerizing the application
Now we need to repeat the entire process, but this time, We would utilize Docker containers. we will start by writing Dockerfiles for both frontend and backend and then move to the project's root directory and configure a docker compose file that will run and configure:

  • The Frontend and Backend
  • The postgres database the backend depends on
  • Adminer
  • Nginx proxy Manager

Let's start by writing the Dockerfile for the backend application

cd devops-stage-2/backend
vim Dockerfile
Enter fullscreen mode Exit fullscreen mode 
# Use the latest official Python image as a base
FROM python:latest

# Install Node.js and npm
RUN apt-get update && apt-get install -y \
    nodejs \
    npm

# Install Poetry using pip
RUN pip install poetry

# Set the working directory
WORKDIR /app

# Copy the application files
COPY . .

# Install dependencies using Poetry
RUN poetry install

# Expose the port FastAPI runs on
EXPOSE 8000

# Run the prestart script and start the server
CMD ["sh", "-c", "poetry run bash ./prestart.sh && poetry run uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload"]
Enter fullscreen mode Exit fullscreen mode

This repeats the entire process we carried out locally all in one file.

Now let's set up the frontend.

cd devops-stage-2/frontend
vim Dockerfile
Enter fullscreen mode Exit fullscreen mode 
# Use the latest official Node.js image as a base
FROM node:latest

# Set the working directory
WORKDIR /app

# Copy the application files
COPY . .

# Install dependencies
RUN npm install

# Expose the port the development server runs on
EXPOSE 5173

# Run the development server
CMD ["npm", "run", "dev", "--", "--host"]
Enter fullscreen mode Exit fullscreen mode

Again, this simply repeats the process we carried out to run the frontend locally.

Step 6

Docker compose setup
Navigate to the project root directory and create a docker-compose.yml file

cd devops-stage-2/
vim docker-compose.yml
Enter fullscreen mode Exit fullscreen mode

Copy this configuration into it

version: '3.8'

services:
  backend:
    build:
      context: ./backend
    container_name: fastapi_app
    ports:
      - "8000:8000"
    depends_on:
      - db
    env_file:
      - ./backend/.env

  frontend:
    build:
      context: ./frontend
    container_name: nodejs_app
    ports:
      - "5173:5173"
    env_file:
      - ./frontend/.env

  db:
    image: postgres:latest
    container_name: postgres_db
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
    env_file:
      - ./backend/.env

  adminer:
    image: adminer
    container_name: adminer
    ports:
      - "8080:8080"

  proxy:
    image: jc21/nginx-proxy-manager:latest
    container_name: nginx_proxy_manager
    ports:
      - "80:80"
      - "443:443"
      - "81:81"
    environment:
      DB_SQLITE_FILE: "/data/database.sqlite"
    volumes:
      - ./data:/data
      - ./letsencrypt:/etc/letsencrypt
    depends_on:
      - db
      - backend
      - frontend
      - adminer

volumes:
  postgres_data:
  data:
  letsencrypt:
Enter fullscreen mode Exit fullscreen mode

Breakdown of the docker-compose.yml File
Here's an explanation of each section in the provided docker-compose.yml file:

Services

Services are the containers that make up the application. Each service runs one image and can define volumes and networks. Each container can connect to any container in the same network using the service name.

Backend Service

backend:
  build:
    context: ./backend
  container_name: fastapi_app
  ports:
    - "8000:8000"
  depends_on:
    - db
  environment:
    POSTGRES_SERVER: ${POSTGRES_SERVER}
    POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
Enter fullscreen mode Exit fullscreen mode
  • build.context: Specifies the build context, pointing to the ./backend directory which contains the Dockerfile for building the FastAPI backend service.
  • container_name: Sets the container name to fastapi_app.
  • ports: Maps port 8000 on the host to port 8000 in the container.
  • depends_on: Ensures the db service is started before the backend service.
  • environment: Injects environment variables from the .env file, used by the FastAPI application to connect to the PostgreSQL database.

Frontend Service

frontend:
  build:
    context: ./frontend
  container_name: nodejs_app
  ports:
    - "5173:5173"
  environment:
    VITE_API_URL: ${VITE_API_URL}
Enter fullscreen mode Exit fullscreen mode
  • build.context: Points to the ./frontend directory for building the Node.js frontend service.
  • container_name: Names the container nodejs_app.
  • ports: Maps port 5173 on the host to port 5173 in the container.
  • environment: Injects the VITE_API_URL environment variable from the .env file, used by the frontend application to connect to the backend API.

Database Service

db:
  image: postgres:latest
  container_name: postgres_db
  ports:
    - "5432:5432"
  volumes:
    - postgres_data:/var/lib/postgresql/data
  environment:
    POSTGRES_USER: ${POSTGRES_USER}
    POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    POSTGRES_DB: ${POSTGRES_DB}
Enter fullscreen mode Exit fullscreen mode
  • image: Uses the latest PostgreSQL image from Docker Hub.
  • container_name: Names the container postgres_db.
  • ports: Maps port 5432 on the host to port 5432 in the container, which is the default port for PostgreSQL.
  • volumes: Mounts a Docker volume postgres_data to persist database data.
  • environment: Sets database-related environment variables from the .env file for initializing PostgreSQL.

Adminer Service

adminer:
  image: adminer
  container_name: adminer
  ports:
    - "8080:8080"
Enter fullscreen mode Exit fullscreen mode
  • image: Uses the Adminer image, a database management tool.
  • container_name: Names the container adminer.
  • ports: Maps port 8080 on the host to port 8080 in the container for accessing the Adminer web interface.

Proxy Service

proxy:
  image: jc21/nginx-proxy-manager:latest
  container_name: nginx_proxy_manager
  ports:
    - "80:80"
    - "443:443"
    - "81:81"
  environment:
    DB_SQLITE_FILE: "/data/database.sqlite"
  volumes:
    - ./data:/data
    - ./letsencrypt:/etc/letsencrypt
  depends_on:
    - db
    - backend
    - frontend
    - adminer
Enter fullscreen mode Exit fullscreen mode
  • image: Uses the latest Nginx Proxy Manager image.
  • container_name: Names the container nginx_proxy_manager.
  • ports: Maps ports 80, 443, and 81 on the host to the same ports in the container for HTTP, HTTPS, and the Nginx Proxy Manager admin interface.
  • environment: Sets the environment variable for the SQLite database location.
  • volumes: Mounts the data directory for storing proxy manager data and the letsencrypt directory for SSL certificates.
  • depends_on: Ensures the db, backend, frontend, and adminer services are started before the proxy service.

Volumes

volumes:
  postgres_data:
  data:
  letsencrypt:
Enter fullscreen mode Exit fullscreen mode

Defines named volumes to persist data across container restarts.

Step 7

Domain Setup
We need to setup domains and subdomains for the frontend, adminer service and Nginx proxy manager.
Remember we are required to route port 80 to both frontend and backend:

  • domain - Frontend
  • domain/api - Backend
  • db.domain - Adminer
  • proxy.domain - Nginx proxy manager

If you don't have a Domain name, you can acquire a subdomain at AfraidDNS. That's where i acquired the domain I used for this project. Ensure you route all the required domains above to the server your application is running on.

Step 8

Routing domains using Nginx proxy manager
We now have everything set up, we can run docker-compose up -d to get our application up and running. We would need to install Docker and Docker-compose first.

Install Docker
Update the package list:

sudo apt-get update
Enter fullscreen mode Exit fullscreen mode

Install required packages:

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common
Enter fullscreen mode Exit fullscreen mode

Add Docker’s official GPG key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
Enter fullscreen mode Exit fullscreen mode

Add the Docker repository to APT sources:

sudo add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) \
    stable"
Enter fullscreen mode Exit fullscreen mode

Update the package list again:

sudo apt-get update
Enter fullscreen mode Exit fullscreen mode

Install Docker:

sudo apt-get install docker-ce
Enter fullscreen mode Exit fullscreen mode

Verify that Docker is installed correctly:

sudo systemctl status docker
Enter fullscreen mode Exit fullscreen mode

Install Docker Compose
Download the latest version of Docker Compose:

sudo curl -L "https://github.com/docker/compose/releases/download/$(curl -s https://api.github.com/repos/docker/compose/releases/latest | grep -oP '"tag_name": "\K(.*)(?=")')" /usr/local/bin/docker-compose
Enter fullscreen mode Exit fullscreen mode

Apply executable permissions to the binary:

sudo chmod +x /usr/local/bin/docker-compose
Enter fullscreen mode Exit fullscreen mode

Verify that Docker Compose is installed correctly:

docker-compose --version
Enter fullscreen mode Exit fullscreen mode

Post-Installation Steps for Docker
Manage Docker as a non-root user:
Create the docker group if it doesn't already exist:

sudo groupadd docker
Enter fullscreen mode Exit fullscreen mode

Add your user to the docker group:

sudo usermod -aG docker $USER
Enter fullscreen mode Exit fullscreen mode

Now we can start up the application.
Ensure you are in the project root directory

cd devops-stage-2
Enter fullscreen mode Exit fullscreen mode

Start the application

docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

If you get a permission denied error, run is as superuser

sudo docker-compose up -d
Enter fullscreen mode Exit fullscreen mode

Docker compose up

Running curl localhost gives us a HTML response that Nginx proxy manager is successfully installed

Step 9

Reverse Proxying and SSL setup with Nginx proxy manager
Access the Proxy manager UI by entering http://:81 in your browser, Ensure that port is open in your security group or firewall.

Proxy manager login

Login with the default Admin credentials

Click on Proxy host and setup the proxy for your frontend and backend
Map your domain name to the service name of your frontend and the port the container is listening on Internally.

Proxy setup

Click on the SSL tab and request a new certificate

SSL

Now to configure the frontend to route api requests to the backend on the same domain, Click on Advanced and paste this configuration

location /api {
    proxy_pass http://backend:8000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

location /docs {
    proxy_pass http://backend:8000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}

location /redoc {
    proxy_pass http://backend:8000;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
}
Enter fullscreen mode Exit fullscreen mode

Advanced setup

Repeat the same process for

  • db.domain: to route to your adminer service on port 8080
  • proxy.domain: to route to the proxy service UI on port 81

You don't need to do the advanced setup on the db and proxy domain

Step 10

Setup Adminer
Access the adminer web interface on db.<your_domain>.com

Adminer login

Login with the db credentials in your backend .env file

Adminer Dashboard

Step 11

Setup Frontend Login
Access your frontend on <your_domain>

Frontend login

Before you login, make sure to change change the API_URL in your frontend .env to the name of your domain

VITE_API_URL=https://<your_domain>
Enter fullscreen mode Exit fullscreen mode

You would need to run docker-compose up -d --build to enable the changes to take effect

Your login should be successful now

Login success

Conclusion

We have now successfully:

  • Configured and tested the full stack application locally
  • Containerized the application
  • Setup Docker compose
  • Configured Adminer for Database management
  • Configured Reverse Proxying with Nginx Proxy Manager
  • Setup SSL certificates for our domains

Thank you for reading ♥
Happy Proxying! 🚀

Image
The Ultimate Guide to Awesome Self-Hosted Solutions

selfhosted, lowcode, opensource, github
Image
Where to Find E-Commerce Developers?

hiring, webdev
Image
Why Bitbucket Pipelines is Slow and How to Optimize

devops, bitbucket, ci, docker
Image
Advancing Maritime Operations with Machine Learning
Image
T-shirt Tuesday!

jokes, watercooler, discuss, tshirt
Image
Ivan Sulusulumaivasa - A Pillar of the Samoan Community

ivansulusulumaivasa, ivanalofa, airmediasamoa
Image
The Rise of Immersive Technology in Canada: Transforming Industries and Experience

ai, devops, testing, development
Image
Mastering Dynamic Editing in Flutter Charts

flutter, chart, development, desktop
Image
Top Professional Fire Safety Training in LA - Get Certified with HFSI

firesafety, losangeles
Image
The Future of B2B Marketing: Integrating AI with Technology Users Data

database
Image
🚀 Guía para principiantes: instalación y uso de GitHub CLI 🛠️

spanish, beginners, productivity, github
Image
Hello World.....Joining a new Tech Community. I will update you guys new things in my tech career. Thanks :)

webdev, beginners, programming, devops
Image
PMP academy

pmp, startup, agile, scrum
Image
Transforming the Maritime Industry with Blockchain Technology
Image
Secure proxy solutions for business
Image
Understanding Runes in Go

beginners, go, coding, programming
Image
(🇪🇸) Conclusiones del Stack Overflow Dev Survey 2024 - Tecnologías más demandadas

javascript, python, webdev, stackoverflow
Image
give me ideas for projetc
Image
Angular + Sanity CMS: Setting Up Your Sanity CMS Project with TypeScript

sanity, angular, typescript, webdev
. . . . . .
Terabox Video Player