Microservice Application Deployed on Kubernetes (EKS) : Video-to-Audio Converter

Table of Contents

  1. Project Overview

  2. System Architecture

  3. Prerequisites

  4. Installation Guide

  5. Database Setup

  6. Microservice Deployment

  7. API Documentation

  8. Troubleshooting

  9. Summary Steps

Project Overview

This project implements a distributed microservice application that converts video files to audio format. The system is deployed on Kubernetes using Helm charts and consists of four core microservices working together to provide authentication, file conversion, API gateway functionality, and email notifications.

System Architecture

Components

  • Auth Service: Handles user authentication using JWT tokens

  • Convert Service: Processes video files from queue and converts to audio

  • Gateway Service: API endpoint router (login/upload/download)

  • Notification Service: Sends email notifications

Data Flow

  1. User authenticates via Gateway Service

  2. Authenticated user uploads video through Gateway

  3. Video is placed in RabbitMQ queue

  4. Convert Service processes video and stores audio in MongoDB

  5. Notification Service sends email confirmation

  6. User downloads converted audio file

Infrastructure

  • Kubernetes Cluster (EKS)

  • PostgreSQL (User authentication)

  • MongoDB (Audio file storage)

  • RabbitMQ (Message queue)

  • Docker (Containerization)

Prerequisites

Tools Required

  • Helm (v3.x)

  • Python 3.x

  • AWS CLI v2

  • Docker

  • kubectl

  • PostgreSQL client

  • MongoDB client

AWS Resources

  • EKS Cluster with:

    • Cluster role (EKSAutoClusterRole) with policies:

      • AmazonEKS_CNI_Policy

      • AmazonEKSClusterPolicy

    • Node role (EKSAutoNodeRole) with required policies

  • Default VPC with public subnets

Installation Guide

1. Kubernetes Cluster Setup

# Connect to your EKS cluster
aws eks update-kubeconfig --name microservices --region ap-south-1

2. Install Mongo, Postgres, RabbitMQ

# Install MongoDB
cd ~/microservices-python-app/Helm_charts/MongoDB
helm install mongo .

# Install PostgreSQL
cd ~/microservices-python-app/Helm_charts/Postgres
helm install postgres .

# Same for the RabbitMQ

3. Network Configuration

Open these ports in security group:

  • 30002: Gateway Service

  • 30003: PostgreSQL

  • 30005: MongoDB

  • 30004: RabbitMQ Management

Database Setup

MongoDB Connection

mongosh mongodb://absk:root@<node-ip>:30005/mp3s?authSource=admin

PostgreSQL Setup

# Login Postgre
psql 'postgres://absk:root@<node-ip>:30003/authdb'

# Create auth table
CREATE TABLE auth_user (
    id integer GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    email VARCHAR (255) NOT NULL,
    password VARCHAR (255) NOT NULL
);

# Add sample user
INSERT INTO auth_user (email, password) VALUES ('youremail.com', '123456');

# Check table is created or not.
\d

# Now check the user.
SELECT * from auth_user;

RabbitMQ Configuration

  1. Access RabbitMQ management console at http://<node-ip>:15672

  2. Login with credentials: guest/guest

  3. Create queues:

    • Name: mp3 (Classic type)

    • Name: video (Classic type)

Microservice Deployment

1. Build and Push Docker Images

For each service (auth, convert, gateway, notification):

  1. Build Docker image

  2. Push to container registry

2. Deploy to Kubernetes

# For each service
cd ~/microservices-python-app/src/<service-name>/manifest/
kubectl apply -f .

API Documentation

Authentication

curl -X POST http://<node-ip>:30002/login -u youremail.com:123456

Response: JWT token for authenticated requests

File Upload

curl -X POST -F 'file=@./video.mp4' \
  -H 'Authorization: Bearer <JWT_TOKEN>' \
  http://<node-ip>:30002/upload

Response: Success message and file ID (sent via email)

File Download

curl --output audio.mp3 -X GET \
  -H 'Authorization: Bearer <JWT_TOKEN>' \
  http://<node-ip>:30002/download?fid=<FILE_ID>

Troubleshooting

Common Issues

  1. Connection refused errors:

    • Verify security group rules

    • Check if pods are running (kubectl get pods)

  2. Authentication failures:

    • Verify PostgreSQL user table exists

    • Check JWT token expiration

  3. File processing stalls:

    • Check RabbitMQ queues

    • Verify Convert Service logs (kubectl logs <pod-name>)

Summary Steps

  1. On your main server, install the important tools: Helm, Python 3, AWS CLI v2, Docker, kubectl, PostgreSQL, and MongoDB.

  2. Now create an EKS cluster.

  3. Now connect the cluster to the main server using CLI.

  4. Now deploy MongoDB, PostgreSQL, and RabbitMQ on the cluster.

  5. Now connect to PostgreSQL. After logging in to PostgreSQL, just create a table.

  6. Now access RabbitMQ by hitting IP:port and create queues for mp3 and video.

  7. Now create a Docker container for every microservice and push it to the Docker registry or Docker Hub.

  8. Now use the manifest of each service and apply it. When we apply it, pods are created for the services.

  9. Now we can upload and download the converted file.

Project Repository
Updated on
Project Images
Project Images