ErlyTalk | Build a highly scalable distributed messaging application with Erlang and Kubernetes on AWS

A few days ago I developed a simple distributed messaging application using Erlang and built a fully automated, highly scalable, and well-designed infrastructure on AWS using Terraform and Ansible.

The entire project codebase is available on GitHub. This project has three components, each residing in separate repositories:

• ErlyTalk – Infrastructure as Code (IaC)
• ErlyTalk – Messaging Service (Backend)
• ErlyTalk – Web (Web interface)

Let’s dive into each component individually:

Infrastructure as Code

It provides a fully automated end-to-end deployment of the ErlyTalk Messaging Service and Web application on AWS. It leverages Terraform and Ansible to provision and configure the necessary AWS resources, allowing for seamless scaling based on the load.

• The various AWS services such as VPC, EC2, Route 53, S3, DynamoDB, CloudWatch, EKS, and IAM are utilized for the infrastructure.
• The infrastructure is provisioned by using Terraform.
• Deployment of micro-services on EKS (Kubernetes) is carried out using Ansible.
• Automatic scaling of the EKS cluster is facilitated by Kubernetes Autoscaler.
• Container Insights is used to monitor the EKS cluster, providing valuable insights into its performance and health.
• DNS records for EKS Ingress and Services are created in Route 53 using ExternalDNS.
• Provisioning of ALB/NLB for EKS Ingress and Services is handled by AWS Load Balancer Controller.

Messaging Service

It’s a distributed WebSocket server in Erlang that handles incoming frames and dispatches them to the designated destination. It provides a scalable and fault-tolerant solution for real-time communication over WebSocket connections in a distributed environment.

• Messages between the Erlang nodes cluster are routed by the Distributed Erlang System.
• Users’ process IDs (PIDs) are stored and synchronized across the Erlang nodes cluster using Syn.
• The HTTP server is handled by Cowboy.
• New Erlang nodes are being discovered automatically by using Kubernetes headless service.

Web

It’s a front-end application in Node.js for the ErlyTalk Messaging Service. It provides a user-friendly interface for real-time communication over WebSocket connections.

Conclusion

By combining Erlang’s concurrent programming capabilities with the power of Kubernetes, we have built a highly scalable and distributed messaging application. The Erlang-based messaging service ensures efficient message handling and fault tolerance, while the Kubernetes-based infrastructure guarantees seamless scalability.