Monolith to Microservices - PGP CC Capstone Project

Team:

Application: – An Online Shopping Portal

Current Business Context and Challenges:

The monolith system is not scalable to the number of user hits and application will tend to crash in the peak load
The legacy system doesn’t support high availability, scalability, automation and reliability
The legacy system lacks interoperability with newer technologies and integration with applications based on new technologies is often expensive in terms of time and efforts
Being a monolith, more frequent releases are always a challenge to deal with
Limitations in security with the current architecture and design
Frequent upload and maintaining product catalog has been a challenge due to lack of automation
Faster to Market – Continuous integration and ease of deployment

The current ecommerce web application is a big monolith application with a legacy front end and Java/Spring services. A single database backend is unable to meet the scaling needs of the deployment. Querying data is a big bottleneck as it gets slower and slower over time. This issue along with the increasing user base forced us to think of a better solution and move to container-based deployment to auto scale and manage resources better in the cloud.

Current Technical Pain points:

The legacy Java monolith application is not scalable, and it is one fat JAR which has to be deployed every time. Due to the size of the application the start time and application initialization time is adversely affected. Furthermore, additional memory pressure among high other system requirements is forcing the team to think about a strategy to control these issues in the long run.

Functional Specification:

Actors:

Users/End Consumers
Seller/Employee/Managers

Modules:

Home Page
Product and Details Page
User Registration and User Login
Manage Carts and Orders (View, Edit and cancel)
Manage Products and inventory – Employee Role
Manage Order clearance and shipping

Use Case: Product Search and Order processing

Proposed Solution:

To circumvent the technical pain points and the business need for a highly scalable solution the monolith application has been broken into 5 microservice bounded contexts (Products, Orders, Users, Cart and Reports). AWS is chosen as the cloud platform due to the ease of use and the cost benefit associated with it for corporates.

Proposed Technology Stack:

Angular 9, Java Spring Boot, PostgreSql, Docker. For AWS services – Refer separate section below

Solution Approach:

Identify core functional business context from existing application and also from eCommerce context
Identify Services which exist and can be containerized as is
Identify the new set of services
Identify different database and entities required for each microservice
Identify how data consistency can be maintained across different databases/microservices
Implement Pub/Sub as the primary mode of asynchronous calls
Identify and implement microservice best practices
Implement serverless leveraging managed services so that the manual intervention and management of those services can either be eliminated or reduced substantially
Identify strict security policies and enforce an appropriate A&A model so that the data is secured and also encrypted throughout the transmission and at rest

Design Principles:

Domain Driven Design – Enabled by implementing Microservices based architecture – Modular routines for each of the functionalities – User, Product, Cart, Order and Reports
Cloud First – Building a highly scalable, resilient, cost effective infrastructure using cloud
Microservices architecture – Aligning each service to have it’s own database
Scalable solution – Implementation of containerized application using Dockers and ECS to enable horizontal scaling
Secured Application – Enabled by API Gateway, Lambda Authorizers, Security group & ACL’s
Caching and Performance– Caching enabled by Redis cache for faster search
Faster to Market – DevOps integration with CI/CD and Code Pipeline
Event driven integration – Communication between decoupled services
Operational excellence – Elastic search and Kibana integration to view logs and errors

Key Characteristics of Architecture:

Microservices implementation using Fargate/Auto Scaling
Highly scalable, available, and secured architecture
Private Load balancer: . We have implemented the NLB in a private subnet and integrated with API gateway through VPC links. By this we have made our Load Balancer URL not accessible publicly rather all requests routed securely through API gateway.
Lambda Authorizer and API gateway implementation: Custom Lambda Authorizer was implemented as an additional security around API invocation. This was written in Node JS to validate the header for the JWT token and make sure that the authenticated calls get to the ECS cluster.
Data Consistency – Saga Pattern with Lambda – Choreography model. Eventual consistency is maintained across services through the Saga pattern implementation with SQS implementation.

AWS Components/Services Used:

Networking: VPC, Subnets, IGW, Route Tables, Elastic IPs, NAT Gateway, NACL
Storage: RDS, Redis, Elasticache, DynamoDB
Compute: Monolith Application (Microservices, Screenshots & Java Spring boot source code), ECR, ECS, Autoscaling, EC2, Load Balancer, Target Groups, Security Group for ECS
Serverless Functions and Managed Services : Lambda
Other Services used: IAM Policy and Roles, Security Groups, Policies, SQS, SNS, SES, API Gateway API Gateway Configuration, Lambda Authorizer Security and Management, Secrets Manager, Parameter Store, CloudWatch, CDN, CloudFront, S3, Route 53, ACM, Devops, Athena, Elasticsearch and Kibana.

This brings us to the end of the capstone project, if you wish to upskill in this domain, join Great Learning’s PGP Cloud Computing Course and build your dream career. You will have access to live mentorship sessions and can work on exciting capstone projects as well.