Understand the challenges involved in adopting microservices and the strategies to mitigate them
The ever-changing business needs of the industry have necessitated the growing need of the right tools and technologies and the evolution of design and architecture methodologies. Keeping pace with the new development approaches such as DevOps has been a challenge in monolithic applications.
Microservices architecture is a result of this growing demand and has become immensely popular over the past few years. It is a variant of service-oriented architecture that structures an application as a collection of loosely coupled, lightweight services that are maintainable, testable, independently deployable and organized around business capabilities. This article examines the strategies, methods and best practices in transitioning a monolith to a microservice.
From monolith to microservice
There are two approaches to developing applications using microservices architecture. These include the following:
1.) Build the application using microservices architecture from the beginning
2.) Build a monolith and the migrate it gradually to microservices
Migrating an existing monolithic application to microservices is a journey that's worthwhile, but not at all an easy one. Transitioning to microservices creates significant challenges for the organizations as well.
Okay, so what are the challenges?
To reap the benefits microservice architecture, you should choose the right strategy, communicate the expectations and requirements to the team members and mitigate the challenges involved in transitioning an existing monolith to a microservice.
The challenges faced in transitioning a monolith to a microservice can broadly be classified into the following:
Decomposition – Microservices architecture is complex than legacy applications. To handle this complexity and reduce the risks involved, you should use the right tools and technologies in place. In transitioning an existing monolith to a microservices, you would typically need to decompose the existing application into granular microservices. During this transitioning process, you would typically need to decompose the monolith to building more and more granular microservices to suit the business needs. Once this is accomplished, there would be more moving parts in the application. As a result, this would lead to operational and infrastructural overheads, i.e., configuration management, security, provisioning, integration, deployment, monitoring, etc. One of the ways to reduce these complexities is in using containerization. In using containerization, provisioning, configuration, and deployment of microservices would be simplified.
Testing - Testing microservices can become challenging, particularly the integration tests. To write an effective integration test case, the QA engineer should have good knowledge of each of the services that are a part of the solution. Another reason why testing a microservices-based application is difficult is because microservices-based applications are generally asynchronous. The best approach to solving these testing challenges is adopting various testing methodologies and tools and leveraging continuous integration capabilities through automation and standard agile methodologies.
Performance - The increase in resource usage may cause a microservices-based application to execute slower. You can overcome this challenge by introducing additional servers. Logging in any application can be a part of the solution - you can log performance data and detect the problems. You should take advantage of logging to store performance data in a repository so that you can analyze the data at a later point of time. You can implement throttling, handle service timeouts, implement dedicated thread pools, implement circuit breakers and take advantage of asynchronous programming to boost the performance of microservice-based applications.
Security - In a typical microservices architecture, communication between the services can be in the same or different machines - even between different data centers. Due to this complexity in the communication, security in a microservices-based application is important to authorize access to a protected resource.
In a monolith the facade pattern to aggregate the data that is retrieved from multiple services. On the contrary, in a microservices-based application, the API Gateway is used for the same purpose. The API Gateway pattern can be used in a microservices-based application to secure access to the microservices by abstracting the underlying microservices from external clients. The other strategies that are adopted include implementation of SSL, OAuth, and containerization.
Inter-service communication - In a distributed system, the components should be able to communicate with each other and in microservices-architecture, it is no exception. In a monolithic application, the components invoke one another through method or function calls. In contrast, a microservices-based application is distributed in nature with each service running isolated from another service. Hence, it is imperative that services in a microservice-based application communicate using inter-process communication (also known as IPC) mechanisms. However, inter-service communication in a microservices-based application poses a lot of challenges.
Since microservices are distributed in nature, remote invocation of these services is a challenge and you should understand the necessary patterns to overcome the challenges involved. Since services in a microservice-based application communicate with one another using IPC mechanisms you should consider issues like, how the services will interact, how to handle failures, etc.
Organizational challenges – There are certain organizational challenges involved in moving from monolithic architecture to microservice architecture as well. One such challenge is the organization's structure. An organization that builds a monolithic application typically have large teams with distinct roles, i.e., development, quality assurance, testing, database administration, etc. For microservices-architecture to be successful, there should be good communication between the team members and the stakeholders.
Conway's law states: "Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure." So, it is evident that such organizations design systems in which the structure of the architecture is a replica of the organization's structure. This kind of structure results in slower development cycles due to the hand-offs involved. You can learn more about hand-offs from this article: https://techbeacon.com/app-dev-testing/how-agile-teams-can-minimize-hand-offs
Persistence – A monolithic application typically contains a single data store. In contrast, a microservices-based application contains multiple databases, typically one for each service. One of the key challenges in transitioning to microservices-based architecture is in understanding and handling how decentralized data management.
Splitting the data model of a monolithic application to fit the autonomous data models of a microservices-based application is challenging. Breaking a monolith data model into separate autonomous data models that are local to each microservice is a daunting task primarily because of the need to maintain data and transactional integrity. Another challenge with a microservices-based application is querying data that spans across multiple services - it was a piece of cake with the monolith-based approach, wasn't it?
Deployment – Deployment of a microservices-based application can be challenging. Unlike a monolith where you typically deploy one application per host/machine and then scale the application vertically, you can deploy a microservices-based application using various patterns, i.e., Multiple service instances per Host, One Service instances per Host or One Service instance per Container.
Operations overhead for running so many services and version management are two of the challenges. You should have an infrastructure management strategy in place and have automated, proactive monitoring of the infrastructure from time to time.
Transaction management - Transaction management in a monolith is simple as it involves in-process invocations. On the contrary, transaction management in a microservices architecture is not straight forward. Each and every microservice maintains its own private database. Added to this, microservices are stateless in nature and hence to fulfill a request there might be a need for several remote invocations. One way to address this challenge is through event-based programming - you can trigger events and notify a microservice when there is a change of state.
Summary
An organization willing to transition from monolith to microservices should evaluate the cost and the benefits of the transition. Albeit the fact that adoption of microservice in most cases can facilitate rapid application development, continuous integration and faster releases, for some organizations this can also be detrimental, i.e., the challenges might outweigh the rewards.
