Polyglot Architecture
In the fast-paced business environment, organizations often struggle to keep up with the ever-evolving demands of technology and business. A single technology stack or programming language may become insufficient, making it crucial to adopt a polyglot architecture to meet the changing needs of the business. A polyglot architecture makes it easier for engineers to decide on which coding language and infrastructure to choose from. It can lead to increased agility, faster time to market, and better outcomes for the business. Organizations can tap into a wider pool of talent and play in almost every space, including big data, IoT, AI, cloud computing, and data analysis. Let’s deep dive and understand the needs, benefits, and challenges associated with a successful implementation.
Understanding the Need?
The main idea behind a polyglot architecture is it makes it easier for engineers to decide on which programming language and infrastructure to use. Many large technology companies have developed to support this type of microservices architecture. Before exploring the concept of a polyglot architecture further, it’s crucial to recognize why an enterprise may want to pursue a multi-language microservice ecosystem.
Business Needs
The demand for advanced computing devices and complex business requirements has pushed services to keep up with constant growth. The trend of moving from a monolithic to a microservices-based system has already passed, and now the focus is on extending the stable power of microservices within a distributed system. This can be seen in implementing measures such as a service mesh for smarter communication between services, distributed tracing for monitoring requests, and canary rollouts for traffic management.
Platform Mindset
As part of the platform team, our responsibility is to create the foundation and necessary tools for the wider organization. Despite supporting multiple groups across various lines of business, we prioritize taking a structured and formal approach to the development of microservices that cater to our traffic. Our ability to deliver personalized user experiences can be limited when supporting multiple teams that use different languages, runtimes, and ecosystems.
Instead of simply hiring more engineers, we are interested in exploring the idea of reducing the effort required for our current developers to create and deliver their microservices using polyglot architecture.
Benefits :
Business Need
- A polyglot architecture helps address the diverse needs of businesses, making it a good fit for both enterprises and startups.
- It enables organizations to provide solutions to meet the demands of big data, IoT, AI, cloud computing, and data analysis.
Engineering Excellence
- Guidelines and standards for microservice development while empowering engineers to make decisions on which language to use.
- This leads to an increase in the scope of creativity and innovation, resulting in better solutions for the business.
Release Velocity
- Ease in choice of technology makes it even easier to increase agility and expedite time to market.
Skilled workforce
- Allows organizations to tap into a wider pool of talent, and limits the barrier of programming language.
Challenges:
Technical Complexity
- Polyglot microservices can add significant complexity to a system, which may require a deeper understanding of libraries and purposeful development of every new service.
- Coordination and consensus between teams must be managed, and strong guardrails may be necessary to balance autonomy and development goals.
- Checklists and thorough documentation are a must.
Scaling Issues
- Containerization makes it easier to build microservices with multiple technologies, but this comes with costs of ownership and maintenance.
- Standardizing services in multiple languages means solving the same problems across all used languages, and also accounting for framework and dependency updates.
Cooking the Perfect Recipe: To Coder’s Delight
At Intuit for our marketplace (sharing) solution, we are building on the lines of polyglot architecture. Let me list the perfect ingredients that ensure a smooth and seamless experience to delight the customers.
- Governance and Schema standardisation
With the growing engagement across multiple teams, Schema governance and standardization becomes a mightier challenge.
Proper practices should be in place where microservices and respective teams can leverage the data from shared pools and datalakes to build meaningful insights to delight the customers. - Loosely Couple Data
With loosely coupled data, standardization changes can be easily managed and scaled at the same time. - Automation
This is paramount to polyglot architecture. Automating various layers in a service, db layer using Liquibase, Flyway, testing layers using Component tests, and other testing frameworks available in the market. Application deployment using docker, Kubernetes, etc, helps to expedite and automate the work. - Observability
Monitoring and operational excellence using solutions like Splunk, Wavefront, NewRelic, AppDynamics, etc, and setting up alerts for governance, standardization, and health checks can ensure smooth and successful implementation - Reusable Structural Patterns
We use Hexagonal Architecture (aka Ports and Adapters) that lets users and external systems input data through an Adapter into a Port and lets the Application output data through a Port to an Adapter. This creates a protective layer for the core application, isolating it from external tools and tech. It’s modular and flexible design that separates the business logic from the infrastructure, providing a clean, maintainable, and testable design that can be easily updated and modified. For instance, we use a flavor of NoSql + Search for our service where backend data is created/updated/deleted in NoSql (typically Cassandra/dynamo DB) and is ingested to Search which is powered by ElasticSearch. Schema standardization and governance help to achieve this pipeline setup for any new offerings in a matter of minutes. Essentially at the application level, ports and adapters are technology agnostic and are truly designed as interfaces to implement functionality on the need basis.
Conclusion
Polyglot architecture enables the use of multiple technology stacks to provide diverse solutions and accelerates time to market. Although it has challenges like managing complexity and standardization, using strategies such as a loosely coupled codebase, framework, automation, and monitoring solutions can help. Polyglot architecture is crucial for staying competitive in the rapidly evolving business landscape and satisfying modern business demands by giving engineers the freedom to innovate.
