One of the most vital and oldest departments of any national or state government in the world is taxation authority. Taxes are the government’s revenue, which is needed for social welfare, infrastructure projects, and sustaining the economy of the country. In this article, I will explain how to convert from batch-based to microservices using this tax system use case as an example.
Apart from being the oldest, this is one of the departments which needs to be very agile due to frequently changing national and international policies. However, in a majority of countries, the current taxation systems were developed a decade or two ago. Any new changes are implemented in silos and in a haphazard manner, which has made the whole system very rigid and very difficult to evolve alongside changing needs.
Traditionally, the tax calculations system is implemented using a monolithic architecture with a mix of SOA for service orchestration. In most cases, it is a very tightly coupled batch-based architecture as shown in the figure below. Because of this, the response time is prolonged. It takes at least 2-3 weeks for tax returns to be calculated and sent to the individuals/corporations. Also, adding a new taxation policy at the national level requires months of implementation and testing.
An example of a tax system using batch-based architecture
Before we convert this into microservices, here is a little refresher on the benefits of event-driven architecture:
What is Event-Driven Architecture and Why Is It Useful?
In the last few years, there has been a significant drive towards moving from monolithic architecture to event-driven architecture (EDA).
“Event-driven architecture (EDA) is a software architecture paradigm promoting the production, detection, consumption of, and reaction to events.” -Wikipedia
In other words, event-driven architecture is an IT strategy that has evolved from SOA, where the flow of events is consumed and produced by microservices, functions, applications in real-time for business logic, insights, AI/ML, and IoT scenarios.
From a business perspective, an event-driven architecture can allow for better customer experiences and quicker times to market. Consider this: a business reacting to real-time data can be more proactive and responsive when it comes to its customers. Using real-time data in unique ways can give a business an edge over its competition, allowing for a quicker response to changing market demands and customer expectations. For example, an airline’s loyalty program could be notified when a check-in event occurs, resulting in a chain of events that includes a congratulatory beverage upon boarding. A check-in event could also trigger other systems that track the likelihood of a plane departing on time. In this ever-changing landscape, new use cases for real-time data can be the key to a competitive advantage that wasn’t previously considered due to technical limitations.
From a technical perspective, being event-driven provides better agility and easy adoption of cloud-native services such as Google Beam and AWS Lambda functions, thus providing an innovative backbone for new business use-cases.
The advantages of event-driven architecture over traditional batch-based architecture.
Now let’s get back to the tax example.
How to Convert from Batch-Based to Microservices – A Tax Assessment System
The high-level component diagram shown above displays three main monolithic applications for doing the tax assessment. They are:
- Process Tax Assessment
- Generate Notice of Assessment
- Generate Correspondence
The monolithic, batch-based tax processes can convert into various event-driven microservices that support specific business functions, as shown in the figure below. Once that happens, they still need to communicate and exchange information – this is done by using an event broker. An event broker helps with the choreography of microservices, which can also be seen in the diagram:
Monolithic batch based tax system applications converted to event-driven microservices with an event broker.
Achieving Elastic Scalability, Load Balancing and Throttling
One of the significant functionalities of an event broker is its ability to dynamically determine if a new consumer is added for the same subscription. Accordingly, it can also load balance the events between the multiple instances of the same consumer. Similar to how REST APIs can do auto-scale-up using API gateways, there can be multiple instances of the same event-driven microservice listening on the same queue and providing parallel processing.
This mechanism allows the flexibility to have in-process auto-scale-up and auto-scale-down depending on the load on the system. Alternatively, we can individually auto-scale only those services which are having a higher load.
An event broker should also be able to throttle the consumer traffic based on how many events a consumer is able to process. It should be able to do ‘weighted round robin’ delivery of the messages. Weighted round robin essentially means keeping track of the TCP acknowledgement from each of the consumers and intelligently pushing the traffic to the consumer based on its performance.
Monolithic batch based tax system applications converted to event-driven microservices with an event broker.
Microservices Serving Independent Tax Functions Within a Business Environment
Now, these microservices can be further divided based on more fine-grained business functions. For example, the “Calculate Tax” microservice can be divided into:
- Calculate Property Tax
- Calculate Individual Income Tax
- Calculate GST
Similarly, validation services would have different logic for different types of taxes.
Once the granularity of microservices has been decided, an event broker can be used to provide choreography of the microservices as shown in the figure below:
Microservices are divided into business functions.
There is a separate microservice for various business functions. This allows one microservice to change, scale, and be reused without affecting any other services.
The Advantages of Event-Driven Microservices over Monolithic Architecture
In a monolithic architecture, the example above would be a separate “if-else” function in the application logic. One can certainly argue that the execution is much faster in this case since there is no network latency (as there would between two microservices), and implementation is quick and easy from the developer’s point of view; however, implementing this as separate microservices has its advantages:
- It provides separation of logic/concerns – a business unit can have full control over implementation of its own business function
- Changes in one business function may not affect another
There is a delicate balance that needs to be maintained, taking into consideration all the factors outlined above, when deciding the granularity of a microservice.
While there may be many microservices with synchronous request-reply patterns which are powered by RESTful APIs, there would be many event-driven microservices that would publish and subscribe to events. Thus, these microservices would be loosely coupled using the power of the topic routing feature of the advanced broker.
Choreographed Microservices – The Power of Topic Routing
Each of the microservices in the tax example emits an event – essentially a message with a topic attached to it. A topic defines the event; it identifies what the event represents. For instance, a calculated private property tax assessment event can have a topic as “tax/calculated/PT/private”. A topic can even be more descriptive if required, including things like version number, tax assessment year, etc. (eg. tax/v1.0/2019/calculated/PT/private).
To learn more about creating a sound topic hierarchy, I encourage you to read this post: Topic Hierarchy and Topic Architecture Best Practices
An event has two parts: topics and payload. A payload is a message itself which would be processed by the service. You can read this older post for more about payload data.
As mentioned earlier, a microservice can publish and consume one or more events. For instance, the “Retrieve Tax Returns” service would be consuming events from the topic “tax/assess/”. It would also publish events on two separate topics for different types of taxes such as: “tax/retrieve/PT” for property tax and “tax/retrieve/CT” for corporate tax.
These events will, in turn, be subscribed to by “Validate Property Tax” and “Validate Corporate Tax” respectively.
In this way, all microservices can be choreographed, as shown in the figure below, only starting its dance (or execution) when an event the service has subscribed to is published.
Choreographed microservices for a tax system.
This architecture provides agility and flexibility, since adding a new microservice will not affect existing functions. New microservices can be added without bringing down the whole system, which would be the case in a monolithic architecture or even with a SOA-based service bus, which uses the orchestration of the services.
In the figure below, we see how audit/logging/error handling microservices can be added seamlessly:
Adding new microservices does not affect existing functions.
Visualizing and Managing Your Events Using Event Portal
As an enterprise starts getting more and more events, managing, discovering, and utilizing those events will eventually become a challenge for the organization. Enterprise architects will have to answer some tough questions such as:
- Who produces and consumes each event?
- What is the event payload format?
- Where do we go to discover events?
- When will a given event be available or deprecated?
- Why does a given event exist – context?
- How do we define event-driven applications?
- How do we generate code?
If events and microservices are not governed appropriately, it won’t be long when the talks within the organization sound a little like this:
Source: https://devops.com/event-driven-environments/
An event portal provides developers and architects tools to design, describe, and discover events within their system. It helps them see the relationships between applications and events, making event-driven applications and microservices easier to design, deploy, and evolve.
From our tax example above and using the PubSub+ Event Portal design space, we can design the following:
The PubSub+ Event Portal can be considered a design-time component, which allows architects to view and design applications while being mindful of their relationships. You can read more about it here or watch this demo to get started.
Wrapping-Up
Event-driven methodology is still quite new and is rapidly making its way across industries. As more and more data is captured, new and innovative use cases are constantly being discovered. An enterprise has a lot to gain from moving towards an event-driven architecture. More agility, better flexibility, reduced time to market, and improved customer experiences are business objectives that go hand-in-hand with a proper foundation for event-driven microservices.
For more on microservices and PubSub+ Event Portal, bookmark these sections on our Resources Hub to keep up to date on a variety of helpful content:
https://solace.com/resources/solace-microservices-resources
https://solace.com/resources/pubsub-event-portal
Explore other posts from categories: For Architects | For Developers
Subscribe to Our Blog
Get the latest trends, solutions, and insights into the event-driven future every week.