3 Challenging Microservices Transaction Management Scenarios – How to Overcome Them
December 30, 2019
Microservices that deal with the Single Responsibility Principle (SRP) raise frequent challenges when it comes to ease of distributed data management. As there are no rock-solid approaches that work in all situations – at least not yet – it becomes necessary to experiment and question scenarios to arrive at the desired resolution.
Where does the data need to be persisted? In a different schema or maybe in other external sources? Are there any additional data sources to be added in the next year or so? If so, how does the rollback work?
Are you, by any chance, writing a ‘touch me not’ code? Is that leading to an ever-extending quality gate cycle, which is eventually affecting your confidence levels? Happens to the best of us.
Here are our recommendations for three scenarios that were resolved holistically to enable effective transaction management in microservices.
Scenario 1
Data to be persisted in multiple schemas, and one DB data source connection (RDBMS).
Recommendation
Manage transactions at the data layer itself. Use simple stored procedures that can access multiple schemas and tables, proceed for the transaction, and finally execute the commit.
Assumption
You have the appropriate access rights to make a transaction.
Code Complexity
A single microservice will own the stored procedure integration have, and everything will be managed at the data layer. This will not only make it easy to maintain the microservices in the long term, but any additional transaction enhancements can also be accommodated without needing a code deployment.
Atomicity
100% transaction management will be achieved, and immediate responses can be shown to the end-customer.
Scenario 2
Different types of data sources that require different ways to connect. Presence of external data sources and NoSQL. Not expecting an immediate response.
Recommendation
Eventual transaction persistence. Use data message streams like Kafka/RabbitMQ that will listen to the topic and reliably persist them to the data source listeners.
Code Complexity
Any failure in the data source listeners will create periodic retry queue topics. This approach line up any additional transactions for future accommodation.
Atomicity
With this approach, there will be eventual persistence, which means that the end-customer will not have real-time access to view the status of a transaction but will need to be notified via email/notification once the persistence is carried out.
Scenario 3
Different types of data sources and some are external services that have less control over them — expecting immediate responses.
Recommendation
Orchestration or choreograph based transaction management. Use Saga Pattern or declarative Spring transaction manager with custom implementations of success/rollback services.
Code Complexity
Say, if we have three transactions to be performed, each will require three more rollback services. Additionally, the transaction will be managed by well-choreographed services.
Atomicity
The success of transactions will be less than 100%. This may lead to an added level of operational complexity – a customer service process to handle end-user queries if/when the rollback services don’t work.
Conclusion
Microservices have gained tremendous popularity across industries and have significantly influenced the software system development so far. Unlike monolithic applications, a microservice may involve many dependencies that make maintenance of data consistency challenging. This, in turn, impedes the seamless creation of interdependent transactions across microservices.
We’ve worked on projects where the clients needed us to build a custom transaction framework for handling their complex needs. This route is expensive and takes time to mature, but that’s precisely what they needed. We’ve also handled projects where all that the clients required were simple pre-designed frameworks with a few tweaks to make it their own.
The key is to think about how transactions occur and which kind makes sense for your offerings. Need help to understand what route you should take? Schedule a free consultation with our experts now.
