How to Implement a Distributed Transaction Across Mysql, Redis, and Mongo | by dongfu ye | Apr, 2022

Combining a number of storage engines to kind a distributed transaction

Photograph by Jan Antonin Kolar on Unsplash

Mysql, Redis, and Mongo are all highly regarded shops, and every has its personal benefits. In sensible functions, it is not uncommon to make use of a number of shops on the similar time, and guaranteeing knowledge consistency throughout a number of shops turns into a requirement.

This text provides an instance of implementing a distributed transaction throughout a number of retailer engines: Mysql, Redis, and Mongo. This instance is predicated on the Distributed Transaction Framework and can hopefully assist to resolve your issues in knowledge consistency throughout microservices.

The power to flexibly mix a number of storage engines to kind a distributed transaction is firstly proposed by DTM, and no different distributed transaction framework has acknowledged the power like this.

Suppose {that a} person is now collaborating in a promotion: they’ve a steadiness, recharge the telephone invoice, and the promotion will give away mall factors. The steadiness is saved in Mysql, the invoice is saved in Redis, and the mall factors are saved in Mongo. As a result of the promotion is restricted in time, there’s a chance that participation might fail, so rollback assist is required.

For the above downside situation, you need to use DTM’s Saga transaction, and we’ll clarify the answer intimately beneath.

Step one is to organize the information. To make it simpler for customers to shortly get began with the examples, we’ve got ready all knowledge at, which incorporates Mysql, Redis, and Mongo, and the particular connection username and password might be discovered at dtm-labs/dtm-examples.

If you wish to put together the information setting domestically your self, you need to use dtm-labs/dtm/blob/main/helper/ to start out Mysql, Redis, Mongo; after which execute scripts in dtm-labs/dtm/tree/main/sqls to organize the information for this instance, the place busi.* is the enterprise knowledge and barrier.* is the auxiliary desk utilized by DTM

Let’s begin with the enterprise code for essentially the most acquainted retailer engine Mysql

The next code is in Golang. Different languages equivalent to C#, PHP, Java might be discovered right here: [DTM SDKs](

func SagaAdjustBalance(db dtmcli.DB, uid int, quantity int) error 
_, err := dtmimp.DBExec(db, "replace dtm_busi.user_account set steadiness = steadiness + ? the place user_id = ?" , quantity, uid)
return err

This code primarily performs the adjustment of the person’s steadiness within the database. In our instance, this a part of the code is used not just for Saga’s ahead operation but additionally for the compensation operation, the place solely a unfavorable quantity must be handed in for compensation.

For Redis and Mongo, the enterprise code is dealt with equally, simply incrementing or decrementing the corresponding balances.

For the Saga transaction sample, when we’ve got a short lived failure within the sub-transaction service, the failed operation shall be retried. This failure might happen earlier than or after the sub-transaction commits, so the sub-transaction operation must be idempotent.

DTM gives helper tables and helper capabilities to assist customers obtain idempotency shortly. For Mysql, it is going to create an auxiliary desk barrier within the enterprise database, when the person begins a transaction to regulate the steadiness, it is going to first insert Gid within the barrier desk. If there’s a duplicate row, then the insertion will fail, after which skip the steadiness adjustment to make sure the idempotent. The code utilizing the helper operate is as follows:

app.POST(BusiAPI+"/SagaBTransIn", dtmutil.WrapHandler2(func(c *gin.Context) interface 
return MustBarrierFromGin(c).Name(txGet(), func(tx *sql.Tx) error
return SagaAdjustBalance(tx, TransInUID, reqFrom(c).Quantity, reqFrom(c).TransInResult)

Mongo handles idempotency in the same solution to Mysql, so I received’t go into element once more.

Redis handles idempotency in another way than Mysql, primarily due to the distinction within the precept of transactions. Redis transactions are primarily ensured by atomic execution of Lua. The DTM helper operate will alter the steadiness through a Lua script. Earlier than adjusting the steadiness, it is going to question Gid in Redis. If Gid exists, it is going to skip the steadiness adjustment; if not, it is going to document Gid and carry out the steadiness adjustment. The code used for the helper operate is as follows:

app.POST(BusiAPI+"/SagaRedisTransOut", dtmutil.WrapHandler2(func(c *gin.Context) interface 
return MustBarrierFromGin(c).RedisCheckAdjustAmount(RedisGet(), GetRedisAccountKey(TransOutUID), -reqFrom(c).Quantity, 7*86400)

For Saga, we additionally must take care of the compensation operation, however the compensation isn’t merely a reverse adjustment, and there are lots of pitfalls that ought to pay attention to.

On the one hand, compensation must take idempotency under consideration, as a result of the failure and retries described within the earlier subsection additionally exist in compensation.

Then again, compensation additionally must take “null compensation” under consideration, for the reason that ahead operation of Saga might return a failure, which can have occurred earlier than or after the information adjustment. For failures the place the adjustment has been dedicated, we have to carry out the reverse adjustment, however for failures the place the adjustment has not been dedicated we have to skip the reverse adjustment.

Within the helper capabilities offered by DTM, on the one hand, it is going to decide whether or not the compensation is a null compensation based mostly on the Gid inserted by the ahead operation, and then again, it is going to insert Gid+’compensate’ once more to find out whether or not the compensation is a reproduction operation. If there’s a regular compensation operation, then it is going to execute the information adjustment on the enterprise; if there’s a null compensation or duplicate compensation, it is going to skip the adjustment on the enterprise.

The Mysql code is as follows.

app.POST(BusiAPI+"/SagaBTransInCom", dtmutil.WrapHandler2(func(c *gin.Context) interface 
return MustBarrierFromGin(c).Name(txGet(), func(tx *sql.Tx) error
return SagaAdjustBalance(tx, TransInUID, -reqFrom(c).Quantity, "")

The code for Redis is as follows.

app.POST(BusiAPI+"/SagaRedisTransOutCom", dtmutil.WrapHandler2(func(c *gin.Context) interface 
return MustBarrierFromGin(c).RedisCheckAdjustAmount(RedisGet(), GetRedisAccountKey(TransOutUID), reqFrom(c).Quantity, 7*86400)

The compensation service code is nearly an identical to the earlier code of the ahead operation, besides that the quantity is multiplied by -1. The DTM helper operate mechanically handles idempotency and null compensation correctly.

When writing ahead operations and compensation operations, there’s truly one other exception known as “Suspension”. A world transaction will roll again when it’s a timeout or the retries have attain the configured restrict. The conventional case is that the ahead operation is carried out earlier than the compensation, however within the case of “course of suspension” the compensation could also be carried out earlier than the ahead operation. So the ahead operation additionally wants to find out whether or not the compensation has been executed, and within the case that it has, the information adjustment must be skipped as nicely.

For DTM customers, these exceptions have been dealt with gracefully and correctly and also you, as a person, want solely comply with the MustBarrierFromGin(c).Name name described above and don’t must care about them in any respect. The precept for DTM dealing with these exceptions is described intimately right here: Exceptions and sub-transaction barriers

After writing the person sub-transaction providers, the next codes of the code provoke a Saga world transaction.

saga := dtmcli.NewSaga(dtmutil.DefaultHTTPServer, dtmcli.MustGenGid(dtmutil.DefaultHTTPServer)).
Add(busi.Busi+"/SagaBTransOut", busi.Busi+"/SagaBTransOutCom", &busi.TransReqAmount: 50).
Add(busi.Busi+"/SagaMongoTransIn", busi.Busi+"/SagaMongoTransInCom", &busi.TransReqAmount: 30).
Add(busi.Busi+"/SagaRedisTransIn", busi.Busi+"/SagaRedisTransOutIn", &busi.TransReqAmount: 20)
err := saga.Submit()

On this a part of the code, a Saga world transaction is created which consists of three sub-transactions.

  • Switch out 50 from Mysql
  • Switch in 30 to Mongo
  • Switch in 20 to Redis

All through the transaction, if all of the sub-transactions full efficiently, then the worldwide transaction succeeds; if one of many sub-transactions returns a enterprise failure, then the worldwide transaction rolls again.

If you wish to run an entire instance of the above, the steps are as follows.

  1. Run DTM
git clone && cd dtm
go run important.go
  1. Run a profitable instance
git clone && cd dtm-examples
go run important.go http_saga_multidb
  1. Run a failed instance
git clone && cd dtm-examples
go run important.go http_saga_multidb_rollback

You may modify the instance to simulate varied short-term failures, null compensation conditions, and varied different exceptions the place the information is constant when your entire world transaction is completed.

This text provides an instance of a distributed transaction throughout Mysql, Redis, and Mongo. It describes intimately the issues that must be handled, and the options.

The rules on this article are appropriate for all storage engines that assist ACID transactions, and you may shortly prolong it for different engines equivalent to TiKV.

Welcome to go to It’s a devoted mission to make distributed transactions in microservices simpler. It helps a number of languages, and a number of patterns like a 2-phase message, Saga, Tcc, and Xa.

More Posts