With the development of time and business , The amount of data in the database table will be larger and larger , Accordingly , Data manipulation , The cost of adding, deleting, modifying and checking will be more and more . therefore , Split some of the large tables into multiple tables in multiple databases .
This article is based on the asynchronous assurance of non transaction message to complete the transaction problem in sub database and sub table .
2.1 Original business
Because after sub database and sub table , The new table is in another database , How to ensure the transaction of the main database and sub database is a problem that must be solved .
terms of settlement ： By creating a flow table in the main library , Map the logic of operating the database to a pipeline record . When the whole big business is done （ The pipeline is inserted into the pipeline table ）, And then execute this flow in other ways , Ensure ultimate consistency .
2.2 Running water
The so-called running water , It can be understood as a transaction message
Above by creating a flow table in the database , Use a pipeline record to represent a business processing logic , therefore , A running water must be able to execute correctly in the end . therefore , When extracting a piece of business code from the pipeline, we must consider :
** Pipeline delay processing .** Pipelining is not real-time , It's implemented asynchronously with an over pipelined actuator . therefore , If in the original logic , Special attention should be paid to whether the subsequent process has real-time dependence on the pipeline （ For example, pipeline results will be used in subsequent business logic to do some calculations, etc ）.
** Pipelining disorder .** Ensure that even if the later generated pipeline is executed first , There can't be a problem .
** The final success of running water .** For each inserted pipeline , This flow of water must ensure that it can be executed successfully
therefore , When we extract water :
The simpler the pipelining, the better
The less we rely on, the better
The extracted pipeline has no real-time dependency in the business logic
2.3 Pipelined processors
Pipelining processor is to ensure that pipelining process as fast as possible , It can also ensure that the running water can be implemented successfully .
Imagine a scenario ： When a pipeline fails to process , If the drain actuator has to wait for the current pipeline to execute successfully before continuing to execute later , Then it will affect the implementation of the follow-up flow , What's more, it's stuck as a record all the time , Cause problems in the whole system
therefore , Set... In the pipeline actuator 2 A mission ：
The first task , Pipelining tasks , The pipeline has been executed at the fastest speed , If pipelining fails , It doesn't affect the following flow process
The second task , Pipeline verification task , This task is to check the flow records in sequence , Make sure that all flows are executed successfully , If you fail , retry , After repeated retries fail, an alarm is issued to allow manual intervention .
2.4 The flow process is complete
Because the flow table is placed in the original database , After the completion of pipeline processing, it is the operation branch library , If the sub database operation is completed, update the old table pipeline message , So it's the quaku business again , How to ensure the update and sub database of pipeline state is also in the process of a transaction ？
The solution is ： Create a flow table in the branch library , When the wastage treatment is completed , Not to update the old watch status , Instead, it is inserted into the branch flow table 、
The benefits of doing this ：
In general, the pipeline is uniquely indexed , So if the pipeline is repeated many times , When inserting the branch library flow table, it must be because the unique index detection fails , The entire transaction will be rolled back （ Of course, we can also make idempotent judgment before dealing with the flow of water ）
In this way, by judging whether the main library pipeline is in the branch library, we can judge whether a pipeline has been executed
Pipelined processors do not contain any business-related processing logic , The core function is ：
Inform the service access party when to handle what kind of flow
Test the success of flow execution
notes ： The pipelining executor does not know what logic the pipelining represents , Specifically, the business system needs to identify and execute the corresponding business logic .
3.1 Running the task
Pipeline processing scheduling task is to scan the pipeline to be processed , Then inform the business system which flow to execute .
The schematic diagram is as follows ：
3.2 Pipeline verification task
The task of pipeline verification is to compare the pipeline records in the main database and the branch database , Reprocess the flow notification business system that fails to execute , If multiple retries fail, an alarm will be issued .
Flow diagram ：
Because it is an existing project to carry out transformation （ I am engaged in Internet finance , Therefore, it is absolutely intolerable that any message is lost or the message processing fails ), Without using transaction messages, there are 1 One reason
Additional message queues need to be introduced , Increases the complexity of the system , It also needs additional logic guarantee and handling when message queue communication fails
Actually 1 Not the main reason , It's because transaction messages need to be manually commit and rollback（ Using a database doesn't require ）, So here comes the question ,spring Transactions are transitive , It's a big problem when we submit the transaction information , for example A.a() It's a business , But another thing B.b() Call... Again A.a() The transaction message commit is placed in A.a() still B.b() What about China? ？
Welcome to my official account 【 Calm as a code 】, Massive Java Related articles , Learning materials will be updated in it , The sorted data will also be put in it .
Like what you write , Just pay attention ！ Focus , Neverlost , Continuous updating ！！！