Project Structure

The project mainly uses the springboot + dubbo + mybatis framework, which is roughly divided into two layers: web and service. web provides api interface for sdk client, service provides mysql database table and other operations, and provides dubbo service support for web.

Business scene

Take a look at this picture:

As shown in the figure, the approximate process is: sdk end incoming appkey (the unique identifier of an app), the web side calls the service remotely through dubbo, and the service needs to be routed to different databases according to different appkeys. The table structure of the database represented by each appkey is the same.

Now the problem is coming, appkey is a non-business field identifier (not in the table, only on the library name), how to change the original dao method signature (no appkey entry), service method signature (no appkey entry), Under the premise of the mapper xml file (without appkey identifier), execute the sql operation on the corresponding database according to the different appkeys reported?


Summarize the requirements of the above questions: `How to non-intrusive database routing by key?

The "no intrusion" here involves two difficulties:

  1. How can I bring the appkey to the service instead of the appkey when the web is tuned?
  2. Assuming the first point is ok, how can the service side execute sql on the specified library without displaying the changed data source?

Consider the second of the above difficulties:

Idea One

There are many examples of multi-data sources for mybatis, but they are not suitable for the above scenarios. It needs to configure multiple data sources to “write dead” first, and you need to explicitly specify the data source to be used on the specific dao method used. It may be more suitable for scenarios such as read-write separation, but it is not enough for the "dynamic" data source in this case!

Idea 2

Data sources can’t be dynamically selected. Can you create data sources dynamically? The answer is yes. But this will make the architecture more complicated. sql under different appkeys needs to use different data sources to execute. In general, each data source must maintain a connection pool. As a result, when there are many appkeys, Very resource intensive. So the way to dynamically create a data source is not feasible!

Feasible solution

In fact, sql itself has given us the answer:

As can be seen from the above process, the entire process does not use any commands such as use test to switch databases, which means that all sql operations are performed in a data source (I am logged in as the root user, the default mysql library) ).

As a result, things became clear, and the remaining goals were: rewrite sql, table name plus library name prefix

Just as mybatis also supports this kind of operation, through the interceptor of mybatis, the sample code (verified) is as follows:

Careful, you can easily find this line:

At this point, the first of the above two difficulties is almost solved. Yes, the RpcContext object here is dubbo for us. A tool that allows you to easily pass context parameters when calling the dubbo service. The principle behind it is based on ThreadLocal.

The problem of the service layer has been largely solved. Now the problem of the web layer is relatively simple. Write a filter to intercept the appkey parameter globally and put it into the dubbo context. All other controllers will bring the appkey parameter as long as the dubbo service is called. as follows:

Still a little bit

The above process seems to be perfect, but due to the design of dubbo, things are not as smooth as imagined. For the convenience of explanation, look at the following code:

I can take a closer look at the code comments section I wrote above. The above conclusions are the ones I have tried.

It is common to call the dubbo service multiple times in a controller, but the data put into RpcContext will be destroyed after the first call to testService.method1(), the second time the dubbo is called. I can’t get it. And filter will only intercept the controller once.

As mentioned earlier, RpcContext is based on the java native ThreadLocal design, and ThreadLocal is bound to the current thread, and the hello method of the above controller is after the first testService.method1() is adjusted. The current thread is still there! It seems that the data does not disappear out of thin air, so it should be explicitly removed from the dubbo source code.

After some search + breakpoint debugging (limited to space, omit the steps), finally found the following two places in the destruction of data:

The code in the red box above is not uncommented before, and friends who need it can modify the dubbo source code and put it into the private warehouse.

At this point, a non-intrusive dynamic routing database solution under microservices is achieved!

to sum up

  1. The interceptor of mybatis is very practical, rewriting sql, statistical execution time, etc., is very useful.
  2. Dubbo originally considered carrying context parameters in remote calls. Using this can do some similar but more lightweight designs.
Last modified: 2019年3月29日



Write a Reply or Comment

Your email address will not be published.