The original intention of the article

In response to the future online ( Especially the wireless terminal ) Multiple growth of business volume , The degree of distributed back-end services needs to be improved , Delay and fault tolerance management of services will face greater challenges , Company framework and open source team choose internal promotion Netflix Of Hystrix, One is to promote the service coverage of various departments , The second is to increase C Sharp Participation in language versions ( At present, at least 30% of the company's services are provided by .NET To write ). This blog post belongs to individuals Hystrix Research and practical experience .

What is? Hystrix?

Hystrix Is the world's largest online video rental service provider Netflix Open source , Delay and fault tolerance libraries for distributed systems . The library is created by Java It's written in , The project originated from Netflix API Team in 2011 Elastic engineering project started in . The project in github Published on , There are already nearly 3000 stars , Only a few excellent open source projects can enjoy the treatment of thousand star level ,Hystrix It's a sign of success .

  Why use Hystrix?

In large and medium-sized Distributed systems in , Usually the system depends a lot on (HTTP,hession,Netty,Dubbo etc. ), Here's the picture :

Under high concurrent access , The stability of these dependencies has a great influence on the system , But dependence has a lot of uncontrollable problems : If the network connection is slow , Resources are busy , Temporarily unavailable , Service offline, etc .

Here's the picture :QPS by 50 Dependence I There is an unavailable , But other dependencies are still available .

When dependent I Blocking time , Most servers Threads The pool is blocked (BLOCK), Affect the stability of the entire online service . Here's the picture :

In a complex distributed environment framework There are a lot of dependencies on your application , Will inevitably fail at some time . If there is no isolation measure for high concurrency dependency failure , At present, application services are at risk of being dragged down .

 for example : A dependency 30 individual SOA Service system , Every service 99.99% You can use .
99.99% Of 30 Power ≈ 99.7%
0.3% It means 100 million requests There will be 3,000,00 Secondary failure
It's about a month 2 Hours of service is not stable .
As the number of Service Dependencies increases , The probability of service instability will increase exponentially .
The solution to the problem : Isolate dependence ,Hystrix It's a framework for dealing with dependency isolation , At the same time, it can also help us to manage and monitor the dependent services .

What can we do ?

1)Hystrix Use command mode HystrixCommand(Command) Wrapping depends on the invocation logic , Each command is in a separate thread / The signal to grant authorization perform

2) Provide fuse assembly , It can be run automatically or invoked manually , Stop the current dependency for a while (10 second ), Fuse default error The rate threshold is 50%, Overruns will run automatically .

3) Configurable dependency calls Overtime Time , The timeout time is generally set as ratio 99.5% The average time is slightly higher . When the call times out , Direct return or execution fallback Logic .

4) Provide a small thread pool for each dependency ( Or signal ), If the thread pool is full, the call is rejected immediately , Queuing is not used by default . Accelerate failure to determine time .

5) Depend on calling result points : success , Failure ( Throw out abnormal ), Overtime , Thread refused to , A short circuit . request was aborted ( abnormal , Refuse , Overtime , A short circuit ) When the fallback( Downgrade ) Logic .

6) Provides near - real-time dependency statistics and monitoring

7) Support asynchronous execution . Supports concurrent request caching . Automatic batch failed request .

Hystrix Design concept

Want to know how to use , We have to understand the core design concept first ,Hystrix Based on command mode , adopt UML First, let's have an intuitive understanding of this design pattern

so ,Command Is in Receiver and Invoker Add a middle layer between ,Command Realized with Receiver Encapsulation . that Hystrix How does the application scenario correspond to the figure above ?

API It can be either Invoker It can be reciever, By inheritance Hystrix Core class HystrixCommand To encapsulate these API( for example , Remote interface call , Database queries and other operations that may cause delays ). You can for API Provides elastic protection .

Hello World

Hello World The purpose of this article is to show , How to carry out intrusive transformation in the project , send API in Hystrix Under protection !

introduce maven rely on

<!-- Dependent version -->
<hystrix-metrics-event-stream.version>1.1.2</hystrix-metrics-event-stream.version> <dependency>
<!-- Warehouse address -->
<name>local private nexus</name>

Here's an unused Hystrix The protection of sayHello Service and its invocation

//API call , There may be delays 
public class HelloService {
public static String sayHello(final String name)
return String.format("Hello %s!", name);
// The client directly calls API
public class Client{
public static void main(String[] args)

Suppose the string generation process is an operation that needs to be protected , Now let's use Hystrix encapsulate .

It should be noted that , Although using command mode , But we don't recommend coverage here execute Method , But to achieve run Template method , Most frameworks are implemented using template Design patterns , And set the template method to protected Signature , The advantage of this is , You can hand over the specific business to the business implementer , You can add other functions to it , And business implementers just need to focus on their own business . Here, for example. HystrixCommand.execute Method is actually called HystrixCommand.queue().get(), and queue Method except the final call run outside , Also need to run Method provides protection functions such as timeout and exception , You can't call non secure directly from outside run Method , This practice is very worthy of our study .

OK, Now we're going to do it through run Method to pack sayHello function , We go through a private domain _name, Passing messages through constructors , Get a copy of the construction parameters to keep invariance .

public class SayHelloCommand extends HystrixCommand<String> {
private final String _name;
public SayHelloCommand(String name)
_name = new String(name);//unmutable }
protected String run() { return String.format("Hello %s!", _name);

API The transformation is as follows , As a facade method, it's best not to change the signature of the function ( Unless the parameters and return types change , This is because the cost of changing the client code is often huge ), Also provide version sayHelloAsync, This method provides asynchronous function

public class HelloService {
// public static String sayHello(final String name)
// {
// return "Hello " + name + "!";
// } /**
* sayHello under protection of Hystrix
* @param name
* @return <code>"Hello " + name + "!"</code>
public static String sayHello(final String name)
return new SayHelloCommand(name).execute();
} /**
* call async
* @param name
* @return
public static Future<String> sayHelloAsync(final String name)
return new SayHelloCommand(name).queue();

So let's see , How to use it in case of over time fusing FallBack Strategy , This is quite useful in projects , For example, access the backup database after timeout , Or return the retrial response directly

First SayHelloCommand Constructor uses Hystrix Of Setter To set the timeout , Here's the explanation Setter This class involves several best practices

1.Setter Use builder Pattern , Think about the constructor having a lot of parameters to set , Passing as a construction parameter greatly reduces readability , Using static factory method to set one by one may cause inconsistency under multithreading concurrency , And this kind of bug It's often very difficult to locate , therefore builder Patterns are very good practices . take Setter Pass on as a constructor to HystrixCommand Constructor for ,Setter There are many static methods in , You can clearly know the meaning of the element through the method name .

2.Setter yes HystrixCommand Inner static class ,Hystrix The code uses a lot of internal static classes , As factory methods for this class , Or the constructor , I think this division makes code responsibilities clearer , Easier to maintain than a separate factory class .

3.Setter Use functional concatenation , Each static factory method returns Setter example , So we can connect the construction process in series , Make the code easier to read .

4.Setter It's an immutable class , Each static factory method returns a new Setter Copy , therefore Setter It's thread safe .

OK,Setter So that's it , Here we continue to set the timeout to 500

public SayHelloCommand(final String name)
//builder for HystrixCommand
_name = new String(name);

run Method we use Thread.sleep(600) To achieve the effect of overtime , At the same time to realize getFallback Method , The program runs as soon as it times out FallBack

protected String getFallback() {
return String.format("[FallBack]Hello %s!", _name);
} @Override
protected String run() throws Exception {
return String.format("Hello %s!", _name);

Final output :

[FallBack]Hello World!

Review the main point

1.Hystrix Flexible protection for distributed services

2.Hystrix Encapsulating calls through command mode , To achieve elastic protection , Inherit HystrixCommand And realize run Method , It's the simplest package .

3. Realization getFallBack Method can provide backup handling method for fusing or exception .

4.HystrixCommand in Setter Class best practices .

5. The practice of template method in framework .

Distributed service elastic framework “Hystrix” Practice and source code research ( One ) More articles about

  1. Spring Framework spring-web web The source code is completely parsed

    Spring Framework spring-web web The source code is completely parsed spring-web yes Spring webMVC The basis of , from http.remoting.web Three parts , The core is web modular .http The module encapsulates ...

  2. How to read gracefully GitHub The open source js Framework and library source code

    How to read gracefully GitHub The open source js Framework and library source code step Total before score , That is to understand the general structure of the framework first , Another overall understanding , In selecting certain and interesting parts , Read... Carefully , Crush one by one : Read with questions , What was used ...

  3. 【 Collections framework 】JDK1.8 Source code analysis HashMap( One ) Reprint

    [ Collections framework ]JDK1.8 Source code analysis HashMap( One )   One . Preface Based on the analysis of jdk1.8 After HashMap Source code , I found that a lot of online analysis is based on the previous jdk, and Java8 Of HashMap I've done a lot of optimization before ...

  4. 【 Collections framework 】JDK1.8 Source code analysis ArrayList Detailed explanation ( One )

    [ Collections framework ]JDK1.8 Source code analysis ArrayList Detailed explanation ( One ) One . from ArrayList On the surface of the word ArrayList Class is named by Array and List A combination of words ,Array It means array in Chinese ,Lis ...

  5. be based on Docker Of TensorFlow Machine learning framework construction and example source code interpretation

    summary : be based on Docker Of TensorFlow Machine learning framework construction and example source code interpretation ,TensorFlow As one of the hottest machine learning frameworks ,Docker Yes, the container , It's a good combination , For machine learning or scientific research personnel to provide convenient ...

  6. MyBatis The use of framework and source code analysis ( 11、 ... and ) StatementHandler

    Let's recall that <MyBatis The use of framework and source code analysis ( Ten ) CacheExecutor,SimpleExecutor,BatchExecutor ,ReuseExecutor> , this 4 individual Ex ...

  7. MyBatis The use of framework and source code analysis ( Nine ) Executor

    from <MyBatis The use of framework and source code analysis ( 8、 ... and ) MapperMethod> In this article, we know that the implementation of Mapper Every interface method of , The last thing to call is MapperMethod.execute Method . And when executed Ma ...

  8. Spring Framework spring-web http The source code is completely parsed

    Spring Framework spring-web http The source code is completely parsed Spring-web yes Spring webMVC The basis of , from http.remoting.web Three parts . http: Encapsulates the http In the agreement ...

  9. Ali sentinel Source code research in depth

    1. Ali sentinel Source code research in depth 1.1. Preface Yesterday I put sentinel Successfully deployed to online environment , Please refer to my last blog , All the holes that should be walked have been walked once , Its current limiting and degrading functions can be used preliminarily , According to my current practice , Current limiting ...

Random recommendation

  1. Use requestAnimationFrame Do animation effect one

    I've been studying recently requestAnimationFrame, After reading Zhang Xinxu, he is straightforward and easy to understand , But in some places the language is too bare ...

  2. Talking about Dynamic The third in the keyword series ( On ):ExpandoObject, DynamicObject, DynamicMetaObject ExpandoObject: Represents an object , This object contains information that can be added dynamically at run time ...

  3. Change Or Set Report Object Property At Run Time In Oracle Forms Using Set_Report_Object_Property Command

    Sets the Report object property at run time in Oracle Forms of an report object. The following are t ...

  4. ADO.NET Additions and deletions 、 check

    The data access Corresponding namespace :System.Data.SqlClient; SqlConnection: Connection object SqlCommand: Command object SqlDataReader: Reader object CommandText: ...

  5. Flask From entry to earth ( Two )—— Request response and Flask Expand

    ———————————————————————————————————————————————————————————— One . Program and request context Flask When a request is received from the client , To enable view functions to access some objects ...

  6. PHP MySQL Order By key word

    ORDER BY Keywords are used to sort the data in the recordset . ORDER BY key word ORDER BY Keywords are used to sort the data in the recordset . ORDER BY Keywords sort records in ascending order by default . If you want to go down ...

  7. tensorflow The basic content

    tensorflow Structure 1. Use the figure (graphs) To represent the computing task 2. It's called a conversation (Session) The context of (context) Performed in the figure 3. Use tensor According to the data 4. Through the variable (Variabl ...

  8. .6- elementary analysis webpack Source code validateSchema modular

    validateSchema modular Let's start with error detection : const webpackOptionsValidationErrors = validateSchema(webpackOptionsSchem ...

  9. day 74 json and ajax Example

    One json The definition of : json(JavaScript object notation,js Object tag ) Is a lightweight data exchange format , It's based on ecmascript(w3c designated js standard ) A subset of , Adopt complete independence ...

  10. Simple Package Tool Study

    Simple Package Tool Study   1.getattr Built in functions getattr(object, name[, default]) python install - ...