2021 summary of the latest Java common open source libraries, the top java development tools necessary for developers

Programmer Xiao Gang 2021-10-14 08:06:37


ZooKeeper Is very popular , There's a basic question :

  • ZooKeeper What is it used for ?
  • Not before ZK, Why was it born ZK?

OK, Answer the question above :( Here's what I said intuitively )

  • ZooKeeper Distributed application development is used to simplify , Shield developers from the underlying details of the distributed application development process
  • ZooKeeper The exposure is simple API, For supporting distributed application development
  • ZooKeeper While providing the above functions , It's still a High performance 、 High availability 、 Highly reliable distributed cluster

That's all it says , To sum up ,ZK Distributed application development can solve the problem ,ZK Can solve the problem very well . To this step , There are more questions :

  1. Distributed application development , What are the common problems ?ZK How to shield these underlying details ?
  2. ZooKeeper Exposed to the outside world API? these API How to support distributed application development ? these API Can it be simplified ?API How about the semantic nature of ?
  3. ZooKeeper Itself is a high performance 、 High availability 、 Highly reliable distributed cluster , There's a simple question :
  • What does high performance mean ?ZooKeeper In order to achieve high performance , What has been done ?
  • High availability is the same as
  • High reliability is the same as

Note: This article wiki It is to solve the first question .( Other questions will be in other blog Step by step )

Why ZooKeeper

An application , When multiple processes are involved in collaboration , There are a lot of complex process collaboration logic in business logic code .

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _Java

The above multi process collaboration logic , Yes 2 Characteristics :

  • Deal with complexity
  • Processing logic is reusable

therefore , Consider the common problems of multi process collaboration , As infrastructure , Give Way RD More focus on business logic development , namely :

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _Java_02

ZooKeeper It is one of the basic services of multi process cooperation .

ZooKeeper Characteristics

ZooKeeper There are several simple features :

  • ZooKeeper Of API: from file system API The inspiration , Provide simple API
  • ZooKeeper Running on a dedicated server , Separate from business logic , To ensure the High fault tolerance and Extensibility

ZooKeeper It's a storage facility , But pay special attention to

  • ZK The data stored on is focused on : Collaborative data Metadata ), Instead of applying data , Application data has its own storage scheme , for example HDFS etc.
  • ZK Essentially , It can be seen as a kind of special FS

In particular :

Application data and metadata , Due to different scenarios , There are differences in requirements for consistency and persistence , therefore , Architecture design 、 In the process of data governance , Should be 2 Class data is viewed independently 、 Independent storage .

ZooKeeper The mission of

ZK The core problem to be solved :

ZK The goal is : Simplify distributed application development , Multi process collaboration problem . For distributed applications , Provide Efficient reliable The distributed coordination service ( Basic services ), for example :

  • Unified naming service
  • Distributed lock
  • Process crash detection
  • Leader The election
  • Configuration Management : When the configuration changes , Send it to all in time Client.

A simple question : What is multi process collaboration ? Nima , Over and over , You have everything , Facing this crazy skull , Let's answer .

Multi process collaboration , The whole is divided into 2 class :

  1. Collaboration : Many processes need to be dealt with together , Some processes take action, and others work properly , for example : Master-slave structure ,M towards S Assigned tasks ,S Will execute , otherwise S Just stay idle
  2. competition : Two processes can't work at the same time , One process must wait for another process to finish executing , for example : Master-slave structure ,M After node failure , quite a lot S All want to be M, At this time , You need mutexes , Only the first to get the lock S Become M

In particular :

  1. No collaboration across networks : Multi process , It can be on the same physical host , Synchronization primitives are very convenient ( such as ? The Conduit 、 Shared memory 、 Message queue 、 Semaphore )
  2. Collaboration across networks : Multi process , Distributed on different physical hosts ,ZK Focus on this category

Multi process collaboration across networks , Process of communication , The basic idea is 2 individual :

  1. Message mechanism : Through the network , Direct information exchange , Multi message passing algorithm , Implement synchronization primitives
  2. Shared memory : Using external shared storage , Achieve multi process collaboration , requirement Shared memory Provide orderly access to ,ZK In this way

In real systems , Cross network communication , There are several common problems :

  1. Message delay : Because of the Internet , Send later and arrive first
  2. Processor performance : Due to system scheduling , When the message arrives , Delays in processing
  3. Clock offset : Different physical hosts , Clock offset

ZK Carefully designed for Block the above 3 A common problem , Make these problems completely transparent at the application service level .

ZooKeeper characteristic

ZooKeeper The essence of the solution

Consistency of distributed systems :

  1. The messaging : Delay , Message sent first , You don't have to arrive first ;
  2. The messaging : Loss of sex , Message sent , May be lost ;
  3. Node crash : Within a distributed system , Any node can crash ;

under these circumstances , How to ensure data consistency ?

  1. Vote on the proposal : Based on voting strategy ,2PC
  2. The election vote : Based on voting strategy , Cast The highest priority node ( The node that contains the latest data )

Paxos The goal is : solve Distributed consistency problem , Improve distributed systems Fault tolerance The consistency algorithm .

Paxos The essence : be based on The messaging Of Highly fault-tolerant Of Consistency algorithm

ZooKeeper location

ZooKeeper yes :

  1. Distributed coordination services
  2. Efficient 、 reliable
  3. Convenient applications , focusing Business logic development , And you don't have to pay too much attention to Details of distributed inter process collaboration

ZooKeeper No direct exposure The original language , It is , Expose a part of Calling method Composed of API, File system like API, Support applications to implement their own The original language .

ZooKeeper characteristic

ZooKeeper The following distributed consistency features can be guaranteed :

  • Sequential consistency : The same Client Initiated transaction request , Strictly follow the order of initiation
  • Atomicity : Transaction request , Or apply to all nodes , Either one node has no application
  • A single view :Client No matter which node you connect to , The server-side data you see is consistent (Note: inaccurate , It's actually Final consistency
  • reliability : Once the transaction is successfully executed , The state remains permanently
  • The real time : Once the transaction is successfully executed ,Client You can't see the latest data immediately , but ZooKeeper Guarantee Final consistency

ZooKeeper Design objectives

ZooKeeper Committed to providing High performance High availability Sequential consistency The distributed coordination service , Guaranteed data Final consistency .

Goal one : High performance ( A simple data model )

  1. use A tree structure Organize data nodes ;
  2. Full data nodes , All stored in memory ;
  3. Follower and Observer Handle non transactional requests directly ;

Goal two : High availability ( Build clusters )

  1. More than half of the machines survive , The service will run normally
  2. Automatically Leader The election

Goal three : Sequential consistency ( The order of transaction operations )

  1. Each transaction request , Will be forwarded to Leader Handle
  2. Every business , The global allocation will be incremental id(zxid,64 position :epoch + Self increasing id)

Goal four : Final consistency

  1. By proposing a vote , Ensure the reliability of transaction commit
  2. Propose a way to vote , Only guarantee Client After receiving the transaction commit successfully , More than half of the nodes can see the latest data

ZooKeeper Before appearance

ZK Before appearance , There are two ways of distributed system , Achieve multi process collaboration :

  1. Distributed lock manager
  2. Distributed database

ZK More focused on process collaboration , It does not provide any lock interface and general storage data interface .( doubt :ZK It can also be provided , We don't need to use it )

application server , common 2 Needs :

  1. Master-Slave?Leader The election : Asked to provide Master Node election function
  2. Process response tracking ? Crash detection : Request to provide a trace of the process's survival state
  3. Distributed lock : Exclusive lock

ZK For the above 2 These strategies provide the basis API.

ZooKeeper Scenarios that are not applicable :

  1. Mass data storage :ZK The essence is special FS, but ZK Used to store Metadata , It needs to be stored separately Application data


Why don't I fully advocate self-study ?
Daniel on the platform basically has many years of working experience , Have you ever thought about the threshold of the industry before , What is the industry threshold now ? In the past, enterprises did not have such high requirements for programmers' ability , Even more than a decade ago, if you could write “Hello World”, You can get started in this industry , So you can get started before .
② Now there are some excellent young cattle , They may also be self-taught , But they must have excellent learning ability , Excellent self-management skills ( time management , Meditation, persistence, etc ) And be good at finding and summarizing problems .
If you think your goal is very clear , Can you do the first ② The points mentioned in point , In the current market , You are really suitable for self-study .

besides , For most people , Registering for a class must be the best way to grow quickly . But there's a problem , At present, the quality of training institutions in the market is uneven , If you don't find a good training class , It's a total waste of energy , Time and money , This needs to be selected by yourself .

I personally suggest that online is more cost-effective than offline , The price of offline training is basically unchanged 2W You can't get down , Online education is now more mature , During this outbreak , Students have basically experienced the online learning mode . Compared with offline , The advantages of online are mainly in the following aspects according to my understanding :
① Price : The online price is basically half of the offline price ;
② teacher : Relatively speaking, the teachers of online education are stronger and richer than offline education , Better coordination of resources ;
③ Time : Learning time is relatively free , Don't learn naked words , Suitable for learning while working , Reduce the stress of life ;
④ Course : In terms of course content , It really goes deeper than offline .

What technologies should be learned to meet the requirements of the enterprise ?( The following figure summarizes )

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _Java_03

2021 newest Java Summary of common open source libraries , The top-level necessary for developers Java development tool _ The backend development _04

This article has been  CODING Open source project :【 A big factory Java Analysis of interview questions + Core summary learning notes + The latest explanation video + Actual project source code 】 Included

Please bring the original link to reprint ,thank
Similar articles