Cloud native API Gateway - open source project Hango gateway design and Practice

Netease Shufan 2021-09-15 10:01:33

Interpretation of open source Hango Cloud native design and practice of gateway , This paper first traces the origin of cloud origin , Interpretation of cloud origin API Gateway selection features ; Further interpretation of Netease Shufan open source Hango Cloud native API Gateway design and large-scale landing practice .

Cloud origin

What is Yunyuan

CNCF( Cloud native Computing Foundation ) The definition of cloud nativity is : Cloud native is an architectural pattern , It is beneficial to organizations in the public cloud 、 In dynamic environments such as private cloud and hybrid cloud , Build and run scalable applications . Representative technologies include , Containers 、 Microservices 、 Immutable infrastructure and declarative API.

As can be seen from the definition of cloud primitiveness , Cloud primitives have fast integration , It has good scalability and high availability . Thanks to the Kubernetes The development of , Cloud native technology is booming , Has entered a new era .

API The meaning of gateway

With the development of container technology and micro service concept , More product organizations focus on exposure API, For client interaction .API The gateway acts as API The function of agent , The gateway receives the request , The request is uniformly routed and forwarded to the back-end service . Gateway as abstraction layer , Provide a unified access layer for the whole microservice system or cluster .

In addition to the agent function ,API Gateway provides unified security for micro service cluster 、 Response conversion 、 Fuse 、 Multi dimensional functions such as degradation and monitoring , Ensure safe and reliable flow .

As Yunyuan physiological concept leads the evolution of micro Services , Application iteration frequency and delivery speed will be accelerated ,API The emergence of gateway can make business development pay more attention to the design of its own business logic , Improve R & D agility . In the cloud native architecture ,API Gateways are also evolving to cloud native ,Hango Gateway is an excellent gateway product born under the cloud native design .

Cloud native API Gateway features

Enter the cloud primary era , In addition to paying attention to tradition API The request proxy provided by the gateway , In addition to routine functions such as traffic management , More people begin to pay attention to cloud native compatibility , Support the diversity of scenes and better performance .

Compared with tradition API gateway , In cloud native mode, the writer thinks API Some thoughts that gateway should have :

  • Service discovery approach : With traditional gateway service discovery, you need to manually specify upstream, Cloud native gateways need to dynamically perceive business changes in service discovery .
  • The pursuit of higher performance : With Kubernetes And the container brings cloud native into a new era and the development of micro service concept ,API The excellent performance of the gateway can reduce the cost of the whole link RT.
  • Cloud native architecture compatible / Integrate : Whether the deployment architecture of the gateway is compatible with the cloud native architecture , Whether it can be seamlessly connected with container and service grid .
  • Dynamic configuration capabilities : The traditional reverse proxy can not achieve dynamic configuration and loading capability , The proxy configuration changes frequently in the cloud native mode , Dynamic configuration and distribution capability is particularly critical .
  • Deployment form and architecture : Whether the gateway deployment architecture can meet the elastic expansion requirements , Whether it supports smooth migration from mixed business department to independent deployment .
  • monitor / Link tracking : In the cloud native architecture , Good monitoring and link tracking capabilities can facilitate service observation and problem location by operation and maintenance personnel .

Hango API Gateway Design

With the large-scale container transformation of business in the group , The demand of services for gateways is becoming stronger and stronger . Our team is committed to building a new generation of high performance API gateway , The purpose is to replace the more widely used in the group Java Asynchronous gateway 、Zull, At the same time, it meets the north-south traffic governance needs of the business gradually going to the cloud .

In terms of selection , We choose Envoy As a key basic component of cloud native gateway practice , With rich features , Include :

  • Rich native functions : Compare with Nginx,HAProxy Basic functions required by equal traffic agent ,Envoy Based on C++ A considerable number of advanced governance functions have been implemented , Including load balancing 、 Fuse 、 Current limiting 、 Fault injection, etc . Richer features make Envoy Born with multiple scenes .
  • Rich observability :Envoy Flexible log configuration 、 Rich indicator data and a variety of tools out of the box Tracing Ability .
  • Flexible scalability :Envoy Provides L4/L7 Filter Mechanism , Can let developers without intruding Envoy On the premise of trunk Envoy Expand and enhance . meanwhile ,xDS The proposal of the agreement , bring Envoy Almost all configurations can be distributed dynamically 、 load 、 take effect , Without reloading the process .

meanwhile ,Envoy The development trend in the industry is also very rapid ,2020 year Cloud native report Show ,Envoy Usage in the industry has increased significantly , The fourth place last year Envoy, exceed F5 and HAProxy from 15% Our share has grown to 29%.

Architecture design

Based on the above typical characteristics , Adhering to the original concept of cloud , Accumulate practical experience through a large number of internal , We're open source Hango API gateway .Hango Chinese name “ Hangu ”, It's a high performance , Scalable , Feature rich cloud native API gateway .

Hango The gateway is based on Envoy And Istio structure . The data plane is based on Envoy Expand , Enhanced plug-in chain ; The control surface is based on Istio Expand . The specific structure is as follows :


The control surface assembly includes Hango Portal, Hango API Plane, Slime as well as Istiod.Hango API Plane Generate Gateway、DestinationRule、VirtualService etc. CRD as well as EnvoyPlugin etc. Slime CRD; adopt Slime Expand , take Slime CRD Dynamic conversion to EnvoyFilter, On the data side envoy Expand .

The figure below shows Hango Data flow of gateway plug-in chain , By creating a EnvoyPlugin CR,Slime Dynamically listen to the aggregation to generate the corresponding EnvoyFilter, Finish right Envoy filter chain Dynamic expansion of .


High performance

And traditional API Gateway compared to , In cloud native mode , Business development is more agile , Business splitting is more detailed . This will make the delay between each hop more sensitive . The gateway undertakes the traffic agent of the whole micro service cluster , Good performance is a prerequisite for a service to choose a gateway .

Hango API The gateway adopts Envoy As the core data plane , Many core functions are implemented through native C++ Realization , Compare with Nginx、HAProxy And other ways to realize the main functions by extending the script , The performance advantage is more obvious .

meanwhile , In the actual landing process , We also optimized the performance of the gateway , Include :

  • Configure filtering : Through the cluster svc as well as DestinationRule Resource configuration filtering , Reduce envoy cluster Number .
  • The plug-in configuration : Provide global plug-in switch and plug-in switch of routing dimension , Reduce the performance overhead of unnecessary plug-in logic .

By optimizing the , be based on 20 individual cluster,500 strip route Configuration scale , We are right. Hango Gradient tests were carried out , Under the container network ,500 strip route Configuration scale ,Hango RPS You can achieve 9.5w.


Multi scene

Hano API The gateway has the function of supporting micro service gateway 、 Seven layer load balancing 、Kubernetes Ingress Wait for multiple scenes . As the north-south traffic portal of the business cloud ,Hango The gateway has the following features :

  • Flexible routing configuration : Support is not limited to requests Path,Header、Method、Query And so on .
  • Multiple registration methods : Support Kubernetes The registry dynamically perceives the changes of business clusters , Meet the business smooth expansion and contraction scenario ; Support traditional logical deployment , So that the business can migrate smoothly .
  • Multi scenario governance : Support multi-dimensional current limiting 、 Fuse 、 Downgrade 、 Retry and other multi scenario traffic management .
  • Multi protocol support : Support HTTP、gRPC、Websocket Wait for multi protocol access .
  • Multidimensional security : Support multi-dimensional black and white list 、 External authentication and other authentication methods .

meanwhile , As a general gateway ,Hango It can also be used for L7 Load Balancer ,Kubernetes Common scenarios such as edge gateway .


Function extension

Hango Gateway open source Rider modular , For function expansion . Users can dynamically extend the gateway function in the form of pluggable plug-ins through a simple tutorial .

Rider As Hango API The gateway is open source module, Provide plug-in development SDK, Make it possible for users to access Lua Write custom plug-ins in the way of extension , It can be dynamically expanded through simple configuration envoy L7 filters, bring Lua Plug ins can be like other Native Http filter It is also configured at run time .


You can refer to Hango Github Webpage Rider Custom plug-in tutorial , Here is a simple out of the box UA Black and white list plugin , Readers can learn about custom plug-ins .

Stretch and stretch

Hango Gateway supports container deployment , Offer based on Kubernetes Deployment architecture of . Based on the isolation of data plane and control plane , Data plane nodes are based on stateless nodes envoy Deployment . be based on xDS Dynamic distribution design of protocol , The data plane can be expanded dynamically , Smooth expansion and contraction in case of large business promotion or traffic growth .

Fault tolerance

Hango The gateway can realize service configuration isolation , Increase business fault tolerance , For key services, gateway clusters can be deployed separately , Improve the availability of the entire gateway product .

Hango The isolation of service configuration can be completed at the beginning of gateway design , be based on Istio Model of , stay Hango The gateway initially considered enhancements in scale practice Istio The ability of , Through to DestinationRule relevant CR Add corresponding Gateway label. adopt label Control by matching cluster Distribution of resources , Play the role of configuration isolation .

Istio The community is also in the process of version evolution (>Istio1.7), A solution is proposed ,cluster The acquisition of can depend on VirtualService(VS) Medium host To configure , adopt VS resources route host And DestinationRule Map Resources , Generate corresponding cluster, Distribute to the designated Proxy. This can ensure the communication between different gateways cluster Configure isolation , Reduce the interaction of different configurations , Improve service stability ; meanwhile , Greatly reduce online cluster The number of , Further improve proxy performance .

As shown below , Isolate by configuration ,a-hango Gateway cluster can only receive app1 Of cluster. This enables online configuration isolation , Improve product stability , At the same time, the performance of the gateway is further improved .

Easy to operate

Hango The gateway provides slave control interface and API To improve the user experience .

Control plane

Provide a simple control surface , Users can manage the life cycle of services and routes through visual operations . For details, please refer to :Hango Operation flow of gateway control surface


In order to facilitate the enterprise user interface Hango, Hango Provided with standard Restful API, It is convenient for enterprise users to complete the export of existing gateway products Hango The smooth migration of .

Hango API The scale of the gateway is on the ground

Within Netease group , be based on Hango The light boat gateway built by the gateway has been put into practice on a large scale , Undertake the daily 10 billion level traffic of Netease group . This paper mainly from the smooth nanotube 、 Gray scale release and observable practice are introduced from three perspectives Hango The implementation practice of gateway in Netease group .

Smooth tube

be based on Hango The scale of the gateway is on the ground , We are interested in container services , Bare metal services, etc Smooth tube And smooth migration , It provides convenient access to the cloud for business , It eliminates the concern that the cloud cannot be smoothed in business .

Within Netease group , Different gateway clusters are divided for different services , Configure isolation from the business , Improve gateway reliability .

Grayscale Publishing

In the process of practice , Grayscale mainly involves two aspects , On the one hand, it is the gray level migration in the process of business containerization transformation , On the other hand, it is the gray strategy of the gateway product itself .

Business container

The business is in the process of container transformation , There are multiple versions of online scenarios , Grayscale publishing needs to be carried out according to different versions .Hango The gateway provides a grayscale process according to different business versions , By calling the service label In the form of , In the process of routing , Choose different label Configure the gray weight .

Gateway grayscale

As the entrance of micro service cluster , The stable operation of the gateway is related to the stability and reliability of the whole cluster traffic . During the field landing , It is very necessary to establish a set of gray system of gateway itself ,Hango The gateway is implemented within Netease group with four layers LB Build the gray system , Through to Hango The data side is marked label In the form of , Control the gateway itself .


be based on Envoy Good observability ,Hango The gateway is in the process of scale implementation within Netease group , Combined with the service grid scenario , Provide rich observation capability , The overall structure is as follows :

  • journal

    Envoy Detailed record of events in ,Hango The gateway is based on Envoy Expand the data plane , Provides flexible and easy to configure AccessLog, Support custom format , Custom filter rules and output .

    be based on filebeat as well as elastic The ability of , Build an integrated log audit platform .

  • monitor

    be based on Envoy cluster mertic Etc , utilize Promethues Build a gateway / Service and other multi-dimensional index systems . meanwhile , For gateway containerized deployment mode , be based on Kubernetes The container corresponds to metrics Build indicator monitoring for container dimension , cover CPU/ Memory / Bandwidth and other multi-dimensional monitoring .

  • Link tracking

    be based on Envoy A variety of out of the box tracing Access , Strong expansion , At present, online has been completed SkyWalking etc. tracing Access to .

    At the end

    Hango As a newborn in the open source field , We will uphold the concept of embracing cloud Nativity , Continue to follow up Istio/Envoy Evolution of , Give play to greater field value . Next stage , We will expand in multiple languages ,LB Multiple scenes such as fusion , We also look forward to paying more attention to cloud native 、API Gateway students can join Hango Open source community construction .

    We sincerely welcome your attention to Hango Gateway

    Reference documents

    Cloud native (Cloud Native) The definition of

    Cloud is about culture , Not the container

    Hango Open source interpretation : Cloud native gateway practice , Why choose Envoy ?

    Author's brief introduction

    Han Jiahao , Open source project Hango Core developers , Netease Shufan senior R & D Engineer , The dominant Hango Gateway open source R & D and design , Responsible for Netease sailing canoe API The gateway is implemented on a large scale and built into products within Netease . Three years of gateway related R & D and large-scale practical experience .

    Share the trailer

    9 month 9 Japan 20:00,Hango Project core Developer , Wang Baiping, a technical expert of Netease Shufan, will be in InfoQ Open class sharing 《 How to base on open source Envoy, Build a high-performance cloud native microservice gateway 》, Open source from Netease API Gateway project Hango Start with practice , Describes how to build on open source Envoy Build high performance 、 Easy to expand 、 Observable cloud native microservice gateway . respectfully invite Click here Sign up for an appointment to watch !

Please bring the original link to reprint ,thank
Similar articles