author Wang Feng   Published on 2014 year 5 month 28 Japan

Preface

Microsoft Azure It's the only solution for Microsoft's public cloud . With this platform , Users can deploy and publish their own applications in a variety of ways . It's an open platform , In addition to Windows The server and SQL Server Support for ,Microsoft Azure It also supports a large number of mainstream open source software and frameworks , Include Linux, Tomcat, Java etc. . How to migrate a system based on open source software to Microsoft Azure platform , And give full play to the elasticity of the cloud 、 Agility, etc ? This article shares some best practices with readers , At the end of the paper, the system migration of China Network TV station is taken as a reference case .

Before reading this article , I hope the readers are right Microsoft Azure Know something about . Readers can log in http://www.windowsazure.cn Get information about the platform . The following concepts are mentioned in this paper :

- Infrastructure as a service (IaaS: Infrastructure as a Service): Consumers get computer infrastructure resources through the Internet , Including computing resources ( The virtual machine ), Storage resources and network resources, etc .

- Platform as a service (PaaS: Platform as a Service): The service providers of cloud platform are interested in resources IaaS On the basis of this, we will do further encapsulation , take IaaS Layers are transparent to users . Consumers use development tools and a range of development languages , Build your own applications on the cloud platform , Without the need to manage and maintain virtual machines 、 Storage 、 Network and other infrastructure .

stay Azure Use in Linux Virtual server :

Azure The platform currently supports three types of Linux operating system : UBUNTU, CENTOS and SUSE. Their versions are :

- Ubuntu Server 12.04 LTS

- Ubuntu Server 12.10

- Ubuntu Server 13.10

- Ubuntu Server 14.04 LTS

- OpenLogic

- SUSE Linux Enterprise Server 11 SP3

stay Azure Platform use Linux The virtual machine , There are the following best practices to help users get a better experience .

by Linux Configure switch partition (SWAP)

stay Linux Keep in swap There are two important reasons for zoning :

  1. When physical memory is insufficient to support systems and Applications ( process ) When it's working , This swap Space can be used for temporary storage of memory paging with low utilization rate , Give the free memory to the badly needed applications ( process ) Use .
  2. Even if your machine has enough physical memory , There are also some programs that transfer the rarely used paging contents of memory to swap Space , To make room for physical memory . For applications that have a chance of memory leaks ( process ),swap Space is more important , Because no one wants to see the system crash due to insufficient physical memory .

stay Azure On the platform , When users go through the image library , Create a Linux Server time , No swap partition is assigned by default . function free command , We can see swap The corresponding number is 0.

The reader may ask , since swap Zoning is so important , Why? Azure The image of the platform is not automatically configured swap Partition ? One important reason is that , swap The size and location of the partition are determined by the user's own usage scenarios ,Azure Keep the flexibility of your own customization , Leave the job to the user , After the virtual machine is created .

How to create swap Partition ? The following example is in /mnt/resource Create under directory 5GB Of swap Partition .

  1. Switch to root user
  2. Run the command : fallocate –l 5g /mnt/resource/swap5g
  3. Run the command : chmod 600 /mnt/resource/swap5g
  4. Run the command : mkswap /mnt/resource/swap5g
  5. Run the command : swapon /mnt/resource/swap5g

Linux Non persistent disk of virtual machine

Each station Linux Virtual servers come with a non persistent disk . The read and write speed of this disk is very high , But it doesn't exist in Azure On persistent storage devices , And on the host . Users must pay attention not to put persistent data in this directory /mnt/resource, Otherwise, when the server restarts , This data will be lost .

stay Azure Use in Tomcat

If the user's original application is deployed in Tomcat Server , There are two migration patterns :IaaS and PaaS. Use IaaS Pattern , User pass Azure Create virtual machine , And put Tomcat Install on a virtual server , Then deploy the application . Use PaaS Pattern , With the help of users Azure Tools provided , Deploy the original application as “ The cloud service ”. If the customer's application does not necessarily require Linux operating system ( That is, it is not used in the application Linux Special functions of the operating system ), The way we recommend it is PaaS. Now let's explain it in detail Tomcat Application deployment as Azure Cloud Service The advantages and convenience of .( How to use Eclipse take Tomcat Application deployment to Azure, Please refer to the official documentation :
http://msdn.microsoft.com/library/azure/hh694271.aspx

When we're in Eclipse Created in Windows Azure After the project , Right click , choice “ Deploy ”. In the deployment dialog box that pops up , We can see that for JDK and Tomcat There are three ways to deploy :

  1. Using the client's local development environment JDK and Tomcat.

    This is the most intuitive way , Customers only need to specify their own machines JDK and Tomcat The location is OK . The disadvantage is that it takes a long time to upload . However, this kind of work only needs to be carried out in the first deployment , You will no longer need to update your application in the future .

  2. Use the information provided by a third party in Windows Azure Upper JDK.
  3. Use Azure In storage JDK and Tomcat.

In this way, customers need to put JDK and Tomcat Upload the compressed package to Azure In storage . Customers can use a range of tools , for example Windows Azure Storage Explorer,
AzCopy etc. . This method is the quickest , It is also recommended by the author .

stay Azure Use in MySQL

adopt Microsoft Azure Image library deployment of CentOS Virtual server , stay YUM Repository I brought it with me MySQL Installation package for . Users can easily go through ”yum install mysql” install MySQL The server . For deployed MySQL Virtual machine of , Please execute the order “chkconfig --list mysql” take MySQL The process is added to the virtual machine startup program . such , When the virtual machine is restarted , Can guarantee MySQL The service automatically returned to normal .

How to achieve MySQL High availability architecture for . Customers have many choices , For example, through MySQL, Heartbeat, DRBD To do virtual machine state monitoring and data synchronization . You can also choose Oracle Version of MySQL Cluster. In the world Azure For instance , Customers can also directly adopt ClearDB Provided MySQL as a Service.

Best practices for large scale applications

Use Azure PowerShell Batch deployment of virtual machines

When we need to deploy hundreds or more virtual machines , Use browser based Azure Management portal It's very time consuming . Especially if we need to get the server ready in a short time , It's even more unrealistic to use graphical interface for manual operation .Azure All the management functions provided by the management portal can be accessed through PowerShell To achieve . For non Windows Environmental Science ,Azure Also provided Command line interface .

for example , We need to deploy 100 Taiwan is based on CentOS Of Apache The server . The recommended method is to first pass Azure The image library deploys one CentOS The server ( The current version is CentOS 6.5). install Apache, And configure it as you need . after , Take this virtual machine as a mirror (image). Refer to :
http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-capture-image/ . After production , We can use this image file , Batch deployment of virtual machines . Refer to the following PowerShell Script .

Be careful : Azure Cloud services (Cloud Service) At present, it can accommodate at most 50 A virtual machine . If your application needs 50 The virtual machine above the platform , You need to create multiple cloud services .

Use Azure Auto scaling function of

A lot of times , Our need for system size is dynamic . Like a voting system , Usually, the user visits are not large . When there are big events ( For example, the Spring Festival Gala ) When , The number of system visits has soared . That's what we need. The size of the system changes with demand . There is also some explosive growth that is unpredictable . For this scenario ,Azure Provides automatic scaling . You can CPU Average percentage of usage or specify scaling based on the number of messages in the queue .

stay Windows Azure Manage the portal's “ The zoom ” Page , You can scale the application manually , You can also set parameters to make it auto zoom . You can zoom to run Web role 、 Applications that assist roles or virtual machines . To zoom, run Web Role or auxiliary role instance application , You need to add or remove role instances to accommodate the workload .

Before you configure scaling for your application , The following information should be considered :

  • You have to add the virtual machine to the availability set , To scale the applications that use them . The virtual machine you added may initially be on or off , But they will open in the extension operation , Will be turned off in the shrink operation . More about virtual machines and availability sets , Please see the Manage the availability of virtual machines .
  • Scaling is affected by kernel usage . Larger role instances or virtual machines use more cores . You can only scale applications within the kernel limits of your subscription . for example , If your subscription is limited to 20 cores , And you have two medium-sized virtual machines ( There are four cores ) Run an application , For other cloud service deployment in the subscription , You can only extend sixteen cores . All virtual machines used to scale applications in the availability set must have the same size .
  • You must first create a queue and associate it with a role or availability set , Then you can scale the application based on the message threshold .
  • You can scale resources linked to cloud services .

Use Azure Virtual networks improve overall performance

When we use Azure Of IaaS When the service creates a virtual machine , It often needs several virtual machines to achieve high-speed interconnection . for example , We created one Windows Server 2008 R2 As an application server , And created a SQL Server 2008 R2 Standard As a database server . When the application server accesses the database server , We want them to go through Azure Internal network ( The network in the background data center ) Talk to each other , Instead of bypassing Internet. To achieve this goal , There are two ways :

  1. Put these virtual machines in the same cloud service .
  2. Put these virtual machines in the same virtual network .

In a classic three-tier system , We usually deploy virtual machines with the help of virtual networks . At present, a cloud service can accommodate at most 50 A virtual machine , Just using cloud services is often not enough . Because as the system expands , Probably more than 50 The limits of Taiwan , So we need to deploy multiple cloud services . In multiple cloud services ,Azure It doesn't guarantee the interconnection of virtual machines. It's the intranet IP. How to use virtual network to deploy virtual machine , Readers can refer to this article :
http://blog.csdn.net/azureincubationchina/article/details/10212871 .

In some cases , The system is deployed to PaaS and IaaS Mixed mode . for example Web The server is deployed as a cloud service , The database server is deployed on a virtual machine . At this time , We still hope Web The server and the database server are in the same virtual network . To get to this point , The user will first IaaS Virtual machines are deployed to a virtual network , Then edit the cloud service *.csfg file , Just add the following paragraph .

China Internet TV (CNTV) Case reference

of CNTV stay Microsoft Azure Application scenarios and business value of , Readers can refer to the following article . This article focuses on the technical architecture .

- Windows Azure Help the webcast of the Spring Festival Gala of the year of the horse , by CNTV Three big surprises : http://blog.sina.com.cn/s/blog_6b8988220102edgt.html

- CNTV The transformation of “ One cloud with multiple screens ” The new media Microsoft Windows Azure Escort : http://www.cnetnews.com.cn/2014/0216/3011306.shtml

Technology Architecture

CNTV The original data center is rich in IT Resources and key data , How to make the original IT System and Windows Azure The fusion , In order to obtain the best utilization rate of system resources and the efficiency of the system, it is the key problem to be considered .CNTV Of VDN Video distribution networks need to process per second 8 Ten thousand user requests , After many tests and demonstrations ,CNTV The original IT The system undertakes 40% User request for , in addition 60% User requests for will have Windows Azure complete , That is to say CNTV There's a baseline inside , The business below the baseline is placed in the original IT In the system , And businesses above the baseline are Windows Azure, For example, every day there are 4000 Ten thousand UV visit , The baseline is 1600 Ten thousand UV, exceed 1600 ten thousand UV The request will be placed in Windows
Azure above , And the whole handover process belongs to seamless handover , Users can't feel it at all .

stay Windows Azure in ,CNTV Of VDN front end Tomcat The server uses Azure Cloud Service Medium Worker Role, Receive requests from clients , And return the result . The user to use Eclipse, install Azure SDK Plug ins as development and release environments . On the front end , The user deployed two Cloud Service, One is deployed at Azure Beijing Data Center , The other is in Shanghai data center . In two VIP The front end of the , User F5 The equipment is right HTTP Request diversion .

When Tomcat After the server receives the request , Need to visit first MEMCACHE The server , Find data , If there is no data to query in the cache , Access the background MySQL database , And update the MEMCACHE. stay Windows Azure in ,MEMCACHE The server and MySQL The server , The load balancing set is composed of multiple instances .

Running in the user's own data center MySQL Main library . The main warehouse passes VPN Connect , Periodically synchronize data to Windows Azure From the library on .

stay Azure Cloud Service in , Each role corresponds to multiple instances , Belong to the same load balancing set . When one of the virtual machines goes wrong ,Windows Azure A new virtual machine replacement will be generated automatically . When the system has background maintenance ,Windows Azure It will also ensure that the virtual machines will not restart at the same time , So as to ensure the continuity of services and the high availability of the system .

For DBMS ,MySQL Master DB In the customer's own data center ,Slave MySQL stay Windows Azure in . Both sides go through MySQL Bin-log Data synchronization . meanwhile , Two slave libraries belong to a high availability set . Through the open source framework Guzz Realized MySQL Separation of reading and writing .

Reference documents

  1. Windows Azure Plugin for Eclipse with Java
  2. http://msdn.microsoft.com/library/azure/hh694271.aspx
  3. download Eclipse IDE: http://www.eclipse.org/downloads/
  4. download Azure Storage Explorer: http://azurestorageexplorer.codeplex.com/
  5. Download for Azure High speed upload and download tools for storage AzCopy: http://blogs.msdn.com/b/windowsazurestorage/archive/2012/12/03/azcopy-uploading-downloading-files-for-windows-azure-blobs.aspx
  6. download Azure PowerShell: http://az412849.vo.msecnd.net/downloads01/windowsazure-powershell.0.6.15.msi
  7. Download cross platform command line interface : http://az412849.vo.msecnd.net/downloads02/windowsazure-cli.0.6.17.msi
  8. stay Azure In Chinese, it means Linux Virtual machine makes image file : http://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-capture-image/
  9. How to scale the application : http://www.windowsazure.cn/zh-cn/manage/services/cloud-services/how-to-scale-a-cloud-service/

thank Ma Guoyao A review of this article , Liu Dawei and Shirley Wang The planning of this article .

Reprinted from : http://www.infoq.com/cn/articles/best-practices-of-building-large-scale-systems

Based on open source software in Azure More articles on best practices for building large-scale systems on platforms

  1. Build high performance clusters based on open source software NAS System , Including load balancing ( Liu Aigui )

    The era of big data has come unstoppable , In the face of the explosive growth of data , Especially semi-structured data and unstructured data ,NoSQL Storage system and distributed file system have become the wave of technology , It has made great progress . Unstructured data is now showing a more rapid growth trend ,IDC ...

  2. [ turn ] stay Azure Best practices for designing large scale services on cloud services

    In this paper, from :http://technet.microsoft.com/zh-cn/magazine/jj717232.aspx English version :http://msdn.microsoft.com/library ...

  3. Fur yes .NET 5 Best practice framework for enterprise application development under the platform .

    Fur yes .NET 5 Best practice framework for enterprise application development under the platform . Try it now Fur It's based on the latest .NET 5 RC2 structure , The goal is to experience new features as soon as possible , For the coming .NET 5 The official version gives the fastest response . the ...

  4. be based on React Native Mobile platform R & D practice sharing

    Reprint :http://blog.csdn.net/haozhenming/article/details/72772787 Contents of this article : One .React Native  It has become the trend of mobile front end technology Two . The base ...

  5. Ruffian scale embedded : Open source software protocol (MIT/BSD/Apache/LGPL/MPL/GPL)

    Hello everyone , I'm ruffian Heng , It's a serious technical ruffian . Today, ruffian Heng is talking about the basic knowledge of open source software protocol . Newton once said :" If I see further than others , That's because I stand on the shoulders of giants ". In software development, if there is a giant ...

  6. Tencent large scale Hadoop Cluster practice [ Turn to programmer magazine ]

    TDW(Tencent distributed Data Warehouse, Tencent distributed data warehouse ) Based on Open Source Software Hadoop and Hive Build , Break the traditional data warehouse can not be linear expansion . The limitation of poor controllability , And according to Teng ...

  7. On a large scale Hadoop Cluster practice : Tencent distributed data warehouse (TDW)

    TDW Tencent is the largest offline data processing platform . This article mainly from the demand . Challenge . Plans and future plans, etc , It introduces TDW In the construction of a single large-scale cluster JobTracker Decentralization and NameNode There are two optimization schemes for high availability . TD ...

  8. Windows Azure Safety best practices - The first 7 part : Tips 、 Tools and coding best practices

    In the process of writing this series of articles , I've come up with a lot of best practices . In this article , I introduced you to WindowsAzure More things to consider when applying . Here are some tools and coding tips and best practices : ·  It runs on the operating system ...

  9. dried food : be based on Git Flow Of Git Best practices ( In addition, we can solve the problems we often encounter )

    Suddenly I want to write this article Git How to use it , There are several main reasons , One is to use it yourself Git There is also fast 3 Years time. , In the meantime, I have experienced some pits , There are also times of confusion , In the small and large teams I have been in , In fact, everyone is not Git Experts , A lot about processes and G ...

Random recommendation

  1. Jsp The difference between static inclusion and dynamic inclusion

    1 <%@include file="xxx.jsp"%> by jsp Compile instructions in , The inclusion of the file occurs in jsp towards servlet The period of transformation , and <jsp:include ...

  2. Linux Basics - Common commands

    Common compression command One .tar 1. Compress :tar zcvf [ Package name ].tar.gz [ Filename to be compressed  ... ] 2. decompression :tar zxvf [ Package name ].tar.gz -C [ The specified decompression Directory ] 3 ...

  3. appear ( linker command failed with exit code 1) Error summary

    This kind of problem , It usually occurs when a third-party library file is added or when multiple people develop it . This kind of problem is usually a link error caused by not finding the file . We can start from the following aspects . 1. Take the following error as an example , If it's multi person development , After synchronization, you find the following error . ...

  4. How to Modify Public Network Information including VIP in Oracle Clusterware ( file ID 276434.1)

    APPLIES TO: Oracle Database - Enterprise Edition - Version 11.2.0.3 to 12.1.0.2 [Release 11.2 to 12. ...

  5. 【 turn 】 Multicore CPU Operation mode

    Multicore CPU There are three main modes of operation : • Asymmetric multiprocessing (Asymmetric multiprocessing,AMP)—— Every CPU The kernel runs a separate operating system or a separate instance of the same operating system (instantiat ...

  6. Mybatis Source brief

    Please quote from :http://www.cnblogs.com/xmzJava/p/8578399.html Daily development ,mybatis If it's wrong, it'll be more troublesome , Because one layer covers the other , If there is no right myba ...

  7. Linux Interprocess communication

    One .Linux Communication mode between processes 1) The Conduit (Pipe) And famous pipes (named pipe): Pipes can be used for communication between processes with kinship , The famous pipeline overcomes the limitation that the pipeline has no name , therefore , In addition to the functions of pipeline , It also allows ...

  8. 07:vue Define routes

    1.1 Define routes 1. explain 1.  Routing is a single page application (SPA) The key to ,Vue Provide routing plug-ins , To use this route, you need to install this plug-in 2.  install : npm install vue-router 3.  Depend on v ...

  9. 【 Reprint 】.NET A necessary toolkit for developers

    The author of this article Spencer It's a teacher who focuses on ASP.NET and C# The programmer , He listed his usual work . Most of the development tools used at home , Most of these tools are focused on Development , Of course, there are also some other uses , For example, image processing . File compression, etc . If you are ...

  10. String type

    One . immutable One String The value of a type cannot be changed , such as ,String china = " China ",“ China ” This string is immutable from its creation to its destruction . Two . String constant pool Literally ...