Status and challenges of automated testing
With the continuous improvement of people's requirements for the quality of life , The frequency of product upgrading in the market also increases , This is a big challenge for product developers . And we explore the significance of automated testing , Is to help the business with better quality , Occupy the market faster .
Compared to manual testing , Automated testing has higher test coverage , In the face of TOP300 Model and TOP500 The test efficiency of the model is also higher . meanwhile , Automated testing can work with continuously integrated tools , A quick response version , When you reach a version, you can do a regression test , Even one code submission can perform a regression test . And automated testing can also integrate a number of special tests .
In recent years, within Tencent , The demand for automated testing continues to rise . from 2019 Year to 2020 According to the data of , The number of automatic test cases of Tencent's head project has increased 50%, The frequency of automated testing has increased 200%. This paper will comprehensively analyze , How to explore and build automated testing within Tencent .
Exploration of intelligent automatic test
Tencent has a large number of internal products , There are many types of coverage , For the access mode 、 There are differences between test scenarios and test requirements . An excellent automatic test platform , Need to help the business solve the following problems ：
- efficiency ： Service access is convenient and fast , Support multiple access modes
- You can use ： Test task execution is stable ：99.9%+
- effect ： Convenient problem analysis and location , It's easy to return
that WeTest As a one-stop test platform recognized by many well-known Tencent products , How to build the technical scheme of intelligent automatic test ？
Construction of full link test capability
WeTest Automated test platform provides from scripting to task triggering 、 Task execution 、 Mission report , Finally, a series of automated testing capabilities for problem debugging . meanwhile , Also available from IAAS、PAAS、SAAS Different levels of access methods . stay IAAS layer ,WeTest It includes signal shielding cabinet 、 Thermostatic laboratory , A series of hardware facilities such as network dedicated lines at home and abroad ; stay PAAS The layer provides different access methods . stay SAAS The layer provides automatic test capability of the whole link . Business can be done only through IAAS Layer access to use these underlying hardware and devices , It can also be done through PAAS Layer to connect the whole test platform , It can also go directly through SAAS The ability to use these automated tests .
Multiple access mechanisms , Meet the docking of various systems
Facing different service access requirements ,WeTest Provide different access methods . for example CI plug-in unit , As a CI A link in automated testing , You can smoke or return to version builds regularly every day ; Facing the business with test platform ,WeTest Can provide REST API Platform level docking in the way of ; According to the different needs of the business side ,WeTest Also available in web Automated tests that can be submitted with one click on the page , And a command line tool that can submit automated tests by executing a simple command .
Mirrored test framework deployment
In automated testing , The automated testing technologies of different business teams are very different , The script framework it uses 、 Script environment 、 The requirements of terminal equipment are different , Even some user scripts may have some security problems .
To solve this kind of problem ,WeTest Provides a mirrored deployment scheme . When the business team deploys the test framework , You only need to generate the corresponding image to use . The script or test framework library that needs to be relied on in the image , Just by writing DockerFile Packaged as a mirror , Then upload it to the image warehouse of the platform . When users submit automated tests , The platform system will pull the image required by the user from the user's image warehouse , And then in the whole K8S In the cluster , Run in the form of pulling .
Use K8S Clusters can also automatically load balance , Let the task execution containers be evenly distributed on different nodes , It is also very convenient for capacity expansion .
Configurable test scheduling
equipment 、 Use cases 、 Scripts are resources that can be configured and scheduled . For example, when users need to schedule use cases , You only need to configure the required execution case and the equipment execution conditions required by the use case , The platform will execute according to the historical execution time of user use cases , Conditions required for use cases 、 The estimated results of the use case and how the use case is executed , Optimal allocation to equipment , So as to speed up the execution of use cases . For the scheduling of models , The platform will be based on certain algorithms , Select the equipment that is idle and can meet the user's test needs , So as to improve the test efficiency .
Centralization 、 Networking 、 clustering
At present, most of the automated test platforms in the industry , The hardware connected to the platform facilities is a splicing of consumer products , As a result, the availability of the mobile phone depends on the availability of all hardware of the whole link . When one of the hardware is unavailable , The phone will not work . This will not only make the single point of failure become the bottleneck of the whole mobile phone , The overall control link will also be very long . In the face of such problems ,WeTest Put forward “ Hardware data centralization 、 Mobile control networking 、 Cluster the execution machines ” The new plan .
Comprehensive test report
In order to find problems quickly and conveniently after the business completes the test 、 Locate the problem and perform regression testing ,WeTest Provides a full range of test reports . Include device logs 、 The performance data 、 network data 、 Screenshot data 、 Wrong stack information 、 Playback of the whole test process, etc . For the problems of individual mobile phones in the report , The platform also provides remote debugging function , Support HD picture and audio input and output .
Strong confidentiality for some businesses 、 Special requirements for equipment timeliness , The platform can provide a proprietary cloud exclusive to the business . It can not only meet the confidentiality requirements ,0 Waiting for equipment timeliness , At the same time, the equipment can be specially configured and deployed , Meet user specific application requirements 、 Don't unload 、 Login status is permanent 、 Automatic Chat 、 Automatic approval and other demand scenarios .
Intelligent automated testing
In order to better protect the product quality of users , Give users a better platform experience . We have been studying the automatic testing technology 、 Explore and think , Constantly pursue technological progress .
For low code tools and AI Areas to explore , Then we developed a new generation of low code automation tools for script recording ——WeAutomator. This low code tool includes a variety of script recording functions , Including the most basic through image 、 Integrated recording of space and coordinates , Intelligent recording according to the edge algorithm of the picture can also be provided . And this tool supports scenarios and code patterns , Support Android And iOS.
King glory skin automated test
The left screen is the user's real game screen , The right screen is a special test map provided by King glory for the automatic test of the screen . It changed time into a logical title , At the same time, the noise of the map is removed , Turn the bottom into a uniform blue , And heroes who don't need to be tested by the enemy 、 Blood volume 、 The hero's name is hidden .
Throughout the process of automated testing , The system will record , After recording, each frame of each video will pass OCR Identify and extract , Then save all the picture frames . Images between different iterations , Will pass ORB Feature matching and accurate comparison of images , Use the picture comparison library to compare , So as to filter out the pictures with differences for annotation , Then judge whether there is UI abnormal . Of the entire automated test bug Discovery is very strong , Equivalent to human ability 5 times . meanwhile , Because the glory of the king has more resources , Conducting an automated test can save... Compared with manual testing 160 Hours .
Wechat client automation test case
Wechat related end-to-end UI Automated testing mainly involves ： Wechat basic client , payment UAT, The basics of applets , Search and other business tests . commonly UI Automated testing takes four steps ： choice UI Driving tools 、 Select the use case organization tool , Write use cases and use case deployment . The wechat client project team will work according to its own business conditions , When deploying use cases , Choose the 3 It's time to get involved in automated testing .
1. Closing inspection
Develop each submission MR Will check the compilation , Startup issues . This opportunity is mainly to check the launch of wechat / Whether the message is successful , use 3 Use cases + Log in to test .
2. Red version test
On this version , It is necessary to ensure that the basic application functions are stable and available , Allow new features or some scenes to bring bug. After compiling the package, intervene in a selected use case （ Probably 40 Use cases ） To test , Verify whether the basic functions are executed successfully , Mainly to detect whether there are some problems that obviously affect the experience .
3. P0 test
The requirements that pass the test will be merged into the release branch for release , After closing, it will trigger packaging . Intervene to execute after packaging P0 automated testing （ Probably 400 Use cases ）, Testing can ensure that 26min Run left and right .
Wechat access WeTest After automated testing , Use cases can be distributed , Use case result storage , Test report and other functions to WeTest, I have more energy to focus on business testing ,WeTest Provides high stability equipment 、 Automation equipment operation and maintenance monitoring , The test stability reaches 99.68%, Test efficiency is improved 40%, So that users don't have to worry about equipment problems , More efficient and convenient inspection of product quality .
1. Equipment dimension ： High equipment stability , Average monthly test volume 100w+ In the case of sets, the equipment success rate and test task execution rate are as high as 99.9%
2. Use case latitude ： High use case coverage , Dynamic distribution through use cases , Shorten the test time , Support flexible use case distribution patterns
3. Efficiency dimension ： More resources , Faster , Test and improve efficiency 40%
- WeTest Tencent has been covered 6 Big BG
tencent WeTest It is a one-stop quality open platform officially launched by Tencent . More than ten years of quality management experience , Committed to the construction of quality standards 、 Product quality improvement . tencent WeTest Provide compatibility testing for mobile developers 、 Yunzhenji 、 Performance testing 、 Excellent R & D tools such as safety protection , Provide solutions for more than 100 industries , Covering products in R & D 、 Test requirements in all stages of operation , Through thousands of products . Team of gold medal experts , adopt 5 Big dimension ,41 Indicators ,360 To guarantee the quality of your products .