MySQL是怎么保证产品质量的?

How to guarantee the quality of products for MySQL?

强大的测试力量+对产品质量的重视+资源投

原文:  http://mysqlserverteam.com/mysql-maintenance-releases/

 

一 多种测试

What kind of testing is done upfront?

We do continuous intergration testing of all our release branches (5.5, 5.6 and trunk). This consists of:
1 Per-push testing: a subset of the MTR test suite on a selected set of platforms.
2 Daily testing: a bigger subset of the MTR test suite on a narrow set of platforms.
3 Weekly testing: the entire MTR test suite on all release platforms.

Random Query Generator (RQG) tests are also run on per-push, daily and weekly.

In addition, we run daily performance regression testing, stress/load testing, replication testing, as well as specialized builds like Werror, Valgrind and AddressSanitizer. On a weekly basis we run upgrade-downgrade testing and additional performance tests.

An important aspect of the work on continuous testing is the human factor. The QA team and the release managers follow up on errors introduced by developers and pushes that break tests are not allowed to remain for long without a fix.

 

二 产品发布所做的测试

How do we produce a maintenance release?

About a month before the projected release date, the release manager cuts off the maintenance release and creates the release branch.

Packages get built automatically in our combined build/test system and they are handed over for integration testing:

Testing with other products (connectors, MEB, Workbench…)
Upgrade-downgrade testing
Performance testing (sysbench, DBT2)
Load-stress testing with and without replication (using mysql-stress-test.pl)
Large volume testing
Package validation/installation/upgrade (on a wide range of platforms and packages)
Windows installer testing

From the point where integration testing starts, no more fixes except showstoppers into the release. If a showstopper is found, we respin the release and retest it.