运维开发网

自动化二期CQC(TAOBAO TOAST框架二次开发)---支持结果展示

运维开发网 https://www.qedev.com 2020-03-07 09:01 出处:网络 作者:运维开发网整理
一.背景: 承接上一篇自动化二期CQC(TAOBAO TOAST框架二次开发)---支持自定义测试环境; maven工程使用surefire插件,执行"mvn -Dtest=测试类 test"命令,stdout并不支持输出成功用例和skipped用例(JUnit中被注解为@Ignore的类或方法)的信息,只输出Failed和error的用例信息,如下: ,故前端无法根据stdout解析出成功的和s

一.背景:

承接上一篇自动化二期CQC(TAOBAO TOAST框架二次开发)---支持自定义测试环境;

maven工程使用surefire插件,执行"mvn -Dtest=测试类 test"命令,stdout并不支持输出成功用例和skipped用例(JUnit中被注解为@Ignore的类或方法)的信息,只输出Failed和error的用例信息,如下:

自动化二期CQC(TAOBAO TOAST框架二次开发)---支持结果展示

,故前端无法根据stdout解析出成功的和skipped的用例详细。

尝试1:修改maven surefire插件源码让它支持输出success and fail的用例信息,查看源码,surefire(https://github.com/apache/maven-surefire/tree/master/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire)工程,见另一篇文章 

修改maven surefire源码,支持stdout出成功和skipped用例,并自定义stdout用例格式

尝试2:用surefire生成的xml文件(在项目target/surefire-reports)进行解析,surefire生成的xml中包含了成功的和skipped用例,但是surefire是根据单独一个测试类生成一个xml结果,而我们的用例,使用了@ClassnameFilters聚合了同一功能的测试类(正则匹配),比如评论功能

@ClassnameFilters({"com.weibo.cases.xiaoyu.*CommentTest", "com.weibo.cases.wanglei16.*CommentTest","com.weibo.cases.xuelian.*CommentTest","com.weibo.cases.hugang.*CommentTest","com.weibo.cases.maincase.*CommentTest"})

这样会生成很多不同被匹配类的xml,不好统计某一功能的结果。

当然可以尝试surefire-report功能,但是这就脱离了TOAST自己解析。

二.stdout支持输出成功用例和skipped用例详情

第二种尝试失败后,转向底层用例实现上。

1.stdout出执行成功用例

通过查看JUnit apidoc(http://junit.org/apidocs/index.html),JUnit提供了类

org.junit.rules.TestWatcher, 记录了测试用例执行过程中的行为,包括用例开始,成功,失败,skipped(a test is skipped due to a failed assumption,而不是@Ignore注释的), 结束等行为。

在每个测试类中添加@Rule,重新定义一个TestWatcher类实例,并重写successed(Description description)方法,stdout出执行成功的用例详情。
        @Rule 
	public TestWatcher testWatcher = new TestWatcher() {  
	    @Override  
	    protected void succeeded(Description description) {  
	        System.out.println("Success: " + description.getDisplayName());  
	    }  
	};
输出格式如下:
Success: testDestroyLikes(com.weibo.cases.wanglei16.LikesGroupTest)

2.stdout出skipped的用例(被注释为@Ignore的测试类或方法)

之前介绍过多线程执行用例,在定义ConcurrentSuite类的构造函数时,初始化 父类ClasspathSuite构造函数,重写了

runnerForClass方法,使用工厂类,创建Runner, 自定义一个新的Runner,通过反射找出带有注解@Ignore的类或方法并stdout。如果类有@Ignore,则只输出类名,因为Junit最后统计结果时,会把@Ignore类记为1个用例,不是计算该类下面的测试方法数(实验验证过);否则,遍历方法,如果方法有@Ignore,则输出该方法。

主要代码如下,完整版请查看多线程执行用例这篇文章:

public ConcurrentSuite(final Class<?> klass) throws InitializationError {
		// 调用父类ClasspathSuite构造函数
		// AllDefaultPossibilitiesBuilder根据不同的测试类定义(@RunWith的信息)返回Runner,使用职责链模式
		super(klass, new AllDefaultPossibilitiesBuilder(true) {
			@Override
			public Runner runnerForClass(Class<?> testClass) throws Throwable {
				List<RunnerBuilder> builders = Arrays
						.asList(new RunnerBuilder[] {
						// 创建Runner, 工厂类,
						// 自定义自己的Runner,找出注解为@Ignore,并输出@Ignore的类和方法名
							new RunnerBuilder() {
							@Override
							public Runner runnerForClass(
									Class<?> testClass)throws Throwable {
									// 获取类的所有方法
									Method[] methods = testClass.getMethods();
                                                        // 如果类有@Ignore,则只输出类名,因为Junit最后计算结果时,会把@Ignore类记为1个用例,
                                                                        // 不是计算类下面的测试方法数(实验验证过)
                                                                        // 否则,遍历方法,如果方法有@Ignore,则输出该方法
									if (testClass.isAnnotationPresent(Ignore.class)) {
										System.out.println("Ignore: "
													+ testClass.getName());

									} else {
									for (Method method : methods) {
											if (method.isAnnotationPresent(Ignore.class)) {
											System.out.println("Ignore: " + testClass.getName() + "." + method.getName());
										}
											}
										}						
										return null;
									}
								}, ignoredBuilder(), annotatedBuilder(),
								suiteMethodBuilder(), junit3Builder(),
								junit4Builder() });
				for (RunnerBuilder each : builders) {
					// 根据不同的测试类定义(@RunWith的信息)返回Runner
					Runner runner = each.safeRunnerForClass(testClass);
					if (runner != null)
						// 方法级别,多线程执行
						// return MulThread(runner);
						return runner;
				}
				return null;
			}
		});
 
 Ignore输出格式: 
 
Ignore: com.weibo.cases.wanglei16.LikeObjectRpcTest
Ignore: com.weibo.cases.wanglei16.LikesGroupTest.test4
结合mvn test stdout出的fail,error用例格式,完整的用例详情stdout格式如下:

skipped用例
Ignore: com.weibo.cases.wanglei16.LikeObjectRpcTest
Ignore: com.weibo.cases.wanglei16.LikesGroupTest.test4

pass用例
Success: testLikeStatus(com.weibo.cases.wanglei16.LikeObjectRpcTest)
Success: testLikesUpdate(com.weibo.cases.wanglei16.LikeObjectRpcTest)

Failed tests: 
  LikeObjectRpcTest.testLikesUpdate:153 
Expected: is "12312"
     but: was "1042018:10012099744"
  LikesByMeBatchTest.testSingleType:183 
Expected: is an empty collection
     but: <[com.weibo.model.Objects@42a6f5df]>

Tests in error: 
  LikesByMeBatchTest.testMultiType:190 » ArrayIndexOutOfBounds -1

对应的正则表达式:

fail
$failPattern = "#\s{2}(\w+\.\w+:\d+)\s[^\S]#";

error
$errorPattern = "#\s{2}(\w+\.\w+:\d{1,}\s[^\s].*)#";

Ignore
$skipPattern = "#Ignore:\s(.*)#";

success
$passPattern = "#Success:\s(.*)#";
修改解析文件:/toast/protected/parsers/JUnitMvnParser.php
<?php
/*
 *@author hugang
 *parse case info
 */
include_once('BaseParser.php');
class JUnitMvnParser extends BaseParser
{
    protected function parseCaseAmount()
    {
        $amountPattern = '#Tests run: (\d+), Failures: (\d+), Errors: (\d+), Skipped: (\d+)[\r\n]+#';
        preg_match_all($amountPattern, $this->output, $amountMatches);
        $this->parserInfo->case_total_amount = array_sum($amountMatches[1]);
        $this->parserInfo->case_failed_amount = array_sum($amountMatches[2]) + array_sum($amountMatches[3]);
        $this->parserInfo->case_skipped_amount = array_sum($amountMatches[4]);
        $this->parserInfo->case_passed_amount = $this->parserInfo->case_total_amount - $this->parserInfo->case_failed_amount - $this->parserInfo->case_skipped_amount;
    }

    protected function parseCases()
    {

        // 下行没有空格,自行清掉;[ INFO ]中了博客关键字
        $idPattern = "#\ [ INFO \ ][^\n]*\nCASE\sID:\s(\d*)#s";
        preg_match_all($idPattern, $this->output, $idMatches);


        // fail case
        $failPattern = "#\s{2}(\w+\.\w+:\d+)\s[^\S]#";
        preg_match_all($failPattern, $this->output, $fmatches);
        for($i = 0; $i < count($fmatches[1]); $i++) {
            $caseInfo = new CaseInfo();
            if (isset($idMatches[1][$i])) {
                $caseInfo->id = trim($idMatches[1][$i]);
            } else if (isset($idMatches[1])) {
                $idx = count($idMatches[1]);
                if (isset($idMatches[1][$idx - 1])) {
                    $caseInfo->id = trim($idMatches[1][$idx - 1]);
                }
            }
            $caseInfo->name = trim($fmatches[1][$i]);
            $caseInfo->info = trim($fmatches[1][$i]);
            $caseInfo->result = CaseInfo::RESULT_FAILED;
            if (!empty($caseInfo->id)) {
                $testcase = TestCase::model()->findByPk($caseInfo->id);
                if ($testcase !== null) {
                    $caseInfo->name = $testcase->name;
                }
            }
            $this->parserInfo->cases[] = $caseInfo;
        }

        // fail Expected info
        $failExpectPattern = "#(Expected:\sis.*)#";
        preg_match_all($failExpectPattern, $this->output, $fematches);
        for($j = 0; $j < count($fematches[1]); $j++){
            if(isset($this->parserInfo->cases[$j])){
                $caseInfo = $this->parserInfo->cases[$j];
                $caseInfo->info .= "\n<b>" .$fematches[1][$j] . "</b>";
                $this->parseInfo->cases[$j] = $caseInfo;
            }
        }


        // fail But info
        $failButPattern = "#\s{6}(but:\s.*)#";
        preg_match_all($failButPattern, $this->output, $fbmatches);
        for($m = 0; $m < count($fbmatches[1]); $m++){
            if(isset($this->parserInfo->cases[$m])){
                $caseInfo = $this->parserInfo->cases[$m];
                $caseInfo->info .= "<b>  " .$fbmatches[1][$m] . "</b>";
                $this->parseInfo->cases[$m] = $caseInfo;
            }
        }

        // error cases info
        $errorPattern = "#\s{2}(\w+\.\w+:\d{1,}\s[^\s].*)#";
        preg_match_all($errorPattern, $this->output, $fematches);
        for($i = 0; $i < count($fematches[1]); $i++) {
            $caseInfo = new CaseInfo();
            if (isset($idMatches[1][$i])) {
                $caseInfo->id = trim($idMatches[1][$i]);
            } else if (isset($idMatches[1])) {
                $idx = count($idMatches[1]);
                if (isset($idMatches[1][$idx - 1])) {
                    $caseInfo->id = trim($idMatches[1][$idx - 1]);
                }
            }
            $caseInfo->name = trim($fematches[1][$i]);
            $caseInfo->info = trim($fematches[1][$i]);
            $caseInfo->result = CaseInfo::RESULT_FAILED;
            if (!empty($caseInfo->id)) {
                $testcase = TestCase::model()->findByPk($caseInfo->id);
                if ($testcase !== null) {
                    $caseInfo->name = $testcase->name;
                }
            }
            $this->parserInfo->cases[] = $caseInfo;
        }
        // ignore cases info
        $skipPattern = "#Ignore:\s(.*)#";
        preg_match_all($skipPattern, $this->output, $smatches);
        foreach ($smatches[1] as $key => $smatch) {
            $caseInfo = new CaseInfo();
            if (isset($idMatches[1][$key])) {
                $caseInfo->id = trim($idMatches[1][$key]);
            } else if (isset($idMatches[1])) {
                $idx = count($idMatches[1]);
                if (isset($idMatches[1][$idx - 1])) {
                    $caseInfo->id = trim($idMatches[1][$idx - 1]);
                }
            }
            $caseInfo->name = trim($smatches[1][$key]);
            $caseInfo->info = trim($smatches[1][$key]);
            $caseInfo->result = CaseInfo::RESULT_SKIPPED;
            if (!empty($caseInfo->id)) {
                $testcase = TestCase::model()->findByPk($caseInfo->id);
                if ($testcase !== null) {
                    $caseInfo->name = $testcase->name;
                }
            }
            $this->parserInfo->cases[] = $caseInfo;
        }


        // success cases info
        $passPattern = "#Success:\s(.*)#";
        preg_match_all($passPattern, $this->output, $pmatches);
        foreach ($pmatches[1] as $key => $pmatch)
        {
            $caseInfo = new CaseInfo();
            if(isset($idMatches[1][$key]))
            {
                $caseInfo->id = trim($idMatches[1][$key]);
            }
            else if(isset($idMatches[1]))
            {
                $idx = count($idMatches[1]);
                if(isset($idMatches[1][$idx-1]))
                {
                    $caseInfo->id = trim($idMatches[1][$idx-1]);
                }
            }
            $caseInfo->name = trim($pmatches[1][$key]);
            $caseInfo->info = trim($pmatch);
            $caseInfo->result = CaseInfo::RESULT_PASSED;
            if(!empty($caseInfo->id))
            {
                $testcase = TestCase::model()->findByPk($caseInfo->id);
                if($testcase !== null)
                {
                    $caseInfo->name = $testcase->name;
                }
            }
            $this->parserInfo->cases[] = $caseInfo;
        }


    }
}
?>
结果展示如下,展示顺序(1.failed cases->2.error cases->3.skipped cases->4.success cases):

自动化二期CQC(TAOBAO TOAST框架二次开发)---支持结果展示

0

精彩评论

暂无评论...
验证码 换一张
取 消