TF框架学习笔记

简介

TF：Trade Federation，一种google官方推出的测试框架，可以运行uiautomator，instrumentation，junit等写的case，支持多台设备同时运行。

source code:https://github.com/testobject/tradefed

该框架分为3个步骤：

developer

用java语言开发module

integrator

配置case

test runner

运行case查看结果

环境配置

需要将git上的source下到本地，然后执行build命令，启动tf环境。

设备

真机、模拟器、无设备

测试生命周期

1.build

2.配置文件

3.跑case

4.监听running过程

参数配置

开发时配置：

public class PhoneCallFuncTest extends IRemoteTest {
    @Option(name = "timeout", description = "How long to wait for connection, in millis")
    private long mWaitTime = 30 * 1000;  // 30 seconds

    @Option(name = "call", description = "Key: Phone number to attempt.  " +
            "Value: DTMF to expect.  May be repeated.")
    private Map<String, String> mCalls = new HashMap<String, String>;

    public PhoneCallFuncTest() {
        mCalls.add("123-456-7890", "01134");  // default
    }

Integrator中配置：

<?xml version="1.0" encoding="utf-8"?>
<configuration description="low-latency default test; low-latency.xml">
    <test class="com.example.PhoneCallFuncTest">
        <option name="timeout" value="5000" />
    </test>
</configuration>

Test runner：

tf >run low-latency.xml --call 111-111-1111 #*#*TEST1*#*# --call 222-222-2222 #*#*TEST2*#*#

tf >run many-numbers.xml --timeout 5000

HelloWorld

Developer：

package com.android.tradefed.example;

import com.android.tradefed.device.DeviceNotAvailableException;
import com.android.tradefed.result.ITestInvocationListener;
import com.android.tradefed.testtype.IRemoteTest;

public class HelloWorldTest implements IRemoteTest {
    @Override
    public void run(ITestInvocationListener listener) throws DeviceNotAvailableException {
        System.out.println("Hello, TF World!");
    }
}

case写完，保存在/tools/tradefederation/prod-tests/src/com/android/tradefed/example/HelloWorldTest.java，重新build。命令：

m -jN

Integrator:

<configuration description="Runs the hello world test">
    <test class="com.android.tradefed.example.HelloWorldTest" />
</configuration>

然后TF会通过该配置文件，利用反射的方式找到HelloWorldTest类，实例化它，调用IRemoteTest类的run方法启动这条case.然后把该配置文件放在本地，例如tmp/helloworld.xml

Runner:

进入tf命令行：

tradefed.sh

参看设备连接：

tf >list devices

连接正常后，启动case,(配置xml放在tmp目录下)

tf> run /tmp/helloworld.xml

当然你可以放在你的java文件同一个目录。exmaple/helloworld.xml

tf >run example/helloworld

与系统交互

public class HelloWorldTest implements IRemoteTest, IDeviceTest {
    private ITestDevice mDevice;
    @Override
    public void setDevice(ITestDevice device) {
        mDevice = device;
    }

    @Override
    public ITestDevice getDevice() {
        return mDevice;
    }
    <pre class="prettyprint"><code><span class="lit">    @Override</span><span class="pln">
</span><span class="kwd">    public</span><span class="pln"> </span><span class="kwd">void</span><span class="pln"> run</span><span class="pun">(</span><span class="typ">ITestInvocationListener</span><span class="pln"> listener</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">throws</span><span class="pln"> </span><span class="typ">DeviceNotAvailableException</span><span class="pln"> </span><span class="pun">{</span><span class="pln">
          </span><span class="typ">System</span><span class="pun">.</span><span class="kwd">out</span><span class="pun">.</span><span class="pln">println</span><span class="pun">(</span><span class="str">"Hello, TF World! I have device "</span><span class="pln"> </span><span class="pun">+</span><span class="pln"> getDevice</span><span class="pun">().</span><span class="pln">getSerialNumber</span><span class="pun">());</span><span class="pln">
</span><span class="pun">    }</span><span class="pln">
</span></code>

 }

TF框架会在IRemoteTest的run方法开启前，通过setDevice将device注入到case中。

$ tradefed.sh
tf >list devices

tf >run example/helloworld
05-12 13:26:18 I/TestInvocation: Starting invocation for target stub on build 0 on device 004ad9880810a548
Hello, TF World! I have device 004ad9880810a548

获得了设备的serial number号。

测试结果

ITestInvocationListener 类为辅助输出结果的类，该类提供了很多方法，在case执行开始到结束的过程中，扑捉case状态，输出相应的log.

@Override
public void run(ITestInvocationListener listener) throws DeviceNotAvailableException {
    System.out.println("Hello, TF World! I have device " + getDevice().getSerialNumber());

    TestIdentifier testId = new TestIdentifier("com.example.TestClassName", "sampleTest");
    listener.testRunStarted("helloworldrun", 1);
    listener.testStarted(testId);
    listener.testFailed(TestFailure.FAILURE, testId, "oh noes, test failed");
    listener.testEnded(testId, Collections.emptyMap());
    listener.testRunEnded(0, Collections.emptyMap());
}

运行结果如下所示：

$ ./tradefed.sh
tf >run example/helloworld
05-16 20:03:15 I/TestInvocation: Starting invocation for target stub on build 0 on device 004ad9880810a548
Hello, TF World! I have device 004ad9880810a548
05-16 20:03:15 I/InvocationToJUnitResultForwarder: run helloworldrun started: 1 tests
Test FAILURE: com.example.TestClassName#sampleTest 
 stack: oh noes, test failed 
05-16 20:03:15 I/InvocationToJUnitResultForwarder: run ended 0 ms

如果你想把log输出到文件中，也是需要配置一个xml文件的。

<configuration description="Runs the hello world test">
    <test class="com.android.tradefed.example.HelloWorldTest" />
    <result_reporter class="com.android.tradefed.result.XmlResultReporter" />
</configuration>

TF框架会通过XmlResultReporter监听器来处理log，将其结果写到一个xml文件中。执行该case:

<pre class="prettyprint"><code><span class="pln">tf </span><span class="pun">></span><span class="pln">run example</span><span class="pun">/</span><span class="pln">helloworld
</span><span class="lit">05</span><span class="pun">-</span><span class="lit">16</span><span class="pln"> </span><span class="lit">21</span><span class="pun">:</span><span class="lit">07</span><span class="pun">:</span><span class="lit">07</span><span class="pln"> I</span><span class="pun">/</span><span class="typ">TestInvocation</span><span class="pun">:</span><span class="pln"> </span><span class="typ">Starting</span><span class="pln"> invocation </span><span class="kwd">for</span><span class="pln"> target stub on build </span><span class="lit">0</span><span class="pln"> on device </span><span class="lit">004ad9880810a548</span><span class="pln">
</span><span class="typ">Hello</span><span class="pun">,</span><span class="pln"> TF </span><span class="typ">World</span><span class="pun">!</span><span class="pln"> I have device </span><span class="lit">004ad9880810a548</span><span class="pln">
</span><span class="lit">05</span><span class="pun">-</span><span class="lit">16</span><span class="pln"> </span><span class="lit">21</span><span class="pun">:</span><span class="lit">07</span><span class="pun">:</span><span class="lit">07</span><span class="pln"> I</span><span class="pun">/</span><span class="typ">XmlResultReporter</span><span class="pun">:</span><span class="pln"> </span><span class="typ">Saved</span><span class="pln"> device_logcat log to </span><span class="pun">/</span><span class="pln">tmp</span><span class="pun">/</span><span class="lit">0</span><span class="pun">/</span><span class="pln">inv_2991649128735283633</span><span class="pun">/</span><span class="pln">device_logcat_6999997036887173857</span><span class="pun">.</span><span class="pln">txt
</span><span class="lit">05</span><span class="pun">-</span><span class="lit">16</span><span class="pln"> </span><span class="lit">21</span><span class="pun">:</span><span class="lit">07</span><span class="pun">:</span><span class="lit">07</span><span class="pln"> I</span><span class="pun">/</span><span class="typ">XmlResultReporter</span><span class="pun">:</span><span class="pln"> </span><span class="typ">Saved</span><span class="pln"> host_log log to </span><span class="pun">/</span><span class="pln">tmp</span><span class="pun">/</span><span class="lit">0</span><span class="pun">/</span><span class="pln">inv_2991649128735283633</span><span class="pun">/</span><span class="pln">host_log_6307746032218561704</span><span class="pun">.</span><span class="pln">txt
</span><span class="lit">05</span><span class="pun">-</span><span class="lit">16</span><span class="pln"> </span><span class="lit">21</span><span class="pun">:</span><span class="lit">07</span><span class="pun">:</span><span class="lit">07</span><span class="pln"> I</span><span class="pun">/</span><span class="typ">XmlResultReporter</span><span class="pun">:</span><span class="pln"> XML test result file generated at </span><span class="pun">/</span><span class="pln">tmp</span><span class="pun">/</span><span class="lit">0</span><span class="pun">/</span><span class="pln">inv_2991649128735283633</span><span class="pun">/</span><span class="pln">test_result_536358148261684076</span><span class="pun">.</span><span class="pln">xml</span><span class="pun">.</span><span class="pln"> </span><span class="typ">Total</span><span class="pln"> tests </span><span class="lit">1</span><span class="pun">,</span><span class="pln"> </span><span class="typ">Failed</span><span class="pln"> </span><span class="lit">1</span><span class="pun">,</span><span class="pln"> </span><span class="typ">Error</span><span class="pln"> </span><span class="lit">0</span></code>

最后一行会生成一个xml文件，打开：

<?xml version='1.0' encoding='UTF-8' ?>
<testsuite name="stub" tests="1" failures="1" errors="0" time="9" timestamp="2011-05-17T04:07:07" hostname="localhost">
  <properties />
  <testcase name="sampleTest" classname="com.example.TestClassName" time="0">
    <failure>oh noes, test failed
    </failure>
  </testcase>
</testsuite>

你也可以写自己的监听器，只要继承自ITestInvocationListener就行！

参数传递

可以以注解的形式配置一些参数：

@Option(name="my_option",
        shortName='m',
        description="this is the option's help text",
        // always display this option in the default help text
        importance=Importance.ALWAYS)
private String mMyOption = "thisisthedefault";

<pre class="prettyprint"><code><span class="lit">@Override</span><span class="pln">
</span><span class="kwd">public</span><span class="pln"> </span><span class="kwd">void</span><span class="pln"> run</span><span class="pun">(</span><span class="typ">ITestInvocationListener</span><span class="pln"> listener</span><span class="pun">)</span><span class="pln"> </span><span class="kwd">throws</span><span class="pln"> </span><span class="typ">DeviceNotAvailableException</span><span class="pln"> </span><span class="pun">{</span><span class="pln">
    </span><span class="pun">…</span><span class="pln">
    </span><span class="typ">CLog</span><span class="pun">.</span><span class="pln">logAndDisplay</span><span class="pun">(</span><span class="typ">LogLevel</span><span class="pun">.</span><span class="pln">INFO</span><span class="pun">,</span><span class="pln"> </span><span class="str">"I received option '%s'"</span><span class="pun">,</span><span class="pln"> mMyOption</span><span class="pun">);</span><span class="pln">
</span></code>

 

然后运行得到输出：

tf> run example/helloworld
…
05-24 18:30:05 I/HelloWorldTest: I received option 'thisisthedefault'

然后在命令行里传参

tf> run example/helloworld --my_option foo
…
05-24 18:33:44 I/HelloWorldTest: I received option 'foo'

也可以在命令行的help菜单里找到该参数

tf> run --help example/helloworld
Printing help for only the important options. To see help for all options, use the --help-all flag

  cmd_options options:
    --[no-]help          display the help text for the most important/critical options. Default: false.
    --[no-]help-all      display the full help text for all options. Default: false.
    --[no-]loop          keep running continuously. Default: false.

  test options:
    -m, --my_option      this is the option's help text Default: thisisthedefault.

  'file' logger options:
    --log-level-display  the minimum log level to display on stdout. Must be one of verbose, debug, info, warn, error, assert. Default: error

importance属性决定其是否显示在帮助菜单里。

除了注解的方式，还可以通过配置文件的信息做到接受命令行的参数：

<test class="com.android.tradefed.example.HelloWorldTest" >
    <option name="my_option" value="fromxml" />
</test>

05-24 20:38:25 I/HelloWorldTest: I received option 'fromxml'

tf> run --help example/helloworld
  test options:
    -m, --my_option      this is the option's help text Default: fromxml.

在输出的时候也可以通过参数来设置log输出的格式：

tf >run --log-level-display info example/helloworld
…
05-24 18:53:50 I/HelloWorldTest: Hello, TF World! I have device 004ad9880810a548