敏捷实践 (2) - appium支持与无法支持的测试

---

1. 简介

在探索App自动化测试工具过程中，主要接触了 Macaca 和 Appium, 已及稍稍看了点 Calabash。

其中，Calabash 与 Appium都支持使用 Cucumber 编写测试用例。由于精力有限，Calabash没有更进一步研究，有兴趣的朋友可以看看，交流一下心得。

Macaca在 敏捷实践 - 我们是如何自动化App验收标准（一） 一文中有提到过，是我的前端小伙伴Lorne向我推荐的，看了文档，也装上试了试，不得不说Macaca非常优秀，上手非常容易，同时支持 NodeJS 与 Python 与 Java 三大主流语言。就是文档稍有些少。推荐大家试试。

Appium是个非常优秀的App自动化测试工具，支持的语言比Macaca多一些。让我选择Appium而不是Macaca的关键因数并非所支持语言的多少，而是Appium支持用Ruby+Cucumber来写测试用例，简直就像为我们量身定造。没错，我最喜欢用的语言首选Ruby，其次NodeJS/Python，再之Java。

我们的测试用例起步于 appium sample-code 中的ruby部分，强烈推荐大家去翻翻代码。

---

2. Appium

Appium的详细介绍信息可以参考 在线文档，在这里我并不打算太多重复，而且网上也有不少教程了，只想概要的说明一下它是如何工作的。

---

2.1 Appium的体系结构

Appium是用NodeJS写的一个HTTP服务，因此使用它需要先安装NodeJS。它又是通过WebDriverAgent组件在移动设备中控制被测试应用执行测试指令。

---

2.2 工作流程图

注意，这个流程图并非官方图，只是我随手画的帮助大家理解的示意图。

appium_flow.png

测试脚本与Appium与被测试的APP是分别各自独立的进程，甚至分署在不同的主机与真实设备中。

测试的执行过程为

1). 测试脚本调用appium_lib提供的方法, 如 find, click
2). appium_lib调用转给Selenium WebDriver的Remote Adapter
3). Selenium WebDriver将请求封装为JSON，以HTTP Rest协议向Appium发送请求
4). Appium收到HTTP请求后，将请求交给相应的AppiumDriver
5). AppiumDriver通过HTTP向运行在终端设备上的WebDriverAgent进程(后台app)发送请求
6). 设备上的WebDriverAgent解析请求后，通过XCUITest测试套件与目标测试App交互，并获取结果
7). WebDriverAgent将结果由HTTPResponse返回给Appium
8). Appium再将结果进行包装处理后，由HTTPResponse返回给3)的Selenium WebDriver
9). Selenium WebDriver再将结果转换成Ruby对象返回给测试脚本。  

---

2.3 Appium常用指令

官方已经有了非常完整的文档，我也就不再重复或做些翻译工作了。
Ruby语言请移步 https://github.com/appium/ruby_lib/blob/master/docs/docs.md

--

Example use of Appium's mobile gesture.

@driver.find_element()

console.rb uses some code from simple_test.rb and is released under the same license as Appium. The Accessibility Inspector is helpful for discovering button names and textfield values.

Long click on an ImageView in Android.

last_image = find_elements(:tag_name, :ImageView).last
long_press(element: last_image)

Rotate examples.

driver.rotate :landscape
driver.rotate :portrait

• status["value"]["build"]["revision"] Discover the Appium rev running on the server.
• driver.keyboard.send_keys "msg" Sends keys to currently active element

generic

• source Prints a JSON view of the current page.
• page Prints the content descriptions and text values on the current page.
• page_class Prints the classes found on the current page.
• (Element) find(value) Returns the first element that contains value.
• (Element) finds(value) Returns all elements containing value (iOS only for now).
• (Element) name(name) Returns the first element containing name. Android name is the content description.
  iOS uses accessibility label with a fallback to text.
• (Array) names(name) Returns all elements containing name.
• (Element) text(text) Returns the first element containing text.
• (Array) texts(text) Returns all elements containing text.
• current_app Returns information about the current app. Android only.

--

alert

1. (void) alert_accept Accept the alert.
2. (String) alert_accept_text Get the text of the alert's accept button.
3. (void) alert_click(value) iOS only Tap the alert button identified by value.
4. (void) alert_dismiss Dismiss the alert.
5. (String) alert_dismiss_text Get the text of the alert's dismiss button.
6. (String) alert_text Get the alert message text.

button

1. (Button) button(index) Find a button by index.
2. (Button) button(text) Get the first button that includes text.
3. (Array, Array) buttons(text = nil) Get an array of button texts or button elements if text is provided.
4. (Array) buttons(text) Get all buttons that include text.
5. (Button) first_button Get the first button element.
6. (Button) last_button Get the last button element.

textfield

1. (Textfield) textfield(index) Find a textfield by index.
2. (Array) textfields Get an array of textfield elements.
3. (Textfield) first_textfield Get the first textfield element.
4. (Textfield) last_textfield Get the last textfield element.
5. (Textfield) textfield_exact(text) Get the first textfield that matches text.
6. (Textfield) textfield(text) Get the first textfield that includes text.

text

The Static Text methods have been prefixed with s_ to avoid conflicting with the generic text methods.

1. (Text) text(index) Find a text by index.
2. (Array) texts Get an array of text elements.
3. (Text) first_text Get the first text element.
4. (Text) last_text Get the last text element.
5. (Text) text_exact(text) Get the first element that matches text.
6. (Text) text(text) Get the first textfield that includes text.

window

1. (Object) window_size Get the window's size.

--

e.name # button, text
e.value # secure, textfield
e.type 'some text' # type text into textfield
e.clear # clear textfield
e.tag_name # calls .type (patch.rb)
e.text
e.size
e.location
e.rel_location
e.click
e.send_keys 'keys to send'
e.set_value 'value to set' # ruby_console specific
e.displayed? # true or false depending if the element is visible
e.selected? # is the tab selected?
e.attribute('checked') # is the checkbox checked?


# alert example without helper methods
alert = $driver.switch_to.alert
alert.text
alert.accept
alert.dismiss

# Secure textfield example.
#
# Find using default value
s = textfield 'Password'
# Enter password
s.send_keys 'hello'
# Check value
s.value == ios_password('hello'.length)

--

Driver

start_driver will restart the driver.

x will quit the driver and exit Pry.

execute_script calls $driver.execute_script

find_element calls $driver.find_element

find_elements calls $driver.find_elements

no_wait will set implicit wait to 0. $driver.manage.timeouts.implicit_wait = 0

set_wait will set implicit wait to default seconds. $driver.manage.timeouts.implicit_wait = default

set_wait(timeout_seconds) will set implicit wait to desired timeout. $driver.manage.timeouts.implicit_wait = timeout

.click to tap an element.
.send_keys to type on an element.

Raw UIAutomation

execute_script "au.lookup('button')[0].tap()" is the same as
execute_script 'UIATarget.localTarget().frontMostApp().buttons()[0].tap()'

See app.js for more au methods.
Note that raw UIAutomation commands are not officially supported.

Advanced au.

In this example we lookup two tags, combine the results, wrap with $, and then return the elements.

s = %(
var t = au.lookup('textfield');
var s = au.lookup('secure');
var r = $(t.concat(s));
au._returnElems(r);
)

execute_script s

XPath(UIAutomation)

See #194 for details.

find_element  :xpath, 'button'
find_elements :xpath, 'button'

find_element  :xpath, 'button[@name="Sign In"]'
find_elements :xpath, 'button[@name="Sign In"]'

find_element  :xpath, 'button[contains(@name, "Sign In")]'
find_elements :xpath, 'button[contains(@name, "Sign")]'

find_element :xpath, 'textfield[@value="Email"]'
find_element :xpath, 'textfield[contains(@value, "Email")]'

find_element  :xpath, 'text[contains(@name, "Reset")]'
find_elements :xpath, 'text[contains(@name, "agree")]'

---

3. Appium支持与无法支持的测试

在 2.2 中，可以看到测试代码并不能直接访问被测试的App已及它的控件。因此能测试的也就是XCUITest (iOS)套件所支持的方法与属性。

支持的测试:
查找元素，获取name value type text size location enabled? displayed? selected?等属性，或者发送点击事件，为文本框输入文字。

这些基本上都属于功能性测试，可以用来操作输入框，点击按钮，查找某段文字是否存在。

如

1) 打开登录页面
2) 找到 “用户名” 输入框，输入 [email protected]
3) 找到 “密码” 输入框， 输入 123456
4) 找到 “登录” 按钮，向它发生点击事件
5) 登录成功，检查用户是否处于主页面

传送门
https://github.com/appium/sample-code/blob/master/sample-code/examples/ruby/simple_test.rb

# GETTING STARTED
# -----------------
# This documentation is intended to show you how to get started with a
# simple Appium & appium_lib test.  This example is written without a specific
# testing framework in mind;  You can use appium_lib on any framework you like.
#
# INSTALLING RVM
# --------------
# If you don't have rvm installed, run the following terminal command
#
# \curl -L https://get.rvm.io | bash -s stable --ruby
#
# INSTALLING GEMS
# ---------------
# Then, change to the example directory:
#   cd appium-location/sample-code/examples/ruby
#
# and install the required gems with bundler by doing:
#   bundle install
#
# RUNNING THE TESTS
# -----------------
# To run the tests, make sure appium is running in another terminal
# window, then from the same window you used for the above commands, type
#
# bundle exec ruby simple_test.rb
#
# It will take a while, but once it's done you should get nothing but a line
# telling you "Tests Succeeded";  You'll see the iOS Simulator cranking away
# doing actions while we're running.
require 'rubygems'
require 'appium_lib'

APP_PATH = '../../apps/TestApp/build/release-iphonesimulator/TestApp.app'

desired_caps = {
  caps:       {
    platformName:  'iOS',
    versionNumber: '8.1',
    deviceName:    'iPhone 6',
    app:           APP_PATH,
  },
  appium_lib: {
    sauce_username:   nil, # don't run on Sauce
    sauce_access_key: nil
  }
}

# Start the driver
Appium::Driver.new(desired_caps).start_driver

module Calculator
  module IOS
    # Add all the Appium library methods to Test to make
    # calling them look nicer.
    Appium.promote_singleton_appium_methods Calculator

    # Add two numbers
    values       = [rand(10), rand(10)]
    expected_sum = values.reduce(&:+)

    # Find every textfield.
    elements     = textfields

    elements.each_with_index do |element, index|
      element.type values[index]
    end

    # Click the first button
    button(1).click

    # Get the first static text field, then get its text
    actual_sum = first_text.text
    raise unless actual_sum == (expected_sum.to_s)

    # Alerts are visible
    button('show alert').click
    find_element :class_name, 'UIAAlert' # Elements can be found by :class_name

    # wait for alert to show
    wait { text 'this alert is so cool' }

    # Or by find
    find('Cancel').click

    # Waits until alert doesn't exist
    wait_true { !exists { tag('UIAAlert') } }

    # Alerts can be switched into
    wait { button('show alert').click } # Get a button by its text
    alert         = driver.switch_to.alert # Get the text of the current alert, using
    # the Selenium::WebDriver directly
    alerting_text = alert.text
    raise Exception unless alerting_text.include? 'Cool title'
    alert_accept # Accept the current alert

    # Window Size is easy to get
    sizes = window_size
    raise Exception unless sizes.height == 667
    raise Exception unless sizes.width == 375

    # Quit when you're done!
    driver_quit
    puts 'Tests Succeeded!'
  end
end

提醒，由于sample-code中的TestApp比较老了，上面的测试代码直接跑，是会失败的。这个是很隐晦的问题导致，新版模拟器(eg. iPhone 6 (iOS 10.0x)), 在运行这个app是，会自动弹出一个告警信息，说“TestApp使用老版本编译的，会影响系统系能，建议用新xcode重新编译”。 就是这个alert框，导致测试脚本找不到相应的控件而失败。

这个事情一开始也卡了我很久，为什么测试用例会通不过？后来是在用Appium Ruby Console的时候找到的。

解决方式有两个：
一、重新编译TestApp;

二、测试代码在“ # Add two numbers” 前加入 sleep(10) 暂停10秒，让你有机会点击 alert 上的 OK 按钮，关掉对话框。

当时不想多事，选择了二。
现在想想，应该还有三： 就是把sleep 换成 alert_accept 来关掉对话框。 有兴趣的可以自己试试。

无法支持的测试:
首先，测试脚本无法直接访问被测对象，其次，来之于上面XCUITest的限制，关于控件有无边框，控件状态，动画效果，字体变化，颜色，对齐等视觉效果，我们是无法测试的。

关于这一点，stackoverflow 上也有老外在吐槽：
http://stackoverflow.com/questions/31250941/xcuielement-obtain-image-value

http://www.danielhall.io/exploring-the-new-ui-testing-features-of-xcode-7

如，

a) 文章的标题需要根据不同的状态，用红色，黑色，灰色等不同的颜色展示。

b) 当点击“上传”按钮后，头像Image应当加载新的头像地址。

这种测试通常就属于无法用自动化来检验的。

尤其是我们的App是用React Native开发的，给我们的测试用例带来了更多的限制与挑战。

4. 解决方案

自动化测试更多的是在功能性上，流程性上的测试起作用，基本能覆盖了80%～90%的功能性测试范围。

对于自动化不好做的AC，我们的处理方式很简单，把它们放到人工测试的范围去，当环境、需求、技术发生变化了，能自动化测试了，再自动化，不要太去纠结。

最后，自动化测试并不能代替一切，也无法覆盖所有的地方。
它的最大价值在于使得App的开发也能够持续集成。
极大的降低了全回归的成本与工作量。