在Agile 2009大会之前的周日，第三届敏捷联盟功能测试研讨会召开了。任何有志于改进现有功能测试（相对单元测试而言）工具的人都可以参加开放空间议程。与会者中包括下列工具的创建者和贡献者：Selenium,、SWAT,、Cucumber,、WebTest,、RobotFramework和Twist。
由于采取了开放空间的形式，会议大纲和日程都是现场决定的，本年度有3个时间档和4块场地，所以可以举办12个议程活动。
Lisa Crispin是Agile Testing一书的联合作者，在她的记录中，第一个议程是多个工具的快速演示，这些工具不像Selenium和Watir那么知名，其中包括：Canoo WebTest、 Twist、Cucumber、Robot Framework和SWAT。Lisa说道：“Robot Framework给我留下了深刻印象，这是一个开源工具，开发者是Pekka Klärck。它非常灵活，使用的表格形式非常类似FitNesse，但是只有一种类型。你可以完成关键字驱动、数据驱动甚至是BDD风格的测试。它还可以使用命令行参数，允许你调用内部和像Swing这样的外部程序库。……SWAT引起了人们很大兴趣，以前没人见过它，我想人们对其IDE尤其印象深刻。”
Paul King是WebTest和Groovy的贡献者，他提议：“开发更多混合与匹配（mixing and matching）测试框架、驱动和实用工具。我们都同意测试运行工具已经足够多了，开发人员们应该把注意力放在解决新问题上。”
在另一天上午的议程中，Matt Wynne、Richard Lawrence、Aslak Hellesøy和笔者讨论了将Cucumber移植至.NET平台需要做的工作。结果发现：只要Cucumber的用户准备好在Ruby下运行测试，如果Cucumber开发人员能够找到一种简单的方式与测试应用通信，那么将Cucumber与.NET关联起来就没多大困难。人们提出了一种类似于FitNesse Slim的方法。在议程结束时，Matt和Richard着手开始完成解决方案。
在去年的议程中，大家同意用文档记录现有的相关工具，但是没有完成。今年，Gerard Meszaros创建了一个电子表格（想获得编辑权限，请联系笔者：mark AT mlevison DOT com），内容基于他上午看到的演示。在下午的一个议程中，一组人聚在一起，将更多工具填入到该表格中，截至今日，他们在表格中记录的工具包括：SWAT、Cucumber、WebTest、RobotFramework、Twist、TestSwarm、JBehave、Fit、FitNesse、FitNesseSlim、UltiFit、Watir、Watin、Abbot、Fest、White、Sahi和Sahi-Java。可以注意到，人们没有记录Selenium。
下午晚些时候，人们举办了一个讨论，议题是像Selenium IDE这样的录制和回放工具。Lisa Crispin认为捕获/回放工具“对于学习新工具很有帮助，而且也能帮人们调试测试脚本，或是找到某个测试中应该使用的正确语句。然而，人们不应该仅仅使用捕获/回放工具，否则就会遇到问题。”Jason Huggins是Selenium的开发人员，他解释道：他对于Selenium IDE的通用性也很困惑（因为那仅仅是个录制/回放工具而已）。这个工具本来就是用来培训“新手”的，就像“刚学飞行的飞行员要从训练机开始。飞行员能从训练机上学到很多东西，不过最终还是要开真正的飞机。”为了让这个区别更加明显，人们提出的建议包括（选自Mike Longin的笔记）：
- 将Selenium IDE的名字改为Selenium Trainer，让大家知道：录制不是自动化的终点。
- 在录制器上创建某种复杂度表，当录制内容过于复杂时要告诉人们：也许应该学习一些新的自动化测试技术了。
Mike这样回应Paul的评论：“我们已经有了数不胜数的驱动程序、框架和运行器，现在也许该开始寻找一些集成方面的技术了，尝试如何利用这些出色的工具，并将它们组合利用。”
Pekka Klärck对Jennitta Andrea和Elisabeth Hendrickson组织今年的研讨会表示了感谢之情。
来源：http://www.infoq.com/cn/news/2009/09/functional-test-tools
作者 Mark Levison 译者 郑柯
Functional Test Tools Workshop
Posted by Mark Levison on Sep 17, 2009
The 3rd workshop of the Agile Alliance Functional Testing Tools group was held the Sunday before Agile2009. The open space session was open to anyone interested in advancing the state of the art with Functional (vs Unit) test tools. Among others, the participants were the creators of and contributors to: Selenium, SWAT, Cucumber, WebTest, RobotFramework and Twist.
As an open space, the agenda and sessions are created on the fly– this year there were 3 time slots and 4 locations allowing for up 12 sessions.
According to Lisa Crispin, co-author of Agile Testing, the first session was a series of lightening talks demoing various tools that aren’t as well known as Selenium and Watir: Canoo WebTest, Twist, Cucumber, Robot Framework and SWAT. Lisa says: “I was impressed by Robot Framework, an open source tool whose developer is Pekka Klärck. It has a lot of flexibility. It uses a tabular format similar to FitNesse except there is only one type of table. You can do keyword driven, data driven and even BDD style tests. It also takes command line arguments. It lets you have internal and external libraries such as Swing … There was a lot of interest in SWAT, nobody had seen it before and I think they were particularly impressed by the IDE.”
Paul King, WebTest and Groovy contributor, proposed: “doing more mixing and matching of testing frameworks/drivers/utilities/tools. He had a nice graphic showing what is available. We all agreed there are enough test runners, and the developer community should focus on solving new problems”
In another morning session, Matt Wynne, Richard Lawrence, Aslak Hellesøy and this reporter discussed what would be required to port Cucumberto .NET. As it turns out, as long as users are prepared to run the Cucumber test run under Ruby then creating .NET bindings for Cucumber is relatively simple if the Cucumber developers can find an simple way of communicating with the test application. A model similar to the FitNesse Slim approach was proposed. By the end of the session Matt and Richard started work on the solution.
In last years session it was agreed that we should document the existing tools in this space but the effort floundered. This year, Gerard Meszaros created a spreadsheet(contact this reporter markAT mlevisonDOT com for editing access)based on what he saw in the morning demos. In one of the afternoon sessions a group got together and helped fill in more tools in the spreadsheet– as of today they spreadsheet documents: SWAT, Cucumber, WebTest, RobotFramework, Twist, TestSwarm,JBehave, Fit, FitNesse, FitNesseSlim, UltiFit,Watir, Watin, Abbot, Fest, White, Sahiand Sahi-Java. Notably missing are notes about Selenium.
In the late afternoon there was a discussion around the role of record and playback tools such as the Selenium IDE. Lisa Crispinthinks that capture/playback tools “can be a great way to help learn a new tool, and also can be helpful in debugging test scripts or figuring out the right statements to use in a particular test. However, people shouldn’t get bogged down in only using capture/playback”. Jason Huggins, Selenium developer, explained that he is troubled by the general use of the SeleniumIDE (really just a record/playback tool). It was originally intended as a “little 'trainer' airplane that jet pilots train on first. The pilots can learn a lot from the trainer, but eventually they have to move up to a real jet.” Suggestions for making this distinction more clear included (notes from Mike Longin):
- Rename Selenium IDE to Selenium Trainer to help promote the idea that recording really isn’t the end all for automation
- Create some kind of complexity meter on the recorder that shows when your recording is getting so complex that maybe its time to learn new automation techniques
Mike echoes Paul’s comment“We have numerous drivers, frameworks, and runners out there and now maybe is the time to start looking at integration techniques. How do we take all of these great tools and start combining them.”
Pekka Klärck thanked Jennitta Andrea and Elisabeth Hendrickson for organizing the workshop again this year.
- This article is part of a featured topic series on Agile2009 and also Testing and also Agile Techniques
- See more Agile2009 content at: http://www.infoq.com/Agile2009 and more Testing content at: http://www.infoq.com/Testing and more Agile Techniques content at: http://www.infoq.com/agile_techniques
- Other recent content items in this topic
- Refactoring Automated Functional Test Scripts with iTest2
- Excerpts from an Interview with James Bach