Python+Selenium 搭建自动化测试环境

基础准备 安装

yum -y update

yum -y install gcc gcc-g++ python python-devel python-pip

yum -y install Xvfb firefox

pip install pyvirtualdisplay

yum -y install gcc gcc-c++ autoconf libjpeg libjpeg-devel libpng libpng-devel freetype freetype-devel libxml2 libxml2-devel zlib zlib-devel glibc glibc-devel glib2 glib2-devel bzip2 bzip2-devel ncurses ncurses-devel curl curl-devel e2fsprogs e2fsprogs-devel krb5 krb5-devel libidn libidn-devel openssl openssl-devel openldap openldap-devel nss_ldap openldap-clients openldap-servers unzip subversion

 

pypi 官网: https://pypi.python.org/pypi

pip 官网: https://pypi.python.org/pypi/pip/

bs4 官网: https://pypi.python.org/pypi/beautifulsoup4/

setuptools 官网: https://pypi.python.org/pypi/setuptools/

selenium 官网:https://pypi.python.org/pypi/selenium

tornado 官网: https://pypi.python.org/pypi/tornado

tornado 官网:http://www.tornadoweb.org

tornado github: https://github.com/tornadoweb/tornado/

PyVirtualDisplay 官网: https://pypi.python.org/pypi/PyVirtualDisplay (包含 xvfb)

PyVirtualDisplay github: https://github.com/ponty/PyVirtualDisplay

firefox 老版本下载:http://ftp.mozilla.org/pub/firefox/releases/   (推荐)

geckodriver 官网:https://github.com/mozilla/geckodriver/releases (selenium + firefox 虚拟浏览器交互的驱动)

Python+Selenium测试环境的安装过程

SetupTools 和 pip 工具安装

这两个工具都是属于python的第三方工具包安装软件,有点类似于linux下的安装包软件 apt-get 或 yum,不过pip比SetupTools功能更强大。

SetupTools 官方解释:Download, build, install, upgrade, and uninstall Python packages -- easily!

pip 官方解释:A tool for installing and managing Python packages.

cmd 进入命令行:easy_install pip 在线安装即可。

备注:此处需要注意的是,当安装SetupTools之后,就可以在python安装目录下看到Script目录,如下图所示:

Python+Selenium 搭建自动化测试环境_第1张图片

安装 Selenium

selenium 官方下载:https://pypi.python.org/pypi/selenium#downloads

selenium 最新版本:selenium-2.53.4.tar.gz

这里因为需要将Python和Selenium进行组合,当然Selenium也提供了基于python的实现,所以就需要把Selenium的包安装到python库中去,以便于python开发时进行调用。

进入命令行:pip install selenium 执行之后,将自动化搜寻最新的selenium版本下载并安装,如下图所示(Windows):

Python+Selenium 搭建自动化测试环境_第2张图片

以上显示,则表明在线安装selenium成功!

 

Python + Selenium 示例

这里可以直接在python的编辑中编写如下程序,并保存hello_selenium.py

1

2

3

4

5

6

7

8

from selenium import webdriver

  

 driver = webdriver.Firefox()

 driver.get(‘https://blog.mimvp.com’)

 assert "blog.mimvp.com 博客".decode('utf-8') in driver.title

 print driver.title

  

 driver.close()

在python编辑器里面操作F5运行即可,看看是否成功调用Firefox浏览器。。。

以上一个基础的Python+Selenium的自动化环境已经搭建完成。

 

Selenium 多浏览器实现

构建Python+Selenium2自动化测试环境完成之后,就需要测试支持python的selenium的版本是否都支持在不同浏览器上运行,当前我们分别在三个最通用的浏览器上(IE,Chrome,FireFox)通过脚本来测试。

1) IE 浏览器

在IE浏览器上运行测试脚本,首先需要下载IEDriverServer.exe,放在IE浏览器的安装目录且同级目录下,脚本如下:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

import os

from selenium import webdriver

from selenium.webdriver.common.keys import Keys

 

iedriver = "C:Program FilesInternet ExplorerIEDriverServer.exe"        # IE driver

os.environ["webdriver.ie.driver"] = iedriver

 

driver = webdriver.Ie(iedriver)

driver.get("https://blog.mimvp.com")

assert "Python" in driver.title

 

elem = driver.find_element_by_name("q")

elem.send_keys("selenium")

elem.send_keys(Keys.RETURN)

assert "mimvp.com" in driver.title

 

driver.close()

driver.quit()

 

2)Chrome 浏览器

在Chrome浏览器上运行测试脚本,首先需要下载ChromeDriver.exe,放在Chrome浏览器的安装目录且同级目录下,脚本如下:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

import os

from selenium import webdriver

from selenium.webdriver.common.keys import Keys

 

chromedriver = "C:Program Files (x86)GoogleChromeApplicationchromedriver.exe"

os.environ["webdriver.chrome.driver"] = chromedriver

driver = webdriver.Chrome(chromedriver)

 

driver.get("https://blog.mimvp.com")

assert "Python" in driver.title

elem = driver.find_element_by_name("q")

elem.send_keys("selenium")

elem.send_keys(Keys.RETURN)

assert "mimvp.com" in driver.title

 

driver.close()

driver.quit()

注意官网的介绍

Chrome Driver is maintained / supported by the Chromium project iteslf.  看来如果使用 new ChromeDriver() 的话,应该要安装 Chromium 而不是 Chrome,我现在懒得折腾了,有兴趣的童鞋可以试验一下。 

 

3) Firefox 浏览器

在Firefox浏览器上运行测试脚本,具体如下:

1

2

3

4

5

6

7

8

9

10

11

12

13

from selenium import webdriver

from selenium.webdriver.common.keys import Keys

 

driver = webdriver.Firefox()

driver.get("https://blog.mimvp.com/")

assert "Python" in driver.title

elem = driver.find_element_by_name("q")

elem.send_keys("selenium")

elem.send_keys(Keys.RETURN)

assert "mimvp.com" in driver.title

 

driver.close()

driver.quit()

 

Selenium 虚拟浏览器静默执行(不打开窗体)

使用selenium打开网页时,FireFox浏览器会在虚拟窗体中显示,不会在当前用户窗体中打开。

应用场景:

非常适合在服务器端执行;非常人性化的不打扰当前用户工作,赞!

On *nix, you can run WebDriver in a headless (virtual) display to hide the browser. This can be done with Xvfb.

I personally use Python on Linux, and the PyVirtualDisplay module to handle Xvfb for me.

Code for running headless would look like this:

#!/usr/bin/env python

from pyvirtualdisplay import Display
from selenium import webdriver

display = Display(visible=0, size=(800, 600))
display.start()

# now Firefox will run in a virtual display. 
# you will not see the browser.
browser = webdriver.Firefox()
browser.get('https://www.mimvp.com')
print browser.title
browser.quit()

display.stop()

Install dependencies on Debian/Ubuntu:

$ sudo apt-get install xvfb python-pip
$ sudo pip install pyvirtualdisplay

或 CentOS 系统上

$ yum install Xvfb python-pip      // yum install xorg-x11-server-Xvfb
$ yum --enablerepo=remi install firefox
$ pip install pyvirtualdisplay

或 AWS EC2 系统上

$ wget https://lambda-linux.io/epll-release-2015.09-1.1.ll1.noarch.rpm
$ yum -y install epll-release-2015.09-1.1.ll1.noarch.rpm
$ yum --enablerepo=epll install firefox-compat

下载firefox,解压,软链接

wget  http://download.firefox.com.cn/releases/firefox/45.0/en-US/Firefox-latest-x86_64.tar.bz2
tar jxvf Firefox-latest-x86_64.tar.bz2
mv Firefox-latest-x86_64 firefox
ln -s  /home/ec2-user/tool-server/firefox/firefox /usr/bin/firefox

参考: Announcing Firefox Browser Support for Amazon Linux

firefox 更多老版本下载:http://ftp.mozilla.org/pub/firefox/releases/  (推荐)

 

错误1:

如果报错,则更新selenium到最新版,错误信息:

Message: Can't load the profile. Profile Dir: %s If you specified a log_file in the FirefoxBinary constructor, check it for details.

 pip install -U selenium  (高版本CentOS 7、Firefox、Selenium,此方法没用)

 

错误2:

easyprocess.EasyProcessCheckInstalledError: cmd=['Xvfb', '-help']

OSError=[Errno 2] No such file or directory

Program install error! 

解决:

1

pip install xvfbwrapper

 

错误3:

selenium.common.exceptions.WebDriverException: Message: 'geckodriver' executable needs to be in PATH. 

或者

KeyError: 'DISPLAY'

这是因为Firefox高版本需要使用 geckodriver 来驱动,不再使用Seleniu默认自带的Firefox webdriver。

我们只需要在 github 下载 geckodriver 并将其所在的路径设为环境变量即可解决。

解决:

1

brew install geckodriver

或下载安装低版本firefox

firefox-45.0.2.tar.bz2

firefox 更多老版本下载:http://ftp.mozilla.org/pub/firefox/releases/  (推荐)

 

错误4:

 #warning This file should not be included by clients, include only
  ^
In file included from _mysql.c:46:0:
/usr/include/mysql/mysql.h:440:3: warning: function declaration isn’t a prototype [-Wstrict-prototypes]
   MYSQL_CLIENT_PLUGIN_HEADER
   ^
_mysql.c: In function ‘_mysql_ConnectionObject_ping’:
_mysql.c:2005:41: error: ‘MYSQL’ has no member named ‘reconnect’
  if ( reconnect != -1 ) self->connection.reconnect = reconnect;
                                         ^
error: command 'gcc' failed with exit status 1

错误原因是 MariaDB 10.2 、10.3 的 MySQL 版本时没有 reconnect

解决:

vim /usr/include/mysql/mysql.h​

搜索 st_mysql 定义结构,添加 reconnect 声明如下:

unsigned int reconnect;              /* matt hacking mysql by mimvp.com */

my_bool reconnect;                 /* set to 1 if automatic reconnect by mimvp.com */

完整代码如下:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

  typedef struct st_mysql {

    NET         net;                    /* Communication parameters */

    void  *unused_0;

    char *host,*user,*passwd,*unix_socket,*server_version,*host_info;

    char *info,*db;

    const struct ma_charset_info_st *charset;      /* character set */

    MYSQL_FIELD *fields;

    MA_MEM_ROOT field_alloc;

    unsigned long long affected_rows;

    unsigned long long insert_id;               /* id if insert on table with NEXTNR */

    unsigned long long extra_info;              /* Used by mysqlshow */

    unsigned long thread_id;            /* Id for connection in server */

    unsigned long packet_length;

    unsigned int port;

    unsigned long client_flag;

    unsigned long server_capabilities;

    unsigned int protocol_version;

    unsigned int field_count;

    unsigned int server_status;

    unsigned int server_language;

    unsigned int warning_count;          /* warning count, added in 4.1 protocol */

    unsigned int reconnect;              /* matt hacking mysql by mimvp.com */

    struct st_mysql_options options;

    enum mysql_status status;

    my_bool     free_me;                 /* If free in mysql_close */

    my_bool     reconnect;               /* set to 1 if automatic reconnect by mimvp.com */

    my_bool     unused_1;

    char                scramble_buff[20+ 1];

    /* madded after 3.23.58 */

    my_bool       unused_2;

    void          *unused_3, *unused_4, *unused_5, *unused_6;

    LIST          *stmts;

    const struct  st_mariadb_methods *methods;

    void          *thd;

    my_bool       *unbuffered_fetch_owner;

    char          *info_buffer;

    struct st_mariadb_extension *extension;

} MYSQL;

然后,编译就通过了,可参考 MySQL-python-1.2.5 will not compile against MariaDB 10.2 

 

应用实例:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

import bs4

from selenium import webdriver

from pyvirtualdisplay import Display

 

 

def spider_url(self, url, index, total):

    print("%d/%d  -  url: %s" % (index, total, url))

 

    content = ''

    browser = None

    table_soup = []

    try:

        display = Display(visible=0, size=(800, 600))

        display.start()

 

        # now Firefox will run in a virtual display.

        # you will not see the browser.

        browser = webdriver.Firefox()       # 打开 FireFox 浏览器

        browser.set_page_load_timeout(60)

        browser.get(url)    

         

        content = browser.find_element_by_class_name('table')       # 通过标记id 获取网页的内容

        content = browser.page_source

             

        self.kill_firefox(browser)

        display.stop()

         

        content = bs4.BeautifulSoup(content, from_encoding='GB18030')

        table_soup = content.find('table', {"class":"table"}).find_all("tr")

    except Exception as ex:

        print("error msg: " + str(ex))

        self.kill_firefox(browser)

 

还有一种静默执行的方法(供参考):

I easily managed to hide the browser window.

Just install PhantomJS. Then, change this line:

driver = webdriver.Firefox()

to:

driver = webdriver.PhantomJS()

The rest of your code won't need to be changed and no browser will open. For debugging purposes, use driver.save_screenshot('screen.png') at different steps of your code.

 

总结

通过以上三个不同浏览器上的测试,说明selenium在python中的运用于其Java版本都是一样。

由于Firefox是默认安装路径,webdriver可以正常访问找到他,如果非系统默认安装路径,则需要跟IE和Chrome一样来设置driver路径。

 

米扑分享

米扑科技的服务器配置成功的几个版本组合

1. CentOS

1)组合一

CentOS 6.5 
Python 2.7.6 、pip 1.5.1 、 PyVirtualDisplay 0.1.5 、 tornado 3.2.2
selenium-2.46.1
Firefox 38.1.0
beautifulsoup4 4.3.2

 

2)组合二

CentOS 7.3.1611
Python 2.7.6 、pip 8.1.2 、 setuptools 1.4.2 、 PyVirtualDisplay 0.2 、xvfbwrapper 0.2.4 、 tornado 2.4
selenium-2.53.4
Firefox 52.1.0
beautifulsoup4 4.3.2

 

3)组合三

CentOS 7.4.1708
Python 2.7.6 、pip 9.0.1 、setuptools 36.2.7 、 PyVirtualDisplay 0.2.1、 tornado 3.2
selenium-2.53.4
Firefox 52.6.0
beautifulsoup4 4.3.2

 

4)组合四

CentOS 7.4.1708
Python 2.7.6 、pip 9.0.1 、setuptools 36.2.7 、 PyVirtualDisplay 0.2.1、 tornado 3.2
selenium-2.53.4
Firefox 52.7.2
beautifulsoup4 4.3.2

 

5)组合五 (2018.7.26)

CentOS 7.5.1804
Python 2.7.6 、pip 18.0 、setuptools 36.4.0 、 PyVirtualDisplay 0.2.1、 tornado 3.2
selenium-3.13.0
Firefox 61.0       geckodriver 0.21.0
beautifulsoup4 4.3.2

 

6)组合六 (2018.9.10)

CentOS 7.4.1708
Python 2.7.6 、pip 18.0 、setuptools 36.4.0 、 PyVirtualDisplay 0.2.1、 tornado 3.2
selenium-3.4.3
Firefox 60.1.0       geckodriver 0.21.0
beautifulsoup4 4.3.2

 

 

2. Ubuntu

Ubuntu 14.04.5 LTS
Python 2.7.6 、pip 9.0.1 、setuptools 38.5.1 、 PyVirtualDisplay 0.1.5、 tornado 3.2.2
selenium-3.8.0
Firefox 57.0.3
beautifulsoup4 4.3.2

 

 

3. MacOS

macOS High Sierra 10.13.2
Python 2.7.13 、pip 9.0.1 、setuptools 28.8.0 、 PyVirtualDisplay 0.2.1、xvfbwrapper 0.2.9  、 tornado 4.5.1
selenium-3.4.2
Firefox 55.0.1     geckodriver 0.19.1
beautifulsoup4 4.3.2

你可能感兴趣的:(linux,python)