3-4 Beautiful Soup解析器

网页解析器

Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式.Beautiful Soup会帮你节省数小时甚至数天的工作时间

# 如果通过show没有显示相应的路径，则可以采用install安装
C:\Users\Administrator>pip show beautifulsoup4
Name: beautifulsoup4
Version: 4.7.1
Summary: Screen-scraping library
Home-page: http://www.crummy.com/software/BeautifulSoup/bs4/
Author: Leonard Richardson
Author-email: leonardr@segfault.org
License: MIT
Location: d:\anaconda3\lib\site-packages
Requires: soupsieve
Required-by: conda-build

C:\Users\Administrator>pip install beautifulsoup4

各种解析器的对比

推荐使用lxml作为解析器,因为效率更高. 在Python2.7.3之前的版本和Python3中3.2.2之前的版本,必须安装lxml或html5lib, 因为那些Python版本的标准库中内置的HTML解析方法不够稳定

快速开始

Python中3引号代表可换行的字符串

html_doc = """


The Dormouse's story

Once upon a time there were three little sisters; and their names were
Elsie,
Lacie and
Tillie;
and they lived at the bottom of a well.

...
"""

采用Soup解析网页

此处大家只需要了解soup可以获取网页中的任何元素，某个元素中父元素，元素中的属性等。下节课我们会对需要的代码进行封装操作

from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc)

soup.title
# 

soup.title.name
# u'title'

soup.title.string
# u'The Dormouse's story'

soup.title.parent.name
# u'head'

soup.p
# 
The Dormouse's story

soup.p['class']
# u'title'

soup.a
# Elsie

soup.find_all('a')
# [Elsie,
#  Lacie,
#  Tillie]

soup.find(id="link3")
# Tillie

# 从文档中找到所有的a标签
for link in soup.find_all('a'):
    print(link.get('href'))
    # http://example.com/elsie
    # http://example.com/lacie
    # http://example.com/tillie