Python网络爬虫与信息提取-Day7-基于bs4库的HTML内容遍历方法

HTML基本格式

具有树形结构的文本信息

<>…构成了所属关系，形成了标签的树形结构

1.标签树的下行遍历

属性	说明
.contents	子节点的列表，将所有儿子节点存入列表
.children	子节点的迭代类型，与.contents类似，用于循环遍历儿子节点
.descendants	子孙节点的迭代类型，包含所有子孙节点，用于循环遍历

BeautifulSoup类型是标签树的根节点

 

>>> soup.head

>>> soup.head.contents

[]

>>> soup.body.contents

['\n',

The demo python introduces several python courses.

, '\n',

Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:

 

Basic Python and Advanced Python.

, '\n']

>>> len(soup.body.contents)

5

>>> soup.body.contents[1]

The demo python introduces several python courses.

 

遍历儿子节点：

for child in soup.body.children:
	print(child)

遍历子孙节点：

for child in soup.body.descendants:
	print(child)

2.标签树的上行遍历

属性	说明
.parent	节点的父亲标签
.parents	节点先辈标签的迭代类型，用于循环遍历先辈节点

 

>>> soup.title.parent

>>> soup.html.parent

The demo python introduces several python courses.

Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:

 

Basic Python and Advanced Python.

>>> soup.parent

>>>

 

>>> soup = BeautifulSoup(demo,"html.parser")

>>> for parent in soup.a.parents:

if parent is None:

print(parent)

else:

print(parent.name)

p

body

html

[document]

>>>

 

遍历所有先辈节点，包括soup本身，所以要区别判断

 

3.标签树的平行遍历

属性	说明
.next_sibling	返回按照HTML文本顺序的下一个平行节点标签
.previous_sibling	返回按照HTML文本顺序的上一个平行节点标签
.next_siblings	迭代类型，返回按照HTML文本顺序的后续所有平行节点标签
.previous_siblings	迭代类型，返回按照HTML文本顺序的前续所有平行节点

 

平行遍历发生在同一个父节点下的各节点间

>>> soup = BeautifulSoup(demo,"html.parser")

>>> soup.a.next_sibling

' and '

>>> soup.a.next_sibling.next_sibling

Advanced Python

>>> soup.a.previous_sibling

'Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:\r\n'

>>> soup.a.previous_sibling.previous_sibling

>>> soup.a.parent

Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses:

 

Basic Python and Advanced Python.

>>>

 

遍历后续节点：

for sibling in soup.a.next_sibling:
	print(sibling)

遍历前续节点：

for sibling in soup.a.previous_sibling:
	print(sibling)

总结：