Python网络爬虫--BeautifulSoup库的基本元素

requests

requests库可以看看这篇文章
http://blog.csdn.net/shanzhizi/article/details/50903748
最近在学习嵩天老师的Python网络爬虫课程,记录一下.

bs4

1.Beautiful Soup库，也叫beautifulsoup4 或bs4
约定引用方式如下，即主要是用BeautifulSoup类

from bs4 import BeautifulSoup
import bs4

2.BeautifulSoup库解析器

3.BeautifulSoup类的基本元素

4.测试代码(太乱了)

import requests
from bs4 import BeautifulSoup

r = requests.get('http://python123.io/ws/demo.html',timeout = 30)
r.raise_for_status()
r.encoding = r.apparent_encoding
# print(r.text)
demo = r.text
soup = BeautifulSoup(demo,"html.parser")
print(soup.title)
tag = soup.a
print(tag)
print(soup.a.parent.name) #通过name获得标签的名字
print(soup.a.parent.parent.name)
print(tag.attrs) #标签的属性,以字典形式打印,href class id
print(tag.attrs['class'])
print(tag.attrs['href'])
print(type(tag))  # a标签的类型 bs4中认可的属性
print(tag.string)
print(soup.p.string)
print(type(soup.p.string))  #bs4.element.NavigableString类型,p标签之间的内容的类型