Python爬虫解析库之pyquery详解

PyQuery解析库·详解

安装

pip3 install pyquery

初始化的三种方法

字符串初始化

from pyquery import PyQuery  as pq
html = '''

    
        
first item
        
third item
    

'''
doc = pq(html)
print(doc('li'))# 这里的选择与css选择器一样，选class加点，选id加#，选标签什么都不加
输出结果为：
<li class="item-0">first item</li>
        <li class="item-1"><a href="link3.html"><span class="bold">third item</span></a></li>

url初始化

from pyquery import PyQuery  as pq
url='http://www.baidu.com'
doc = pq(url)
print(doc('head'))

文件初始化
在这里就不举出例子了，与前面两种方法类似

基本CSS选择器

from pyquery import PyQuery as pq
 
html = '''

    

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    

'''
doc = pq(html)
print(doc('#container .list li'))
 
输出结果为：
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>

查找元素

查找子元素，find方法，查找元素里面包含的元素

from pyquery import PyQuery as pq
 
html = '''

    

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    

'''
doc = pq(html)
items = doc('.list')
print(type(items))
print(items)
lis = items.find('li')
print(type(lis))
print(lis)
 
输出结果为：
<class 'pyquery.pyquery.PyQuery'>
<ul class="list">
        <li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
     
</ul>
<class 'pyquery.pyquery.PyQuery'>
<li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>

查找父元素

from pyquery import PyQuery as pq
 
html = '''

    

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    

'''
doc = pq(html)
items = doc('.list')
print(items.parent())
输出结果为：
<div id="container">
    <ul class="list">
        <li class="item-0">first item</li>
        <li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
     
</ul></div>

还有parents方法，查找祖先节点，不只是父节点，父节点的父节点也会查找到

查找兄弟元素

from pyquery import PyQuery as pq
 
html = '''

    

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    

'''
doc = pq(html)
items = doc('.list .item-0.active')
print(items.siblings())
print(items.siblings('.active'))<br>#在查找的时候，可以进行进一步满足条件的筛选
输出结果为：
<li class="item-1"><a href="link2.html">second item</a></li>
        <li class="item-0">first item</li>
        <li class="item-1 active"><a href="link4.html">fourth item</a></li>
        <li class="item-0"><a href="link5.html">fifth item</a></li>
     
<li class="item-1 active"><a href="link4.html">fourth item</a></li>

遍历，items()方法：实际上就是产生了一个产生器，再用for循环进行遍历

from pyquery import PyQuery as pq
 
html = '''

    

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    

'''
doc = pq(html)
lis = doc('li').items()
for li in lis:
    print(li)
 
输出结果为：
<li class="item-0">first item</li>
         
<li class="item-1"><a href="link2.html">second item</a></li>
         
<li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         
<li class="item-1 active"><a href="link4.html">fourth item</a></li>
         
<li class="item-0"><a href="link5.html">fifth item</a></li>

获取信息

获取属性，比如要获取item元素的属性：item.attr(‘属性名称’)，或者：item.attr.属性名称

from pyquery import PyQuery as pq
 
html = '''

    

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    

'''
doc = pq(html)
li = doc('.item-0.active a')
print(li.attr.href)
print(li.attr('href'))
 
输出结果为：
link3.html
link3.html　　

获取文本，text()方法

还有DOM操作，因为使用的比较少，这里就不详细介绍了

下面是pyquery的官方文档
https://pyquery.readthedocs.io/en/latest/api.html

欢迎各位朋友的关注与评论