pyquery 库详解

安装

pyquery官网地址

pip install pyquery

验证安装

In [1]: import pyquery

没报错即表示安装成功

pyquery

pyquery 介绍

虽然 xpath 与 Beautiful Soup 已经很强大了，不过语法还是过于啰嗦，pyquery提供了更加简洁优雅的语法，你可以像写jquery一般提取数据

数据初始化

In [1]: html = '''
   ...: 

   ...:     

   ...:         
first item
   ...:         
second item
   ...:         
third item
   ...:         
fourth item
   ...:         
fifth item
   ...:     
   ...: 
   ...: '''

In [2]: from pyquery import PyQuery as pq

In [3]: doc = pq(html)

In [4]: print(doc('li'))
first item
        
second item
        
third item
        
fourth item
        
fifth item

初始化的数据还可以从url中获取

In [11]: doc_url = pq(url='https://www.python.org/')

In [12]: print(doc_url('title'))

除此，也可以传入文件初始化

doc_file = pq(filename='demo.html')

基本css选择器

In [2]: html = '''
   ...: 

   ...:     

   ...:         
first item
   ...:         
second item
   ...:         
third item
   ...:         
fourth item
   ...:         
fifth item
   ...:     
   ...: 
   ...: '''

In [4]: from pyquery import PyQuery as pq

In [5]: doc = pq(html)

In [6]: print(doc('#container .list li'))
first item
        
second item
        
third item
        
fourth item
        
fifth item


In [7]: print(type(doc('#container .list li')))

查找节点

find()

In [8]: items = doc('.list')

In [10]: print(type(items))


In [11]: print(items)

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    


In [12]: lis = items.find('li')

In [13]: print(type(lis))


In [14]: print(lis)
first item
        
second item
        
third item
        
fourth item
        
fifth item

find()的查找范围是节点的所有后代节点

children()

查找直接子节点

In [15]: lis = items.children()

In [16]: print(type(lis))


In [17]: print(lis)
first item
        
second item
        
third item
        
fourth item
        
fifth item


In [18]: items.children('.active')
Out[18]: []

In [19]: print(items.children('.active'))
third item

parent()

查找父节点

In [20]: container = items.parent()

In [21]: print(type(container))


In [22]: print(container)

    

        
first item
        
second item
        
third item
        
fourth item
        
fifth item
    

parents()

查找祖先节点

siblings()

兄弟节点

遍历节点

In [26]: lis = doc('li').items()

In [27]: print(type(lis))


In [28]: for li in lis:
    ...:     print(li, type(li))
    ...:
first item
         
second item
         
third item
         
fourth item
         
fifth item
     

获取属性

# 选中class为item-0及active的a节点
In [29]: a = doc('.item-0.active a')

In [30]: print(a, type(a))
third item 

In [31]: print(a.attr('href'))
link3.html

如果是多个节点的属性，就需要遍历节点获取

In [32]: a = doc('a')

In [33]: for item in a.items():
    ...:     print(item.attr('href'))
    ...:
link2.html
link3.html
link4.html
link5.html

获取文本

text()

In [35]: a = doc('.item-0.active a')

In [36]: print(a)
third item

In [37]: print(a.text())
third item

html()

In [38]: li = doc('.item-0.active')

In [39]: print(li)
third item

In [41]: print(li.html())
third item

在多个节点的情况下，如果要获取每个节点的内部html，需要遍历调用html()。而text()不需要遍历，会将所有节点取出文本后合并成一个字符串

节点操作

addClass 和 removeClass

In [42]: li = doc('.item-0.active')

In [43]: print(li)
third item


In [44]: li.removeClass('active')
Out[44]: []

In [45]: print(li)
third item


In [46]: li.addClass('active')
Out[46]: []

In [47]: print(li)
third item

attr, text 和 html

In [48]: li = doc('.item-0.active')

In [49]: print(li)
third item


In [50]: li.attr('name', 'link')
Out[50]: []

In [51]: print(li)
third item


In [52]: li.text('changed item')
Out[52]: []

In [53]: print(li)
changed item


In [54]: li.html('changed item')
Out[54]: []

In [55]: print(li)
changed item

remove

In [56]: text = '''
    ...: 

    ...:     Hello, World
    ...:     
This is a paragraph
    ...: 
    ...: '''

In [57]: doc = pq(text)

In [58]: wrap = doc('.wrap')

In [59]: print(wrap.text())
Hello, World
This is a paragraph

如果我们不想要p节点内的text，就可以用remove将其移除

In [60]: wrap.find('p').remove()
Out[60]: [
]

In [61]: print(wrap.text())
Hello, World

伪类选择器

In [62]: doc = pq(html)

In [63]: li = doc('li:first-child')

In [64]: print(li)
first item


In [65]: li = doc('li:last-child')

In [66]: print(li)
fifth item


In [67]: li = doc('li:nth-child(2)')

In [68]: print(li)
second item


In [69]: li = doc('li:gt(2)')

In [70]: print(li)
fourth item
        
fifth item


In [71]: li = doc('li:nth-child(2n)')

In [72]: print(li)
second item
        
fourth item


In [73]: li = doc('li:contains(second)')

In [74]: print(li)
second item