Scrapy commands

.extract() - list of items
.extract_first() - the string converted from unicode ('u')
response.urljoin(relative url) - make absolute url
scrapy.htttp.request or selector.follow - simulate clicking onto next page, follow allows relative path

xpath

  • contains
xpath('//*[contains(@class, "star-rating")]/@class')
  • following-sibling::
xpath(''//*[@id="product_description"]/following-sibling::p/text()'')

你可能感兴趣的:(Scrapy commands)