scrapy的helloworld

http://scrapy.org/

安装python virlualenv
yum install libxslt-devel
yum install python-devel libffi-devel
cp /usr/lib64/python2.6/lib-dynload/bz2.so Scrapy/lib/python2.7/
否则可能汇报bz2的错误

pip install scrapy

from scrapy import Spider, Item, Field

class Post(Item):
    title = Field()

class BlogSpider(Spider):
    name, start_urls = 'blogspider', ['http://blog.scrapinghub.com']

    def parse(self, response):
        return [Post(title=e.extract()) for e in response.css("h2 a::text")]

EOF
 scrapy runspider myspider.py

参考
http://scrapy-chs.readthedocs.org/zh_CN/0.24/intro/tutorial.html

scrapy shell "http://app.shafa.com/apk/baobaoanquanjiaoyu.html"


response.xpath('//title')
response.xpath('//title').extract()
response.xpath('//title/text()')
response.xpath('//title/text()').re('(\w+):')
response.xpath('//section[@class="app-info"]//div[contains(@class,"meta-info")][2]//div[@class="info-title"]/following-sibling::span[1]/text()').extract()

http://www.shumeipai.net/forum.php?mod=viewthread&tid=20894&highlight=%D0%A1%B3%B5