scrapy 通过命令行传参数批量爬取

class EypSpider(scrapy.Spider):
    name = 'eyp'
    def __init__(self, category=None, *args, **kwargs):
        super(EypSpider, self).__init__(*args, **kwargs)
        cat = [category, category]
        self.start_urls = ['https://search.jd.com/Search?keyword=%s&enc=utf-8&wq=%s' % (cat[0], cat[1])]

命令行输入:scrapy crawl eyp -a category='鞋'

你可能感兴趣的:(Python,爬虫,数据)