Scrapy连接Mongodb

piplines.py:

from scrapy.conf import settings
import pymongo

class Mongo66Pipeline(object):
    def __init__(self):
        self.connection = pymongo.MongoClient(settings['MONHOST'],settings['MONPORT'])
        db = self.connection[settings['MONDB']]
        self.collection = db[settings['COLLECTION']]
    def process_item(self, item,spider):
        self.collection.insert(dict(item))
        return item

settings.py:

# mongodb连接配置
MONHOST = '192.168.6.24'
MONPORT = 27017
MONDB = 'mydb' #库名
COLLECTION = 'host' #表名

你可能感兴趣的:(Scrapy连接Mongodb)