量化交易-2-数据获取

量化交易-2

@(金融)

数据获取，加载

通过tushare获取数据，不能每次都去请求，一般可以加载到本地存储，我选择存储到mysql,以后获取的时候，可以去mysql加载。然后每天更新数据就好了。

数据存储模块如下

    def down_history(self, stock):
        '''
        下载历史至今天的数据,可以用于下载新股票
        date,open,high,close,low,volume,amount
        '''
        print '--'*10,"downing ",stock,'--'*10
        date = self.basic.ix[stock]['timeToMarket']
        #20100115  竟然是个整数
        start_year = date/10000
        today =datetime.date.today()
        end_year = int(today.strftime("%Y"))
        suffix = "-" + str(date)[4:6] + "-" + str(date)[6:8]

        raw_data = None

        #针对次新股，今年的股票
        if start_year == end_year:
            raw_data = ts.get_h_data('603600')
        for year in range(start_year, end_year):
            start = str(year) + suffix
            right = datetime.datetime.strptime(str(year+1) + suffix, "%Y-%m-%d")-datetime.timedelta(days=1)
            #跨年的应该没有那天上市的公司，所以不存在bug
            end = right.strftime("%Y-%m-%d")
            print start, "-----",end
            data = ts.get_h_data(stock,start=start,end=end)
            if data is None:
                print None
            else:
                print data.shape
            raw_data = pd.concat([raw_data, data], axis=0)

            #看看是否需要补充最后一段时间的数据
            if (year+1) == end_year and end < today.strftime("%Y-%m-%d"):
                this_year_start = str(year+1) + suffix
                print this_year_start, "-------",today.strftime("%Y-%m-%d")
                data = ts.get_h_data(stock, start=this_year_start, end=today.strftime("%Y-%m-%d"))
                print data.shape
                raw_data = pd.concat([raw_data, data], axis=0)

        raw_data = raw_data.sort_index(ascending=True)
        raw_data.to_sql('day_'+stock, self.engine)
        return raw_data

    def down_all_day_stick(self):
        '''
        下载所有股票的历史数据
        '''
        for stock in self.basic['code']:
            self.down_history(stock)

更新数据模块

    def append_days(self,stock, start, end):
        '''
        添加stock,指定时间范围内的数据
        '''
        data = ts.get_h_data(stock,start=start,end=end)
        data = data.sort_index(ascending=True)  
        data.to_sql('day_'+stock, self.engine,if_exists='append')

    def append_all_days(self, start=None, end=None):
        '''
        添加所有股票数据
        '''
        if start == None:
            start = datetime.datetime.today()
            end = start
        for stock in self.basic['code']:
            self.append_days(stock, start, end)

加载模块

 def load_data(self, stock):
        '''
        加载股票历史数据
        '''
        search_sql = "select * from {0}".format('day_'+stock)
        raw_data = SQL.read_sql(search_sql, self.engine)
        return raw_data

有了这些代码，基本上日线级别的数据就有了，与日线级别及以上的指标都可以算出来了