解决方案：ZIPLINE导入BUNDLE出错KEYERROR: TIMESTAMP(‘2019-09-09 23:57:00+0000′, TZ=’UTC’)

文章更新于https://www.itbook5.com/2019/09/11613/

错误如下：

(envs_zipline) C:\Users\hoddy>zipline ingest -b forex-csvdir-bundle
Loading custom pricing data:   [##################------------------]   50% | EURUSD: sid 0
Merging minute equity files:  [------------------------------------]  0
Traceback (most recent call last):
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\base.py", line 2393, in get_loc
    return self._engine.get_loc(key)
  File "pandas\_libs\index.pyx", line 436, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:10160)
  File "pandas\_libs\index.pyx", line 455, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9683)
KeyError: Timestamp('2019-09-09 23:57:00+0000', tz='UTC')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Anaconda3\envs\envs_zipline\Scripts\zipline-script.py", line 11, in 
    load_entry_point('zipline==1.3.0', 'console_scripts', 'zipline')()
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 697, in main
    rv = self.invoke(ctx)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\click\core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\__main__.py", line 348, in ingest
    show_progress,
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\bundles\core.py", line 451, in ingest
    pth.data_path([name, timestr], environ=environ),
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\bundles\csvdir.py", line 94, in ingest
    self.csvdir)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\bundles\csvdir.py", line 156, in csvdir_bundle
    show_progress=show_progress)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\minute_bars.py", line 697, in write
    write_sid(*e, invalid_data_behavior=invalid_data_behavior)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\minute_bars.py", line 730, in write_sid
    self._write_cols(sid, dts, cols, invalid_data_behavior)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\zipline\data\minute_bars.py", line 810, in _write_cols
    latest_min_count = all_minutes.get_loc(last_minute_to_write)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\datetimes.py", line 1418, in get_loc
    return Index.get_loc(self, key, method, tolerance)
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\base.py", line 2395, in get_loc
    return self._engine.get_loc(self._maybe_cast_indexer(key))
  File "pandas\_libs\index.pyx", line 436, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:10160)
  File "pandas\_libs\index.pyx", line 455, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9683)
KeyError: Timestamp('2019-09-09 23:57:00+0000', tz='UTC')

有时候错误是下面的显示：

(envs_zipline) C:\Users\>zipline ingest -b forex-csvdir-bundle
Loading custom pricing data:   [##################------------------]   50% | EURUSD: sid 0
Merging minute equity files:  [------------------------------------]  0
Traceback (most recent call last):
  File "pandas\_libs\index.pyx", line 464, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9810)
  File "pandas\_libs\hashtable_class_helper.pxi", line 756, in pandas._libs.hashtable.Int64HashTable.get_item (pandas\_libs\hashtable.c:13913)
  File "pandas\_libs\hashtable_class_helper.pxi", line 762, in pandas._libs.hashtable.Int64HashTable.get_item (pandas\_libs\hashtable.c:13857)
KeyError: 1568073420000000000

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Anaconda3\envs\envs_zipline\lib\site-packages\pandas\core\indexes\base.py", line 2393, in get_loc
    return self._engine.get_loc(key)
  File "pandas\_libs\index.pyx", line 436, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:10160)
  File "pandas\_libs\index.pyx", line 466, in pandas._libs.index.DatetimeEngine.get_loc (pandas\_libs\index.c:9897)
KeyError: Timestamp('2019-09-09 23:57:00+0000', tz='UTC')

两种情况，都是一样的，类似的问题。

zipline网上都说是文档最多的回测平台，或许是吧，可是官方的文档写的真是非常的差，其中太多的坑。难道是有意的，这样社区才有热度？

本来是想改为适合外汇历史数据的bundle和 TradingCalendaryy就是我们要修改的重点，这样才能导入我们自己的数据，zipline在数据规范上做的太多，个人觉得这就是最大的坑，特别对于外汇市场来说，各平台数据和开盘时间，相差不少。

为了在数据上不出错，我们选择7*24小时的交易日历。其实他已经自带了，可是官方就是什么也不写，自己找吧，搞的每个人都去研究他的源码。

下面给出正确的文件源码，每一行都是必要的。

解决方案

自定义bundle
在.zipline/extension.py中写入

import pandas as pd
import datetime
from zipline.data.bundles import register
from zipline.data.bundles.csvdir import csvdir_equities

start_session = pd.Timestamp('2000-01-01', tz='utc')
end_session = pd.Timestamp(datetime.datetime.now().strftime('%Y-%m-%d'), tz='utc')
register(
    'forex-csvdir-bundle',
    csvdir_equities(
        ['minute'], #daily,minute
        'D:/zipline/',
    ),
    calendar_name='FOREX', # NYSE，FOREX
    start_session=start_session,
    end_session=end_session,
    minutes_per_day=1440 #这就坑，交易日历会根据这个生成时间表，不写的话默认390
)

在环境路径中找到envs_zipline\Lib\site-packages\trading_calendars\calendar_utils.py，在_default_calendar_aliases中新增一项自定义的bundles名称与calendar的对应关系

_default_calendar_aliases = {
    'FOREX': '24/7',# 新增这个bundles名称与calendar的对应关系，24/7这个是默认就有的。
    'NYSE': 'XNYS',
    'NASDAQ': 'XNYS',
    'BATS': 'XNYS',
    'FWB': 'XFRA',
    'LSE': 'XLON',
    'TSX': 'XTSE',
    'BMF': 'BVMF',
    'CME': 'CMES',
    'CBOT': 'CMES',
    'COMEX': 'CMES',
    'NYMEX': 'CMES',
    'ICE': 'IEPA',
    'ICEUS': 'IEPA',
    'NYFE': 'IEPA',
    'CFE': 'XCBF',
}