Pandas —— resample()重采样和asfreq()频度转换
一、resample
-
resample()进行重采样。
-
重采样（Resampling）指的是把时间序列的频度变为另一个频度的过程。把高频度的数据变为低频度叫做降采样（downsampling），把低频度变为高频度叫做升采样（upsampling）
resample参数如下：
resample(rule, how=None, axis=0, fill_method=None, closed=None, label=None, convention='start',kind=None, loffset=None, limit=None, base=0)降采样
考虑因素：
-
各区间哪边是闭合的（参数：closed）
-
如何标记各聚合面元，用区间的开头还是末尾（参数：label）
freq取值如下：
In [232]: ts_index = pd.date_range('2018-08-03',periods =12,freq = 'T')
In [233]: ts = pd.Series(np.arange(12),index = ts_index)
In [234]: ts
Out[234]:
2018-08-03 00:00:00 0
2018-08-03 00:01:00 1
2018-08-03 00:02:00 2
2018-08-03 00:03:00 3
2018-08-03 00:04:00 4
2018-08-03 00:05:00 5
2018-08-03 00:06:00 6
2018-08-03 00:07:00 7
2018-08-03 00:08:00 8
2018-08-03 00:09:00 9
2018-08-03 00:10:00 10
2018-08-03 00:11:00 11
Freq: T, dtype: int32
默认使用左标签（label=‘left’），左闭合（closed='left’）
此时第一个区间为：2018-08-03 00:00:00~2018-08-03 00:04:59，故sum为10，label为：2018-08-03 00:00:00
In [235]: ts.resample('5min').sum()
Out[235]:
2018-08-03 00:00:00 10
2018-08-03 00:05:00 35
2018-08-03 00:10:00 21
Freq: 5T, dtype: int32
可以指定为右闭合（closed='right’），默认使用左标签（label=‘left’）
此时第一个区间为：2018-08-02 23:55:01~2018-08-03 00:00:00，故sum为0，label为：2018-08-02 23:55:00
In [236]: ts.resample('5min',closed='right').sum()
Out[236]:
2018-08-02 23:55:00 0
2018-08-03 00:00:00 15
2018-08-03 00:05:00 40
2018-08-03 00:10:00 11
Freq: 5T, dtype: int32
可以指定为右闭合（closed='right’），右标签（label=‘right’）
此时第一个区间为：2018-08-02 23:55:01~2018-08-03 00:00:00，故sum为0，label为：2018-08-03 00:00:00
In [237]: ts.resample('5min',closed='right',label='right').sum()
Out[237]:
2018-08-03 00:00:00 0
2018-08-03 00:05:00 15
2018-08-03 00:10:00 40
2018-08-03 00:15:00 11
Freq: 5T, dtype: int32
升采样
考虑因素：
- 没有聚合，但是需要填充
In [244]: frame = pd.DataFrame(np.random.randn(2, 4),
...: index=pd.date_range('1/1/2000', periods=2,
...: freq='W-WED'), # freq='W-WED'表示每周三
...: columns=['Colorado', 'Texas', 'New York', 'Ohio'])
In [245]: frame
Out[245]:
Colorado Texas New York Ohio
2000-01-05 1.201713 0.029819 -1.366082 -1.325252
2000-01-12 -0.711291 -1.070133 1.469272 0.809806
当我们对这个数据进行聚合的的时候，每个组只有一个值，以及gap（间隔）之间的缺失值。在不使用任何聚合函数的情况下，我们使用asfreq方法将其转换为高频度：
In [246]: df_daily = frame.resample('D').asfreq()
In [247]: df_daily
Out[247]:
Colorado Texas New York Ohio
2000-01-05 1.201713 0.029819 -1.366082 -1.325252
2000-01-06 NaN NaN NaN NaN
2000-01-07 NaN NaN NaN NaN
2000-01-08 NaN NaN NaN NaN
2000-01-09 NaN NaN NaN NaN
2000-01-10 NaN NaN NaN NaN
2000-01-11 NaN NaN NaN NaN
2000-01-12 -0.711291 -1.070133 1.469272 0.809806
使用ffill()进行前向填充
In [248]: frame.resample('D').ffill()
Out[248]:
Colorado Texas New York Ohio
2000-01-05 1.201713 0.029819 -1.366082 -1.325252
2000-01-06 1.201713 0.029819 -1.366082 -1.325252
2000-01-07 1.201713 0.029819 -1.366082 -1.325252
2000-01-08 1.201713 0.029819 -1.366082 -1.325252
2000-01-09 1.201713 0.029819 -1.366082 -1.325252
2000-01-10 1.201713 0.029819 -1.366082 -1.325252
2000-01-11 1.201713 0.029819 -1.366082 -1.325252
2000-01-12 -0.711291 -1.070133 1.469272 0.809806
In [249]: frame.resample('D').ffill(limit=2)
Out[249]:
Colorado Texas New York Ohio
2000-01-05 1.201713 0.029819 -1.366082 -1.325252
2000-01-06 1.201713 0.029819 -1.366082 -1.325252
2000-01-07 1.201713 0.029819 -1.366082 -1.325252
2000-01-08 NaN NaN NaN NaN
2000-01-09 NaN NaN NaN NaN
2000-01-10 NaN NaN NaN NaN
2000-01-11 NaN NaN NaN NaN
2000-01-12 -0.711291 -1.070133 1.469272 0.809806
新的日期索引没必要跟旧的重叠
In [250]: frame.resample('W-THU').ffill()
Out[250]:
Colorado Texas New York Ohio
2000-01-06 1.201713 0.029819 -1.366082 -1.325252
2000-01-13 -0.711291 -1.070133 1.469272 0.809806
分组重采样
In [279]: times = pd.date_range('2018-08-3 00:00', freq='1min', periods=10)
In [280]: df2 = pd.DataFrame({'time': times.repeat(3),
...: 'key': np.tile(['a', 'b', 'c'], 10),
...: 'value': np.arange(30)})
In [281]: df2[:5]
Out[281]:
key time value
0 a 2018-08-03 00:00:00 0
1 b 2018-08-03 00:00:00 1
2 c 2018-08-03 00:00:00 2
3 a 2018-08-03 00:01:00 3
4 b 2018-08-03 00:01:00 4
In [282]: df2.groupby(['key',pd.Grouper(key='time',freq='5min')]).sum()
Out[282]:
value
key time
a 2018-08-03 00:00:00 30
2018-08-03 00:05:00 105
b 2018-08-03 00:00:00 35
2018-08-03 00:05:00 110
c 2018-08-03 00:00:00 40
2018-08-03 00:05:00 115
asfreq()
- asfreq()进行频度转换。
- 当您要将DatetimeIndex更改为具有不同的频率同时在当前索引处保留相同的值时,可以使用asfreq.
- asfreq() : 采样时间点的value
- resample() : 采样时间段内value
ts如上图：
ts.asfreq('D').sum()-1.0655834142614131
ts.resample('D').sum()
-2.494026
-
缺失值的填充方式.
- method : {‘backfill’, ‘bfill’, ‘pad’, ‘ffill’, None}
>>> index = pd.date_range('1/1/2000', periods=4, freq='T')
>>> series = pd.Series([0.0, None, 2.0, 3.0], index=index)
>>> df = pd.DataFrame({'s':series})
>>> df
s
2000-01-01 00:00:00 0.0
2000-01-01 00:01:00 NaN
2000-01-01 00:02:00 2.0
2000-01-01 00:03:00 3.0
将频度转换为30s
>>> df.asfreq(freq='30S')
s
2000-01-01 00:00:00 0.0
2000-01-01 00:00:30 NaN
2000-01-01 00:01:00 NaN
2000-01-01 00:01:30 NaN
2000-01-01 00:02:00 2.0
2000-01-01 00:02:30 NaN
2000-01-01 00:03:00 3.0
将频度转换为2min，不会进行重采样（与resample的不同之处）
>>> df.asfreq(freq='2min')
s
2000-01-01 00:00:00 0.0
2000-01-01 00:02:00 2.0使用bfill()进行后向填充
>>> df.asfreq(freq='30S').bfill()
s
2000-01-01 00:00:00 0.0
2000-01-01 00:00:30 NaN
2000-01-01 00:01:00 NaN
2000-01-01 00:01:30 2.0
2000-01-01 00:02:00 2.0
2000-01-01 00:02:30 3.0
2000-01-01 00:03:00 3.0
补充：day3Ts.resample('D').interpolate('linear') # 线性插值---------------------------------------------------------------------------------------------------------
原文：https://blog.csdn.net/starter_____/article/details/81437626