pandas 入门 -- date_range

pandas date_range

start 开始时间

end 结束时间

periods 时间长度

freq 时间频率,默认为‘D’, 可选H(our) , W(eek), B(usiness), S(emi-)M(onth),(min)T(es), S(econd), A(year),…

In [23]: import datetime

In [24]: datetime.datetime.strptime('2010-01-01', '%Y-%m-%d')  # 将字符串转成时间
Out[24]: datetime.datetime(2010, 1, 1, 0, 0)

In [25]: # 第一个参数是时间字符串,第二个参数是时间格式

In [26]: import dateutil  # 可能会出错 from dateutil import parser

In [27]: dateutil.parser.parse('2001-01-01')
Out[27]: datetime.datetime(2001, 1, 1, 0, 0)

In [28]: # 像 datatime 一样 但这个省去时间格式

In [29]: dateutil.parser.parse('2001/01/01')
Out[29]: datetime.datetime(2001, 1, 1, 0, 0)

In [30]: dateutil.parser.parse('01/01/2001')
Out[30]: datetime.datetime(2001, 1, 1, 0, 0)

In [31]: dateutil.parser.parse('JAN/01/2001')
Out[31]: datetime.datetime(2001, 1, 1, 0, 0)

In [33]: pd.to_datetime(['2001-01-01', '2010/Feb/02'])
Out[33]: DatetimeIndex(['2001-01-01', '2010-02-02'], dtype='datetime64[ns]', freq=None)

In [34]: # 将不同的时间对象字符串转成时间

In [35]: pd.date_range('2010-01-01','2010-01-15')
Out[35]:
DatetimeIndex(['2010-01-01', '2010-01-02', '2010-01-03', '2010-01-04',
               '2010-01-05', '2010-01-06', '2010-01-07', '2010-01-08',
               '2010-01-09', '2010-01-10', '2010-01-11', '2010-01-12',
               '2010-01-13', '2010-01-14', '2010-01-15'],
              dtype='datetime64[ns]', freq='D')

In [36]: pd.date_range('2010-01-01',periods=20)
Out[36]:
DatetimeIndex(['2010-01-01', '2010-01-02', '2010-01-03', '2010-01-04',
               '2010-01-05', '2010-01-06', '2010-01-07', '2010-01-08',
               '2010-01-09', '2010-01-10', '2010-01-11', '2010-01-12',
               '2010-01-13', '2010-01-14', '2010-01-15', '2010-01-16',
               '2010-01-17', '2010-01-18', '2010-01-19', '2010-01-20'],
              dtype='datetime64[ns]', freq='D')

In [37]: # periods 是要生成的时间长度

In [38]: pd.date_range('2010-01-01',periods=20, freq='W')
Out[38]:
DatetimeIndex(['2010-01-03', '2010-01-10', '2010-01-17', '2010-01-24',
               '2010-01-31', '2010-02-07', '2010-02-14', '2010-02-21',
               '2010-02-28', '2010-03-07', '2010-03-14', '2010-03-21',
               '2010-03-28', '2010-04-04', '2010-04-11', '2010-04-18',
               '2010-04-25', '2010-05-02', '2010-05-09', '2010-05-16'],
              dtype='datetime64[ns]', freq='W-SUN')

In [39]: # 按照每周

In [40]: pd.date_range('2010-01-01',periods=20, freq='W-MON')
Out[40]:
DatetimeIndex(['2010-01-04', '2010-01-11', '2010-01-18', '2010-01-25',
               '2010-02-01', '2010-02-08', '2010-02-15', '2010-02-22',
               '2010-03-01', '2010-03-08', '2010-03-15', '2010-03-22',
               '2010-03-29', '2010-04-05', '2010-04-12', '2010-04-19',
               '2010-04-26', '2010-05-03', '2010-05-10', '2010-05-17'],
              dtype='datetime64[ns]', freq='W-MON')

In [41]: # 按照每周一

In [42]: pd.date_range('2010-01-01',periods=20, freq='B')
Out[42]:
DatetimeIndex(['2010-01-01', '2010-01-04', '2010-01-05', '2010-01-06',
               '2010-01-07', '2010-01-08', '2010-01-11', '2010-01-12',
               '2010-01-13', '2010-01-14', '2010-01-15', '2010-01-18',
               '2010-01-19', '2010-01-20', '2010-01-21', '2010-01-22',
               '2010-01-25', '2010-01-26', '2010-01-27', '2010-01-28'],
              dtype='datetime64[ns]', freq='B')

In [43]: # 去掉周六/日

In [45]: dt = _

In [46]: dt[0]
Out[46]: Timestamp('2010-01-01 00:00:00', offset='B')

In [47]: dt[0].to_pydatetime()
Out[47]: datetime.datetime(2010, 1, 1, 0, 0)

In [48]: # 转成 python 的 datatime

In [49]: pd.date_range('2010-01-01',periods=20, freq='1h20min')
Out[49]:
DatetimeIndex(['2010-01-01 00:00:00', '2010-01-01 01:20:00',
               '2010-01-01 02:40:00', '2010-01-01 04:00:00',
               '2010-01-01 05:20:00', '2010-01-01 06:40:00',
               '2010-01-01 08:00:00', '2010-01-01 09:20:00',
               '2010-01-01 10:40:00', '2010-01-01 12:00:00',
               '2010-01-01 13:20:00', '2010-01-01 14:40:00',
               '2010-01-01 16:00:00', '2010-01-01 17:20:00',
               '2010-01-01 18:40:00', '2010-01-01 20:00:00',
               '2010-01-01 21:20:00', '2010-01-01 22:40:00',
               '2010-01-02 00:00:00', '2010-01-02 01:20:00'],
              dtype='datetime64[ns]', freq='80T')

In [50]: # 任意时间

pandas 常用函数

In [2]: import pandas as pd

In [3]: df = pd.DataFrame({'one':[1,2,3,4], 'two':[4,5,6,7]}, index=['a','b','c'
   ...: ,'d'])

In [4]: df
Out[4]:
   one  two
a    1    4
b    2    5
c    3    6
d    4    7

In [5]: df.mean()
Out[5]:
one    2.5
two    5.5
dtype: float64

In [6]: df.mean(axis=1)
Out[6]:
a    2.5
b    3.5
c    4.5
d    5.5
dtype: float64

In [7]: # 按行求平均值

In [8]: df.sum()
Out[8]:
one    10
two    22
dtype: int64

In [9]: df.sum(axis=1)
Out[9]:
a     5
b     7
c     9
d    11
dtype: int64

In [10]: # 按行求和

In [11]: df.sort_values(by='one')
Out[11]:
   one  two
a    1    4
b    2    5
c    3    6
d    4    7

In [12]: df.sort_values(by='one', ascending=False)
Out[12]:
   one  two
d    4    7
c    3    6
b    2    5
a    1    4

In [13]: # ascending 按列 one 降序排列

In [16]: # 当列中有 NaN 时 ascending 排序 NaN 始终在最后

In [17]: df.sort_index()
Out[17]:
   one  two
a    1    4
b    2    5
c    3    6
d    4    7

In [18]: # 按索引排序

In [19]: df.sort_index(ascending=False)
Out[19]:
   one  two
d    4    7
c    3    6
b    2    5
a    1    4

In [20]: # 降序列排序

In [21]: df.sort_index(ascending=False, axis=1)
Out[21]:
   two  one
a    4    1
b    5    2
c    6    3
d    7    4

In [22]: # 列索引排序

In [23]:

pandas - 时间序列

时间序列就是以时间为索引的Series或DataFrame

datetime对象作为索引时时存储在DatetimeIndex对象中的

时间序列特殊功能:

​ 传入“年” 或着 “年月” 作为切片方式

​ 传入日期范围作为切片方式

​ 函数支持:resample(), truncate(),…

In [2]: import pandas as pd
In [2]: import pandas as pd

In [3]: pd.date_range("2010-01-01", '2010-01-20')
Out[3]:
DatetimeIndex(['2010-01-01', '2010-01-02', '2010-01-03', '2010-01-04',
               '2010-01-05', '2010-01-06', '2010-01-07', '2010-01-08',
               '2010-01-09', '2010-01-10', '2010-01-11', '2010-01-12',
               '2010-01-13', '2010-01-14', '2010-01-15', '2010-01-16',
               '2010-01-17', '2010-01-18', '2010-01-19', '2010-01-20'],
              dtype='datetime64[ns]', freq='D')

In [4]: import numpy as np

In [5]: sr = pd.Series(np.arange(20),index=pd.date_range('2017-01-01',periods=20
   ...: ))

In [6]: sr
Out[6]:
2017-01-01     0
2017-01-02     1
2017-01-03     2
2017-01-04     3
2017-01-05     4
2017-01-06     5
2017-01-07     6
2017-01-08     7
2017-01-09     8
2017-01-10     9
2017-01-11    10
2017-01-12    11
2017-01-13    12
2017-01-14    13
2017-01-15    14
2017-01-16    15
2017-01-17    16
2017-01-18    17
2017-01-19    18
2017-01-20    19
Freq: D, dtype: int64

In [7]: sr.index
Out[7]:
DatetimeIndex(['2017-01-01', '2017-01-02', '2017-01-03', '2017-01-04',
               '2017-01-05', '2017-01-06', '2017-01-07', '2017-01-08',
               '2017-01-09', '2017-01-10', '2017-01-11', '2017-01-12',
               '2017-01-13', '2017-01-14', '2017-01-15', '2017-01-16',
               '2017-01-17', '2017-01-18', '2017-01-19', '2017-01-20'],
              dtype='datetime64[ns]', freq='D')

In [8]: # 我们在这创建的 sr 的 index 是 DatetimeIndex类型的

In [9]: # 此时sr 就是一个时间序列

In [10]: sr['2017-01']  # 获取2017年1月的数据
Out[10]:
2017-01-01     0
2017-01-02     1
2017-01-03     2
2017-01-04     3
2017-01-05     4
2017-01-06     5
2017-01-07     6
2017-01-08     7
2017-01-09     8
2017-01-10     9
2017-01-11    10
2017-01-12    11
2017-01-13    12
2017-01-14    13
2017-01-15    14
2017-01-16    15
2017-01-17    16
2017-01-18    17
2017-01-19    18
2017-01-20    19
Freq: D, dtype: int64

In [11]: sr['2017-01-01':'2017-01-09']
Out[11]:
2017-01-01    0
2017-01-02    1
2017-01-03    2
2017-01-04    3
2017-01-05    4
2017-01-06    5
2017-01-07    6
2017-01-08    7
2017-01-09    8
Freq: D, dtype: int64

In [12]: sr.resample('W').sum()
Out[12]:
2017-01-01     0
2017-01-08    28
2017-01-15    77
2017-01-22    85
Freq: W-SUN, dtype: int64

In [13]: # 一周数字总和

In [14]: sr.resample('M').sum()
Out[14]:
2017-01-31    190
Freq: M, dtype: int64

In [15]: # 一月数字总和

In [16]: sr.resample('M').mean()
Out[16]:
2017-01-31    9.5
Freq: M, dtype: float64

In [17]: sr.truncate(before='2017-01-03') # 还有个 after 可以使用
Out[17]:
2017-01-03     2
2017-01-04     3
2017-01-05     4
2017-01-06     5
2017-01-07     6
2017-01-08     7
2017-01-09     8
2017-01-10     9
2017-01-11    10
2017-01-12    11
2017-01-13    12
2017-01-14    13
2017-01-15    14
2017-01-16    15
2017-01-17    16
2017-01-18    17
2017-01-19    18
2017-01-20    19
Freq: D, dtype: int64

In [18]:

Pandas 文件读取

数据文件常用格式:csv

pandas读取文件:从。文件名、url、文件对戏那个中加载数据

​ read_csv 默认分隔符为逗号 “,”

​ read_table 默认分隔符为制表符 “\t”

read_csv、 read_table 函数主要参数:

​ sep 指定分隔符,可以使用正则表达式如 ‘\s+’

​ header=None 指定文件没有列名

​ names 指定列名

​ index_col 指定某列索引

​ skiprows 跳过某些行 [1,2,3 ] # 即表示为跳过1,2,3行

​ na_values 指定某些字符串表示缺失值 [‘None’,‘null’ ] # 指定None 和 null字符串被解释成NaN

​ parse_dates 指定某些列被解析成日期,类型为bool值或者列表[]

In [19]: # pd.read_csv("文件名.csv", index_col='') # index_col 表示指定某一列作
    ...: 为行索引

In [20]: # pd.read_csv("文件名.csv", index_col=0) # index_col 表示指定第一列作为
    ...: 行索引

In [21]: # pd.read_csv("文件名.csv", index_col=0, parse_dates=True) # parse_date
    ...: s将文件中所有第时间对象转换成

In [22]: # 把能解释成时间序列第列都解释出来

In [23]: # pd.read_csv("文件名.csv", index_col=0, parse_dates=['列',‘列’]) # pa
    ...: rse_dates将文件中指定列名的时间对象转换成时间序列

In [24]:

In [24]: # 有些csv文件可能没有列名,当读取的时候会把第一行数据当作列名,这样就不
    ...: 太好了

In [25]: # pd.read_csv('文件名.csv', header=None) # 当指定 header=None 的时候 pd
    ...: 就会自己创建一个列名0123... 这样就不会影响数据了

In [26]: # pd.read_csv('文件名.csv', header=None, names=['列名1', ‘列名2’]) #
    ...: 当指定 header=None 的时候 pd就会自己创建一个我们提供的names列表作为列名

In [27]: # read_table 和read_csv没有大区别,table默认的分隔符为制表符

In [28]:

写入到csv文件:to_csv 函数

主要参数

sep 指定文件分隔符

na_rep 指定缺失值转换到字符串,默认为空字符串

header=False 不输出列名一行

index=False 不输出行索引一列

columns 指定输出到列,传输列表

Pandas 支持其他的文件类型:

​ json、xml、html、数据库、pickle、excel…

你可能感兴趣的:(python)