2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 1/26100 pandas puzzlesInspired by 100 Numpy exerises (https://github.com/rougier/numpy-100), here are 100* short puzzles fortesting your knowledge of pandas (http://pandas.pydata.org/) power.Since pandas is a large library with many different specialist features and functions, these excercises focusmainly on the fundamentals of manipulating data (indexing, grouping, aggregating, cleaning), making use of thecore DataFrame and Series objects.Many of the excerises here are stright-forward in that the solutions require no more than a few lines of code (inpandas or NumPy... dont go using pure Python or Cython!). Choosing the right methods and following bestpractices is the underlying goal.The exercises are loosely divided in sections. Each section has a difficulty rating; these ratings are subjective,of course, but should be a seen as a rough guide as to how inventive the required solution is.If youre just starting out with pandas and you are looking for some other resources, the official documentationis very extensive. In particular, some good places get a broader overview of pandas are...10 minutes to pandas (http://pandas.pydata.org/pandas-docs/stable/10min.html)pandas basics (http://pandas.pydata.org/pandas-docs/stable/basics.html)tutorials (http://pandas.pydata.org/pandas-docs/stable/tutorials.html)cookbook and idioms (http://pandas.pydata.org/pandas-docs/stable/cookbook.html#cookbook)Enjoy the puzzles!* the list of exercises is not yet complete! Pull requests or suggestions for additional exercises, corrections andimprovements are welcomed.Importing pandasGetting started and checking your pandas setupDifficulty: easy1. Import pandas under the name pd .In[1]:2. Print the version of pandas that has been imported.import pandas as pdimport numpy as np2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 2/26In[2]:3. Print out all the version information of the libraries that are required by the pandas library.Out[2]:0.23.4pd.__version__2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 3/26In[3]:DataFrame basicsINSTALLED VERSIONS------------------commit: Nonepython: 3.6.6.final.0python-bits: 64OS: DarwinOS-release: 17.7.0machine: x86_64processor: i386byteorder: littleLC_ALL: NoneLANG: zh_CN.UTF-8LOCALE: zh_CN.UTF-8pandas: 0.23.4pytest: Nonepip: 18.1setuptools: 39.1.0Cython: Nonenumpy: 1.14.5scipy: 1.1.0pyarrow: Nonexarray: NoneIPython: 6.5.0sphinx: Nonepatsy: Nonedateutil: 2.7.3pytz: 2018.5blosc: Nonebottleneck: Nonetables: Nonenumexpr: Nonefeather: Nonematplotlib: 2.2.2openpyxl: Nonexlrd: Nonexlwt: Nonexlsxwriter: Nonelxml: Nonebs4: Nonehtml5lib: 1.0.1sqlalchemy: Nonepymysql: Nonepsycopg2: Nonejinja2: 2.10s3fs: Nonefastparquet: Nonepandas_gbq: Nonepandas_datareader: Nonepd.show_versions()2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 4/26A few of the fundamental routines for selecting, sorting, adding and aggregatingdata in DataFramesDifficulty: easyNote: remember to import numpy using:import numpy as npConsider the following Python dictionary data and Python list labels :data = {animal: [cat, cat, snake, dog, dog, cat, snake, cat, dog, dog], age: [2.5, 3, 0.5, np.nan, 5, 2, 4.5, np.nan, 7, 3], visits: [1, 3, 2, 3, 2, 3, 1, 1, 2, 1], priority: [yes, yes, no, yes, no, no, no, yes, no, no]}labels = [a, b, c, d, e, f, g, h, i, j](This is just some meaningless data I made up with the theme of animals and trips to a vet.)4. Create a DataFrame df from this dictionary data which has the index labels .In[2]:5. Display a summary of the basic information about this DataFrame and its data.data = {animal: [cat, cat, snake, dog, dog, cat, snake, cat, dog age: [2.5, 3, 0.5, np.nan, 5, 2, 4.5, np.nan, 7, 3], visits: [1, 3, 2, 3, 2, 3, 1, 1, 2, 1], priority: [yes, yes, no, yes, no, no, no, yes, no, no]labels = [a, b, c, d, e, f, g, h, i, j]df = pd.DataFrame(data, index=labels)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 5/26In[5]:6. Return the first 3 rows of the DataFrame df .In?[6]:Index: 10 entries, a to jData columns (total 4 columns):animal 10 non-null objectage 8 non-null float64visits 10 non-null int64priority 10 non-null objectdtypes: float64(1), int64(1), object(2)memory usage: 400.0+ bytesOut[5]:age visitscount 8.000000 10.000000mean 3.437500 1.900000std 2.007797 0.875595min 0.500000 1.00000025% 2.375000 1.00000050% 3.000000 2.00000075% 4.625000 2.750000max 7.000000 3.000000Out[6]:animal age visits prioritya cat 2.5 1 yesb cat 3.0 3 yesc snake 0.5 2 nodf.info()# ...or...df.describe()df.iloc[:3]# or equivalentlydf.head(3)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 6/267. Select just the animal and age columns from the DataFrame df .In?[7]:8. Select the data in rows [3, 4, 8] and in columns [animal, age] .In?[3]:9. Select only the rows where the number of visits is greater than 3.Out[7]:animal agea cat 2.5b cat 3.0c snake 0.5d dog NaNe dog 5.0f cat 2.0g snake 4.5h cat NaNi dog 7.0j dog 3.0Out[3]:animal aged dog NaNe dog 5.0i dog 7.0df.loc[:, [animal, age]]# ordf[[animal, age]]df.loc[df.index[[3, 4, 8]], [animal, age]]2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 7/26In[4]:10. Select the rows where the age is missing, i.e. is NaN .In[5]:11. Select the rows where the animal is a cat and the age is less than 3.In[6]:12. Select the rows the age is between 2 and 4 (inclusive).Out[4]:animal age visits priorityOut[5]:animal age visits priorityd dog NaN 3 yesh cat NaN 1 yesOut[6]:animal age visits prioritya cat 2.5 1 yesf cat 2.0 3 nodf[df[visits] > 3]df[df[age].isnull()]df[(df[animal] == cat) & (df[age] http://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 8/26In[7]:13. Change the age in row f to 1.5.In[]:14. Calculate the sum of all visits (the total number of visits).In[]:15. Calculate the mean age for each different animal in df .In[8]:16. Append a new row k to df with your choice of values for each column. Then delete that row to return theoriginal DataFrame.Out[7]:animal age visits prioritya cat 2.5 1 yesb cat 3.0 3 yesf cat 2.0 3 noj dog 3.0 1 noOut[8]:animalcat 2.5dog 5.0snake 2.5Name: age, dtype: float64df[df[age].between(2, 4)]df.loc[f, age] = 1.5df[visits].sum()df.groupby(animal)[age].mean()2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 9/26In[]:17. Count the number of each type of animal in df .In[9]:18. Sort df first by the values in the age in decending order, then by the value in the visit column inascending order.In[10]:19. The priority column contains the values yes and no. Replace this column with a column of booleanvalues: yes should be True and no should be False .Out[9]:cat 4dog 4snake 2Name: animal, dtype: int64Out[10]:animal age visits priorityi dog 7.0 2 noe dog 5.0 2 nog snake 4.5 1 noj dog 3.0 1 nob cat 3.0 3 yesa cat 2.5 1 yesf cat 2.0 3 noc snake 0.5 2 noh cat NaN 1 yesd dog NaN 3 yesdf.loc[k] = [5.5, dog, no, 2]# and then deleting the new row...df = df.drop(k)df[animal].value_counts()df.sort_values(by=[age, visits], ascending=[False, True])2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 10/26In?[?]:20. In the animal column, change the snake entries to python.In?[14]:21. For each animal type and each number of visits, find the mean age. In other words, each row is an animal,each column is a number of visits and the values are the mean ages (hint: use a pivot table).In[15]:DataFrames: beyond the basicsSlightly trickier: you may need to combine two or more methods to get the rightanswerDifficulty: mediumThe previous section was tour through some basic but essential DataFrame operations. Below are some waysthat you might need to cut your data, but for which there is no single out of the box method. animal age visits prioritya cat 2.5 1 yesb cat 3.0 3 yesc python 0.5 2 nod dog NaN 3 yese dog 5.0 2 nof cat 2.0 3 nog python 4.5 1 noh cat NaN 1 yesi dog 7.0 2 noj dog 3.0 1 noOut[15]:visits 1 2 3animalcat 2.5 NaN 2.5dog 3.0 6.0 NaNpython 4.5 0.5 NaNdf[priority] = df[priority].map({yes: True, no: False})df[animal] = df[animal].replace(snake, python)print(df)df.pivot_table(index=animal, columns=visits, values=age, aggfunc=mean)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 11/2622. You have a DataFrame df with a column A of integers. For example:df = pd.DataFrame({A: [1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7]})How do you filter out rows which contain the same integer as the row immediately above?In[16]:23. Given a DataFrame of numeric values, saydf = pd.DataFrame(np.random.random(size=(5, 3))) # a 5x3 frame of float valueshow do you subtract the row mean from each element in the row?In[]:24. Suppose you have DataFrame with 10 columns of real numbers, for example:df = pd.DataFrame(np.random.random(size=(5, 10)), columns=list(abcdefghij))Which column of numbers has the smallest sum? (Find that columns label.)Out[16]:A0 11 23 34 45 58 69 7df = pd.DataFrame({A: [1, 2, 2, 3, 4, 5, 5, 5, 6, 7, 7]})df.loc[df[A].shift() != df[A]]# Alternatively, we could use drop_duplicates() here. Note# that this removes *all* duplicates though, so it wont# work as desired if A is [1, 1, 2, 2, 1, 1] for example.df.drop_duplicates(subset=A)df.sub(df.mean(axis=1), axis=0)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 12/26In[17]:25. How do you count how many unique rows a DataFrame has (i.e. ignore all rows that are duplicates)?In[]:The next three puzzles are slightly harder...26. You have a DataFrame that consists of 10 columns of floating--point numbers. Suppose that exactly 5entries in each row are NaN values. For each row of the DataFrame, find the column which contains the thirdNaN value.(You should return a Series of column labels.)In[]:27. A DataFrame has a column of groups grps and and column of numbers vals. For example:df = pd.DataFrame({grps: list(aaabbcaabcccbbc), vals: [12,345,3,1,45,14,4,52,54,23,235,21,57,3,87]})For each group, find the sum of the three greatest values.In?[?]:28. A DataFrame has two integer columns A and B. The values in A are between 1 and 100 (inclusive). Foreach group of 10 consecutive integers in A (i.e. (0, 10] , (10, 20] , ...), calculate the sum of thecorresponding values in column B.Out[17]:Adf.sum().idxmin()len(df) - df.duplicated(keep=False).sum()# or perhaps more simply...len(df.drop_duplicates(keep=False))(df.isnull().cumsum(axis=1) == 3).idxmax(axis=1)df.groupby(grp)[vals].nlargest(3).sum(level=0)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 13/26In[]:DataFrames: harder problemsThese might require a bit of thinking outside the box......but all are solvable using just the usual pandas/NumPy methods (and so avoid using explicit for loops).Difficulty: hard29. Consider a DataFrame df where there is an integer column X:df = pd.DataFrame({X: [7, 2, 0, 3, 4, 2, 5, 0, 3, 4]})For each value, count the difference back to the previous zero (or the start of the Series, whichever is closer).These values should therefore be [1, 2, 0, 1, 2, 3, 4, 0, 1, 2] . Make this a new column Y.In[]:Heres an alternative approach based on a cookbook recipe (http://pandas.pydata.org/pandasdocs/stable/cookbook.html#grouping):In[]:And another approach using a groupby:In[]:30. Consider a DataFrame containing rows and columns of purely numerical data. Create a list of the rowdf.groupby(pd.cut(df[A],np.arange(0, 101, 10)))[B].sum()izero = np.r_[-1, (df[X] == 0).nonzero()[0]] # indices of zerosidx = np.arange(len(df))df[Y] = idx - izero[np.searchsorted(izero - 1, idx) - 1]# http://stackoverflow.com/questions/30730981/how-to-count-distance-to-the-previous-# credit: Behzad Nourix = (df[X] != 0).cumsum()y = x != x.shift()df[Y] = y.groupby((y != y.shift()).cumsum()).cumsum()df[Y] = df.groupby((df[X] == 0).cumsum()).cumcount()# Were off by one before we reach the first zero.first_zero_idx = (df[X] == 0).idxmax()df[Y].iloc[0:first_zero_idx] += 12018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 14/26column index locations of the 3 largest values.In[]:31. Given a DataFrame with a column of group IDs, grps, and a column of corresponding integer values,vals, replace any negative values in vals with the group mean.In[]:32. Implement a rolling mean over groups with window size 3, which ignores NaN value. For example considerthe following DataFrame:>>> df = pd.DataFrame({group: list(aabbabbbabab), value: [1, 2, 3, np.nan, 2, 3, np.nan, 1, 7, 3, np.nan, 8]})>>> df group value0 a 1.01 a 2.02 b 3.03 b NaN4 a 2.05 b 3.06 b NaN7 b 1.08 a 7.09 b 3.010 a NaN11 b 8.0The goal is to compute the Series:df.unstack().sort_values()[-3:].index.tolist()# http://stackoverflow.com/questions/14941261/index-and-column-for-the-max-value-in-# credit: DSMdef replace(group): mask = group group[mask] = group[~mask].mean() return groupdf.groupby([grps])[vals].transform(replace)# http://stackoverflow.com/questions/14760757/replacing-values-with-groupby-means/# credit: unutbu2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 15/260 1.0000001 1.5000002 3.0000003 3.0000004 1.6666675 3.0000006 3.0000007 2.0000008 3.6666679 2.00000010 4.50000011 4.000000E.g. the first window of size three for group b has values 3.0, NaN and 3.0 and occurs at row index 5. Insteadof being NaN the value in the new column at this row index should be 3.0 (just the two non-NaN values areused to compute the mean (3+3)/2)In[]:Series and DatetimeIndexExercises for creating and manipulating Series with datetime dataDifficulty: easy/mediumpandas is fant代写pandas-puzzles作业、代做Python程序设计作业、代写Cython留学生作业、代做Python编程作业astic for working with dates and times. These puzzles explore some of this functionality.33. Create a DatetimeIndex that contains each business day of 2015 and use it to index a Series of randomnumbers. Lets call this Series s .In?[?]:34. Find the sum of the values in s for every Wednesday.g1 = df.groupby([group])[value] # group valuesg2 = df.fillna(0).groupby([group])[value] # fillna, then group valuess = g2.rolling(3, min_periods=1).sum() / g1.rolling(3, min_periods=1).count() # comps.reset_index(level=0, drop=True).sort_index() # drop/sort index# http://stackoverflow.com/questions/36988123/pandas-groupby-and-rolling-apply-ignordti = pd.date_range(start=2015-01-01, end=2015-12-31, freq=B)s = pd.Series(np.random.rand(len(dti)), index=dti)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 16/26In[]:35. For each calendar month in s , find the mean of values.In[]:36. For each group of four consecutive calendar months in s , find the date on which the highest valueoccurred.In[]:37. Create a DateTimeIndex consisting of the third Thursday in each month for the years 2015 and 2016.In]:Cleaning DataMaking a DataFrame easier to work withDifficulty: easy/mediumIt happens all the time: someone gives you data containing malformed strings, Python, lists and missing data.How do you tidy it up so you can get on with the analysis?Take this monstrosity as the DataFrame to use in the following puzzles:df = pd.DataFrame({From_To: [LoNDon_paris, MAdrid_miLAN, londON_StockhOlm, Budapest_PaRis, Brussels_londOn], FlightNumber: [10045, np.nan, 10065, np.nan, 10085], RecentDelays: [[23, 47], [], [24, 43, 87], [13], [67, 32]], Airline: [KLM(!), (12), (British Airways. ), 12. Air France, Swiss Air]})(Its some flight data I made up; its not meant to be accurate in any way.)38. Some values in the the FlightNumber column are missing. These numbers are meant to increase by 10 withs[s.index.weekday == 2].sum()s.resample(M).mean()s.groupby(pd.TimeGrouper(4M)).idxmax()pd.date_range(2015-01-01, 2016-12-31, freq=WOM-3THU)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 17/26each row so 10055 and 10075 need to be put in place. Fill in these missing numbers and make the column aninteger column (instead of a float column).In[]:39. The From_To column would be better as two separate columns! Split each string on the underscoredelimiter _ to give a new temporary DataFrame with the correct values. Assign the correct column names tothis temporary DataFrame.In[]:40. Notice how the capitalisation of the city names is all mixed up in this temporary DataFrame. Standardisethe strings so that only the first letter is uppercase (e.g. londON should become London.)In[]:41. Delete the From_To column from df and attach the temporary DataFrame from the previous questions.In[]:42. In the Airline column, you can see some extra puctuation and symbols have appeared around the airlinenames. Pull out just the airline name. E.g. (British Airways. ) should become BritishAirways .In[]:43. In the RecentDelays column, the values have been entered into the DataFrame as a list. We would like eachfirst value in its own column, each second value in its own column, and so on. If there isnt an Nth value, thevalue should be NaN.Expand the Series of lists into a DataFrame named delays , rename the columns delay_1 , delay_2 ,etc. and replace the unwanted RecentDelays column in df with delays .df[FlightNumber] = df[FlightNumber].interpolate().astype(int)temp = df.From_To.str.split(_, expand=True)temp.columns = [From, To]temp[From] = temp[From].str.capitalize()temp[To] = temp[To].str.capitalize()df = df.drop(From_To, axis=1)df = df.join(temp)df[Airline] = df[Airline].str.extract(([a-zA-Z\s]+), expand=False).str.strip()# note: using .strip() gets rid of any leading/trailing spaces2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 18/26In[]:The DataFrame should look much better now.Using MultiIndexesGo beyond flat DataFrames with additional index levelsDifficulty: mediumPrevious exercises have seen us analysing data from DataFrames equipped with a single index level. However,pandas also gives you the possibilty of indexing your data using multiple levels. This is very much like addingnew dimensions to a Series or a DataFrame. For example, a Series is 1D, but by using a MultiIndex with 2levels we gain of much the same functionality as a 2D DataFrame.The set of puzzles below explores how you might use multiple index levels to enhance data analysis.To warm up, well look make a Series with two index levels.44. Given the lists letters = [A, B, C] and numbers = list(range(10)) , construct aMultiIndex object from the product of the two lists. Use it to index a Series of random numbers. Call this Seriess .In[]:45. Check the index of s is lexicographically sorted (this is a necessary proprty for indexing to work correctlywith a MultiIndex).In[]:# there are several ways to do this, but the following approach is possibly the simpdelays = df[RecentDelays].apply(pd.Series)delays.columns = [delay_{}.format(n) for n in range(1, len(delays.columns)+1)]df = df.drop(RecentDelays, axis=1).join(delays)letters = [A, B, C]numbers = list(range(10))mi = pd.MultiIndex.from_product([letters, numbers])s = pd.Series(np.random.rand(30), index=mi)s.index.is_lexsorted()# or more verbosely...s.index.lexsort_depth == s.index.nlevels2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 19/2646. Select the labels 1 , 3 and 6 from the second level of the MultiIndexed Series.In[]:47. Slice the Series s ; slice up to label B for the first level and from label 5 onwards for the second level.In[]:48. Sum the values in s for each label in the first level (you should have Series giving you a total for labels A,B and C).In[]:49. Suppose that sum() (and other methods) did not accept a level keyword argument. How else couldyou perform the equivalent of s.sum(level=1) ?In[]:50. Exchange the levels of the MultiIndex so we have an index of the form (letters, numbers). Is this new Seriesproperly lexsorted? If not, sort it.In[]:Minesweepers.loc[:, [1, 3, 6]]s.loc[pd.IndexSlice[:B, 5:]]# or equivalently without IndexSlice...s.loc[slice(None, B), slice(5, None)]s.sum(level=0)# One way is to use .unstack()...# This method should convince you that s is essentially# just a regular DataFrame in disguise!s.unstack().sum(axis=0)new_s = s.swaplevel(0, 1)# checknew_s.index.is_lexsorted()# sortnew_s = new_s.sort_index()2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 20/26Generate the numbers for safe squares in a Minesweeper gridDifficulty: medium to hardIf youve ever used an older version of Windows, theres a good chance youve played with [Minesweeper](https://en.wikipedia.org/wiki/Minesweeper_(video_game)(https://en.wikipedia.org/wiki/Minesweeper_(video_game)). If youre not familiar with the game, imagine a gridof squares: some of these squares conceal a mine. If you click on a mine, you lose instantly. If you click on asafe square, you reveal a number telling you how many mines are found in the squares that are immediatelyadjacent. The aim of the game is to uncover all squares in the grid that do not contain a mine.In this section, well make a DataFrame that contains the necessary data for a game of Minesweeper:coordinates of the squares, whether the square contains a mine and the number of mines found on adjacentsquares.51. Lets suppose were playing Minesweeper on a 5 by 4 grid, i.e.X = 5Y = 4To begin, generate a DataFrame df with two columns, x and y containing every coordinate for thisgrid. That is, the DataFrame should start: x y0 0 01 0 12 0 2In[]:52. For this DataFrame df , create a new column of zeros (safe) and ones (mine). The probability of a mineoccuring at each location should be 0.4.In[]:53. Now create a new column for this DataFrame called adjacent . This column should contain thenumber of mines found on adjacent squares in the grid.(E.g. for the first row, which is the entry for the coordinate (0, 0) , count how many mines are found on thecoordinates (0, 1) , (1, 0) and (1, 1) .)p = pd.tools.util.cartesian_product([np.arange(X), np.arange(Y)])df = pd.DataFrame(np.asarray(p).T, columns=[x, y])# One way is to draw samples from a binomial distribution.df[mine] = np.random.binomial(1, 0.4, X*Y)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 21/26In[]:54. For rows of the DataFrame that contain a mine, set the value in the adjacent column to NaN.In[]:55. Finally, convert the DataFrame to grid of the adjacent mine counts: columns are the x coordinate, rowsare the y coordinate.In[]:PlottingVisualize trends and patterns in dataDifficulty: mediumTo really get a good understanding of the data contained in your DataFrame, it is often essential to create plots:if youre lucky, trends and anomalies will jump right out at you. This functionality is baked into pandas and thepuzzles below explore some of whats possible with the library.# Here is one way to solve using merges.# Its not necessary the optimal way, just# the solution I thought of first...df[adjacent] = \ df.merge(df + [ 1, 1, 0], on=[x, y], how=left)\ .merge(df + [ 1, -1, 0], on=[x, y], how=left)\ .merge(df + [-1, 1, 0], on=[x, y], how=left)\ .merge(df + [-1, -1, 0], on=[x, y], how=left)\ .merge(df + [ 1, 0, 0], on=[x, y], how=left)\ .merge(df + [-1, 0, 0], on=[x, y], how=left)\ .merge(df + [ 0, 1, 0], on=[x, y], how=left)\ .merge(df + [ 0, -1, 0], on=[x, y], how=left)\ .iloc[:, 3:]\ .sum(axis=1)# An alternative solution is to pivot the DataFrame# to form the actual grid of mines and use convolution.# See https://github.com/jakevdp/matplotlib_pydata2013/blob/master/examples/minesweefrom scipy.signal import convolve2dmine_grid = df.pivot_table(columns=x, index=y, values=mine)counts = convolve2d(mine_grid.astype(complex), np.ones((3, 3)), mode=same).real.asdf[adjacent] = (counts - mine_grid).ravel(F)df.loc[df[mine] == 1, adjacent] = np.nandf.drop(mine, axis=1)\ .set_index([y, x]).unstack()2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 22/2656. Pandas is highly integrated with the plotting library matplotlib, and makes plotting DataFrames very userfriendly!Plotting in a notebook environment usually makes use of the following boilerplate:import matplotlib.pyplot as plt%matplotlib inlineplt.style.use(ggplot)matplotlib is the plotting library which pandas plotting functionality is built upon, and it is usually aliased toplt .%matplotlib inline tells the notebook to show plots inline, instead of creating them in a separatewindow.plt.style.use(ggplot) is a style theme that most people find agreeable, based upon the styling ofRs ggplot package.For starters, make a scatter plot of this random data, but use black Xs instead of the default markers.df = pd.DataFrame({xs:[1,5,2,8,1], ys:[4,2,1,9,6]})Consult the documentation (https://pandas.pydata.org/pandasdocs/stable/generated/pandas.DataFrame.plot.html)if you get stuck!In[]:57. Columns in your DataFrame can also be used to modify colors and sizes. Bill has been keeping track of hisperformance at work over time, as well as how good he was feeling that day, and whether he had a cup ofcoffee in the morning. Make a plot which incorporates all four features of this DataFrame.(Hint: If youre having trouble seeing the plot, try multiplying the Series which you choose to represent size by10 or more)The chart doesnt have to be pretty: this isnt a course in data viz!df = pd.DataFrame({productivity:[5,2,3,1,4,5,6,7,8,3,4,8,9], hours_in :[1,9,6,5,3,9,2,9,1,7,4,2,2], happiness :[2,1,3,2,3,1,2,3,1,2,2,1,3], caffienated :[0,0,1,1,0,0,0,0,1,1,0,1,0]})import matplotlib.pyplot as plt%matplotlib inlineplt.style.use(ggplot)df = pd.DataFrame({xs:[1,5,2,8,1], ys:[4,2,1,9,6]})df.plot.scatter(xs, ys, color = black, marker = x)2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 23/26In[]:58. What if we want to plot multiple things? Pandas allows you to pass in a matplotlib Axis object for plots, andplots will also return an Axis object.Make a bar plot of monthly revenue with a line plot of monthly advertising spending (numbers in millions)df = pd.DataFrame({revenue:[57,68,63,71,72,90,80,62,59,51,47,52], advertising:[2.1,1.9,2.7,3.0,3.6,3.2,2.7,2.4,1.8,1.6,1.3,1.9], month:range(12) })In[]:Now were finally ready to create a candlestick chart, which is a very common tool used to analyze stock pricedata. A candlestick chart shows the opening, closing, highest, and lowest price for a stock during a timewindow. The color of the candle (the thick part of the bar) is green if the stock closed above its opening price,or red if below.df = pd.DataFrame({productivity:[5,2,3,1,4,5,6,7,8,3,4,8,9], hours_in :[1,9,6,5,3,9,2,9,1,7,4,2,2], happiness :[2,1,3,2,3,1,2,3,1,2,2,1,3], caffienated :[0,0,1,1,0,0,0,0,1,1,0,1,0]})df.plot.scatter(hours_in, productivity, s = df.happiness * 30, c = df.caffienatedf = pd.DataFrame({revenue:[57,68,63,71,72,90,80,62,59,51,47,52], advertising:[2.1,1.9,2.7,3.0,3.6,3.2,2.7,2.4,1.8,1.6,1.3,1.9], month:range(12) })ax = df.plot.bar(month, revenue, color = green)df.plot.line(month, advertising, secondary_y = True, ax = ax)ax.set_xlim((-1,12))2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 24/26This was initially designed to be a pandas plotting challenge, but it just so happens that this type of plot is justnot feasible using pandas methods. If you are unfamiliar with matplotlib, we have provided a function that willplot the chart for you so long as you can use pandas to get the data into the correct format.Your first step should be to get the data in the correct format using pandas time-series grouping function. Wewould like each candle to represent an hours worth of data. You can write your own aggregation functionwhich returns the open/high/low/close, but pandas has a built-in which also does this.The below cell contains helper functions. Call day_stock_data() to generate a DataFrame containing theprices a hypothetical stock sold for, and the time the sale occurred. Call plot_candlestick(df) on yourproperly aggregated and formatted stock data to print the candlestick chart.2018/11/8 100-pandas-puzzles-with-solutionshttp://localhost:8889/notebooks/100-pandas-puzzles-with-solutions.ipynb 25/26In[]:59. Generate a days worth of random stock data, and aggregate / reformat it so that it has hourly summariesof the opening, highest, lowest, and closing pricesIn[]:#This function is designed to create semi-interesting r转自：http://ass.3daixie.com/2018121440483895.html
