今天是讀《python數(shù)據(jù)分析基礎(chǔ)》的第18天，讀書筆記的內(nèi)容是使用pandas進行數(shù)據(jù)清洗以及探索
由于原始數(shù)據(jù)在某種程度上是“臟”的，原始數(shù)據(jù)并不能完全使用于分析。因此，需要為其進行清洗。而為了解數(shù)據(jù)的大致情況，則需要進行數(shù)據(jù)探索。
以下是進行簡單的數(shù)據(jù)清洗以及探索的代碼：

注：數(shù)據(jù)來源于https://github.com/cbrownley/foundations-for-analytics-with-python/tree/master/statistics/churn.csv

#數(shù)據(jù)探索

import pandas as pd
import numpy as np


#導入數(shù)據(jù)
inputCsv='文件路徑'
churn=pd.read_csv(inputCsv)


#數(shù)據(jù)預處理:將表頭的空格、引號以及問號去除,以及將大寫字母轉(zhuǎn)換為小寫字母
#print(churn.columns)
churn.columns=churn.columns.str.replace(' ','_').str.replace('\'','').str.strip('?')
churn.columns=churn.columns.str.lower()
#將churn列中元素末尾的'.'去除
churn.churn=churn.churn.str.strip('.')
#將churn轉(zhuǎn)換為01編碼并創(chuàng)建新列churn01
churn['churn01']=np.where(churn.churn=='True',1,0)
print(churn.head())


#按制定類別變量state分組計算其他變量的均值、最大值、最小值
print(churn.groupby('churn')[['day_calls','eve_calls','night_calls']].agg(['count','mean','std']))


#按指定值分組計算，并計算不同變量各自的統(tǒng)計值
print(churn.groupby('churn').agg({'day_calls':['mean','max','min'],'eve_calls':['mean','max','min'],'night_calls':['mean','max','min'],'day_mins':['mean','std'],'eve_mins':['mean','std'],'night_mins':['mean','std']}))


#按特定連續(xù)型變量total_charge將數(shù)據(jù)分箱并計算各組的統(tǒng)計值
#生成total_charge變量
churn['total_charge']=churn['day_charge']+churn['eve_charge']+churn['night_charge']+churn['intl_charge']
#按記錄數(shù)量分箱
cut_cat=pd.cut(churn.total_charge,5,precision=2)
qcut_cat=pd.qcut(churn.total_charge,[0,0.25,0.5,0.75,1])
#按cut_cat和qcut_cat分別分組計算total_chage的均值
print('cut',churn.groupby(cut_cat)[['total_charge']].agg(['mean']))
print('qcut',churn.groupby(qcut_cat)[['total_charge']].agg(['mean']))


#對變量intl_plan進行獨熱編碼：將離散型變量轉(zhuǎn)換為一組二值變量
dummies=pd.get_dummies(churn.intl_plan)