<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Data Processing
We need to make a column for average stats for each 'mon
We need to label each 'mon by its generation
(We should figure out a way to ignore non-stat changed formes i.e. Arceus, as he may be upsetting the Gen IV data)
Step1:  Some Stats
Step2:  Machine Learning and Clustering
Step3:  PCA
Step4:  K-Means Clustering
<ASSISTANT_TASK:>
Python Code:
mons["AVERAGE_STAT"] = mons["STAT_TOTAL"]/6
gens = pd.Series([0 for i in range(len(mons.index))], index=mons.index)
for ID, mon in mons.iterrows():
    if 0<mon.DEXID<=151:
        gens[ID] = 1
    elif 151<mon.DEXID<=251:
        gens[ID] = 2
    elif 251<mon.DEXID<=386:
        gens[ID] = 3
    elif 386<mon.DEXID<=493:
        gens[ID] = 4
    elif 493<mon.DEXID<=649:
        gens[ID] = 5
    elif 649<mon.DEXID<=721:
        gens[ID] = 6
    elif 721<mon.DEXID<=805:
        gens[ID] = 7
    else:
        gens[ID] = 0
mons["GEN"] = gens
mons.to_csv("./data/pokemon_preUSUM_data.csv")
gen = {}
for i in range(1,8):
    gen[i] = mons[mons.GEN == i]
plt.figure(100)
colors = sns.color_palette("colorblind", 7)
for i in range(1,8):
    sns.distplot( mons[mons["GEN"] == i]["STAT_TOTAL"], hist=False,kde=True, color=colors[i-1], label=f"Gen {i}")
plt.legend()
plt.show()
Explanation: Data Processing
We need to make a column for average stats for each 'mon
We need to label each 'mon by its generation
(We should figure out a way to ignore non-stat changed formes i.e. Arceus, as he may be upsetting the Gen IV data)
End of explanation
stat_averages_by_gen = {i:gen[i].AVERAGE_STAT for i in range(1,8)}
testable_data = list(stat_averages_by_gen.values())
data = [list(gen) for gen in testable_data]
data = np.array(data)
averages = {i: stat_averages_by_gen[i].mean() for i in range(1,8)}
averages
stats.kruskal(*data)
recarray = mons.to_records()
test = comp.pairwise_tukeyhsd(recarray["AVERAGE_STAT"], recarray["GEN"])
test.summary()
Explanation: Some Stats
End of explanation
np.random.seed(525_600)
stats_gens = mons[['HP', 'ATTACK', 'DEFENSE',
       'SPECIAL_ATTACK', 'SPECIAL_DEFENSE', 'SPEED', 'GEN']]
X = np.c_[stats_gens]
Explanation: Machine Learning and Clustering
End of explanation
pca = decomposition.PCA()
pca.fit(X)
pca.explained_variance_
pca.n_components = 3
X_reduced = pca.fit_transform(X)
X_reduced.shape
pca.get_params()
Explanation: PCA
End of explanation
from sklearn import cluster
k_means = cluster.KMeans(n_clusters = 6)
k_means.fit(X)
mons["KMEANS_LABEL"] = pd.Series(k_means.labels_)
plotData = mons[["GEN", "STAT_TOTAL", "KMEANS_LABEL"]]
colors = sns.color_palette("colorblind", 7)
for i in range(1,8):
    sns.distplot( plotData[plotData["GEN"] == i]["STAT_TOTAL"], color=colors[i-1])
plt.figure(925)
sns.boxplot(x="KMEANS_LABEL", y="STAT_TOTAL", data=plotData)
plt.show()
plt.figure(9050624)
sns.pairplot(plotData, kind="scatter", hue="GEN", palette=colors)
plt.show()
plotData.to_csv("./data/kmeans.csv")
Explanation: K-Means Clustering
End of explanation
<END_TASK>