Simple plots of eigenvectors for sklearn.decomposition.PCA
<p>I'm trying to understand how <code>Principal Component Analysis</code> works and I am testing it on the <code>sklearn.datasets.load_iris</code> dataset.  I understand how each step works (e.g. standardize the data, covariance, eigendecomposition, sort for highest eigenvalue, transform original data to new axis using <code>K</code> selected dimensions).</p>
<p>The next step is to visualize where these <code>eigenvectors</code> are being projected into on the dataset (on the <code>PC1 vs. PC2 plot</code>, right?).</p>
<h3>Can someone explain how to plot [PC1, PC2, PC3] eigenvectors on a 3D plot of the reduced dimension dataset?</h3>
<p>Also, am I plotting this 2D version correctly? I'm not sure why my first eigenvector has a shorter length.  Should I multiply by the eigenvalue?</p>
<hr />
<p><strong>Here is some of the research I have done to accomplish this:</strong></p>
<p>The PCA method that I'm following is from :
<a href="https://plot.ly/ipython-notebooks/principal-component-analysis/#Shortcut---PCA-in-scikit-learn" rel="nofollow noreferrer">https://plot.ly/ipython-notebooks/principal-component-analysis/#Shortcut---PCA-in-scikit-learn</a> (although I don't want to use <code>plotly</code>.  I want to stick with <code>pandas, numpy, sklearn, matplotlib, scipy, and seaborn</code>)</p>
<p>I've been following this tutorial for plotting eigenvectors it seems pretty simple: <a href="https://stackoverflow.com/questions/18299523/basic-example-for-pca-with-matplotlib">Basic example for PCA with matplotlib</a> but I can't seem to replicate the results with my data.</p>
<p>I found this but it seems overly complicated for what I'm trying to do and I don't want to have to create a <code>FancyArrowPatch</code>: <a href="https://stackoverflow.com/questions/35508141/plotting-the-eigenvector-of-covariance-matrix-using-matplotlib-and-np-linalg">plotting the eigenvector of covariance matrix using matplotlib and np.linalg</a></p>
<hr />
<p><strong>I've tried to make my code as straightforward as possible to follow the other tutorials:</strong></p>
<pre><code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler
from sklearn import decomposition
import seaborn as sns; sns.set_style(&quot;whitegrid&quot;, {'axes.grid' : False})

%matplotlib inline
np.random.seed(0)

# Iris dataset
DF_data = pd.DataFrame(load_iris().data, 
                       index = [&quot;iris_%d&quot; % i for i in range(load_iris().data.shape[0])],
                       columns = load_iris().feature_names)

Se_targets = pd.Series(load_iris().target, 
                       index = [&quot;iris_%d&quot; % i for i in range(load_iris().data.shape[0])], 
                       name = &quot;Species&quot;)

# Scaling mean = 0, var = 1
DF_standard = pd.DataFrame(StandardScaler().fit_transform(DF_data), 
                           index = DF_data.index,
                           columns = DF_data.columns)

# Sklearn for Principal Componenet Analysis

# Dims
m = DF_standard.shape[1]
K = 2

# PCA (How I tend to set it up)
M_PCA = decomposition.PCA(n_components=m)
DF_PCA = pd.DataFrame(M_PCA.fit_transform(DF_standard), 
                columns=[&quot;PC%d&quot; % k for k in range(1,m + 1)]).iloc[:,:K]


# Plot the eigenvectors
#https://stackoverflow.com/questions/18299523/basic-example-for-pca-with-matplotlib

# This is where stuff gets weird...
data = DF_standard

mu = data.mean(axis=0)
eigenvectors, eigenvalues = M_PCA.components_, M_PCA.explained_variance_ #eigenvectors, eigenvalues, V = np.linalg.svd(data.T, full_matrices=False)
projected_data = DF_PCA #np.dot(data, eigenvectors)

sigma = projected_data.std(axis=0).mean()

fig, ax = plt.subplots(figsize=(10,10))
ax.scatter(projected_data[&quot;PC1&quot;], projected_data[&quot;PC2&quot;])
for axis, color in zip(eigenvectors[:K], [&quot;red&quot;,&quot;green&quot;]):
#     start, end = mu, mu + sigma * axis ### leads to &quot;ValueError: too many values to unpack (expected 2)&quot;

    # So I tried this but I don't think it's correct
    start, end = (mu)[:K], (mu + sigma * axis)[:K] 
    ax.annotate('', xy=end,xytext=start, arrowprops=dict(facecolor=color, width=1.0))
    
ax.set_aspect('equal')
plt.show()
</code></pre>
<p><a href="https://i.stack.imgur.com/t9st0.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/t9st0.png" alt="enter image description here" /></a></p>