<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Clustering
Version 0.1
By AA Miller (Northwestern/CIERA)
03 Mar 2022
In this notebook, we will explore the use of clustering methods in low dimensional data spaces. 
Problem 1) Load and plot Iris data set
Problem 1a
Import the iris data set from scikit-learn.
Step1:  Problem 1b
As a baseline for reference, make a scatter plot of the iris data in the sepal length-sepal width plane.
Step2:  Problem 2) $k$-means clustering
As a subfield of unsupervised learning, clustering aims to group/separate sources in the multidimensional feature space. The "unsupervised" comes from the fact that there are no target labels provided to the algorithm, so the machine is asked to cluster the data "on its own." The lack of labels means there is no (simple) method for validating the accuracy of the solution provided by the machine (though sometimes simple examination can show the results are terrible).
For this reason, "classic" unsupervised methods are not particularly useful for astronomy.$^\dagger$ Supposing one did find some useful clustering structure, an adversarial researcher could always claim that the current feature space does not accurately capture the physics of the system and as such the clustering result is not interesting or, worse, erroneous.
$^\dagger$This is my (AAM) opinion and there are many others who disagree.
We start today with the most famous, and simple, clustering algorithm
Step3:  Problem 2b 
Fit a $k = 2$, $k$-means model to the iris data. Plot the resulting clusters in the sepal length-sepal width plane.
Step4:  Problem 2c 
Fit a $k = 3$, $k$-means model to the iris data. Plot the resulting clusters in the sepal length-sepal width plane.
Step5:  Problem 2d
Pretend that you do not know which iris sources belong to which class. Given this, which of the two clustering solutions ($k=2$ or $k=3$) would you identify as superior? 
Knowing that there are in fact 3 different clusters, which of the two clustering solutions would you identify as superior?
write your answer here
Problem 2e 
How do the results change if the 3 cluster model is called with n_init = 1 and init = 'random' options? Use rs for the random state [this allows me to cheat in service of making a point].
*Note - the respective defaults for these two parameters are 10 and k-means++, respectively. Read the docs to see why these choices are, likely, better than those in 2b.
Step6:  That doesn't look right at all! 
So in addition to not knowing the correct number of clusters in the data, we see that the results are also sensitive to how the cluster positions are initiated. 
$k$-means evaluates the Euclidean distance between individual sources and cluster centers, thus, the magnitude of the individual features has a strong effect on the final clustering outcome.
Problem 2f 
Calculate the mean, standard deviation, min, and max of each feature in the iris data set. Based on these summaries, which feature is most important for clustering?
Step7:  write your answer here
Since $k$-means is built on Euclidean distance measures in the feature space, it can be really useful to re-scale all the features prior to applying the clustering algorithm. 
(Two notes – (a) some algorithms are extremely sensitive to feature scaling so this is a always a good thing to keep in mind, and (b) the iris data set is small and of relatively similar scale so the effects will not be that dramatic)
Imagine you are classifying stellar light curves
Step8:  write your answer here
How do I test the accuracy of my clusters?
Essentially - you don't. There are some methods that are available, but they essentially compare clusters to labeled samples, and if the samples are labeled it is likely that supervised learning is more useful anyway. If you are curious, scikit-learn does provide some built-in functions for analyzing clustering, but again, it is difficult to evaluate the validity of any newly discovered clusters. 
What if I don't know how many clusters are present in the data?
An excellent question, as you will almost never know this a priori. Many algorithms, like $k$-means, do require the number of clusters to be specified, but some other methods do not.
Problem 3) DBSCAN
During the lecture we saw that DBSCAN can be used to identify clusters without the pre-specification of the number of clusters to search for. 
In brief, DBSCAN requires two parameters
Step9:  Problem 3b
Adjust the tuning parameters to see how they affect the final clustering results. How does the use of DBSCAN compare to $k$-means? Can you obtain 3 clusters with DBSCAN? If not, given the knowledge that the iris dataset has 3 classes - does this invalidate DBSCAN as a viable algorithm?
Step10:  write your answer here
Problem 4) Cluster SDSS Galaxy Data
The following query will select 10k likely galaxies from the SDSS database and return the results of that query into an astropy.Table object. (For now, if you are not familiar with the SDSS DB schema, don't worry about this query, just know that it returns a bunch of photometric features.)
from astroquery.sdss import SDSS  # enables direct queries to the SDSS database
GALquery = 
Step12:  Problem 4b 
Using the SDSS galaxy data, identify interesting clusters within the data. This question is intentionally very open ended. If you uncover anything especially exciting you'll have a chance to share it with the group. Feel free to use the algorithms discussed above, or any other packages available via sklearn. Can you make sense of the clusters in the context of galaxy evolution? 
Hint - don't fret if you know nothing about galaxy evolution (neither do I!). Just take a critical look at the clusters that are identified
<ASSISTANT_TASK:>
Python Code:
from sklearn import datasets
iris = # complete
Explanation: Clustering
Version 0.1
By AA Miller (Northwestern/CIERA)
03 Mar 2022
In this notebook, we will explore the use of clustering methods in low dimensional data spaces. 
Problem 1) Load and plot Iris data set
Problem 1a
Import the iris data set from scikit-learn.
End of explanation
fig, ax = plt.subplots()
ax.scatter( # complete
# complete
# complete
# complete
Explanation: Problem 1b
As a baseline for reference, make a scatter plot of the iris data in the sepal length-sepal width plane.
End of explanation
from sklearn.cluster import # complete
Explanation: Problem 2) $k$-means clustering
As a subfield of unsupervised learning, clustering aims to group/separate sources in the multidimensional feature space. The "unsupervised" comes from the fact that there are no target labels provided to the algorithm, so the machine is asked to cluster the data "on its own." The lack of labels means there is no (simple) method for validating the accuracy of the solution provided by the machine (though sometimes simple examination can show the results are terrible).
For this reason, "classic" unsupervised methods are not particularly useful for astronomy.$^\dagger$ Supposing one did find some useful clustering structure, an adversarial researcher could always claim that the current feature space does not accurately capture the physics of the system and as such the clustering result is not interesting or, worse, erroneous.
$^\dagger$This is my (AAM) opinion and there are many others who disagree.
We start today with the most famous, and simple, clustering algorithm: $k$-means. $k$-means clustering looks to identify $k$ convex clusters, where $k$ is a user defined number. And here-in lies the rub: if we truly knew the number of clusters in advance, we likely wouldn't need to perform any clustering in the first place. This is the major downside to $k$-means. 
As a reminder from lecture, the pseudocode for $k$-means: 
initiate search by identifying k points (i.e. the cluster centers)
loop 
    assign each point in the data set to the closest cluster center
    calculate new cluster centers based on mean position of all points within each cluster
    if diff(new center - old center) &lt; threshold:
        stop (i.e. clusters are defined)
The threshold is defined by the user, though in some cases the total number of iterations can also be used as a stopping criteria. An advantage of $k$-means is that the solution will always converge, though the solution may only be a local minimum. Disadvantages include the assumption of convexity, i.e. difficult to capture complex geometry, and the curse of dimensionality (though as discussed in lecture it is possible to apply dimensionality reduction techniques prior to applying clustering).
In scikit-learn the KMeans algorithm is implemented as part of the sklearn.cluster module. 
Problem 2a 
Import KMeans
End of explanation
Kcluster = # complete
Kcluster.fit( # complete
fig, ax = plt.subplots()
ax.scatter( # complete
# complete
# complete
# complete
Explanation: Problem 2b 
Fit a $k = 2$, $k$-means model to the iris data. Plot the resulting clusters in the sepal length-sepal width plane.
End of explanation
Kcluster = # complete
# complete
fig, ax = plt.subplots()
ax.scatter( # complete
Explanation: Problem 2c 
Fit a $k = 3$, $k$-means model to the iris data. Plot the resulting clusters in the sepal length-sepal width plane.
End of explanation
rs = 14
Kcluster1 = KMeans( # complete
# complete
    
fig, ax = plt.subplots()
ax.scatter( # complete
Explanation: Problem 2d
Pretend that you do not know which iris sources belong to which class. Given this, which of the two clustering solutions ($k=2$ or $k=3$) would you identify as superior? 
Knowing that there are in fact 3 different clusters, which of the two clustering solutions would you identify as superior?
write your answer here
Problem 2e 
How do the results change if the 3 cluster model is called with n_init = 1 and init = 'random' options? Use rs for the random state [this allows me to cheat in service of making a point].
*Note - the respective defaults for these two parameters are 10 and k-means++, respectively. Read the docs to see why these choices are, likely, better than those in 2b.
End of explanation
# complete
# complete
# complete
# complete
# complete
# complete
Explanation: That doesn't look right at all! 
So in addition to not knowing the correct number of clusters in the data, we see that the results are also sensitive to how the cluster positions are initiated. 
$k$-means evaluates the Euclidean distance between individual sources and cluster centers, thus, the magnitude of the individual features has a strong effect on the final clustering outcome.
Problem 2f 
Calculate the mean, standard deviation, min, and max of each feature in the iris data set. Based on these summaries, which feature is most important for clustering?
End of explanation
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler().fit( # complete
Kcluster = # complete
# complete
    
fig, ax = plt.subplots()
ax.scatter( # complete
Explanation: write your answer here
Since $k$-means is built on Euclidean distance measures in the feature space, it can be really useful to re-scale all the features prior to applying the clustering algorithm. 
(Two notes – (a) some algorithms are extremely sensitive to feature scaling so this is a always a good thing to keep in mind, and (b) the iris data set is small and of relatively similar scale so the effects will not be that dramatic)
Imagine you are classifying stellar light curves: the data set will include binary white dwarf stars with periods of $\sim 0.01 \; \mathrm{d}$ and Mira variables with periods of $\sim 1000 \; \mathrm{d}$. Without re-scaling, this feature that covers 6 orders of magnitude! Without rescaling all other features will add little weight to any final clustering solution.
The two most common forms of re-scaling are to rescale to a guassian with mean $= 0$ and variance $= 1$, or to rescale the min and max of the feature to $[0, 1]$. The best normalization is problem dependent. The sklearn.preprocessing module makes it easy to re-scale the feature set.$^\dagger$  
$\dagger$ For supervised methods, it is essential that the same scaling used for the training set be used for all other data run through the model. The testing, validation, and field observations cannot be re-scaled independently. This would result in meaningless final classifications/predictions.
Problem 2g 
Re-scale the features to normal distributions, and perform $k$-means clustering on the iris data. How do the results compare to those obtained earlier? 
Hint - you may find 'StandardScaler()' within the sklearn.preprocessing module useful.
End of explanation
from sklearn.cluster import DBSCAN
dbs = # complete
dbs.fit( # complete
fig, ax = plt.subplots()
ax.scatter( # complete
Explanation: write your answer here
How do I test the accuracy of my clusters?
Essentially - you don't. There are some methods that are available, but they essentially compare clusters to labeled samples, and if the samples are labeled it is likely that supervised learning is more useful anyway. If you are curious, scikit-learn does provide some built-in functions for analyzing clustering, but again, it is difficult to evaluate the validity of any newly discovered clusters. 
What if I don't know how many clusters are present in the data?
An excellent question, as you will almost never know this a priori. Many algorithms, like $k$-means, do require the number of clusters to be specified, but some other methods do not.
Problem 3) DBSCAN
During the lecture we saw that DBSCAN can be used to identify clusters without the pre-specification of the number of clusters to search for. 
In brief, DBSCAN requires two parameters: minPts, the minimum number of points necessary for a cluster, and $\epsilon$, a distance measure (see the lecture for the full pseudocode). 
The general downsides for DBSCAN are that the results are highly dependent on the two tuning parameters, and that clusters of highly different densities can be difficult to recover (because $\epsilon$ and minPts is specified for all clusters. 
In scitkit-learn the 
DBSCAN algorithm is part of the sklearn.cluster module. $\epsilon$ and minPts are set by eps and min_samples, respectively. 
Problem 3a 
Cluster the iris data using DBSCAN. Use the scikit-learn defaults. Plot the results in the sepal width-sepal length plane. 
Note - DBSCAN labels outliers as $-1$, and thus, plt.scatter(), will plot all these points as the same color.
End of explanation
dbs = DBSCAN( # complete
dbs.fit(# complete
    
fig, ax = plt.subplots()
ax.scatter( # complete
Explanation: Problem 3b
Adjust the tuning parameters to see how they affect the final clustering results. How does the use of DBSCAN compare to $k$-means? Can you obtain 3 clusters with DBSCAN? If not, given the knowledge that the iris dataset has 3 classes - does this invalidate DBSCAN as a viable algorithm?
End of explanation
# complete
# complete
# complete
Explanation: write your answer here
Problem 4) Cluster SDSS Galaxy Data
The following query will select 10k likely galaxies from the SDSS database and return the results of that query into an astropy.Table object. (For now, if you are not familiar with the SDSS DB schema, don't worry about this query, just know that it returns a bunch of photometric features.)
from astroquery.sdss import SDSS  # enables direct queries to the SDSS database
GALquery = SELECT TOP 5000 
         p.dered_u - p.dered_g as ug, p.dered_g - p.dered_r as gr, 
         p.dered_g - p.dered_i as gi, p.dered_g - p.dered_z as gz,             
         p.petroRad_i, p.petroR50_i, p.deVAB_i, p.fracDev_i
         FROM PhotoObjAll AS p JOIN specObjAll s ON s.bestobjid = p.objid
         WHERE p.mode = 1 AND s.sciencePrimary = 1 AND p.clean = 1 AND p.type = 3
         AND p.deVAB_i &gt; -999 AND p.petroRad_i &gt; -999 AND p.petroR50_i &gt; -999 AND p.dered_r &lt; 20
           
SDSSgals = SDSS.query_sql(GALquery)
SDSSgals
Problem 4a 
Download the SDSS galaxy data
Read in the file galaxy_clustering.csv, and convert the data into a feature array X.
End of explanation
# complete
# complete
# complete
# complete
# complete
# complete
# complete
# complete
# complete
# complete
Explanation: Problem 4b 
Using the SDSS galaxy data, identify interesting clusters within the data. This question is intentionally very open ended. If you uncover anything especially exciting you'll have a chance to share it with the group. Feel free to use the algorithms discussed above, or any other packages available via sklearn. Can you make sense of the clusters in the context of galaxy evolution? 
Hint - don't fret if you know nothing about galaxy evolution (neither do I!). Just take a critical look at the clusters that are identified
End of explanation
<END_TASK>