classifying new member into existing cluster in Python
<p>There is a group of texts <code>GROUP</code> from <code>text[:n]</code> (say 1-100 as an example) and a matrix of their cosine similarity to each other:</p>

<pre><code>     001   002   003  ...   100
001    1  0.53  0.47  ...  0.79
002 0.53     1  0.55  ...  0.30
003 0.47  0.55     1  ...  0.21
...  ...   ...   ...    1   ...
100 0.79  0.30  0.21  ...     1
</code></pre>

<p>When a new member <code>text[n+1]</code> is created, I would like to know whether <code>text[n+1]</code> is <strong>something new</strong> to the group. That is, it does not belong to any of the potential clusters in <code>text[1-100]</code> based on their surface similarity.</p>

<p>Knowing just a little about the variety of clustering techniques, I would like to know how to best complete this classification in Python. There are two rather general ways I am now doing some research about:</p>

<p>Whenever a new member <code>text[n+1]</code> is added to the existing <code>GROUP</code>:<code>text[:n]</code>:</p>

<ol>
<li><p>Run a clustering algorithm to cluster <code>text[:n]</code> based on their cosine similarity, then calculate the probability that <code>text[n+1]</code> belong to one of the existing clusters and the probability that it should be in a new cluster on its own. -> then use the probability to decide whether <code>text[n+1]</code> is "something new" or "something that similar stuff is already in our data before"</p></li>
<li><p>Alternatively, if I don't care about how many clusters are in <code>text[:n]</code>, but simply whether <code>text[n+1</code> is something new, is it possible to simplify the process to classifying <code>text[n+1]</code> to a binary <code>0 (something seen before)</code> or <code>1 (something new)</code> with better computational efficiency?</p></li>
</ol>

<p>Also, would something like <em>Chinese restaurant process</em> be potentially relevant to the question here? I appreciate any suggestions and resources you can point me towards as well.</p>