How to avoid duplicate document indexing in Lucene 6.0
<p>I am creating a Lucene Index for values got from database. I have set Index <code>OpenMode</code> as <code>OpenMode.CREATE_OR_APPEND</code>. </p>

<p>Index creation step is part of a Spring Batch Job. </p>

<p>My understanding is that when I run job for the first time, indexing might take a while but when I rerun the job again for <strong>same unchanged source data</strong>, it should be fast because document is already there so <strong>UPDATE OR INSERT</strong> has not be performed. </p>

<p>But for my case, subsequent indexing attempts for <strong>same unchanged source data</strong> gets slower and slower. </p>

<p>Answer to <a href="https://stackoverflow.com/questions/9731671/lucene-how-to-add-document-without-duplication">this question</a> says that it will be handled automatically <strong>based on a term.</strong></p>

<p>I am not sure as how to I define the term in my case to handle this?</p>

<p>Below is my sample code, </p>

<pre><code>        public Integer createIndex(IndexWriter writer, String str, LuceneIndexerInputVO luceneInputVO) throws Exception {
            Integer count = 0;
            Document d = null;
            txtFieldType.setTokenized(false);
            strFieldType.setTokenized(false);

            List&lt;IndexVO&gt; indexVO = null;

            indexVO = jdbcTemplate.
                    query(Constants.SELECT_FROM_TABLE1, 
                            new Object[] {luceneInputVO.getId1(), luceneInputVO.getId2(), str}, 
                            new IndexRowMapper());

            while (!indexVO.isEmpty()) {
                d = new Document();
                d.add(getStringField(Constants.ID, String.valueOf(luceneInputVO.getId())));
                .....
                ....
                writer.addDocument(d);
                indexVO.remove(indexVO.get(count));
                count++;
            }
            return count;
        }
</code></pre>

<p>What should I change in above code to not perform indexing when there is no change in source data?</p>

<p>I am a beginner to Lucene and not sure as how to define that <code>Term</code> which would decide about duplicity. </p>

<p>I don't want indices to be recreated and I wish new <code>Document</code> to be skipped ( don't do anything ) if exactly same <code>Document</code> already exists in Index. </p>

<p><strong>EDIT -</strong> I asked a long question but after reading SO for few Lucene related questions, I realize that I am simply asking for <strong>incremental indexing</strong> approach while focusing on duplicate avoidance provided a document represents a row of a RDBMS table having a primary key. If DB row is changed, update document otherwise not and add docs for new rows. </p>

<p><a href="https://stackoverflow.com/questions/12462652/lucene-how-to-perform-an-incremental-indexing-and-avoid-delete-and-redo">Question 1</a>,<a href="https://stackoverflow.com/questions/11450416/incremental-indexing-lucene">Question 2</a></p>