More efficient way to compare two huge directories and replace same files
<p><code>F:/original_images</code>, <code>E:/resized_images</code></p>

<p>I have <strong>two HDDs</strong> containing a really lot of directories and image(jpg) files, each total size is about <strong>1.5TB(original) and 400GB(resized)</strong>.</p>

<p>Each have same file names but different size(resized). Then I have to replace the resized ones with the originals. Unfortunately, <strong>each directory hierarchy is totally different.</strong></p>

<p>I managed to do this job, but it takes a really really long time. I'm expecting few days to complete. It has two loops(<code>Files.walkFileTree()</code>) that just searching for the matched from A to Z. Not smart at all.</p>

<pre><code>public static void main(String[] args) throws IOException {
        FileWriter ostream = new FileWriter("result.txt");
        BufferedWriter out = new BufferedWriter(ostream);

        String fromDir = "F:/original_images";
        String toDir = "E:/resized_images";
        final Path source = Paths.get(fromDir);
        final Path target = Paths.get(toDir);

        Files.walkFileTree(source, EnumSet.of(FileVisitOption.FOLLOW_LINKS), Integer.MAX_VALUE,
                            new SimpleFileVisitor&lt;Path&gt;() {
            @Override
            public FileVisitResult visitFile(Path sourceFile,
                    BasicFileAttributes attrs) throws IOException {
                // if jpg (there are no jpeg)
                if(sourceFile.toString().toLowerCase().endsWith("jpg")) {

                    // search for the matching file                 
                    // start ** inner of [Files.walkFileTree()]
                    Files.walkFileTree(target, EnumSet.of(FileVisitOption.FOLLOW_LINKS), Integer.MAX_VALUE,
                            new SimpleFileVisitor&lt;Path&gt;() {
                        @Override
                        public FileVisitResult visitFile(Path Targetfile, BasicFileAttributes attrs) throws IOException {

                            if(sourceFile.getFileName().equals(Targetfile.getFileName())) {
                                out.write("replace : [" + sourceFile + "] -&gt; [" + Targetfile + "]");
                                try {
                                    // copy..
                                    Files.copy(sourceFile, Targetfile, REPLACE_EXISTING);
                                }catch(Exception e) {
                                    out.write(e.toString());
                                }
                                // stop searching for this file.
                                return FileVisitResult.TERMINATE;
                            }else
                                return FileVisitResult.CONTINUE;
                        }
                    });
                    // end ** inner of [Files.walkFileTree()]
                }
                return FileVisitResult.CONTINUE;
            }
        });
        out.write("[completed folder] " + fromDir);
        out.close();
    }
</code></pre>

<p>I believe there must be the smarter way. </p>

<p>(My guess is storing the file names in indexed array, coz it's much faster to compare.)</p>

<p>How would you do this?</p>

<p><strong>update</strong>(solved)</p>

<p>By adopting the idea of two answers, I finally came to do it.</p>

<p>The source code is too long to show, but the concise is :</p>

<ol>
<li><p>loop 'resized_images' and store files_info into the hashmap(key:file_name, value:full_Path).</p></li>
<li><p>loop 'original_images' and store files_info into the hashmap(key:file_name, value:full_Path). I made each hashmap for
each sub-directory for the efficiency.</p></li>
<li><p>compare and replace each 'resized' and 'original' hashmap.</p></li>
</ol>

<p>The result is much much faster than before. Most of the execution time is when copying files. Except that, it take less than 10 minutes.</p>