Remove duplicate value in csv on combine
<p>I encountered the below code online and modified this on my need.
I just wanna ask since i am new to batch file if there is a way to remove duplicate values after the combine.</p>

<pre><code>@echo off

ECHO Set working directory
pushd %~dp0

ECHO Deleting existing combined file
del combined.csv

setlocal ENABLEDELAYEDEXPANSION


set cnt=1


for %%i in (*.csv) do (
  if !cnt!==1 (
    for /f "delims=" %%j in ('type "%%i"') do echo %%j &gt;&gt; combined.csv
  ) else if %%i NEQ combined.csv (
    for /f "skip=1 delims=" %%j in ('type "%%i"') do echo %%j &gt;&gt; combined.csv
  )
  set /a cnt+=1
)
</code></pre>