python update csv cells that meet criteria
<p>I am setting up a scheduled scrape of a car sales website. I have a csv with previously scraped ads in the following format (most columns omitted for neatness):</p>

<pre><code>id              year    first scan  last scan   active
OAG-AD-5459832  2005    31/01/2016  31/01/2016  Y
OAG-AD-12013782 2013    31/01/2016  31/01/2016  Y
OAG-AD-10487293 2015    31/01/2016  31/01/2016  Y
OAG-AD-12358252 1972    31/01/2016  31/01/2016  Y
AGC-AD-17247844 2015    31/01/2016  31/01/2016  Y
</code></pre>

<p>Each successive scrape currently returns a list of dicts, like so:</p>

<pre><code>[{'body': u'\xa0Coupe','id': u'SSE-AD-3469148', 'odo': u'288', 'adtype': u'Private Seller Car'},
 {'body': u'\xa0Coupe', 'id': u'OAG-AD-12014474', 'odo': u'0', 'adtype': u'Dealer: Demo',}]
</code></pre>

<p>I have three cases I need to work on:</p>

<ul>
<li>a) If the scraped ad is not in the csv, append it to the end. </li>
<li>b) If the
scraped ad is in the csv and active = 'Y', update 'last scan' to
today's date </li>
<li>c) For any active = 'Y' ads in the csv that aren't in the
list of scraped ads, change  active to 'N'</li>
</ul>

<p>My current code for a) is below. I am not sure how to approach this to complete b) and c) efficiently, without copying new rows in for existing ads. Also - I am using pandas to open the csv file, not sure if I should keep using it for the manipulation or work with dicts.</p>

<pre><code>date = datetime.datetime.now().strftime('%d/%m/%y') #last scan date     
addb = pd.read_csv('pd output.csv') #csv of previously scraped ads
ids = addb['id'].tolist() 
addbdict = addb.to_dict(orient='records')

newads = []
for ad in adscrape:
    ad['last scan'] = date
    ad['active'] = 'Y'
    if ad['id'] not in ids:
        ad['first scan'] = date
        newads.append(ad)

addb2 = addb.append(newads)           
addb2.to_csv('pd output2.csv', index=False)
</code></pre>