<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Processing Test
Consolidating the returned CSVs into one is relatively painless
Main issue is that for some reason the time is still in GMT, and needs 5 hours in milliseconds subtracted from the epoch
Validating against Weather Underground read from O'Hare
Step1:  NEXRAD at O'Hare Zip 60666
Step2:  Wunderground
<ASSISTANT_TASK:>
Python Code:
s3_client = boto3.client('s3')
resource = boto3.resource('s3')
# Disable signing for anonymous requests to public bucket
resource.meta.client.meta.events.register('choose-signer.s3.*', disable_signing)
def file_list(client, bucket, prefix=''):
    paginator = client.get_paginator('list_objects')
    for result in client.list_objects(Bucket=bucket, Prefix=prefix, Delimiter='/')['Contents']:
        yield result.get('Key')
gen_s3_files = list(file_list(s3_client, 'nexrad-etl', prefix='test-aug3/'))
for i, f in enumerate(gen_s3_files):
   s3_client.download_file('nexrad-etl',f,'test-aug3/nexrad{}.csv'.format(i))
folder_files = os.listdir(os.path.join(os.getcwd(), 'test-aug3'))
nexrad_df_list = list()
for f in folder_files:
    if f.endswith('.csv'):
        try:
            nexrad_df_list.append(pd.read_csv('test-aug3/{}'.format(f)))
        except:
            #print(f)
            pass
print(len(nexrad_df_list))
merged_nexrad = pd.concat(nexrad_df_list)
merged_nexrad['timestamp'] = pd.to_datetime(((merged_nexrad['timestamp'] / 1000) - (5*3600*1000)), unit='ms')
#merged_nexrad['timestamp'] = pd.to_datetime(merged_nexrad['timestamp'] / 1000, unit='ms')
merged_nexrad = merged_nexrad.set_index(pd.DatetimeIndex(merged_nexrad['timestamp']))
merged_nexrad = merged_nexrad.sort_values('timestamp')
merged_nexrad = merged_nexrad.fillna(0.0)
# Get diff between previous two reads
merged_nexrad['diff'] = merged_nexrad['timestamp'].diff()
merged_nexrad = merged_nexrad[1:]
print(merged_nexrad.shape)
merged_nexrad.index.min()
merged_nexrad['diff'] = (merged_nexrad['diff'] / np.timedelta64(1, 'm')).astype(float) / 60
merged_nexrad.head()
aug_day_ohare = merged_nexrad['2016-08-12'][['timestamp','60666','diff']]
aug_day_ohare.head()
aug_day_ohare['60666'] = (aug_day_ohare['60666']*aug_day_ohare['diff'])/25.4
aug_day_ohare.head()
Explanation: Processing Test
Consolidating the returned CSVs into one is relatively painless
Main issue is that for some reason the time is still in GMT, and needs 5 hours in milliseconds subtracted from the epoch
Validating against Weather Underground read from O'Hare
End of explanation
# Checking against Weather Underground read for O'Hare on this day
print(aug_day_ohare['60666'].sum())
aug_day_ohare['60666'].plot()
Explanation: NEXRAD at O'Hare Zip 60666
End of explanation
wunderground = pd.read_csv('test-aug3/aug-12.csv')
wunderground['PrecipitationIn'] = wunderground['PrecipitationIn'].fillna(0.0)
wunderground['TimeCDT'] = pd.to_datetime(wunderground['TimeCDT'])
wunderground = wunderground.set_index(pd.DatetimeIndex(wunderground['TimeCDT']))
wund_hour = wunderground['PrecipitationIn'].resample('1H').max()
print(wund_hour.sum())
wund_hour.plot()
Explanation: Wunderground
End of explanation
<END_TASK>