<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Notebook for Marking Extraction from Panopotes Classification CSV
Step1:  Read-in and examine content of "classifications" table.
Step2:  Read-in and examine content of "subjects" and "workflows" table.
Step3:  Expanding JSON Fields
These commands converts JSON strings into python dictionaries, providing access to key-value pairs.
Step4:  Understanding the fields of the annotations data
Step5:  Creating a new column based on classification data
Step6:  Helpful (optional) subselection of classification data
Step7:  Creating A New Output File
Step8:  Make Overlay Images
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import json
Explanation: Notebook for Marking Extraction from Panopotes Classification CSV
End of explanation
classfile_in = 'ap-aas229-test-classifications.csv'
classifications = pd.read_csv(classfile_in)
len(classifications)
classifications
Explanation: Read-in and examine content of "classifications" table.
End of explanation
subjectsfile_in = 'ap-aas229-test-subjects.csv'
subjects = pd.read_csv(subjectsfile_in)
workflowsfile_in = 'ap-aas229-test-workflows.csv'
workflows = pd.read_csv(workflowsfile_in)
subjects
workflows
Explanation: Read-in and examine content of "subjects" and "workflows" table.
End of explanation
classifications['metadata_json'] = [json.loads(q) for q in classifications.metadata]
classifications['annotations_json'] = [json.loads(q) for q in classifications.annotations]
classifications['subject_data_json'] = [json.loads(q) for q in classifications.subject_data]
Explanation: Expanding JSON Fields
These commands converts JSON strings into python dictionaries, providing access to key-value pairs.
End of explanation
classifications['annotations_json'][1][0]['value'][1]
classifications['annotations_json'][1][0]['value'][0]['x']
Explanation: Understanding the fields of the annotations data:
table | table column | classification index | task index | task column = value | marking index | marking column = data
End of explanation
classifications['n_markings'] = [ len(q[0]['value']) for q in classifications.annotations_json ]
Explanation: Creating a new column based on classification data:
End of explanation
iclass = classifications[classifications.workflow_version == classifications['workflow_version'].max()]
len(iclass)
Explanation: Helpful (optional) subselection of classification data: only work with data from most recent version of site & workflow.
End of explanation
# output markings from 31 classifications in iclass to csv
clist=[]
for index, c in iclass.iterrows():
    if c['n_markings'] > 0:
        for q in c.annotations_json[0]['value']:
            clist.append({'classification_id':c.classification_id, 'user_name':c.user_name, 'user_id':c.user_id,
                          'created_at':c.created_at, 'subject_ids':c.subject_ids, 'tool':q['tool'], 
                          'tool_label':q['tool_label'], 'x':q['x'], 'y':q['y'], 'r':q['r'], 'frame':q['frame']})
col_order=['classification_id','user_name','user_id','created_at','subject_ids',
           'tool','tool_label','x','y','r','frame']
out=pd.DataFrame(clist)[col_order]
out
out.to_csv('ap-aas229-test-markings.csv',index_label='mark_id')
Explanation: Creating A New Output File
End of explanation
import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
# Unique Subjects
usub = classifications.drop_duplicates(subset='subject_ids')
len(usub)
# Make Images
# Select Subject Index Number
subsel=12
marks = out[out.subject_ids == usub.subject_ids[subsel]]
imgnm = list(json.loads(usub.subject_data[subsel]).values())[0]['IMG_COLOR']
imgpath='../ap_sampledata/'
imgfile=imgpath+imgnm
fig, ax = plt.subplots()
im = plt.imread(imgfile)
ax.imshow(im)
for index, m in marks.iterrows():
    if m.tool == 0:
        ax.add_artist(plt.Circle( (m.x, m.y), m.r, color='b', fill=False))
    if m.tool == 1:
        ax.add_artist(plt.Circle( (m.x, m.y), m.r, color='r', fill=False))
    if m.tool > 1:
        ax.add_artist(plt.Circle( (m.x, m.y), m.r, color='g', fill=False))
plt.axis('off')
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)
plt.savefig('out_'+imgnm, dpi=120, bbox_inches='tight', pad_inches=0)
Explanation: Make Overlay Images
End of explanation
<END_TASK>