<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 UK MPs - Register of Interests - Quick Sketch
Couple of hours hack around register of interests data...
Get Data
Seems we can find some from http
Step1:  Simple entity extraction
Quick pass at trying to extract entities locally using simple natural language extractor.
This is not necessarily that spophisticated - but it's a start...
Step2:  We might then try to reconcile things classed as an ORG using something like OpenCorporates API.
Step3:  Third Party Taggers
Examples of using third party taggers.
Thomson Reuters OpenCalais
Step4:  Observations
The free text has items that can be parsed out - e.g. Name of donor
Step5:  Define a regular expression to pull out the data in structured form if the text conforms to a conventional format.
<ASSISTANT_TASK:>
Python Code:
url='http://downloads.membersinterests.org.uk/register/170707.zip'
!mkdir -p tmp/
!mkdir -p data/
!wget {url} -O tmp/temp.zip; unzip tmp/temp.zip -d data/ ; rm tmp/temp.zip
#Preview the data
!head -n  3 data/170707.csv  
#View data in datatable
import pandas as pd
df=pd.read_csv('data/170707.csv',header=None)
df.columns=['Name','Constituency','Party','URL','Item']
df.head()
Explanation: UK MPs - Register of Interests - Quick Sketch
Couple of hours hack around register of interests data...
Get Data
Seems we can find some from http://www.membersinterests.org.uk/.
End of explanation
#!pip3 install spacy
#!python3 -m spacy download en
from spacy.en import English
parser = English()
def entities(example, show=False):
    if show: print(example)
    parsedEx = parser(example)
    print("-------------- entities only ---------------")
    # if you just want the entities and nothing else, you can do access the parsed examples "ents" property like this:
    ents = list(parsedEx.ents)
    tags={}
    for entity in ents:
        #print(entity.label, entity.label_, ' '.join(t.orth_ for t in entity))
        term=' '.join(t.orth_ for t in entity)
        if ' '.join(term) not in tags:
            tags[term]=[(entity.label, entity.label_)]
        else:
            tags[term].append((entity.label, entity.label_))
    print(tags)
#Get a single register line item to play with
txt=df.iloc[0]['Item']
txt
entities(txt, True)
Explanation: Simple entity extraction
Quick pass at trying to extract entities locally using simple natural language extractor.
This is not necessarily that spophisticated - but it's a start...
End of explanation
import requests
ocrecURL='http://opencorporates.com/reconcile/gb'
rq=requests.get(ocrecURL,params={'query':'Guardian News & Media'})
rq.json()
Explanation: We might then try to reconcile things classed as an ORG using something like OpenCorporates API.
End of explanation
CALAIS_KEY=""
import requests
import json
def calais(text, calaisKey=CALAIS_KEY):
    calais_url = 'https://api.thomsonreuters.com/permid/calais'
    headers = {'X-AG-Access-Token' : calaisKey, 'Content-Type' : 'text/raw', 'outputformat' : 'application/json'}
    
    response = requests.post(calais_url, files={'file':text}, headers=headers, timeout=80)
    return response.json()
def cleaner(txt):
    txt=txt.replace('Address of','. Address of')
    return txt
oc=calais( cleaner(txt) )
def ocQuickView(oc):
    items={}
    for k in oc.keys():
        if '_typeGroup' in oc[k] and oc[k]['_typeGroup'] in ['entities','relations','socialTag','topics']:
            k2=oc[k]['_typeGroup']
            if k2 not in items: items[k2]=[]
            record={}
            #if '_type' in oc[k]:
            #    record['typ']=oc[k]['_type']
            if 'instances' in oc[k]:
                record['instances']=[i['exact'] for i in oc[k]['instances'] if 'exact' in i]
            for k3 in ['name','address','_type']:
                if k3 in oc[k]: record[k3] = oc[k][k3]
            items[k2].append(record)
    return items
ocQuickView(oc)
ix=155
txt=cleaner(df.iloc[ix]['Item'])
print('{}\n---\n{}'.format(txt, ocQuickView(calais(txt))))
ix=299
txt=cleaner(df.iloc[ix]['Item'])
print('{}\n---\n{}'.format(txt, ocQuickView(calais(txt))))
ix=863
txt=cleaner(df.iloc[ix]['Item'])
print('{}\n---\n{}'.format(txt, ocQuickView(calais(txt))))
Explanation: Third Party Taggers
Examples of using third party taggers.
Thomson Reuters OpenCalais
End of explanation
txt="Name of donor: Nael FarargyAddress of donor: privateAmount of donation or nature and value of donation in kind: Â£20,000 to hire a part time member of staff and meet office and staff expensesDate received: 12 April 2017Date accepted: 12 April 2017Donor status: individual(Registered 18 April 2017) "
txt
Explanation: Observations
The free text has items that can be parsed out - e.g. Name of donor:, Amount of donation or nature and value if donation in kind:, etc.
End of explanation
extractor1='Name of donor:(?P<name>.*)Address of donor:(?P<address>.*)Amount of donation or nature and value of donation in kind:(?P<amount>.*)Date received:(?P<rxd>.*)Date accepted:(?P<accptd>.*)Donor status(?P<status>.*)'
import re
r=re.compile(extractor1)
r.match(txt).groupdict()
#Looking at the response values, we could catch for whitespace in the regex or do a cleaning pass to strip whitespace
Explanation: Define a regular expression to pull out the data in structured form if the text conforms to a conventional format.
End of explanation
<END_TASK>