<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Go down for licence and other metadata about this presentation
\newpage
Preamble
Licence
Unless stated otherwise all content is released under a [CC0]+BY licence. I'd appreciate it if you reference this but it is not necessary.
\newpage
Using Ipython for presentations
A short video showing how to use Ipython for presentations
Step1:  The environment
In order to replicate my environment you need to know what I have installed!
Set up watermark
This describes the versions of software used during the creation. 
Please note that critical libraries can also be watermarked as follows
Step2:  Running dynamic presentations
You need to install the RISE Ipython Library from Damián Avila for dynamic presentations
To convert and run this as a static presentation run the following command
Step5:  To close this instances press control 'c' in the ipython notebook terminal console
Static presentations allow the presenter to see speakers notes (use the 's' key)
If running dynamically run the scripts below
Pre load some useful libraries
Step6:  \newpage
About me
Honorary Research Fellow, University of Nottingham
Step7:  A potted history of mapping
In the beginning was the geoword
and the word was cartography
\newpage
Cartography was king. 
Static representations of spatial knowledge with the cartographer deciding what to represent. 
Hence, maps are domain specific knowledge repositories for spatial data
\newpage
And then there was data .........
\newpage
Restrictive data
\newpage
Disconnected data with different
Step8:  \newpage
Here's something I prepared earlier
A live presentation (for those who weren't at the event).....
Step9:  \newpage
A more robust logic
Would need to decouple licence incompatibility from licence name into licence clause (see table below)
Deal with all licence type
Provide recommendations based on desired derivative licence type
Link this through to the type of process in a workflow
<ASSISTANT_TASK:>
Python Code:
from IPython.display import YouTubeVideo
YouTubeVideo('F4rFuIb1Ie4')
## PDF output using pandoc
import os
### Export this notebook as markdown
commandLineSyntax = 'ipython nbconvert --to markdown 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb'
print (commandLineSyntax)
os.system(commandLineSyntax)
### Export this notebook and the document header as PDF using Pandoc
commandLineSyntax = 'pandoc  -f markdown -t latex -N -V geometry:margin=1in DocumentHeader.md 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.md --filter pandoc-citeproc  --latex-engine=xelatex --toc -o interim.pdf '
os.system(commandLineSyntax)
### Remove cruft from the pdf
commandLineSyntax = 'pdftk interim.pdf cat 1-3 16-end output 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.pdf'
os.system(commandLineSyntax)
### Remove the interim pdf
commandLineSyntax = 'rm interim.pdf'
os.system(commandLineSyntax)
Explanation: Go down for licence and other metadata about this presentation
\newpage
Preamble
Licence
Unless stated otherwise all content is released under a [CC0]+BY licence. I'd appreciate it if you reference this but it is not necessary.
\newpage
Using Ipython for presentations
A short video showing how to use Ipython for presentations
End of explanation
%install_ext https://raw.githubusercontent.com/rasbt/python_reference/master/ipython_magic/watermark.py
%load_ext watermark
%watermark -a "Anthony Beck"  -d -v -m -g
#List of installed conda packages
!conda list
#List of installed pip packages
!pip list
Explanation: The environment
In order to replicate my environment you need to know what I have installed!
Set up watermark
This describes the versions of software used during the creation. 
Please note that critical libraries can also be watermarked as follows:
python
%watermark -v -m -p numpy,scipy
End of explanation
# I can't see notes in this - Not sure what the issue is. Try this in a solely python 2 environment
#PRIGINAL !ipython nbconvert 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb --to slides --post serve
!ipython nbconvert 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb --to slides  --reveal-prefix https://cdn.jsdelivr.net/reveal.js/2.6.2 --post serve
#ipython nbconvert --to slides Analysis-scheme1.ipynb --reveal-prefix https://cdn.jsdelivr.net/reveal.js/2.6.2
!ipython nbconvert 2016_Nottingham_GIServices_Lecture3_Beck_InteroperabilitySemanticsAndOpenData.ipynb --to slides  --reveal-prefix https://cdn.jsdelivr.net/reveal.js/2.6.2
Explanation: Running dynamic presentations
You need to install the RISE Ipython Library from Damián Avila for dynamic presentations
To convert and run this as a static presentation run the following command:
End of explanation
#Future proof python 2
from __future__ import print_function #For python3 print syntax
from __future__ import division
# def
import IPython.core.display
# A function to collect user input - ipynb_input(varname='username', prompt='What is your username')
def ipynb_input(varname, prompt=''):
        Prompt user for input and assign string val to given variable name.
        js_code = (
            var value = prompt("{prompt}","");
            var py_code = "{varname} = '" + value + "'";
            IPython.notebook.kernel.execute(py_code);
        ).format(prompt=prompt, varname=varname)
        return IPython.core.display.Javascript(js_code)
    
# inline
%pylab inline
Explanation: To close this instances press control 'c' in the ipython notebook terminal console
Static presentations allow the presenter to see speakers notes (use the 's' key)
If running dynamically run the scripts below
Pre load some useful libraries
End of explanation
# THINGS TO DISCUSS
## Data needs to be credible, accessible, properly licenceed. Bring in Internet of Things
Explanation: \newpage
About me
Honorary Research Fellow, University of Nottingham: orcid
Director, Geolytics Limited - A spatial data analytics consultancy
About this presentation
Available on GitHub - https://github.com/AntArch/Presentations_Github/
Fully referenced PDF
\newpage
Contribution to GIScience learning outcomes
This presentation contributes to the following learning outcomes for this course.
Knowledge and Understanding:
Appreciate the importance of standards for Geographic Information and the role of the Open Geospatial Consortium.
Understand the term 'interoperability'.
Appreciate the different models for database design.
Understand the basis of Linked Data.
Find UK government open data and understand some of the complexities in the use of this data.
Appreciate the data issues involved in managing large distributed databases, Location-Based Services and the emergence of real-time data gathering through the 'Sensor-Web'.
Understand the different models for creating international Spatial Data Infrastructures.
Intellectual Skills:
Evaluate the role of standards and professional bodies in GIS.
Articulate the meaning and importance of interoperability, semantics and ontologies.
Assess the technical and organisational issues which come into play when attempting to design large distributed geographic databases aimed at supporting 'real-world' problems.
Suggested Questions:
What do you think are the challenges for National Mapping Agencies and major Volunteer Geographic Data initiatives (OSM) are over the next 5 to 10 years?
This questions aims to get the student to think about how technology, policy and practice frameworks will impact upon the GI landscape. Will, for example, OSM make the Ordnance Survey irrelevant? What would then happen to organisations like the Land Registry that require credible data products to support their activites. How will things like the semantic web impact on GI. Can the aspirations of Open Data be realised and bring a wealth of nwe products and services. What impact will the Internet of Things have on GI. 
Describe the issues surrounding interoperability. How have the OGC, OSGeo and INSPIRE helped and what challenges remain.
This should be a simple description of data silos, technical interoperability (syntactic, semantic, schematic), standards, standards bodies and inspire. Full dscussions will consider non-technical interoperability. 
Is cartography relevant to modern GI? Discuss this in relation to map representations, the range of available open data and the needs of different users.
This is a somewhat open ended question with the purpose of getting the student to consider the differences between mapping as a broadly static data representation and GI as a dynamic data rich discipline. Obviously cartography or visualization is important - this is where issues like clutter, colour, visualization, symbology, generalisation can all be discussed. The open data point is to get the students to consider the range of available geo and attribute data and to think of GI as a spatial data stack. This should lead into the student being able to discuss generic products (providing similar service to the masses (i.e. maps))  and bespoke services (providing specific services to niche communities) which both use the same underlying data sources. Good answers will bring in some of the non-technical interoperability issues - particularly licencing. 
SORT OUT DROPBOX and then let Ana know my details.
End of explanation
from IPython.display import YouTubeVideo
YouTubeVideo('jUzGF401vLc')
Explanation: A potted history of mapping
In the beginning was the geoword
and the word was cartography
\newpage
Cartography was king. 
Static representations of spatial knowledge with the cartographer deciding what to represent. 
Hence, maps are domain specific knowledge repositories for spatial data
\newpage
And then there was data .........
\newpage
Restrictive data
\newpage
Disconnected data with different:
Standards
Quality
Databases
Semantics
\newpage
Why is this an issue: INSPIRE
describe inspire
\newpage
Making data interoperable and open
\newpage
Interoperability
Put in interoperability quote here (plus references)
Defense interoperability
As defined by DoD policy, interoperability is the ability of systems, units, or forces to provide data, information, material, and services to, and accept the same from, other systems, units, or forces; and to use the data, information, materiel, and services so exchanged to enable them to operate effectively together. IT and NSS interoperability includes both the technical exchange of information and the end-to-end operational effectiveness of that exchanged information as required for mission accomplishment. Interoperability is more than just information exchange; it includes systems, processes, procedures, organizations, and missions over the life cycle and must be balanced with information assurance.
\newpage
Technical interoperability - levelling the field
\newpage
Discuss each interoperability and its resolution.
Bring in:
OGC  - standards body (remember the standards must be open (read data/write data)
GML (interoperable foramt) - interchange
\newpage
What did technical interoperability facilitate
From Map to Model The changing paradigm of map creation from cartography to data driven visualization
\newpage
\newpage
\newpage
\newpage
The world was a happy place.......
Our data was interoperable!
Our data became open data
\newpage
Background - open..... nesss
in communities
in government
in academia
\newpage
\newpage
Open..... nesss -- in government
The Shakespeare review [-@shakespeare_shakespeare_2013] indicate that the amount of government Open Data, at least in the UK, is only going to grow.
Open data has the potential to trigger a revolution in how governments think about providing services to citizens and how they measure their success: this produces societal impact.
This will require an understanding of citizen needs, behaviours, and mental models, and how to use data to improve services.
\newpage
A McKinsey Global Institute report examines the economic impact of Open Data [@mckinsey_open_2013] and estimates that globally open data could be worth a minimum of $3 trillion annually. 
\newpage
Background - open..... nesss
in academia
Open inquiry is at the heart of the scientific enterprise..... Science’s powerful capacity for self-correction comes from this openness to scrutiny and challenge.
Science as an open enterprise [@royal_society_science_2012 p. 7].
The Royal Society’s report Science as an open enterprise [-@royal_society_science_2012] identifies how 21^st^ century communication technologies are changing the ways in which scientists conduct, and society engages with, science. The report recognises that ‘open’ enquiry is pivotal for the success of science, both in research and in society.
This goes beyond open access to publications (Open Access), to include access to data and other research outputs (Open Data), and the process by which data is turned into knowledge (Open Science).
The next generation open data in academia
Zenodo is a DATA REPOSITORY which offers:
* accreditation 
* different licences 
* different exposure (private (closed), public (open) and embargoed (timestamped)) 
* DOIs 
* is free at the point of use
* is likely to be around for a long time 
    * supported by Horizon 2020 and delivered by CREN
\newpage
The underlying rationale of Open Data is:
unfettered access to large amounts of ‘raw’ data 
enables patterns of re-use and knowledge creation that were previously impossible. 
improves transparency and efficiency
encourages innovative service delivery
introduces a range of data-mining and visualisation challenges, 
which require multi-disciplinary collaboration across domains
catalyst to research and industry
supports the generation of new products, services and markets
the prize for succeeding is improved knowledge-led policy and practice that transforms 
communities, 
practitioners, 
science and 
society
the Open Data Landscape
Formal and informal data.OS versus VGI (Open Street Map)
Conclude with Linked Data + Open Data = Linked Open Data
So where are these new data products?
Data, data everywhere - but where are the new derivatives and services?
\newpage
Non-technical interoperability issues?
Issues surrounding non-technical interoperability include:
Policy interoperabilty
Licence interoperability
Legal interoperability
Social interoperability
We will focus on licence interoperability
\newpage
Policy Interoperability
Access control
Are you allowed  to use data from this country?
From international journal of digital curation
Social Interoperability
Social interoperability is concerned about the environment and the human processes involved in the information exchange. 
see here
It's a bit wooley
Legal Interoperability
{The purpose of the Case Studies is to:}(https://rd-alliance.org/groups/rdacodata-legal-interoperability-ig.html)
Provide more specific information on best practices, as well as barriers and constraints in a number of different scientific domains and communities of practice (lessons learned); 
Illustrate the variety of legal frameworks that govern research data, different approaches to intellectual property and copyright across jurisdictions, different disciplinary expectations and norms, and alternative mechanisms to address the legal interoperability of data that have been tried in practice; and
Identify opportunities for cross-disciplinary, cross-domain fertilization and collaboration and for new initiatives to address key barriers and constraints.
Also look here
Chatham house paper
Licence Interoperability
A specific form of legal interoperability
Example of applying the semantic web to licence interoperability
There is a multitude of formal and informal data. 
\newpage
What is a licence?
Wikipedia state:
A license may be granted by a party ("licensor") to another party ("licensee") as an element of an agreement between those parties. 
A shorthand definition of a license is "an authorization (by the licensor) to use the licensed material (by the licensee)."
Each of these data objects can be licenced in a different way. This shows some of the licences described by the RDFLicence ontology
\newpage
Concepts (derived from Formal Concept Analysis) surrounding licences
\newpage
Two lead organisations have developed legal frameworks for content licensing:
Creative Commons (CC) and 
Open Data Commons (ODC). 
Until the release of CC version 4, published in November 2013, the CC licence did not cover data. Between them, CC and ODC licences can cover all forms of digital work.
There are many other licence types
Many are bespoke
Bespoke licences are difficult to manage
Many legacy datasets have bespoke licences
I'll describe CC in more detail
\newpage
Creative Commons Zero
Creative Commons Zero (CC0) is essentially public domain which allows:
Reproduction
Distribution
Derivations
Constraints on CC0
The following clauses constrain CC0:
Permissions
ND – No derivatives: the licensee can not derive new content from the resource.
Requirements
BY – By attribution: the licensee must attribute the source.
SA – Share-alike: if the licensee adapts the resource, it must be released under the same licence.
Prohibitions
NC – Non commercial: the licensee must not use the work commercially without prior approval.
CC license combinations
License|Reproduction|Distribution|Derivation|BY|SA|NC
----|----|----|----|----|----|----
CC0|X|X|X|||
CC-BY-ND|X|X||X||
CC-BY-NC-ND|X|X||X||X
CC-BY|X|X|X|X||
CC-BY-SA|X|X|X|X|X|
CC-BY-NC|X|X|X|X||X
CC-BY-NC-SA|X|X|X|X|X|X
Table: Creative Commons license combinations
\newpage
Why are licenses important?
They tell you what you can and can't do with 'stuff'
Very significant when multiple datasets are combined
It then becomes an issue of license compatibility
\newpage
Which is important when we mash up data
Certain licences when combined:
Are incompatible
Creating data islands
Inhibit commercial exploitation (NC)
Force the adoption of certain licences
If you want people to commercially exploit your stuff don't incorporate CC-BY-NC-SA data!
Stops the derivation of new works
A conceptual licence processing workflow. The licence processing service analyses the incoming licence metadata and determines if the data can be legally integrated and any resulting licence implications for the derived product.
\newpage
A rudimentry logic example
```
Data1 hasDerivedContentIn NewThing.
Data1 hasLicence a cc-by-sa.
What hasLicence a cc-by-sa? #reason here
If X hasDerivedContentIn Y and hasLicence Z then Y hasLicence Z. #reason here
Data2 hasDerivedContentIn NewThing.
Data2 hasLicence a cc-by-nc-sa.
What hasLicence a cc-by-nc-sa? #reason here
Nothing hasLicence a cc-by-nc-sa and hasLicence a cc-by-sa. #reason here
```
And processing this within the Protege reasoning environment
End of explanation
from IPython.display import YouTubeVideo
YouTubeVideo('tkRB5Rp1_W4')
Explanation: \newpage
Here's something I prepared earlier
A live presentation (for those who weren't at the event).....
End of explanation
and more
```
If X spatiallyContains Y and X is a city then Y is a place and Y is a cityPart.
Every city is a place.
What is a place.
```
and more
```
UK isPartOf Europe.
UK is sameAs UnitedKingdom.
If X has a port then X borders Water.
What borders Water?
```
Explanation: \newpage
A more robust logic
Would need to decouple licence incompatibility from licence name into licence clause (see table below)
Deal with all licence type
Provide recommendations based on desired derivative licence type
Link this through to the type of process in a workflow:
data derivation is, from a licence position, very different to contextual display
License|Reproduction|Distribution|Derivation|BY|SA|NC
----|----|----|----|----|----|----
CC0|X|X|X|||
CC-BY-ND|X|X||X||
CC-BY-NC-ND|X|X||X||X
CC-BY|X|X|X|X||
CC-BY-SA|X|X|X|X|X|
CC-BY-NC|X|X|X|X||X
CC-BY-NC-SA|X|X|X|X|X|X
ODC-PDDL|X|X|X|||
ODC-BY|X|X|X|X||
ODC-ODbL|X|X|X|X|X|
OGL 2.0|X|X|X|X||
OS OpenData|X|X|X|X|?|
Table: Creative Commons license combinations
\newpage
OGC and Licence interoperability
The geo business landscape is increasingly based on integrating heterogeneous data to develop new products
Licence heterogeneity is a barrier to data integration and interoperability
A licence calculus can help resolve and identify heterogenties leading to
legal compliance
confidence
Use of standards and collaboration with organisations is crucial
Open Data Licensing ontology
The Open Data Institute
Failure to do this could lead to breaches in data licenses
and we all know where that puts us........
\newpage
Ontologies, semantics and linked data
We've already been introduced to ontologies in the licence calculus example. 
SLides to describe ontologies, semantics and linked data
Inlcude geo-logic from Brandon
Geo example:
```
Leeds is a city.
Yorkshire is a county.
Sheffield is a city.
Lancaster is a city.
Lancashire is a county.
Lancaster has a port.
What is Leeds?
Leeds isIn Yorkshire.
Sheffield isIn Yorkshire.
Lancaster isIn Lancashire.
What isIn Yorkshire?
If X isIn Y then Y contains X.
What contains Leeds?
Yorkshire borders Lancashire.
If X borders Y then Y borders X.
What borders Lancashire?
Yorkshire isIn UnitedKingdom.
Lancashire isIn UnitedKingdom.
Transitivity
If X isIn Y and Y isIn Z then X isIn Z.
If X contains Y and Y contains Z then X contains Z
```
using proper isIn
```
Leeds is a city.
Yorkshire is a county.
Sheffield is a city.
Lancaster is a city.
Lancashire is a county.
Lancaster has a port.
What is Leeds?
Leeds is spatiallyWithin Yorkshire.
Sheffield is spatiallyWithin Yorkshire.
Lancaster is spatiallyWithin Lancashire.
What is spatiallyWithin Yorkshire?
If X is spatiallyWithin Y then Y spatiallyContains X.
What spatiallyContains Leeds?
Yorkshire borders Lancashire.
If X borders Y then Y borders X.
What borders Lancashire?
Yorkshire is spatiallyWithin UnitedKingdom.
Lancashire is spatiallyWithin UnitedKingdom.
Transitivity
If X is spatiallyWithin Y and Y is spatiallyWithin Z then X is spatiallyWithin Z.
If X spatiallyContains Y and Y spatiallyContains Z then X spatiallyContains Z
What is spatiallyWithin UnitedKingdom?
```
Adding more......
```
Pudsey is spatiallyWithin Leeds.
Kirkstall is spatiallyWithin Leeds.
Meanwood is spatiallyWithin Leeds.
Roundhay is spatiallyWithin Leeds.
Scarcroft is spatiallyWithin Leeds.
```
and more
```
UnitedKingdom isPartOf Europe.
UnitedKingdom is a country.
If X isPartOf Y and X spatiallyContains Z then Z isPartOf Y.
What isPartOf Europe?
```
End of explanation
<END_TASK>