<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
Step1:  IOOS CSW queries
Step2:  GeoPortal uuid's for NGDC IOOS Collections
Step3:  Query 1
Step4:  Run CSW query
Step5:  Note that the COOPS SOS is not returned. This is probably due to an improper setup for the temporal extent in the COOPS SOS' ISO metadata record.
CSW query w/o the date filter and limited to the NANOOS, NOS/CO-OPS and NDBC collections
Just to double-check what's returned when the date filter is removed, but limited to those 3 collections so that a reasonable and relevant set of records is returned.
Step6:  Query 2
Step7:  NOTE for NANOOS Collection
Step8:  Query 3
Step9:  In keywords, apiso
Step10:  In apiso
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
from owslib.csw import CatalogueServiceWeb
from owslib import fes
from pprint import pprint
fmt = '{:*^64}'.format
def fes_date_filter(start, stop, constraint='overlaps'):
    
    Take datetime-like objects and returns a fes filter for date range
    (begin and end inclusive).
    NOTE: Truncates the minutes!!!
    Examples
    --------
    >>> from datetime import datetime, timedelta
    >>> stop = datetime(2010, 1, 1, 12, 30, 59).replace(tzinfo=pytz.utc)
    >>> start = stop - timedelta(days=7)
    >>> begin, end = fes_date_filter(start, stop, constraint='overlaps')
    >>> begin.literal, end.literal
    ('2010-01-01 12:00', '2009-12-25 12:00')
    >>> begin.propertyoperator, end.propertyoperator
    ('ogc:PropertyIsLessThanOrEqualTo', 'ogc:PropertyIsGreaterThanOrEqualTo')
    >>> begin, end = fes_date_filter(start, stop, constraint='within')
    >>> begin.literal, end.literal
    ('2009-12-25 12:00', '2010-01-01 12:00')
    >>> begin.propertyoperator, end.propertyoperator
    ('ogc:PropertyIsGreaterThanOrEqualTo', 'ogc:PropertyIsLessThanOrEqualTo')
    
    start = start.strftime('%Y-%m-%d %H:00')
    stop = stop.strftime('%Y-%m-%d %H:00')
    if constraint == 'overlaps':
        propertyname = 'apiso:TempExtent_begin'
        begin = fes.PropertyIsLessThanOrEqualTo(propertyname=propertyname,
                                                literal=stop)
        propertyname = 'apiso:TempExtent_end'
        end = fes.PropertyIsGreaterThanOrEqualTo(propertyname=propertyname,
                                                 literal=start)
    elif constraint == 'within':
        propertyname = 'apiso:TempExtent_begin'
        begin = fes.PropertyIsGreaterThanOrEqualTo(propertyname=propertyname,
                                                   literal=start)
        propertyname = 'apiso:TempExtent_end'
        end = fes.PropertyIsLessThanOrEqualTo(propertyname=propertyname,
                                              literal=stop)
    else:
        raise NameError('Unrecognized constraint {}'.format(constraint))
    return begin, end
def run_ngdc_csw(filter_list):
    csw = CatalogueServiceWeb('http://www.ngdc.noaa.gov/geoportal/csw',
                              timeout=60)
    csw.getrecords2(constraints=filter_list, maxrecords=1000, esn='full')
    
    print(fmt(' Catalog information '))
    print("CSW version: {}".format(csw.version))
    print("Number of datasets available: {}".format(len(csw.records.keys())))
    
    return csw
def service_urls(csw):
    df = []
    for key, rec in csw.records.items():
        df.append(pd.DataFrame(rec.references))
    df = pd.concat(df, ignore_index=True)
    df['scheme'] = [scheme.split(':')[-2] for scheme in df['scheme']]
    return df.set_index('scheme').sort_index().stack()
Explanation: IOOS CSW queries: NANOOS & OOI Endurance tests
12/30/2015. Emilio Mayorga, NANOOS. Much of this is adapted from notebooks from Filipe Fernandes.
End of explanation
# IOOS NGDC collection uuid's
# uuidscsv_url = 'https://raw.githubusercontent.com/ioos/registry/master/uuid.csv'
# The separator used is messy! It's ',\t' in most cases,
# except OceanSITES
# That's too messy! I've created my own, cleaned-up copy, for now:
uuidscsv_url = 'https://raw.githubusercontent.com/emiliom/stuff/master/ioos_ngdc_registry_uuid.csv'
uuids_df = pd.read_csv(uuidscsv_url)
uuids_df.set_index('MetadataCollection', inplace=True)
uuids = uuids_df.to_dict()['UUID']
uuids
Explanation: GeoPortal uuid's for NGDC IOOS Collections
End of explanation
bbox = [-127, 43, -123.75, 48]
from datetime import datetime, timedelta
dt = 5
now = datetime.utcnow()
start = now - timedelta(days=dt)
stop = now + timedelta(days=dt)
sos_name = 'sea_water_temperature'
name_list = ['sea_water_temperature',
             'sea_surface_temperature',
             'sea_water_potential_temperature',
             'equivalent_potential_temperature',
             'sea_water_conservative_temperature',
             'pseudo_equivalent_potential_temperature']
# Not includeing 'Water Temperature'
begin, end = fes_date_filter(start, stop)
kw = dict(wildCard='*',
          escapeChar='\\',
          singleChar='?',
          propertyname='apiso:AnyText')
variable_names_filt = fes.Or([fes.PropertyIsLike(literal=('*%s*' % val), **kw)
                              for val in name_list])
filter_list = [fes.And([begin, end, fes.BBox(bbox), variable_names_filt])]
Explanation: Query 1: OOI Endurance bbox, datetime range, and SST
Set up filters
OOI Endurance bbox, and "now" +/- 5 days, and 'sea_water_temperature'
End of explanation
csw = run_ngdc_csw(filter_list)
csw.records.keys()
Explanation: Run CSW query
End of explanation
collection_uuids = fes.Or(
    [fes.PropertyIsEqualTo(propertyname='dc:source', literal=uuids[collection])
     for collection in ['NANOOS', 'NOS/CO-OPS', 'NDBC']]
    )
filter_list = [fes.And([fes.BBox(bbox), variable_names_filt, collection_uuids])]
csw = run_ngdc_csw(filter_list)
csw.records.keys()
Explanation: Note that the COOPS SOS is not returned. This is probably due to an improper setup for the temporal extent in the COOPS SOS' ISO metadata record.
CSW query w/o the date filter and limited to the NANOOS, NOS/CO-OPS and NDBC collections
Just to double-check what's returned when the date filter is removed, but limited to those 3 collections so that a reasonable and relevant set of records is returned.
End of explanation
uuid_selection = ['NANOOS', 'NOS/CO-OPS']
for collection in uuid_selection:
    collection_uuid = fes.PropertyIsEqualTo(propertyname='dc:source',
                                            literal=uuids[collection])
    filter_list = [collection_uuid]
    
    print(fmt(' Collection: %s ' % collection))
    csw = run_ngdc_csw(filter_list)
    pprint(csw.records.keys())
    services = service_urls(csw)
    print('---------- OPeNDAP services:')
    pprint(set(services['odp'].values.tolist()))
    print('---------- SOS services:')
    pprint(set(services['sos'].values.tolist()))
Explanation: Query 2: IOOS NGDC Collection UUID's for NANOOS and NOS/CO-OPS
End of explanation
collection_uuid = fes.PropertyIsEqualTo(propertyname='dc:source',
                                        literal=uuids['NOS/CO-OPS'])
filter_list = [collection_uuid]
csw = run_ngdc_csw(filter_list)
print(csw.records['NOAA.NOS.CO-OPS SOS'].xml)
Explanation: NOTE for NANOOS Collection: {725A8A9B-1937-4DBF-ADF4-F6405543DC70} is the NANOOS GeoServer WMS
Examine the NOS/CO-OPS SOS record
End of explanation
kw = dict(wildCard='*', escapeChar='\\', singleChar='?')
Explanation: Query 3: NANOOS in keywords or any text
End of explanation
kw['propertyname'] = 'apiso:Subject'
filt_nanoos = fes.PropertyIsLike(literal=('*%s*' % 'NANOOS'), **kw)
filter_list = [filt_nanoos]
csw = run_ngdc_csw(filter_list)
csw.records.keys()
Explanation: In keywords, apiso:Subject
End of explanation
kw['propertyname'] = 'apiso:AnyText'
filt_nanoos = fes.PropertyIsLike(literal=('*%s*' % 'NANOOS'), **kw)
filter_list = [filt_nanoos]
csw = run_ngdc_csw(filter_list)
csw.records.keys()
Explanation: In apiso:AnyText
End of explanation
<END_TASK>