<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Examples of object type customization
Listing techniques per CPT measurement
Step1:  While performing CPT measurements, different techniques can be used. Since these can have an impact on the results, it can be interesting to download this additional information in order to better comprehend the CPT data.
Different CPT techniques can be applied at various depths, so the easiest way to add this to pydov is to use a new Sondering subtype Techniek, as shown below. The result will be that one can then choose to query CPT measurements and either retrieve a dataframe with the measurements themselves, or a dataframe with the techniques applied. The user can subsequently compare or merge the two dataframes at will.
Step2:  A new subtype has to be a subclass of the AbstractDovSubType class and implement two class variables
Step3:  In order to be able to use this subtype in a search query, we have to create a subclass of the original main type (Sondering) and register our new subtype
Step4:  The next step is to instantiate the SonderingSearch class with our newly created type
Step5:  If everything worked out, you should be able to see the new fields in the get_fields output
Step6:  Querying is exactly the same as with the default Sondering type
Step7:  One can use the values from the XSD type to add a human-readably column with the different techniques
Step8:  Adding location and height details to Boring dataframe
There is more to the location of a borehole than meets the eye! The default dataframe lists already multiple fields regarding the location of the borehole, both planimetric as altimetric
Step9:  When instantiating our BoringSearch object, we now explicitly set our new type as objecttype to search
Step10:  Searching for boreholes remains exactly the same, but will reveal the extra information in the output dataframe
Step11:  Listing GxG for GrondwaterFilters
For some of our Grondwaterfilters precalculated groundwaterlevel statistics (GxG) are available next to the individual measurements (peilmetingen) themselves. These statistics give information about the average high, low and medium groundwater levels at that location.
They can be obtained by defining a new subtype for the GrondwaterFilter type
Step12:  Adding color to the Borehole dataframe
The default dataframe of the Boring datatype contains information about the 'boormethode' as its subtype. However, in the DOV database there is information about the observed color of the soil in the various depth intervals available as well. We can access this data in pydov by defining a new subtype 'Kleur'
Step13:  List all glauconite fields for Grondmonsters
In the default Grondmonster dataframe, only the total glauconite is available. Using the following example, you can however request all different glauconite fields available in DOV
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
Explanation: Examples of object type customization
Listing techniques per CPT measurement
End of explanation
from pydov.types.fields import XmlField, XsdType
from pydov.types.abstract import AbstractDovSubType
from pydov.types.sondering import Sondering
Explanation: While performing CPT measurements, different techniques can be used. Since these can have an impact on the results, it can be interesting to download this additional information in order to better comprehend the CPT data.
Different CPT techniques can be applied at various depths, so the easiest way to add this to pydov is to use a new Sondering subtype Techniek, as shown below. The result will be that one can then choose to query CPT measurements and either retrieve a dataframe with the measurements themselves, or a dataframe with the techniques applied. The user can subsequently compare or merge the two dataframes at will.
End of explanation
class Techniek(AbstractDovSubType):
    
    rootpath = './/sondering/sondeonderzoek/penetratietest/technieken'
    fields = [
        XmlField(name='techniek_diepte_van',
                 source_xpath='/diepte_van',
                 definition='Enkel van toepassing voor het plaatsen van voerbuizen - '
                            '(code V) of het boren door een harde laag (code B).',
                 datatype='float'),
        XmlField(name='techniek_diepte',
                 source_xpath='/diepte_techniek',
                 definition='Diepte waarop techniek toegepast werd.',
                 datatype='float'),
        XmlField(name='techniek',
                 source_xpath='/techniek',
                 definition='De gebruikte techniek.',
                 datatype='string',
                 xsd_type=XsdType(
                     xsd_schema='https://www.dov.vlaanderen.be/xdov/schema/latest/xsd/kern/sondering/SonderingDataCodes.xsd',
                     typename='SondeerTechniekEnumType')),
        XmlField(name='techniek_andere',
                 source_xpath='/techniek_andere',
                 definition="De gebruikte techniek (enkel van toepassing indien de techniek = 'andere').",
                 datatype='string')
    ]
Explanation: A new subtype has to be a subclass of the AbstractDovSubType class and implement two class variables: rootpath and fields.
The rootpath is the XML XPath expression matching all instances of this subtype in the XML. One instance of this subtype will be created for each element matched by the rootpath XPath expression.
In the fields all the fields of this subtype are listed. These are instances of XmlField and should have at minimum a name, a source_xpath and a datatype. Additionally, a field should have a definition and can have a reference to an XSD schema type. The latter will be resolved and parsed at runtime, resulting in a list of values of this field.
End of explanation
class SonderingTechnieken(Sondering):
    subtypes = [Techniek]
Explanation: In order to be able to use this subtype in a search query, we have to create a subclass of the original main type (Sondering) and register our new subtype:
End of explanation
from pydov.search.sondering import SonderingSearch
cpts = SonderingSearch(objecttype=SonderingTechnieken)
Explanation: The next step is to instantiate the SonderingSearch class with our newly created type:
End of explanation
cpts.get_fields()['techniek']
Explanation: If everything worked out, you should be able to see the new fields in the get_fields output:
End of explanation
from pydov.util.location import WithinDistance, Point
df = cpts.search(location=WithinDistance(Point(150000, 150000), 10000, 'meter'))
Explanation: Querying is exactly the same as with the default Sondering type:
End of explanation
df['techniek_label'] = df['techniek'].map(cpts.get_fields()['techniek']['values'])
df.head()
Explanation: One can use the values from the XSD type to add a human-readably column with the different techniques:
End of explanation
from pydov.types.fields import XmlField, XsdType
from pydov.types.boring import Boring
class BoringMethodeXyz(Boring):
    
    __generiekeDataCodes = 'https://www.dov.vlaanderen.be/xdov/schema/latest/xsd/kern/generiek/GeneriekeDataCodes.xsd'
    
    fields = Boring.extend_fields([
        XmlField(name='methode_xy',
                 source_xpath='/boring/xy/methode_opmeten',
                 definition='Methode waarop de x en y-coordinaat opgemeten werden.',
                 datatype='string',
                 xsd_type=XsdType(
                     xsd_schema=__generiekeDataCodes,
                     typename='MethodeOpmetenXyEnumType')),
        XmlField(name='betrouwbaarheid_xy',
                 source_xpath='/boring/xy/betrouwbaarheid',
                 definition='Betrouwbaarheid van het opmeten van de x en y-coordinaat.',
                 datatype='string',
                 xsd_type=XsdType(
                     xsd_schema=__generiekeDataCodes,
                     typename='BetrouwbaarheidXyzEnumType')),
        XmlField(name='methode_mv',
                 source_xpath='/boring/oorspronkelijk_maaiveld/methode_opmeten',
                 definition='Methode waarop de Z-coördinaat van het maaiveld opgemeten werd.',
                 datatype='string',
                 xsd_type=XsdType(
                     xsd_schema=__generiekeDataCodes,
                     typename='MethodeOpmetenZEnumType')),
        XmlField(name='betrouwbaarheid_mv',
                 source_xpath='/boring/oorspronkelijk_maaiveld/betrouwbaarheid',
                 definition='Betrouwbaarheid van het opmeten van de z-coordinaat van het maaiveld.',
                 datatype='string',
                 xsd_type=XsdType(
                     xsd_schema=__generiekeDataCodes,
                     typename='BetrouwbaarheidXyzEnumType')),
        XmlField(name='aanvangspeil_mtaw',
                 source_xpath='/boring/aanvangspeil/waarde',
                 definition='Hoogte in mTAW van het startpunt van de boring (boortafel, bouwput etc).',
                 datatype='float'),
        XmlField(name='methode_aanvangspeil',
                 source_xpath='/boring/aanvangspeil/methode_opmeten',
                 definition='Methode waarop de Z-coördinaat van het aanvangspeil opgemeten werd.',
                 datatype='string',
                 xsd_type=XsdType(
                     xsd_schema=__generiekeDataCodes,
                     typename='MethodeOpmetenZEnumType')),
        XmlField(name='betrouwbaarheid_aanvangspeil',
                 source_xpath='/boring/aanvangspeil/betrouwbaarheid',
                 definition='Betrouwbaarheid van het opmeten van de z-coordinaat van het aanvangspeil.',
                 datatype='string',
                 xsd_type=XsdType(
                     xsd_schema=__generiekeDataCodes,
                     typename='MethodeOpmetenZEnumType')),
    ])
Explanation: Adding location and height details to Boring dataframe
There is more to the location of a borehole than meets the eye! The default dataframe lists already multiple fields regarding the location of the borehole, both planimetric as altimetric:
x and y are the planimetric coordinates of the borehole
start_boring_mtaw is the height of the start (aanvangspeil) of the borehole
mv_mtaw is the height of the ground level at the time of the making of the borehole
However, we have more information available regarding the (origin of) these coordinates. Each of them has an associated method (methode) and reliability (betrouwbaarheid).
We also make the distinction between the height of the ground level (maaiveld) and the height of the start of the borehole (aanvangspeil). If the borehole was started at ground level both are the same, but this is not necessarily the case. Furthermore the height of the start of the borehole can be either absolute (measured individually) or relative to the ground level.
If we want to have all this extra information available when retrieving the borehole dataframe output (or that of another DOV type), we can add the extra XML fields in a subclass of the Boring type:
End of explanation
from pydov.search.boring import BoringSearch
bs = BoringSearch(objecttype=BoringMethodeXyz)
bs.get_fields()['mv_mtaw']
Explanation: When instantiating our BoringSearch object, we now explicitly set our new type as objecttype to search:
End of explanation
from pydov.util.location import WithinDistance, Point
df = bs.search(location=WithinDistance(Point(150000, 150000), 10000, 'meter'),
               return_fields=('pkey_boring', 'boornummer', 'x', 'y', 'methode_xy', 'betrouwbaarheid_xy', 
                              'mv_mtaw', 'methode_mv', 'betrouwbaarheid_mv', 'aanvangspeil_mtaw', 
                              'methode_aanvangspeil', 'betrouwbaarheid_aanvangspeil', 'start_boring_mtaw'))
df.head()
df_plot = df.groupby('methode_xy').size().sort_values()
ax = df_plot.plot.barh()
ax.set_xlabel("count");
ax.set_title("Number of boreholes per coordinate digitization method");
df_plot = df.groupby('methode_mv').size().sort_values()
ax = df_plot.plot.barh()
ax.set_xlabel("count");
ax.set_title("Number of boreholes per height digitization method");
df_plot = df.groupby('methode_aanvangspeil').size().sort_values()
ax = df_plot.plot.barh()
ax.set_xlabel("count");
ax.set_title("Number of boreholes per `aanvangspeil` digitization method");
Explanation: Searching for boreholes remains exactly the same, but will reveal the extra information in the output dataframe:
End of explanation
from pydov.types.fields import XmlField, XsdType
from pydov.types.abstract import AbstractDovSubType
from pydov.types.grondwaterfilter import GrondwaterFilter
class Gxg(AbstractDovSubType):
    
    rootpath = './/filtermeting/gxg'
    fields = [
        XmlField(name='gxg_jaar',
                 source_xpath='/jaar',
                 definition='jaar (hydrologisch jaar voor lg3 en hg3, kalenderjaar voor vg3)',
                 datatype='integer'),
        XmlField(name='gxg_hg3',
                 source_xpath='/hg3',
                 definition='gemiddelde van de drie hoogste grondwaterstanden in een hydrologisch '
                            'jaar (1 april t/m 31 maart) bij een meetfrequentie van tweemaal per maand',
                 datatype='float'),
        XmlField(name='gxg_lg3',
                 source_xpath='/lg3',
                 definition='gemiddelde van de drie laagste grondwaterstanden in een hydrologisch jaar '
                            '(1 april t/m 31 maart) bij een meetfrequentie van tweemaal per maand',
                 datatype='float'),
        XmlField(name='gxg_vg3',
                 source_xpath='/vg3',
                 definition='gemiddelde van de grondwaterstanden op 14 maart, 28 maart en 14 april in '
                            'een bepaald kalenderjaar',
                 datatype='float')
    ]
class GrondwaterFilterGxg(GrondwaterFilter):
    subtypes = [Gxg]
from pydov.search.grondwaterfilter import GrondwaterFilterSearch
from owslib.fes import PropertyIsEqualTo
fs = GrondwaterFilterSearch(objecttype=GrondwaterFilterGxg)
fs.get_fields()['gxg_vg3']
df = fs.search(
    query=PropertyIsEqualTo('pkey_filter', 'https://www.dov.vlaanderen.be/data/filter/1999-009146')
)
df.head()
Explanation: Listing GxG for GrondwaterFilters
For some of our Grondwaterfilters precalculated groundwaterlevel statistics (GxG) are available next to the individual measurements (peilmetingen) themselves. These statistics give information about the average high, low and medium groundwater levels at that location.
They can be obtained by defining a new subtype for the GrondwaterFilter type:
End of explanation
from pydov.types.abstract import AbstractDovSubType
from pydov.types.boring import Boring
class Kleur(AbstractDovSubType):
    
    rootpath = './/boring/details/kleur'
    fields = [
        XmlField(name='diepte_kleur_van',
                 source_xpath='/van',
                 definition='Bovenkant van de laag met een bepaalde bekisting, in meter.',
                 datatype='float'),
        XmlField(name='diepte_kleur_tot',
                 source_xpath='/tot',
                 definition='Onderkant van de laag met een bepaalde bekisting, in meter.',
                 datatype='float'),
        XmlField(name='kleur',
                 source_xpath='/kleur',
                 definition='Grondkleur voor het diepte-interval',
                 datatype='string')
    ]
class BoringKleur(Boring):
    subtypes = [Kleur]
from pydov.search.boring import BoringSearch
from owslib.fes import PropertyIsEqualTo
bs = BoringSearch(objecttype=BoringKleur)
bs.get_fields()['kleur']
df = bs.search(
    query=PropertyIsEqualTo('pkey_boring', 'https://www.dov.vlaanderen.be/data/boring/2004-103984')
)
df.head()
Explanation: Adding color to the Borehole dataframe
The default dataframe of the Boring datatype contains information about the 'boormethode' as its subtype. However, in the DOV database there is information about the observed color of the soil in the various depth intervals available as well. We can access this data in pydov by defining a new subtype 'Kleur':
End of explanation
from pydov.types.fields import XmlField
from owslib.fes import PropertyIsEqualTo
from pydov.search.grondmonster import GrondmonsterSearch
from pydov.types.grondmonster import Grondmonster
class GlauconietWaarden(Grondmonster):
    fields = Grondmonster.extend_fields([
        XmlField(name='glauconiet_gt500',
                 source_xpath='/grondmonster/observatieData/observatie['
                              'parameter="GLAUCONIET_GT500"]/'
                              'waarde_numeriek',
                 definition='Glauconiet fractie groter 500 micron (%)',
                 datatype='float'),
        XmlField(name='glauconiet_tss',
                 source_xpath='/grondmonster/observatieData/observatie['
                              'parameter="GLAUCONIET_TSS"]/'
                              'waarde_numeriek',
                 definition='Glauconiet fractie kleiner 500micron en groter 63micron (%)',
                 datatype='float'),
        XmlField(name='glauconiet_kl63',
                 source_xpath='/grondmonster/observatieData/observatie['
                              'parameter="GLAUCONIET_KL63"]/'
                              'waarde_numeriek',
                 definition='Glauconiet fractie kleiner 63micron (%)',
                 datatype='float')
    ])
gm = GrondmonsterSearch(objecttype=GlauconietWaarden)
df = gm.search(
    query=PropertyIsEqualTo(
        propertyname='pkey_grondmonster', literal='https://www.dov.vlaanderen.be/data/grondmonster/2017-172111')
)
df.head()
Explanation: List all glauconite fields for Grondmonsters
In the default Grondmonster dataframe, only the total glauconite is available. Using the following example, you can however request all different glauconite fields available in DOV: 'glauconiet_totaal', 'glauconiet_KL63', 'glauconiet_TSS' and 'glauconiet_GR500'.
End of explanation
<END_TASK>