<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 QC Configuration
Objective
Step1:  load_cfg(), just for demonstration
Here we will import the load_cfg() function to illustrate different procedures. This is typically not necessary since ProfileQC does that for us. The cfgname argument for load_cfg is the same for ProfileQC, thus when we call
ProfileQC(dataset, cfgname='argo')
the procedure applied to dataset is the same shown by
load_cfg(cfgname='argo')
We will take advantage on that and simplify this notebook by inspecting only the configuration without actually applying it.
Step2:  Built-in tests
The easiest way to configure a QC procedure is by using one of the built-in tests, for example the GTSPP procedure for realtime data, here named 'gtspp_realtime'.
Step3:  The output cfg is a dictionary type of object, more specifically it is an ordered dictionary. The configuration has
Step4:  So, for GTSSP realtime assessement, all variables must be associated with a valid time and a valid location that is at sea.
Step5:  GTSPP evaluates temperature and salinity. Here we use CF standard names, so temperature is sea_water_temperature. But which tests are applied on temperature measurements?
Step6:  Let's inspect the spike test.
Step7:  There is one single item, the threshold, here defined as 2, so that any measured temperature with a spike greater than this threshold will fail on this spike test.
Let's check the global range test.
Step8:  Here there are two limit values, the minimum acceptable value and the maximum one. Anything beyond these limits will fail this test.
Check CoTeDe's manual to see what each test does and the possible parameters for each one.
Explicit inline
A QC procedure can also be explicitly defined with a dictionary. For instance, let's consider that we want to evaluate the temperature of a dataset with a single test, the spike test, using a threshold equal to one,
Step9:  Note that load_cfg took care for us to format it with the 0.21 standard, adding the revision and variables. If a revision is not defined, it is assumed a pre-0.21.
Compound procedure
Many of the recommended QC procedures share several tests in common. One way to simplify a QC procedure definition is by using inheritance to define a QC procedure to be used as a template. For example, let's create a new QC procedure that is based on GTSPP realtime and add a new test to that, the World Ocean Atlas Climatology comparison for temperature, with a threshold of 3 standard deviations.
Step10:  There is a new item, inherit
Step11:  And now sea_water_temperature has all the GTSPP realtime tests plus the WOA comparison,
Step12:  This new definition is actually the GTSPP recommended procedure for non-realtime data, i.e. the delayed mode. The built-in GTSPP procedure is actually written by inheriting the GTSPP realtime.
Step13:  The inheritance can also be used to modify any parameter from the parent template procedure. For example, let's use the GTSPP recommended procedure but with a more restricted threshold, equal to 1,
<ASSISTANT_TASK:>
Python Code:
# A different version of CoTeDe might give slightly different outputs.
# Please let me know if you see something that I should update.
import cotede
print("CoTeDe version: {}".format(cotede.__version__))
Explanation: QC Configuration
Objective:
Show different ways to configure a quality control (QC) procedure - explicit inline or calling a pre-set configuration.
For CoTeDe, the most important component is the human operator, hence it should be easy to control which tests to apply and the specific parameters of each test. CoTeDe is based on the principle of a single engine for multiple applications by using a dictionary to describe the QC procedure to be used, since 2011.
End of explanation
from cotede.utils import load_cfg
Explanation: load_cfg(), just for demonstration
Here we will import the load_cfg() function to illustrate different procedures. This is typically not necessary since ProfileQC does that for us. The cfgname argument for load_cfg is the same for ProfileQC, thus when we call
ProfileQC(dataset, cfgname='argo')
the procedure applied to dataset is the same shown by
load_cfg(cfgname='argo')
We will take advantage on that and simplify this notebook by inspecting only the configuration without actually applying it.
End of explanation
cfg = load_cfg('gtspp_realtime')
print(list(cfg.keys()))
Explanation: Built-in tests
The easiest way to configure a QC procedure is by using one of the built-in tests, for example the GTSPP procedure for realtime data, here named 'gtspp_realtime'.
End of explanation
cfg['revision']
print(list(cfg['common'].keys()))
Explanation: The output cfg is a dictionary type of object, more specifically it is an ordered dictionary. The configuration has:
A revision to help to determine how to handle this configuration.
A common item with the common tests for the whole dataset, i.e. the tests that are valid for all variables. For instance, a valid date and time is the same if we are evaluating temperature, salinity, or chlorophyll fluorescence.
A variables, with a list of the variables to evaluate.
Let's check each item:
End of explanation
print(list(cfg['variables'].keys()))
Explanation: So, for GTSSP realtime assessement, all variables must be associated with a valid time and a valid location that is at sea.
End of explanation
print(list(cfg['variables']['sea_water_temperature'].keys()))
Explanation: GTSPP evaluates temperature and salinity. Here we use CF standard names, so temperature is sea_water_temperature. But which tests are applied on temperature measurements?
End of explanation
print(cfg['variables']['sea_water_temperature']['spike'])
Explanation: Let's inspect the spike test.
End of explanation
print(list(cfg['variables']['sea_water_temperature']['global_range']))
Explanation: There is one single item, the threshold, here defined as 2, so that any measured temperature with a spike greater than this threshold will fail on this spike test.
Let's check the global range test.
End of explanation
my_config = {"sea_water_temperature":
               {"spike": {
                   "threshold": 1
                   }
               }
           }
cfg = load_cfg(my_config)
print(cfg)
Explanation: Here there are two limit values, the minimum acceptable value and the maximum one. Anything beyond these limits will fail this test.
Check CoTeDe's manual to see what each test does and the possible parameters for each one.
Explicit inline
A QC procedure can also be explicitly defined with a dictionary. For instance, let's consider that we want to evaluate the temperature of a dataset with a single test, the spike test, using a threshold equal to one,
End of explanation
my_config = {"inherit": "gtspp_realtime",
             "sea_water_temperature":
               {"woa_normbias": {
                   "threshold": 3
                   }
               }
           }
cfg = load_cfg(my_config)
print(cfg.keys())
Explanation: Note that load_cfg took care for us to format it with the 0.21 standard, adding the revision and variables. If a revision is not defined, it is assumed a pre-0.21.
Compound procedure
Many of the recommended QC procedures share several tests in common. One way to simplify a QC procedure definition is by using inheritance to define a QC procedure to be used as a template. For example, let's create a new QC procedure that is based on GTSPP realtime and add a new test to that, the World Ocean Atlas Climatology comparison for temperature, with a threshold of 3 standard deviations.
End of explanation
print(cfg['inherit'])
Explanation: There is a new item, inherit
End of explanation
print(cfg['variables']['sea_water_temperature'].keys())
Explanation: And now sea_water_temperature has all the GTSPP realtime tests plus the WOA comparison,
End of explanation
cfg = load_cfg('gtspp')
print(cfg['inherit'])
Explanation: This new definition is actually the GTSPP recommended procedure for non-realtime data, i.e. the delayed mode. The built-in GTSPP procedure is actually written by inheriting the GTSPP realtime.
End of explanation
my_config = {"inherit": "gtspp_realtime",
             "sea_water_temperature":
               {"spike": {
                   "threshold": 1
                   }
               }
           }
cfg = load_cfg(my_config)
print(cfg['variables']['sea_water_temperature']['spike'])
Explanation: The inheritance can also be used to modify any parameter from the parent template procedure. For example, let's use the GTSPP recommended procedure but with a more restricted threshold, equal to 1,
End of explanation
<END_TASK>