<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Observability facilities
Third party software stack to keep an eye on your experiment or gather some metrics.
Website
Step1:  A simple load generator
We'll install a simple load generator
Step2:  Monitoring with dstat
Dstat is a simple monitoring tool
Step3:  Visualization
All the CSVs files are available under the backup_dir inside subdirectories named after the corresponding remote host alias
Step4:  Packet sniffing with tcpdump
Capture
Step5:  Visualization
Step6:  Capture on a specific network
You can start a capture on a dedicated network by specifying it to TCPDump
This will sniff all the packet that go through an interface configured in this specific network
You need to call sync_info first to enable the translation (network logical name)->interface name
Step7:  Monitoring with Telegraf/[InfluxDB|prometheus]/grafana
Step8:  To not forget to close the tunnel you can use a context manager
Step9:  Cleaning
<ASSISTANT_TASK:>
Python Code:
import enoslib as en
# Enable rich logging
_ = en.init_logging()
# claim the resources
network = en.G5kNetworkConf(type="prod", roles=["my_network"], site="rennes")
conf = (
    en.G5kConf.from_settings(job_type="allow_classic_ssh", job_name="enoslib_observability")
    .add_network_conf(network)
    .add_machine(
        roles=["control", "xp"], cluster="parasilo", nodes=1, primary_network=network
    )
    .add_machine(
        roles=["agent", "xp"], cluster="parasilo", nodes=1, primary_network=network
    )
    .finalize()
)
conf
provider = en.G5k(conf)
roles, networks = provider.init()
roles
Explanation: Observability facilities
Third party software stack to keep an eye on your experiment or gather some metrics.
Website: https://discovery.gitlabpages.inria.fr/enoslib/index.html
Instant chat: https://framateam.org/enoslib
Source code: https://gitlab.inria.fr/discovery/enoslib
More generally, useful service for the experimentation are consolidated through Services.
In particular, EnOSlib has some some Services which deal with the problem of getting some insight on what's running on remote nodes. 
A Service is a python object which exposes three main methods:
deploy: which deploy the service
destroy: remove stop the service
backup: retrieve some states of the services (e.g monitoring information)
Usually a service is used as follow:
```python
service = Service(args, *kwargs)
service.deploy()
...
do stuffs
...
service.backup()
service.destroy()
```
But it's sometime useful to use a Context Manager when working with module:
python
with Service(*args, **kwargs) as service:
    ...
    # do stuffs
    ...
This allows for
running the service for some time depending on what's inside the context manager
cleaning (and backuping) stuffs automatically at the end
Prerequisites:
- A Grid'5000 account
- A working EnOSlib environment and Jupyter (not included in EnOSlib dependencies, but pip install jupyterlab will install it)
- Some visualization tools: matplotlib, seaborn
- Packet manipulation tool: scapy
Common setup
End of explanation
with en.actions(roles=roles["agent"]) as a:
    a.apt(name="stress", state="present")
Explanation: A simple load generator
We'll install a simple load generator: stress available in the debian packages.
End of explanation
# Start a capture on all nodes
# - stress on some nodes
import time
with en.Dstat(nodes=roles["xp"]) as d:
    time.sleep(5)
    en.run_command("stress --cpu 4 --timeout 10", roles=roles["agent"])
    time.sleep(5)
    backup_dir = d.backup_dir
Explanation: Monitoring with dstat
Dstat is a simple monitoring tool: https://github.com/dstat-real/dstat#information
It runs as a single process and collect metrics from various sources. 
That makes it a good candidate for getting a quick insight on the resources consumptions during an experiment.
The EnOSlib implementation lets you easily 
- start Dstat processes on remote machine and start dumping the metrics into a csv file( it's the purpose deploy method of the Dstat service)
- retrieve all the csvs file (one per remote node) on your local machine ( that's the purpose of the backup method)
- stop every remote Dstat processes (that's the purpose of the destroy method)
Capture
Let's start with a single capture implemented using a context manager.
The context manager runs deploy when entering, and backup/destroy when exiting.
End of explanation
import pandas as pd
import seaborn as sns 
print(backup_dir)
#Create a dictionnary of (alias) -> list of pandas df
result = pd.DataFrame()
for host in roles["xp"]:
    host_dir = backup_dir / host.alias
    csvs = host_dir.rglob("*.csv")
    for csv in csvs:
        print(csv)
        df = pd.read_csv(csv, skiprows=5, index_col=False)
        df["host"] = host.alias
        df["csv"] = csv
        result = pd.concat([result, df], axis=0, ignore_index=True)
result
# let's show the metrics !
sns.lineplot(data=result, x="epoch", y="usr", hue="host", markers=True, style="host")
Explanation: Visualization
All the CSVs files are available under the backup_dir inside subdirectories named after the corresponding remote host alias:
bash
&lt;backup_sir&gt; / host1 / ... / &lt;metrics&gt;.csv
             / host2 / ..../ &lt;metrics&gt;.csv
The following bunch of python lines will recursively look for any csv file inside these directories and build a DataFrame and a visualization
End of explanation
# start a capture
# - on all the interface configured on the my_network network
# - we dump icmp traffic only
# - for the duration of the commands (here a client is pigging the server)
with en.TCPDump(
    hosts=roles["xp"], ifnames=["any"], options="icmp"
) as t:
    backup_dir = t.backup_dir
    _ = en.run(f"ping -c10 {roles['control'][0].address}", roles["agent"])
Explanation: Packet sniffing with tcpdump
Capture
End of explanation
from scapy.all import rdpcap
import tarfile
# Examples:
# create a dictionnary of (alias, if) -> list of decoded packets by scapy
decoded_pcaps = dict()
for host in roles["control"]:
    host_dir = backup_dir / host.alias
    t = tarfile.open(host_dir / "tcpdump.tar.gz")
    t.extractall(host_dir / "extracted")
    # get all extracted pcap for this host
    pcaps = (host_dir / "extracted").rglob("*.pcap")
    for pcap in pcaps:
        decoded_pcaps.setdefault((host.alias, pcap.with_suffix("").name),
                                 rdpcap(str(pcap)))
# Displaying some packets
for (host, ifs), packets in decoded_pcaps.items():
    print(host, ifs)
    packets[0].show()
    packets[1].show()
Explanation: Visualization
End of explanation
roles = en.sync_info(roles, networks)
# start a capture
# - on all the interface configured on the my_network network
# - we dump icmp traffic only
# - for the duration of the commands (here a client is pigging the server)
with en.TCPDump(
    hosts=roles["xp"], networks=networks["my_network"], options="icmp"
) as t:
    backup_dir = t.backup_dir
    _ = en.run(f"ping -c10 {roles['control'][0].address}", roles["agent"])
Explanation: Capture on a specific network
You can start a capture on a dedicated network by specifying it to TCPDump
This will sniff all the packet that go through an interface configured in this specific network
You need to call sync_info first to enable the translation (network logical name)->interface name
End of explanation
monitoring = en.TIGMonitoring(collector=roles["control"][0], agent=roles["agent"], ui=roles["control"][0])
monitoring
monitoring.deploy()
en.run_command("stress --cpu 4 --timeout 60", roles=roles["agent"], background=True)
# create a tunnel to the service running inside g5k
tunnel = en.G5kTunnel(address=monitoring.ui.address, port=3000)
local_address, local_port, _ = tunnel.start()
print(f"The service is running at http://localhost:{local_port} (admin:admin)")
# don't forget to close it
tunnel.close()
Explanation: Monitoring with Telegraf/[InfluxDB|prometheus]/grafana
End of explanation
import time
with en.G5kTunnel(address=monitoring.ui.address, port=3000) as (_, local_port, _):
    print(f"The service is running at http://localhost:{local_port}")
    time.sleep(60)
    
Explanation: To not forget to close the tunnel you can use a context manager: the tunnel will be closed automatically when exiting the context manager.
End of explanation
provider.destroy()
Explanation: Cleaning
End of explanation
<END_TASK>