<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Let us have a look at a 20 GB Rosbag file
Note data can be found for instance at https
Step1:  Show that the we can read the index
Solved the issue https
Step2:  Create the Spark Session or get an existing one
Step3:  Create an RDD from the Rosbag file
Note
<ASSISTANT_TASK:>
Python Code:
%%bash
ls -tralFh /root/project/doc/el_camino_north.bag
%%bash
# same size, no worries, just the -h (human) formating differs in rounding 
hdfs dfs -ls -h
Explanation: Let us have a look at a 20 GB Rosbag file
Note data can be found for instance at https://github.com/udacity/self-driving-car/tree/master/datasets published under MIT License.
The file is not distributed over the Dockerfile but you can download it and put it into HDFS.
End of explanation
%%time
out = !java -jar ../lib/rosbaginputformat.jar -f /root/project/doc/el_camino_north.bag
%%bash
ls -tralFh /root/project/doc/el_camino_north.bag*
Explanation: Show that the we can read the index
Solved the issue https://github.com/valtech/ros_hadoop/issues/6 
The issue was due to ByteBuffer being limitted by JVM Integer size and has nothing to do with Spark or how the RosbagMapInputFormat works within Spark. It was only problematic to extract the conf index with the jar.
Integer.MAX_SIZE is 2 GB !!
End of explanation
from pyspark import SparkContext, SparkConf
from pyspark.sql import SparkSession
sparkConf = SparkConf()
sparkConf.setMaster("local[*]")
sparkConf.setAppName("ros_hadoop")
sparkConf.set("spark.jars", "../lib/protobuf-java-3.3.0.jar,../lib/rosbaginputformat.jar,../lib/scala-library-2.11.8.jar")
spark = SparkSession.builder.config(conf=sparkConf).getOrCreate()
sc = spark.sparkContext
Explanation: Create the Spark Session or get an existing one
End of explanation
fin = sc.newAPIHadoopFile(
    path =             "hdfs://127.0.0.1:9000/user/root/el_camino_north.bag",
    inputFormatClass = "de.valtech.foss.RosbagMapInputFormat",
    keyClass =         "org.apache.hadoop.io.LongWritable",
    valueClass =       "org.apache.hadoop.io.MapWritable",
    conf =             {"RosbagInputFormat.chunkIdx":"/root/project/doc/el_camino_north.bag.idx.bin"})
fin
Explanation: Create an RDD from the Rosbag file
Note: your HDFS address might differ.
End of explanation
<END_TASK>