How can I use request module in Spark?
<p>this is the code I used.</p>

<pre><code>from __future__ import print_function
import sys
from pyspark.sql import SparkSession
import sys
sys.path.append('/usr/local/lib/python2.7/site-packages')
import requests

if __name__ == "__main__":
    s = Session()
    toGet = s.get

    spark = SparkSession\
        .builder\
        .appName("PythonDockerRepoStat")\
        .getOrCreate()

    lines = spark.read.text('/data/urls.txt').rdd.map(lambda r: r[0])
    res = lines.flatMap(lambda x: x.split("\n"))\
        .map(lambda x: toGet(x))
    output = res.collect()
    print(output)
</code></pre>

<p>However, I got this error: <em>ImportError: No module named requests.sessions</em></p>