Running pySpark in Jupyter notebooks - Windows
<p>I would like to run pySpark from Jupyter notebook. I downloaded and installed Anaconda which had Juptyer. I created the following lines </p>

<pre><code> from pyspark import SparkConf, SparkContext
 conf = SparkConf().setMaster("local").setAppName("My App")
 sc = SparkContext(conf = conf)
</code></pre>

<p>I get the following error</p>

<pre><code>ImportError                               Traceback (most recent call last)
&lt;ipython-input-3-98c83f0bd5ff&gt; in &lt;module&gt;()
  ----&gt; 1 from pyspark import SparkConf, SparkContext
  2 conf = SparkConf().setMaster("local").setAppName("My App")
  3 sc = SparkContext(conf = conf)

 C:\software\spark\spark-1.6.2-bin-hadoop2.6\python\pyspark\__init__.py in   &lt;module&gt;()
 39 
 40 from pyspark.conf import SparkConf
  ---&gt; 41 from pyspark.context import SparkContext
 42 from pyspark.rdd import RDD
 43 from pyspark.files import SparkFiles

 C:\software\spark\spark-1.6.2-bin-hadoop2.6\python\pyspark\context.py in &lt;module&gt;()
 26 from tempfile import NamedTemporaryFile
 27 
 ---&gt; 28 from pyspark import accumulators
 29 from pyspark.accumulators import Accumulator
 30 from pyspark.broadcast import Broadcast

 ImportError: cannot import name accumulators
</code></pre>

<p>I tried adding the following environment variable PYTHONPATH which points to the spark/python  directory, based on an answer in Stackoverflow  <a href="https://stackoverflow.com/questions/23256536/importing-pyspark-in-python-shell">importing pyspark in python shell</a> </p>

<p>but this was of no help</p>