Dependency is not distributed to Spark cluster
<p>I'm trying to execute Spark job on Mesos cluster that depends on <a href="https://github.com/datastax/spark-cassandra-connector" rel="nofollow">spark-cassandra-connector</a> library, but it keeps failing with</p>

<pre><code>Exception in thread "main" java.lang.NoClassDefFoundError: com/datastax/spark/connector/package$
</code></pre>

<p>As I understand from <a href="http://spark.apache.org/docs/1.6.1/submitting-applications.html" rel="nofollow">spark documentation</a></p>

<blockquote>
  <p>JARs and files are copied to the working directory for each SparkContext on the executor nodes.
  ...
  Users may also include any other dependencies by supplying a comma-delimited list of maven coordinates with --packages.</p>
</blockquote>

<p>But it seems that only <code>pucker-assembly-1.0.jar</code> task jar is distributed.</p>

<p>I'm running  spark 1.6.1 with scala 2.10.6.
And here's <code>spark-submit</code> command I'm executing:</p>

<pre><code>spark-submit --deploy-mode cluster 
             --master mesos://localhost:57811 
             --conf spark.ssl.noCertVerification=true 
             --packages datastax:spark-cassandra-connector:1.5.1-s_2.10
             --conf spark.cassandra.connection.host=10.0.1.83,10.0.1.86,10.0.1.85 
             --driver-cores 3 
             --driver-memory 4000M 
             --class SimpleApp 
             https://dripit-spark.s3.amazonaws.com/pucker-assembly-1.0.jar
             s3n://logs/E1SR85P3DEM3LU.2016-05-05-11.ceaeb015.gz
</code></pre>

<p>So why isn't <code>spark-cassandra-connector</code> distributed to all my spark executers?</p>