How to resolve external packages with spark-shell when behind a corporate proxy?
<p>I would like to run spark-shell with a external package behind a corporate proxy. Unfortunately external packages passed via <code>--packages</code> option are not resolved.</p>

<p>E.g., when running</p>

<pre><code>bin/spark-shell --packages datastax:spark-cassandra-connector:1.5.0-s_2.10
</code></pre>

<p>the cassandra connector package is not resolved (stuck at last line):</p>

<pre><code>Ivy Default Cache set to: /root/.ivy2/cache
The jars for the packages stored in: /root/.ivy2/jars
:: loading settings :: url = jar:file:/opt/spark/lib/spark-assembly-1.6.1-hadoop2.6.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
datastax#spark-cassandra-connector added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent;1.0
        confs: [default]
</code></pre>

<p>After some time the connection times out containing error messages like this:</p>

<pre><code>:::: ERRORS
        Server access error at url https://repo1.maven.org/maven2/datastax/spark-cassandra-connector/1.5.0-s_2.10/spark-cassandra-connector-1.5.0-s_2.10.pom (java.net.ConnectException: Connection timed out)
</code></pre>

<p>When i deactivate the VPN with the corporate proxy the package gets resolved and downloaded immediately.</p>

<p>What i tried so far:</p>

<p>Exposing proxies as environment variables:</p>

<pre><code>export http_proxy=&lt;proxyHost&gt;:&lt;proxyPort&gt;
export https_proxy=&lt;proxyHost&gt;:&lt;proxyPort&gt;
export JAVA_OPTS="-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;"
export ANT_OPTS="-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;"
</code></pre>

<p>Running spark-shell with extra java options:</p>

<pre><code>bin/spark-shell --conf "spark.driver.extraJavaOptions=-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;" --conf "spark.executor.extraJavaOptions=-Dhttp.proxyHost=&lt;proxyHost&gt; -Dhttp.proxyPort=&lt;proxyPort&gt;" --packages datastax:spark-cassandra-connector:1.6.0-M1-s_2.10
</code></pre>

<p>Is there some other configuration possibility i am missing?</p>