Spark can access Hive table from pyspark but not from spark-submit
<p>So, when running from pyspark i would type in (without specifying any contexts) :</p>

<pre><code>df_openings_latest = sqlContext.sql('select * from experian_int_openings_latest_orc')
</code></pre>

<p>.. and it works fine.</p>

<p>However, when i run my script from <code>spark-submit</code>, like </p>

<p><code>spark-submit script.py</code> i put the following in </p>

<pre><code>from pyspark.sql import SQLContext
from pyspark import SparkConf, SparkContext
conf = SparkConf().setAppName('inc_dd_openings')
sc = SparkContext(conf=conf)
sqlContext = SQLContext(sc)

df_openings_latest = sqlContext.sql('select * from experian_int_openings_latest_orc')
</code></pre>

<p>But it gives me an error </p>

<blockquote>
  <p>pyspark.sql.utils.AnalysisException: u'Table not found:
  experian_int_openings_latest_orc;'</p>
</blockquote>

<p>So it doesnt see my table. </p>

<p>What am I doing wrong? Please help</p>

<p>P.S. Spark version is 1.6 running on Amazon EMR</p>