Oozie Sqoop job - cannot restore job
<p>On HDP 2.3.4, using Oozie 4.2.0 and Sqoop 1.4.2, I'm trying to create a coordinator app that will execute sqoop jobs on a daily basis. I need the sqoop action to execute jobs because these are incremental imports. </p>

<p>I've configured <code>sqoop-site.xml</code> and started the <code>sqoop-metastore</code> and I'm able to create, list, and delete jobs via the command line but the workflow encounters the error: <strong>Cannot restore job: streamsummary_incremental</strong></p>

<p><strong>stderr</strong></p>

<pre><code>Sqoop command arguments :
             job
             --exec
             streamsummary_incremental
Fetching child yarn jobs
tag id : oozie-26fcd4dc0afd8f53316fc929ac38eae2
2016-02-03 09:46:47,193 INFO  [main] client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at &lt;myHost&gt;/&lt;myIP&gt;:8032
Child yarn jobs are found - 
=================================================================

&gt;&gt;&gt; Invoking Sqoop command line now &gt;&gt;&gt;

2241 [main] WARN  org.apache.sqoop.tool.SqoopTool  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
2016-02-03 09:46:47,404 WARN  [main] tool.SqoopTool (SqoopTool.java:loadPluginsFromConfDir(177)) - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
2263 [main] INFO  org.apache.sqoop.Sqoop  - Running Sqoop version: 1.4.6.2.3.4.0-3485
2016-02-03 09:46:47,426 INFO  [main] sqoop.Sqoop (Sqoop.java:&lt;init&gt;(97)) - Running Sqoop version: 1.4.6.2.3.4.0-3485
2552 [main] ERROR org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage  - Cannot restore job: streamsummary_incremental
2016-02-03 09:46:47,715 ERROR [main] hsqldb.HsqldbJobStorage (HsqldbJobStorage.java:read(254)) - Cannot restore job: streamsummary_incremental
2552 [main] ERROR org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage  - (No such job)
2016-02-03 09:46:47,715 ERROR [main] hsqldb.HsqldbJobStorage (HsqldbJobStorage.java:read(255)) - (No such job)
2553 [main] ERROR org.apache.sqoop.tool.JobTool  - I/O error performing job operation: java.io.IOException: Cannot restore missing job streamsummary_incremental
    at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.read(HsqldbJobStorage.java:256)
    at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:198)
    at org.apache.sqoop.tool.JobTool.run(JobTool.java:283)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
    at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197)
    at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:177)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
    at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:46)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:241)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
</code></pre>

<p><strong>sqoop-site.xml</strong></p>

<pre><code>  &lt;property&gt;
    &lt;name&gt;sqoop.metastore.client.enable.autoconnect&lt;/name&gt;
    &lt;value&gt;false&lt;/value&gt;
    &lt;description&gt;If true, Sqoop will connect to a local metastore
      for job management when no other metastore arguments are
      provided.
    &lt;/description&gt;
  &lt;/property&gt;

  &lt;property&gt;
    &lt;name&gt;sqoop.metastore.client.autoconnect.url&lt;/name&gt;
    &lt;value&gt;jdbc:hsqldb:hsql://&lt;myhost&gt;:12345&lt;/value&gt;
    &lt;description&gt;The connect string to use when connecting to a
      job-management metastore. If unspecified, uses ~/.sqoop/.
      You can specify a different path here.
    &lt;/description&gt;
  &lt;/property&gt;

  &lt;property&gt;
    &lt;name&gt;sqoop.metastore.client.autoconnect.username&lt;/name&gt;
    &lt;value&gt;SA&lt;/value&gt;
    &lt;description&gt;The username to bind to the metastore.
    &lt;/description&gt;
  &lt;/property&gt;

  &lt;property&gt;
    &lt;name&gt;sqoop.metastore.client.autoconnect.password&lt;/name&gt;
    &lt;value&gt;&lt;/value&gt;
    &lt;description&gt;The password to bind to the metastore.
    &lt;/description&gt;
  &lt;/property&gt;

  &lt;property&gt;
    &lt;name&gt;sqoop.metastore.server.location&lt;/name&gt;
    &lt;value&gt;/tmp/sqoop-metastore/shared.db&lt;/value&gt;
    &lt;description&gt;Path to the shared metastore database files.
    If this is not set, it will be placed in ~/.sqoop/.
    &lt;/description&gt;
  &lt;/property&gt;

  &lt;property&gt;
    &lt;name&gt;sqoop.metastore.server.port&lt;/name&gt;
    &lt;value&gt;12345&lt;/value&gt;
    &lt;description&gt;Port that this metastore should listen on.
    &lt;/description&gt;
  &lt;/property&gt;
</code></pre>

<p><strong>workflow.xml</strong></p>

<pre><code>  &lt;action name="sqoop-import-job"&gt;
    &lt;sqoop xmlns="uri:oozie:sqoop-action:0.2"&gt;
      &lt;job-tracker&gt;${jobTracker}&lt;/job-tracker&gt;
      &lt;name-node&gt;${nameNode}&lt;/name-node&gt;
      &lt;prepare&gt;
        &lt;delete path="${outputDir}"/&gt;
      &lt;/prepare&gt;
      &lt;arg&gt;job&lt;/arg&gt;
      &lt;arg&gt;--exec&lt;/arg&gt;
      &lt;arg&gt;${jobId}&lt;/arg&gt;
    &lt;/sqoop&gt;
    &lt;ok to="hive-load"/&gt;
    &lt;error to="kill-sqoop"/&gt;
  &lt;/action&gt;
</code></pre>

<p>Additional info:</p>

<ul>
<li>We're only running a single-node cluster.</li>
<li>Only Sqoop Client is
installed.</li>
</ul>

<p>I'm thinking maybe Oozie isn't able to connect to the metastore because we don't have sqoop server? Could anyone confirm this? If not that, could I have missed anything else?</p>

<p>Thanks!</p>