Trigger a spark job whenever an event occurs
<p>I have a spark application which should run whenever it receives a kafka message on a topic.</p>

<p>I won't be receiving more than 5-6 messages a day so I don't want to take spark streaming approach. Instead I tried to submit the application using <code>SparkLauncher</code> but I don't like the approach as I have to set spark and Java classpath programmatically within my code along with all the necessary spark properties like executor cores, executor memory etc.</p>

<p>How do I trigger the spark application to run from <code>spark-submit</code> but make it wait until it receives a message?</p>

<p>Any pointers are very helpful.</p>