getting CSV sink metrics files from spark-submit at run time
<p>Having metrics.properties in /conf  (enabling CSV sink) as follows (see  configuration below), collects metrics every time you submit a job (using spark-submit) and it works by saving it to /tmp/</p>

<pre><code># Enable CsvSink for all instances
*.sink.csv.class=org.apache.spark.metrics.sink.CsvSink

# Polling period for CsvSink
*.sink.csv.period=1

*.sink.csv.unit=minutes

# Polling directory for CsvSink
*.sink.csv.directory=/tmp/

# Worker instance overlap polling period
worker.sink.csv.period=1

worker.sink.csv.unit=minutes
</code></pre>

<p><strong>Now I want to give metrics.properties file at run time (using the same configuration as above), and I gave the arguments for spark-submit as follows:</strong></p>

<pre><code>$spark_home/bin/spark-submit --files=file:///home/log_properties/metrics.properties --conf spark.metrics.conf=./metrics.properties --class com.myClass job1.jar
</code></pre>

<p><strong>And I get the following warning and I don't have any Graphite configuration in my metrics.properties file (I just used the metrics.template and enabled the above csv configurations only)</strong></p>

<pre><code> WARN graphite.GraphiteReporter: Unable to report to Graphite
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at java.net.Socket.connect(Socket.java:538)
        at java.net.Socket.&lt;init&gt;(Socket.java:434)
        at java.net.Socket.&lt;init&gt;(Socket.java:244)
        at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:277)
        at com.codahale.metrics.graphite.Graphite.connect(Graphite.java:118)
        at com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:167)
        at com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162)
        at org.apache.spark.metrics.sink.GraphiteSink.report(GraphiteSink.scala:91)
        at org.apache.spark.metrics.MetricsSystem$$anonfun$report$1.apply(MetricsSystem.scala:114)
        at org.apache.spark.metrics.MetricsSystem$$anonfun$report$1.apply(MetricsSystem.scala:114)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
        at org.apache.spark.metrics.MetricsSystem.report(MetricsSystem.scala:114)
        at org.apache.spark.SparkContext$$anonfun$stop$3.apply$mcV$sp(SparkContext.scala:1715)
        at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1219)
        at org.apache.spark.SparkContext.stop(SparkContext.scala:1714)
        at org.apache.spark.SparkContext$$anonfun$3.apply$mcV$sp(SparkContext.scala:596)
        at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:267)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ShutdownHookManager.scala:239)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:239)
        at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:239)
</code></pre>

<p><strong>Is it defaulted to report to Graphite and is ignoring my metrics.properties (which only  is enabled for CSV sink)????</strong> </p>