Does Spring Cloud Dataflow Local Server need to connect zookeeper?
<p>I am trying to set up scdf on docker containers.</p>

<p><strong>All the containers are in one network</strong></p>

<p>The two <em>kafka</em>s are exposed by port <em>2181</em></p>

<p>and I am sure the <em>kafka</em>s are connecting to the <em>zookeeper</em>s. (if not, they will not be started normally)</p>

<p>But during the creation of the simple stream </p>

<pre><code>stream create --name test --definition "time | log" --deploy
</code></pre>

<p>I got these logs in the <code>stdout_0.log</code> file which stands for the <code>log</code> app logs.</p>

<pre><code>2016-12-14 01:18:03.264  INFO 62 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-14 01:18:03.265  WARN 62 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn          : Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect

java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_111-internal]
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[na:1.8.0_111-internal]
    at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) ~[zookeeper-3.4.6.jar!/:3.4.6-1569965]
    at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) ~[zookeeper-3.4.6.jar!/:3.4.6-1569965]
</code></pre>

<p>in <code>stderr_1.log</code>, I have</p>

<pre><code>Caused by: org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 10000
    at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
    at org.I0Itec.zkclient.ZkClient.&lt;init&gt;(ZkClient.java:98)
    at org.I0Itec.zkclient.ZkClient.&lt;init&gt;(ZkClient.java:84)
    at org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.ensureTopicCreated(KafkaMessageChannelBinder.java:354)
    at org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.createKafkaConsumer(KafkaMessageChannelBinder.java:441)
    at org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.doBindConsumer(KafkaMessageChannelBinder.java:282)
    at org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.doBindConsumer(KafkaMessageChannelBinder.java:121)
    at org.springframework.cloud.stream.binder.AbstractBinder.bindConsumer(AbstractBinder.java:177)
    at org.springframework.cloud.stream.binding.ChannelBindingService.bindConsumer(ChannelBindingService.java:93)
    at org.springframework.cloud.stream.binding.BindableProxyFactory.bindInputs(BindableProxyFactory.java:189)
    at org.springframework.cloud.stream.binding.InputBindingLifecycle.start(InputBindingLifecycle.java:55)
    at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:173)
    ... 27 more
</code></pre>

<p>The first log said I am using <code>localhost:0.0.0.0:2181</code> for <em>zookeeper</em> connection, which I am not, and I have configured the real IP address for <em>kafka</em>.</p>

<p>the second log said the <em>timeout</em> for connection is <strong>10000</strong>, which in kafka, I set the connection timeout to <strong>6000</strong></p>

<p>So these are not from <em>kafka</em>, then they are from <em>scdf</em>.</p>

<p>the fact is, when I look up the reference of <em>scdf</em> <a href="http://docs.spring.io/spring-cloud-dataflow/docs/1.0.1.RELEASE/reference/htmlsingle/#_zookeeper" rel="nofollow noreferrer">link</a>, here it says, the <em>zookeeper</em> is not used (<em>or maybe not directly used</em>) in this architecture.</p>

<p>which confused me... if these logs not from <em>kafka</em> and not from <em>scdf</em>, then where does them from? </p>

<p>If <em>scdf</em> does use <em>zookeeper</em>, then how to configure the connection for <em>zookeeper</em>?</p>

<p><strong>UPDATE</strong></p>

<p>I am thinking about when <em>scdf</em> is trying to create a <em>topic</em>, from what I know about creating a <em>topic</em> you have to specify the zookeeper connection, <em>scdf</em> uses <em>localhost</em> for <em>producer</em>s or <em>consumer</em>s to register on <em>zookeeper</em>, so <em>scdf</em> is actually using <em>zookeeper</em> somehow. but how to configure this?</p>