set spark.streaming.kafka.maxRatePerPartition for createDirectStream
<p>I need to increase the input rate per partition for my application and I have use <code>.set("spark.streaming.kafka.maxRatePerPartition",100)</code> for the config. The stream duration is 10s so I expect process <code>5*100*10=5000</code> messages for this batch. However, the input rate I received is just about 500. Can You suggest any modifications to increase this rate?</p>