Logstash : get URL params into hash
<p>I'm trying to use Logstash and ElasticSearch to monitor my Apache webserver activity. At this time, it works pretty well but I need to more specific informations about my request field.
At this time my logstash configuration is : </p>

<pre><code>filter {
  grok { match =&gt; { "message" =&gt; "%{COMBINEDAPACHELOG}" } }
  grok { match =&gt; { "request" =&gt; [ "url", "%{URIPATH:url_path}%{URIPARAM:url_params}?" ]} }
   urldecode{ field =&gt; "url_path" }
   mutate { gsub =&gt;  ["url_params","\?","" ] }
   kv {
     field_split =&gt; "&amp;"
     source =&gt; "url_params"
     prefix =&gt; "url_param_"
   }
   date { match =&gt; [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ] }
   geoip { source =&gt; "clientip" }
   useragent { source =&gt; "agent" }
 }
</code></pre>

<p>Taking a basic apache log : </p>

<pre><code>255.254.230.10 - - [11/Dec/2013:00:01:45 -0800] "GET /xampp/boreal%3A123456/status.php?pretty=true&amp;test=boreal%3A12345 HTTP/1.1" 200 3891 "http://cadenza/xampp/navi.php" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0"
</code></pre>

<p>The result of this first configuration is : </p>

<pre><code>{
         "message" =&gt; "255.254.230.10 - - [11/Dec/2013:00:01:45 -0800] \"GET /xampp/boreal%3A123456/status.php?pretty=true&amp;test=boreal:%3A12345 HTTP/1.1\" 200 3891 \"http://cadenza/xampp/navi.php\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:25.0) Gecko/20100101 Firefox/25.0\"",
        "@version" =&gt; "1",
      "@timestamp" =&gt; "2013-12-11T08:01:45.000Z",
            ...
         "request" =&gt; "/xampp/boreal%3A123456/status.php?pretty=true&amp;test=boreal%3A12345",
        "url_path" =&gt; "/xampp/boreal:123456/status.php",
      "url_params" =&gt; "pretty=true&amp;test=boreal%3A12345",
"url_param_pretty" =&gt; "true",
  "url_param_test" =&gt; "boreal%3A12345",
           ...    
}
</code></pre>

<p>And (in a dream world), I would like to have this response for url params :</p>

<pre><code>{
         ...
         "request" =&gt; "/xampp/boreal%3A123456/status.php?pretty=true&amp;test=boreal%3A12345",
        "url_path" =&gt; "/xampp/boreal:123456/status.php",
      "url_params" =&gt; {
                "pretty" =&gt; "true",
        "url_param_test" =&gt; "boreal:12345"
      },
           ...    
}
</code></pre>

<p><strong>My whishes</strong></p>

<ul>
<li>url_params become a hash array.</li>
<li>each key of this hash will be the name of the param</li>
<li>each corresponding value will be the urldecode value</li>
</ul>

<p><strong>Questions</strong></p>

<ul>
<li>Does I need to create my own plugin (I'm not yet familiar with ruby) ? </li>
<li>Is it exist a existing plugin (I didn't found ... maybe bad search) ?</li>
<li>Is it a way to do that without a plugin ?</li>
</ul>

<p>Thanks for your help (and sorry for my english)</p>

<p>Renaud </p>

<p><strong>Solution :</strong></p>

<p>Thanks to Val, He found the solution. I changed my configuration to :</p>

<pre><code>grok { match =&gt; { "request" =&gt; [ "url", "%{URIPATH:url_path}%{URIPARAM:url_params}?" ]} }
urldecode{ field =&gt; "url_path" }
mutate { gsub =&gt;  ["url_params","\?","" ] }
kv {
  field_split =&gt; "&amp;"
  source =&gt; "url_params"
  target =&gt; "url_params_hash"
}
urldecode{ field =&gt; "url_params_hash" }
</code></pre>

<p>Using this solution, even if an "&amp;"(%26) character are into url_params string the splitting is correct.</p>