{"action":"created","issue":{"url":"https://api.github.com/repos/floragunncom/search-guard/issues/3","labels_url":"https://api.github.com/repos/floragunncom/search-guard/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/floragunncom/search-guard/issues/3/comments","events_url":"https://api.github.com/repos/floragunncom/search-guard/issues/3/events","html_url":"https://github.com/floragunncom/search-guard/issues/3","id":84909487,"number":3,"title":"Config examples","user":{"login":"pemontto","id":939704,"avatar_url":"https://avatars.githubusercontent.com/u/939704?v=3","gravatar_id":"","url":"https://api.github.com/users/pemontto","html_url":"https://github.com/pemontto","followers_url":"https://api.github.com/users/pemontto/followers","following_url":"https://api.github.com/users/pemontto/following{/other_user}","gists_url":"https://api.github.com/users/pemontto/gists{/gist_id}","starred_url":"https://api.github.com/users/pemontto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pemontto/subscriptions","organizations_url":"https://api.github.com/users/pemontto/orgs","repos_url":"https://api.github.com/users/pemontto/repos","events_url":"https://api.github.com/users/pemontto/events{/privacy}","received_events_url":"https://api.github.com/users/pemontto/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/floragunncom/search-guard/labels/doc","name":"doc","color":"d4c5f9"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-06-04T04:45:40Z","updated_at":"2015-09-14T13:14:00Z","closed_at":null,"body":"It would be great if you supplied more in depth example configurations. For instance showing how to replicate the Shield config guides for particular clients would be very helpful. Guides for [Kibana3](https://www.elastic.co/guide/en/shield/current/_shield_with_kibana_3.html), [Kibana4](https://www.elastic.co/guide/en/shield/current/_shield_with_kibana_4.html), [Marvel](https://www.elastic.co/guide/en/shield/current/marvel.html), [Logstash](https://www.elastic.co/guide/en/shield/current/logstash.html)"},"comment":{"url":"https://api.github.com/repos/floragunncom/search-guard/issues/comments/140070557","html_url":"https://github.com/floragunncom/search-guard/issues/3#issuecomment-140070557","issue_url":"https://api.github.com/repos/floragunncom/search-guard/issues/3","id":140070557,"user":{"login":"SergeyBear","id":13977551,"avatar_url":"https://avatars.githubusercontent.com/u/13977551?v=3","gravatar_id":"","url":"https://api.github.com/users/SergeyBear","html_url":"https://github.com/SergeyBear","followers_url":"https://api.github.com/users/SergeyBear/followers","following_url":"https://api.github.com/users/SergeyBear/following{/other_user}","gists_url":"https://api.github.com/users/SergeyBear/gists{/gist_id}","starred_url":"https://api.github.com/users/SergeyBear/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SergeyBear/subscriptions","organizations_url":"https://api.github.com/users/SergeyBear/orgs","repos_url":"https://api.github.com/users/SergeyBear/repos","events_url":"https://api.github.com/users/SergeyBear/events{/privacy}","received_events_url":"https://api.github.com/users/SergeyBear/received_events","type":"User","site_admin":false},"created_at":"2015-09-14T13:13:59Z","updated_at":"2015-09-14T13:13:59Z","body":"Here is my example of connection to FreeIPA.\r\nFor completness I left identical to previous example.\r\n\r\n# Required\r\n* CentOS 7.1\r\n* FreeIPA 4.1 in https://ipa.domain.com with REALM domain.com\r\n* Elasticsearch 1.5.2\r\n* Search-Guard 0.5\r\n* Kibana 4.1.1\r\n* Logstash 1.5.4\r\n* Elasticsearch and Logstash installed from official repos\r\n* Kibana installed to **/opt/kibana** and runned as user **kibana**\r\n* All ELK tested to work properly with installed but not enabled search-guard\r\n\r\n# Getting started\r\n## Prepearing FreeIPA\r\nIn most cases anonymous binds disabled in production LDAP. So we will think that it is the case.\r\n\r\n**NOTICE**: System user cannot write to directory server and cannot login to IPA UI. But he can read everithing from directory server. So keep it safe!\r\n\r\n**NOTICE**: To create system user you need to have directory server account. \r\n\r\nOn FreeIPA server host:\r\n\r\n```\r\nldapmodify -x -D 'cn=Directory Manager' -W\r\ndn: uid=elastic,cn=sysaccounts,cn=etc,dc=example,dc=com\r\nchangetype: add\r\nobjectclass: account\r\nobjectclass: simplesecurityobject\r\nuid: mx\r\nuserPassword: passwd111\r\npasswordExpirationTime: 20380119031407Z\r\nnsIdleTimeout: 0\r\n```\r\n\r\nWith IPA UI or IPA CLI:\r\n1. Create user: **logstash**, **kibana**, **kibana.admin** and **kibana.user**\r\n2. Create group **services** and add users **kibana** and **logstash** to it\r\n3. Create group **kibana-admins** and add user **kibana.admin** to it\r\n4. Create group **kibana-users** and add user **kibana.user** to it\r\n5. Create group **elastic-users** and add to it groups **kibana-admins** and **kibana-users** and users **kibana** and **logstash**\r\n6. Create password policy for group **services** with maximum password expiration time (max is 20000)\r\n5. Login with each user to https://ipa.domain.com to ensure that everything is ok.\r\n\r\n\r\n## Preparing Java\r\nBy default FreeIPA create it's own self-signed CA certificate during installation. In this case, you need to add IPA CA cert to java thrusted store on elasticsearch host or you will get an error while trying to connect IPA.\r\n\r\n**NOTICE**: If your FreeIPA installed with CA cert signed by thrusted authority (lucky you! :-) ), then you can skip this block and proceed to ELK configurations.\r\n\r\nOn elasticsearch host:\r\n\r\n```\r\ncd /etc/pki/ca-trust/extracted/java\r\ncp cacerts cacerts.bak\r\n\r\nwget --no-check-certificate https://ipa.domain.com/ipa/config/ca.crt\r\nkeytool -keystore cacerts -importcert -alias freeipa -file ca.crt\r\n```\r\n\r\nDefault java keystore password is **changeit**\r\n\r\n```\r\nkeytool -list -keystore cacerts\r\n```\r\n\r\nCheck command output that certificate added\r\n\r\n\r\n## Preparing ELK\r\nAdd to **/etc/elasticsearch/elasticsearch.yml**\r\n\r\n```\r\nsearchguard.enabled: true\r\nsearchguard.key_path: /etc/elasticsearch/\r\n#searchguard.allow_all_from_loopback: true\r\n\r\nsearchguard.http.xforwardedfor.header: DUMMY\r\n\r\nsearchguard.authentication.authentication_backend.impl: com.floragunn.searchguard.authentication.backend.ldap.LDAPAuthenticationBackend\r\nsearchguard.authentication.authentication_backend.cache.enable: false\r\nsearchguard.authentication.authorizer.impl: com.floragunn.searchguard.authorization.ldap.LDAPAuthorizator\r\nsearchguard.authentication.authorizer.cache.enable: false\r\nsearchguard.authentication.http_authenticator.impl: com.floragunn.searchguard.authentication.http.basic.HTTPBasicAuthenticator\r\n\r\nsearchguard.authentication.ldap.host: [\"ipa.domain.com:389\"]\r\nsearchguard.authentication.ldap.ldaps.ssl.enabled: false\r\nsearchguard.authentication.ldap.ldaps.starttls.enabled: true\r\n\r\n## Comment out next 3 lines in case of using thrusted certificate in FreeIPA\r\nsearchguard.authentication.ldap.ldaps.truststore_type: JKS\r\nsearchguard.authentication.ldap.ldaps.truststore_filepath: \"/etc/pki/ca-trust/extracted/java/cacerts\"\r\nsearchguard.authentication.ldap.ldaps.truststore_password: \"changeit\"\r\n\r\nsearchguard.authentication.ldap.bind_dn: \"uid=elastic,cn=sysaccounts,cn=etc,dc=domain,dc=com\"\r\nsearchguard.authentication.ldap.password: passwd111\r\nsearchguard.authentication.ldap.userbase: \"cn=users,cn=accounts,dc=domain,dc=com\"\r\nsearchguard.authentication.ldap.usersearch: (&(objectClass=posixAccount)(memberOf:=cn=elastic-users,cn=groups,cn=accounts,dc=domain,dc=com)(uid={0})(!(nsaccountlock=TRUE)))\r\nsearchguard.authentication.ldap.username_attribute: uid\r\n\r\nsearchguard.authentication.authorization.ldap.rolebase: \"cn=groups,cn=accounts,dc=domain,dc=com\"\r\nsearchguard.authentication.authorization.ldap.rolesearch: (&(objectClass=posixGroup)(memberOf:=cn=elastic-users,cn=groups,cn=accounts,dc=domain,dc=com)(member={0}))\r\nsearchguard.authentication.authorization.ldap.rolename: cn\r\nsearchguard.authentication.authorization.ldap.resolve_nested_roles: false\r\n\r\nsearchguard.actionrequestfilter.names: [\"kibana-server\", \"kibana-user\", \"kibana-admin\", \"logstash-server\", \"logstash-user\"]\r\nsearchguard.actionrequestfilter.kibana-server.allowed_actions: [\r\n  \"cluster:monitor/nodes/info*\",\r\n  \"cluster:monitor/health*\",\r\n  \"indices:admin/create*\",\r\n  \"indices:admin/exists*\",\r\n  \"indices:admin/mapping/put*\",\r\n  \"indices:admin/mappings/fields/get*\",\r\n  \"indices:admin/refresh*\",\r\n  \"indices:admin/validate/query*\",\r\n  \"indices:data/read/get*\",\r\n  \"indices:data/read/mget*\",\r\n  \"indices:data/read/search*\",\r\n  \"indices:data/write/delete*\",\r\n  \"indices:data/write/index*\",\r\n  \"indices:data/write/update*\"\r\n]\r\nsearchguard.actionrequestfilter.kibana-user.allowed_actions: [\r\n  \"cluster:monitor/nodes/info*\",\r\n  \"cluster:monitor/health*\",\r\n  \"indices:admin/exists*\",\r\n  \"indices:admin/mappings/fields/get*\",\r\n  \"indices:admin/validate/query*\",\r\n  \"indices:data/read/get*\",\r\n  \"indices:data/read/mget*\",\r\n  \"indices:data/read/search*\"\r\n]\r\nsearchguard.actionrequestfilter.kibana-admin.allowed_actions: [\r\n  \"cluster:monitor/nodes/info*\",\r\n  \"cluster:monitor/health*\",\r\n  \"indices:admin/exists*\",\r\n  \"indices:admin/mapping/put*\",\r\n  \"indices:admin/mappings/fields/get*\",\r\n  \"indices:admin/refresh*\",\r\n  \"indices:admin/validate/query*\",\r\n  \"indices:data/read/get*\",\r\n  \"indices:data/read/mget*\",\r\n  \"indices:data/read/search*\",\r\n  \"indices:data/write/delete*\",\r\n  \"indices:data/write/index*\",\r\n  \"indices:data/write/update*\",\r\n  \"indices:admin/create*\"\r\n]\r\nsearchguard.actionrequestfilter.logstash-server.allowed_actions: [\r\n  \"indices:admin/template/get*\",\r\n  \"indices:admin/template/put*\",\r\n  \"indices:admin/create*\",\r\n  \"indices:data/write/bulk*\",\r\n  \"indices:data/write/index*\",\r\n  \"indices:data/write/delete*\",\r\n  \"indices:data/write/update*\",\r\n  \"indices:data/read/search*\",\r\n  \"indices:data/read/scroll*\"\r\n]\r\nsearchguard.actionrequestfilter.logstash-user.allowed_actions: [\r\n  \"indices:admin/mappings/fields/get*\",\r\n  \"indices:admin/validate/query*\",\r\n  \"indices:data/read/search*\",\r\n  \"indices:data/read/msearch*\",\r\n  \"indices:admin/get*\"\r\n]\r\n```\r\n\r\n**NOTICE**: If you want to use nested groups, then set **searchguard.authentication.authorization.ldap.resolve_nested_roles** to **true**.\r\n\r\nAdd to **/opt/kibana/config/kibana.yml** login and password for IPA user\r\n\r\n```\r\nkibana_elasticsearch_username: kibana\r\nkibana_elasticsearch_password: passwd222\r\n```\r\n\r\nAdd to **/etc/logstash/conf.d/30-lumberjack-output.conf** login and password for IPA user\r\n\r\n```\r\noutput {\r\n  elasticsearch {\r\n    protocol => http\r\n    host => localhost\r\n    user => logstash\r\n    password => passwd333\r\n    #index => \"logstash-%{type}-%{+YYYY.MM.dd}\"\r\n  }\r\n  stdout { codec => rubydebug }\r\n}\r\n```\r\n\r\n**NOTICE**: To create separate index per each log type uncomment **index** parameter. In this case specified in logstash-forwarder field **type** will automatically be added to indice name. You also can use any fields or multiple fields you like added by logstash-forwarder. It can be very usefull  for separating access to different user roles.\r\n\r\nThen change owners and mod of config files with passwords to be sure that only root and services can read them:\r\n\r\n```\r\nchown -R root.elasticsearch /etc/elasticsearch\r\nchmod 640 /etc/elasticsearch/elasticsearch.yml\r\nchmod 750 /etc/elasticsearch\r\n\r\nchown root.logstash /etc/logstash/conf.d/30-lumberjack-output.conf\r\nchmod 640 /etc/logstash/conf.d/30-lumberjack-output.conf\r\n\r\nchown root.kibana /opt/kibana/config/kibana.yml\r\nchmod 640 /opt/kibana/config/kibana.yml\r\n```\r\n\r\nBefore adding ACL make sure you deny all access to Kibana for users.\r\nUncomment in **elasticsearch.yml** parameter **searchguard.allow_all_from_loopback: true**.\r\nAnd execute on elasticsearch host following:\r\n\r\n```\r\nservice elasticsearch restart\r\n\r\ncurl -XPUT \"http://localhost:9200/searchguard/ac/ac\" -d '{\r\n    \"acl\": [\r\n    {\r\n        \"__Comment__\": \"By default no filters are executed and no filters a by-passed. In such a case an exception is thrown and access will be denied.\",\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": []\r\n    },\r\n    {\r\n        \"__Comment__\": \"Allow Logstash server user manage logstash indices\",\r\n        \"users\": [\"kibana\"],\r\n        \"indices\": [\".kibana\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.kibana-server\"]\r\n    },\r\n    {\r\n        \"__Comment__\": \"Allow Kibana server user manage logstash indices\",\r\n        \"users\": [\"logstash\"],\r\n        \"indices\": [\"logstash-*\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.logstash-server\"]\r\n    },\r\n    {\r\n        \"__Comment__\": \"Allow administrators group manage Kibana indices\",\r\n        \"roles\": [\"kibana-admins\"],\r\n        \"indices\": [\".kibana\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.kibana-admin\"]\r\n    },\r\n    {\r\n        \"__Comment__\": \"Allow administrators group read all Logstash indices\",\r\n        \"roles\": [\"kibana-admins\"],\r\n        \"indices\": [\"logstash-*\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.logstash-user\"]\r\n    },\r\n    {\r\n        \"__Comment__\": \"Allow users group read Kibana indices\",\r\n        \"roles\": [\"kibana-users\"],\r\n        \"indices\": [\".kibana\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.kibana-user\"]\r\n    },\r\n    {\r\n        \"__Comment__\": \"Allow users group read all Logstash indices\",\r\n        \"roles\": [\"kibana-users\"],\r\n        \"indices\": [\"logstash-*\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.logstash-user\"]\r\n    }\r\n    ]\r\n}'\r\n\r\n```\r\n\r\nThen comment back **searchguard.allow_all_from_loopback: true** and restart all together:\r\n\r\n```\r\nservice elasticsearch restart && service kibana restart && service logstash restart\r\n```\r\n\r\nNow you can open Kibana to users and test authentication.\r\n\r\n# How it works\r\n* All IPA users in IPA group **kibana-users** can only read kibana and logstash indices.\r\n* All IPA users in IPA group **kibana-admins** can edit kibana and read logstash indices.\r\n* All grants were taken from official Shield documentation.\r\n* If you need group **kibana-devs** that can only read indices **logstash-nginx-*** then\r\n** in IPA create user **kibana.dev**\r\n** in IPA create group **kibana-devs**\r\n** in IPA add group **kibana-devs** to group **elastic-users**\r\n** in IPA add user **kibana.dev** to group **kibana-devs**\r\n** Apply ACL with added rule:\r\n\r\n``` \r\n    {\r\n        \"__Comment__\": \"Allow devs group read Kibana indices\",\r\n        \"roles\": [\"kibana-devs\"],\r\n        \"indices\": [\".kibana\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.kibana-user\"]\r\n    },\r\n    {\r\n        \"__Comment__\": \"\"Allow devs group read specific Logstash indices\",\r\n        \"roles\": [\"kibana-devs\"],\r\n        \"indices\": [\"logstash-nginx-*\"],\r\n        \"filters_bypass\": [],\r\n        \"filters_execute\": [\"actionrequestfilter.logstash-user\"]\r\n    }\r\n```\r\n\r\n# Problems\r\n* There is a bug that allow authenticated users to see LDAP Bind username and passwords in elasticsearch by executing query:\r\n\r\n```\r\ncurl -XGET -u user -v -XGET 'http://localhost:9200/_nodes?pretty'\r\n```\r\n\r\nThis can be overrided by using digest passwords **searchguard.authentication.settingsdb.digest**, but I don't see any much profit of using digest, because while you don't give access to elasticsearch directly (not through Kibana) it's not a problem at all. Fix me if I wrong :-)\r\n* In current state I found that **dlsfilters** and **flsfilters** of search-guard breaks Kibana, so for now you need to use per indice-access (like Shield does)\r\n* To apply ACL elasticsearch must be started with **searchguard.allow_all_from_loopback: true** and set to **false** and restarted after ACL applied."}}