{"action":"created","issue":{"url":"https://api.github.com/repos/Mashape/kong/issues/15","labels_url":"https://api.github.com/repos/Mashape/kong/issues/15/labels{/name}","comments_url":"https://api.github.com/repos/Mashape/kong/issues/15/comments","events_url":"https://api.github.com/repos/Mashape/kong/issues/15/events","html_url":"https://github.com/Mashape/kong/issues/15","id":57876778,"number":15,"title":"Caching/Invalidation strategy for APIs, Applications and Plugins","user":{"login":"thefosk","id":964813,"avatar_url":"https://avatars.githubusercontent.com/u/964813?v=3","gravatar_id":"","url":"https://api.github.com/users/thefosk","html_url":"https://github.com/thefosk","followers_url":"https://api.github.com/users/thefosk/followers","following_url":"https://api.github.com/users/thefosk/following{/other_user}","gists_url":"https://api.github.com/users/thefosk/gists{/gist_id}","starred_url":"https://api.github.com/users/thefosk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thefosk/subscriptions","organizations_url":"https://api.github.com/users/thefosk/orgs","repos_url":"https://api.github.com/users/thefosk/repos","events_url":"https://api.github.com/users/thefosk/events{/privacy}","received_events_url":"https://api.github.com/users/thefosk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/Mashape/kong/milestones/9","html_url":"https://github.com/Mashape/kong/milestones/0.6.0","labels_url":"https://api.github.com/repos/Mashape/kong/milestones/9/labels","id":1182243,"number":9,"title":"0.6.0","description":"","creator":{"login":"sinzone","id":1213643,"avatar_url":"https://avatars.githubusercontent.com/u/1213643?v=3","gravatar_id":"","url":"https://api.github.com/users/sinzone","html_url":"https://github.com/sinzone","followers_url":"https://api.github.com/users/sinzone/followers","following_url":"https://api.github.com/users/sinzone/following{/other_user}","gists_url":"https://api.github.com/users/sinzone/gists{/gist_id}","starred_url":"https://api.github.com/users/sinzone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sinzone/subscriptions","organizations_url":"https://api.github.com/users/sinzone/orgs","repos_url":"https://api.github.com/users/sinzone/repos","events_url":"https://api.github.com/users/sinzone/events{/privacy}","received_events_url":"https://api.github.com/users/sinzone/received_events","type":"User","site_admin":false},"open_issues":2,"closed_issues":0,"state":"open","created_at":"2015-06-26T08:38:35Z","updated_at":"2015-10-05T21:03:49Z","due_on":"2015-10-30T07:00:00Z","closed_at":null},"comments":18,"created_at":"2015-02-17T02:40:15Z","updated_at":"2015-10-05T23:11:31Z","closed_at":null,"body":"Kong fetches `Api`, `Application` and `Plugin` data from the datastore on every request. The performance of the system can be improved by implementing either a caching strategy, or an invalidation strategy.\r\n\r\nThe caching strategy is easier to build, but inefficient, since data will be fetched again every `n` seconds, even the data that didn't change.\r\n\r\nThe invalidation strategy is more efficient, but harder to build, since every node needs to invalidate the data, and there needs to be a mechanism that keeps the invalidations in sync on every node. There are two ways of implementing the invalidation strategy:\r\n\r\n* Every time an `Api`, `Application` or `Plugin` is being updated/deleted, the system iterates over all the nodes in the cluster and issues a `PURGE` HTTP request to invalidate the data. This means that each node needs to be aware of other nodes in the cluster, so we need to introduce the concept of *cluster* and *cluster nodes* in the LUA code. This can have performance issues on large node installations, because in a 100-node system that means 100 `PURGE` requests to be sent.\r\n\r\n* Another option is storing the invalidation data in the datastore in an appropriate `invalidations` table. Each node will be responsible for checking periodically the table and only invalidate the data that's being inserted into the table, and storing in the same table the number of nodes that have deleted the data. When the number of nodes that have deleted the data matches the number of nodes in the cluster, then the data is supposed to have been invalidated in every node and can be removed from the table. This can lead problems when the invalidation of the data happens at the same time when new nodes are being added or removed.\r\n\r\nThis issue is a work in progress and more options may be available."},"comment":{"url":"https://api.github.com/repos/Mashape/kong/issues/comments/145695174","html_url":"https://github.com/Mashape/kong/issues/15#issuecomment-145695174","issue_url":"https://api.github.com/repos/Mashape/kong/issues/15","id":145695174,"user":{"login":"thibaultCha","id":1125431,"avatar_url":"https://avatars.githubusercontent.com/u/1125431?v=3","gravatar_id":"","url":"https://api.github.com/users/thibaultCha","html_url":"https://github.com/thibaultCha","followers_url":"https://api.github.com/users/thibaultCha/followers","following_url":"https://api.github.com/users/thibaultCha/following{/other_user}","gists_url":"https://api.github.com/users/thibaultCha/gists{/gist_id}","starred_url":"https://api.github.com/users/thibaultCha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thibaultCha/subscriptions","organizations_url":"https://api.github.com/users/thibaultCha/orgs","repos_url":"https://api.github.com/users/thibaultCha/repos","events_url":"https://api.github.com/users/thibaultCha/events{/privacy}","received_events_url":"https://api.github.com/users/thibaultCha/received_events","type":"User","site_admin":false},"created_at":"2015-10-05T23:11:31Z","updated_at":"2015-10-05T23:11:31Z","body":"Yeah that's my point "}}