{"action":"created","issue":{"url":"https://api.github.com/repos/DIRACGrid/DIRAC/issues/2607","labels_url":"https://api.github.com/repos/DIRACGrid/DIRAC/issues/2607/labels{/name}","comments_url":"https://api.github.com/repos/DIRACGrid/DIRAC/issues/2607/comments","events_url":"https://api.github.com/repos/DIRACGrid/DIRAC/issues/2607/events","html_url":"https://github.com/DIRACGrid/DIRAC/pull/2607","id":106923813,"number":2607,"title":"[v6r16] new data logging","user":{"login":"chaen","id":3728211,"avatar_url":"https://avatars.githubusercontent.com/u/3728211?v=3","gravatar_id":"","url":"https://api.github.com/users/chaen","html_url":"https://github.com/chaen","followers_url":"https://api.github.com/users/chaen/followers","following_url":"https://api.github.com/users/chaen/following{/other_user}","gists_url":"https://api.github.com/users/chaen/gists{/gist_id}","starred_url":"https://api.github.com/users/chaen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chaen/subscriptions","organizations_url":"https://api.github.com/users/chaen/orgs","repos_url":"https://api.github.com/users/chaen/repos","events_url":"https://api.github.com/users/chaen/events{/privacy}","received_events_url":"https://api.github.com/users/chaen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-09-17T07:02:56Z","updated_at":"2015-11-12T07:15:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/DIRACGrid/DIRAC/pulls/2607","html_url":"https://github.com/DIRACGrid/DIRAC/pull/2607","diff_url":"https://github.com/DIRACGrid/DIRAC/pull/2607.diff","patch_url":"https://github.com/DIRACGrid/DIRAC/pull/2607.patch"},"body":"Replaces https://github.com/DIRACGrid/DIRAC/pull/2575\r\nThis is the data logging system for DIRAC. It allows to track all operations on files. It is transparent for users, it does not change the behavior of DIRAC. It is flexible for future additional.\r\n\r\nHere is an example of the logging of a put and register operation and of a remove file operation :\r\n$ python dirac/DIRAC/DataManagementSystem/scripts/dirac-dls-sequence.py /lhcb/sub/file2.data\r\n\r\nSequence 1, Caller dirac-dms-add-file.py , UserName coberger , HostName localhost.localdomain , Group devGroup\r\nDataManager.putAndRegister, 2015-08-19 11:50:13, Successful, TargetSE StorageTest1\r\nDataManager.registerFile, 2015-08-19 11:50:14, Successful, TargetSE StorageTest1\r\nFileCatalog.addFile, 2015-08-19 11:50:14, Successful, TargetSE StorageTest1\r\nStorageElement.putFile, 2015-08-19 11:50:14, Successful, TargetSE StorageTest1\r\n\r\nSequence 2, Caller dirac-dms-remove-files.py , UserName coberger , HostName localhost.localdomain , Group devGroup\r\nDataManager.removeFile, 2015-08-19 12:38:20, Successful\r\nFileCatalog.removeFile, 2015-08-19 12:38:20, Successful\r\nFileCatalog.removeReplica, 2015-08-19 12:38:20, Successful, TargetSE StorageTest1\r\nStorageElement.removeFile, 2015-08-19 12:38:20, Successful, TargetSE StorageTest1\r\n\r\nIt is also possible to have all method calls about one LFN , here is the output:\r\n\r\n$ python dirac/DIRAC/DataManagementSystem/scripts/dirac-dls-method-call.py /lhcb/sub/file2.data\r\n\r\nFound 8 method calls\r\n2015-08-19 11:50:13, DataManager.putAndRegister, TargetSE StorageTest1, Successful, SequenceID 1\r\n2015-08-19 11:50:14, DataManager.registerFile, TargetSE StorageTest1, Successful, SequenceID 1\r\n2015-08-19 11:50:14, StorageElement.putFile, TargetSE StorageTest1, Successful, SequenceID 1\r\n2015-08-19 11:50:14, FileCatalog.addFile, TargetSE StorageTest1, Successful, SequenceID 1\r\n2015-08-19 12:38:20, FileCatalog.removeFile, Successful, SequenceID 2\r\n2015-08-19 12:38:20, DataManager.removeFile, Successful, SequenceID 2\r\n2015-08-19 12:38:20, StorageElement.removeFile, TargetSE StorageTest1, Successful, SequenceID 2\r\n2015-08-19 12:38:20, FileCatalog.removeReplica, TargetSE StorageTest1, Successful, SequenceID 2\r\n\r\nIt is possible to get method calls for one method name, it is also possible to precise a date before, a date after or an operation Status.\r\n\r\nFor sequences, it is possible to get all sequences for an LFN, a user name, a group name, an host name, etc."},"comment":{"url":"https://api.github.com/repos/DIRACGrid/DIRAC/issues/comments/156020957","html_url":"https://github.com/DIRACGrid/DIRAC/pull/2607#issuecomment-156020957","issue_url":"https://api.github.com/repos/DIRACGrid/DIRAC/issues/2607","id":156020957,"user":{"login":"chaen","id":3728211,"avatar_url":"https://avatars.githubusercontent.com/u/3728211?v=3","gravatar_id":"","url":"https://api.github.com/users/chaen","html_url":"https://github.com/chaen","followers_url":"https://api.github.com/users/chaen/followers","following_url":"https://api.github.com/users/chaen/following{/other_user}","gists_url":"https://api.github.com/users/chaen/gists{/gist_id}","starred_url":"https://api.github.com/users/chaen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chaen/subscriptions","organizations_url":"https://api.github.com/users/chaen/orgs","repos_url":"https://api.github.com/users/chaen/repos","events_url":"https://api.github.com/users/chaen/events{/privacy}","received_events_url":"https://api.github.com/users/chaen/received_events","type":"User","site_admin":false},"created_at":"2015-11-12T07:15:18Z","updated_at":"2015-11-12T07:15:18Z","body":"Yes, for the first point it is obvious, and I meant to add that. For the two others, we can certainly use the FC filtering mechanism I implemented.\r\nBtw, scalability tests were done "}}