{"action":"closed","issue":{"url":"https://api.github.com/repos/dmwm/DAS/issues/3873","labels_url":"https://api.github.com/repos/dmwm/DAS/issues/3873/labels{/name}","comments_url":"https://api.github.com/repos/dmwm/DAS/issues/3873/comments","events_url":"https://api.github.com/repos/dmwm/DAS/issues/3873/events","html_url":"https://github.com/dmwm/DAS/issues/3873","id":5809808,"number":3873,"title":"Add attribute look-up via auto-learning of entity attributes","user":{"login":"vkuznet","id":125138,"avatar_url":"https://avatars.githubusercontent.com/u/125138?v=3","gravatar_id":"","url":"https://api.github.com/users/vkuznet","html_url":"https://github.com/vkuznet","followers_url":"https://api.github.com/users/vkuznet/followers","following_url":"https://api.github.com/users/vkuznet/following{/other_user}","gists_url":"https://api.github.com/users/vkuznet/gists{/gist_id}","starred_url":"https://api.github.com/users/vkuznet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vkuznet/subscriptions","organizations_url":"https://api.github.com/users/vkuznet/orgs","repos_url":"https://api.github.com/users/vkuznet/repos","events_url":"https://api.github.com/users/vkuznet/events{/privacy}","received_events_url":"https://api.github.com/users/vkuznet/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dmwm/DAS/labels/research%20task","name":"research task","color":"cc66ff"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2012-06-21T23:18:31Z","updated_at":"2015-02-04T01:01:58Z","closed_at":"2015-02-04T01:01:58Z","body":"**Tasks to be done:**\r\n\r\n- [ ] keylearning shall ignore fields that are part of input query\r\n  * e.g. file run=123 shall not crate run.number as part of file entity\r\n\r\n\r\n\r\n\r\n\r\n\r\nIf we'll add auto-learning entity attribute feature, i.e. analyze output of entity look-up, then it would be possible to perform attribute look-up in DAS query. For example, dataset=/a/b/c produce dataset DAS record. This record has different attributes, e.g. tag, nevents, etc. If we'll store entity attributes in separate collection (we'll learn it), then we can add attribute queries like\r\ntag dataset=/a/b/c\r\nnevents dataset=/a/b/c\r\netc.\r\nwhich is more intuitive than dataset=/a/b/c | grep dataset.<att>\r\n\r\nThis can be done by getting all keys, see web/cms_representation.py fltpage function:\r\n{{{\r\npkey = row['das']['primary_key'] # primary key\r\nmkey = pkey.split('.')[0]\r\nrowkeys = [k for k in DotDict(row).get_keys(mkey)]\r\n}}}\r\nand storing those as entity record, e.g. \r\n{{{\r\n{'dataset':['dataset.nevents', 'dataset.tag', 'dataset.nfiles', ...]}\r\n}}}\r\nand storing them into dedicated collection. Then, the DAS query class can look-up the keys and create proper DAS query via DAS filters. For example, user place the query\r\n{{{\r\ntag dataset=/a/b/c\r\n}}}\r\nwe found in attribute collection that tag is an attribute of dataset, so we can construct DAS MongoDB query as \r\n{{{\r\n{'dataset':'/a/b/c', 'filters':{'grep':'dataset.tag'}}\r\n}}}\r\nThe problem will be to identically construct query hash, e.g. qhash(tag dataset=abc) == qhash(dataset=abc | grep dataset.tag)"}}