{"action":"created","issue":{"url":"https://api.github.com/repos/sherpa/sherpa/issues/38","labels_url":"https://api.github.com/repos/sherpa/sherpa/issues/38/labels{/name}","comments_url":"https://api.github.com/repos/sherpa/sherpa/issues/38/comments","events_url":"https://api.github.com/repos/sherpa/sherpa/issues/38/events","html_url":"https://github.com/sherpa/sherpa/issues/38","id":88502005,"number":38,"title":"Importing Bugs #13545 - grouping twice gives an IndexError exception","user":{"login":"anetasie","id":1213746,"avatar_url":"https://avatars.githubusercontent.com/u/1213746?v=3","gravatar_id":"","url":"https://api.github.com/users/anetasie","html_url":"https://github.com/anetasie","followers_url":"https://api.github.com/users/anetasie/followers","following_url":"https://api.github.com/users/anetasie/following{/other_user}","gists_url":"https://api.github.com/users/anetasie/gists{/gist_id}","starred_url":"https://api.github.com/users/anetasie/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anetasie/subscriptions","organizations_url":"https://api.github.com/users/anetasie/orgs","repos_url":"https://api.github.com/users/anetasie/repos","events_url":"https://api.github.com/users/anetasie/events{/privacy}","received_events_url":"https://api.github.com/users/anetasie/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/sherpa/sherpa/labels/3%20%20-%20Working","name":"3  - Working","color":"ededed"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/area:code","name":"area:code","color":"d4c5f9"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/area:science","name":"area:science","color":"d4c5f9"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/note:can-workaround","name":"note:can-workaround","color":"bfd4f2"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/note:cxc-ticket","name":"note:cxc-ticket","color":"bfd4f2"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/note:need-more-info","name":"note:need-more-info","color":"fef2c0"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/priority:high","name":"priority:high","color":"fad8c7"},{"url":"https://api.github.com/repos/sherpa/sherpa/labels/type:bug","name":"type:bug","color":"e11d21"}],"state":"open","locked":false,"assignee":{"login":"olaurino","id":5692321,"avatar_url":"https://avatars.githubusercontent.com/u/5692321?v=3","gravatar_id":"","url":"https://api.github.com/users/olaurino","html_url":"https://github.com/olaurino","followers_url":"https://api.github.com/users/olaurino/followers","following_url":"https://api.github.com/users/olaurino/following{/other_user}","gists_url":"https://api.github.com/users/olaurino/gists{/gist_id}","starred_url":"https://api.github.com/users/olaurino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olaurino/subscriptions","organizations_url":"https://api.github.com/users/olaurino/orgs","repos_url":"https://api.github.com/users/olaurino/repos","events_url":"https://api.github.com/users/olaurino/events{/privacy}","received_events_url":"https://api.github.com/users/olaurino/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/sherpa/sherpa/milestones/1","html_url":"https://github.com/sherpa/sherpa/milestones/4.8rc1","labels_url":"https://api.github.com/repos/sherpa/sherpa/milestones/1/labels","id":1120770,"number":1,"title":"4.8rc1","description":"","creator":{"login":"olaurino","id":5692321,"avatar_url":"https://avatars.githubusercontent.com/u/5692321?v=3","gravatar_id":"","url":"https://api.github.com/users/olaurino","html_url":"https://github.com/olaurino","followers_url":"https://api.github.com/users/olaurino/followers","following_url":"https://api.github.com/users/olaurino/following{/other_user}","gists_url":"https://api.github.com/users/olaurino/gists{/gist_id}","starred_url":"https://api.github.com/users/olaurino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olaurino/subscriptions","organizations_url":"https://api.github.com/users/olaurino/orgs","repos_url":"https://api.github.com/users/olaurino/repos","events_url":"https://api.github.com/users/olaurino/events{/privacy}","received_events_url":"https://api.github.com/users/olaurino/received_events","type":"User","site_admin":false},"open_issues":14,"closed_issues":20,"state":"open","created_at":"2015-05-20T11:33:14Z","updated_at":"2015-06-26T22:02:41Z","due_on":"2015-10-01T04:00:00Z","closed_at":null},"comments":1,"created_at":"2015-06-15T18:22:35Z","updated_at":"2015-06-29T14:33:38Z","closed_at":null,"body":"This is a bug imported from an older bug tracking system.\r\n\r\nThe bug #13545 shows up in the current version in both CIAO and standalone.\r\nGrouping twice gives an IndexError.\r\n\r\nHere is the current output in my local version.\r\n\r\n~~~~ python\r\n(sherpa-test)13545 $ ipython\r\nPython 2.7.10 |Continuum Analytics, Inc.| (default, May 28 2015, 17:04:42) \r\nType \"copyright\", \"credits\" or \"license\" for more information.\r\n\r\nIPython 3.1.0 -- An enhanced Interactive Python.\r\nAnaconda is brought to you by Continuum Analytics.\r\nPlease check out: http://continuum.io/thanks and https://anaconda.org\r\n?         -> Introduction and overview of IPython's features.\r\n%quickref -> Quick reference.\r\nhelp      -> Python's own help system.\r\nobject?   -> Details about 'object', use 'object??' for extra details.\r\n\r\nIn [1]: from sherpa.astro.ui import *\r\nWARNING: imaging routines will not be available, \r\nfailed to import sherpa.image.ds9_backend due to \r\n'RuntimeErr: DS9Win unusable: Could not find ds9 on your PATH'\r\nWARNING: failed to import sherpa.astro.xspec; XSPEC models will not be available\r\n\r\nIn [2]: load_data(\"xte.pi\")\r\nread ARF file xte.warf\r\nread RMF file xte.wrmf\r\nread ARF (background) file xte_bkg.warf\r\nread RMF (background) file xte_bkg.wrmf\r\nread background file xte_bkg.pi\r\n\r\nIn [3]: notice(0.3,2)\r\n\r\nIn [4]: show_data()\r\nData Set: 1\r\nFilter: 0.2993-1.9929 Energy (keV)\r\nBkg Scale: 0.332619\r\nNoticed Channels: 21-137\r\nname           = xte.pi\r\nchannel        = Float64[1024]\r\ncounts         = Float64[1024]\r\nstaterror      = None\r\nsyserror       = None\r\nbin_lo         = None\r\nbin_hi         = None\r\ngrouping       = None\r\nquality        = None\r\nexposure       = 18513.2782715\r\nbackscal       = 2.18391968294e-07\r\nareascal       = 1.0\r\ngrouped        = False\r\nsubtracted     = False\r\nunits          = energy\r\nrate           = True\r\nplot_fac       = 0\r\nresponse_ids   = [1]\r\nbackground_ids = [1]\r\n\r\nRMF Data Set: 1:1\r\nname     = xte.wrmf\r\ndetchans = 1024\r\nenerg_lo = Float64[1070]\r\nenerg_hi = Float64[1070]\r\nn_grp    = UInt64[1070]\r\nf_chan   = UInt64[1354]\r\nn_chan   = UInt64[1354]\r\nmatrix   = Float64[382802]\r\noffset   = 1\r\ne_min    = Float64[1024]\r\ne_max    = Float64[1024]\r\n\r\nIn [5]: group_counts(30)\r\n\r\nIn [6]: import matplotlib\r\n\r\nIn [7]: plot_data()\r\n\r\nIn [8]: show_data()\r\nData Set: 1\r\nFilter: 0.3322-2.1754 Energy (keV)\r\nBkg Scale: 0.332619\r\nNoticed Channels: 1-165\r\nname           = xte.pi\r\nchannel        = Float64[1024]\r\ncounts         = Float64[1024]\r\nstaterror      = None\r\nsyserror       = None\r\nbin_lo         = None\r\nbin_hi         = None\r\ngrouping       = Float64[1024]\r\nquality        = Float64[1024]\r\nexposure       = 18513.2782715\r\nbackscal       = 2.18391968294e-07\r\nareascal       = 1.0\r\ngrouped        = True\r\nsubtracted     = False\r\nunits          = energy\r\nrate           = True\r\nplot_fac       = 0\r\nresponse_ids   = [1]\r\nbackground_ids = [1]\r\n\r\nRMF Data Set: 1:1\r\nname     = xte.wrmf\r\ndetchans = 1024\r\nenerg_lo = Float64[1070]\r\nenerg_hi = Float64[1070]\r\nn_grp    = UInt64[1070]\r\nf_chan   = UInt64[1354]\r\nn_chan   = UInt64[1354]\r\nmatrix   = Float64[382802]\r\noffset   = 1\r\ne_min    = Float64[1024]\r\ne_max    = Float64[1024]\r\n\r\nARF Data Set: 1:1\r\nname     = xte.warf\r\nenerg_lo = Float64[1070]\r\nenerg_hi = Float64[1070]\r\nspecresp = Float64[1070]\r\nbin_lo   = None\r\nbin_hi   = None\r\nexposure = 18513.2782715\r\n\r\nBackground Data Set: 1:1\r\nFilter: 7.4789 Energy (keV)\r\nNoticed Channels: 1-1024\r\nname           = xte_bkg.pi\r\nchannel        = Float64[1024]\r\ncounts         = Float64[1024]\r\nstaterror      = None\r\nsyserror       = None\r\nbin_lo         = None\r\nbin_hi         = None\r\ngrouping       = Float64[1024]\r\nquality        = Float64[1024]\r\nexposure       = 18513.2782715\r\nbackscal       = 6.56582415104e-07\r\nareascal       = 1.0\r\ngrouped        = True\r\nsubtracted     = False\r\nunits          = energy\r\nrate           = True\r\nplot_fac       = 0\r\nresponse_ids   = [1]\r\nbackground_ids = []\r\n\r\nBackground RMF Data Set: 1:1\r\nname     = xte_bkg.wrmf\r\ndetchans = 1024\r\n\r\nIn [9]: group_counts(15)\r\n---------------------------------------------------------------------------\r\nIndexError                                Traceback (most recent call last)\r\n<ipython-input-9-f2f20366e574> in <module>()\r\n----> 1 group_counts(15)\r\n\r\n<string> in group_counts(id, num, bkg_id, maxLength, tabStops)\r\n\r\n/Users/aneta/git/sherpa/sherpa/astro/ui/utils.pyc in group_counts(self, id, num, bkg_id, maxLength, tabStops)\r\n   7552         if bkg_id is not None:\r\n   7553             data = self.get_bkg(id, bkg_id)\r\n-> 7554         data.group_counts(num, maxLength, tabStops)\r\n   7555 \r\n   7556     ### DOC-TODO: check the Poisson stats claim; I'm guessing it means\r\n\r\n/Users/aneta/git/sherpa/sherpa/astro/data.pyc in group_counts(self, num, maxLength, tabStops)\r\n   1005             bkg = self.get_background(bkg_id)\r\n   1006             if (hasattr(bkg, \"group_counts\")):\r\n-> 1007                 bkg.group_counts(num, maxLength=maxLength, tabStops=tabStops)\r\n   1008 \r\n   1009     ### DOC-TODO: see discussion in astro.ui.utils regarding errorCol\r\n\r\n/Users/aneta/git/sherpa/sherpa/astro/data.pyc in group_counts(self, num, maxLength, tabStops)\r\n   1001             raise ImportErr('importfailed', 'group', 'dynamic grouping')\r\n   1002         self._dynamic_group(pygroup.grpNumCounts, self.counts, num,\r\n-> 1003                             maxLength=maxLength, tabStops=tabStops)\r\n   1004         for bkg_id in self.background_ids:\r\n   1005             bkg = self.get_background(bkg_id)\r\n\r\n/Users/aneta/git/sherpa/sherpa/astro/data.pyc in _dynamic_group(self, group_func, *args, **kwargs)\r\n    837                 kwargs.pop(key)\r\n    838 \r\n--> 839         old_filter = self.get_filter(group=False)\r\n    840         do_notice = numpy.iterable(self.mask)\r\n    841 \r\n\r\n/Users/aneta/git/sherpa/sherpa/astro/data.pyc in get_filter(self, group, format, delim)\r\n   1596             x = x[::-1]\r\n   1597             mask = mask[::-1]\r\n-> 1598         return create_expr(x, mask, format, delim)\r\n   1599 \r\n   1600     def get_filter_expr(self):\r\n\r\n/Users/aneta/git/sherpa/sherpa/utils/__init__.pyc in create_expr(vals, mask, format, delim)\r\n    571             expr.append(format % vals[ii])\r\n    572             expr.append(',')\r\n--> 573     if expr[-1] in (',',delim):\r\n    574         expr.append(format % vals[-1])\r\n    575 \r\n\r\nIndexError: list index out of range\r\n\r\n~~~~\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/sherpa/sherpa/issues/comments/116705991","html_url":"https://github.com/sherpa/sherpa/issues/38#issuecomment-116705991","issue_url":"https://api.github.com/repos/sherpa/sherpa/issues/38","id":116705991,"user":{"login":"olaurino","id":5692321,"avatar_url":"https://avatars.githubusercontent.com/u/5692321?v=3","gravatar_id":"","url":"https://api.github.com/users/olaurino","html_url":"https://github.com/olaurino","followers_url":"https://api.github.com/users/olaurino/followers","following_url":"https://api.github.com/users/olaurino/following{/other_user}","gists_url":"https://api.github.com/users/olaurino/gists{/gist_id}","starred_url":"https://api.github.com/users/olaurino/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olaurino/subscriptions","organizations_url":"https://api.github.com/users/olaurino/orgs","repos_url":"https://api.github.com/users/olaurino/repos","events_url":"https://api.github.com/users/olaurino/events{/privacy}","received_events_url":"https://api.github.com/users/olaurino/received_events","type":"User","site_admin":false},"created_at":"2015-06-29T14:33:38Z","updated_at":"2015-06-29T14:33:38Z","body":"@anetasie I can reproduce this. A simple workaround is to call `ungroup()` before grouping again.\r\n\r\nI guess it makes sense that data should be ungrouped before it can be regrouped, it this correct?\r\n\r\nIf it is, I need more information regarding the fix: the simplest fix I can think of involves refusing to apply a new group function on a dataset that has already been grouped, and showing the user a WARNING.\r\n\r\nAn alternative involves automatically calling `ungroup()` if a group function is called and the dataset has already been grouped.\r\n\r\nI can also see possible combinations of these two possibilities (e.g. refuse to group a grouped dataset, but accept a keyword argument to force the application of `ungroup()`).\r\n\r\nWhat do you think?"}}