{"action":"created","issue":{"url":"https://api.github.com/repos/ss3sim/ss3sim/issues/201","labels_url":"https://api.github.com/repos/ss3sim/ss3sim/issues/201/labels{/name}","comments_url":"https://api.github.com/repos/ss3sim/ss3sim/issues/201/comments","events_url":"https://api.github.com/repos/ss3sim/ss3sim/issues/201/events","html_url":"https://github.com/ss3sim/ss3sim/issues/201","id":57428518,"number":201,"title":"EM re-binning of conditional age-at-length data","user":{"login":"seananderson","id":19349,"avatar_url":"https://avatars.githubusercontent.com/u/19349?v=3","gravatar_id":"","url":"https://api.github.com/users/seananderson","html_url":"https://github.com/seananderson","followers_url":"https://api.github.com/users/seananderson/followers","following_url":"https://api.github.com/users/seananderson/following{/other_user}","gists_url":"https://api.github.com/users/seananderson/gists{/gist_id}","starred_url":"https://api.github.com/users/seananderson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seananderson/subscriptions","organizations_url":"https://api.github.com/users/seananderson/orgs","repos_url":"https://api.github.com/users/seananderson/repos","events_url":"https://api.github.com/users/seananderson/events{/privacy}","received_events_url":"https://api.github.com/users/seananderson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-02-12T07:56:57Z","updated_at":"2015-02-17T01:27:10Z","closed_at":null,"body":"I think I have dynamic re-binning of the conditional age-at-length data working. The meat of the new code is here: \r\n\r\nhttps://github.com/ss3sim/ss3sim/blob/cal-rebin/R/change_em_binning.r#L167-L212\r\n\r\nThe strange structure of the CAL data meant that following the format Kot used for the length re-binning was going to be a huge pain, so I pulled out dplyr.\r\n\r\nThis seems to work with the simple example I coded up, but I'm probably overlooking all sorts of edge cases right now. I'm also not sure how much I can rely on the id column names being the same every time. Maybe it's OK here since the data file will have always gone through change_data first?\r\n\r\nOutput from `dev_example(\"change_em_binning\")`:\r\n```\r\n> f <- system.file(\"extdata\", \"models\", \"cod-om\", \"codOM.dat\", package = \"ss3sim\")\r\n> d <- r4ss::SS_readdat(f)\r\n> # Add catch at length data (and simplify the bin structure for this example)\r\n> datfile <- ss3sim::change_data(d, outfile = NULL, write_file = FALSE,\r\n+   types = c(\"len\", \"age\", \"cal\"), fleets = 1, years = seq(2000, 2002),\r\n+   age_bins = 1:3, len_bins = 4:8)\r\n> datfile$agecomp\r\n     Yr Seas Flt Gender Part AgeErr Lbin_lo Lbin_hi Nsamp a1 a2 a3\r\n1  2000    1   1      0    0      1      -1      -1     3  1  1  1\r\n2  2001    1   1      0    0      1      -1      -1     3  1  1  1\r\n3  2002    1   1      0    0      1      -1      -1     3  1  1  1\r\n4  2000    1   1      0    0      1       1       1     3  1  1  1\r\n5  2000    1   1      0    0      1       2       2     3  1  1  1\r\n6  2000    1   1      0    0      1       3       3     3  1  1  1\r\n7  2000    1   1      0    0      1       4       4     3  1  1  1\r\n8  2000    1   1      0    0      1       5       5     3  1  1  1\r\n9  2001    1   1      0    0      1       1       1     3  1  1  1\r\n10 2001    1   1      0    0      1       2       2     3  1  1  1\r\n11 2001    1   1      0    0      1       3       3     3  1  1  1\r\n12 2001    1   1      0    0      1       4       4     3  1  1  1\r\n13 2001    1   1      0    0      1       5       5     3  1  1  1\r\n14 2002    1   1      0    0      1       1       1     3  1  1  1\r\n15 2002    1   1      0    0      1       2       2     3  1  1  1\r\n16 2002    1   1      0    0      1       3       3     3  1  1  1\r\n17 2002    1   1      0    0      1       4       4     3  1  1  1\r\n18 2002    1   1      0    0      1       5       5     3  1  1  1\r\n> newdat <- change_em_binning(datfile, file_out = NULL, bin_vector = c(4, 6, 8),\r\n+   lbin_method = 1, rebin_cal = TRUE, write_file = FALSE)\r\n> newdat$agecomp\r\n     Yr Seas Flt Gender Part AgeErr Lbin_lo Lbin_hi Nsamp a1 a2 a3\r\n1  2000    1   1      0    0      1      -1      -1     3  1  1  1\r\n2  2001    1   1      0    0      1      -1      -1     3  1  1  1\r\n3  2002    1   1      0    0      1      -1      -1     3  1  1  1\r\n4  2000    1   1      0    0      1       1       1     3  2  2  2\r\n5  2000    1   1      0    0      1       2       2     3  2  2  2\r\n6  2000    1   1      0    0      1       3       3     3  1  1  1\r\n7  2001    1   1      0    0      1       1       1     3  2  2  2\r\n8  2001    1   1      0    0      1       2       2     3  2  2  2\r\n9  2001    1   1      0    0      1       3       3     3  1  1  1\r\n10 2002    1   1      0    0      1       1       1     3  2  2  2\r\n11 2002    1   1      0    0      1       2       2     3  2  2  2\r\n12 2002    1   1      0    0      1       3       3     3  1  1  1\r\n```"},"comment":{"url":"https://api.github.com/repos/ss3sim/ss3sim/issues/comments/74603591","html_url":"https://github.com/ss3sim/ss3sim/issues/201#issuecomment-74603591","issue_url":"https://api.github.com/repos/ss3sim/ss3sim/issues/201","id":74603591,"user":{"login":"colemonnahan","id":4804974,"avatar_url":"https://avatars.githubusercontent.com/u/4804974?v=3","gravatar_id":"","url":"https://api.github.com/users/colemonnahan","html_url":"https://github.com/colemonnahan","followers_url":"https://api.github.com/users/colemonnahan/followers","following_url":"https://api.github.com/users/colemonnahan/following{/other_user}","gists_url":"https://api.github.com/users/colemonnahan/gists{/gist_id}","starred_url":"https://api.github.com/users/colemonnahan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/colemonnahan/subscriptions","organizations_url":"https://api.github.com/users/colemonnahan/orgs","repos_url":"https://api.github.com/users/colemonnahan/repos","events_url":"https://api.github.com/users/colemonnahan/events{/privacy}","received_events_url":"https://api.github.com/users/colemonnahan/received_events","type":"User","site_admin":false},"created_at":"2015-02-17T01:27:10Z","updated_at":"2015-02-17T01:27:10Z","body":"I'm having a look at this now (finally!). \r\n\r\nI originally used `Lbin_method=2` because that was how the original data file I happened to use was set up. In other words, it was by chance. Now that I think about it, I do think we should switch to `Lbin_method=3` so that we can see the actual bins. I think it would be more clear this way, especially when we start doing uneven bins. If we go this route the only thing needing to be changed is in the `make_dummy_data_calcomp` function. If no one objects I will move in that direction.\r\n\r\nSean: I modified your example to have original bins of length 1cm. This is a more realistic scenario since the \"old\" data file represents measured fish at a natural length in their unprocessed form. I don't think anyone measures them to anything other than 1cm, with the possible except of .5cm. So we can focus on those two bins in the original data file.\r\n\r\nThe SS3 manual says that the length bins given are: \r\n> Vector containing lower edge of length bins. The last length bin will have same width as next lower bin\r\n\r\nSo in the case where our new bins are L20, L22, L24, ... L30, the bin 'L20' would contain all fish between 20 and 22cm, which would be l20 and l21 of the original bins, but NOT l22 since that includes fish that are greater than 22cm. When I run the `change_em_binning` function it gives this for lengths (and calcomps but not shown here):\r\n\r\n````\r\nl20\tl21\tl22\tl23\tl24\tl25\tl26\tl27\tl28\tl29\tl30\r\n1\t1\t1\t1\t1\t1\t1\t1\t1\t1\t1\r\n````\r\nbecomes\r\n````\r\nl20\tl22\tl24\tl26\tl28\tl30\r\n3\t2\t3\t2\t2\t1\r\n````\r\nBut shouldn't that be\r\n````\r\nl20\tl22\tl24\tl26\tl28\tl30\r\n2\t2\t2\t2\t2\t1\r\n````\r\nWhere L20=l20+l21; L22=l22+l23, etc., and then L30 is all greater than or equal to it.\r\n\r\n@Kotkot  and @seananderson what are your thoughts? My impression is that the logic should be this: Assume new bin N with length X cm. Let old bin B1 be the largest bin >=X. Let old bin B2 be the largest bin that is strictly less than the next new bin N+1. Sum across bins B1 to B2. Right now we're not handling the boundaries in this way.\r\n"}}