{"action":"created","issue":{"url":"https://api.github.com/repos/cjfields/redmine-test/issues/37","labels_url":"https://api.github.com/repos/cjfields/redmine-test/issues/37/labels{/name}","comments_url":"https://api.github.com/repos/cjfields/redmine-test/issues/37/comments","events_url":"https://api.github.com/repos/cjfields/redmine-test/issues/37/events","html_url":"https://github.com/cjfields/redmine-test/issues/37","id":107444536,"number":37,"title":"crashes on downloading NCBI records","user":{"login":"cjfields","id":49265,"avatar_url":"https://avatars.githubusercontent.com/u/49265?v=3","gravatar_id":"","url":"https://api.github.com/users/cjfields","html_url":"https://github.com/cjfields","followers_url":"https://api.github.com/users/cjfields/followers","following_url":"https://api.github.com/users/cjfields/following{/other_user}","gists_url":"https://api.github.com/users/cjfields/gists{/gist_id}","starred_url":"https://api.github.com/users/cjfields/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cjfields/subscriptions","organizations_url":"https://api.github.com/users/cjfields/orgs","repos_url":"https://api.github.com/users/cjfields/repos","events_url":"https://api.github.com/users/cjfields/events{/privacy}","received_events_url":"https://api.github.com/users/cjfields/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cjfields/redmine-test/labels/Priority:%20Normal","name":"Priority: Normal","color":"666600"},{"url":"https://api.github.com/repos/cjfields/redmine-test/labels/Status:%20New","name":"Status: New","color":"FF6600"},{"url":"https://api.github.com/repos/cjfields/redmine-test/labels/Tracker:%20Bug","name":"Tracker: Bug","color":"FF0000"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-09-21T04:55:53Z","updated_at":"2015-09-21T04:56:17Z","closed_at":null,"body":"---\n\n\nAuthor Name: **Dmitry Samborskiy** (Dmitry Samborskiy)\nOriginal Redmine Issue: 2213, https://redmine.open-bio.org/issues/2213\nOriginal Date: 2007-02-20\nOriginal Assignee: Bioperl Guts\n\n---\n\nHi All,\n\nI tried to use modern BioPerl-DB code for storing NCBI virus genome records in MySQL database (I used to use BioPerl-1.0 before).\n\nI prepare a fresh BioSQL database with loaded NCBI taxonomy database.\nThen I try to store some genomes.\n\nTo my surprise it crashes with following diagnostic:\n\nCould not store AY593801:\n——————- EXCEPTION ——————-\nMSG: The supplied lineage does not start near ‘Foot-and-mouth disease virus - type A’ (I was supplied ‘Foot-and-mouth disease virus | Aphthovirus | Picornaviridae’)\nSTACK Bio::Species::classification /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/Species.pm:174\nSTACK Bio::DB::Persistent::PersistentObject::AUTOLOAD /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/Persistent/PersistentObject.pm:552\nSTACK Bio::DB::BioSQL::SpeciesAdaptor::populate\\_from\\_row /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/SpeciesAdaptor.pm:281\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::\\_build\\_object /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:1305\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::\\_find\\_by\\_unique\\_key /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:973\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::find\\_by\\_unique\\_key /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:852\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::create /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:182\nSTACK Bio::DB::Persistent::PersistentObject::create /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/Persistent/PersistentObject.pm:244\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::create /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:169\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::store /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:251\nSTACK Bio::DB::Persistent::PersistentObject::store /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/Persistent/PersistentObject.pm:271\nSTACK (eval) ./load\\_seqdatabase.pl:620\nSTACK toplevel ./load\\_seqdatabase.pl:602\n\n------------------------------------------------------------------------\n\nThe AY593801 genome was downloaded recently from NCBI site.\nIn my script I used only standard scripts from BioPerl-DB/BioSQL packages\nthat makes all the test steps (db creation/taxonomy loading/sequence loading /…). See the attachment archive.\n\nIt seems that the problem originates from the fact that\nAY593801’s taxonomy (taxon:12111) has a special form:\n\nmysql\\> SELECT name.name, node.node\\_rank FROM taxon node, taxon taxon, taxon\\_name name WHERE name.taxon\\_id = node.taxon\\_id AND taxon.left\\_value BETWEEN node.left\\_value AND node.right\\_value AND taxon.taxon\\_id = 12111 AND name.name\\_class = ‘scientific name’ ORDER BY node.left\\_value;\n*——————————————————————-*—————-+\n| name | node\\_rank |\n*——————————————————————-*—————-+\n| root | no rank |\n| Viruses | no rank |\n| ssRNA positive-strand viruses, no DNA stage | no rank |\n| Picornaviridae | family |\n| Aphthovirus | genus |\n| Foot-and-mouth disease virus | species |\n| Foot-and-mouth disease virus - type A | no rank |\n*——————————————————————-*—————-+\n7 rows in set (0.30 sec)\n\nOne can see that “Foot-and-mouth disease virus” is a prefix of\n“Foot-and-mouth disease virus - type A”.\nI found that in all these cases BioPerl-DB crashes.\n\nApparently, it’s unexpected consequence of the “massaging” trick in the\nSpeciesAdaptor::populate\\_from\\_row method. Indeed, the problem has gone\nwhen I commented out the trick code and commented accordingly the\nspecies correction in the mysql::SpeciesAdaptorDriver::insert\\_object method.\nSee my patch code:\n\n—- Bio/DB/BioSQL/SpeciesAdaptor.pm.ORIG 2006-12-06 19:46:51.000000000 +0\n300\n**+ Bio/DB/BioSQL/SpeciesAdaptor.pm 2007-02-20 23:08:44.000000000 +0300\n`@ -256,6 +256,8 `@\n while($clf-\\>[0]~~\\>[1] && ($clf~~\\>[0]-\\>[1] eq “no rank”)) {\n shift(`$clf);\n            }\n+# skip this corrections (see also mysql/SpeciesAdaptorDriver::insert_object):\n+=pod\n            # in the species object we store the species element without the\n            # genus, and similarly for the sub-species and variant\n            for(my $i = scalar(`$clf)–2; $i \\>= 0; $i—) {\n`@ -278,6 +280,7 `@\n $rank = $clf-\\>[scalar(`$clf)-1]->[1];\n            }\n            # done massaging, store away\n+=cut\n            $obj->classification([reverse(map { $_->[0]; } `$clf)], “FORCE”);\n }\n if($rows-\\>[4] && (! $obj-\\>classification)) {\n—- Bio/DB/BioSQL/SpeciesAdaptorDriver.pm.ORIG 2006-12-06 19:46:51.000000000 +0\n300\n**+ Bio/DB/BioSQL/SpeciesAdaptorDriver.pm 2007-02-20 23:10:00.000000000 +0\n300\n`@ -312,7 +312,8 `@\n $clf[0]-\\>[1] = “species”;\n $clf[1]-\\>[1] = “genus”;\n \\# also, convention is species to equal the binomial, not just first name\n- $clf[0]~~\\>[0] = $obj~~\\>binomial();\n+ \\# Don’t do that (see also SpeciesAdaptor::populate\\_from\\_row):\n+ \\#$clf[0]~~\\>[0] = $obj~~\\>binomial();\n \\# sub-species and variant are to be prepended, also as full names\n if($obj-\\>sub\\_species) {\n unshift(@clf, [$obj-\\>binomial() .\" \". $obj-\\>sub\\_species(),\n\nAnother one crash case occures with my old NC\\_001846 record which was\ndownloaded few years ago (see NC\\_001846.seq file in the archive).\nIn this case ORGANISM line doesn’t match NCBI taxonomy for the supplied\ntaxon:11138 (it did correspond several years ago, though).\n\nPerhaps, BioPerl-DB code is not intelligent enough to find and fix this\ndiscrepancy. And my patch does NOT help there.\n\nMany thanks in advance.\n\nBest regards,\n Dmitry Samborskiy\n\n\n\n\n"},"comment":{"url":"https://api.github.com/repos/cjfields/redmine-test/issues/comments/141877960","html_url":"https://github.com/cjfields/redmine-test/issues/37#issuecomment-141877960","issue_url":"https://api.github.com/repos/cjfields/redmine-test/issues/37","id":141877960,"user":{"login":"cjfields","id":49265,"avatar_url":"https://avatars.githubusercontent.com/u/49265?v=3","gravatar_id":"","url":"https://api.github.com/users/cjfields","html_url":"https://github.com/cjfields","followers_url":"https://api.github.com/users/cjfields/followers","following_url":"https://api.github.com/users/cjfields/following{/other_user}","gists_url":"https://api.github.com/users/cjfields/gists{/gist_id}","starred_url":"https://api.github.com/users/cjfields/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cjfields/subscriptions","organizations_url":"https://api.github.com/users/cjfields/orgs","repos_url":"https://api.github.com/users/cjfields/repos","events_url":"https://api.github.com/users/cjfields/events{/privacy}","received_events_url":"https://api.github.com/users/cjfields/received_events","type":"User","site_admin":false},"created_at":"2015-09-21T04:56:17Z","updated_at":"2015-09-21T04:56:17Z","body":"---\n\nOriginal Redmine Comment\nAuthor Name: **Dmitry Samborskiy** (Dmitry Samborskiy)\nOriginal Date: 2007-02-23T03:54:08Z\n\n---\n\n(In reply to comment \\#6)\n\\> First, you must remember that rel. 1.5.2 is still considered a developer\n\\> release. bioperl-db is also still considered under active development, though\n\\> it should be stable enough for most purposes thanks to Hilmar.\n\nYou are right. Indeed I assumed 1.5.2 to be a stable release, the “Bioperl 1.5.2 has been released!” message was so inviting (http://www.bioperl.org/wiki/Main\\_Page) that I’ve overlooked “developer release” warning. It was also confusing that the last version marked as really stable was released almost 4 years ago. So, I decided that 1.5.2 is a milestone release which is ready to use…\n\nThank you for your explanation, I’ll try to read thoroughly the code and the threads you mentioned.\nI think for now I’d better to use my correction until the problem will be solved\nby BioPerl community. Then I could fix taxonomy records in our database if the final solution will differ from my solution.\n\nMany thanks for your attention!\n\n"}}