{"action":"opened","issue":{"url":"https://api.github.com/repos/bioperl/bioperl-db/issues/3","labels_url":"https://api.github.com/repos/bioperl/bioperl-db/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/bioperl/bioperl-db/issues/3/comments","events_url":"https://api.github.com/repos/bioperl/bioperl-db/issues/3/events","html_url":"https://github.com/bioperl/bioperl-db/issues/3","id":110342552,"number":3,"title":"crashes on downloading NCBI records","user":{"login":"cjfields","id":49265,"avatar_url":"https://avatars.githubusercontent.com/u/49265?v=3","gravatar_id":"","url":"https://api.github.com/users/cjfields","html_url":"https://github.com/cjfields","followers_url":"https://api.github.com/users/cjfields/followers","following_url":"https://api.github.com/users/cjfields/following{/other_user}","gists_url":"https://api.github.com/users/cjfields/gists{/gist_id}","starred_url":"https://api.github.com/users/cjfields/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cjfields/subscriptions","organizations_url":"https://api.github.com/users/cjfields/orgs","repos_url":"https://api.github.com/users/cjfields/repos","events_url":"https://api.github.com/users/cjfields/events{/privacy}","received_events_url":"https://api.github.com/users/cjfields/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bioperl/bioperl-db/labels/Priority:%20Normal","name":"Priority: Normal","color":"666600"},{"url":"https://api.github.com/repos/bioperl/bioperl-db/labels/Status:%20New","name":"Status: New","color":"FF6600"},{"url":"https://api.github.com/repos/bioperl/bioperl-db/labels/Tracker:%20Bug","name":"Tracker: Bug","color":"FF0000"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-10-07T23:47:42Z","updated_at":"2015-10-07T23:47:43Z","closed_at":null,"body":"---\n\n\nAuthor Name: **Dmitry Samborskiy** (Dmitry Samborskiy)\nOriginal Redmine Issue: 2213, https://redmine.open-bio.org/issues/2213\nOriginal Date: 2007-02-20\nOriginal Assignee: Bioperl Guts\n\n---\n\nHi All,\n\nI tried to use modern BioPerl-DB code for storing NCBI virus genome records in MySQL database (I used to use BioPerl-1.0 before).\n\nI prepare a fresh BioSQL database with loaded NCBI taxonomy database.\nThen I try to store some genomes.\n\nTo my surprise it crashes with following diagnostic:\n\nCould not store AY593801:\n——————- EXCEPTION ——————-\nMSG: The supplied lineage does not start near ‘Foot-and-mouth disease virus - type A’ (I was supplied ‘Foot-and-mouth disease virus | Aphthovirus | Picornaviridae’)\nSTACK Bio::Species::classification /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/Species.pm:174\nSTACK Bio::DB::Persistent::PersistentObject::AUTOLOAD /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/Persistent/PersistentObject.pm:552\nSTACK Bio::DB::BioSQL::SpeciesAdaptor::populate\\_from\\_row /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/SpeciesAdaptor.pm:281\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::\\_build\\_object /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:1305\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::\\_find\\_by\\_unique\\_key /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:973\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::find\\_by\\_unique\\_key /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:852\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::create /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:182\nSTACK Bio::DB::Persistent::PersistentObject::create /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/Persistent/PersistentObject.pm:244\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::create /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:169\nSTACK Bio::DB::BioSQL::BasePersistenceAdaptor::store /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/BioSQL/BasePersistenceAdaptor.pm:251\nSTACK Bio::DB::Persistent::PersistentObject::store /tmp/perl/lib/perl5/site\\_perl/5.8.6/Bio/DB/Persistent/PersistentObject.pm:271\nSTACK (eval) ./load\\_seqdatabase.pl:620\nSTACK toplevel ./load\\_seqdatabase.pl:602\n\n------------------------------------------------------------------------\n\nThe AY593801 genome was downloaded recently from NCBI site.\nIn my script I used only standard scripts from BioPerl-DB/BioSQL packages\nthat makes all the test steps (db creation/taxonomy loading/sequence loading /…). See the attachment archive.\n\nIt seems that the problem originates from the fact that\nAY593801’s taxonomy (taxon:12111) has a special form:\n\nmysql\\> SELECT name.name, node.node\\_rank FROM taxon node, taxon taxon, taxon\\_name name WHERE name.taxon\\_id = node.taxon\\_id AND taxon.left\\_value BETWEEN node.left\\_value AND node.right\\_value AND taxon.taxon\\_id = 12111 AND name.name\\_class = ‘scientific name’ ORDER BY node.left\\_value;\n*——————————————————————-*—————-+\n| name | node\\_rank |\n*——————————————————————-*—————-+\n| root | no rank |\n| Viruses | no rank |\n| ssRNA positive-strand viruses, no DNA stage | no rank |\n| Picornaviridae | family |\n| Aphthovirus | genus |\n| Foot-and-mouth disease virus | species |\n| Foot-and-mouth disease virus - type A | no rank |\n*——————————————————————-*—————-+\n7 rows in set (0.30 sec)\n\nOne can see that “Foot-and-mouth disease virus” is a prefix of\n“Foot-and-mouth disease virus - type A”.\nI found that in all these cases BioPerl-DB crashes.\n\nApparently, it’s unexpected consequence of the “massaging” trick in the\nSpeciesAdaptor::populate\\_from\\_row method. Indeed, the problem has gone\nwhen I commented out the trick code and commented accordingly the\nspecies correction in the mysql::SpeciesAdaptorDriver::insert\\_object method.\nSee my patch code:\n\n—- Bio/DB/BioSQL/SpeciesAdaptor.pm.ORIG 2006-12-06 19:46:51.000000000 +0\n300\n**+ Bio/DB/BioSQL/SpeciesAdaptor.pm 2007-02-20 23:08:44.000000000 +0300\n`@ -256,6 +256,8 `@\n while($clf-\\>[0]~~\\>[1] && ($clf~~\\>[0]-\\>[1] eq “no rank”)) {\n shift(`$clf);\n            }\n+# skip this corrections (see also mysql/SpeciesAdaptorDriver::insert_object):\n+=pod\n            # in the species object we store the species element without the\n            # genus, and similarly for the sub-species and variant\n            for(my $i = scalar(`$clf)–2; $i \\>= 0; $i—) {\n`@ -278,6 +280,7 `@\n $rank = $clf-\\>[scalar(`$clf)-1]->[1];\n            }\n            # done massaging, store away\n+=cut\n            $obj->classification([reverse(map { $_->[0]; } `$clf)], “FORCE”);\n }\n if($rows-\\>[4] && (! $obj-\\>classification)) {\n—- Bio/DB/BioSQL/SpeciesAdaptorDriver.pm.ORIG 2006-12-06 19:46:51.000000000 +0\n300\n**+ Bio/DB/BioSQL/SpeciesAdaptorDriver.pm 2007-02-20 23:10:00.000000000 +0\n300\n`@ -312,7 +312,8 `@\n $clf[0]-\\>[1] = “species”;\n $clf[1]-\\>[1] = “genus”;\n \\# also, convention is species to equal the binomial, not just first name\n- $clf[0]~~\\>[0] = $obj~~\\>binomial();\n+ \\# Don’t do that (see also SpeciesAdaptor::populate\\_from\\_row):\n+ \\#$clf[0]~~\\>[0] = $obj~~\\>binomial();\n \\# sub-species and variant are to be prepended, also as full names\n if($obj-\\>sub\\_species) {\n unshift(@clf, [$obj-\\>binomial() .\" \". $obj-\\>sub\\_species(),\n\nAnother one crash case occures with my old NC\\_001846 record which was\ndownloaded few years ago (see NC\\_001846.seq file in the archive).\nIn this case ORGANISM line doesn’t match NCBI taxonomy for the supplied\ntaxon:11138 (it did correspond several years ago, though).\n\nPerhaps, BioPerl-DB code is not intelligent enough to find and fix this\ndiscrepancy. And my patch does NOT help there.\n\nMany thanks in advance.\n\nBest regards,\n Dmitry Samborskiy\n\n\n\n\n"}}