{"action":"created","issue":{"url":"https://api.github.com/repos/thinkaurelius/titan/issues/963","labels_url":"https://api.github.com/repos/thinkaurelius/titan/issues/963/labels{/name}","comments_url":"https://api.github.com/repos/thinkaurelius/titan/issues/963/comments","events_url":"https://api.github.com/repos/thinkaurelius/titan/issues/963/events","html_url":"https://github.com/thinkaurelius/titan/issues/963","id":57083420,"number":963,"title":"Add unit test of titan-hadoop Cassandra widerows, fix spurious row-breaks","user":{"login":"dalaro","id":260045,"avatar_url":"https://avatars.githubusercontent.com/u/260045?v=3","gravatar_id":"","url":"https://api.github.com/users/dalaro","html_url":"https://github.com/dalaro","followers_url":"https://api.github.com/users/dalaro/followers","following_url":"https://api.github.com/users/dalaro/following{/other_user}","gists_url":"https://api.github.com/users/dalaro/gists{/gist_id}","starred_url":"https://api.github.com/users/dalaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dalaro/subscriptions","organizations_url":"https://api.github.com/users/dalaro/orgs","repos_url":"https://api.github.com/users/dalaro/repos","events_url":"https://api.github.com/users/dalaro/events{/privacy}","received_events_url":"https://api.github.com/users/dalaro/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/thinkaurelius/titan/labels/bug","name":"bug","color":"fc2929"}],"state":"closed","locked":false,"assignee":{"login":"dalaro","id":260045,"avatar_url":"https://avatars.githubusercontent.com/u/260045?v=3","gravatar_id":"","url":"https://api.github.com/users/dalaro","html_url":"https://github.com/dalaro","followers_url":"https://api.github.com/users/dalaro/followers","following_url":"https://api.github.com/users/dalaro/following{/other_user}","gists_url":"https://api.github.com/users/dalaro/gists{/gist_id}","starred_url":"https://api.github.com/users/dalaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dalaro/subscriptions","organizations_url":"https://api.github.com/users/dalaro/orgs","repos_url":"https://api.github.com/users/dalaro/repos","events_url":"https://api.github.com/users/dalaro/events{/privacy}","received_events_url":"https://api.github.com/users/dalaro/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/thinkaurelius/titan/milestones/19","labels_url":"https://api.github.com/repos/thinkaurelius/titan/milestones/19/labels","id":924607,"number":19,"title":"Titan0.5.4","description":"","creator":{"login":"mbroecheler","id":758061,"avatar_url":"https://avatars.githubusercontent.com/u/758061?v=3","gravatar_id":"","url":"https://api.github.com/users/mbroecheler","html_url":"https://github.com/mbroecheler","followers_url":"https://api.github.com/users/mbroecheler/followers","following_url":"https://api.github.com/users/mbroecheler/following{/other_user}","gists_url":"https://api.github.com/users/mbroecheler/gists{/gist_id}","starred_url":"https://api.github.com/users/mbroecheler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbroecheler/subscriptions","organizations_url":"https://api.github.com/users/mbroecheler/orgs","repos_url":"https://api.github.com/users/mbroecheler/repos","events_url":"https://api.github.com/users/mbroecheler/events{/privacy}","received_events_url":"https://api.github.com/users/mbroecheler/received_events","type":"User","site_admin":false},"open_issues":5,"closed_issues":5,"state":"open","created_at":"2015-01-06T22:40:55Z","updated_at":"2015-02-10T06:35:00Z","due_on":null,"closed_at":null},"comments":1,"created_at":"2015-02-09T20:15:25Z","updated_at":"2015-02-10T06:35:00Z","closed_at":"2015-02-10T06:35:00Z","body":"ColumnFamilyRecordReader's widerows option is technically honored when provided in titan-hadoop's config since #694, but because TitanCassandraRecordReader incorrectly assumes it will still get all of a row's columns in one call even when widerows is enabled, it's unusuable.  TCRR produces a bunch of fragmentary vertices from distinct chunks of a single row's columns and chaos ensues.\r\n\r\nThe binary-row-to-FaunusVertex logic in `TitanHadoopGraph.readHadoopVertex` may need to be broken into a builder pattern that can accommodate incremental vertex construction as the widerow iterator passes in columns over several calls."},"comment":{"url":"https://api.github.com/repos/thinkaurelius/titan/issues/comments/73651597","html_url":"https://github.com/thinkaurelius/titan/issues/963#issuecomment-73651597","issue_url":"https://api.github.com/repos/thinkaurelius/titan/issues/963","id":73651597,"user":{"login":"dalaro","id":260045,"avatar_url":"https://avatars.githubusercontent.com/u/260045?v=3","gravatar_id":"","url":"https://api.github.com/users/dalaro","html_url":"https://github.com/dalaro","followers_url":"https://api.github.com/users/dalaro/followers","following_url":"https://api.github.com/users/dalaro/following{/other_user}","gists_url":"https://api.github.com/users/dalaro/gists{/gist_id}","starred_url":"https://api.github.com/users/dalaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dalaro/subscriptions","organizations_url":"https://api.github.com/users/dalaro/orgs","repos_url":"https://api.github.com/users/dalaro/repos","events_url":"https://api.github.com/users/dalaro/events{/privacy}","received_events_url":"https://api.github.com/users/dalaro/received_events","type":"User","site_admin":false},"created_at":"2015-02-10T06:35:00Z","updated_at":"2015-02-10T06:35:00Z","body":"I added a test that reads from Titan using widerows=true (3c776d48e6d7d40359c2d7080409fd590d695728).  That broke as expected.  3736b37212d3a9e2278e44e4b070ba02b32cceea is the fix.  The fix just pulls all of the columns into memory, processes each chunk, and then releases that chunk before moving onto the next.  It only passes a FaunusVertex object reflecting all those chunks of columns up to the task layer when it reaches the end of a row or the end of the query iterator.  So, this doesn't save any memory in the task processes -- it's still only intended to avoid thrift frame size exceeded exceptions."}}