How to iterate through very large text files and merge lines only when a certain condition exists
<p>The underlying purpose is for me to become a python expert (long run) and the immediate goal is as follows...</p>

<p>I want to merge two massive lists into one. The lists are very large text files comprised of millions of lines which would look like the following- </p>

<pre><code>         bigfile1                        bigfile2           
 (10,'red','blue','orange')         (10,'31','false','true')                         
 (11,'black','blue','green')        (11,'88','true','true')
 (12,'blue','blue','green')         random junk once in a while
 (13,'red','blue','yellow')         (12,'3','false','false')
 (14,'brown','red','red')           (15,'6','true','true')
</code></pre>

<p>Using Python, I would like to:</p>

<ol>
<li>merge the lines from each list and write them to a new list if the       "usernumbers" before the first comma are the same.</li>
<li>Have program complete before our sun runs out of
hydrogen, so that ruled out iterating every line through every line.</li>
</ol>

<p>I then learned about (a,b) in zip but then i had a new problem. I want to merge the lines from the lists only when the first number in the lines are the same...and within the lists, some numbers are skipped in one list, duplicates, garbled trash once in a while, etc. so I can't just go line by line and can't figure out if there's a way to iterate through <strong>only</strong> a or b when using zip. I realize these are files that should be in a database and you query from there, but it's an exercise for me to learn more Python. </p>

<p>I'm using Python3.4 on windows. If anyone has suggestions on completing the following, or start from scratch, I would greatly appreciate it!</p>

<p>I want the lines with same usernumbers to be merged together in a new file. My current code follows: </p>

<pre><code>list1 = open('bigfile1.txt', 'r', errors = 'ignore')    
list2 = open('bigfile2.txt', 'r', errors = 'ignore')

for a,b in zip(list1,list2):   

    c = (''.join(a.split("(")[1:])).rstrip()
    d = ''.join(c.split(",")[:1])

    e = (''.join(b.split("(")[1:])).rstrip()
    f = ''.join(e.split(",")[:1])

    if d == f:
        #FILE.write()
        print (a,b)

    elif d != f:
#####  I'm STUCK!!  #####

#FILE.close()
list1.close()
list2.close()        
</code></pre>