Take the 2-minute tour ×  I have a problem that involves biology
area. Right now I have 4 VERY LARGE files(each with 0.1 billion
lines), but the structure is rather simple, each line of these
files has only 2 fields, both stands for a type of gene.  My
goal is: design an efficient algorithm that can achieves the
following: Find a circle within the contents of these 4 files.
The circle is defined as:  field #1 in a line in file 1 == field
#1 in a line in file 2 and field #2 in a line in file 2 == field
#1 in a line in file 3 and field #2 in a line in file 3 == field
#1 in a line in file 4 and field #2 in a line in file 4 == field
#2 in a line in file 1  I cannot think of a decent way to solve
this, so I just wrote a brute-force-stupid-4-layer-nested loop
for now. I'm thinking about sorting them as alphabetical order,
even if that might help a little, but then it's also obvious
that the computer memory would not allow me to load everything
at once. Can anybody tell me a good way to solve this problem in
a both