Take the 2-minute tour ×  What I'm asking is rather a
methodology, than a concrete solution. I will start by
describing the situation I found challenging, and will then
proceed to the question. Hope it makes more sense to do it this
way.  I'm dealing with data extracted from natural language.
This data must be later analysed against a sort of "knowledge
base" (it's quoted because it's not really a knowledge base,
I'll get to it later). The knowledge base is large, its volume,
so far theoretically, but soon practically will surpass what is
possible to store in-memory. My two concerns are these:    •
Moving the data over to a database server will mean speed
reduction by a factor... well, I don't know what factor, but it
could be easily several orders of magnitude. I.e. a task of
finding a piece of data in an object native to the runtime
located in memory is significantly faster, then querying the
database.    • The entire huge volume of data is not required at
any one time. In fact, only a very small fraction i