Take the 2-minute tour ×  I need to extract a few field contents
from a large XML file. I currently do this though a combination
of xmlstarlet and a Python script (using ElementTree). The idea
was to trim the XML file from useless data with xmlstarlet and
then process the smaller file with Python (using Python directly
on the file was not doable - memory and CPU were hogged and some
files never got processed). It basically works but:    • it is
not efficient   • it is not particularly flexible   • it is
quite ugly (the least of my concerns, but a concern nevertheless
from a maintenance perspective)  I am looking for advice on how
best to handle such a case (the amount of extracted data is
about 5% of the initial file). I am open to anything reasonable
(a specific language, maybe dumping the XML file into a DB and
then extract what I need before dumping the DB?, ...)
share|improve this question  1 Answer 1  up vote 1 down vote
accepted  Are you using ElementTree's iterparse? It should be
able to efficiently h