using lxml and iterparse() to parse a big (+- 1Gb) XML file

by 
for event, element in etree.iterparse(path_to_file, tag="BlogPost"):
  for child in element:
    print(child.tag, child.text)
    element.clear()

the final clear will stop you from using too much memory.

[update:] to get “everything between … as a string” i guess you want one of:

for event, element in etree.iterparse(path_to_file, tag="BlogPost"):
  print(etree.tostring(element))
  element.clear()

or

for event, element in etree.iterparse(path_to_file, tag="BlogPost"):
  print(''.join([etree.tostring(child) for child in element]))
  element.clear()

or perhaps even:

for event, element in etree.iterparse(path_to_file, tag="BlogPost"):
  print(''.join([child.text for child in element]))
  element.clear()

More Related Contents:

  1. Pretty print in lxml is failing when I add tags to a parsed tree
  2. Why is lxml.etree.iterparse() eating up all my memory?
  3. How to convert an XML file to nice pandas dataframe?
  4. Using Python Iterparse For Large XML Files
  5. how to remove an element in lxml
  6. What is the fastest way to parse large XML docs in Python?
  7. Get all text inside a tag in lxml
  8. How can this function be rewritten to implement OrderedDict? [duplicate]
  9. Remove namespace and prefix from xml in python using lxml
  10. Empty list returned from ElementTree findall
  11. How do I use a default namespace in an lxml xpath query?
  12. Python code to remove HTML tags from a string [duplicate]
  13. Parsing a YAML file in Python, and accessing the data?
  14. Python BeautifulSoup XML Parsing
  15. parsing xml containing default namespace to get an element value using lxml
  16. Parse SGML with Open Arbitrary Tags in Python 3
  17. parsing XML file gets UnicodeEncodeError (ElementTree) / ValueError (lxml)
  18. Simple CSV to XML Conversion – Python
  19. python – lxml: enforcing a specific order for attributes
  20. How to include the namespaces into a xml file using lxml?
  21. Equivalent to InnerHTML when using lxml.html to parse HTML
  22. Comparing two xml files in python
  23. Why does this xpath fail using lxml in python?
  24. Why doesn’t xpath work when processing an XHTML document with lxml (in python)?
  25. Why is ElementTree raising a ParseError?
  26. Parsing broken XML with lxml.etree.iterparse
  27. lxml: add namespace to input file
  28. Parsing XML with namespace in Python via ‘ElementTree’
  29. pandas read_html ValueError: No tables found
  30. How to use regular expression in lxml xpath?

Leave a Comment