Parse SGML with Open Arbitrary Tags in Python 3

by

If you can find an SGML DTD for the documents that you work with, a solution could be to use the osx SGML to XML converter from the OpenSP SGML toolkit to turn the documents into XML.

Here is a simple example. Let’s say that we have the following SGML document (company.sgml; with a root element):

<!DOCTYPE ROOT SYSTEM "company.dtd">
<ROOT>
<COMPANY>Awesome Corp
<FORM> 24-7
<ADDRESS>
<STREET>101 PARSNIP LN
<ZIP>31337
</ADDRESS>

The DTD (company.dtd) looks like this:

<!ELEMENT ROOT       -  o (COMPANY, FORM, ADDRESS) >
<!ELEMENT COMPANY    -  o (#PCDATA) >
<!ELEMENT FORM       -  o (#PCDATA) >
<!ELEMENT ADDRESS    -  - (STREET, ZIP) >
<!ELEMENT STREET     -  o (#PCDATA) >
<!ELEMENT ZIP        -  o (#PCDATA) >

The - o bit means that the end tag can be omitted.

The SGML document can be parsed with osx, and the output can be formatted with xmllint, as follows:

osx company.sgml | xmllint --format -

Output from the above command: 

<?xml version="1.0"?>
<ROOT>
  <COMPANY>Awesome Corp</COMPANY>
  <FORM> 24-7</FORM>
  <ADDRESS>
    <STREET>101 PARSNIP LN</STREET>
    <ZIP>31337</ZIP>
  </ADDRESS>
</ROOT>

Now we have well-formed XML that can be processed with lxml or other XML tools.

I don’t know if there is a complete DTD for the document that you link to. The following PDF file contains related information about EDGAR, including a DTD that might be useful: http://www.sec.gov/info/edgar/pdsdissemspec910.pdf (I found it via this answer). But the linked SGML document contains elements (SEC-HEADER, for example) that are not mentioned in the PDF file.

More Related Contents:

  1. Using Python Iterparse For Large XML Files
  2. how to remove an element in lxml
  3. How can this function be rewritten to implement OrderedDict? [duplicate]
  4. Remove namespace and prefix from xml in python using lxml
  5. using lxml and iterparse() to parse a big (+- 1Gb) XML file
  6. builtins.TypeError: must be str, not bytes
  7. How do I use a default namespace in an lxml xpath query?
  8. parsing xml containing default namespace to get an element value using lxml
  9. parsing XML file gets UnicodeEncodeError (ElementTree) / ValueError (lxml)
  10. python – lxml: enforcing a specific order for attributes
  11. How to include the namespaces into a xml file using lxml?
  12. How to install lxml on Windows
  13. Is there a way to get a line number from an ElementTree Element
  14. Why does this xpath fail using lxml in python?
  15. Pretty print in lxml is failing when I add tags to a parsed tree
  16. Why doesn’t xpath work when processing an XHTML document with lxml (in python)?
  17. Why is lxml.etree.iterparse() eating up all my memory?
  18. Parsing broken XML with lxml.etree.iterparse
  19. lxml: add namespace to input file
  20. How to annotate function that takes a tuple of variable length? (variadic tuple type annotation)
  21. python: xml.etree.ElementTree, removing “namespaces”
  22. Customizing the order of legends in plotly
  23. Set encoding in Python 3 CGI scripts
  24. How to write List of lists in csv file in python [duplicate]
  25. Python Pandas – Lookup a variable column depending on another column’s value
  26. With “args” vs without “args” to pass arguments to a thread in Python
  27. How to make print() output UTF-8 in Python 3.0?
  28. Static variable in Python?
  29. Sending multiple files through a TCP socket
  30. Subprocess check_output returned non-zero exit status 1

Leave a Comment