Parsing Huge XML Files in PHP

by

There are only two php APIs that are really suited for processing large files. The first is the old expat api, and the second is the newer XMLreader functions. These apis read continuous streams rather than loading the entire tree into memory (which is what simplexml and DOM does).

For an example, you might want to look at this partial parser of the DMOZ-catalog:

<?php

class SimpleDMOZParser
{
    protected $_stack = array();
    protected $_file = "";
    protected $_parser = null;

    protected $_currentId = "";
    protected $_current = "";

    public function __construct($file)
    {
        $this->_file = $file;

        $this->_parser = xml_parser_create("UTF-8");
        xml_set_object($this->_parser, $this);
        xml_set_element_handler($this->_parser, "startTag", "endTag");
    }

    public function startTag($parser, $name, $attribs)
    {
        array_push($this->_stack, $this->_current);

        if ($name == "TOPIC" && count($attribs)) {
            $this->_currentId = $attribs["R:ID"];
        }

        if ($name == "LINK" && strpos($this->_currentId, "Top/Home/Consumer_Information/Electronics/") === 0) {
            echo $attribs["R:RESOURCE"] . "\n";
        }

        $this->_current = $name;
    }

    public function endTag($parser, $name)
    {
        $this->_current = array_pop($this->_stack);
    }

    public function parse()
    {
        $fh = fopen($this->_file, "r");
        if (!$fh) {
            die("Epic fail!\n");
        }

        while (!feof($fh)) {
            $data = fread($fh, 4096);
            xml_parse($this->_parser, $data, feof($fh));
        }
    }
}

$parser = new SimpleDMOZParser("content.rdf.u8");
$parser->parse();

More Related Contents:

  1. Best way to process large XML in PHP [duplicate]
  2. Best XML Parser for PHP [duplicate]
  3. How to use XMLReader in PHP?
  4. preg_match(); – Unknown modifier ‘+’ [duplicate]
  5. Simplify PHP DOM XML parsing – how?
  6. I keep getting these kind of errors in terms of HttpUrlConnection and BackGroundTask in android
  7. Accessing @attribute from SimpleXML
  8. How to evaluate formula passed as string in PHP?
  9. Processing large JSON files in PHP
  10. PHP XML how to output nice format
  11. Reading very large files in PHP
  12. Is there a PDF parser for PHP? [closed]
  13. Parse HTML with PHP’s HTML DOMDocument
  14. How exactly is a PHP script executed?
  15. SOAP PHP fault parsing WSDL: failed to load external entity?
  16. How to solve JSON_ERROR_UTF8 error in php json_decode?
  17. Very large uploads with PHP
  18. php count xml elements
  19. Create new XML file and write data to it?
  20. How to set an Arrays internal pointer to a specific position? PHP/XML
  21. Merge XML files in PHP
  22. using php to download files, not working on large files? [duplicate]
  23. CakePHP Xml utility library triggers DOMDocument warning
  24. simplexml error handling php
  25. PHP SimpleXML Namespace Problem
  26. edit XML with simpleXML
  27. Call to undefined method DOMDocument::getElementsByClassName() [duplicate]
  28. Inspect XML created by PHP SoapClient call before/without sending the request
  29. How to save changed SimpleXML object back to file?
  30. XPath query with PHP

Leave a Comment