How to parse an XML file to an R data frame?

by

Data in XML format are rarely organized in a way that would allow the xmlToDataFrame function to work. You’re better off extracting everything in lists and then binding the lists together in a data frame:

require(XML)
data <- xmlParse("http://forecast.weather.gov/MapClick.php?lat=29.803&lon=-82.411&FcstType=digitalDWML")

xml_data <- xmlToList(data)

In the case of your example data, getting location and start time is fairly straightforward:

location <- as.list(xml_data[["data"]][["location"]][["point"]])

start_time <- unlist(xml_data[["data"]][["time-layout"]][
    names(xml_data[["data"]][["time-layout"]]) == "start-valid-time"])

Temperature data is a bit more complicated. First you need to get to the node that contains the temperature lists. Then you need extract both the lists, look within each one, and pick the one that has “hourly” as one of its values. Then you need to select only that list but only keep the values that have the “value” label:

temps <- xml_data[["data"]][["parameters"]]
temps <- temps[names(temps) == "temperature"]
temps <- temps[sapply(temps, function(x) any(unlist(x) == "hourly"))]
temps <- unlist(temps[[1]][sapply(temps, names) == "value"])

out <- data.frame(
  as.list(location),
  "start_valid_time" = start_time,
  "hourly_temperature" = temps)

head(out)
  latitude longitude          start_valid_time hourly_temperature
1    29.81    -82.42 2013-06-19T16:00:00-04:00                 91
2    29.81    -82.42 2013-06-19T17:00:00-04:00                 90
3    29.81    -82.42 2013-06-19T18:00:00-04:00                 89
4    29.81    -82.42 2013-06-19T19:00:00-04:00                 85
5    29.81    -82.42 2013-06-19T20:00:00-04:00                 83
6    29.81    -82.42 2013-06-19T21:00:00-04:00                 80

More Related Contents:

  1. How to parse XML to R data frame
  2. Scrape password-protected website in R
  3. R: convert XML data to data frame
  4. Combine values in huge XML-files
  5. How to transform XML data into a data.frame?
  6. How to webscrape secured pages in R (https links) (using readHTMLTable from XML package)?
  7. extract data from raw html in R
  8. Unable to install R package in Ubuntu 11.04 [closed]
  9. “Non Zero Exit Status” R 3.0.1 ‘XML’ and ‘RCurl’
  10. How can I use xpath querying using R’s XML library?
  11. How to create xml from R objects, e.g., is there a ‘listToXml’ function?
  12. XSLT Transform doesn’t work until I remove root node
  13. SOAP or REST for Web Services? [closed]
  14. XSLT not working in web browser
  15. JSON and XML comparison [closed]
  16. XPath to select element based on childs child value
  17. How to pass variable parameter into XPath expression?
  18. Using XPATH to search text containing  
  19. split function in xslt 1.0
  20. What Content-Type value should I send for my XML sitemap?
  21. What is the difference between // and .// in XPath?
  22. How to insert a new element under another with xmlstarlet?
  23. Should I use Elements or Attributes in XML? [duplicate]
  24. VBA Output to file using UTF-16
  25. How can I escape “&” in XML? [duplicate]
  26. XML validation with XSD: how to avoid caring about the sequence of the elements?
  27. How do I generate a comma-separated list with XSLT/XPath?
  28. XML tagname starting with number is not working
  29. Cvc-elt.1: Cannot Find The Declaration Of Element ‘soap:Envelope’
  30. Matching a node based on a sibling’s value with XPath

Leave a Comment