Summary: When feeding UTF-8 to Nokogiri through open-uri, use open(…).read and pass the resulting string to Nokogiri. Analysis: If I fetch the page using curl, the headers properly show Content-Type: text/html; charset=UTF-8 and the file content includes valid UTF-8, e.g. “Genealogía de Jesucristo”. But even with a magic comment on the Ruby file and setting … Read more
I managed to install Nokogiri under Yosemite (OS X 10.10 Preview). Step 1: Install Brew Skip this if brew was installed. ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)” Step 2: Install brew libs brew tap homebrew/dupes brew install libxml2 libxslt brew install libiconv Step 3: Download and install Apple Commandline Tools for 10.10 It’s important that you … Read more
Previous advises didn’t help me, here is the solution for OS 10.9: brew install libxml2 bundle config build.nokogiri “–use-system-libraries –with-xml2-include=/usr/local/opt/libxml2/include/libxml2” bundle install
You need to have all the necessary libraries installed on your machine. When you installed RVM , it should have listed this for you. On the current version of rvm, you can run rvm requirements to see the exact list. Right now, that list is: sudo apt-get install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g … Read more
If you installed Nokogiri with gem install nokogiri, you can resolve this warning by running gem pristine nokogiri to recompile the gem’s C extension. If you installed Nokogiri with bundle install, you can resolve this warning by running bundle exec gem pristine nokogiri to recompile the C extension of the gem wherever Bundler installed it.
All namespaces need to be registered when parsing. Nokogiri automatically registers namespaces on the root node. Any namespaces that are not on the root node you have to register yourself. This should work: puts doc.xpath(‘//dc:title’, ‘dc’ => “URI”) Alternately, you can remove namespaces altogether. Only do this if you are certain there will be no … Read more
If you want to convert a Nokogiri XML document to a hash, just do the following: require ‘active_support/core_ext/hash/conversions’ hash = Hash.from_xml(nokogiri_document.to_s)
I had the same issue. Unfortunately the “Installing Nokogiri” doesn’t cover Iconv issues. Here’s how I resolved the issue. First install homebrew, it’ll make your life easier. If you already have it installed, be sure to grab the latest formulae by updating like so: brew update Note: In OSX 10.9+ you may need to install … Read more
If you want to build DOM you can use jsdom. There’s also cheerio, it has the jQuery interface and it’s a lot faster than older versions of jsdom, although these days they are similar in performance. You might wanna have a look at htmlparser2, which is a streaming parser, and according to its benchmark, it … Read more
This is an easily solved problem that results from not reading the documentation about how text behaves when used on a NodeSet versus a Node (or Element). The NodeSet documentation says text will: Get the inner text of all contained Node objects Which is what we’re seeing happen with: doc = Nokogiri::HTML(<<EOT) <html> <body> <p>foo</p> … Read more