Why do my Perl tests fail with use encoding ‘utf8’?

by

Do not use the encoding pragma. It’s broken. (Juerd Waalboer gave a great talk where he mentioned this at YAPC::EU 2k8.)

It does at least two things at once that do not belong together:

  1. It specifies an encoding for your source file.
  2. It specifies an encoding for your file input/output.

And to add injury to insult it also does #1 in a broken fashion: it reinterprets \xNN sequences as being undecoded octets as opposed to treating them like codepoints, and decodes them, preventing you from being able to express characters outside the encoding you specified and making your source code mean different things depending on the encoding. That’s just astonishingly wrong.

Write your source code in ASCII or UTF-8 only. In the latter case, the utf8 pragma is the correct thing to use. If you don’t want to use UTF-8, but you do want to include non-ASCII charcters, escape or decode them explicitly.

And use I/O layers explicitly or set them using the open pragma to have I/O automatically transcoded properly.

More Related Contents:

  1. Why does modern Perl avoid UTF-8 by default?
  2. How can I output UTF-8 from Perl?
  3. Use of ‘use utf8;’ gives me ‘Wide character in print’
  4. How can I treat command-line arguments as UTF-8 in Perl?
  5. How can I convert my shell script to Perl? [closed]
  6. search using perl one liner within a file [closed]
  7. How to re-direct the contents to a file instead of printing it to terminal
  8. How do I remove duplicate items from an array in Perl?
  9. What is the difference between library files and modules?
  10. How do I read in the contents of a directory in Perl?
  11. How can I pass command-line arguments to a Perl program?
  12. How do I efficiently parse a CSV file in Perl?
  13. How can I generate all permutations of an array in Perl?
  14. Why can’t I match my string from standard input in Perl?
  15. How do I choose a package name for a custom Perl module that does not collide with builtin or CPAN packages names?
  16. What’s broken about exceptions in Perl?
  17. Delete ClearCase Views Script
  18. How to pass a variable from a child process (fork by Parallel::ForkManager)?
  19. What’s the difference between single and double quotes in Perl?
  20. How do I get the full path to a Perl script that is executing?
  21. Perl subroutine arguments
  22. What does the function declaration “sub function($$)” mean?
  23. How can I call a shell command in my Perl script?
  24. Which cpan installer is the right one? (CPAN.pm/CPANPLUS/cpanminus)
  25. How do I lock a file in Perl?
  26. Why is the Switch module deprecated in Perl?
  27. Which framework should I use to write modules? [closed]
  28. How do I capture Perl’s STDOUT in a variable?
  29. Is object oriented exception handling in Perl worth it?
  30. How can I check whether my Perl installation is 32 or 64 bit?

Leave a Comment