Why does Runtime.exec(String) work for some but not all commands?

Why do some commands fail?

This happens because the command passed to Runtime.exec(String) is not executed in a shell. The shell performs a lot of common support services for programs, and when the shell is not around to do them, the command will fail.

When do commands fail?

A command will fail whenever it depends on a shell features. The shell does a lot of common, useful things we don’t normally think about:

1. The shell splits correctly on quotes and spaces

   This makes sure the filename in "My File.txt" remains a single argument.

   Runtime.exec(String) naively splits on spaces and would pass this as two separate filenames. This obviously fails.

2. The shell expands globs/wildcards

   When you run ls *.doc, the shell rewrites it into ls letter.doc notes.doc.

   Runtime.exec(String) doesn’t, it just passes them as arguments.

   ls has no idea what * is, so the command fails.

3. The shell manages pipes and redirections.

   When you run ls mydir > output.txt, the shell opens “output.txt” for command output and removes it from the command line, giving ls mydir.

   Runtime.exec(String) doesn’t. It just passes them as arguments.

   ls has no idea what > means, so the command fails.

4. The shell expands variables and commands

   When you run ls "$HOME" or ls "$(pwd)", the shell rewrites it into ls /home/myuser.

   Runtime.exec(String) doesn’t, it just passes them as arguments.

   ls has no idea what $ means, so the command fails.

What can you do instead?

There are two ways to execute arbitrarily complex commands:

Simple and sloppy: delegate to a shell.

You can just use Runtime.exec(String[]) (note the array parameter) and pass your command directly to a shell that can do all the heavy lifting:

// Simple, sloppy fix. May have security and robustness implications
String myFile = "some filename.txt";
String myCommand = "cp -R '" + myFile + "' $HOME 2> errorlog";
Runtime.getRuntime().exec(new String[] { "bash", "-c", myCommand });

Secure and robust: take on the responsibilities of the shell.

This is not a fix that can be mechanically applied, but requires an understanding the Unix execution model, what shells do, and how you can do the same. However, you can get a solid, secure and robust solution by taking the shell out of the picture. This is facilitated by ProcessBuilder.

The command from the previous example that requires someone to handle 1. quotes, 2. variables, and 3. redirections, can be written as:

String myFile = "some filename.txt";
ProcessBuilder builder = new ProcessBuilder(
    "cp", "-R", myFile,        // We handle word splitting
       System.getenv("HOME")); // We handle variables
builder.redirectError(         // We set up redirections
    ProcessBuilder.Redirect.to(new File("errorlog")));
builder.start();