Custom sorting (non-alphabetical)

The levels should be specified explicitly:

A$animal <- factor(A$animal, levels = c("dog", "elephant","cat"))
A$color <- factor(A$color, levels = c("green", "blue", "red"))

Then you order by the 2 columns simultaneously:

A[order(A$animal,A$color),]

# animal color
# 6      dog green
# 4      dog  blue
# 5      dog   red
# 9 elephant green
# 7 elephant  blue
# 8 elephant   red
# 3      cat green
# 1      cat  blue
# 2      cat   red