Typically you’d do something like this:
ann_text <- data.frame(mpg = 15,wt = 5,lab = "Text",
cyl = factor(8,levels = c("4","6","8")))
p + geom_text(data = ann_text,label = "Text")
It should work without specifying the factor variable completely, but will probably throw some warnings: