Just one minor change solves the issue. You need to specify group=x
inside your geom_text(aes(...))
call.
ggplot(df) +
geom_bar(aes(z, y, fill=x), position=position_dodge(width=1), stat="identity") +
geom_text(aes(z,y,label=perc, group=x), position=position_dodge(width=1), size=4)