For me, the problem was that there was a build error (see Problems tab) which was preventing compilation; oops! The reason you see the error is that the run macro proceeds despite the failed compilation step, and attempts to run class files it expects to be there; they don’t exist because there was a build error preventing compilation, so it says it can’t find Main (not compiled).
Problem goes away when build can complete successfully, i.e. errors are fixed.
I guess, theoretically, there may be more complicated reasons your build is not completing successfully that are not listed in Problems.