There is no simple answer to this question. The specification leaves room for different implementation strategies and even within one implementation, it will depend on how the class is been used. Further, your question is more about “when can it fail” which depends on why it may fail, e.g. a class may be loaded at one point of time to verify its existence but initialized at a later time when it is actually first‑time used. At both points of time, the operation may fail causing a NoClassDefFoundError.
We may categorize referenced classes into three groups:
- classes that must be resolved at linkage time (e.g. the superclass)
- classes that must be loaded at verification time
- classes whose loading can be deferred to it’s first actual use
In your case, BouncyCastleProvider must be loaded at verification time, rather than its first actual use, because you are passing the result of new BouncyCastleProvider() as argument to the method Security.insertProviderAt(…) and the verifier must check whether the class actually extends the Provider type which the method’s formal parameter mandates.
When verification will happen, is also implementation specific as at least the following possibilities are allowed:
- eagerly traversing all referenced classes
- on the loading of the containing class or its first use
- on the containing method’s first use
- right before executing the offending instruction
Oracle’s JVM prefers a method’s first use, read: verify it at the method entry, thus, moving the invocation into another method that won’t get executed, be it in another class or not, is sufficient in your case.
But to be compatible with other JVMs, moving it into another class is safer, but to comply to all possible JVMs, you need to even load this other class via Reflection to avoid a direct reference that can be traversed by an eager implementation (or instantiate BouncyCastleProvider reflectively via Class.forName("… .BouncyCastleProvider").newInstance() in the first place).