Only use ref
if the method needs to alter the parameters, and these changes need to be passed onto the calling code. You should only optimize this if you have run it through a profiler and determined that the bottleneck is indeed the CLR copying the method parameters onto the stack.
Bear in mind the CLR is heavily optimized for calling methods with parameters, so I shouldn’t think this would be the issue.