CUDA limit seems to be reached, but what limit is that?

by

The resource which is being exhausted is time. On all current CUDA platforms, the display driver includes a watchdog timer which will kill any kernel which takes more than a few seconds to execute. Running code on a card which is running a display is subject to this limit.

On the WDDM Windows platforms you are using, there are three possible solutions/work-arounds:

  1. Get a Telsa card and use the TCC driver, which eliminates the problem completely
  2. Try modifying registry settings to increase the timer limit (google for the TdrDelay registry key for more information, but I am not a Windows user and can’t be more specific than that)
  3. Modify your kernel code to be “re-entrant” and process the data parallel work load in several kernel launches rather than one. Kernel launch overhead isn’t all that large and processing the workload over several kernel runs is often pretty easy to achieve, depending on the algorithm you are using.

More Related Contents:

  1. Modifying registry to increase GPU timeout, windows 7
  2. sending 3d array to CUDA kernel
  3. How to measure the inner kernel time in NVIDIA CUDA?
  4. Passing Host Function as a function pointer in __global__ OR __device__ function in CUDA
  5. How do CUDA blocks/warps/threads map onto CUDA cores?
  6. How are 2D / 3D CUDA blocks divided into warps?
  7. nvidia-smi Volatile GPU-Utilization explanation?
  8. How do I use Nvidia Multi-process Service (MPS) to run multiple non-MPI CUDA applications?
  9. Should I unify two similar kernels with an ‘if’ statement, risking performance loss?
  10. CUDA: How many concurrent threads in total?
  11. CUDA apps time out & fail after several seconds – how to work around this?
  12. Why is NVIDIA Pascal GPUs slow on running CUDA Kernels when using cudaMallocManaged
  13. How does CUDA assign device IDs to GPUs?
  14. Best approach for GPGPU/CUDA/OpenCL in Java?
  15. What is the canonical way to check for errors using the CUDA runtime API?
  16. CUDA incompatible with my gcc version
  17. How to use Thrust to sort the rows of a matrix?
  18. GPU Emulator for CUDA programming without the hardware [closed]
  19. Using GPU from a docker container?
  20. How to get the CUDA version?
  21. How to generalize fast matrix multiplication on GPU using numba
  22. Why does nvcc fails to compile a CUDA file with boost::spirit?
  23. In CUDA, what is memory coalescing, and how is it achieved?
  24. CUBLAS: Incorrect inversion for matrix with zero pivot
  25. Using maximum shared memory in Cuda
  26. Cuda Mutex, why deadlock?
  27. Inconsistency of IDs between ‘nvidia-smi -L’ and cuDeviceGetName()
  28. CUDA stream compaction algorithm
  29. CUDA allocation alignment is 256 bytes – seriously?
  30. CUDA : How to allocate memory for data member of a class

Leave a Comment