What is the purpose of using multiple “arch” flags in Nvidia’s NVCC compiler?

by

Roughly speaking, the code compilation flow goes like this:

CUDA C/C++ device code source –> PTX –> SASS

The virtual architecture (e.g. compute_20, whatever is specified by -arch compute...) determines what type of PTX code will be generated. The additional switches (e.g. -code sm_21) determine what type of SASS code will be generated. SASS is actually executable object code for a GPU (machine language). An executable can contain multiple versions of SASS and/or PTX, and there is a runtime loader mechanism that will pick appropriate versions based on the GPU actually being used.

As you point out, one of the handy features of GPU operation is JIT-compile. JIT-compile will be done by the GPU driver (does not require the CUDA toolkit to be installed) anytime a suitable PTX code is available but a suitable SASS code is not. The definition of a “suitable PTX” code is one which is numerically equal to or lower than the GPU architecture being targeted for running the code. To pick an example, specifying arch=compute_30,code=compute_30 would tell nvcc to embed cc3.0 PTX code in the executable. This PTX code could be used to generate SASS code for any future architecture that the GPU driver supports. Currently this would include architectures like Pascal, Volta, Turing, etc. assuming the GPU driver supports those architectures.

One advantage of including multiple virtual architectures (i.e. multiple versions of PTX), then, is that you have executable compatibility with a wider variety of target GPU devices (although some devices may trigger a JIT-compile to create the necessary SASS).

One advantage of including multiple “real GPU targets” (i.e. multiple SASS versions) is that you can avoid the JIT-compile step, when one of those target devices is present.

If you specify a bad set of options, it’s possible to create an executable that won’t run (correctly) on a particular GPU.

One possible disadvantage of specifying a lot of these options is code size bloat. Another possible disadvantage is compile time, which will generally be longer as you specify more options.

It’s also possible to create excutables that contain no PTX, which may be of interest to those trying to obscure their IP.

Creating PTX suitable for JIT should be done by specifying a virtual architecture for the code switch.

More Related Contents:

  1. CUDA: How to use -arch and -code and SM vs COMPUTE
  2. CUDA and nvcc: using the preprocessor to choose between float or double
  3. How to disable a specific nvcc compiler warnings
  4. What is the canonical way to check for errors using the CUDA runtime API?
  5. Unspecified launch failure on Memcpy
  6. Different CUDA versions shown by nvcc and NVIDIA-smi
  7. Thrust inside user written kernels
  8. GPU Emulator for CUDA programming without the hardware [closed]
  9. How to measure the inner kernel time in NVIDIA CUDA?
  10. How do CUDA blocks/warps/threads map onto CUDA cores?
  11. Understanding CUDA grid dimensions, block dimensions and threads organization (simple explanation) [closed]
  12. How to get the CUDA version?
  13. Copy an object to device?
  14. What kind of variables consume registers in CUDA?
  15. Why does nvcc fails to compile a CUDA file with boost::spirit?
  16. In CUDA, what is memory coalescing, and how is it achieved?
  17. CUDA apps time out & fail after several seconds – how to work around this?
  18. Why is NVIDIA Pascal GPUs slow on running CUDA Kernels when using cudaMallocManaged
  19. How to use 2D Arrays in CUDA?
  20. How to create a CUDA context?
  21. CUBLAS: Incorrect inversion for matrix with zero pivot
  22. Using maximum shared memory in Cuda
  23. Why does my CUDA kernel crash (unspecified launch failure) with a different dataset size?
  24. Cuda Mutex, why deadlock?
  25. Inconsistency of IDs between ‘nvidia-smi -L’ and cuDeviceGetName()
  26. CUDA allocation alignment is 256 bytes – seriously?
  27. printf() in my CUDA kernel doesn’t result produce any output
  28. Multiply Rectangular Matrices in CUDA
  29. Which Compute Capability is supported by which CUDA versions?
  30. “invalid configuration argument ” error for the call of CUDA kernel?

Leave a Comment