I got the message:
«cutilCheckMsg() CUTIL CUDA error :
kernel launch failure : CUDA driver
version is insufficient for CUDA
runtime version.»
While trying to run an example source code. Also happens for the function cutilSafeCall.
I am using:
- Windows 7 64bits
- Visual studio 2008
- CUDA developer driver, toolkit, and SDK 3.1
- Emulation mode
double-beep
4,97617 gold badges32 silver badges41 bronze badges
asked Jul 15, 2010 at 7:13
2
You need to ensure that your driver version matches or exceeds your CUDA Toolkit version.
For 2.3 you need a 190.x driver, for 3.0 you need 195.x and for 3.1 you need 256.x (actually anything up to the next multiple of five is ok, e.g. 258.x for 3.1).
You can check your driver version by either running the deviceQueryDrv SDK sample or go into the NVIDIA Control Panel and choose System Information.
Download an updated driver from www.nvidia.com/drivers.
answered Jul 20, 2010 at 9:26
TomTom
20.8k4 gold badges42 silver badges54 bronze badges
I saw the same at runtime with the latest driver on Mac OS 10.6.
cudaError_t error = cudaGetDevice(&device);
printf("%sn", cudaGetErrorString(error));
I went back to the developer site, downloaded the driver again and now it runs.
http://developer.nvidia.com/object/cuda_3_1_downloads.html#MacOS
double-beep
4,97617 gold badges32 silver badges41 bronze badges
answered Sep 2, 2010 at 20:57
FrankFrank
1342 bronze badges
You can either download the latest driver OR use an older toolkit version to compile your code.
answered Dec 13, 2011 at 7:50
MeghanaMeghana
511 silver badge1 bronze badge
1
Counterintuitively, this error also happens if libcuda.so is not found, even when versions reported by nvidia-smi match perfectly. This library is part of nvidia-drivers package (On CentOS: nvidia-driver-latest-cuda-libs, on Gentoo x11-drivers/nvidia-drivers). It is possible to have the CUDA Tookit with nvcc and libcudart installed and building your app fine, but the drivers part not installed, causing this error.
To diagnose whether this is the reason, use strace:
strace -f -e trace=file ./your_cuda_app
and check for open calls to libcuda.so*, at least one of them should return with a success code, like so:
4928 open("/lib64/libcuda.so.1", O_RDONLY|O_CLOEXEC) = 3
answered Apr 19, 2021 at 15:50
alexeialexei
1,9911 gold badge25 silver badges27 bronze badges
My cent,
with Linux/Unix this error may be related to the selected GPU mode (Performance/Power Saving Mode), when you select (with nvidia-settings utiliy) the integrated Intel GPU and you execute the deviceQuery script… you get this error:
-> CUDA driver version is insufficient for CUDA runtime version
But this error is misleading, by selecting back the NVIDIA(Performance mode) with nvidia-settings utility the problem disappears.
It is not a version problem.
Regards
P.s: «Power Saving Mode» tells Optimus to activate the CPU integrated Intel GPU
answered Mar 12, 2018 at 9:28
CUDA driver version is insufficient for CUDA runtime version: means your GPU can`t been manipulated by the CUDA runtime API, so you need to update your driver.
answered Oct 10, 2014 at 17:02
1
In my case, I had to run my docker container with nvidia-docker run ... instead of docker run ...
answered Sep 3, 2021 at 11:35
Kees SchollaartKees Schollaart
6861 gold badge5 silver badges4 bronze badges
I also had similar problem, updated my graphic driver but the problem still remained. I finally decided to remove Cuda 9.2 and install Cuda 8, it solved my issue.
answered Sep 13, 2018 at 23:47
This problem can also be because of incorrect environment setup, e.g. Docker image setup. Although the driver itself is correct, sufficient for your program. If your LD_LIBRARY_PATH points to the wrong driver, it can throw this error. In my case, i get this error when using /usr/local/nvidia/lib/libcuda.so, and if I use /usr/local/nvidia/lib64/libcuda.so everything goes right.
answered Sep 23, 2022 at 12:26
leninlenin
6214 silver badges3 bronze badges
Maybe it is related to the TBB lib:
Error OpenCV with CUDA using TBB for multiple GPUs
Try rebuilding it making sure you passed the following parameters to CMake (assuming you already installed «tbb» and «tbb-devel» packages:
-D WITH_TBB=YES -D TBB_INCLUDE_DIRS=/usr/include/tbb
answered Oct 13, 2014 at 21:22
herreraherrera
1171 silver badge10 bronze badges
Hi
I encountered this error: Check failed: error == cudaSuccess (35 vs. 0) CUDA driver version is insufficient for CUDA runtime version during my app processing.
Machine details:
GCE with GPU Tesla K80
OS: Ubuntu 14.04
The Driver installed is: NVIDIA-Linux-x86_64-375.66.run with nvidia-docker nvidia-docker_1.0.1.
Im using the following docker image, based on nvidia/cuda:7.0-cudnn4-devel-ubuntu14.04NVIDIA official Docker image
kaixhin/cuda-caffe:7.0
On top of that I compiled OpenCV 3.0.0
$nvidia-smi / run --rm nvidia/cuda:7.0-runtime nvidia-smi
Thu Jun 15 19:24:53 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.66 Driver Version: 375.66 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 0000:00:04.0 Off | 0 |
| N/A 55C P8 31W / 149W | 0MiB / 11439MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
$ldconfig -p | grep cuda
libicudata.so.52 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libicudata.so.52
libcuda.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libcuda.so.1
libcuda.so (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libcuda.so
$nvidia-docker run --rm device
./deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device(s)
Device 0: "Tesla K80"
CUDA Driver Version / Runtime Version 8.0 / 8.0
CUDA Capability Major/Minor version number: 3.7
Total amount of global memory: 11440 MBytes (11995578368 bytes)
(13) Multiprocessors, (192) CUDA Cores/MP: 2496 CUDA Cores
GPU Max Clock rate: 824 MHz (0.82 GHz)
Memory Clock rate: 2505 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 1572864 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Enabled
Device supports Unified Addressing (UVA): Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 0 / 4
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = Tesla K80
Result = PASS
Can you pls help me solve this issue?
D.
Loading
If you are a CUDA user, you might have come across the ‘CUDA Driver Version is Insufficient for CUDA Runtime Version’ error message. This error usually occurs when the version of the CUDA Driver installed on your computer is not compatible with the version of the CUDA Runtime you are trying to use. In this guide, we will provide you with a comprehensive solution to fix this error.
What is CUDA?
Before we dive into the solution, let’s first understand what CUDA is. CUDA (Compute Unified Device Architecture) is a parallel computing platform and application programming interface (API) model created by Nvidia. It enables developers to use the power of Nvidia GPUs to accelerate computing tasks in their applications.
CUDA includes a driver and a runtime library that work together to provide access to the GPU’s processing power. The driver is responsible for managing the GPU and communicating with the operating system, while the runtime library provides the API for developers to write CUDA code.
Why does the ‘CUDA Driver Version is Insufficient for CUDA Runtime Version’ error occur?
The ‘CUDA Driver Version is Insufficient for CUDA Runtime Version’ error occurs when the version of the CUDA Driver installed on your computer is not compatible with the version of the CUDA Runtime you are trying to use. This can happen when you install a new version of the CUDA Runtime without updating the CUDA Driver, or when you install a new CUDA Driver without updating the CUDA Runtime.
To fix the ‘CUDA Driver Version is Insufficient for CUDA Runtime Version’ error, you need to make sure that the version of the CUDA Driver installed on your computer is compatible with the version of the CUDA Runtime you are trying to use. Follow these steps to fix the error:
Step 1: Check the CUDA version
The first step is to check the version of the CUDA Runtime you are using. You can do this by running the following command in the terminal:
nvcc --version
This will display the version of the CUDA Runtime installed on your computer.
Step 2: Check the CUDA Driver version
Next, you need to check the version of the CUDA Driver installed on your computer. You can do this by running the following command in the terminal:
cat /proc/driver/nvidia/version
This will display the version of the CUDA Driver installed on your computer.
Step 3: Install the compatible CUDA Driver
If the version of the CUDA Driver installed on your computer is not compatible with the version of the CUDA Runtime you are trying to use, you need to install the compatible CUDA Driver. You can download the compatible CUDA Driver from the Nvidia website.
Before installing the new CUDA Driver, make sure to uninstall the old one by running the following command in the terminal:
sudo apt-get --purge remove nvidia-*
After uninstalling the old CUDA Driver, you can install the new one by following the instructions provided by Nvidia.
Step 4: Test the CUDA installation
Once you have installed the compatible CUDA Driver, you need to test the CUDA installation to make sure it is working properly. You can do this by running the following command in the terminal:
cuda-install-samples-10.0.sh ~/cuda-samples
cd ~/cuda-samples/NVIDIA_CUDA-*
make
This will compile and run the CUDA samples on your computer. If the samples run without any errors, it means that the CUDA installation is working properly.
FAQ
Q1. What is the CUDA Toolkit?
The CUDA Toolkit is a software development kit (SDK) provided by Nvidia that includes everything you need to develop CUDA applications. It includes the CUDA Runtime, the CUDA Driver, and various tools and libraries for developing and debugging CUDA code.
Q2. Can I install multiple versions of the CUDA Toolkit on my computer?
Yes, you can install multiple versions of the CUDA Toolkit on your computer. However, you need to make sure that the versions are compatible with each other.
Q3. How do I know which version of the CUDA Toolkit to install?
The version of the CUDA Toolkit you should install depends on the version of the Nvidia GPU you have and the version of the operating system you are using. You can find the recommended versions on the Nvidia website.
Q4. Can I use CUDA on non-Nvidia GPUs?
No, you cannot use CUDA on non-Nvidia GPUs. CUDA is a proprietary technology developed by Nvidia and is only compatible with Nvidia GPUs.
Q5. How do I update the CUDA Toolkit?
To update the CUDA Toolkit, you need to download the latest version from the Nvidia website and follow the instructions provided by Nvidia to install it. Before installing the new version, make sure to uninstall the old one.
- CUDA Toolkit download page
- CUDA Toolkit documentation
- NVIDIA GPU driver download page
So I have a very similar question to:
What can I do against ‘CUDA driver version is insufficient for CUDA runtime version’?
When I make and run deviceQuery, I get the exact same error:
./deviceQuery Starting...
CUDA Device Query (Runtime API) version (CUDART static linking)
cudaGetDeviceCount returned 35
-> CUDA driver version is insufficient for CUDA runtime version
Result = FAIL
Here’s my system:
andycui97@andycui97-Z10PE-D8-WS:~$ nvidia-settings -q NvidiaDriverVersion
Attribute 'NvidiaDriverVersion' (andycui97-Z10PE-D8-WS:0.0): 367.35
Attribute 'NvidiaDriverVersion' (andycui97-Z10PE-D8-WS:0[gpu:0]): 367.35
andycui97@andycui97-Z10PE-D8-WS:~$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 367.35 Mon Jul 11 23:14:21 PDT 2016
GCC version: gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.1)
andycui97@andycui97-Z10PE-D8-WS:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Wed_May__4_21:01:56_CDT_2016
Cuda compilation tools, release 8.0, V8.0.26
andycui97@andycui97-Z10PE-D8-WS:~$ nvidia-smi
Sat Jul 16 17:48:19 2016
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.35 Driver Version: 367.35 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 1070 Off | 0000:03:00.0 On | N/A |
| 27% 39C P5 12W / 151W | 545MiB / 8106MiB | 31% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 900 G /usr/lib/xorg/Xorg 241MiB |
| 0 1556 G compiz 140MiB |
| 0 7455 G ...s-passed-by-fd --v8-snapshot-passed-by-fd 136MiB |
| 0 9861 G /home/andycui97/.steam/ubuntu12_32/steam 25MiB |
+-----------------------------------------------------------------------------+
So I have a gtx 1070 and I installed cuda 8rc from the runfile for linux 16.04
If I’m not mistaken, my driver version is the absolute latest, literally released a day ago according to
http://www.nvidia.com/download/driverResults.aspx/105343/en-us, so I am confused as to how my CUDA driver version is insufficient.
Any help would be appreciated!