Linking frustration -lcuda fails

This one should be pretty easy, since I know I am missing something simple.

I’m linking an NVCC generated .o file with another plain .o file from gcc.
My problem is that the final link can’t find the cuda libraries. Which is crazy since I’ve checked about 12 times that they’re there.
64 bit Ubuntu 9.04 with CUDA 2.3, driver 190.18.

nvcc -arch sm_13 -c -I/usr/local/cuda/include  ${SRC}/kernel.cu 
   g++ -c main.cpp 
   g++ main.o kernel.o -L/usr/lib64 -L/usr/local/cuda/lib64 -lcuda

my $LD_LIBRARY_PATH has /usr/local/cuda/lib64 in it (as well as /lib, which is symlinked to /usr/lib64)

If I run the three build commands above, the link fails (with step #3) with
/usr/bin/ld: cannot find -lcuda

If I don’t put -lcuda, I get about 40 errors about can’t finding CUDA functions like:
tmpxft_00003c34_00000000-10_kernel.ii:(.text+0xdbbd): undefined reference to `__cudaRegisterFatBinary’

-lcudart also fails at link (not found).

Over and over I keep checking that inside /usr/lib64 there’s a libcuda.so , and there is. It’s a symlink (set up by the toolkit installer) to libcuda.so.1 and libcuda.so.1 is a symlink to the hard file libcuda.so.190.18

The SDK examples are installed and I can build them successfully. The complexity of the SDK makefile makes it hard to boil down to my app.

A single .cu file (with its own main() ) will work with a single nvcc invocation (and not need -lcuda)
nvcc -arch sm_11 simple.cu -o simple

What am I missing? This has got to be easy but I’m still lost in head-banging frustration.
Thanks very very much for any rescue!

From the looks of the missing symbols, you are using the runtime api, so you should be linking against libcudart rather than libcuda, which means providing -lcudart and -L to the linker. It really is that simple. Something like this (this example also links againts cublas and the SDK cutil library)

avid@cuda:~/build/CUDA_CG$ /opt/cuda/bin/nvcc -c -arch sm_13 -I$HOME/NVIDIA_GPU_Computing_SDK/C/common/inc -I/opt/cuda/include fQCG.cu -o fQCG.o

avid@cuda:~/build/CUDA_CG$ /opt/cuda/bin/nvcc -c -arch sm_13 -I$HOME/NVIDIA_GPU_Computing_SDK/C/common/inc -I/opt/cuda/include dQCG.cu -o dQCG.o

avid@cuda:~/build/CUDA_CG$ gcc -c -I$HOME/NVIDIA_GPU_Computing_SDK/C/common/inc -I/opt/cuda/include main.c -o main.o

avid@cuda:~/build/CUDA_CG$ g++ -L/opt/cuda/lib64 -L$HOME/NVIDIA_GPU_Computing_SDK/C/lib -o cuda_cg dQCG.o fQCG.o main.o -lcublas -lcudart -lcutil

avid@cuda:~/build/CUDA_CG$ export LD_LIBRARY_PATH=/opt/cuda/lib64 

avid@cuda:~/build/CUDA_CG$ LD_LIBRARY_PATH=/opt/cuda/lib64 ldd ./cuda_cg

	linux-vdso.so.1 =>  (0x00007fffb01fe000)

	libcublas.so.2 => /opt/cuda/lib64/libcublas.so.2 (0x00007f8fa6e9c000)

	libcudart.so.2 => /opt/cuda/lib64/libcudart.so.2 (0x00007f8fa6c5c000)

	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f8fa694f000)

	libm.so.6 => /lib/libm.so.6 (0x00007f8fa66ca000)

	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f8fa64b2000)

	libc.so.6 => /lib/libc.so.6 (0x00007f8fa6140000)

	libpthread.so.0 => /lib/libpthread.so.0 (0x00007f8fa5f24000)

	libdl.so.2 => /lib/libdl.so.2 (0x00007f8fa5d20000)

	librt.so.1 => /lib/librt.so.1 (0x00007f8fa5b18000)

	/lib64/ld-linux-x86-64.so.2 (0x00007f8fa7f86000)

You will definitely find it easier in the long run to use a makefile, but that is how it is done.

avid, thanks for the fast reply!

Unfortunately, it’s not an issue of using -lcuda or -lcudart. They both fail.
libcuda.so lives in /usr/lib64 and libcudart.so lives in /usr/local/cuda/lib64.
Both of those paths are in my LD_LIBRARY_PATH. Both of those paths are being passed to the linker with -L .
But using either -lcuda or -lcudart results in “/usr/bin/ld: cannot find -lcudart” or “/usr/bin/ld: cannot find -lcudart”

I am using a makefile… these are the build rules I’m trying to set up.

I appreciate the quoted build lines you’re using! And checking them, I still don’t see anything I’m missing. :-(
Since it’s the linker failing, I suspect that it’s my Linux/g++ setup and not CUDA specifically.

Ok, a further test. I can use “-lreadline” and it finds the libreadline library with no error. So I can link some libraries at least.
But I cannot use “-lsmime3” even though libsmine3.so lives in that same /usr/lib64 directory!
But the difference is that there is a libreadline.a file in the directory as well as libreadline.so . There is no libsmime3.a.
So maybe this is a static versus dynamic library issue!? But my compile steps pretty much match aviday’s.

libcuda does not have a libcuda.a , only a libcuda.so .
Is there supposed to be a libcuda.a file too? (Please check your /lib directory… do you have both files?)

This post says that the libcuda.a file is installed by the driver. However I only have libcuda.so.
Should I reinstall the latest driver?

You have several “levels” of problems. Not linking against the correct library is one of them. The linker not finding the library is another.

LD_LIBRARY_PATH is irrelevant to this discussion. It only controls the runtime linkers ability to locate shared libraries. It has no effect on compilation behaviour.

This is undoubtedly where your problem is. Could you post the exact compilation statements or makefile you are using.

I very much doubt it. The mere fact you can build the SDK examples completely disproves it.

CUDA is only supplied as shared libraries.

Boom, solved. Programmer stupidity.
Avid, thanks for insisting on a simple test case. I made a test and it failed, but then I decided to make a command-line only version to simplify even more and it WORKED.

So the problem was hidden in my makefile. I had flags of “CFLAGS= -ffast-math -static -O3”

Guess what, the -static flag means that you must have a static .a file. There is no static CUDA lib, so it was failing.
My fault. Though to be honest I wish GCC’s linker had better error error messages. If it just said “-static link option cannot find libcuda.a in library paths.” it would have helped enormously!

Avid, thanks very very much for jumping in to help. I’d still be screaming obscenities if you hadn’t asked for the exact statements… my makefile test was only using a couple lines and it was so easy to think there was no problem there.

Problem and solution summary: “Don’t use -static when linking CUDA apps.”

Hello! I need help about this:

I want to compile bandwidthTest.cu within makefile from sdk, and I use this: nvcc -0 bandwidthTest bandwithTest.cu -lcuda

/tmp/tmpxft_00007852_00000000-12_bandwidthTest.o: In function testDeviceToDeviceTransfer(unsigned int)': tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe4ac): undefined reference to cutCreateTimer’
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe555): undefined reference to cutStartTimer' tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe5bd): undefined reference to cutStopTimer’
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe5d6): undefined reference to cutGetTimerValue' tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe664): undefined reference to cutDeleteTimer’
/tmp/tmpxft_00007852_00000000-12_bandwidthTest.o: In function testHostToDeviceTransfer(unsigned int, memoryMode)': tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe6ed): undefined reference to cutCreateTimer’
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe7de): undefined reference to cutStartTimer' tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe81d): undefined reference to cutStopTimer’
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe836): undefined reference to cutGetTimerValue' tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe84f): undefined reference to cutResetTimer’
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe924): undefined reference to cutDeleteTimer' /tmp/tmpxft_00007852_00000000-12_bandwidthTest.o: In function testDeviceToHostTransfer(unsigned int, memoryMode)‘:
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xe9d7): undefined reference to cutCreateTimer' tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xeab1): undefined reference to cutStartTimer’
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xeb03): undefined reference to cutStopTimer' tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xeb1c): undefined reference to cutGetTimerValue’
tmpxft_00007852_00000000-11_bandwidthTest.ii:(.text+0xeba6): undefined reference to `cutDeleteTimer’
collect2: ld devolvió el estado de salida 1

Thanks

You need to link with the cutil library from the SDK (and you need to tell nvcc where to find it)

Many thank avvi. Another cuestion, Is it posible mix driver library and api library in the same code #include <cutil_inline.h>
#include <cuda.h>

I´m trying to compile, but only work if I separate the code :(.