Is it possible to have cuFFT callback routines in two or more shared libraries? For example, I have a basic project where I just FFT input data, then scale and IFFT it back. I am following the steps in this example https://devblogs.nvidia.com/cuda-pro-tip-use-cufft-callbacks-custom-data-processing/
I have one class that computes a padded FFT using a load routine and compiles into a shared library libZeroPaddedFFT.so
Another class computes an IFFT and scales by the FFT length using a store routine. This class is compiled into shared library libScaledFFT.so
I have tested each library and know they both work exactly how I want them to individually. My problem is when I try to link both libraries into one main.cpp. I am receiving the runtime error
CUDA CALL failed at scaledFFT.cu:55 : invalid device symbol
which corresponds to this line, where I am copying the callback function pointer from the device to the host
CUDA_CALL(cudaMemcpyFromSymbol(&hostCallbackPtr, devCallbackPtr, sizeof(hostCallbackPtr)));
Also for reference, here are the commands for how I am building my libZeroPaddedFFT.so
/opt/cuda-9.0/bin/nvcc -ccbin g++ -Xcompiler "-O3 -DNDEBUG -Wall -Wextra -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-long-long -fno-strict-aliasing -Wno-variadic-macros -Wno-missing-field-initializers -fPIC" -I./ -I/opt/cuda-9.0/include -I/opt/cuda-9.0/samples/common/inc -m64 -dc -gencode arch=compute_61,code=sm_61 zeroPaddedFFT.cu -o zeroPaddedFFT.o
/opt/cuda-9.0/bin/nvcc -ccbin g++ -Xcompiler "-O3 -DNDEBUG -Wall -Wextra -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-long-long -fno-strict-aliasing -Wno-variadic-macros -Wno-missing-field-initializers -fPIC" -I./ -I/opt/cuda-9.0/include -I/opt/cuda-9.0/samples/common/inc -m64 -dlink -gencode arch=compute_61,code=sm_61 zeroPaddedFFT.o -o zeroPadLink.o -L/opt/cuda-9.0/lib64 -L. -lcufft_static -lculibos
gcc -o libZeroPaddedFFT.so -rdynamic -shared -Wl,-rpath=/opt/cuda9.0/lib64 zeroPaddedFFT.o zeroPadLink.o -L/opt/cuda-9.0/lib64 -L. -lcufft_static -lculibos
Then to compile my main.cpp,
g++ -I./ -I/opt/cuda-9.0/include -I/opt/cuda-9.0/samples/common/inc -m64 -o callbackTest main.cpp -L/opt/cuda-9.0/lib64 -L. -lcudart -lZeroPaddedFFT -lScaledFFT
Again, I know it can’t be a syntax error because both of my libraries work individually. Can anyone explain to me why this happens or a potential work around? Thank you in advance!