Trying to compile nccl, can't finish making caffe2 til I get this straightened out

Graphics / Linux Linux
1

The most recent thing I tried to use is the package manager, but everywhere I try to compile it it’s the same error message:

==> nccl dependencies:
 - cuda (already installed)

==> Continue building nccl ? [Y/n]
==> ------------------------------
==> U

==> Building and installing package
==> Making package: nccl 1.3.0.1-5 (Sun Apr 23 17:50:51 CDT 2017)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Downloading nccl-1.3.0-1.tar.gz...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   121    0   121    0     0     94      0 --:--:--  0:00:01 --:--:--    94
100 41418    0 41418    0     0   9379      0 --:--:--  0:00:04 --:--:-- 13326
==> Validating source files with sha256sums...
    nccl-1.3.0-1.tar.gz ... Passed
==> Extracting sources...
  -> Extracting nccl-1.3.0-1.tar.gz with bsdtar
==> Starting prepare()...
==> Starting build()...
Grabbing  src/nccl.h                > /tmp/yaourt-tmp-daenerys/aur-nccl/include/nccl.h
Compiling src/libwrap.cu            > /tmp/yaourt-tmp-daenerys/aur-nccl/obj/libwrap.o
-gencode=arch=compute_35,code=sm_35: No such file or directory
make: *** [Makefile:99: /tmp/yaourt-tmp-daenerys/aur-nccl/obj/libwrap.o] Error 1
==> ERROR: A failure occurred in build().
    Aborting...
==> ERROR: Makepkg was unable to build nccl.
==> Restart building nccl ? [y/N]
==> -----------------------------
==>

I can only assume that my repeated issue with this is a bug on NVidia’s end. I followed the correct instructions when I did it myself, same “No such file or directory libwrap.o” issue (and I looked, it’s not in there where it’s supposed to be), and even the package manager has the same issue. If I’m missing something I humbly apologize but this is making me tear my hair out I just want Caffe2 working and this eternally stands in my way. It was the same in the third-party directory inside of caffe2, it was the same when I git cloned it myself.

2

Caffe 0.16 Compilation Error [blocking_queue.cpp] with USE_NCCL:=1 flag. NCCL v1.3.4-1
[Also tried 0.15.13 and 0.15.9]

Environment: AWS G3 Large Instance with Tesla M60 - Ubuntu 16.04

NCCL 1.3.4 is successfully compiled and has nccl.h file under /usr/local/library/ and
has other dependent files in /usr/local/lib.

libnccl-dev/now 1.3.4-1+cuda8.0 amd64 [installed,upgradable to: 2.0.4-1+cuda8.0]
libnccl1/now 1.3.4-1+cuda8.0 amd64 [installed,local]

Compilation Error

CXX src/caffe/util/blocking_queue.cpp
In file included from ./include/caffe/parallel.hpp:19:0,
from src/caffe/util/blocking_queue.cpp:6:
./include/caffe/util/nccl.hpp:25:16: error: ‘ncclDataType_t’ does not name a type
static const ncclDataType_t type = ncclFloat;
^
./include/caffe/util/nccl.hpp:29:16: error: ‘ncclDataType_t’ does not name a type
static const ncclDataType_t type = ncclDouble;
^
./include/caffe/util/nccl.hpp:34:16: error: ‘ncclDataType_t’ does not name a type
static const ncclDataType_t type = ncclHalf;
^
./include/caffe/util/nccl.hpp:38:8: error: ‘ncclDataType_t’ does not name a type
inline ncclDataType_t nccl_type(Type type) {
^
In file included from src/caffe/util/blocking_queue.cpp:6:0:
./include/caffe/parallel.hpp:79:3: error: ‘ncclUniqueId’ does not name a type
ncclUniqueId nccl_id_;
^
./include/caffe/parallel.hpp:112:3: error: ‘ncclComm_t’ does not name a type
ncclComm_t nccl_comm_;
^
./include/caffe/parallel.hpp:113:3: error: ‘ncclUniqueId’ does not name a type
ncclUniqueId nccl_id_;
^
Makefile:611: recipe for target ‘.build_release/src/caffe/util/blocking_queue.o’ failed
make: *** [.build_release/src/caffe/util/blocking_queue.o] Error 1