I was thinking of using some libraries instead of writing my own this time, so I was checking through the CuDNN user guide and it seems a tensor only supports up to 8 dimensions. From what I read a tensor is the equivalent of a case vector. Most case vectors have 100+ dimensions on real world applications, so how do I implement these types of problems with CuDNN?
Or is CuDNN more focused on image recognition, rather than general machine learning problems?
Thanks in advance.
cuDNN is not focused on general machine learning, but is specifically focused on deep neural networks.
since when does deep learning only apply to images? its very myopic view of deep learning!
you realise deep learning can solve just as many problems greater than 8 inputs? lol! Nvidia should have called it CuImageRecognition, not CuDNN
having said that i hope i am not right here and someone can clarify the terminologies for me! thanks!
It seems to me that “DNN” is perfect shorthand for “deep neural networks”, so there is actually truth in advertising in the naming.
Presumably NVIDIA focuses its software efforts where there is the largest customer demand. After all, while the software is free to customers, it does represent a cost to NVIDIA. In such scenarios, aiming for the largest !/$ seems like a perfectly rational approach.
CuDNN does fully connected layers so theoretically it should be able to do more than 8 dimensions. so what is a tensor if it is not a case vector, someone who actually uses the library? thanks.
I didn’t say deep learning only applied to images, nor did I say cuDNN only handles images.
I said cuDNN applies to deep neural networks, not to the more general case of machine learning.
Machine Learning includes many sub-areas such as SVM, k-means clustering, etc. which cuDNN is not directly designed to handle. It is designed to handle deep neural networks. But those deep neural networks don’t have to be limited to images.
For deep neural networks, a tensor captures the information necessary to perform the calculations associated with a particular layer of the network.
This blog article mentions using cuDNN for a speech application, for example:
if cudnn only suports 8 dimensions its not really a DNN library. it should be called VLDDNN very low dimensional deep neural network.
Are you sure you’re not confusing number of dimensions with dimension size? For example, if you’re doing linear regression on 100 features then you would normally parallelize this by using a 1-D tensor of size 100.
The number of tensor dimensions used by cuDNN is related to data organization rather than number of features. For example, for a convolutional net you’d typically use 4-D tensors (width x height x channels x batch).
yes that was it.