"Folder does not exist or is not reachable" on DIGITS AMI
Hi, I have an instance of Ubuntu 16.04 running DIGITS 6. I have downloaded and successfully expanded the MS-COCO dataset as detailed in https://github.com/dusty-nv/jetson-inference. When I run DIGITS and enter the paths to reach the training and validation data folders, I get the error in DIGITS "Folder does not exist or is not reachable". 1. I have checked the paths are correct and tried variants ( from root, home etc). 2. I have followed instructions to make sure that www-data has read access. Guidance from : https://github.com/NVIDIA/DIGITS/issues/1026 and also from https://askubuntu.com/questions/244406/how-do-i-give-www-data-user-to-a-folder-in-my-home-folder. All help most welcome! Thanks! Mike
Hi,

I have an instance of Ubuntu 16.04 running DIGITS 6.
I have downloaded and successfully expanded the MS-COCO dataset as detailed in https://github.com/dusty-nv/jetson-inference.


When I run DIGITS and enter the paths to reach the training and validation data folders, I get the error in DIGITS "Folder does not exist or is not reachable".

1. I have checked the paths are correct and tried variants ( from root, home etc).
2. I have followed instructions to make sure that www-data has read access. Guidance from : https://github.com/NVIDIA/DIGITS/issues/1026 and also from https://askubuntu.com/questions/244406/how-do-i-give-www-data-user-to-a-folder-in-my-home-folder.


All help most welcome! Thanks!

Mike

#1
Posted 11/22/2017 02:56 PM   
Is this the DIGITS AMI, or is it the NGC AMI running the the NGC DIGITS container? (The two are somewhat different.)
Is this the DIGITS AMI, or is it the NGC AMI running the the NGC DIGITS container? (The two are somewhat different.)

#2
Posted 11/22/2017 03:26 PM   
It's the AWS AMI for DIGITS 6 - so I suspect the first of the two options above. It is selected in AWS from EC2 -> AWS Marketplace -> NVIDIA DIGITS 6 on Ubuntu 16.04 Is there a better place I can be doing this? I am trying to get Object Detection working on my TX2, but do not have a suitable machine for building the models - hence working in the cloud to get DIGITS operational. Thanks for the response!
It's the AWS AMI for DIGITS 6 - so I suspect the first of the two options above. It is selected in AWS from EC2 -> AWS Marketplace -> NVIDIA DIGITS 6 on Ubuntu 16.04

Is there a better place I can be doing this?

I am trying to get Object Detection working on my TX2, but do not have a suitable machine for building the models - hence working in the cloud to get DIGITS operational.

Thanks for the response!

#3
Posted 11/22/2017 03:37 PM   
I have now also looked again at the NGC docker containers at https://ngc.nvidia.com/registry/nvidia-digits and I'm afraid I find the instructions completely baffling. For example, mention is made of an 'Actions' column, but I don't think there is one. It also says use 'docker pull', without explaining how to do that. Apologies for being slow! In fact, it was orignally from this page that I opted to follow the links to AWS... If I can be doing this a better way, please let me know, as I am spending a great deal of time trying to get a simple build environment working. Cheers, Mike
I have now also looked again at the NGC docker containers at https://ngc.nvidia.com/registry/nvidia-digits and I'm afraid I find the instructions completely baffling.

For example, mention is made of an 'Actions' column, but I don't think there is one.

It also says use 'docker pull', without explaining how to do that.

Apologies for being slow!

In fact, it was orignally from this page that I opted to follow the links to AWS...

If I can be doing this a better way, please let me know, as I am spending a great deal of time trying to get a simple build environment working.

Cheers, Mike

#4
Posted 11/22/2017 04:12 PM   
Hi Cliff, I am working through instructions for the NGC container now... However, I'm still hoping to be able to get the AWS AMI image working - no resaon it shouldn't? Thanks for your help. Cheers Mike
Hi Cliff,

I am working through instructions for the NGC container now...

However, I'm still hoping to be able to get the AWS AMI image working - no resaon it shouldn't?

Thanks for your help.

Cheers

Mike

#5
Posted 11/22/2017 04:26 PM   
Ah, thanks for pointing out about the "actions" - looks like they renamed that column in the NGC UI to "Pull" after we wrote the text of the README. We'll get that updated. Beyond that, the instructions in the NGC README - and this forum, generally - assume you're using the NGC AMI, not the DIGITS AMI. (I was working on finding you an alternative place to ask the question in the context of the DIGITS AMI before suggesting you start over in a different AMI.) The fact that we have two paths to getting DIGITS on AWS is a bit of a historical artifact we're looking to reconcile in the future. From first principles, the main thing will be to know what paths are involved, where and how those volumes are mounted, and what user account DIGITS is running as. I'm seeking answers to at least the latter of those questions internally, but any extra details you could offer as well would be helpful.
Ah, thanks for pointing out about the "actions" - looks like they renamed that column in the NGC UI to "Pull" after we wrote the text of the README. We'll get that updated.

Beyond that, the instructions in the NGC README - and this forum, generally - assume you're using the NGC AMI, not the DIGITS AMI. (I was working on finding you an alternative place to ask the question in the context of the DIGITS AMI before suggesting you start over in a different AMI.) The fact that we have two paths to getting DIGITS on AWS is a bit of a historical artifact we're looking to reconcile in the future.

From first principles, the main thing will be to know what paths are involved, where and how those volumes are mounted, and what user account DIGITS is running as. I'm seeking answers to at least the latter of those questions internally, but any extra details you could offer as well would be helpful.

#6
Posted 11/22/2017 04:29 PM   
Hi Cliff, Thanks for getting back. I think the 'penny had dropped' on the NGC container approach. However it seems that the Volta instance is 5 times more expensive per hour, which is significant for a self-funded learner when getting started. So as I ssh in, the prompt is $ ubuntu@ip-privateipaddress. Following Dusty's instructions, a simple ls from here shows the directory 'coco' seems correctly listed and drilling down, all the subdirectories and data apper to be present. i.e. Training image folder: coco/train/images/dog Training label folder: coco/train/labels/dog Validation image folder: coco/val/images/dog Validation label folder: coco/val/labels/dog ubuntu@ip:~$ cd coco ubuntu@ip-:~/coco$ ls coco2kitti.py train val ubuntu@ip-:~/coco$ cd train ubuntu@ip-:~/coco/train$ ls images labels ubuntu@ip-:~/coco/train$ cd images ubuntu@ip-:~/coco/train/images$ ls airplane bottle chair dog ubuntu@ip-:~/coco/train/images$ cd dog ubuntu@ip-:~/coco/train/images/dog$ Permissions are as follows ( I have reloaded the datasets so permissions have not been modified in any way) ubuntu@ip-:~/coco$ ls -l total 12 -rw-rw-r-- 1 ubuntu ubuntu 2512 May 1 2017 coco2kitti.py drwxrwxr-x 4 ubuntu ubuntu 4096 May 3 2017 train drwxrwxr-x 4 ubuntu ubuntu 4096 May 2 2017 val By way of accounts, does this help? /home$ ls digits ubuntu Thanks again, Mike
Hi Cliff,
Thanks for getting back.

I think the 'penny had dropped' on the NGC container approach. However it seems that the Volta instance is 5 times more expensive per hour, which is significant for a self-funded learner when getting started.

So as I ssh in, the prompt is $ ubuntu@ip-privateipaddress.

Following Dusty's instructions, a simple ls from here shows the directory 'coco' seems correctly listed and drilling down, all the subdirectories and data apper to be present. i.e.

Training image folder: coco/train/images/dog
Training label folder: coco/train/labels/dog
Validation image folder: coco/val/images/dog
Validation label folder: coco/val/labels/dog

ubuntu@ip:~$ cd coco
ubuntu@ip-:~/coco$ ls
coco2kitti.py train val
ubuntu@ip-:~/coco$ cd train
ubuntu@ip-:~/coco/train$ ls
images labels
ubuntu@ip-:~/coco/train$ cd images
ubuntu@ip-:~/coco/train/images$ ls
airplane bottle chair dog
ubuntu@ip-:~/coco/train/images$ cd dog
ubuntu@ip-:~/coco/train/images/dog$

Permissions are as follows ( I have reloaded the datasets so permissions have not been modified in any way)

ubuntu@ip-:~/coco$ ls -l
total 12
-rw-rw-r-- 1 ubuntu ubuntu 2512 May 1 2017 coco2kitti.py
drwxrwxr-x 4 ubuntu ubuntu 4096 May 3 2017 train
drwxrwxr-x 4 ubuntu ubuntu 4096 May 2 2017 val


By way of accounts, does this help?
/home$ ls
digits ubuntu

Thanks again,

Mike

#7
Posted 11/22/2017 04:50 PM   
So one thought is that I am logged in as ubuntu and installing data owned by that account and in that path. However, maybe the server is accessing using the digits account? This might mean: 1. the server is looking in the wrong account path. 2. It doesn't have permission to see the folders. Could that be a possibility?
So one thought is that I am logged in as ubuntu and installing data owned by that account and in that path. However, maybe the server is accessing using the digits account? This might mean:

1. the server is looking in the wrong account path.
2. It doesn't have permission to see the folders.

Could that be a possibility?

#8
Posted 11/22/2017 05:51 PM   
There are two possible issues at play here. One or both might be at play depending on which AMI you're using. (1) You've got your dataset in /home/ubuntu/coco . However, /home/ubuntu (and therefore subdirectories of the same) would normally only be accessible to the 'ubuntu' user, and DIGITS -- whether containerized or not -- usually runs as a different user, leading to possible permissions problems accessing the dataset. The solution is to move the dataset to some shared directory that all users on the system can access. (2) When running in a containerized environment, the dataset is probably mounted at different paths as viewed from outside the container (say it's /host/path/to/coco ) versus inside the container (normally just /data for DIGITS containers). It's the [b]-v[/b] argument to [b]docker run[/b] that configures this; for example, if you're doing something like [b]docker run ... -v /host/path/to/coco:/data ...[/b], then when you go into the DIGITS UI and set up the dataset, the path you use to tell DIGITS where to find the dataset needs to be /data, not /host/path/to/coco. Hope this helps, Cliff
There are two possible issues at play here. One or both might be at play depending on which AMI you're using.

(1) You've got your dataset in /home/ubuntu/coco . However, /home/ubuntu (and therefore subdirectories of the same) would normally only be accessible to the 'ubuntu' user, and DIGITS -- whether containerized or not -- usually runs as a different user, leading to possible permissions problems accessing the dataset. The solution is to move the dataset to some shared directory that all users on the system can access.

(2) When running in a containerized environment, the dataset is probably mounted at different paths as viewed from outside the container (say it's /host/path/to/coco ) versus inside the container (normally just /data for DIGITS containers). It's the -v argument to docker run that configures this; for example, if you're doing something like docker run ... -v /host/path/to/coco:/data ..., then when you go into the DIGITS UI and set up the dataset, the path you use to tell DIGITS where to find the dataset needs to be /data, not /host/path/to/coco.

Hope this helps,
Cliff

#9
Posted 11/22/2017 06:31 PM   
Hi Cliff, I have managed to get it running through the combination of actions as suggested. Thanks for your help! Mike
Hi Cliff,

I have managed to get it running through the combination of actions as suggested.

Thanks for your help!

Mike

#10
Posted 11/22/2017 08:56 PM   
Glad to hear it!
Glad to hear it!

#11
Posted 11/22/2017 08:58 PM   
Hi! My question is almost exactly the same as the one above. - I am using the AMI with NVIDIA DIGITS 6 on Ubuntu 16.04: https://aws.amazon.com/marketplace/pp/B076DHKCZJ - I have downloaded and successfully expanded the MS-COCO dataset as detailed in https://github.com/dusty-nv/jetson-inference. - I keep getting the error in DIGITS that says "Folder does not exist or is not reachable". I think the problem probably has to do with giving www-data read access, but I'm having trouble figuring out the appropriate combination of commands to do so with my directory structure. I am logged in as ubuntu, and I am trying to put the 'coco' directory into /home/digits/data. Here are the current permissions: ubuntu@ip:/home/digits$ ls -la total 28 drwxr-xr-x 4 digits digits 4096 Oct 2 22:34 . drwxr-xr-x 4 root root 4096 Oct 2 22:34 .. -rw-r--r-- 1 digits digits 220 Oct 2 22:34 .bash_logout -rw-r--r-- 1 digits digits 3771 Oct 2 22:34 .bashrc drwxrwx--- 3 digits digits 4096 Nov 24 19:43 data drwxr-xr-x 2 digits digits 4096 Nov 8 03:33 jobs -rw-r--r-- 1 digits digits 655 Oct 2 22:34 .profile Should I be trying to put the 'coco' directory into /data instead? Any help would be much appreciated!
Hi! My question is almost exactly the same as the one above.

- I am using the AMI with NVIDIA DIGITS 6 on Ubuntu 16.04: https://aws.amazon.com/marketplace/pp/B076DHKCZJ

- I have downloaded and successfully expanded the MS-COCO dataset as detailed in https://github.com/dusty-nv/jetson-inference.

- I keep getting the error in DIGITS that says "Folder does not exist or is not reachable".

I think the problem probably has to do with giving www-data read access, but I'm having trouble figuring out the appropriate combination of commands to do so with my directory structure. I am logged in as ubuntu, and I am trying to put the 'coco' directory into /home/digits/data. Here are the current permissions:

ubuntu@ip:/home/digits$ ls -la
total 28
drwxr-xr-x 4 digits digits 4096 Oct 2 22:34 .
drwxr-xr-x 4 root root 4096 Oct 2 22:34 ..
-rw-r--r-- 1 digits digits 220 Oct 2 22:34 .bash_logout
-rw-r--r-- 1 digits digits 3771 Oct 2 22:34 .bashrc
drwxrwx--- 3 digits digits 4096 Nov 24 19:43 data
drwxr-xr-x 2 digits digits 4096 Nov 8 03:33 jobs
-rw-r--r-- 1 digits digits 655 Oct 2 22:34 .profile

Should I be trying to put the 'coco' directory into /data instead? Any help would be much appreciated!

#12
Posted 11/24/2017 09:21 PM   
Hi pineapple_flora, So firstly move all the data into the digits/data folder. home/digits/data/coco/... In DIGITS, simply specify this path as /data/coco... Secondly set permissions on the entire data folder (recursively) to be readable for all. This will also allow www.data access. If you are following the dusty-nv instructions, you may also find the network in prototext will not work with the specified dataset. I learnt that after 14 hours of build time, only to find the model hadn't been learning anything. That's what I'm working on now...
Hi pineapple_flora,

So firstly move all the data into the digits/data folder. home/digits/data/coco/...

In DIGITS, simply specify this path as /data/coco...

Secondly set permissions on the entire data folder (recursively) to be readable for all. This will also allow www.data access.

If you are following the dusty-nv instructions, you may also find the network in prototext will not work with the specified dataset. I learnt that after 14 hours of build time, only to find the model hadn't been learning anything. That's what I'm working on now...

#13
Posted 11/24/2017 11:15 PM   
Hi mikeisted, Thanks for the suggestions! I followed them and am not seeing a "Folder does not exist or is not reachable" error any more, hooray! I was able to get a little further in the dusty-nv tutorial, but I ran into another roadblock when trying to launch the DetectNet-COCO-Dog model. At a high level, the error that I'm seeing is: "ERROR: Check failed: error == cudaSuccess (8 vs. 0) invalid device function." It might make sense for me to start another thread about this topic or to search through the issues at dusty-nv/jetson-inference -- I was just curious if you had experienced anything similar with your DIGITS 6 AMI.
Hi mikeisted,

Thanks for the suggestions! I followed them and am not seeing a "Folder does not exist or is not reachable" error any more, hooray!

I was able to get a little further in the dusty-nv tutorial, but I ran into another roadblock when trying to launch the DetectNet-COCO-Dog model. At a high level, the error that I'm seeing is: "ERROR: Check failed: error == cudaSuccess (8 vs. 0) invalid device function." It might make sense for me to start another thread about this topic or to search through the issues at dusty-nv/jetson-inference -- I was just curious if you had experienced anything similar with your DIGITS 6 AMI.

#14
Posted 11/26/2017 06:33 PM   
invalid device function usually means that the architecture a CUDA code was compiled for does not match the architecture of the GPU you are running on. What sort of AWS instance type did you launch the DIGITS AMI on? And for clarity are we talking about NGC or not NGC? NGC: https://ngc.nvidia.com/registry/nvidia-digits Not NGC: AWS Marketplace -> NVIDIA DIGITS 6 (I asssume this is "not NGC" here)
invalid device function usually means that the architecture a CUDA code was compiled for does not match the architecture of the GPU you are running on.

What sort of AWS instance type did you launch the DIGITS AMI on? And for clarity are we talking about NGC or not NGC?

NGC: https://ngc.nvidia.com/registry/nvidia-digits


Not NGC: AWS Marketplace -> NVIDIA DIGITS 6

(I asssume this is "not NGC" here)

#15
Posted 11/26/2017 08:05 PM   
Scroll To Top

Add Reply