Network boot

Is it possible to boot the Jetson Nano Developer Kit over the network? PXE?

Not sure if NFS boot would work. Need to check.

@kosmon
Please check below document for the NFS support.

[url]https://docs.nvidia.com/jetson/l4t/Tegra%20Linux%20Driver%20Package%20Development%20Guide/getting_started.html#wwpID0E02B0HA[/url]

hi @ShaneCCC,

can you clarify to me the steps in the documentation? it is not clear for me what steps are to be done on the target (jetson nano?). also i dont understand what should be inside the nfsroot and rootfs folders on both the host and the target.

do i share the boot files or the whole OS on the host system?

maybe i dont understand in essence how boot over the network works, maybe a small clarification could help me understand.

thanks

@ibondokji
This chapter is talk about how to flash the device from Network. Not sure if that’s what you want?

@ShaneCCC,

what i am looking for turns out to be called wake on lan. do you know if it works on jetson nano? i have read that it is supported on the boared, but i was not able to make it work.

@ibondokji
Sorry to tell, due to missing low power mode capability, WoL (Wake on LAN) not workable on Nano.

@ShaneCCC

I see, thanks for your response.

Hello @ShaneCCC,
I am having a bit of hard time getting my Jetson Nano Developer Kit to boot from network.

As I understand, I need to create a DTB for CBoot. At https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fgetting_started.html%23wwpID0E0NG0HA it says “Locate the cbo.dts file in this directory: $ cd /Linux_for_Tegra/bootloader/”. But in the Jetson-210_Linux_R32.2.0-0.tar there is no such file:

mkudlacek@goliath:~/jetson/Linux_for_Tegra/bootloader$ find
.
./bmp.blob
./l4t_bup_gen.func
./mkbctpart
./tegrarcm
./mksparse
./tegrahost
./LICENSE.u-boot
./tegraflash_internal.py
./LICENSE.bpmp_t210b01
./BUP_generator.py
./t210ref
./t210ref/nvtboot.bin
./t210ref/cboot.bin
./t210ref/warmboot.bin
./t210ref/sc7entry-firmware.bin
./t210ref/BCT
./t210ref/BCT/P2530_E01_3GB_Samsung_lpddr4_204Mhz_P968_v2.cfg
./t210ref/BCT/P3448_A00_4GB_Micron_4GB_lpddr4_204Mhz_P987.cfg
./t210ref/BCT/E2220_LP3_DSC_931.2Mhz.cfg
./t210ref/BCT/P2894_A00_Samsung_3GB_lpddr4_204Mhz_P984_v2.cfg
./t210ref/BCT/P2180_A00_LP4_DSC_204Mhz.cfg
./t210ref/p2371-0000
./t210ref/p2371-0000/u-boot.bin
./t210ref/nvtboot_t210b01.bin
./t210ref/e2220-1170
./t210ref/e2220-1170/u-boot.bin
./t210ref/p3450-porg
./t210ref/p3450-porg/u-boot.bin
./t210ref/LICENSE.cboot
./t210ref/LICENSE.sc7entry-firmware
./t210ref/p2371-2180
./t210ref/p2371-2180/u-boot.bin
./t210ref/cfg
./t210ref/cfg/flash_l4t_t210_emmc_p3448.xml
./t210ref/cfg/board_config_p3448.xml
./t210ref/cfg/board_config_p2597-devkit.xml
./t210ref/cfg/flash_l4t_t210_spi_p3448.xml
./t210ref/cfg/board_config_p2597.xml
./t210ref/cfg/gnu_linux_tegraboot_emmc_full.xml
./t210ref/cfg/board_config_ers_e2220.xml
./t210ref/cfg/flash_l4t_t210_spi_sd_p3448.xml
./t210ref/cfg/flash_l4t_t214_spi.xml
./t210ref/cfg/board_config_p2595.xml
./t210ref/cfg/flash_l4t_t214_emmc.xml
./nvtboot_recovery_t210b01.bin
./LICENSE.mksparse
./tegrasign
./chkbdinfo
./rp4.blob
./nvidia-l4t-bootloader_32.2.0-20190716172031_arm64.deb
./nvtboot_recovery_cpu.bin
./LICENSE.mkbctpart
./mkbootimg
./bpmp_t210b01.bin
./LICENSE.tos-mon-only.img.arm-trusted-firmware
./LICENSE.mkgpt
./NVIDIA_Trademark_License_Addendum_SW.pdf
./l4t_initrd.img
./tegraparser
./extlinux.conf
./tos-mon-only.img
./nvtboot_cpu.bin
./tegraflash.py
./mkgpt
./tegradevflash
./tegrabct
./LICENSE.mkbootimg
./LICENSE
./nvtboot_recovery.bin
./nv_boot_control.conf
./eks.img
./LICENSE.chkbdinfo
./tos.img

Can you please point me in the right direction?

@martin.kudlacek
Looks like the cbo.dts only for the TX2 and Xavier.
Could you skip this.

@ShaneCCC
How can I specify the tftp and nfs settings for netboot?

Hi @ShaneCCC,
any updates on this?

Thanks,
Martin

We are trace this internally will update to you once got any result.

Hello martin,

We have been able to successfully boot Jetson NANO boot over network. Here are the steps that we have followed:

  1. Untar u-boot source shipped with the “L4T Sources” for TX1/NANO.
  2. Since the “saveenv” is not present on u-boot for NANO; do following changes to the u-boot_src files:
diff -rpubw uboot-t210.orig/u-boot/include/config_distro_bootcmd.h uboot-t210/u-boot/include/config_distro_bootcmd.h
--- uboot-t210.orig/u-boot/include/config_distro_bootcmd.h	2019-07-16 19:40:54.000000000 -0400
+++ uboot-t210/u-boot/include/config_distro_bootcmd.h	2019-07-30 08:31:39.663611217 -0400
@@ -296,7 +296,6 @@
 	"bootcmd_pxe=" \
 		BOOTENV_RUN_NET_USB_START \
 		BOOTENV_RUN_NET_PCI_ENUM \
-		"dhcp; " \
 		"if pxe get; then " \
 			"pxe boot; " \
 		"fi\0"
diff -rpubw uboot-t210.orig/u-boot/include/configs/p3450-porg.h uboot-t210/u-boot/include/configs/p3450-porg.h
--- uboot-t210.orig/u-boot/include/configs/p3450-porg.h	2019-07-16 19:40:54.000000000 -0400
+++ uboot-t210/u-boot/include/configs/p3450-porg.h	2019-08-01 10:54:35.650579379 -0400
@@ -29,9 +29,9 @@
 
 /* Only MMC1/PXE/DHCP for now, add USB back in later when supported */
 #define BOOT_TARGET_DEVICES(func) \
+	func(PXE, pxe, na) \
 	func(MMC, mmc, 1) \
 	func(MMC, mmc, 0) \
-	func(PXE, pxe, na) \
 	func(DHCP, dhcp, na)
 
 /* Environment s/b at end of SPI, fix it later */
@@ -61,6 +61,9 @@
 
 /* General networking support */
 
+#define CONFIG_IPADDR			<target IP addr>
+#define CONFIG_SERVERIP		<server IP addr>
+
 #include "tegra-common-usb-gadget.h"
 #include "tegra-common-post.h"
  1. Follow compilation steps from:https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Fuboot_guide.html%23
    NOTE: NANO’s “board_and_rev” is p3450-porg.

Depending on your setup, either you will need to flash the board with the NFS parameters or just the newly built u-boot.

Hope this helps.
Once the

1 Like

@anish.aney

I’m attempting to do something very similar with the Jetson TX2. I’ve already gotten connectivity with a PXE server, so the next step is to get a bootable image to be downloaded. I’m new to PXE, and hours of Googling hasn’t really helped me learn how to package things into a PXE-friendly format. Do you have any suggestions or know of any good resources that can help?

Thanks!
~Alec

Hello marin-at-RLR,

You need to download a file looking similar to extlinux.conf file using u-boot’s PXE mechanism.

The path PXE looks for in order to “get” the file is /var/tftpboot/01-. The MAC address must have “-” instead of “:”. This file will be read and something similar to extlinux.conf’s output will appear on the screen. Select the kernel you wish to boot to.

Hope I have answered your query.

Thanks for sharing, this really increases productivity, Nvidia should inlcude this in their doc.

1 Like

Hi @AnishAney sorry to reopen the thread but, I’m also trying to boot Linux with root file system via NFS.

So far i tried to use the kernel in sdcard and pass the kernel arguments[1] to enable rootfs via NFS, the following is the serial output:

U-Boot 2016.07 (Dec 07 2019 - 05:37:58 +0000)

TEGRA210
Model: NVIDIA P3450-Porg
Board: NVIDIA P3450-PORG
DRAM:  4 GiB
MMC:   Tegra SD/MMC: 0, Tegra SD/MMC: 1
SF: Detected MX25U3235F with page size 256 Bytes, erase size 4 KiB, total 4 MiB
In:    serial
Out:   serial
Err:   serial
Net:   No ethernet found.
Hit any key to stop autoboot:  0
 
[i][b]switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
317 bytes read in 74 ms (3.9 KiB/s)
1:      Poky (Yocto Project Reference)
Retrieving file: /boot/extlinux/../initrd
2000251 bytes read in 190 ms (10 MiB/s)
Retrieving file: /boot/extlinux/../Image
34557960 bytes read in 1578 ms (20.9 MiB/s)
append: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize[/b]
usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8
[/i]
## Flattened Device Tree blob at 83100000
   Booting using the fdt blob at 0x83100000
   reserving fdt memory region: addr=80000000 size=20000
   Using Device Tree in place at 0000000083100000, end 000000008317724e

Starting kernel ...

<hit enter to activate fiq debugger>
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.9.140-l4t-r32.2.1+gf03f5aa305c2 (oe-user@oe-host) (gcc version 7.3.0 (GCC) ) #1 SMP PREEMPT Sun Dec 1 02:18:11 UTC 2019

In /boot/extlinux/extlinux.conf i have the following:

LABEL Poky (Yocto Project Reference Distro)
    KERNEL ../Image
    INITRD ../initrd
    APPEND ${cbootargs} ${bootargs} root=/dev/nfs nfsroot=192.168.1.78:/var/lib/tftpboot ip=dhcp rootwait rw earlyprintk console=ttyS0,115200 console=tty0

As you can see in the previous serial output looks like my kernel args are appended but the system get stuck after some time, the following are the last messages in console before it freeze

[    8.042555] IP-Config: Got DHCP answer from 192.168.1.254, my address is 192.168.1.70
[    8.050389] IP-Config: Complete:
[    8.053622]      device=eth0, hwaddr=00:e0:4c:68:00:01, ipaddr=192.168.1.70, mask=255.255.255.0, gw=192.168.1.254
[    8.063911]      host=192.168.1.70, domain=rga.ip, nis-domain=(none)
[    8.070262]      bootserver=0.0.0.0, rootserver=192.168.1.78, rootpath=     nameserver0=192.168.1.254
[    8.080035] vdd-fan: disabling
[    8.083110] vdd-usb-vbus: disabling
[    8.086601] vdd-usb-vbus2: disabling
[    8.090176] avdd-io-edp-1v05: disabling
[    8.094110] ALSA device list:
[    8.097087]   #0: tegra-hda at 0x70038000 irq 83
[    8.101709]   #1: tegra-snd-t210ref-mobile-rt565x
[    8.109505] Freeing unused kernel memory: 8576K
[    8.175701] usb 2-1: new SuperSpeed USB device number 2 using tegra-xusb
[    8.215325] usb 2-1: New USB device found, idVendor=0bda, idProduct=0411
[    8.222033] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    8.229170] usb 2-1: Product: 4-Port USB 3.1 Hub
[    8.233795] usb 2-1: Manufacturer: Generic
[    8.243807] hub 2-1:1.0: USB hub found
[    8.249085] hub 2-1:1.0: 4 ports detected
[    8.410562] usb 2-1: usb_suspend_both: status 0
[    8.415237] usb usb2: usb_suspend_both: status 0
[    9.974518] tegra-xusb 70090000.xusb: entering ELPG
[    9.980645] tegra-pmc: PMC tegra_pmc_utmi_phy_enable_sleepwalk : port 1, speed 3
[    9.988235] tegra-pmc: PMC tegra_pmc_utmi_phy_enable_sleepwalk : port 2, speed 0
[    9.997184] tegra-xusb 70090000.xusb: entering ELPG done
[   24.633782] random: crng init done

The only related message that i can see is rootpath=, i also tried to set this variable from extlinux.conf but the variable keeps empty anyway:

[    8.050389] IP-Config: Complete:
...
[    8.070262]      bootserver=0.0.0.0, rootserver=192.168.1.78, rootpath=     nameserver0=192.168.1.254

HINTS:

  • I'm using Jetson Nano
  • I'm using Yocto and meta-tegra to build my image
  • I'm new with PXE tools, that is why i tried the kernel command line approach before trying something like PXE from uboot. Also, i want to have the option to boot a kernel image from sdcard and NFS, i'm not sure if i can use PXE for that

Thank you so much for your time!

[1] kernel docs: https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt

Hello ing.mtma,

From the logs it looks like the kernel has loaded correctly, but I think it failed while loading/mounting the rootfs. Does /var/lib/tftpboot have the complete rootfs? Also, I am assuming that the tftpserver (192.168.1.78) is setup as a tftp server.
There may be some log entries whether you are getting any requests from your board.

I am not familiar with Yocto so if something’s missing on that front then I am no help.

I think you should be able to switch between PXE and SD if you continue using local extlinux.conf file. Just create another entry with root=/dev/mmcblk0p1, etc.

What we have done is that our board receives the configuration file (similar to extlinux.conf) from the server and then we can either boot over NFS or do a localboot where it will fall back to next “boot_targets” entry of U-Boot. Once it sees mmc0/mmc1, we choose the kernel to boot to.

Hi @AnishAney,

Thanks for the answer!

Yes, at least is the same as the one i have in my sdcard and i’m assuming it is fine because without the changes i made to extlinux.conf it runs fine.

Running journalctl -f in my Host(F29). There is no request from my target(jetson nano) at boot time, but, if i run it without my changes and mount the rootfs manually i.e.

root@jetson-nano:~# mount -v -o rw,vers=3 192.168.1.78:/var/lib/tftpboot /mnt

In Host i can see the mount request, i don’t know if this is enough to say that my server setup is working fine:

Dec 12 11:11:23 localhost.localdomain rpc.mountd[8717]: authenticated mount request from 192.168.1.70:988 for /var/lib/tftpboot (/var/lib/tftpboot)
Dec 12 11:11:23 localhost.localdomain kernel: nfsd: exp_rootfh(/var/lib/tftpboot [00000000d612d2df] 192.168.1.101/24,192.168.1.78/24:dm-1/789519)

And it is mounted:

root@jetson-nano:~# ls -la /mnt/
drwxr-xr-x   18 root     root          4096 Dec 10  2019 .
drwxr-xr-x   17 root     root          4096 Dec 10  2019 ..
drwxr-xr-x    2 root     root          4096 Dec  7  2019 bin
drwxr-xr-x    3 root     root          4096 Dec  7  2019 boot
-rw-r--r--    1 root     root           514 Dec 10  2019 cmdline
drwxr-xr-x    2 root     root          4096 Dec  1  2019 dev
drwxr-xr-x   16 root     root          4096 Dec  7  2019 etc
drwxr-xr-x    3 root     root          4096 Dec  7  2019 home
drwxr-xr-x    4 root     root          4096 Dec  1  2019 lib
drwx------    2 root     root          4096 Dec  7  2019 lost+found
drwxr-xr-x    2 root     root          4096 Dec  1  2019 media
drwxr-xr-x    2 root     root          4096 Dec  1  2019 mnt
dr-xr-xr-x    2 root     root          4096 Dec  1  2019 proc
drwxr-xr-x    2 root     root          4096 Dec  1  2019 run
drwxr-xr-x    2 root     root          4096 Dec  7  2019 sbin
dr-xr-xr-x    2 root     root          4096 Dec  1  2019 sys
-rw-r--r--    1 root     root             0 Aug 25  2019 textfile.txt
drwxrwxrwt    2 root     root          4096 Dec  1  2019 tmp
drwxr-xr-x   10 root     root          4096 Dec  1  2019 usr
drwxr-xr-x    8 root     root          4096 Dec  7  2019 var

I will try to use PXE i just have a quick question… you said The path PXE looks for in order to “get” the file is /var/tftpboot/01-.
Based on my last reply my path will be /var/tftpboot/00-e0-4c-68-00-01 ?

However, if i’m able to load the extlinux.conf from Uboot, How do you tell the Kernel that the rootfs will be over NFS? is there a specific OPTION in extlinux or you just APPEND kernel args?

BR,
-Miguel Miranda