usb 3.0 device mode RNDIS ethernet speed （12MB/s --18MB/s）

I use it with a jetson TX2 board and a virtual machine(ubuntu), passed the 3.0 cable plug J19.I find ethernet speed（12MB/s --18MB/s）,Others test up to 70MB/s.Please help me see how to solve it.on ubuntu side, execute （dmesg）,I find (xhci_hcd 0000:03:00.0: ERROR unknown event type 37).

I refer to the following methods.
link:
https://devtalk.nvidia.com/default/topic/1014096/jetson-tx2/how-to-set-tx2-otg-usb-as-device-mode-/1
https://devtalk.nvidia.com/default/topic/1041816/jetson-agx-xavier/usb-device-mode-ethernet-speed/1

kernel and rootfs use version:
Tegra_Linux_Sample-Root-Filesystem_R28.1.0_aarch64
Tegra186_Linux_R28.1.0_aarch64
kernel-4.4

I steps and debugging information.

ubuntu(vmware) side:

root@zgq-virtual-machine:/home/nuctech# ifconfig ens160u1 192.168.55.5
root@zgq-virtual-machine:/home/nuctech# dd if=/dev/zero bs=1024K count=512 | nc -v 192.168.55.21 8888
Connection to 192.168.55.21 8888 port [tcp/*] succeeded!
512+0 records in
512+0 records out
536870912 bytes (537 MB, 512 MiB) copied, 32.1936 s, 16.7 MB/s

root@zgq-virtual-machine:/home/nuctech# lsusb -t

/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
|__ Port 1: Dev 26, If 0, Class=Communications, Driver=rndis_host, 5000M
|__ Port 1: Dev 26, If 1, Class=CDC Data, Driver=rndis_host, 5000M
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
|__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 3: Dev 4, If 0, Class=Hub, Driver=hub/7p, 12M
|__ Port 4: Dev 5, If 0, Class=Hub, Driver=hub/7p, 480M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/7p, 12M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/6p, 480M

root@zgq-virtual-machine:/home/nuctech# usb-devices

T: Bus=04 Lev=01 Prnt=01 Port=00 Cnt=01 Dev#= 26 Spd=5000 MxCh= 0
D: Ver= 3.10 Cls=ef(misc ) Sub=02 Prot=03 MxPS= 9 #Cfgs= 1
P: Vendor=0955 ProdID=7020 Rev=00.01
S: Manufacturer=NVIDIA
S: Product=Linux for Tegra
S: SerialNumber=0421118035065
C: #Ifs= 2 Cfg#= 1 Atr=80 MxPwr=8mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=02(commc) Sub=02 Prot=ff Driver=rndis_host
I: If#= 1 Alt= 0 #EPs= 2 Cls=0a(data ) Sub=00 Prot=00 Driver=rndis_host

root@zgq-virtual-machine:/home/nuctech# ifconfig

ens160u1 Link encap:Ethernet HWaddr 06:aa:bb:34:ad:90
inet addr:192.168.55.5 Bcast:192.168.55.255 Mask:255.255.255.0
inet6 addr: fe80::4aa:bbff:fe34:ad90/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:217086 errors:0 dropped:0 overruns:0 frame:0
TX packets:401116 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11304351 (11.3 MB) TX bytes:581107521 (581.1 MB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:13271 errors:0 dropped:0 overruns:0 frame:0
TX packets:13271 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:816231 (816.2 KB) TX bytes:816231 (816.2 KB)

root@zgq-virtual-machine:/home/nuctech# dmesg

[115583.245046] usb 4-1: new SuperSpeed USB device number 25 using xhci_hcd
[115583.275652] usb 4-1: New USB device found, idVendor=0955, idProduct=7020
[115583.275656] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[115583.275711] usb 4-1: Product: Linux for Tegra
[115583.275715] usb 4-1: Manufacturer: NVIDIA
[115583.275718] usb 4-1: SerialNumber: 0421118035065
[115583.417437] rndis_host 4-1:1.0 usb0: register ‘rndis_host’ at usb-0000:03:00.0-1, RNDIS device, 22:a3:f0:28:d7:28
[115583.800072] rndis_host 4-1:1.0 ens160u1: renamed from usb0
[115584.725193] IPv6: ADDRCONF(NETDEV_UP): ens160u1: link is not ready
[115703.382728] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115704.695709] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115711.939512] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115712.885806] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115715.570756] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115715.838726] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115715.845269] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115716.082096] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115716.093517] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115716.118601] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115717.947446] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115720.538324] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.045077] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.083227] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.104719] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.121738] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.139556] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.157480] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.173985] xhci_hcd 0000:03:00.0: ERROR unknown event type 37
[115745.192965] xhci_hcd 0000:03:00.0: ERROR unknown event type 37

TX2 side:

nvidia@tegra-ubuntu:~$ sudo /home/configfs.sh

nvidia@tegra-ubuntu:~$ sudo ifconfig usb0 192.168.55.21
nvidia@tegra-ubuntu:~$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:04:4b:a5:0d:10
inet addr:10.15.165.199 Bcast:10.15.165.255 Mask:255.255.254.0
inet6 addr: fe80::204:4bff:fea5:d10/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:23154 errors:0 dropped:359 overruns:0 frame:0
TX packets:18104 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1483900 (1.4 MB) TX bytes:675990893 (675.9 MB)
Interrupt:42

l4tbr0 Link encap:Ethernet HWaddr ba:f3:ad:ae:c5:50
inet addr:192.168.55.1 Bcast:192.168.55.255 Mask:255.255.255.0
inet6 addr: fe80::dc9f:70ff:fe06:362f/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:1381 (1.3 KB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:11 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:753 (753.0 B) TX bytes:753 (753.0 B)

usb0 Link encap:Ethernet HWaddr ba:f3:ad:ae:c5:50
inet addr:192.168.55.21 Bcast:192.168.55.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

nvidia@tegra-ubuntu:~$ sudo su
root@tegra-ubuntu:/home/nvidia# echo 0x1 > /sys/class/extcon/extcon0/state
root@tegra-ubuntu:/home/nvidia# nc -v -l 8888 > /dev/null
Listening on [0.0.0.0] (family 0, port 8888)
Connection from [192.168.55.5] port 8888 [tcp/*] accepted (family 2, sport 35326)
root@tegra-ubuntu:/home/nvidia#

root@tegra-ubuntu:/home/nvidia# dmesg

[ 154.536389] using random self ethernet address
[ 154.540931] using random host ethernet address
[ 154.551915] usb0: HOST MAC da:24:96:84:c8:ee
[ 154.556359] usb0: MAC ba:f3:ad:ae:c5:50
[ 154.561599] tegra-xudc-new 3550000.xudc: exiting ELPG
[ 154.570142] tegra-xudc-new 3550000.xudc: exiting ELPG done
[ 154.575717] tegra-xudc-new 3550000.xudc: ep 0 (type: 0, dir: out) enabled
[ 154.582609] tegra-xudc-new 3550000.xudc: entering ELPG
[ 154.589828] tegra-xudc-new 3550000.xudc: entering ELPG done
[ 154.597287] tegra-xudc-new 3550000.xudc: exiting ELPG
[ 154.605594] tegra-xudc-new 3550000.xudc: exiting ELPG done
[ 154.611228] tegra-xudc-new 3550000.xudc: entering ELPG
[ 154.618389] tegra-xudc-new 3550000.xudc: entering ELPG done
[ 154.636328] device usb0 entered promiscuous mode
[ 154.645680] IPv6: ADDRCONF(NETDEV_UP): usb0: link is not ready
[ 192.776633] tegra-xotg xotg: update_vbus_state: VBUS detected
[ 193.321195] tegra186-padctl 3520000.pinctrl: tegra18x_phy_xusb_utmi_pad_dcd: DCD timeout 418 ms
[ 193.371178] tegra-usb-cd usb_cd: notification status (0x0, 0x1)
[ 193.377301] tegra-usb-cd usb_cd: set current 0ma
[ 193.382022] tegra-xudc-new 3550000.xudc: exiting ELPG
[ 193.389505] tegra-xudc-new 3550000.xudc: exiting ELPG done
[ 193.395117] tegra-xudc-new 3550000.xudc: device mode on
[ 195.383252] tegra-usb-cd usb_cd: notification status (0x1, 0x20)
[ 195.389349] tegra-usb-cd usb_cd: connected to non-standard charger
[ 195.395622] tegra-usb-cd usb_cd: set current 500ma
[ 225.826992] android_work: did not send uevent (0 0 (null))
[ 226.036466] android_work: sent uevent USB_STATE=CONNECTED
[ 226.394403] android_work: sent uevent USB_STATE=DISCONNECTED
[ 226.449659] android_work: sent uevent USB_STATE=CONNECTED
[ 226.478186] tegra-xudc-new 3550000.xudc: setup request failed: -22
[ 226.485610] tegra-xudc-new 3550000.xudc: setup request failed: -22
[ 226.658881] configfs-gadget gadget: super-speed config #1: b
[ 226.664825] tegra-xudc-new 3550000.xudc: ep 5 (type: 3, dir: in) enabled
[ 226.671661] tegra-xudc-new 3550000.xudc: ep 3 (type: 2, dir: in) enabled
[ 226.678489] tegra-xudc-new 3550000.xudc: ep 2 (type: 2, dir: out) enabled
[ 226.686836] IPv6: ADDRCONF(NETDEV_CHANGE): usb0: link becomes ready
[ 226.694189] l4tbr0: port 1(usb0) entered forwarding state
[ 226.699977] l4tbr0: port 1(usb0) entered forwarding state
[ 226.706411] android_work: sent uevent USB_STATE=CONFIGURED
[ 241.706897] l4tbr0: port 1(usb0) entered forwarding state
[ 516.449373] tegradc 15210000.nvdisplay: blank - powerdown
hardware-nvidia-platform-t18x-quill.patch.txt (3.54 KB)
hardware-nvidia-platform-t18x-common.patch.txt (1.4 KB)
TX2_device_log.txt (4.85 KB)
vmware_host_log.txt (4.53 KB)
configfs.sh.txt (6.38 KB)

Hi zhangguiqing1999,
Can you please share your usecase?
You have a PC(host) connecting TX2(device) and want to use TX2 ethernet port? Usually general PCs have ethernet port. Why do you need TX2 ethernet port?

TX2 is Coprocessor, main processor is HISILICON Soc. HISILICON Soc use video deal, TX2 use AI Calculation. HISILICON Soc connected to TX2 over 3.0 cable,use data transfer or cmd. HISILICON Soc board is being designed. I want use ubuntu connected TX2 software code test.use RNDIS mode.

Please help me to see the configuration error.

Hi,
We have ipertests and the result looks good:

1. TCP Downlink:
On DUT: iperf3 -s
On Server: iperf3 -c <<DUT's IP>> -t 120

2. TCP Uplink:
On DUT: iperf3 -c <<Server's IP>> -t 120
On Server: iperf3 -s

It can achieve 820+Mbps in downlink and 850Mbps in uplink.

We don’t have experience in running RNDIS. other users may share their experiences.

Hi DaneLLL

Can give me a USB device gadget sample, Read-Write Data passed USB 3.0 mode. I want transfer video data ,use USB 3.0. But, USB device gadget no experience.

FYI, the TX2’s only compatible port for device mode (assuming dev carrier) is the micro-OTG port. This port is not capable of USB3 mode…it lacks the wiring.

USB in device mode allows the Jetson to act as if it is a device instead of as a host. The “gadget” framework exists to make this easier, but you have to use one of the standard USB classes, it isn’t a general data pipe like a UART. One example is that it could become a hard drive or an ethernet adapter. See:
https://www.kernel.org/doc/Documentation/usb/gadget_configfs.txt
http://www.linux-usb.org/gadget/

On the more recent releases there is a sample device mode configuration. Explore here:

/opt/nvidia/l4t-usb-device-mode/

You should know that the “OTG” type port (“on the go”) allows either a type-A connector (used at the host end) to be connected, or a type-B connector (used at the device end). There is an ID pin to detect which type. The full-sized connectors do not allow this as they lack both an ID pin and any ability to attach type-B.

One more important note: The OTG port does not support what is known as “isochronous” data streams. Some video or audio types might need to be “isochronous”.

Hi linuxdev

the full-size type-A ports(J19) use 3.0 device mode, I find RNDIS_host at 5000M mode. But, speed is low (12-18MB/s).

(/opt/nvidia/l4t-usb-device-mode/) no support read-write data at High speed.

root@zgq-virtual-machine:/home/nuctech# lsusb -t

/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
|__ Port 1: Dev 26, If 0, Class=Communications, Driver=rndis_host, 5000M
|__ Port 1: Dev 26, If 1, Class=CDC Data, Driver=rndis_host, 5000M
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 480M
|__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 3: Dev 4, If 0, Class=Hub, Driver=hub/7p, 12M
|__ Port 4: Dev 5, If 0, Class=Hub, Driver=hub/7p, 480M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/7p, 12M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/6p, 480M

The full size type-A ports use USB3 mode, but technically this is invalid for device mode. Did you make a custom cable (basically the USB equivalent to “crossover”…type-A to type-A is illegal/invalid)? If you built the cable, is it truly high enough quality for those data rates (a simple wire cut and solder with heat shrink probably won’t work at those rates without the signal quality suffering)?

Unfortunately, I don’t know much about RNDIS, but obviously it is being emulated in software (there isn’t real network hardware) and so I’d expect it to have a lot of possible bottlenecks (bottlenecks which may not be due to the TX2, but instead due to software emulation of hardware…don’t know for sure, I’m just speculating). I also have no idea how performance will be impacted by the VM (I assume this is only running on the desktop PC). Do be sure that on the TX2 you have first maximized performance:

sudo nvpmodel -m 0
sudo ~ubuntu/jetson_clocks.sh

Just to see if a CPU core is being saturated watch to see if a core goes near 100% during your testing. You might install “htop” (“sudo apt-get install htop”). I will guess CPU0 is going to go up a lot, but maybe not.