In the L4T R28 for TX1 the USB lane mapping is bug?

chunsheng.geng · November 2, 2017, 5:43am

In the Tegra_Linux_Driver_Package_TX1_Adaptation_Guide.pdf(2017-10-03)
“R28 Example 2” show :
pcie-controller {
[…]
pci@1,0 {
- nvidia,num-lanes = <4>;
+ nvidia,num-lanes = <1>;
};

		pci@2,0 { 
				nvidia,num-lanes = <1>; 
		}; 
   };

[…]

It is not a valid configuration and it is falling back to 4x1, 1x1 configuration which is default in tegra_pcie_get_xbar_config(),if set ‘nvidia,num-lanes = <1>’ for ‘pci@1,0’.

The first question:Is that a mistake at “R28 Example 2” in the Tegra_Linux_Driver_Package_TX1_Adaptation_Guide.pdf ?

Now I setting the ‘nvidia,num-lanes = <2>’ for ‘pci@1,0’.
The USB_SS#2 on Lane3 can work very well.But PCIe#0_0 and PCIe#1_0 are both can not to work.

And if setting the ‘nvidia,num-lanes = <4>’ for ‘pci@1,0’.
The USB_SS#2 on Lane3 and PCIe#1_0 can work very well. But PCIe#0_0 can not to work.

How to fix this issue?

DaneLLL · November 2, 2017, 6:11am

It seems like the DTB is not well flashed. Do you follow [url]Jetson/TX2 DTB - eLinux.org ?

chunsheng.geng · November 2, 2017, 6:36am

Hi DaneLLL,

sudo ./flash -r -k DTB jetson-tx1 mmcblk0p1

sudo ./flash jetson-tx1 mmcblk0p1

I’ve tested these two methods。

chunsheng.geng · November 2, 2017, 6:38am

I found the same problem，it doesn’t seem to work out too.

DaneLLL · November 2, 2017, 7:32am

Hi chunsheng,
For PCIe#1_0 @ PEX1, you shall need to change num-lanes of PCIe#0_0,

pci@1,0 { 
- nvidia,num-lanes = <4>; 
+ nvidia,num-lanes = <1>; 
};

apply the patch,
https://devtalk.nvidia.com/default/topic/1022604/jetson-tx1/r28-1-how-to-change-to-pcie-config-4-/post/5207806/#5207806

and check E50 per
https://devtalk.nvidia.com/default/topic/1025513/jetson-tx2/pcie-usb-bridge-can-t-be-detected-by-tx2/post/5216264/#5216264

chunsheng.geng · November 2, 2017, 9:04am

Update the DTB again: TEGRA_GPIO(I, 2) set output low,nvidia,num-lanes = <1>.
The USB_SS#2 on Lane3 and PCIe#0_0 can not to work.

lspci
00:02.0 PCI bridge: NVIDIA Corporation Device 0faf (rev a1)
01:00.0 Multimedia video controller: Device 1cd7:0011 (rev 01)

lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-tegra/4p, 5000M
|__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
|__ Port 3: Dev 3, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 1: Dev 4, If 0, Class=Hub, Driver=hub/4p, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci-tegra/5p, 480M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 3: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 4: Dev 5, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 4: Dev 5, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M

dmesg

[    3.579144] xhci-tegra 70090000.xusb: can't get usb3-0 phy (-517)
[    3.579809] tegra21x-padctl-uphy 7009f000.pinctrl: padctl mmio start 0x000000007009f000 end 0x000000007009ffff
[    3.579819] tegra21x-padctl-uphy 7009f000.pinctrl: TEGRA_FUSE_SKU_CALIB_0 0xaa4a414
[    3.579827] tegra21x-padctl-uphy 7009f000.pinctrl: TEGRA_FUSE_USB_CALIB_EXT_0 0x4
[    3.581154] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-1 is used by XUSB
[    3.581172] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 6, set usb3_lanes = 0x40
[    3.581188] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 0 maps to USB2 port 1
[    3.581195] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-0 is used by XUSB
[    3.581209] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 3, set usb3_lanes = 0x48
[    3.581222] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 2 maps to USB2 port 0
[    3.581229] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-2 is used by XUSB
[    3.581241] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 5, set usb3_lanes = 0x68
[    3.581254] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 1 maps to USB2 port 2
[    3.581260] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-3 is used by XUSB
[    3.581277] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 4, set pcie_lanes = 0x10
[    3.581295] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 0, set pcie_lanes = 0x11
[    3.581315] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 7, set sata_lanes = 0x80
[    3.582091] usb-vbus1: supplied by vdd-3v3
[    3.584685] tegra21x-padctl-uphy 7009f000.pinctrl: Done tegra21x_padctl_uphy_probe
[    3.587384] tegra-pcie 1003000.pcie-controller: wrong configuration updated in DT, switching to default 4x1, 1x1 configuration
[    3.587614] tegra-pcie 1003000.pcie-controller: PCIE: Enable power rails
[    3.588316] clk_cbus_round_rate: no gbus parent
[    3.588323] tegra_gpu_edp gpu_edp: unable to get max GPU freq
[    3.588716] tegra-pcie 1003000.pcie-controller: probing port 0, using 4 lanes
[    3.590551] gk20a 57000000.gpu: GPCPLL initial settings: NA mode, M=1, N=34, P=3
[    3.590947] tegra-pcie 1003000.pcie-controller: probing port 1, using 1 lanes

dts file fragment

gpio@6000d000 {
		wlan-default-output-low {
			gpio-hog;
			output-low;
			gpios = <
				TEGRA_GPIO(I, 2) 0
				>;
		};
	};
	
	
	pcie-controller@1003000 {
		nvidia,wake-gpio = <&gpio TEGRA_GPIO(A, 2) 0>;
		dvdd-pex-pll-supply = <&max77620_ldo1>;
		l0-dvddio-pex-supply = <&max77620_ldo1>;
		l1-dvddio-pex-supply = <&max77620_ldo1>;
		l2-dvddio-pex-supply = <&max77620_ldo1>;
		l3-dvddio-pex-supply = <&max77620_ldo1>;
		l4-dvddio-pex-supply = <&max77620_ldo1>;
		l5-dvddio-pex-supply = <&max77620_ldo1>;
		l6-dvddio-pex-supply = <&max77620_ldo1>;
		hvdd-pex-pll-e-supply = <&max77620_sd3>;
		l0-hvddio-pex-supply = <&max77620_sd3>;
		l1-hvddio-pex-supply = <&max77620_sd3>;
		l2-hvddio-pex-supply = <&max77620_sd3>;
		l3-hvddio-pex-supply = <&max77620_sd3>;
		l4-hvddio-pex-supply = <&max77620_sd3>;
		l5-hvddio-pex-supply = <&max77620_sd3>;
		l6-hvddio-pex-supply = <&max77620_sd3>;
		vddio-pex-ctl-supply = <&max77620_sd3>;
		status = "okay";

		
		pci@1,0 {
			nvidia,num-lanes = <1>;
			status = "okay";
		};

		pci@2,0 {
			nvidia,num-lanes = <1>;
			status = "okay";
		};
	};	
	

	pinctrl@7009f000 {
		status = "okay";
		pinctrl-0 = <&tegra_padctl_uphy_pinmux_default>;
		pinctrl-names = "default";
		
		vbus-2-supply = <&battery_reg>;
		
		tegra_padctl_uphy_pinmux_default: pinmux {
			usb2-eth {
				 nvidia,lanes = "otg-1";
				 nvidia,function = "xusb";
				 nvidia,port-cap =
					<TEGRA_PADCTL_PORT_HOST_ONLY>;
			};
			usb3-eth {
				 nvidia,lanes = "uphy-lane-6";
				 nvidia,function = "usb3";
				 nvidia,usb3-port = <0>;
				 nvidia,usb2-map = <1>;
				 nvidia,port-cap =
					<TEGRA_PADCTL_PORT_HOST_ONLY>;
			};
			
			
			usb2-bottom {
				 nvidia,lanes = "otg-0";
				 nvidia,function = "xusb";
				 nvidia,port-cap =
					<TEGRA_PADCTL_PORT_HOST_ONLY>;
			};
			

			usb3-bottom {
				 nvidia,lanes = "uphy-lane-3";
				 nvidia,function = "usb3";
				 nvidia,usb3-port = <2>;
				 nvidia,usb2-map = <0>;
				 nvidia,port-cap =
					<TEGRA_PADCTL_PORT_HOST_ONLY>;
			};
	
		
			usb2-top {
				 nvidia,lanes = "otg-2";
				 nvidia,function = "xusb";
				 nvidia,port-cap =
					<TEGRA_PADCTL_PORT_HOST_ONLY>;
			};
			
			usb3-top {
				 nvidia,lanes = "uphy-lane-5";
				 nvidia,function = "usb3";
				 nvidia,usb3-port = <1>;
				 nvidia,usb2-map = <2>;
				 nvidia,port-cap =
					<TEGRA_PADCTL_PORT_HOST_ONLY>;
			};	
			
		
			usb2-mini-pcie {
				 nvidia,lanes = "otg-3";
				 nvidia,function = "xusb";
				 nvidia,port-cap =
					<TEGRA_PADCTL_PORT_HOST_ONLY>;
			};			

			hsic {
				 nvidia,lanes = "hsic-0";
				 nvidia,function = "hsic";
			};
		
			pcie-eth {
				 nvidia,lanes =	"uphy-lane-4";			
				 nvidia,function = "pcie";
				 nvidia,pcie-controller = <0>;
				 nvidia,pcie-lane-select =
					<TEGRA_PADCTL_PCIE_LANE_X1>;
			};

		
			pcie-mini {
				nvidia,lanes = "uphy-lane-0";
				nvidia,function = "pcie";
				nvidia,pcie-controller = <1>;
				nvidia,pcie-lane-select =
					<TEGRA_PADCTL_PCIE_LANE_X1>;
			};
		
			sata {
				 nvidia,lanes = "uphy-lane-7";
				 nvidia,function = "sata";
			};
		};
	};

	/* XUSB host mode*/
	xusb@70090000 {
		status = "okay";
		extcon-cables = <&vbus_gpio_extcon 1>;
		extcon-cable-names = "id";
		#extcon-cells = <1>;
		phys =
			<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(1)>,
			<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_USB3_P(0)>,
			<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(0)>,
			<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_USB3_P(2)>,
			<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(2)>,
			<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_USB3_P(1)>,
			<&tegra_padctl_uphy TEGRA_PADCTL_UPHY_UTMI_P(3)>;		
		phy-names = "utmi-1","usb3-0","utmi-0","usb3-2", "utmi-2","usb3-1", "utmi-3";
		nvidia,pmc-wakeup =
			<&tegra_pmc
				PMC_WAKE_TYPE_EVENT 41 PMC_TRIGGER_TYPE_HIGH>,
			<&tegra_pmc
				PMC_WAKE_TYPE_EVENT 44 PMC_TRIGGER_TYPE_HIGH>;
	};

DaneLLL · November 8, 2017, 1:08am

Hi chunsheng,
Can you share HW design of your board? On default carrier board, [B39 B40] are connected to USB2.0 micro AB, but it looks to be connected to USB3.0 type A on your board.

chunsheng.geng · November 8, 2017, 3:49am

Hi DaneLLL,

This is my HW design.
utmi-1 and USB_SS#0 connected to Ethernet;

utmi-0(B39,B40) and USB_SS#2(uphy-lane-3:D42,D43,G42,G43) connected to USB3.0 type A(USB3.0 PORT1);
utmi-2(A38,A39) and USB_SS#1(uphy-lane-5:C43,C44,F43,F44) connected to USB3.0 type A(USB3.0 PORT2);
utmi-3(B42,B43) connected to MiniPCIe(uphy-lane-0).

DaneLLL · November 14, 2017, 5:31am

Hi chunsheng,
By setting

pci@1,0 { 
nvidia,num-lanes = <4>; 
};

Only PCIe#0_0 does not work?

Have you checked reset pin PEX0_RST#(C49) per
https://devtalk.nvidia.com/default/topic/1025513/jetson-tx2/pcie-usb-bridge-can-t-be-detected-by-tx2/post/5216264/#5216264

DaneLLL · November 14, 2017, 9:07am

Hi chunsheng,
We have confirmed the following change is not correct:

pci@1,0 {
-  nvidia,num-lanes = <4>;
+  nvidia,num-lanes = <1>;
};

Please keep it unchanged and do

pcie {
-         nvidia,lanes = "uphy-lane-1", "uphy-lane-2",
-                         "uphy-lane-3", "uphy-lane-4"; 
+         nvidia,lanes = "uphy-lane-4";
          nvidia,function = "pcie";
          nvidia,pcie-controller = <0>;
          nvidia,pcie-lane-select =
-                          <TEGRA_PADCTL_PCIE_LANE_X4>;
+                          <TEGRA_PADCTL_PCIE_LANE_X1>;
};

It is SW configuration. If you still see issues, please check reset pin PEX0_RST#(C49) per
https://devtalk.nvidia.com/default/topic/1025513/jetson-tx2/pcie-usb-bridge-can-t-be-detected-by-tx2/post/5216264/#5216264

chunsheng.geng · November 15, 2017, 3:16am

Hi, DaneLLL

The PEX0 connect to a WG82574L chipset on our custom board ,and PEX0_RST connnect to PE_RST_N which on WG82574L pin17.

The WG82574L can be found if PCIE#0 work on LANE_X4 mode,so I think the PEX0_RST#(C49) is normal.

Follow the link:

I also changed the pci-tegra.c,but is still no progress.

vidyas:

you can have following change to delay applying reset to end point by 100ms after refclk started flowing into it

diff --git a/drivers/pci/host/pci-tegra.c b/drivers/pci/host/pci-tegra.c
index 35d63342cabd..10243cab2c38 100644
--- a/drivers/pci/host/pci-tegra.c
+++ b/drivers/pci/host/pci-tegra.c
@@ -2033,6 +2033,7 @@ static void tegra_pcie_port_enable(struct tegra_pcie_port *port)
 
     afi_writel(port->pcie, value, ctrl);
 
+    mdelay(100);
     tegra_pcie_port_reset(port);
 
     /* On platforms where MXM is not directly connected to Tegra root port,

The log show “tegra-pcie 1003000.pcie-controller: failed to get PHY: -517”.

[    0.473305] tegra21x-padctl-uphy 7009f000.pinctrl: padctl mmio start 0x000000007009f000 end 0x000000007009ffff
[    0.473351] tegra21x-padctl-uphy 7009f000.pinctrl: TEGRA_FUSE_SKU_CALIB_0 0xaa4a414
[    0.473385] tegra21x-padctl-uphy 7009f000.pinctrl: TEGRA_FUSE_USB_CALIB_EXT_0 0x4
[    0.474925] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-1 is used by XUSB
[    0.474971] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 6, set usb3_lanes = 0x40
[    0.475016] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 0 maps to USB2 port 1
[    0.475051] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-0 is used by XUSB
[    0.475085] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 3, set usb3_lanes = 0x48
[    0.475125] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 2 maps to USB2 port 0
[    0.475158] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-2 is used by XUSB
[    0.475190] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 5, set usb3_lanes = 0x68
[    0.475230] tegra21x-padctl-uphy 7009f000.pinctrl: USB3 port 1 maps to USB2 port 2
[    0.475263] tegra21x-padctl-uphy 7009f000.pinctrl: UTMI-3 is used by XUSB
[    0.475300] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 4, set pcie_lanes = 0x10
[    0.475346] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 0, set pcie_lanes = 0x11
[    0.475393] tegra21x-padctl-uphy 7009f000.pinctrl: uphy_lane = 7, set sata_lanes = 0x80
[    0.475820] tegra21x-padctl-uphy 7009f000.pinctrl: mailbox is not ready yet
[    0.476863] tegra-pwm 7000a000.pwm: PWM clk can sleep in ops
[    0.477958] tegra-dfll-pwm 70110000.pwm: DFLL pwm-rate: 12800000
[    0.478713] Adding domain pcie-pd to PM domain mc-clk-pd
[    0.483406] tegra-pcie 1003000.pcie-controller: 4x1, 1x1 configuration
[    0.483474] tegra-pcie 1003000.pcie-controller: failed to get PHY: -517
[    0.483501] tegra-pcie 1003000.pcie-controller: failed to get PHYs: -517
[    0.487049] tsec 54500000.tsec: initialized
[    0.488414] tsec 54100000.tsecb: initialized
[    0.491547] nvdec 54480000.nvdec: initialized

dmesg.log (58.4 KB)

DaneLLL · November 15, 2017, 5:06am

Hi chunsheng,
Can you please try

pcie {
-         nvidia,lanes = "uphy-lane-1", "uphy-lane-2",
-                         "uphy-lane-3", "uphy-lane-4"; 
+         nvidia,lanes = "uphy-lane-4";
          nvidia,function = "pcie";
          nvidia,pcie-controller = <0>;
          nvidia,pcie-lane-select =
                          <TEGRA_PADCTL_PCIE_LANE_X4>;
};

chunsheng.geng · November 15, 2017, 5:44am

Hi DaneLLL,
Follow your advice,PCIe#0 and PCIe#1 are both can not detecte the devices if set “pcie-lane-select =<TEGRA_PADCTL_PCIE_LANE_X4>”.

DaneLLL:

pcie {
-         nvidia,lanes = "uphy-lane-1", "uphy-lane-2",
-                         "uphy-lane-3", "uphy-lane-4"; 
+         nvidia,lanes = "uphy-lane-4";
          nvidia,function = "pcie";
          nvidia,pcie-controller = <0>;
          nvidia,pcie-lane-select =
                          <TEGRA_PADCTL_PCIE_LANE_X4>;
};

DaneLLL · November 15, 2017, 5:51am

Hi chunsheng,
You mean by setting lane-0 to TEGRA_PADCTL_PCIE_LANE_X1 and lane-4 to TEGRA_PADCTL_PCIE_LANE_X4. Both do not work?

pcie-eth {
				 nvidia,lanes =	"uphy-lane-4";			
				 nvidia,function = "pcie";
				 nvidia,pcie-controller = <0>;
				 nvidia,pcie-lane-select =
					<TEGRA_PADCTL_PCIE_LANE_X4>;
			};
			pcie-mini {
				nvidia,lanes = "uphy-lane-0";
				nvidia,function = "pcie";
				nvidia,pcie-controller = <1>;
				nvidia,pcie-lane-select =
					<TEGRA_PADCTL_PCIE_LANE_X1>;
			};

chunsheng.geng · November 15, 2017, 6:18am

Hi DaneLLL,
Sorry,I just made a mistake.

Well, there’s good news,PCIe#0,PCIe#1,USB3-0,USB3-1,USB3-2 they are both can work,when set “pcie-lane-select =<TEGRA_PADCTL_PCIE_LANE_X4>”.

Thank you very much.

nvidia@tegra-ubuntu:~$ lspci 
	00:01.0 PCI bridge: NVIDIA Corporation Device 0fae (rev a1)
	00:02.0 PCI bridge: NVIDIA Corporation Device 0faf (rev a1)
	01:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
	02:00.0 Multimedia video controller: Device 1cd7:0013 (rev 01)
nvidia@tegra-ubuntu:~$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-tegra/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
    |__ Port 2: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
    |__ Port 3: Dev 3, If 0, Class=Hub, Driver=hub/4p, 5000M
        |__ Port 1: Dev 5, If 0, Class=Hub, Driver=hub/4p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci-tegra/5p, 480M
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 3, If 0, Class=Hub, Driver=hub/4p, 480M
            |__ Port 3: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
            |__ Port 3: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
            |__ Port 4: Dev 5, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M