I have a strange issue. I am doing some tests running fastboot as my bootloader in place of u-boot and all functionality of the kernel and bootloader work fine except for one major function, Mini PCIe does not work even though the USB part of the mini pcie wifi module is working. So I have Bluetooth USB through the module but no PCIe wifi. Same exact DTB and Kernel PCIe Wifi/BT works fine running u-boot.
Does fastboot not have PCIe functionality or I am I missing something in DTB that enables PCIe?
Any help is immensely appreciated…
This is all just some thoughts on the topic, I can’t say for sure…
The USB D+/D- pair is optional in a mini-PCIe slot. Something using this is not dependent at all upon PCIe (though both could be used, the USB part is actually independent of PCIe). So it isn’t a PCIe problem at all if it is the USB data lines which don’t work.
I’m not sure about the TK1, but on TX1/TX2 (and probably TK1) only the USB2 drivers have been supported under U-Boot…USB3 mode requires a Tegra-specific driver to use with that controller and so USB3 under U-Boot implies porting a driver. If the controller of the D+/D- lanes are routed to can reach a USB2-capable controller (I’m not sure about what device tree options are available to those traces), then it might be a case of enabling the default XHCI drivers in fastboot (the driver already exists in U-Boot, possibly it exists in fastboot and only needs to be enabled). If those lanes go to a USB3 controller, then you’re probably out of luck. Find out first if your card requires USB3, or if USB2 works…then look for XHCI enable if USB2 will work. Once XHCI is enabled, check device tree for those traces to go to a USB2 controller not in conflict with device tree once the kernel loads.
Thanks for the response linuxdev. Some really good pointers, thank you. I did notice that an older flash.cfg that uses fastboot has a partition named DFI with xusb firmware. Maybe this is whats missing combined with xhci. I’ll do some more testing… Thanks again my friend!
[partition]
name=DFI
id=7
type=data
allocation_policy=sequential
filesystem_type=basic
size=2097152
file_system_attribute=0
partition_attribute=0
allocation_attribute=8
percent_reserved=0
filename=xusb_sil_rel_fw
Here is my board dts port mapping:
pcie-controller {
nvidia,hot-plug-gpio = <&gpio TEGRA_GPIO(O, 1) 0>;
nvidia,wake-gpio = <&gpio TEGRA_GPIO(DD, 3) 0>;
hvdd-pex-supply = <®_3v3_supply>;
hvdd-pex-pll-e-supply = <®_3v3_supply>;
dvddio-pex-supply = <&as3722_sd4>;
avddio-pex-supply = <&as3722_sd4>;
avdd-pex-pll-supply = <&as3722_sd4>;
vddio-pex-ctl-supply = <&as3722_sd4>;
nvidia,lane-map = <0x10>;
status = "okay";
pci@1,0 {
nvidia,disable-clock-request;
status = "okay";
};
pci@2,0 {
nvidia,disable-clock-request;
status = "okay";
};
};
xusb@70090000 {
/* nvidia,uses_external_pmic;
nvidia,gpio_controls_muxed_ss_lanes; */
nvidia,gpio_ss1_sata = <0>;
nvidia,portmap = <0x701>; /* SSP0, USB2P0, USBP2 */
nvidia,ss_portmap = <0x72>; /* SSP0 on USB2P2 */
nvidia,lane_owner = <6>; /* USB3P0 USB3P1 */
nvidia,ulpicap = <0>; /* No ulpi support. can we remove */
status = "okay";
};
Still no luck. Something is preventing the pcie rail from initializing. From what I understand the rtl8168 also uses the pcie port and I have eth0 so pcie must be working but not initializing the first port.
My home directory was lost and is being rebuilt, so my notes on this are not complete (it could be wrong). However, I think the TX1/TX2 connect the ethernet through a USB controller, not PCI. If that is the case, then it is probably also true for TK1.
Can someone verify if the rtl8168 on TK1 goes through a USB D+/D- lane, or from a PCIe lane? (Being without my notes is almost as bad as running out of coffee)
I am almost positive ethernet uses pci.
lspci in fastboot
01:00.0 Class 0200: 10ec:8168 ← using lane 1:00 on fastboot
Ethernet controller: Realtek Semiconductor Co., Ltd.
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
lspci in u-boot
01:00.0 Class 0280: 14e4:43ec ← Wont initialize in fastboot (pulling my hair out lol)
Broadcom 94356 Combo Wifi/BT Wireless Module
02:00.0 Class 0200: 10ec:8168 ← using lane 2:00 in u-boot
RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
Maybe this will help debug also:
fastboot
echo 1 > /sys/bus/pci/rescan
[ 550.419397] pcieport 0000:00:02.0: PCI bridge to [bus 01]
[ 550.424844] pcieport 0000:00:02.0: bridge window [io 0x1000-0x1fff]
[ 550.431414] pcieport 0000:00:02.0: bridge window [mem 0x13000000-0x130fffff]
[ 550.438668] pcieport 0000:00:02.0: bridge window [mem 0x20000000-0x200fffff 64bit pref]
uboot with pci enabled
echo 1 > /sys/bus/pci/rescan
[ 147.593700] pcieport 0000:00:01.0: PCI bridge to [bus 01]
[ 147.599395] pcieport 0000:00:01.0: bridge window [mem 0x13000000-0x135fffff]
[ 209.086158] pcieport 0000:00:02.0: PCI bridge to [bus 02]
[ 209.091613] pcieport 0000:00:02.0: bridge window [io 0x1000-0x1fff]
[ 209.098180] pcieport 0000:00:02.0: bridge window [mem 0x13600000-0x136fffff]
[ 209.105449] pcieport 0000:00:02.0: bridge window [mem 0x20000000-0x200fffff 64bit pref]
I can see some differences in the mem above so that may be my issue now how to fix?
Full log with pci working in u-boot:
[ 0.610346] tegra-pcie 1003000.pcie-controller: PCIE: Enable power rails
[ 0.611141] tegra-pcie 1003000.pcie-controller: probing port 0, using 4 lanes and lane map as 0x10
[ 0.613223] tegra-pcie 1003000.pcie-controller: probing port 1, using 1 lanes and lane map as 0x10
[ 0.648797] tegra-pcie 1003000.pcie-controller: PCI host bridge to bus 0000:00
[ 0.648817] pci_bus 0000:00: root bus resource [mem 0x13000000-0x1fffffff]
[ 0.648833] pci_bus 0000:00: root bus resource [mem 0x20000000-0x3fffffff pref]
[ 0.648875] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.648891] pci_bus 0000:00: root bus resource [io 0x1000-0xffff]
[ 0.648941] pci 0000:00:01.0: [10de:0e12] type 01 class 0x060400
[ 0.649075] pci 0000:00:01.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.649344] pci 0000:00:02.0: [10de:0e13] type 01 class 0x060400
[ 0.649447] pci 0000:00:02.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.649729] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.649749] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.650014] pci 0000:01:00.0: [14e4:43ec] type 00 class 0x028000
[ 0.650055] pci 0000:01:00.0: reg 10: [mem 0x00000000-0x00007fff 64bit]
[ 0.650104] pci 0000:01:00.0: reg 18: [mem 0x00000000-0x003fffff 64bit]
[ 0.650271] pci 0000:01:00.0: supports D1 D2
[ 0.650282] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.651987] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[ 0.652212] pci 0000:02:00.0: [10ec:8168] type 00 class 0x020000
[ 0.652264] pci 0000:02:00.0: reg 10: [io 0x0000-0x00ff]
[ 0.652329] pci 0000:02:00.0: reg 18: [mem 0x00000000-0x00000fff 64bit]
[ 0.652363] pci 0000:02:00.0: reg 20: [mem 0x00000000-0x00003fff 64bit pref]
[ 0.652485] pci 0000:02:00.0: supports D1 D2
[ 0.652496] pci 0000:02:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.653973] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
[ 0.654123] pci 0000:00:01.0: BAR 8: assigned [mem 0x13000000-0x135fffff]
[ 0.654141] pci 0000:00:02.0: BAR 8: assigned [mem 0x13600000-0x136fffff]
[ 0.654177] pci 0000:00:02.0: BAR 9: assigned [mem 0x20000000-0x200fffff 64bit pref]
[ 0.654195] pci 0000:00:02.0: BAR 7: assigned [io 0x1000-0x1fff]
[ 0.654226] pci 0000:01:00.0: BAR 2: assigned [mem 0x13000000-0x133fffff 64bit]
[ 0.654259] pci 0000:01:00.0: BAR 0: assigned [mem 0x13400000-0x13407fff 64bit]
[ 0.654287] pci 0000:00:01.0: PCI bridge to [bus 01]
[ 0.654311] pci 0000:00:01.0: bridge window [mem 0x13000000-0x135fffff]
[ 0.654334] pci 0000:02:00.0: BAR 4: assigned [mem 0x20000000-0x20003fff 64bit pref]
[ 0.654364] pci 0000:02:00.0: BAR 2: assigned [mem 0x13600000-0x13600fff 64bit]
[ 0.654393] pci 0000:02:00.0: BAR 0: assigned [io 0x1000-0x10ff]
[ 0.654410] pci 0000:00:02.0: PCI bridge to [bus 02]
[ 0.654443] pci 0000:00:02.0: bridge window [io 0x1000-0x1fff]
[ 0.654459] pci 0000:00:02.0: bridge window [mem 0x13600000-0x136fffff]
[ 0.654489] pci 0000:00:02.0: bridge window [mem 0x20000000-0x200fffff 64bit pref]
[ 0.654734] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
[ 0.654762] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[ 0.654779] pcie_pme 0000:00:01.0:pcie01: service driver pcie_pme loaded
[ 0.655045] pcieport 0000:00:02.0: Signaling PME through PCIe PME interrupt
[ 0.655061] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
[ 0.655077] pcie_pme 0000:00:02.0:pcie01: service driver pcie_pme loaded
[ 0.655184] tegra-pcie 1003000.pcie-controller: PCIE: No Link speed change happened
[ 4.423331] ehci-pci: EHCI PCI platform driver
[ 9.809250] dhdpcie_init: found adapter info 'DHD generic adapter'
[ 143.093970] pci_bus 0000:01: busn_res: [bus 01] end is updated to 01
[ 143.094008] pci_bus 0000:02: busn_res: [bus 02] end is updated to 02
[ 143.094052] pcieport 0000:00:01.0: PCI bridge to [bus 01]
[ 143.099910] pcieport 0000:00:01.0: bridge window [mem 0x13000000-0x135fffff]
[ 143.107286] pcieport 0000:00:02.0: PCI bridge to [bus 02]
[ 143.112746] pcieport 0000:00:02.0: bridge window [io 0x1000-0x1fff]
[ 143.119494] pcieport 0000:00:02.0: bridge window [mem 0x13600000-0x136fffff]
[ 143.126853] pcieport 0000:00:02.0: bridge window [mem 0x20000000-0x200fffff 64bit pref]
[ 147.593620] pci_bus 0000:01: busn_res: [bus 01] end is updated to 01
[ 147.593658] pci_bus 0000:02: busn_res: [bus 02] end is updated to 02
[ 147.593700] pcieport 0000:00:01.0: PCI bridge to [bus 01]
[ 147.599395] pcieport 0000:00:01.0: bridge window [mem 0x13000000-0x135fffff]
[ 147.607018] pcieport 0000:00:02.0: PCI bridge to [bus 02]
[ 147.612472] pcieport 0000:00:02.0: bridge window [io 0x1000-0x1fff]
[ 147.619401] pcieport 0000:00:02.0: bridge window [mem 0x13600000-0x136fffff]
[ 147.626711] pcieport 0000:00:02.0: bridge window [mem 0x20000000-0x200fffff 64bit pref]
[ 180.838965] dhdpcie_download_code_file: download firmware /system/vendor/firmware/fw_bcmdhd.bin
[ 181.095113] dhdpcie_bus_write_vars: Download, Upload and compare of NVRAM succeeded.
[ 209.072101] pci_bus 0000:01: busn_res: [bus 01] end is updated to 01
[ 209.072137] pci_bus 0000:02: busn_res: [bus 02] end is updated to 02
[ 209.072177] pcieport 0000:00:01.0: PCI bridge to [bus 01]
[ 209.078892] pcieport 0000:00:01.0: bridge window [mem 0x13000000-0x135fffff]
[ 209.086158] pcieport 0000:00:02.0: PCI bridge to [bus 02]
[ 209.091613] pcieport 0000:00:02.0: bridge window [io 0x1000-0x1fff]
[ 209.098180] pcieport 0000:00:02.0: bridge window [mem 0x13600000-0x136fffff]
[ 209.105449] pcieport 0000:00:02.0: bridge window [mem 0x20000000-0x200fffff 64bit pref]
What I would suggest next is to examine the mini-PCIe card itself. Does it specify any specific requirements on PCIe? Are there any detailed descriptions of the card you are using?
Also adjusting the lane_owner_info=0/lane_owner_info=6 does nothing. I am assuming pci is being disabled at the bootloader level somehow since u-boot works perfectly same kernel same dtb…
NVIDIA team please comment on this issue. Thank you kindly for any response with any insight.
Thanks for the response linuxdev ;)
I wouldn’t think i need to check anything since I can flash u-boot and the card works perfect same kernel same dtb same OS.
The reason I suggest finding some technical details of the card is that some PCI features are optional. Not all options are supported equally among drivers. A reason for working in the kernel, but not in U-Boot, could be nothing more than an option which is missing in U-Boot…seeing which options are required might trigger some good info.
Okay I know its not a specific card issue because I now have tried 3 different pcie cards I know worked in u-boot but all 3 do not work in fastboot.
u-boot works just fine kernel is working fine under u-boot. When using fastboot everything works fine except for pcie. I’’ look into more documentation on pcie wifi but this just seems like a fastboot issue. u-boot must enable pcie but fastboot is missing the function to enable pcie.
Would it be possible to test the PCIe voltage between u-boot and fastboot to make sure the pcie rail is getting power and if so how? Occiliscop and or voltage meter?
I couldn’t tell you how to verify if PCIe has rail power…but this would not be a good test anyway since rails may be shared and rail setup would often be correct even if the rest of the drivers related to this are not installed.
Does anyone here who has worked on fastboot know if basic PCIe function is installed?
Okay I spent the day doing some lane map testing and sure enough my lane mappings were off for pci which is odd because the map 0x10 I used for uboot worked somehow but now with fastboot 0x14 seems to have enabled the second pci lane. I still haven’t tried 0x12 but I just wanted to report this issue is solved :D
Glad to hear it works! I would also be curious about the nature of how lane mappings would require differences between the two boot loaders if you find more information on that topic.
I will do some uboot vs fastboot tests for different lane mapping senarios and report back.
0x14 was wrong actually 0x12 enables both pcie lanes for rtl eth0 and wlan0 and you must also add:
iommus = <&smmu TEGRA_SWGROUP_AFI>; ← Fixes DMA mapping for the Realtek LAN chip
to the jetson-tk1 dts like so:
pcie-controller {
hvdd-pex-supply = <®_3v3_supply>;
hvdd-pex-pll-e-supply = <®_3v3_supply>;
dvddio-pex-supply = <&as3722_sd4>;
avddio-pex-supply = <&as3722_sd4>;
avdd-pex-pll-supply = <&as3722_sd4>;
vddio-pex-ctl-supply = <&as3722_sd4>;
nvidia,lane-map = <0x12>;
status = "okay";
iommus = <&smmu TEGRA_SWGROUP_AFI>;
pci@1,0 {
nvidia,disable-clock-request;
status = "okay";
};
pci@2,0 {
nvidia,disable-clock-request;
status = "okay";
};
};