参考:sprugx8b.pdf
DM816x_AM389x_PCI_Express_Root_Complex_Driver_User_Guide.pdf
DM816x_AM389x_PCI_Express_Endpoint_Boot_Driver_User_Guide.pdf
TMS320dm8168.pdf
http://processors.wiki.ti.com/index.php/TI81XX_PSP_PCI_Express_Root_Complex_Driver_User_Guide#RC_Driver_Source_Files
对其中有助于PCIe开发的细节进行提取,虽然文档总结费时间,但是对于经验扩散来说无可替代。
列出几个关键的特征,应该在PCIe开发过程中牢记,部分没有列出参考<17.1.2 sprugx8b.pdf>
Ø x2 Lanes
Ø Maximum outbound payload size of 128bytes。对于TLP的负载设置应该用的到。
Ø Maximum inbound payload size of 256bytes.
Ø Maximum read request size of 256bytes.此参数在于EP的通信中很重要。
Ø 不支持热插拔
Ø 不支持WAKE#
1.5节,1.16节。sprugx8b.pdf
17.1.4.1节sprugx8b.pdf,使用PCIe的前提是配置DM8168PCIe接口的工作模式,在这里我们配置成RC模式。其配置寄存器为:
baseaddr参考1.5.2.1节Table1-12 L4Standard Peripheral Memory Map, Control Module baseaddr0x4814000~0x4815FFFF. 结合Table1-207可知PCIe_CFG baseaddr为0x48140640, 参考1.16.1.3.16节设置PCIE_DEVTYPE为0b10,Root Complex(RC)operation.
配置62005时钟的时候注意时钟误差范围不要超过300ppm,62005可以设置。
根据17.1.5描述,PCIe子系统支持一个link,也就是说DM8168只能连接一个EP,所以DM8168不能当做switch使用了,而且lane 0必须优先使用,lane 1不能单独使用,x1 只能是lane0 ,x2 是lane0 lane1。
250Mhz is aclock for OCP interfaces of the subsystem, is present at SYSCLK5(17.2.1 sprugx8b.pdf), which in turn uses the 27-Mhz crystal input from the main pLL.
只有两种类型的地址空间需要映射,分别是Configuration 和 Memory 地址空间。物理设备(physical device)的地址空间分为两部分,分别是Range0和Range1,Range0被PCI configuration任务使用,作为configuraton space。Range1用作memory映射。17.2.3
51000000-51003fff: pcie-regs,此段地址空间保存8168PCIE register的值,如下图所示。
应该需要主控配置inbound translation bar,用mem_rdwr.out工具查看初始状态可以看出,内核并没有配置这段寄存器。如下图所示:
8168内核扫描XX卡时内核打印:
ti81xx_pcie:PCI scan done.
pci0000:00:00.0: BAR 9: can't assign mem pref (size 0x20000000)
pci 0000:00:00.0: BAR 8: assigned [mem 0x20000000-0x206fffff]
pci 0000:00:00.0: BAR 7: assigned [io 0x40000000-0x40001fff]
pci 0000:01:00.0: BAR 9: can't assign mem pref (size 0x20000000)
pci 0000:01:00.0: BAR 8: assigned [mem 0x20000000-0x205fffff]
pci 0000:01:00.0: BAR 0: assigned [mem 0x20600000-0x2061ffff]
pci 0000:01:00.0: BAR 0: set to [mem 0x20600000-0x2061ffff] (PCI address[0x20600000-0x2061ffff])
pci 0000:01:00.0: BAR 7: assigned [io 0x40000000-0x40001fff]
pci 0000:02:05.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:06.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:08.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:09.0: BAR 9: can't assign mem pref (size 0x8000000)
pci 0000:02:05.0: BAR 8: assigned [mem 0x20000000-0x201fffff]
pci 0000:02:06.0: BAR 8: assigned [mem 0x20200000-0x202fffff]
pci 0000:02:08.0: BAR 8: assigned [mem 0x20300000-0x203fffff]
pci 0000:02:09.0: BAR 8: assigned [mem 0x20400000-0x205fffff]
pci 0000:02:05.0: BAR 7: assigned [io 0x40000000-0x40000fff]
pci 0000:02:09.0: BAR 7: assigned [io 0x40001000-0x40001fff]
pci 0000:02:04.0: PCI bridge to [bus 03-03]
pci 0000:02:04.0: bridge window [io disabled]
pci 0000:02:04.0: bridge window [mem disabled]
pci 0000:02:04.0: bridge window [mem pref disabled]
pci 0000:04:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:04:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:04:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:04:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:04:00.0: BAR 0: assigned [mem 0x20000000-0x20000fff]
pci 0000:04:00.0: BAR 0: set to [mem 0x20000000-0x20000fff] (PCI address[0x20000000-0x20000fff])
pci 0000:02:05.0: PCI bridge to [bus 04-04]
pci 0000:02:05.0: bridge window [io 0x40000000-0x40000fff]
pci 0000:02:05.0: bridge window [mem 0x20000000-0x201fffff]
pci 0000:02:05.0: bridge window [mem pref disabled]
pci 0000:05:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:05:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:05:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:05:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:05:00.0: BAR 0: assigned [mem 0x20200000-0x20200fff]
pci 0000:05:00.0: BAR 0: set to [mem 0x20200000-0x20200fff] (PCI address[0x20200000-0x20200fff])
pci 0000:02:06.0: PCI bridge to [bus 05-05]
pci 0000:02:06.0: bridge window [io disabled]
pci 0000:02:06.0: bridge window [mem 0x20200000-0x202fffff]
pci 0000:02:06.0: bridge window [mem pref disabled]
pci 0000:06:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:06:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:06:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:06:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:06:00.0: BAR 0: assigned [mem 0x20300000-0x20300fff]
pci 0000:06:00.0: BAR 0: set to [mem 0x20300000-0x20300fff] (PCI address[0x20300000-0x20300fff])
pci 0000:02:08.0: PCI bridge to [bus 06-06]
pci 0000:02:08.0: bridge window [io disabled]
pci 0000:02:08.0: bridge window [mem 0x20300000-0x203fffff]
pci 0000:02:08.0: bridge window [mem pref disabled]
pci 0000:07:00.0: BAR 4: can't assign mem pref (size 0x4000000)
pci 0000:07:00.0: BAR 3: can't assign mem pref (size 0x2000000)
pci 0000:07:00.0: BAR 1: can't assign mem pref (size 0x1000000)
pci 0000:07:00.0: BAR 2: can't assign mem pref (size 0x1000000)
pci 0000:07:00.0: BAR 0: assigned [mem 0x20400000-0x20400fff]
pci 0000:07:00.0: BAR 0: set to [mem 0x20400000-0x20400fff] (PCI address[0x20400000-0x20400fff])
pci 0000:02:09.0: PCI bridge to [bus 07-07]
pci 0000:02:09.0: bridge window [io 0x40001000-0x40001fff]
pci 0000:02:09.0: bridge window [mem 0x20400000-0x205fffff]
pci 0000:02:09.0: bridge window [mem pref disabled]
pci 0000:01:00.0: PCI bridge to [bus 02-07]
pci 0000:01:00.0: bridge window [io 0x40000000-0x40001fff]
pci 0000:01:00.0: bridge window [mem 0x20000000-0x205fffff]
pci 0000:01:00.0: bridge window [mem pref disabled]
pci 0000:00:00.0: PCI bridge to [bus 01-07]
pci 0000:00:00.0: bridge window [io 0x40000000-0x40001fff]
pci 0000:00:00.0: bridge window [mem 0x20000000-0x206fffff]
pci 0000:00:00.0: bridge window [mem pref disabled]
PCI: enabling device 0000:00:00.0 (0140 -> 0143)
pci 0000:00:00.0: BAR 9: assigned [mem 0x20700000-0x20afffff pref]
pci 0000:01:00.0: BAR 9: assigned [mem 0x20700000-0x20afffff 64bit pref]
pci 0000:02:05.0: BAR 9: assigned [mem 0x20700000-0x208fffff 64bit pref]
pci 0000:02:09.0: BAR 9: assigned [mem 0x20900000-0x20afffff 64bit pref]
bio: create slab
gpio_vr gpio_vr: failed to get the gpios
从打印可以看出< pci 0000:00:00.0:BAR 9: can't assign mem pref (size 0x20000000)>有512M的空间没有分配成功。
在DM816x_AM389x_PCI_Express_Root_Complex_Driver_User_Guide.pdf中有说明:
也就是说内核初始分配空间256MB,而ti6678申请的空间溢出。这个问题可以通过增大outbound Memory解决,在:kernel/arch/arm/mach-omap2/devices.c中修改如下:
将其增加到1G(实际最大能到2G,先改成1G试试),从新编译内核烧写。打印如下:
ti81xx_pcie: PCI scan done.
pci 0000:00:00.0: BAR 9: assigned [mem0x20000000-0x3fffffff pref]
pci 0000:00:00.0: BAR 8: assigned [mem0x40000000-0x406fffff]
pci 0000:00:00.0: BAR 7: assigned [io 0x40000000-0x40001fff]
pci 0000:01:00.0: BAR 9: assigned [mem0x20000000-0x3fffffff pref]
pci 0000:01:00.0: BAR 8: assigned [mem0x40000000-0x405fffff]
pci 0000:01:00.0: BAR 0: assigned [mem0x40600000-0x4061ffff]
pci 0000:01:00.0: BAR 0: set to [mem0x40600000-0x4061ffff] (PCI address [0x40600000-0x4061ffff])
pci 0000:01:00.0: BAR 7: assigned [io 0x40000000-0x40001fff]
pci 0000:02:05.0: BAR 9: assigned [mem0x20000000-0x27ffffff pref]
pci 0000:02:06.0: BAR 9: assigned [mem0x28000000-0x2fffffff pref]
pci 0000:02:08.0: BAR 9: assigned [mem0x30000000-0x37ffffff pref]
pci 0000:02:09.0: BAR 9: assigned [mem0x38000000-0x3fffffff pref]
pci 0000:02:05.0: BAR 8: assigned [mem0x40000000-0x401fffff]
pci 0000:02:06.0: BAR 8: assigned [mem0x40200000-0x402fffff]
pci 0000:02:08.0: BAR 8: assigned [mem0x40300000-0x403fffff]
pci 0000:02:09.0: BAR 8: assigned [mem0x40400000-0x405fffff]
pci 0000:02:05.0: BAR 7: assigned [io 0x40000000-0x40000fff]
pci 0000:02:09.0: BAR 7: assigned [io 0x40001000-0x40001fff]
pci 0000:02:04.0: PCI bridge to [bus 03-03]
pci 0000:02:04.0: bridge window [io disabled]
pci 0000:02:04.0: bridge window [mem disabled]
pci 0000:02:04.0: bridge window [mem pref disabled]
pci 0000:04:00.0: BAR 4: assigned [mem 0x20000000-0x23ffffffpref]
pci 0000:04:00.0: BAR 4: set to [mem0x20000000-0x23ffffff pref] (PCI address [0x20000000-0x23ffffff])
pci 0000:04:00.0: BAR 3: assigned [mem0x24000000-0x25ffffff pref]
pci 0000:04:00.0: BAR 3: set to [mem0x24000000-0x25ffffff pref] (PCI address [0x24000000-0x25ffffff])
pci 0000:04:00.0: BAR 1: assigned [mem0x26000000-0x26ffffff pref]
pci 0000:04:00.0: BAR 1: set to [mem0x26000000-0x26ffffff pref] (PCI address [0x26000000-0x26ffffff])
pci 0000:04:00.0: BAR 2: assigned [mem 0x27000000-0x27ffffffpref]
pci 0000:04:00.0: BAR 2: set to [mem0x27000000-0x27ffffff pref] (PCI address [0x27000000-0x27ffffff])
pci 0000:04:00.0: BAR 0: assigned [mem0x40000000-0x40000fff]
pci 0000:04:00.0: BAR 0: set to [mem0x40000000-0x40000fff] (PCI address [0x40000000-0x40000fff])
pci 0000:02:05.0: PCI bridge to [bus 04-04]
pci 0000:02:05.0: bridge window [io 0x40000000-0x40000fff]
pci 0000:02:05.0: bridge window [mem 0x40000000-0x401fffff]
pci 0000:02:05.0: bridge window [mem 0x20000000-0x27ffffffpref]
pci 0000:05:00.0: BAR 4: assigned [mem0x28000000-0x2bffffff pref]
pci 0000:05:00.0: BAR 4: set to [mem0x28000000-0x2bffffff pref] (PCI address [0x28000000-0x2bffffff])
pci 0000:05:00.0: BAR 3: assigned [mem0x2c000000-0x2dffffff pref]
pci 0000:05:00.0: BAR 3: set to [mem0x2c000000-0x2dffffff pref] (PCI address [0x2c000000-0x2dffffff])
pci 0000:05:00.0: BAR 1: assigned [mem0x2e000000-0x2effffff pref]
pci 0000:05:00.0: BAR 1: set to [mem0x2e000000-0x2effffff pref] (PCI address [0x2e000000-0x2effffff])
pci 0000:05:00.0: BAR 2: assigned [mem0x2f000000-0x2fffffff pref]
pci 0000:05:00.0: BAR 2: set to [mem0x2f000000-0x2fffffff pref] (PCI address [0x2f000000-0x2fffffff])
pci 0000:05:00.0: BAR 0: assigned [mem0x40200000-0x40200fff]
pci 0000:05:00.0: BAR 0: set to [mem0x40200000-0x40200fff] (PCI address [0x40200000-0x40200fff])
pci 0000:02:06.0: PCI bridge to [bus 05-05]
pci 0000:02:06.0: bridge window [io disabled]
pci 0000:02:06.0: bridge window [mem 0x40200000-0x402fffff]
pci 0000:02:06.0: bridge window [mem 0x28000000-0x2fffffffpref]
pci 0000:06:00.0: BAR 4: assigned [mem0x30000000-0x33ffffff pref]
pci 0000:06:00.0: BAR 4: set to [mem0x30000000-0x33ffffff pref] (PCI address [0x30000000-0x33ffffff])
pci 0000:06:00.0: BAR 3: assigned [mem0x34000000-0x35ffffff pref]
pci 0000:06:00.0: BAR 3: set to [mem0x34000000-0x35ffffff pref] (PCI address [0x34000000-0x35ffffff])
pci 0000:06:00.0: BAR 1: assigned [mem0x36000000-0x36ffffff pref]
pci 0000:06:00.0: BAR 1: set to [mem0x36000000-0x36ffffff pref] (PCI address [0x36000000-0x36ffffff])
pci 0000:06:00.0: BAR 2: assigned [mem0x37000000-0x37ffffff pref]
pci 0000:06:00.0: BAR 2: set to [mem0x37000000-0x37ffffff pref] (PCI address [0x37000000-0x37ffffff])
pci 0000:06:00.0: BAR 0: assigned [mem0x40300000-0x40300fff]
pci 0000:06:00.0: BAR 0: set to [mem0x40300000-0x40300fff] (PCI address [0x40300000-0x40300fff])
pci 0000:02:08.0: PCI bridge to [bus 06-06]
pci 0000:02:08.0: bridge window [io disabled]
pci 0000:02:08.0: bridge window [mem 0x40300000-0x403fffff]
pci 0000:02:08.0: bridge window [mem 0x30000000-0x37ffffffpref]
pci 0000:07:00.0: BAR 4: assigned [mem0x38000000-0x3bffffff pref]
pci 0000:07:00.0: BAR 4: set to [mem0x38000000-0x3bffffff pref] (PCI address [0x38000000-0x3bffffff])
pci 0000:07:00.0: BAR 3: assigned [mem0x3c000000-0x3dffffff pref]
pci 0000:07:00.0: BAR 3: set to [mem0x3c000000-0x3dffffff pref] (PCI address [0x3c000000-0x3dffffff])
pci 0000:07:00.0: BAR 1: assigned [mem0x3e000000-0x3effffff pref]
pci 0000:07:00.0: BAR 1: set to [mem0x3e000000-0x3effffff pref] (PCI address [0x3e000000-0x3effffff])
pci 0000:07:00.0: BAR 2: assigned [mem0x3f000000-0x3fffffff pref]
pci 0000:07:00.0: BAR 2: set to [mem0x3f000000-0x3fffffff pref] (PCI address [0x3f000000-0x3fffffff])
pci 0000:07:00.0: BAR 0: assigned [mem0x40400000-0x40400fff]
pci 0000:07:00.0: BAR 0: set to [mem0x40400000-0x40400fff] (PCI address [0x40400000-0x40400fff])
pci 0000:02:09.0: PCI bridge to [bus 07-07]
pci 0000:02:09.0: bridge window [io 0x40001000-0x40001fff]
pci 0000:02:09.0: bridge window [mem 0x40400000-0x405fffff]
pci 0000:02:09.0: bridge window [mem 0x38000000-0x3fffffffpref]
pci 0000:01:00.0: PCI bridge to [bus 02-07]
pci 0000:01:00.0: bridge window [io 0x40000000-0x40001fff]
pci 0000:01:00.0: bridge window [mem 0x40000000-0x405fffff]
pci 0000:01:00.0: bridge window [mem 0x20000000-0x3fffffffpref]
pci 0000:00:00.0: PCI bridge to [bus 01-07]
pci 0000:00:00.0: bridge window [io 0x40000000-0x40001fff]
pci 0000:00:00.0: bridge window [mem 0x40000000-0x406fffff]
pci 0000:00:00.0: bridge window [mem 0x20000000-0x3fffffffpref]
PCI: enabling device 0000:00:00.0 (0140-> 0143)
资源分配成功。