Received-SPF: pass (google.com: domain of linux-kernel+bounces-150447-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
From: Frank Li <Frank.Li@nxp.com>
Subject: [PATCH v8 0/5] PCI: dwc: Add common pme_turn_off message by using
 outbound iATU
Date: Thu, 18 Apr 2024 12:04:23 -0400
Message-Id: <20240418-pme_msg-v8-0-a54265c39742@nxp.com>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
To: Bjorn Helgaas <bhelgaas@google.com>, Jingoo Han <jingoohan1@gmail.com>, 
 Gustavo Pimentel <gustavo.pimentel@synopsys.com>, 
 Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>, 
 Lorenzo Pieralisi <lpieralisi@kernel.org>, 
 =?utf-8?q?Krzysztof_Wilczy=C5=84ski?= <kw@linux.com>, 
 Rob Herring <robh@kernel.org>, 
 Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>, 
 Conor Dooley <conor+dt@kernel.org>, imx@lists.linux.dev
Cc: linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, 
 devicetree@vger.kernel.org, Frank Li <Frank.Li@nxp.com>, 
 Yoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>, 
 Serge Semin <fancer.lancer@gmail.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?bitKcStXUEZiUEM1UEJLYVd5YUdaZGZEa0RXdE9udlh2UHhRV25neXZpblZV?=
 =?utf-8?B?MHZuQzRCSXp6LzVMYWZsdGpFeEpVWDhMVitHWkRMUHY3NkVZeDNtajV0Um1T?=
 =?utf-8?B?bVBoVm8zVUhjNHlQM0VSUzZwRCtTdXEwRkQ1UjFmdklxTHJ5T1p6WjBGd1Ew?=
 =?utf-8?B?NEMvcFhMUmh6Q2pMZWZqenFkWVIzSTBRMjJMY1RiWEhGRWk5LzNMWk1WaTJY?=
 =?utf-8?B?eEZXTGRJMDE3RWNTSm9uRGNZWE90YWVHaVpONWt0TlRvV3VPWWlUSzZDSmVk?=
 =?utf-8?B?aFdsNmE0aWZ1VFhxNFZYNEhROWMxb0FQOHYyTmtTMENEc2FUS1lvM1dVVHRq?=
 =?utf-8?B?ZFJQeXc5dTF3WnBFSkJVRnBOY3UvN3BTekhyRUZ1LzVaK3dFYXFwTHI0V1p3?=
 =?utf-8?B?d1d0VStla1BMdXJQbmhsK2Jnb2dUc3l2anhSNlpjVHIzbnJ4MTJldHVqSEN5?=
 =?utf-8?B?aDBRVVhOZlhITXRvVWFqUVhVa1BLM0lvUkpGaC9aUzVZUkREK25jWWtYN2FP?=
 =?utf-8?B?TDNIa3VCYkJPZWJ6QkdDK2x3YUVXK1oxbXlZUHhqT1NIUEEwNkZYWWMybGZF?=
 =?utf-8?B?MHVnM1luUFRUU2VpSHhHUkFrWlE4NHBYdGlkUGxkbjBYWGY0cXVneDJ1YzZy?=
 =?utf-8?B?RkkzMXNoRTV2dDNZOWw3NXJpKzBvdjZDTnNNaDdFWGQ1ODZ5eVZaL0kyRnZ4?=
 =?utf-8?B?UTQxRGpaOEF5aVhzN1F5NU5ZV0JnVUtTOTZLN1ljWnYyc3FweGRlNHNUbDRV?=
 =?utf-8?B?SUJ4SDJhMjFGU2g0NFk0ZGYxN1JhZGR2RFhPeTNzcFBvYVIwdjB6SVF6U2Fx?=
 =?utf-8?B?ZzBDL3BOS2xhSk5WOEVDbmJUYUJRMWNrelRSL2NpeU5JMVRMbkw5bStZbTI0?=
 =?utf-8?B?bXdtMFZtUHdYTGtJVlRtN0JhZ1BqYkJ1VWNXck9xN212TnIxYStsZ3RMdHpo?=
 =?utf-8?B?SGdGQ1N2TndId0ZvSktscHFmUXVKMDJDRUxSVXZIQmI3MEhaSTdPMUNNZ2tW?=
 =?utf-8?B?dWg2cEpLelFwZUhZc202WCs0ZzlobStmdExPT1E5TGFUb2dnNnRGU2hkWXk2?=
 =?utf-8?B?L0lkSmNCb1lPZUdydzdHV3FWY3FYOTd2c282bUI4aVIwamJiUm1wdjlZaXpn?=
 =?utf-8?B?N3RVNU1nYzVoQUVyRVlyWDk3d0w0ZU1hTHdtc3EzTmY4eUtiMittT3d3VWlY?=
 =?utf-8?B?enhhQklNWDIrMCtWR3FnK3k5UEhQbGV3c1EvQm05blRwMjNMUGZHL0FqV1hO?=
 =?utf-8?B?d2VoMWtmOUNqYSs0NlF6elE1R1BpZFBrZHQ5RGhwMTNGeG1wbDNXdGloU2c0?=
 =?utf-8?B?L0s3b2RaYmVvR1B3UUp6TjJYU3N0RE1GQVRic09RWEV0ZUJyS041a3ROU1dL?=
 =?utf-8?B?RWFjUEtqY1A2aDNCV0lLc2VNTDZSNFRVZjVRbno5MWtTSnVHclhtWDFEWFpH?=
 =?utf-8?B?ZGRhaHkvUmVmMUI5MzFpbHUzdFpLaXNvVzNDemIyYXlsV0xqeCtSUjkrbzB2?=
 =?utf-8?B?RkpQcDYxR2o1SDNZajNKNzdmcVBDL2FCOUl3THN2TENMRHJDWEJOd2Y4MXZD?=
 =?utf-8?B?dURvSXhKSDJRd1VqaTVJeU16UnVidUN1bSt6WU5aM2MxTXFHU2FuYjFabHhN?=
 =?utf-8?B?NTlFMGJVNjZuRDh6WHZrTGVaeFQxZ3FDcjFoMmxUbkFZUkVZTmdrNC8ybURC?=
 =?utf-8?B?OU1BcXlEbG5TRTlTYmhRZXJKTXJUSFIrY2RFZGkyRnFnWkNSN1ZJUnJwMW93?=
 =?utf-8?B?aGtITXZmRDU1NHdMUkFzb2YvN2NPNGtITTZkMndpRzhpeXBiMjMzdEN4ek1F?=
 =?utf-8?B?bDFCQllBUW1LZGloRVFIUEhiV3RveG5DWmd6TnlpV1NRc3VSV3RoajVGQlhv?=
 =?utf-8?B?alI2ZWtlRXNnU2Nmc1o2TFp1bTFpMUs5bE5KR2tjV1lUbHB0VnFvNXE3dnZG?=
 =?utf-8?B?M3g1L2Qxd0xwdHh3U05CMzkyekxLTFB1ZmN4YXEzY2hRTHdGWXluc1hnVndN?=
 =?utf-8?B?YnQvcGJZTXRhK1BHdytkNktpRCtNSTVCODJBZ285TkVlSDV4N0w1cDVab1FY?=
 =?utf-8?B?U004Rm0yN1BYdUhySER4NHhVendNZVBBZkdDZ0U1U2JXK3pNMEJyTGJlRm04?=
 =?utf-8?Q?/NXGCWBipl23/WJu3IBauJ7RM?=
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6a6a5022-da70-41a3-09db-08dc5fc14184
X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB9642.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2024 16:04:41.4810
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UOXTMB5Ude+rIJgCTYjXCJr86/n/PiT1gn0HtiCQ2tXzWAArlvQT1Sx/Fe8cd9GtKWPU7C0CrHNOjwyKTbsuNg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR04MB8057

Involve an new and common mathod to send pme_turn_off() message. Previously
pme_turn_off() implement by platform related special register to trigge    
it.                                                                        
                                                                           
But Yoshihiro give good idea by using iATU to send out message. Previously 
Yoshihiro provide patches to raise INTx message by dummy write to outbound 
iATU.                                                                      
                                                                           
Use similar mathod to send out pme_turn_off message.                       
                                                                           
Previous two patches is picked from Yoshihiro' big patch serialise.        
 PCI: dwc: Change arguments of dw_pcie_prog_outbound_atu()                 
 PCI: Add INTx Mechanism Messages macros                                   
                                                                           
PCI: Add PME_TURN_OFF message macro                                        
dt-bindings: PCI: dwc: Add 'msg" register region, Add "msg" region to use  
to map PCI msg.                                                            
                                                                           
PCI: dwc: Add common pme_turn_off message method                           
Using common pme_turn_off() message if platform have not define their.

Signed-off-by: Frank Li <Frank.Li@nxp.com>
---
Changes in v8:
- Total use devm_ manage msg_res. after read devres.c code to better
understand release sequence. It use reverse order to release resource. So
it is safe for msg_res.
- Add Reviewed tag of Manivannan Sadhasivam in patch 4/5
- Link to v7: https://lore.kernel.org/r/20240417-pme_msg-v7-0-9c6208cda90f@nxp.com

Changes in v7:
  patch 4/5 add
  /* Power Management Messages; PCIe r6.0, sec 2.2.8.2 */
  patch 1/5 change comment to
  /* Message Routing (r[2:0]); PCIe r6.0, sec 2.2.8 */
  /* INTx Mechanism Messages; PCIe r6.0, sec 2.2.8.1 */

  order according to value.
  Add Acked-by: Bjorn Helgaas <bhelgaas@google.com> in patch 4/5

  patch 5/5
  Apply all mani's comments.

- Link to v6: https://lore.kernel.org/r/20240415-pme_msg-v6-0-56dad968ad3a@nxp.com

Changes in v6:
- Move PCIE_MSG_CODE_PME_TURN_OFF ahead before INTx to keep the codes
sorted.
- Added reason why not alloc_resesource in commit messag.
- Change title PCI: dwc: Add generic MSG TLP support for sending
PME_Turn_Off when system suspend.
- Rework commit message
  "Instead of relying on the vendor specific implementations to send the
PME_Turn_Off message, let's introduce a generic way of sending the message using
the MSG TLP.

This is achieved by reserving a region for MSG TLP of size 'pci->region_align',
at the end of the first IORESOURCE_MEM window of the host bridge. And then
sending the PME_Turn_Off message during system suspend with the help of iATU.

It should be noted that this generic implementation is optional for the glue
drivers and can be overridden by a custom 'pme_turn_off' callback.

- using devm_request_resource()
- using mani's suggest comments

	/*
	 * Allocate the resource for MSG TLP before programming the iATU
	 * outbound window in dw_pcie_setup_rc(). Since the allocation depends
	 * on the value of 'region_align', this has to be done after
	 * dw_pcie_iatu_detect().
	 */

- Add comments, about use_atu_msg, need glue driver set it.

- m -> mem in dw_pcie_pme_turn_off()

- remove redundant check() if (!atu.size) in dw_pcie_pme_turn_off().

- The reason can't adjust windows->end
This windows already reported to pci system before this function. It is
not good to modify window-res-end. It just add child resource like below.

windows is root resource, which will create may child when call
request_resource.
          bridge -> windows
		child1 -> msg
		child2 -> pci ep1
		child3 -> pci_ep2.
		...

Although you see whole bridge window, 'msg' already used and put under root
resource,  new pci devices will never use 'msg' resource. 

If change windows->res->end here, I worry about it may broken resource
tree.

Detail below thread:

- Link to v5: https://lore.kernel.org/r/20240319-pme_msg-v5-0-af9ffe57f432@nxp.com

Changes in v5:
- Default disable allocate TLP message memory windows. If driver need use
this feature, need set use_atu_msg = true before call dw_host_init().

- Link to v4: https://lore.kernel.org/r/20240213-pme_msg-v4-0-e2acd4d7a292@nxp.com

Changes in v4:
- Remove dt-binding patch. Needn't change any dts file and binding doc.
  Reserve a region at end of first IORESOURCE_MEM window by call
  request_resource(). So PCIe stack will not use this reserve region to any
PCIe devices.
  I tested it by reserve at begin of IORESOURCE_MEM window. PCIe stack
will skip it as expection.

  Fixed a issue, forget set iATU index when sent PME_turn_off.

- Link to v3: https://lore.kernel.org/r/20240202-pme_msg-v3-0-ff2af57a02ad@nxp.com

Changes in v3:
- fix 'MSG"
- Add pcie spec ref in head file
- using function name dw_pci_pme_turn_off()
- Using PCIE_ prefix macro
- Link to v2: https://lore.kernel.org/r/20240201-pme_msg-v2-0-6767052fe6a4@nxp.com

Changes in v2:
  - Add my sign off at PCI: dwc: Add outbound MSG TLPs support
  - Add Bjorn review tag at  Add INTx Mechanism Messages macros
  - using PME_Turn_Off match PCIe spec
  - ref to pcie spec v6.1
  - using section number.

- Link to v1: https://lore.kernel.org/r/20240130-pme_msg-v1-0-d52b0add5c7c@nxp.com

---
Frank Li (2):
      PCI: Add PCIE_MSG_CODE_PME_TURN_OFF message macro
      PCI: dwc: Add generic MSG TLP support for sending PME_Turn_Off when system suspend

Yoshihiro Shimoda (3):
      PCI: Add INTx Mechanism Messages macros
      PCI: dwc: Consolidate args of dw_pcie_prog_outbound_atu() into a structure
      PCI: dwc: Add outbound MSG TLPs support

 drivers/pci/controller/dwc/pcie-designware-ep.c   |  21 ++--
 drivers/pci/controller/dwc/pcie-designware-host.c | 145 +++++++++++++++++++---
 drivers/pci/controller/dwc/pcie-designware.c      |  54 ++++----
 drivers/pci/controller/dwc/pcie-designware.h      |  22 +++-
 drivers/pci/pci.h                                 |  21 ++++
 5 files changed, 199 insertions(+), 64 deletions(-)
---
base-commit: ef6a734484e939cf67bc41c45bc936a78ea210f0
change-id: 20240130-pme_msg-dd2d81ee9886

Best regards,
---
Frank Li <Frank.Li@nxp.com>