Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 28 Dec 2018 20:29:23 -0600
From:   Bjorn Helgaas <helgaas@kernel.org>
To:     yu <kkndyu@gmail.com>
Cc:     linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
        Logan Gunthorpe <logang@deltatee.com>,
        Stephen Bates <sbates@raithlin.com>,
        Jonathan Cameron <jonathan.cameron@huawei.com>,
        Eric Wehage <Eric.Wehage@huawei.com>,
        Alexander Duyck <alexander.h.duyck@linux.intel.com>
Subject: Re: How to force RC to forward p2p TLPs
Message-ID: <20181229022923.GA159477@google.com>
References: <CADayL-pxr_dpJykyiTeQDO13XmKPoKUs544zVrB2taHTVEx9Pw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CADayL-pxr_dpJykyiTeQDO13XmKPoKUs544zVrB2taHTVEx9Pw@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

[+cc Logan, Stephen, Jonathan, Eric, Alex]

On Sat, Dec 22, 2018 at 12:50:19PM +0800, yu wrote:
> Hi all,
> 
> We have a PCIE card which has a PEX8732 switch on-board,  and there
> are two endpoint SOCs like graphic decoder behind the switch, and by
> default the ACS is enabled in 8732.
> 
> We use the p2p DMA to transfer data between these two endpoint SOCs,
> and if the host server is not enable ACS in BIOS, the p2p works well,
> but when ACS is enabled in BIOS, the p2p is always failed. With the
> help of a protocol analyzer, we can see that the TLP is redirected to
> RC, and RC just discard it.
> 
> I tried to find how to make RC forward redirected TLP to its original
> target, but nothing found, it seems this is highly related to the RC
> vendors.
> 
> In the PCIE 4.0 spec, the section of the RC behavior of the p2p
> request redirect  said that ''implementation-specific logic within the
> RC that determines whether the request is directed towards its
> original target, or blocked as an ACS Violation error. the algorithms
> and specific controls for making this determination are not
> architected by this spec''.
> 
> 
> So is there some spec or document to describe how to set the RC? Any
> suggestion is appreciated.

Not that I'm aware of, but the folks I cc'd would know a lot more
about this area.

Bjorn