Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.112.32 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.112.32; helo=mail.nvidia.com;
Subject: Re: GTE - The hardware timestamping engine
To:     Kent Gibson <warthog618@gmail.com>
CC:     Linus Walleij <linus.walleij@linaro.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "thierry.reding@gmail.com" <thierry.reding@gmail.com>,
        Jon Hunter <jonathanh@nvidia.com>,
        "Bartosz Golaszewski" <bgolaszewski@baylibre.com>,
        "open list:GPIO SUBSYSTEM" <linux-gpio@vger.kernel.org>,
        linux-tegra <linux-tegra@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Arnd Bergmann <arnd@arndb.de>,
        "Richard Cochran" <richardcochran@gmail.com>
References: <4c46726d-fa35-1a95-4295-bca37c8b6fe3@nvidia.com>
 <CACRpkdbmqww6UQ8CFYo=+bCtVYBJwjMxVixc4vS6D3B+dUHScw@mail.gmail.com>
 <20210322060047.GA226745@sol>
 <d48fa7b5-8c17-c3d7-10a9-a9811c410a39@nvidia.com> <20210323003208.GA6105@sol>
 <7961d9df-4120-e37c-d042-528655bd0270@nvidia.com>
 <20210323025933.GA10669@sol>
From:   Dipen Patel <dipenp@nvidia.com>
Message-ID: <16ff9b8a-15d9-f2fd-24f4-817a7078c40e@nvidia.com>
Date:   Mon, 22 Mar 2021 21:09:50 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210323025933.GA10669@sol>
Content-Type: text/plain; charset="utf-8"
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2021 04:05:23.5220
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: ad624d5f-f25d-441a-abce-08d8edb0e218
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.32];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT042.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR12MB2723
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 3/22/21 7:59 PM, Kent Gibson wrote:
> On Mon, Mar 22, 2021 at 06:53:10PM -0700, Dipen Patel wrote:
>>
>>
>> On 3/22/21 5:32 PM, Kent Gibson wrote:
>>> On Mon, Mar 22, 2021 at 01:21:46PM -0700, Dipen Patel wrote:
>>>> Hi Linus and Kent,
>>>>
> 
> [snip]
> 
>>> In response to all your comments above...
>>>
>>> Firstly, I'm not suggesting that other kernel modules would use the
>>> cdev lineevents, only that they would use the same mechanism that
>>> gpiolib-cdev would use to timestamp the lineevents for userspace.
>>>
>> Sure, I just wanted to mention the different scenarios and wanted to know
>> how can we fit all those together. Having said that, shouldn't this serve
>> an opportunity to extend the linevent framework to accommodate kernel
>> drivers as a clients?
>>
>> If we can't, then there is a risk of duplicating lineevent mechanism in all
>> of those kernel drivers or at least in GTE framework/infrastructure as far
>> as GPIO related GTE part is concerned.
>>  
> 
> In-kernel the lineevents are just IRQs so anything needing a "lineevent"
> can request the IRQ directly.  Or am I missing something?
> 

In the GPIO context, I meant we can extend line_event_timestamp to kernel
drivers as well in that way, both userspace and kernel drivers requesting
particular GPIO for the hardware timestamp would be managed by same
lineevent_* infrastructure from the gpiolib. Something like lineevent_create
version of the kernel drivers, so if they need hardware timestamp on the
GPIO line, they can request with some flags. In that way, GTE can leverage
linevent* codes from gpiolib to cover its both the GPIO related use cases i.e.
userspace app and kernel drivers.

>>> As to that mechanism, my current thinking is that the approach of
>>> associating GTE event FIFO entries with particular physical IRQ events is
>>> problematic, as keeping the two in sync would be difficult, if not
>>> impossible.
>>>
>>> A more robust approach is to ignore the physical IRQs and instead service
>>> the GTE event FIFO, generating IRQs from those events in software -
>>> essentially a pre-timestamped IRQ.  The IRQ framework could provide the
>>> timestamping functionality, equivalent to line_event_timestamp(), for
>>> the IRQ handler/thread and in this case provide the timestamp from the GTE
>>> event.
>>>
>>
>> I have not fully understood above two paragraphs (except about
>> lineevent_event_timestamp related part).
>>
>> I have no idea what it means to "ignore the physical IRQs and service the
>> GTE event FIFO". Just like GPIO clients, there could be IRQ clients which
>> want to monitor certain IRQ line, like ethernet driver wanted to retrieve
>> timestamp for its IRQ line and so on.
>>
> 
> I mean that in the IRQ framework, rather than enabling the physical IRQ
> line it would leave that masked and would instead enable the FIFO line to
> service the FIFO, configure the GTE to generate the events for that
> line, and then generate IRQs in response to the FIFO events.
> That way the client requesting the IRQ is guaranteed to only receive an
> IRQ that corresponds to a GTE FIFO event and the timestamp stored in the
> IRQ framework would match.
> 

I do not think we need to do such things, for example, below is
the rough sequence how GTE can notify its clients be it GPIO or IRQ
lines. I believe this will also help understand better ongoing GPIO
discussions.

1. Configure GTE FIFO watermark or threshold, lets assume 1, i.e
   generate GTE interrupt when FIFO depth is 1.
2. In the GTE ISR or ISR thread, drain internal FIFO entries
3. Through GTE driver's internal mapping, identify which IRQ or
   GPIO number this entry belongs to. (This is possible as GTE
   has predefined bits for each supported signals, for example GTE
   supports 40 GPIOs and 352 IRQ lines, and it has multliple GTE instances
   which can take care all of them)
4. GTE driver pushes the event data (in this case it will be timestamp and
   direction of the event ie.rising or falling) to the GTE generic framework
5. GTE framework will store per event data to its per client/event sw FIFO
6. wake up any sleeping client thread
7. Points 3 to 6 are happening in GTE ISR context. 
8. gte_retrieve_event (which can block if no event) at later convenient
   time do whatever it wants with it. We can extend it to non blocking
   version where some sort of client callbacks can be implemented.

> And that is what I mean by this being an IRQ feature.
> We need feedback from the IRQ guys as to whether that makes sense to
> them.
> 
> Cheers,
> Kent.
> 
Best Regards,
Dipen Patel