Received: by 2002:a25:e74b:0:0:0:0:0 with SMTP id e72csp1943378ybh; Fri, 17 Jul 2020 05:38:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxZXzVRmAgyONnt+CpkHBN9SNyxaarawE+OfbYcwLwQyOxM7+h5IO6D3gHeg/Z6PILXPAJ/ X-Received: by 2002:a17:906:b0d:: with SMTP id u13mr8157003ejg.342.1594989483020; Fri, 17 Jul 2020 05:38:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594989483; cv=none; d=google.com; s=arc-20160816; b=gijw6x9iRO1jMQ7jpVYV24v/90YBQCgaJ+iCZ6SfrxMX5HvC/g4Z91tLEBAW+vwAsa WneodmSgidBmaM5Wp5ib5YpXIWAO5FVOrwTExpKguOB8qwAMhM9afasfJ9Av9UX3LS7n +QrB4xLpack6rQVLqt5j2dI3q6+XD1OgJMXx5Fob0q/MY2q4m/qcn13vz+AMKnpz1lk4 gR/RCsITtaHUmvdTtJaNwNiH/LbzbbhbzQSlQzC/6kOk2Qcgj7LGtzIwu/43vd2Je0f/ frw4Mx/rhmtJWqgi+nvlTuDmXU99XpWI1pcjtwfVNEBwrmGUMiXpBMkRnQa+AdOs2TdF 0DSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:user-agent:references :in-reply-to:subject:cc:to:from:message-id:date:dkim-signature; bh=bCLYwsQVfP2+Fu/lT1X0R3b3gPFkuCjpbX9FCuUneoE=; b=TAWNSEylteMZvobX/UPwbEvBLkj+QbB4ek7nZxA1510/EKawIEa8UcxLsLBlv7T90p xj+6cGnt3ICqxcWEAl+o3u8uOCWMvgF4y7ZDNTDYSwtNqmCKm7g+5t3P/JoOaxefD0Pk Gdi7I46MLR/WnCMSoUburRfwBuRMAHYkma11LUGEhzbr09cU5FD+FB/SeC0W34+dMo+U REKQ+8fEwZy2qmQNgU3LZEbzHyqwJIC5zNy8baR5iTtMeEu/INdiAAhzZWc+hDPqdN2N ZpJR7RaZWrp41VOYBE4POmQOO/vcpNiUB/GBSpWzdphNRHwFXuNJBLp0IMbn4ASTRJBW 2ukw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=0e9um7Sq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id 25si4851528ejy.507.2020.07.17.05.37.40; Fri, 17 Jul 2020 05:38:03 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=0e9um7Sq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726830AbgGQMgz (ORCPT + 99 others); Fri, 17 Jul 2020 08:36:55 -0400 Received: from mail.kernel.org ([198.145.29.99]:40866 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726201AbgGQMgz (ORCPT ); Fri, 17 Jul 2020 08:36:55 -0400 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 7DB3220717; Fri, 17 Jul 2020 12:36:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1594989414; bh=CzYltP1Sifi3ml98UjZKVgBinI/OHmbuTS3rMgZxE8E=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=0e9um7SqZnczNDpi+ds0gGfq8RWvy6Nor1Gu2YtLcIi3u0ie9uYYaJUivARtAe/Bt 3hCEvBtkue8BvP8bF9UUjwDkWg9On3V7xFm6pzyB7DwcI14v625OofcGxwfTKuvOQw 8UyIiD+y0AcEgAUr27p1fjuq7N3Z4X0z0YleXNtE= Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jwPbg-00Ccl9-JA; Fri, 17 Jul 2020 13:36:52 +0100 Date: Fri, 17 Jul 2020 13:36:51 +0100 Message-ID: <87ft9qxqqk.wl-maz@kernel.org> From: Marc Zyngier To: Grzegorz Jaszczyk Cc: tglx@linutronix.de, jason@lakedaemon.net, robh+dt@kernel.org, Lee Jones , devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-omap@vger.kernel.org, linux-arm-kernel@lists.infradead.org, david@lechnology.com, "Mills, William" , "Andrew F . Davis" , Roger Quadros , Suman Anna Subject: Re: [PATCHv3 2/6] irqchip/irq-pruss-intc: Add a PRUSS irqchip driver for PRUSS interrupts In-Reply-To: References: <1593699479-1445-1-git-send-email-grzegorz.jaszczyk@linaro.org> <1593699479-1445-3-git-send-email-grzegorz.jaszczyk@linaro.org> <12db6d22c12369b6d64f410aa2434b03@kernel.org> <53d39d8fbd63c6638dbf0584c7016ee0@kernel.org> <3501f3a6-0613-df1c-2c6d-5ac4610a226d@ti.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/26.3 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: grzegorz.jaszczyk@linaro.org, tglx@linutronix.de, jason@lakedaemon.net, robh+dt@kernel.org, lee.jones@linaro.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-omap@vger.kernel.org, linux-arm-kernel@lists.infradead.org, david@lechnology.com, wmills@ti.com, afd@ti.com, rogerq@ti.com, s-anna@ti.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Suman, Grzegorz, On Wed, 15 Jul 2020 14:38:05 +0100, Grzegorz Jaszczyk wrote: > > Hi Marc, > > > On 7/8/20 5:47 AM, Marc Zyngier wrote: > > > On 2020-07-08 08:04, Grzegorz Jaszczyk wrote: > > >> On Sun, 5 Jul 2020 at 22:45, Marc Zyngier wrote: > > >>> > > >>> On 2020-07-05 14:26, Grzegorz Jaszczyk wrote: > > >>> > On Sat, 4 Jul 2020 at 11:39, Marc Zyngier wrote: > > >>> >> > > >>> >> On 2020-07-03 15:28, Grzegorz Jaszczyk wrote: > > >>> > > >>> [...] > > >>> > > >>> >> It still begs the question: if the HW can support both edge and level > > >>> >> triggered interrupts, why isn't the driver supporting this diversity? > > >>> >> I appreciate that your HW may only have level interrupts so far, but > > >>> >> what guarantees that this will forever be true? It would imply a > > >>> >> change > > >>> >> in the DT binding, which isn't desirable. > > >>> > > > >>> > Ok, I've got your point. I will try to come up with something later > > >>> > on. Probably extending interrupt-cells by one and passing interrupt > > >>> > type will be enough for now. Extending this driver to actually support > > >>> > it can be handled later if needed. Hope it works for you. > > >>> > > >>> Writing a set_type callback to deal with this should be pretty easy. > > >>> Don't delay doing the right thing. > > >> > > >> Ok. > > > > Sorry for the typo in my comment causing this confusion. > > > > The h/w actually doesn't support the edge-interrupts. Likewise, the > > polarity is always high. The individual register bit descriptions > > mention what the bit values 0 and 1 mean, but there is additional > > description in the TRMs on all the SoCs that says > > "always write 1 to the bits of this register" for PRUSS_INTC_SIPR(x) and > > "always write 0 to the bits of this register" for PRUSS_INTC_SITR(x). > > FWIW, these are also the reset values. > > > > Eg: AM335x TRM - https://www.ti.com/lit/pdf/spruh73 > > Please see Section 4.4.2.5 and the register descriptions in 4.5.3.49, > > 4.5.3.51. Please also see Section 4.4.2.3 that explains the PRUSS INTC > > methodology. > > > > >> > > >>> > > >>> [...] > > >>> > > >>> >> >> > + hwirq = hipir & GENMASK(9, 0); > > >>> >> >> > + virq = irq_linear_revmap(intc->domain, hwirq); > > >>> >> >> > > >>> >> >> And this is where I worry. You seems to have a single irqdomain > > >>> >> >> for all the muxes. Are you guaranteed that you will have no > > >>> >> >> overlap between muxes? And please use irq_find_mapping(), as > > >>> >> >> I have top-secret plans to kill irq_linear_revmap(). > > >>> >> > > > >>> >> > Regarding irq_find_mapping - sure. > > >>> >> > > > >>> >> > Regarding irqdomains: > > >>> >> > It is a single irqdomain since the hwirq (system event) can be > > >>> mapped > > >>> >> > to different irq_host (muxes). Patch #6 > > >>> >> > https://lkml.org/lkml/2020/7/2/616 implements and describes how > > >>> input > > >>> >> > events can be mapped to some output host interrupts through 2 > > >>> levels > > >>> >> > of many-to-one mapping i.e. events to channel mapping and > > >>> channels to > > >>> >> > host interrupts. Mentioned implementation ensures that specific > > >>> system > > >>> >> > event (hwirq) can be mapped through PRUSS specific channel into a > > >>> >> > single host interrupt. > > >>> >> > > >>> >> Patch #6 is a nightmare of its own, and I haven't fully groked it > > >>> yet. > > >>> >> Also, this driver seems to totally ignore the 2-level routing. Where > > >>> >> is it set up? map/unmap in this driver do exactly *nothing*, so > > >>> >> something somewhere must set it up. > > >>> > > > >>> > The map/unmap is updated in patch #6 and it deals with those 2-level > > >>> > routing setup. Map is responsible for programming the Channel Map > > >>> > Registers (CMRx) and Host-Interrupt Map Registers (HMRx) basing on > > >>> > provided configuration from the one parsed in the xlate function. > > >>> > Unmap undo whatever was done on the map. More details can be found in > > >>> > patch #6. > > >>> > > > >>> > Maybe it would be better to squash patch #6 with this one so it would > > >>> > be less confusing. What is your advice? > > >>> > > >>> So am I right in understanding that without patch #6, this driver does > > >>> exactly nothing? If so, it has been a waste of review time. > > >>> > > >>> Please split patch #6 so that this driver does something useful > > >>> for Linux, without any of the PRU interrupt routing stuff. I want > > >>> to see a Linux-only driver that works and doesn't rely on any other > > >>> exotic feature. > > >>> > > >> > > >> Patch #6 provides PRU specific 2-level routing setup. This step is > > >> required and it is part of the entire patch-set. Theoretically routing > > >> setup could be done by other platform driver (not irq one) or e.g. by > > >> PRU firmware. In such case this driver would be functional without > > >> patch #6 but I do not think it would be proper. > > > > > > Then this whole driver is non-functional until the last patch that > > > comes with the PRU-specific "value-add". > > > > It is all moot actually and the interrupts work only when the PRU > > remoteproc/clients have invoked the irq_create_fwspec_mapping() > > for all of the desired system events. It does not make much difference > > if it was a separate patch or squashed in, patch #6 is a replacement for > > the previous logic, and since it was complex, it was done in a separate > > patch to better explain the usage (same reason on v1 and v2 as > > well). It may make no difference to you, but it does for me, as I'm the lucky idiot reviewing this code. So I am going to say it again: please keep anything that only exists for the PRU subsystem benefit out of the initial patches. I want to see something that works for Linux, and only for Linux. Once we have that working, we'll see to add more stuff. But stop throwing the PRU business into the early patches, as all you are achieving is to delay the whole thing. > > > > > > > > [...] > > > > > >> I am open to any suggestion if there is a better way of handling > > >> 2-level routing. I will also appreciate if you could elaborate about > > >> issues that you see with patch #6. > > > > > > The two level routing has to be part of this (or another) irqchip > > > driver (specially given that it appears to me like another set of > > > crossbar). There should only be a *single* binding for all interrupts, > > > including those targeting the PRU (you seem to have two). > > > > > > > Yeah, there hasn't been a clean way of doing this. Our previous attempt > > was to do this through custom exported functions so that the PRU > > remoteproc driver can set these up correctly, but that was shot down and > > this is the direction we are pointed to. > > > > We do want to leverage the "interrupts" property in the PRU user nodes > > instead of inventing our own paradigm through a non-irqchip driver, and > > at the same time, be able to configure this at the run time only when > > that PRU driver is running, and remove the mappings once that driver is > > removed allowing another PRU application/driver. We treat PRUs as an > > exclusive resource, so everything needs to go along with an appropriate > > client user. > > I will just add an explanation about interrupt binding. So actually > there is one dt-binding defined in yaml (interrupt-cells = 1). The > reason why you see xlate allowing to proceed with 1 or 3 parameters is > because linux can change the PRU firmware at run-time (thorough linux > remoteproc framework) and different firmware may require different > kinds of interrupt mapping. Therefore during firmware load, the new > mapping is created through irq_create_fwspec_mapping() and in this > case 3 parameters are passed: system event, channel and host irq. > Similarly the mapping is disposed during remoteproc stop by invoking > irq_dispose_mapping. This allows to create new mapping, in the same > way, for next firmware loaded through Linux remote-proc at runtime > (depending on the needs of new remoteproc firmware). > > On the other hand dt-bindings defines interrupt-cells = 1, so when the > interrupt is registered the xlate function (proceed with 1 parameter) > checks if this event already has valid mapping - if yes we are fine, > if not we return -EINVAL. It means that interrupts declared in DT get their two-level routing via the kernel driver, while PRU interrupts get their routing via some external blob that Linux is not in control of? If so, this looks broken. What if you get a resource allocation conflict because the kernel and the blob are stepping into each other's toes? Why should an end-point client decide on the routing of the interrupt? All the end-point should provide is the ID of the input signal, and to which PRU this is routed. Interrupts described in DT should have the exact same model (input signal, target). All the intermediate routing logic should be handled by the Linux driver for *all* interrupts in the system. > > > > > > And the non-CPU interrupt code has to be in its own patch, because > > > it is pretty borderline anyway (I'm still not completely convinced > > > this is Linux's job). > > > > The logic for non-CPU interrupt code is exactly the same as the CPU > > interrupt code, as they are all setup through the > > irq_create_fwspec_mapping(). The CPU-specific pieces are primarily the > > chained interrupt handling. > > > > We have already argued internally about the last part, but our firmware > > developers literally don't have any IRAM space (we have a lot of > > Industrial protocols working out of 4K/8K memory), and have pushed all > > one-time setup to the OS running (Linux or otherwise) on the main ARM > > core, and INTC is one among the other many such settings. Every word in > > Instruction RAM was crucial for them. And that's fine. Just push *all* of it into Linux, and not just the programming of the registers. > > > > So, we are all ears if there is still an elegant way of doing this. Look > > forward to any suggestions you may have. > > Yes, the non-CPU logic is exactly the same as the CPU interrupt code > as Suman described. There is no distinction between routing setup for > main CPU and PRU core, both use exactly the same logic, just different > numbers are passed through irq_create_fwspec_mapping. It obviously isn't the same at the moment. You have two distinct code paths, two ways to describe a mapping, and a potential resource allocation issue. M. -- Without deviation from the norm, progress is not possible.