Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Sender: vjitta=codeaurora.org@mg.codeaurora.org
DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 76B32C433F1
Subject: Re: [PATCH v2 2/2] iommu/iova: Free global iova rcache on iova alloc
 failure
From:   Vijayanand Jitta <vjitta@codeaurora.org>
To:     Robin Murphy <robin.murphy@arm.com>, joro@8bytes.org,
        iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Cc:     vinmenon@codeaurora.org, kernel-team@android.com
References: <1597927761-24441-1-git-send-email-vjitta@codeaurora.org>
 <1597927761-24441-2-git-send-email-vjitta@codeaurora.org>
 <2f20160a-b9da-4fa3-3796-ed90c6175ebe@arm.com>
 <9dac89a4-553a-efe2-08a1-6a3a5fbc97a8@codeaurora.org>
Message-ID: <bb4c4fe4-50b6-5218-0cb5-43bcba9e9bff@codeaurora.org>
Date:   Wed, 30 Sep 2020 11:29:12 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.0
MIME-Version: 1.0
In-Reply-To: <9dac89a4-553a-efe2-08a1-6a3a5fbc97a8@codeaurora.org>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 9/28/2020 6:11 PM, Vijayanand Jitta wrote:
> 
> 
> On 9/18/2020 8:11 PM, Robin Murphy wrote:
>> On 2020-08-20 13:49, vjitta@codeaurora.org wrote:
>>> From: Vijayanand Jitta <vjitta@codeaurora.org>
>>>
>>> When ever an iova alloc request fails we free the iova
>>> ranges present in the percpu iova rcaches and then retry
>>> but the global iova rcache is not freed as a result we could
>>> still see iova alloc failure even after retry as global
>>> rcache is holding the iova's which can cause fragmentation.
>>> So, free the global iova rcache as well and then go for the
>>> retry.
>>>
>>> Signed-off-by: Vijayanand Jitta <vjitta@codeaurora.org>
>>> ---
>>>   drivers/iommu/iova.c | 23 +++++++++++++++++++++++
>>>   include/linux/iova.h |  6 ++++++
>>>   2 files changed, 29 insertions(+)
>>>
>>> diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c
>>> index 4e77116..5836c87 100644
>>> --- a/drivers/iommu/iova.c
>>> +++ b/drivers/iommu/iova.c
>>> @@ -442,6 +442,7 @@ struct iova *find_iova(struct iova_domain *iovad,
>>> unsigned long pfn)
>>>           flush_rcache = false;
>>>           for_each_online_cpu(cpu)
>>>               free_cpu_cached_iovas(cpu, iovad);
>>> +        free_global_cached_iovas(iovad);
>>>           goto retry;
>>>       }
>>>   @@ -1055,5 +1056,27 @@ void free_cpu_cached_iovas(unsigned int cpu,
>>> struct iova_domain *iovad)
>>>       }
>>>   }
>>>   +/*
>>> + * free all the IOVA ranges of global cache
>>> + */
>>> +void free_global_cached_iovas(struct iova_domain *iovad)
>>
>> As John pointed out last time, this should be static and the header
>> changes dropped.
>>
>> (TBH we should probably register our own hotplug notifier instance for a
>> flush queue, so that external code has no need to poke at the per-CPU
>> caches either)
>>
>> Robin.
>>
> 
> Right, I have made it static and dropped header changes in v3.
> can you please review that.
> 
> Thanks,
> Vijay

Please review v4 instead of v3, I have updated other patch as well in v4.

Thanks,
Vijay
>>> +{
>>> +    struct iova_rcache *rcache;
>>> +    unsigned long flags;
>>> +    int i, j;
>>> +
>>> +    for (i = 0; i < IOVA_RANGE_CACHE_MAX_SIZE; ++i) {
>>> +        rcache = &iovad->rcaches[i];
>>> +        spin_lock_irqsave(&rcache->lock, flags);
>>> +        for (j = 0; j < rcache->depot_size; ++j) {
>>> +            iova_magazine_free_pfns(rcache->depot[j], iovad);
>>> +            iova_magazine_free(rcache->depot[j]);
>>> +            rcache->depot[j] = NULL;
>>> +        }
>>> +        rcache->depot_size = 0;
>>> +        spin_unlock_irqrestore(&rcache->lock, flags);
>>> +    }
>>> +}
>>> +
>>>   MODULE_AUTHOR("Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>");
>>>   MODULE_LICENSE("GPL");
>>> diff --git a/include/linux/iova.h b/include/linux/iova.h
>>> index a0637ab..a905726 100644
>>> --- a/include/linux/iova.h
>>> +++ b/include/linux/iova.h
>>> @@ -163,6 +163,7 @@ int init_iova_flush_queue(struct iova_domain *iovad,
>>>   struct iova *split_and_remove_iova(struct iova_domain *iovad,
>>>       struct iova *iova, unsigned long pfn_lo, unsigned long pfn_hi);
>>>   void free_cpu_cached_iovas(unsigned int cpu, struct iova_domain
>>> *iovad);
>>> +void free_global_cached_iovas(struct iova_domain *iovad);
>>>   #else
>>>   static inline int iova_cache_get(void)
>>>   {
>>> @@ -270,6 +271,11 @@ static inline void free_cpu_cached_iovas(unsigned
>>> int cpu,
>>>                        struct iova_domain *iovad)
>>>   {
>>>   }
>>> +
>>> +static inline void free_global_cached_iovas(struct iova_domain *iovad)
>>> +{
>>> +}
>>> +
>>>   #endif
>>>     #endif
>>>
> 

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a
member of Code Aurora Forum, hosted by The Linux Foundation