Received-SPF: pass (google.com: domain of linux-kernel+bounces-18524-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <23cc4bc9-9519-40e9-bd50-f89b0ebc6c5c@linux.intel.com>
Date: Sat, 6 Jan 2024 15:33:19 +0800
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 0/2] iommu/iova: Make the rcache depot properly
 flexible
To: "zhangzekun (A)" <zhangzekun11@huawei.com>
Cc: joro@8bytes.org, will@kernel.org, iommu@lists.linux.dev,
 linux-kernel@vger.kernel.org, john.g.garry@oracle.com,
 dheerajkumar.srivastava@amd.com, jsnitsel@redhat.com,
 Ido Schimmel <idosch@idosch.org>, Robin Murphy <robin.murphy@arm.com>
References: <cover.1694535580.git.robin.murphy@arm.com>
 <ZY1osaGLyT-sdKE8@shredder> <ZZO6NV-Kq-ktnyo5@shredder>
 <f8ae34f0-dab2-48fa-b3f8-5e969ed028b7@linux.intel.com>
 <20924c0f-b752-4f58-91ce-21ed3b305481@huawei.com>
From: Ethan Zhao <haifeng.zhao@linux.intel.com>
In-Reply-To: <20924c0f-b752-4f58-91ce-21ed3b305481@huawei.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 1/6/2024 3:07 PM, zhangzekun (A) wrote:
>
>
> 在 2024/1/6 12:21, Ethan Zhao 写道:
>>
>> On 1/2/2024 3:24 PM, Ido Schimmel wrote:
>>> On Thu, Dec 28, 2023 at 02:23:20PM +0200, Ido Schimmel wrote:
>>>> On Tue, Sep 12, 2023 at 05:28:04PM +0100, Robin Murphy wrote:
>>>>> v2: 
>>>>> https://lore.kernel.org/linux-iommu/cover.1692641204.git.robin.murphy@arm.com/
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I hope this is good to go now, just fixed the locking (and threw
>>>>> lockdep at it to confirm, which of course I should have done to begin
>>>>> with...) and picked up tags.
>>>> Hi,
>>>>
>>>> After pulling the v6.7 changes we started seeing the following memory
>>>> leaks [1] of 'struct iova_magazine'. I'm not sure how to reproduce it,
>>>> which is why I didn't perform bisection. However, looking at the
>>>> mentioned code paths, they seem to have been changed in v6.7 as 
>>>> part of
>>>> this patchset. I reverted both patches and didn't see any memory leaks
>>>> when running a full regression (~10 hours), but I will repeat it to be
>>>> sure.
>>> FYI, we didn't see the leaks since reverting these two patches whereas
>>> before we saw them almost everyday, so I'm quite sure they introduced
>>> the leaks.
>>
>> Seems some magazines were not freed when one CPU is dead (hot 
>> unplugged) ?
>>
>> static void free_cpu_cached_iovas(unsigned int cpu, struct 
>> iova_domain *iovad)
>> {
>>     struct iova_cpu_rcache *cpu_rcache;
>>     struct iova_rcache *rcache;
>>     unsigned long flags;
>>     int i;
>>
>>     for (i = 0; i < IOVA_RANGE_CACHE_MAX_SIZE; ++i) {
>>         rcache = &iovad->rcaches[i];
>>         cpu_rcache = per_cpu_ptr(rcache->cpu_rcaches, cpu);
>>         spin_lock_irqsave(&cpu_rcache->lock, flags);
>>         iova_magazine_free_pfns(cpu_rcache->loaded, iovad);
>>
>> +     iova_magazine_free(cpu_rcache->loaded);
>>
>>         iova_magazine_free_pfns(cpu_rcache->prev, iovad);
>>
>> +     iova_magazine_free(cpu_rcache->prev);
>>
>>         spin_unlock_irqrestore(&cpu_rcache->lock, flags);
>>     }
>> }
> It seems cpu_rcache->loaded and cpu_rcache->prev will be freed in 
> free_iova_rcaches(), and it should not cause memory leak because 
> iova_magazine_free() will be called for each possible cpu.
> free_cpu_cached_iovas() is used to free cached iovas in magazines.

Yup, looked  closely.  possible cpu, not online cpu.

Thanks,

Ethan

>
> Thanks,
> Zekun
>
>
>