Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   "He, Roger" <Hongbo.He@amd.com>
To:     Michal Hocko <mhocko@kernel.org>,
        "Koenig, Christian" <Christian.Koenig@amd.com>
CC:     "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>
Subject: RE: [PATCH] mm/swap: add function get_total_swap_pages to expose
 total_swap_pages
Thread-Topic: [PATCH] mm/swap: add function get_total_swap_pages to expose
 total_swap_pages
Thread-Index: AQHTmNtYoBUw/9kRPkGbswRNbnupYaOLC3EAgAClcwCAAFzmgIAAEfKAgAAV3oCAAAQIgIAAIG6AgAEekqA=
Date:   Wed, 31 Jan 2018 05:52:26 +0000
Message-ID: <MWHPR1201MB0127ECADF14022B0A383081EFDFB0@MWHPR1201MB0127.namprd12.prod.outlook.com>
References: <1517214582-30880-1-git-send-email-Hongbo.He@amd.com>
 <20180129163114.GH21609@dhcp22.suse.cz>
 <MWHPR1201MB01278542F6EE848ABD187BDBFDE40@MWHPR1201MB0127.namprd12.prod.outlook.com>
 <20180130075553.GM21609@dhcp22.suse.cz>
 <9060281e-62dd-8775-2903-339ff836b436@amd.com>
 <20180130101823.GX21609@dhcp22.suse.cz>
 <7d5ce7ab-d16d-36bc-7953-e1da2db350bf@amd.com>
 <20180130122853.GC21609@dhcp22.suse.cz>
In-Reply-To: <20180130122853.GC21609@dhcp22.suse.cz>
Accept-Language: en-US
Content-Language: en-US
received-spf: None (protection.outlook.com: amd.com does not designate
 permitted sender hosts)
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 48bf06e1-62b5-49ab-6b8c-08d5686ecec6
X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Jan 2018 05:52:26.6657
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB2559
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

	I do think you should completely ignore the size of the swap space. IMHO y=
ou should forbid further allocations when your current 	buffer storage cann=
ot be reclaimed. So you need some form of feedback mechanism that would tel=
l you: "Your buffers have 	grown too much". If you cannot do that then simp=
ly assume that you cannot swap at all rather than rely on having some porti=
on 	of it for yourself.=20

If we assume the swap cache size is zero always, that is overkill for GTT s=
ize actually user can get. And not make sense as well I think.

	There are many other users of memory outside of your subsystem. Any scalin=
g based on the 50% of resource belonging to me is 	simply broken.

And that is only a threshold to avoid  overuse  rather than really reserved=
 to TTM at the start. In addition, for most cases TTM only uses a little or=
 not use swap disk at all. Only special test case use more or probably that=
 is intentional.


Thanks
Roger(Hongbo.He)

-----Original Message-----
From: Michal Hocko [mailto:mhocko@kernel.org]=20
Sent: Tuesday, January 30, 2018 8:29 PM
To: Koenig, Christian <Christian.Koenig@amd.com>
Cc: He, Roger <Hongbo.He@amd.com>; linux-mm@kvack.org; linux-kernel@vger.ke=
rnel.org; dri-devel@lists.freedesktop.org
Subject: Re: [PATCH] mm/swap: add function get_total_swap_pages to expose t=
otal_swap_pages

On Tue 30-01-18 11:32:49, Christian K=F6nig wrote:
> Am 30.01.2018 um 11:18 schrieb Michal Hocko:
> > On Tue 30-01-18 10:00:07, Christian K=F6nig wrote:
> > > Am 30.01.2018 um 08:55 schrieb Michal Hocko:
> > > > On Tue 30-01-18 02:56:51, He, Roger wrote:
> > > > > Hi Michal:
> > > > >=20
> > > > > We need a API to tell TTM module the system totally has how=20
> > > > > many swap cache.  Then TTM module can use it to restrict how=20
> > > > > many the swap cache it can use to prevent triggering OOM.  For=20
> > > > > Now we set the threshold of swap size TTM used as 1/2 * total=20
> > > > > size and leave the rest for others use.
> > > > Why do you so much memory? Are you going to use TB of memory on=20
> > > > large systems? What about memory hotplug when the memory is added/r=
eleased?
> > > For graphics and compute applications on GPUs it isn't unusual to=20
> > > use large amounts of system memory.
> > >=20
> > > Our standard policy in TTM is to allow 50% of system memory to be=20
> > > pinned for use with GPUs (the hardware can't do page faults).
> > >=20
> > > When that limit is exceeded (or the shrinker callbacks tell us to=20
> > > make room) we wait for any GPU work to finish and copy buffer=20
> > > content into a shmem file.
> > >=20
> > > This copy into a shmem file can easily trigger the OOM killer if=20
> > > there isn't any swap space left and that is something we want to avoi=
d.
> > >=20
> > > So what we want to do is to apply this 50% rule to swap space as=20
> > > well and deny allocation of buffer objects when it is exceeded.
> > How does that help when the rest of the system might eat swap?
>=20
> Well it doesn't, but that is not the problem here.
>=20
> When an application keeps calling malloc() it sooner or later is=20
> confronted with an OOM killer.
>=20
> But when it keeps for example allocating OpenGL textures the=20
> expectation is that this sooner or later starts to fail because we run=20
> out of memory and not trigger the OOM killer.

There is nothing like running out of memory and not triggering the OOM kill=
er. You can make a _particular_ allocation to bail out without the oom kill=
er. Just use __GFP_NORETRY. But that doesn't make much difference when you =
have already depleted your memory and live with the bare remainings. Any de=
sperate soul trying to get its memory will simply trigger the OOM.

> So what we do is to allow the application to use all of video memory +=20
> a certain amount of system memory + swap space as last resort fallback (e=
.g.
> when you Alt+Tab from your full screen game back to your browser).
>=20
> The problem we try to solve is that we haven't limited the use of swap=20
> space somehow.

I do think you should completely ignore the size of the swap space. IMHO yo=
u should forbid further allocations when your current buffer storage cannot=
 be reclaimed. So you need some form of feedback mechanism that would tell =
you: "Your buffers have grown too much". If you cannot do that then simply =
assume that you cannot swap at all rather than rely on having some portion =
of it for yourself. There are many other users of memory outside of your su=
bsystem. Any scaling based on the 50% of resource belonging to me is simply=
 broken.
--
Michal Hocko
SUSE Labs