Received-SPF: pass (google.com: domain of linux-kernel+bounces-170835-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <d41a1f6e2940e9747f9efcb3721646186940da03.camel@xry111.site>
Subject: Re: [PATCH] LoongArch: Update the flush cache policy
From: Xi Ruoyao <xry111@xry111.site>
To: lijun <lijun01@kylinos.cn>, chenhuacai@kernel.org, kernel@xen0n.name, 
	lvjianmin@loongson.cn, dongbiao@loongson.cn, zhangbaoqi@loongson.cn
Cc: loongarch@lists.linux.dev, linux-kernel@vger.kernel.org
Date: Tue, 07 May 2024 15:48:05 +0800
In-Reply-To: <18342368-ce79-9820-26be-01834876fc9e@kylinos.cn>
References: <20240506092419.4109941-1-lijun01@kylinos.cn>
	 <8a8135eb0f1dc724cfbb4402dc6daf08db5b0bc7.camel@xry111.site>
	 <cbb24599-8b40-cd27-6ce7-215476c0ddf4@kylinos.cn>
	 <cbd6ed9d5be1d7112d69117a72e0cb0081f9b64b.camel@xry111.site>
	 <8809f5a7-de6e-0794-feab-726c26f87344@kylinos.cn>
	 <cf5df629b88fe38ae04cfa5714b9de2a44792704.camel@xry111.site>
	 <18342368-ce79-9820-26be-01834876fc9e@kylinos.cn>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.1 
Precedence: bulk
MIME-Version: 1.0

On Tue, 2024-05-07 at 14:38 +0800, lijun wrote:
> I guess, final the value of addr is not important, just all of addr must=
=20
> read once is very important,
>=20
> so use two 'for()' and 'volatile' to flush all of addr=C2=A0 's cache,=
=20
> exactly as the name of the function is
>=20
> "flush_cache_last_level".

AFAIK do a read does not mean flushing the cache.  The cacop instruction
is intended to do the cache flushing.

So why the cacop instruction in flush_cache_leaf is not enough to flush
it?  Is there something wrong in flush_cache_leaf?  It currently reads:

static void flush_cache_leaf(unsigned int leaf)
{
    int i, j, nr_nodes;
    uint64_t addr =3D CSR_DMW0_BASE;
    struct cache_desc *cdesc =3D current_cpu_data.cache_leaves + leaf;

    nr_nodes =3D cache_private(cdesc) ? 1 : loongson_sysconf.nr_nodes;

    do {
        for (i =3D 0; i < cdesc->sets; i++) {
            for (j =3D 0; j < cdesc->ways; j++) {
                flush_cache_line(leaf, addr);
                addr++;
            }

            addr -=3D cdesc->ways;
            addr +=3D cdesc->linesz;
        }
        addr +=3D (1ULL << NODE_ADDRSPACE_SHIFT);
    } while (--nr_nodes > 0);=20
}

There is something bizarre: the flush_cache_line function uses the cacop
instruction in the Index Invalidate / Invalidate and Writeback mode, and
if I read the LoongArch manual correctly in this mode CSR_DMW0_BASE and
1ULL << NODE_ADDRSPACE_SHIFT are just ignored.

Is this something undocumented or flush_cache_leaf is just "incorrect"?

--=20
Xi Ruoyao <xry111@xry111.site>
School of Aerospace Science and Technology, Xidian University