Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <0000000000005a800a05cd849c36@google.com> <CACT4Y+ZRrxmLoor53nkD54sA5PJcRjWqheo262tudjrLO2rXzQ@mail.gmail.com>
 <20211004131516.GV3544071@ziepe.ca> <CACT4Y+bTB3DCGnem7V2ODpwgmiQdGuJae+h93kfniYn1Pr_x2g@mail.gmail.com>
 <4F4604B1-6EF7-4435-BB12-87664EF852C3@oracle.com>
In-Reply-To: <4F4604B1-6EF7-4435-BB12-87664EF852C3@oracle.com>
From:   Zhu Yanjun <zyjzyj2000@gmail.com>
Date:   Tue, 5 Oct 2021 21:11:12 +0800
Message-ID: <CAD=hENdbCdjPCEnfz0-to81qGGAN4ONkHdrhQEPc1bC+-peYMQ@mail.gmail.com>
Subject: Re: [syzbot] BUG: RESTRACK detected leak of resources
To:     Haakon Bugge <haakon.bugge@oracle.com>
Cc:     Dmitry Vyukov <dvyukov@google.com>, Jason Gunthorpe <jgg@ziepe.ca>,
        Doug Ledford <dledford@redhat.com>,
        syzbot <syzbot+3a992c9e4fd9f0e6fd0e@syzkaller.appspotmail.com>,
        Leon Romanovsky <leon@kernel.org>,
        OFED mailing list <linux-rdma@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        "syzkaller-bugs@googlegroups.com" <syzkaller-bugs@googlegroups.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Oct 5, 2021 at 1:56 AM Haakon Bugge <haakon.bugge@oracle.com> wrote=
:
>
>
>
> > On 4 Oct 2021, at 15:22, Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > On Mon, 4 Oct 2021 at 15:15, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> >>
> >> On Mon, Oct 04, 2021 at 02:42:11PM +0200, Dmitry Vyukov wrote:
> >>> On Mon, 4 Oct 2021 at 12:45, syzbot
> >>> <syzbot+3a992c9e4fd9f0e6fd0e@syzkaller.appspotmail.com> wrote:
> >>>>
> >>>> Hello,
> >>>>
> >>>> syzbot found the following issue on:
> >>>>
> >>>> HEAD commit:    c7b4d0e56a1d Add linux-next specific files for 20210=
930
> >>>> git tree:       linux-next
> >>>> console output: https://syzkaller.appspot.com/x/log.txt?x=3D104be6cb=
300000
> >>>> kernel config:  https://syzkaller.appspot.com/x/.config?x=3Dc9a1f668=
5aeb48bd
> >>>> dashboard link: https://syzkaller.appspot.com/bug?extid=3D3a992c9e4f=
d9f0e6fd0e
> >>>> compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU B=
inutils for Debian) 2.35.2
> >>>>
> >>>> Unfortunately, I don't have any reproducer for this issue yet.
> >>>>
> >>>> IMPORTANT: if you fix the issue, please add the following tag to the=
 commit:
> >>>> Reported-by: syzbot+3a992c9e4fd9f0e6fd0e@syzkaller.appspotmail.com
> >>>
> >>> +RESTRACK maintainers
> >>>
> >>> (it would also be good if RESTRACK would print a more standard oops
> >>> with stack/filenames, so that testing systems can attribute issues to
> >>> files/maintainers).
> >>
> >> restrack certainly should trigger a WARN_ON to stop the kernel.. But I
> >> don't know what stack track would be useful here. The culprit is
> >> always the underlying driver, not the core code..
> >
> > There seems to be a significant overlap between
> > drivers/infiniband/core/restrack.c and drivers/infiniband/sw/rxe/rxe.c
> > maintainers, so perhaps restrack.c is good enough approximation to
> > extract relevant people (definitely better then no CC at all :))
>
> Looks to me as this is rxe:
>
> [ 1892.778632][ T8958] BUG: KASAN: use-after-free in __rxe_drop_index_loc=
ked+0xb5/0x100
> [snip]
> [ 1892.822375][ T8958] Call Trace:
> [ 1892.825655][ T8958]  <TASK>
> [ 1892.828594][ T8958]  dump_stack_lvl+0xcd/0x134
> [ 1892.833273][ T8958]  print_address_description.constprop.0.cold+0x6c/0=
x30c
> [ 1892.840316][ T8958]  ? __rxe_drop_index_locked+0xb5/0x100
> [ 1892.845864][ T8958]  ? __rxe_drop_index_locked+0xb5/0x100
> [ 1892.851424][ T8958]  kasan_report.cold+0x83/0xdf
> [ 1892.856200][ T8958]  ? __rxe_drop_index_locked+0xb5/0x100
> [ 1892.861761][ T8958]  kasan_check_range+0x13d/0x180
> [ 1892.866780][ T8958]  __rxe_drop_index_locked+0xb5/0x100
> [ 1892.872164][ T8958]  __rxe_drop_index+0x3f/0x60
> [ 1892.876850][ T8958]  rxe_dereg_mr+0x14b/0x240
> [ 1892.881381][ T8958]  ib_dealloc_pd_user+0x96/0x230
> [ 1892.886566][ T8958]  rds_ib_dev_free+0xd4/0x3a0
>
> So, RDS de-allocs its PD, ib core must first de-register the PD's local M=
R, calls rxe_dereg_mr(), ...

int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata)
{
        struct rxe_mr *mr =3D to_rmr(ibmr);

        if (atomic_read(&mr->num_mw) > 0) {
                pr_warn("%s: Attempt to deregister an MR while bound to MWs=
\n",
                        __func__);
                return -EINVAL;
        }

        mr->state =3D RXE_MR_STATE_ZOMBIE;
        rxe_drop_ref(mr_pd(mr));
        rxe_drop_index(mr);            <-------This is call trace beginning=
.
        rxe_drop_ref(mr);

        return 0;
}

struct rxe_mr {
        struct rxe_pool_entry   pelem; <-----A ref_cnt in this struct.
        struct ib_mr            ibmr;

        struct ib_umem          *umem;

struct rxe_pool_entry {
        struct rxe_pool         *pool;
        struct kref             ref_cnt;        <-------This ref_cnt may he=
lp.
        struct list_head        list;

Zhu Yanjun

>
>
> Thxs, H=C3=A5kon
>
>
> >
> >> Anyhow, this report is either rxe or rds by the look of it.
> >>
> >> Jason
>