Received: by 2002:ac0:a5b6:0:0:0:0:0 with SMTP id m51-v6csp3598164imm; Tue, 29 May 2018 09:56:20 -0700 (PDT) X-Google-Smtp-Source: AB8JxZpXe9pbkmZv1lVIJ6Z4raQIrtTz62RoSBYu9sTNB6Woze6Z/WSTtmVaqXXRB5u7osf6fhF3 X-Received: by 2002:a63:9302:: with SMTP id b2-v6mr14655261pge.263.1527612980325; Tue, 29 May 2018 09:56:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527612980; cv=none; d=google.com; s=arc-20160816; b=Pxwo+CiPCtqBHMyQAbzJsjEUHjqjN8QDWnFk3IgSky7ugvgUA+U9+Ju5beetxU9hwd FLIsAntVpgVz3McvUn3Tkh6OS50p/qeOzmq1oC96eyObQ35fn18gHHeqU4cWVqwOllAE +i9koAPbxsT/E3EPxKyrvNGjuE+rJ2F9wlVO4RUmaj/5HsRQsAkR27P5Xe0wv0UV/Eot ikmiqp8oQLM3mRYybxdy/baADwBtyvziv+KKpqRVDL9q2f50M/XIB7T3tcwL8YsOKgsD 3I9nnXKx72gLkL7QuQvmr5AxWM6pArmCa08aADHHNl1uI5iW7R/qX5ObNjyLICTEdv+7 CV+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:content-transfer-encoding :content-id:spamdiagnosticmetadata:spamdiagnosticoutput :content-language:accept-language:in-reply-to:references:message-id :date:thread-index:thread-topic:subject:cc:to:from:dkim-signature :dkim-signature:arc-authentication-results; bh=pxRsQu8pZEUCesugjRHWv+zPIbwxtBb+cnCmDSqsb7E=; b=Bck91kDpXM247Nkbzony5outNvrpZRsmrOMT2I+1UzuHqnhLDdF38vOraUFA7oA5K4 O0LVyK5kxVgU6RZPxzeXMtZudM8jQC3sTRzOCYe3Q1/Y4akeMzMIbYFiLPRVRdpBaGpl 3tQHJDOO80MNt0Dcy4IYVqwRJL4uvysFNpQvQ80qsICgGJv3rfgHM8pUbBqGnpkglaHl 0X4ulY9pQ1mYZzZ2RfN86sUsAu0UXj0vBYQr8Iu9RaM1xFzU80xHL6LrZCA9Wz6r/VJa 7n4ovrOhE3CwnTLeJeGDNnJurlZx6zp07z6uwwQQbvm+MyVshXk7uwK6t/Ovk+FuiWcR dfUw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=TP7Lkvwk; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=DGDln+G6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u15-v6si25909352pgv.355.2018.05.29.09.55.53; Tue, 29 May 2018 09:56:20 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@fb.com header.s=facebook header.b=TP7Lkvwk; dkim=pass header.i=@fb.onmicrosoft.com header.s=selector1-fb-com header.b=DGDln+G6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=fb.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S936904AbeE2QzR (ORCPT + 99 others); Tue, 29 May 2018 12:55:17 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:40178 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935486AbeE2QzP (ORCPT ); Tue, 29 May 2018 12:55:15 -0400 Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4TGr0tN001236; Tue, 29 May 2018 09:55:06 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=facebook; bh=pxRsQu8pZEUCesugjRHWv+zPIbwxtBb+cnCmDSqsb7E=; b=TP7LkvwkHM2BuvC/ypriXfA6UL3l8ij/P0x49AK/Zppwvf3bNYFPHFPB6A7h/PlDvU1b sExsu6MZ+upMimmUw5/XUqewoxQH1q6JTzqYCVvEzvBcJnBxeKLWlBiYM/md8ZbJqmO1 QyuF48m8LFtCr82SvAEuxnIQ3di+iuC4Eo4= Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0a-00082601.pphosted.com with ESMTP id 2j97w3gh53-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 29 May 2018 09:55:05 -0700 Received: from NAM02-CY1-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.25) with Microsoft SMTP Server (TLS) id 14.3.361.1; Tue, 29 May 2018 12:55:03 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pxRsQu8pZEUCesugjRHWv+zPIbwxtBb+cnCmDSqsb7E=; b=DGDln+G6jq2jci5o1IQ2QbilbgcfTTohW8YuFIP7Z5oFSosZ8e1m8hUWgc55gLUQ5V3BhUtU5w5G212RnePG59i4yIzwBaWRQBFGUfqpiohazoIOCQudupgCcOgFdQE5oco5wSH9z0fzah8s534BlO4t8/uKjxYqo8GIUqtZ6TQ= Received: from MWHPR15MB1165.namprd15.prod.outlook.com (10.175.2.19) by MWHPR15MB1631.namprd15.prod.outlook.com (10.175.135.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.797.11; Tue, 29 May 2018 16:55:00 +0000 Received: from MWHPR15MB1165.namprd15.prod.outlook.com ([fe80::9485:7727:4b71:8cc0]) by MWHPR15MB1165.namprd15.prod.outlook.com ([fe80::9485:7727:4b71:8cc0%14]) with mapi id 15.20.0797.018; Tue, 29 May 2018 16:55:00 +0000 From: Song Liu To: Thomas Gleixner CC: Song Liu , Tariq Toukan , Dmitry Safonov <0x7f454c46@gmail.com>, open list , Maor Gottlieb , "Kernel Team" Subject: Re: WARNING and PANIC in irq_matrix_free Thread-Topic: WARNING and PANIC in irq_matrix_free Thread-Index: AQHT9GSPvVlQSgBGI0+pRnF2w/wY8aRE+2SAgACA+QCAABp+gIAAM+8AgACcbQCAAIuNAA== Date: Tue, 29 May 2018 16:54:59 +0000 Message-ID: <7954157A-E523-4041-825B-828A3A38E51B@fb.com> References: <16f47fa4-1555-cddb-3dfb-7d56fb992ea1@mellanox.com> <09A22A95-6BBD-48EB-A2FE-42BF6244F751@fb.com> <3F47F523-64C5-422B-B9B0-73B8D105CF71@fb.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: Apple Mail (2.3445.6.18) x-originating-ip: [2620:10d:c090:200::4:40e8] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;MWHPR15MB1631;7:97sWd0OPcNwk3veg1tF3stnm5u2b40Q+vsoJt5CrqWc7Oy2OPrMawoym+35vxT7i21qWlVrqEargT+FynELn6t2EavKWfmc9Jwz20SgZhSvJC/vHdkT/F4ZNoQKbbCeAuKQVGTbzfTZfrEyUZRUzJOOYINoS73nagCph/Z+HTBsO6FothE0HCLzTH2tRW1rwbz+2t6ZOjDTWgZ4ukN8cXlV55xATd3Ibb5Xtrg+v8tChwt7aK8lcRLoCds/Xs2w+;20:pXMtjnlTIw1yrs32pjAsi1h6jRnXn3cZUA4E9zX3KwKl/wYJspQW/p7QRSD10n+ZjFTKcmWTf9m3varen3oYX+sBy8HuQGHZDOwkTKBh7CTBlMFFmcA1XImwNWngdOAIG9Eh+Q+rfqrnkV1oQ96dNHUz2pnh+JxREnrQKQgc37U= x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020);SRVR:MWHPR15MB1631; x-ms-traffictypediagnostic: MWHPR15MB1631: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(209352067349851); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(3231254)(11241501184)(944501410)(52105095)(10201501046)(149027)(150027)(6041310)(20161123558120)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123564045)(6072148)(201708071742011)(7699016);SRVR:MWHPR15MB1631;BCL:0;PCL:0;RULEID:;SRVR:MWHPR15MB1631; x-forefront-prvs: 0687389FB0 x-forefront-antispam-report: SFV:NSPM;SFS:(10019020)(396003)(366004)(346002)(39380400002)(376002)(39860400002)(199004)(189003)(2906002)(6512007)(2616005)(446003)(39060400002)(11346002)(6116002)(229853002)(33656002)(6486002)(486006)(5250100002)(53946003)(76176011)(6506007)(6436002)(5890100001)(53546011)(102836004)(478600001)(4326008)(14454004)(476003)(3660700001)(99286004)(7736002)(53936002)(59450400001)(106356001)(105586002)(2900100001)(46003)(3280700002)(25786009)(57306001)(68736007)(305945005)(36756003)(186003)(50226002)(97736004)(8676002)(5660300001)(6916009)(81166006)(81156014)(93886005)(54906003)(6246003)(8936002)(316002)(86362001)(82746002)(83716003);DIR:OUT;SFP:1102;SCL:1;SRVR:MWHPR15MB1631;H:MWHPR15MB1165.namprd15.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;A:1;MX:1; received-spf: None (protection.outlook.com: fb.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: jINbYZVCbNt/gREr6hqoc0nLmCJFJYHN7d8LmwzgaGsX6lT74O/mgdC1dxZVChUrCHoEQ+CmaMEHDkzHZCJAsh47xqatVRrws+wxll8EsoHYFVoRzXssOY8P6w8TWGpZBqG9Xfx7URxjDajFZqO3zavubkDbj3/oXTbsP9iqspIY2EKn9KJEhpi/DBm3hn4r spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-ID: <32919D25A0BD8C44AC747BB8326A27C3@namprd15.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 4b7f04bd-8193-4c7a-b9cc-08d5c584ea68 X-MS-Exchange-CrossTenant-Network-Message-Id: 4b7f04bd-8193-4c7a-b9cc-08d5c584ea68 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 May 2018 16:54:59.9692 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1631 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-05-29_06:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > On May 29, 2018, at 1:35 AM, Thomas Gleixner wrote: >=20 > On Mon, 28 May 2018, Song Liu wrote: >>> On May 28, 2018, at 1:09 PM, Thomas Gleixner wrote= : >>>=20 >>> On Mon, 28 May 2018, Song Liu wrote: >>>> This doesn't fix the issue with bnxt. Here is a trace with this patch: >>>=20 >>> That's just the backtrace which is not really helpful. The real questio= n is >>> what leads to this scenaria. >>>=20 >>> What I was asking for is to enable the irq_vector and irq_matrix trace >>> points along with ftrace_dump_on_oops. The latter dumps the trace buffe= r >>> when the problem happens and that should give us a hint what actually >>> causes that. >>=20 >> Attached is the dump I got. It seems pretty noisy, with many message los= t. >>=20 >> Maybe we cannot enable all trace points under irq_vectors/ and irq_matri= x. >=20 > Right. Sorry, I forgot to say that we only need the irq_vectors ones whic= h > are related to vector allocation, i.e.: irq_vectors/vector_*=20 >=20 > Thanks, >=20 > tglx Here is the ftrace dump: [ 1609.472697] <...>-53363 19d... 1610359225us : vector_deactivate: irq= =3D29 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472698] <...>-53363 19d... 1610359229us : vector_clear: irq=3D29= vector=3D33 cpu=3D21 prev_vector=3D0 prev_cpu=3D0 [ 1609.472699] <...>-53363 19d... 1610359230us : irq_matrix_free: bit= =3D33 cpu=3D21 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11239, global_rsvd=3D23, total_alloc=3D17 [ 1609.472699] <...>-53363 19d... 1610359231us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11239, global_rsvd=3D24, total_alloc=3D17 [ 1609.472700] <...>-53363 19d... 1610359232us : vector_reserve: irq=3D= 29 ret=3D0 [ 1609.472700] <...>-53363 19d... 1610359233us : vector_config: irq=3D2= 9 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472701] <...>-53363 19d... 1610359242us : vector_deactivate: irq= =3D30 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472701] <...>-53363 19d... 1610359242us : vector_clear: irq=3D30= vector=3D33 cpu=3D11 prev_vector=3D0 prev_cpu=3D1 [ 1609.472701] <...>-53363 19d... 1610359242us : irq_matrix_free: bit= =3D33 cpu=3D11 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11240, global_rsvd=3D24, total_alloc=3D16 [ 1609.472702] <...>-53363 19d... 1610359243us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11240, global_rsvd=3D25, total_alloc=3D16 [ 1609.472702] <...>-53363 19d... 1610359243us : vector_reserve: irq=3D= 30 ret=3D0 [ 1609.472703] <...>-53363 19d... 1610359243us : vector_config: irq=3D3= 0 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472703] <...>-53363 19d... 1610359248us : vector_deactivate: irq= =3D31 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472703] <...>-53363 19d... 1610359248us : vector_clear: irq=3D31= vector=3D33 cpu=3D20 prev_vector=3D0 prev_cpu=3D2 [ 1609.472704] <...>-53363 19d... 1610359249us : irq_matrix_free: bit= =3D33 cpu=3D20 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11241, global_rsvd=3D25, total_alloc=3D15 [ 1609.472704] <...>-53363 19d... 1610359249us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11241, global_rsvd=3D26, total_alloc=3D15 [ 1609.472705] <...>-53363 19d... 1610359249us : vector_reserve: irq=3D= 31 ret=3D0 [ 1609.472705] <...>-53363 19d... 1610359249us : vector_config: irq=3D3= 1 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472705] <...>-53363 19d... 1610359255us : vector_deactivate: irq= =3D32 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472706] <...>-53363 19d... 1610359255us : vector_clear: irq=3D32= vector=3D33 cpu=3D9 prev_vector=3D0 prev_cpu=3D3 [ 1609.472707] <...>-53363 19d... 1610359255us : irq_matrix_free: bit= =3D33 cpu=3D9 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 g= lobal_avl=3D11242, global_rsvd=3D26, total_alloc=3D14 [ 1609.472707] <...>-53363 19d... 1610359256us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11242, global_rsvd=3D27, total_alloc=3D14 [ 1609.472708] <...>-53363 19d... 1610359256us : vector_reserve: irq=3D= 32 ret=3D0 [ 1609.472708] <...>-53363 19d... 1610359256us : vector_config: irq=3D3= 2 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472709] <...>-53363 19d... 1610359261us : vector_deactivate: irq= =3D33 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472709] <...>-53363 19d... 1610359261us : vector_clear: irq=3D33= vector=3D33 cpu=3D18 prev_vector=3D0 prev_cpu=3D4 [ 1609.472710] <...>-53363 19d... 1610359262us : irq_matrix_free: bit= =3D33 cpu=3D18 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11243, global_rsvd=3D27, total_alloc=3D13 [ 1609.472710] <...>-53363 19d... 1610359262us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11243, global_rsvd=3D28, total_alloc=3D13 [ 1609.472711] <...>-53363 19d... 1610359262us : vector_reserve: irq=3D= 33 ret=3D0 [ 1609.472711] <...>-53363 19d... 1610359262us : vector_config: irq=3D3= 3 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472711] <...>-53363 19d... 1610359268us : vector_deactivate: irq= =3D34 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472712] <...>-53363 19d... 1610359268us : vector_clear: irq=3D34= vector=3D34 cpu=3D7 prev_vector=3D0 prev_cpu=3D5 [ 1609.472712] <...>-53363 19d... 1610359268us : irq_matrix_free: bit= =3D34 cpu=3D7 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 g= lobal_avl=3D11244, global_rsvd=3D28, total_alloc=3D12 [ 1609.472713] <...>-53363 19d... 1610359268us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11244, global_rsvd=3D29, total_alloc=3D12 [ 1609.472713] <...>-53363 19d... 1610359269us : vector_reserve: irq=3D= 34 ret=3D0 [ 1609.472713] <...>-53363 19d... 1610359269us : vector_config: irq=3D3= 4 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472714] <...>-53363 19d... 1610359279us : vector_deactivate: irq= =3D35 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472714] <...>-53363 19d... 1610359279us : vector_clear: irq=3D35= vector=3D33 cpu=3D26 prev_vector=3D0 prev_cpu=3D6 [ 1609.472715] <...>-53363 19d... 1610359279us : irq_matrix_free: bit= =3D33 cpu=3D26 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11245, global_rsvd=3D29, total_alloc=3D11 [ 1609.472715] <...>-53363 19d... 1610359279us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11245, global_rsvd=3D30, total_alloc=3D11 [ 1609.472715] <...>-53363 19d... 1610359280us : vector_reserve: irq=3D= 35 ret=3D0 [ 1609.472716] <...>-53363 19d... 1610359280us : vector_config: irq=3D3= 5 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472716] <...>-53363 19d... 1610359285us : vector_deactivate: irq= =3D36 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472717] <...>-53363 19d... 1610359285us : vector_clear: irq=3D36= vector=3D33 cpu=3D10 prev_vector=3D0 prev_cpu=3D7 [ 1609.472717] <...>-53363 19d... 1610359286us : irq_matrix_free: bit= =3D33 cpu=3D10 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11246, global_rsvd=3D30, total_alloc=3D10 [ 1609.472718] <...>-53363 19d... 1610359286us : irq_matrix_reserve: on= line_maps=3D56 global_avl=3D11246, global_rsvd=3D31, total_alloc=3D10 [ 1609.472718] <...>-53363 19d... 1610359286us : vector_reserve: irq=3D= 36 ret=3D0 [ 1609.472718] <...>-53363 19d... 1610359286us : vector_config: irq=3D3= 6 vector=3D239 cpu=3D0 apicdest=3D0x00000000 [ 1609.472719] <...>-53363 20d... 1610366617us : vector_activate: irq= =3D29 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472719] <...>-53363 20d... 1610366619us : vector_alloc: irq=3D29= vector=3D4294967268 reserved=3D1 ret=3D0 [ 1609.472719] <...>-53363 20d... 1610366621us : irq_matrix_alloc: bit= =3D33 cpu=3D7 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 g= lobal_avl=3D11245, global_rsvd=3D30, total_alloc=3D11 [ 1609.472720] <...>-53363 20d... 1610366623us : vector_update: irq=3D2= 9 vector=3D33 cpu=3D7 prev_vector=3D0 prev_cpu=3D21 [ 1609.472720] <...>-53363 20d... 1610366623us : vector_alloc: irq=3D29= vector=3D33 reserved=3D1 ret=3D0 [ 1609.472721] <...>-53363 20d... 1610366623us : vector_config: irq=3D2= 9 vector=3D33 cpu=3D7 apicdest=3D0x00000010 [ 1609.472721] <...>-53363 20d... 1610366629us : irq_matrix_alloc: bit= =3D33 cpu=3D21 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 = global_avl=3D11244, global_rsvd=3D30, total_alloc=3D12 [ 1609.472722] <...>-53363 20d... 1610366629us : vector_update: irq=3D2= 9 vector=3D33 cpu=3D21 prev_vector=3D33 prev_cpu=3D7 [ 1609.472723] <...>-53363 20d... 1610366629us : vector_alloc: irq=3D29= vector=3D33 reserved=3D1 ret=3D0 [ 1609.472723] <...>-53363 20d... 1610366630us : vector_config: irq=3D2= 9 vector=3D33 cpu=3D21 apicdest=3D0x00000030 [ 1609.472724] <...>-53363 20d... 1610366637us : irq_matrix_alloc: bit= =3D35 cpu=3D0 online=3D1 avl=3D197 alloc=3D4 managed=3D0 online_maps=3D56 g= lobal_avl=3D11243, global_rsvd=3D30, total_alloc=3D13 [ 1609.472724] <...>-53363 20d... 1610366637us : vector_update: irq=3D2= 9 vector=3D35 cpu=3D0 prev_vector=3D33 prev_cpu=3D21 [ 1609.472725] <...>-53363 20d... 1610366637us : vector_alloc: irq=3D29= vector=3D35 reserved=3D1 ret=3D0 [ 1609.472725] <...>-53363 20d... 1610366638us : vector_config: irq=3D2= 9 vector=3D35 cpu=3D0 apicdest=3D0x00000000 [ 1609.472725] <...>-53363 20dN.. 1610366643us : vector_activate: irq= =3D30 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472726] <...>-53363 20dN.. 1610366644us : irq_matrix_alloc: bit= =3D33 cpu=3D11 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 = global_avl=3D11242, global_rsvd=3D29, total_alloc=3D14 [ 1609.472726] <...>-53363 20dN.. 1610366644us : vector_update: irq=3D3= 0 vector=3D33 cpu=3D11 prev_vector=3D0 prev_cpu=3D11 [ 1609.472727] <...>-53363 20dN.. 1610366644us : vector_alloc: irq=3D30= vector=3D33 reserved=3D1 ret=3D0 [ 1609.472727] <...>-53363 20dN.. 1610366644us : vector_config: irq=3D3= 0 vector=3D33 cpu=3D11 apicdest=3D0x00000018 [ 1609.472727] <...>-53363 20dN.. 1610366647us : vector_config: irq=3D3= 0 vector=3D33 cpu=3D11 apicdest=3D0x00000018 [ 1609.472728] <...>-53363 20dN.. 1610366650us : irq_matrix_alloc: bit= =3D34 cpu=3D1 online=3D1 avl=3D199 alloc=3D2 managed=3D0 online_maps=3D56 g= lobal_avl=3D11241, global_rsvd=3D29, total_alloc=3D15 [ 1609.472728] <...>-53363 20dN.. 1610366651us : vector_update: irq=3D3= 0 vector=3D34 cpu=3D1 prev_vector=3D33 prev_cpu=3D11 [ 1609.472729] <...>-53363 20dN.. 1610366651us : vector_alloc: irq=3D30= vector=3D34 reserved=3D1 ret=3D0 [ 1609.472729] <...>-53363 20dN.. 1610366651us : vector_config: irq=3D3= 0 vector=3D34 cpu=3D1 apicdest=3D0x00000002 [ 1609.472729] <...>-53363 20dN.. 1610366654us : vector_activate: irq= =3D31 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472730] <...>-53363 20dN.. 1610366654us : vector_alloc: irq=3D31= vector=3D4294967268 reserved=3D1 ret=3D0 [ 1609.472730] <...>-53363 20dN.. 1610366655us : irq_matrix_alloc: bit= =3D33 cpu=3D9 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 g= lobal_avl=3D11240, global_rsvd=3D28, total_alloc=3D16 [ 1609.472731] <...>-53363 20dN.. 1610366655us : vector_update: irq=3D3= 1 vector=3D33 cpu=3D9 prev_vector=3D0 prev_cpu=3D20 [ 1609.472731] <...>-53363 20dN.. 1610366656us : vector_alloc: irq=3D31= vector=3D33 reserved=3D1 ret=3D0 [ 1609.472731] <...>-53363 20dN.. 1610366656us : vector_config: irq=3D3= 1 vector=3D33 cpu=3D9 apicdest=3D0x00000014 [ 1609.472732] -0 7d.h. 1610366658us : vector_free_moved: irq= =3D29 cpu=3D21 vector=3D33 is_managed=3D0 [ 1609.472732] -0 7d.h. 1610366660us : irq_matrix_free: bit= =3D33 cpu=3D21 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11241, global_rsvd=3D28, total_alloc=3D15 [ 1609.472733] <...>-53363 20dN.. 1610366662us : irq_matrix_alloc: bit= =3D33 cpu=3D20 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 = global_avl=3D11240, global_rsvd=3D28, total_alloc=3D16 [ 1609.472733] <...>-53363 20dN.. 1610366662us : vector_update: irq=3D3= 1 vector=3D33 cpu=3D20 prev_vector=3D33 prev_cpu=3D9 [ 1609.472734] <...>-53363 20dN.. 1610366662us : vector_alloc: irq=3D31= vector=3D33 reserved=3D1 ret=3D0 [ 1609.472734] <...>-53363 20dN.. 1610366662us : vector_config: irq=3D3= 1 vector=3D33 cpu=3D20 apicdest=3D0x0000002c [ 1609.472735] <...>-53363 20dN.. 1610366666us : irq_matrix_alloc: bit= =3D34 cpu=3D2 online=3D1 avl=3D199 alloc=3D2 managed=3D0 online_maps=3D56 g= lobal_avl=3D11239, global_rsvd=3D28, total_alloc=3D17 [ 1609.472735] <...>-53363 20dN.. 1610366666us : vector_update: irq=3D3= 1 vector=3D34 cpu=3D2 prev_vector=3D33 prev_cpu=3D20 [ 1609.472736] <...>-53363 20dN.. 1610366666us : vector_alloc: irq=3D31= vector=3D34 reserved=3D1 ret=3D0 [ 1609.472736] <...>-53363 20dN.. 1610366666us : vector_config: irq=3D3= 1 vector=3D34 cpu=3D2 apicdest=3D0x00000004 [ 1609.472737] <...>-53363 20dNh. 1610366669us : vector_free_moved: irq= =3D31 cpu=3D20 vector=3D33 is_managed=3D0 [ 1609.472737] <...>-53363 20dNh. 1610366670us : irq_matrix_free: bit= =3D33 cpu=3D20 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11240, global_rsvd=3D28, total_alloc=3D16 [ 1609.472738] <...>-53363 20dN.. 1610366671us : vector_activate: irq= =3D32 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472739] <...>-53363 20dN.. 1610366671us : irq_matrix_alloc: bit= =3D34 cpu=3D9 online=3D1 avl=3D199 alloc=3D2 managed=3D0 online_maps=3D56 g= lobal_avl=3D11239, global_rsvd=3D27, total_alloc=3D17 [ 1609.472739] <...>-53363 20dN.. 1610366671us : vector_update: irq=3D3= 2 vector=3D34 cpu=3D9 prev_vector=3D0 prev_cpu=3D9 [ 1609.472739] <...>-53363 20dN.. 1610366671us : vector_alloc: irq=3D32= vector=3D34 reserved=3D1 ret=3D0 [ 1609.472740] <...>-53363 20dN.. 1610366672us : vector_config: irq=3D3= 2 vector=3D34 cpu=3D9 apicdest=3D0x00000014 [ 1609.472740] <...>-53363 20dN.. 1610366674us : vector_config: irq=3D3= 2 vector=3D34 cpu=3D9 apicdest=3D0x00000014 [ 1609.472741] <...>-53363 20dN.. 1610366678us : irq_matrix_alloc: bit= =3D34 cpu=3D3 online=3D1 avl=3D199 alloc=3D2 managed=3D0 online_maps=3D56 g= lobal_avl=3D11238, global_rsvd=3D27, total_alloc=3D18 [ 1609.472741] <...>-53363 20dN.. 1610366678us : vector_update: irq=3D3= 2 vector=3D34 cpu=3D3 prev_vector=3D34 prev_cpu=3D9 [ 1609.472741] <...>-53363 20dN.. 1610366679us : vector_alloc: irq=3D32= vector=3D34 reserved=3D1 ret=3D0 [ 1609.472742] <...>-53363 20dN.. 1610366679us : vector_config: irq=3D3= 2 vector=3D34 cpu=3D3 apicdest=3D0x00000006 [ 1609.472742] -0 11d.h. 1610366680us : vector_free_moved: irq= =3D30 cpu=3D11 vector=3D33 is_managed=3D0 [ 1609.472743] -0 11d.h. 1610366681us : irq_matrix_free: bit= =3D33 cpu=3D11 online=3D1 avl=3D201 alloc=3D0 managed=3D0 online_maps=3D56 = global_avl=3D11239, global_rsvd=3D27, total_alloc=3D17 [ 1609.472743] <...>-53363 20dN.. 1610366683us : vector_activate: irq= =3D33 is_managed=3D0 can_reserve=3D1 reserve=3D0 [ 1609.472743] <...>-53363 20dN.. 1610366683us : vector_alloc: irq=3D33= vector=3D4294967268 reserved=3D1 ret=3D0 [ 1609.472744] <...>-53363 20dN.. 1610366684us : irq_matrix_alloc: bit= =3D33 cpu=3D10 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 = global_avl=3D11238, global_rsvd=3D26, total_alloc=3D18 [ 1609.472744] <...>-53363 20dN.. 1610366684us : vector_update: irq=3D3= 3 vector=3D33 cpu=3D10 prev_vector=3D0 prev_cpu=3D18 [ 1609.472745] <...>-53363 20dN.. 1610366684us : vector_alloc: irq=3D33= vector=3D33 reserved=3D1 ret=3D0 [ 1609.472745] <...>-53363 20dN.. 1610366685us : vector_config: irq=3D3= 3 vector=3D33 cpu=3D10 apicdest=3D0x00000016 [ 1609.472745] <...>-53363 20dN.. 1610366688us : irq_matrix_alloc: bit= =3D33 cpu=3D18 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 = global_avl=3D11237, global_rsvd=3D26, total_alloc=3D19 [ 1609.472746] <...>-53363 20dN.. 1610366688us : vector_update: irq=3D3= 3 vector=3D33 cpu=3D18 prev_vector=3D33 prev_cpu=3D10 [ 1609.472746] <...>-53363 20dN.. 1610366688us : vector_alloc: irq=3D33= vector=3D33 reserved=3D1 ret=3D0 [ 1609.472747] <...>-53363 20dN.. 1610366688us : vector_config: irq=3D3= 3 vector=3D33 cpu=3D18 apicdest=3D0x00000028 [ 1609.472747] <...>-53363 20dN.. 1610366692us : irq_matrix_alloc: bit= =3D34 cpu=3D4 online=3D1 avl=3D199 alloc=3D2 managed=3D0 online_maps=3D56 g= lobal_avl=3D11236, global_rsvd=3D26, total_alloc=3D20 [ 1609.472747] <...>-53363 20dN.. 1610366692us : vector_update: irq=3D3= 3 vector=3D34 cpu=3D4 prev_vector=3D33 prev_cpu=3D18 [ 1609.472748] <...>-53363 20dN.. 1610366692us : vector_alloc: irq=3D33= vector=3D34 reserved=3D1 ret=3D0 [ 1609.472748] <...>-53363 20dN.. 1610366693us : vector_config: irq=3D3= 3 vector=3D34 cpu=3D4 apicdest=3D0x00000008 [ 1609.472749] -0 9d.h. 1610366694us : vector_free_moved: irq= =3D32 cpu=3D9 vector=3D34 is_managed=3D0 [ 1609.472749] -0 9d.h. 1610366695us : irq_matrix_free: bit= =3D34 cpu=3D9 online=3D1 avl=3D200 alloc=3D1 managed=3D0 online_maps=3D56 g= lobal_avl=3D11237, global_rsvd=3D26, total_alloc=3D19 [ 1609.472750] -0 9d.h. 1610366696us : vector_free_moved: irq= =3D31 cpu=3D20 vector=3D0 is_managed=3D0 Thanks, Song=