Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH v2] xen/events: Fix race in set_evtchn_to_irq
To:     Maximilian Heyne <mheyne@amazon.de>
Cc:     Amit Shah <aams@amazon.de>, Juergen Gross <jgross@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Wei Liu <wei.liu@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Jan Beulich <jbeulich@suse.com>,
        Malcolm Crossley <malcolm.crossley@citrix.com>,
        David Vrabel <david.vrabel@citrix.com>,
        Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
        xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <20210812130930.127134-1-mheyne@amazon.de>
From:   Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <da6174e0-3c1e-190e-6181-5c83f724a417@oracle.com>
Date:   Thu, 12 Aug 2021 16:23:27 -0400
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.12.0
In-Reply-To: <20210812130930.127134-1-mheyne@amazon.de>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [10.74.96.128] (160.34.88.128) by SA0PR11CA0032.namprd11.prod.outlook.com (2603:10b6:806:d0::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.16 via Frontend Transport; Thu, 12 Aug 2021 20:23:30 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 25dcba26-f649-428d-0b76-08d95dcf0db9
X-MS-TrafficTypeDiagnostic: BL3PR10MB5489:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BL3PR10MB54896C1AC3E3C491EDE50C768AF99@BL3PR10MB5489.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:8882;
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: U1OUQrZ3OjDP9oL3vh+AdEgwcc4CaNrsQdAlxP/AGeT3dav6BrueL28Gre5htR+2d20LkKBVb63IfViqU7JgOPBcdvJ/C/5zo0J5IZ8EnqjBo1+N7auXB8EmGVmXz+5lZAyjTiU+fo3m0duVSmV4zenzd+CO1SztViNlFoh7+D/xUZVIDy65QZnzkiHogQasEhAlyWM81OwKeLgElZdV/edhLcCAhxIqSJB+z0xdTKU/xHUT8zaE5OfPVnanRcNnGI9c1x8zpyXq8aXJbYM3kvpcEZDTW2UQksHkCItV6JmmZIAT45/r3iF7aWP12k5bI8KUv8PPCUPMYY3YEkv/MJ4PrMCG+wQVVgCrNJg/+9I6H8KkzUnVij9OlyGfuxC7h3cluWlW6Gr18IMiN5qt0XCwTmr6pz2DUiwb14L+WNnQVGmIW/g6PZFRuhwQaVMi+3RUcZ8nAGJI9OVmuFriF5sz9fI6SgQd1vYcmmKRFEw5zuirq9/XQ8DNasUY0MK/GRR41cUdOk3HZhSYdtPeI+aIMHJacHE1zgnpR+wU7B0Yjpqi6PxgLyaj3B1xTwmdteN0/R1sq9SlVtK2jQDWuY8Y6sKrYjWzexox8ke4h2oBv0BXv8gzsgkt/hm1nVu9unvhFyv/YJqTQs6DdfDTQWCkfv0lo4wiiBX2CNFD8CJCq/IuCwDnWWY01/wx/G44Ry1KBAsCxaFPY+aMEt91tNR2hL1W2oEzWIeiWJC+fhiqCMTTbhpOGbMXJgGXk+Ih
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BLAPR10MB5009.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(7416002)(316002)(16576012)(38100700002)(66946007)(6666004)(66556008)(31686004)(53546011)(4326008)(66476007)(6916009)(6486002)(2906002)(8676002)(5660300002)(31696002)(508600001)(2616005)(186003)(956004)(44832011)(8936002)(83380400001)(86362001)(26005)(54906003)(36756003)(43740500002)(45980500001);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RENQNzAzY2ZlQ3FwcU42cExzR3V2dzdQUjVMRjlsbTlYQjhSUFBmcDVLd1JW?=
 =?utf-8?B?QzcrRGY3eHRhM25xUjgxUkV2RXpQVGpVZis1bnZsR2hNVFVvM2RJWEdpNzNV?=
 =?utf-8?B?Lzl4NngvaXZWR0QzZjlnb0VvOGNTS0lrck9seG1BV1BWOXkwNUJvSXVvSUtS?=
 =?utf-8?B?VjcrcCtMeitvN3JnRURmU3ZtVkU2K2NhN1dyVTRjZlNYMXBGcXBuUVhVY0Zl?=
 =?utf-8?B?RzlWdkdqcnk5QmJGOGFDTFRQa1RDSUtlWFBqclBnUitmdkpBLzdxTWRmblFS?=
 =?utf-8?B?TGdFckxubm1qbjFZSTJ4MUNnREFxNDJ5c005MloxRU44RTNBQXJQU0ZTWlRS?=
 =?utf-8?B?RVhWbFJBYlV6T0JZellyZ3FhWFlrcXBITFd4Z1pJNEw4MmdWT0xkenBwejA5?=
 =?utf-8?B?ZWJaams2U2I2TVNOcGg0TFdUaHNZTGphQlhvWk9RMk45eVRXZDU4ZDdFazNr?=
 =?utf-8?B?L1RoYUR4elNJUGtLclJjLytNODR0dUtuVWx5ZzF5QUhzdlpsQlplK1BIdmwz?=
 =?utf-8?B?U21ydVI5NkE5L3lyVExTeHd1MUdleUxYUTRxbXVaOEZpOVhLcnZHRlc5Wmx6?=
 =?utf-8?B?TmVJSk9RT2IyS0FlUGprUUIreGx3TmJuai8xQTBYL1MvcXpXazFSVnAwZDI1?=
 =?utf-8?B?Q1M3TGN5cDN4Y2RBY3IvKzZOWDM2Y1J4cWhPOHh5dDgvaHA4b0dlcXZjTG5Y?=
 =?utf-8?B?MTFvZE81dnNqVUJ4Nk1kT2ZqdnFMSjdUTWhsLzNaWVNac3JRWldPMGl2dUFG?=
 =?utf-8?B?akxZYTNEQVJhZFVvTnZoaExwdGdnbTI3dnFmVGZCRys3QzVsUWlCN05LQlUv?=
 =?utf-8?B?VTNkWmJjbmhnZVBUSmhBQmtTWWJMclUxMTRmelBVRDNHUjVtZGVJaHZmNW5R?=
 =?utf-8?B?K0kzWmsydEE3SE11ekw5SW5yQUwvd2VUbERFaFlUNk0xNVJaVG1zWURscFJS?=
 =?utf-8?B?TDc2dlpEdkxSV1R3TldKZzVURXl2RDk1Y3I1SUVaRGkrL2JkU2kyZU0yUU1k?=
 =?utf-8?B?TkNyWXN4QmxVK0gyYWcxbDdoQ1RmYUNiNHIvUy80eUcwUlBJTlgrUk9hME5x?=
 =?utf-8?B?M0IxNWdTa0xjR29xdklBNzNiSHNiSWpSbk5kMjgvdzQreUZuMFY3TWhvcDho?=
 =?utf-8?B?cTlzMlZENlhVbS9vNkZrTVB0YmxOVUMybXF5SXJ6UU1zTTJIQXZORVhZT3FT?=
 =?utf-8?B?NVhZbHhYeksxMDdYWlY4TWVFRktycHNYRkVDNHBqTFovVzdhSnRLWE1RWEVQ?=
 =?utf-8?B?OXQ0K1V6VmZOMzl4VlVNWkJoVUdrYTQyQTlaUUt3WERwWExEdExMY1NjWmlq?=
 =?utf-8?B?cEltelQ3TlZBQVF1bDlKQ2w4V1BRSEFiRTFySGVreTNBL3Rhek1Zb01xa0Uy?=
 =?utf-8?B?QWhpVW1Oa2RQUFJ0Y1pHWGYxMDVldmhBa2pmaVRVZThQSjllNitJNlZzbWR6?=
 =?utf-8?B?WnJiWG44ZzJKWUJEYmtVWklTVHNsaVBBRnhOMmtpc2k5WjJXVnN4Z0lSclph?=
 =?utf-8?B?U08rK1hyVlpvTnpOYUtIcUxUeWM4SVozYWN6eElzSW9GSDR2VWNwUG1yZ0Zv?=
 =?utf-8?B?TFZlUndVRTFYWUVSRFRGcWtqdEpVRnpCQzZHQmhGdkFMNUlRRFVDMjl0cFdT?=
 =?utf-8?B?b0tieFNnL3VmaFZuUXF3dEcwcHgzTXA3U3k5enNnWDVFU2c2aWRZbzBVMWNJ?=
 =?utf-8?B?b2ErMlA5WEl4cXQ4Z1kvTHlOOTJ4RHIzSmduUUVNSnNSV2hVR2JWUWZhSVlN?=
 =?utf-8?Q?wgKP+I6kwakSUlpdtHOFLCattNr2Oco502YPvyU?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 25dcba26-f649-428d-0b76-08d95dcf0db9
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Aug 2021 20:23:32.1262
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AksGuPL1n9JUZoKRgjvL9MrAR2VDUGEVdDUXfHz7JkkS+G7joSepNRFlahuOT1zV0SdPYelvTT9pdSRnmg7wVPv5vDVqwPeZ7wH/JjOq1Hs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR10MB5489
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=10074 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 mlxscore=0
 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 spamscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2107140000 definitions=main-2108120132
X-Proofpoint-GUID: itarf4YV2cm3Or6BVqk-_ptLKOQXjn9J
X-Proofpoint-ORIG-GUID: itarf4YV2cm3Or6BVqk-_ptLKOQXjn9J
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 8/12/21 9:09 AM, Maximilian Heyne wrote:
> There is a TOCTOU issue in set_evtchn_to_irq. Rows in the evtchn_to_irq
> mapping are lazily allocated in this function. The check whether the row
> is already present and the row initialization is not synchronized. Two
> threads can at the same time allocate a new row for evtchn_to_irq and
> add the irq mapping to the their newly allocated row. One thread will
> overwrite what the other has set for evtchn_to_irq[row] and therefore
> the irq mapping is lost. This will trigger a BUG_ON later in
> bind_evtchn_to_cpu:
>
>   INFO: pci 0000:1a:15.4: [1d0f:8061] type 00 class 0x010802
>   INFO: nvme 0000:1a:12.1: enabling device (0000 -> 0002)
>   INFO: nvme nvme77: 1/0/0 default/read/poll queues
>   CRIT: kernel BUG at drivers/xen/events/events_base.c:427!
>   WARN: invalid opcode: 0000 [#1] SMP NOPTI
>   WARN: Workqueue: nvme-reset-wq nvme_reset_work [nvme]
>   WARN: RIP: e030:bind_evtchn_to_cpu+0xc2/0xd0
>   WARN: Call Trace:
>   WARN:  set_affinity_irq+0x121/0x150
>   WARN:  irq_do_set_affinity+0x37/0xe0
>   WARN:  irq_setup_affinity+0xf6/0x170
>   WARN:  irq_startup+0x64/0xe0
>   WARN:  __setup_irq+0x69e/0x740
>   WARN:  ? request_threaded_irq+0xad/0x160
>   WARN:  request_threaded_irq+0xf5/0x160
>   WARN:  ? nvme_timeout+0x2f0/0x2f0 [nvme]
>   WARN:  pci_request_irq+0xa9/0xf0
>   WARN:  ? pci_alloc_irq_vectors_affinity+0xbb/0x130
>   WARN:  queue_request_irq+0x4c/0x70 [nvme]
>   WARN:  nvme_reset_work+0x82d/0x1550 [nvme]
>   WARN:  ? check_preempt_wakeup+0x14f/0x230
>   WARN:  ? check_preempt_curr+0x29/0x80
>   WARN:  ? nvme_irq_check+0x30/0x30 [nvme]
>   WARN:  process_one_work+0x18e/0x3c0
>   WARN:  worker_thread+0x30/0x3a0
>   WARN:  ? process_one_work+0x3c0/0x3c0
>   WARN:  kthread+0x113/0x130
>   WARN:  ? kthread_park+0x90/0x90
>   WARN:  ret_from_fork+0x3a/0x50
>
> This patch sets evtchn_to_irq rows via a cmpxchg operation so that they
> will be set only once. The row is now cleared before writing it to
> evtchn_to_irq in order to not create a race once the row is visible for
> other threads.
>
> While at it, do not require the page to be zeroed, because it will be
> overwritten with -1's in clear_evtchn_to_irq_row anyway.
>
> Signed-off-by: Maximilian Heyne <mheyne@amazon.de>
> Fixes: d0b075ffeede ("xen/events: Refactor evtchn_to_irq array to be dynamically allocated")


Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>