Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: Problems starting Xen domU after latest stable update
To:     Michael D Labriola <michael.d.labriola@gmail.com>,
        David Woodhouse <dwmw@amazon.co.uk>,
        Juergen Gross <jgross@suse.com>,
        Sasha Levin <sashal@kernel.org>,
        Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
        Roger Pau Monne <roger.pau@citrix.com>,
        xen-devel@lists.xenproject.org, linux-kernel@vger.kernel.org
References: <2nft2kipqg.fsf@aragorn.infrastructure.cah>
From:   Boris Ostrovsky <boris.ostrovsky@oracle.com>
Message-ID: <983b87d6-edb8-21ea-7d6f-f653f5c0d048@oracle.com>
Date:   Thu, 28 Jan 2021 19:03:00 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.6.1
In-Reply-To: <2nft2kipqg.fsf@aragorn.infrastructure.cah>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from [10.74.101.127] (138.3.200.63) by CH0PR04CA0102.namprd04.prod.outlook.com (2603:10b6:610:75::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3805.16 via Frontend Transport; Fri, 29 Jan 2021 00:03:03 +0000
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: 17694743-63b1-4c86-3f63-08d8c3e94147
X-MS-TrafficTypeDiagnostic: BYAPR10MB3238:
X-MS-Exchange-Transport-Forked: True
X-Microsoft-Antispam-PRVS: <BYAPR10MB323844B23CE70927B56189DA8AB99@BYAPR10MB3238.namprd10.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:115;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: bjo4f3wHycI9Jlsh7GIkbRSyjjz0MmlB+v0N1I6wmq9xD0uSk6fzfUcLWwZaRkyGbzjBuXC8V3IoXNw2AXslGX7N/L3WIlvHoPEqFKqjzd0Q2i55kUEZJ4dBd3ND1jzV2Jvqt2NTU8Y7fJHjf7EXrsjHFtz1QjkOG4xYeEoxg6H/RB39epDpVHLZkW9/OY9HXz1WLVJExPX3/1X4xtKL+eovjJqXcQM3+QCGMJGWSE2geOL38t4fHOxJrlhU00t965UfP1k0UAMTgAjnwL9q7X0ACutSEo3r7T4i7CU5CHaM9bJKyyA7w4aWNixl2iBX2UiJOcz4Zdu0tX88XPPu0K7JxwE9KtREjS/PeS7kOTycYSiKjRpdjd62HMYlOL8wnpM0m6axpMhkwYhtuAiQqSCFS6nT60fYQj6+LEc1WfSxeSWEnsbHMwTTsrYMFS9e8blpMwHwdd1ZWUTBcu+KpqziFItlmt6ttI1t83ICoSlYJVK5psJ9cobLDR0y9I/S/fbN1FRe2/FrIoDyTq44PoTsqjHl3eMenOUaGr4GzLzKJ59kRT5tcSz63y/ST93jBFuf5XJCrFwYhARW5P3cdXrwC70CILOH/30Hyo/C1yU=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3288.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(376002)(366004)(396003)(136003)(346002)(44832011)(110136005)(8936002)(36756003)(316002)(86362001)(66946007)(2906002)(8676002)(66556008)(15650500001)(31686004)(956004)(16576012)(6486002)(31696002)(66476007)(478600001)(5660300002)(26005)(16526019)(186003)(53546011)(83380400001)(2616005)(45980500001)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: =?utf-8?B?TCtIcmxYeGd6a1J0bTUyOURmUFJLL1B6dytqNzBuOXdITXRQVG9BVW0ydWw4?=
 =?utf-8?B?aEsxVVo5TW5ZemU3TlowVGNzeFpPUk5aanFtbS9VK1dRZ0s5RWdYNWdYRmkw?=
 =?utf-8?B?eVNzVlhIMW4zL1FYdUVuOGZhNTJ6TTdGM3U2S0VRczZKdnFrU2FCNmlGU2Ja?=
 =?utf-8?B?QTFHcXliQWwzVzVrYitTOTRRclJzQUxjV29pa2Q4M2xiUUxWaFdzbko0RzdM?=
 =?utf-8?B?d24yWkxJanVHWTJrOW5Pck9zOXFaU3NQOCtPYjlrUkxkN2xiL3hldVNwR3FE?=
 =?utf-8?B?ajZoVHNHK2lmNUU5cER3Z1dSU2YrWkhSS2hDNVpBcm5QaVNTbUMxOVZ3K2J5?=
 =?utf-8?B?aUZnZjhjS1FraXhBcDRNMUl2T1dlRzkwYU9pQzlqaG9aWllsY1NGZTZKNXZa?=
 =?utf-8?B?aUJkZWhjaGhIMXJkWlVYenZPbTZZWWlRM05KSUQzNnh2Kyt2K2lVZS9sZkhx?=
 =?utf-8?B?NjhtTnhUcmJ4MzBFTUZSek04TTFIT0NhWHowVk5hRDFnbUNHdkJnUkRWZVV4?=
 =?utf-8?B?MmtmRkF5OCtnVTRiV3FUQVROWFRUWncwN1diVGZtaXJkSTdsZ21aRFJQT3Zm?=
 =?utf-8?B?VUl3eU9GeUMyd3NML2M3SGxvY1p4NzJrQWZBdllsN09RWWw5UVVBY21lb3ZC?=
 =?utf-8?B?N2V0RXdRekxyM0VSSzZ4QmV0NEwrSnpHdmxuZU9odkUzTlEwUDZNcS9menZw?=
 =?utf-8?B?RkEzNEJZaDdCQ0JOeUZTL0ovakk5eUZEOHdpdzlXRFRiamZHZXVXY2RrVmVH?=
 =?utf-8?B?dXc3REhma0h6WHlvbGQ2V0hqTGFra0laVGJCcmthZjRyVDBhaWQzYmlJV0Zy?=
 =?utf-8?B?d1VPeC9abW9Ia3l4K2dvZjR3QlN4K3BLb3VhQzJXQzk1bWZVb3lrOUM4U1Fx?=
 =?utf-8?B?TnJTdVZHZEpPcHpObTM0VjlRdEFVM1BhRCt4RXFiZXc3bWoyME1MOGl2RHU1?=
 =?utf-8?B?K09MOVRkeSt3M2pVdGFPSnF5b2tleFozdTczbGtCWVNBZWN3bmgzQlNJUnRa?=
 =?utf-8?B?QkVoYmQ0QkcwSGY1WXN3THFjZEZ5N2VEV0lCOXBlR0JwSHJkSkJzZ1k0SkFD?=
 =?utf-8?B?aGVoNERYN1o5RTVJRytwejcza3RKSHQxaWFxeW5KeEFiRWN2UmJjc2Jpakp5?=
 =?utf-8?B?ZjJ5U01wZndkd1VmRFRXelpkcTN2enM3QWhVVnBZdENJZEVjWW1IaGptcG9N?=
 =?utf-8?B?eVM2T2dsa1YxUkRrUjJ6MFpRT3ZXbDFZcG8wc2l1M05xcisvMUFYU1NLUlMw?=
 =?utf-8?B?OGtvM3RKMGEvV1A5WndjTzlXVHVId2J5dUg0eU9OcUJLNm5FTktsUTFBaklz?=
 =?utf-8?B?Vno1MFNqUDZwekJaN1F3dGNCSEFacmhkalZCRmZacUtQc2xaYnpRY3hjdkRs?=
 =?utf-8?B?TXdtNllGQWpWSmh2a3BILzY3dlFYSHR2T0g3WnQ0a1BVa2s4MEhtRmExTS9U?=
 =?utf-8?Q?UsYStyNZ?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 17694743-63b1-4c86-3f63-08d8c3e94147
X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3288.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jan 2021 00:03:06.4109
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TJnOPcRMDKgMht8J15ot0e9eZmjSttLMIQCzJo8xZopSnWpvTReWUKO9DmcYs+yh+Z79Cn6jKRPAbx4s1eHpkPtQataRBwIsYGholNrrDSI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3238
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9878 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 phishscore=0
 adultscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2101280115
X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9878 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 phishscore=0
 adultscore=0 impostorscore=0 malwarescore=0 lowpriorityscore=0 bulkscore=0
 priorityscore=1501 mlxscore=0 clxscore=1011 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000
 definitions=main-2101280115
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 1/28/21 6:52 PM, Michael D Labriola wrote:
> Hey, everyone.  I've run into problems starting up my Xen domUs as of
> the latest batch of stable updates.  Whenever I try to create one, I
> get a bunch of block device errors like this:
>
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to add device with path /local/domain/0/backend/vbd/4/51712
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to add device with path /local/domain/0/backend/vbd/4/51728
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to add device with path /local/domain/0/backend/vbd/4/51744
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to add device with path /local/domain/0/backend/vbd/4/51760
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to add device with path /local/domain/0/backend/vbd/4/51776
> libxl: error: libxl_create.c:1452:domcreate_launch_dm: Domain 4:unable to add disk devices
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to remove device with path /local/domain/0/backend/vbd/4/51712
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to remove device with path /local/domain/0/backend/vbd/4/51728
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to remove device with path /local/domain/0/backend/vbd/4/51744
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to remove device with path /local/domain/0/backend/vbd/4/51760
> libxl: error: libxl_device.c:1105:device_backend_callback: Domain 4:unable to remove device with path /local/domain/0/backend/vbd/4/51776
> libxl: error: libxl_domain.c:1290:devices_destroy_cb: Domain 4:libxl__devices_destroy failed
> libxl: error: libxl_domain.c:1177:libxl__destroy_domid: Domain 4:Non-existant domain
> libxl: error: libxl_domain.c:1131:domain_destroy_callback: Domain 4:Unable to destroy guest
> libxl: error: libxl_domain.c:1058:domain_destroy_cb: Domain 4:Destruction of domain failed
>
> I'm using Xen 4.13.1 on the box I've been testing with.
>
> I bisected down to this commit, and reverting it does indeed fix my
> problem.  Well, this commit upstream and it's cherry-picked variants
> on linux-5.4.y and linux-5.10.y.


You most likely need 5f46400f7a6a4fad635d5a79e2aa5a04a30ffea1. It hit Linus tree a few hours ago.


-boris