Received: by 2002:a05:6a10:1d13:0:0:0:0 with SMTP id pp19csp268817pxb; Wed, 25 Aug 2021 02:47:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwe/v9HK2v+mg793+IC7leHi6PJwKjAg5VWehiqP+hNkyTMiold5H4ClMTJukbGx4wb+I3N X-Received: by 2002:a17:906:c443:: with SMTP id ck3mr43974291ejb.195.1629884874253; Wed, 25 Aug 2021 02:47:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1629884874; cv=none; d=google.com; s=arc-20160816; b=YGDO1gNSKhePOZfE7ywK30OfomvVw+atRNvnMeyj8wGgLgsKOYAiOxtuZtFTHfzJNn ZT74VOYZG2gSEstrzX2F5hGsrVvOvXpmZCPydiQy3KOGOCh2ERn+eCcA6qhcaXUze/qL qC5PEq152tPpFgGsQVg+o8AkXWA2FX3sDmM8GQVnPXCHj7kR/ny70l9K8q+MTGjWQxmL bLt1CvUkYDCGPslN/7cIN4t6nVtbB/XfzZP5F0hzLYuqyEB+TCjCWH2DzdNmUxMHSOOT 616F+vISaVYGkWGsd3/oFO+bf1aaIvOd2JKprmoQmU+/amMgipzwbR2Bu6zxJ/uWlNqv QBkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:dkim-signature; bh=VDB3aoy5vP8AQ/06F9afgv21i9+28hAv1v7LqW0YHDY=; b=jk1G2owG148iUflyw4YTGKXk2qrcNlFRgHgKuljA4LiQgmhHJKNSNlObohx+/xZtzh LILQRv/kaer6y4iTA3PHfrnd2ZU7EBArn3FM52tq1AQUavPmYyi5+GbnZ6d7Wxj5PKws yL5yw6ZXjNOgDRD4+Vya2fbVvkRyWEevmRQtzYI7gCLP7dt59avO6tjHB0BGNBFlnIgg ypVVglVncUMAiThYqNoHlubCt1kQqhm59PCj3IY284lx4Lwch1uQV/DKZyFhDDFEIZQ1 HUrJ1jBZRWan7rxn2O8izeJG5q5Tms3Dl3PA54s628vfEV+azyWptHENGCBpOyp6hfBp 0H4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=SQGXVvBi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id x24si19101902ejv.514.2021.08.25.02.47.31; Wed, 25 Aug 2021 02:47:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=SQGXVvBi; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237091AbhHYJoc (ORCPT + 99 others); Wed, 25 Aug 2021 05:44:32 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43552 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235088AbhHYJob (ORCPT ); Wed, 25 Aug 2021 05:44:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1629884625; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VDB3aoy5vP8AQ/06F9afgv21i9+28hAv1v7LqW0YHDY=; b=SQGXVvBiB99lQ2dNono6OgbtvEbwzcJ0wrEMHEV2OsKNTyHrRXuR/YAmI56RUNBWOxLkLt 4JSPrKb8s1j3+EyUpa3wJHSj7Rkk0KgTkKyLhRPHDJP7P5Jf02I0pvbem+Ss7EU2JHsQbQ 4FQgct5rTt0q03ISCGFRLA8S2PHuL2g= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-534-yaWgLdI-PUa6YrHjakcE2w-1; Wed, 25 Aug 2021 05:43:44 -0400 X-MC-Unique: yaWgLdI-PUa6YrHjakcE2w-1 Received: by mail-wr1-f70.google.com with SMTP id b7-20020a5d4d87000000b001575d1053d2so1602172wru.23 for ; Wed, 25 Aug 2021 02:43:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=VDB3aoy5vP8AQ/06F9afgv21i9+28hAv1v7LqW0YHDY=; b=MDK7sVrJGYRxThs+cS8Cyylduu+9Q/D4KhVhdqqFDItQl/jXmiNQpg2z9t3xj4yH48 lLc3fl/McT/S4Dlk70eazpLdZ/z6AChpbWWIxTYtRhOdJz17f6R588w5pZukMNgNqv+u 2WlVqelThM+5mvbkjTYc/jxe68E5vOl9BCnpHrVBqtDJxYM3rUGIVBopiElct+hp0T0+ ze2uw7SdDNNbYd2xLkRHb1GsTTNibqzx4FxSkemI7u1Q8hOK9EQjn1Zkzl4q9djwdz25 /tzUTVuWEaHcUTyBgPPOtqr4X3/WVzHaRne2zPt/0vJC7aK6ir6mxVCavZZbd7xLMZo4 FysA== X-Gm-Message-State: AOAM5330LLd1312F11xEp8Rl/mRS2P2UnBjL6h3fKCVc1cDUHFh/670N 2m1M5KISnPwcl58pyA45Spj1RXdpiclylKKjsO5rhGomBgquAUXhAXLRf7dQ6wuZ3bFKRNdvY/u fWzPiGb1FOjDR0+bJZYlRA/lh X-Received: by 2002:a7b:c351:: with SMTP id l17mr8464205wmj.120.1629884623224; Wed, 25 Aug 2021 02:43:43 -0700 (PDT) X-Received: by 2002:a7b:c351:: with SMTP id l17mr8464193wmj.120.1629884622944; Wed, 25 Aug 2021 02:43:42 -0700 (PDT) Received: from vitty.brq.redhat.com (g-server-2.ign.cz. [91.219.240.2]) by smtp.gmail.com with ESMTPSA id g76sm4735096wme.16.2021.08.25.02.43.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Aug 2021 02:43:42 -0700 (PDT) From: Vitaly Kuznetsov To: Maxim Levitsky Cc: kvm@vger.kernel.org, Paolo Bonzini , Sean Christopherson , Wanpeng Li , Jim Mattson , "Dr. David Alan Gilbert" , Nitesh Narayan Lal , linux-kernel@vger.kernel.org, Eduardo Habkost Subject: Re: [PATCH v2 4/4] KVM: x86: Fix stack-out-of-bounds memory access from ioapic_write_indirect() In-Reply-To: <36b6656637d1e6aaa2ab5098f7ebc27644466294.camel@redhat.com> References: <20210823143028.649818-1-vkuznets@redhat.com> <20210823143028.649818-5-vkuznets@redhat.com> <20210823185841.ov7ejn2thwebcwqk@habkost.net> <87mtp7jowv.fsf@vitty.brq.redhat.com> <87k0kakip9.fsf@vitty.brq.redhat.com> <2df0b6d18115fb7f2701587b7937d8ddae38e36a.camel@redhat.com> <87h7fej5ov.fsf@vitty.brq.redhat.com> <36b6656637d1e6aaa2ab5098f7ebc27644466294.camel@redhat.com> Date: Wed, 25 Aug 2021 11:43:41 +0200 Message-ID: <87bl5lkgfm.fsf@vitty.brq.redhat.com> MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Maxim Levitsky writes: > On Wed, 2021-08-25 at 10:21 +0200, Vitaly Kuznetsov wrote: >> Maxim Levitsky writes: >> >> > On Tue, 2021-08-24 at 16:42 +0200, Vitaly Kuznetsov wrote: >> ... >> > Not a classical review but, >> > I did some digital archaeology with this one, trying to understand what is going on: >> > >> > >> > I think that 16 bit vcpu bitmap is due to the fact that IOAPIC spec states that >> > it can address up to 16 cpus in physical destination mode. >> > >> > In logical destination mode, assuming flat addressing and that logical id = 1 << physical id >> > which KVM hardcodes, it is also only possible to address 8 CPUs. >> > >> > However(!) in flat cluster mode, the logical apic id is split in two. >> > We have 16 clusters and each have 4 CPUs, so it is possible to address 64 CPUs, >> > and unlike the logical ID, the KVM does honour cluster ID, >> > thus one can stick say cluster ID 0 to any vCPU. >> > >> > >> > Let's look at ioapic_write_indirect. >> > It does: >> > >> > -> bitmap_zero(&vcpu_bitmap, 16); >> > -> kvm_bitmap_or_dest_vcpus(ioapic->kvm, &irq, &vcpu_bitmap); >> > -> kvm_make_scan_ioapic_request_mask(ioapic->kvm, &vcpu_bitmap); // use of the above bitmap >> > >> > >> > When we call kvm_bitmap_or_dest_vcpus, we can already overflow the bitmap, >> > since we pass all 8 bit of the destination even when it is physical. >> > >> > >> > Lets examine the kvm_bitmap_or_dest_vcpus: >> > >> > -> It calls the kvm_apic_map_get_dest_lapic which >> > >> > -> for physical destinations, it just sets the bitmap, which can overflow >> > if we pass it 8 bit destination (which basically includes reserved bits + 4 bit destination). >> > >> > >> > -> For logical apic ID, it seems to truncate the result to 16 bit, which isn't correct as I explained >> > above, but should not overflow the result. >> > >> > >> > -> If call to kvm_apic_map_get_dest_lapic fails, it goes over all vcpus and tries to match the destination >> > This can overflow as well. >> > >> > >> > I also don't like that ioapic_write_indirect calls the kvm_bitmap_or_dest_vcpus twice, >> > and second time with 'old_dest_id' >> > >> > I am not 100% sure why old_dest_id/old_dest_mode are needed as I don't see anything in the >> > function changing them. >> > I think only the guest can change them, so maybe the code deals with the guest changing them >> > while the code is running from a different vcpu? >> > >> > The commit that introduced this code is 7ee30bc132c683d06a6d9e360e39e483e3990708 >> > Nitesh Narayan Lal, maybe you remember something about it? >> > >> >> Before posting this patch I've contacted Nitesh privately, he's >> currently on vacation but will take a look when he gets back. >> >> > Also I worry a lot about other callers of kvm_apic_map_get_dest_lapic >> > >> > It is also called from kvm_irq_delivery_to_apic_fast, and from kvm_intr_is_single_vcpu_fast >> > and both seem to also use 'unsigned long' for bitmap, and then only use 16 bits of it. >> > >> > I haven't dug into them, but these don't seem to be IOAPIC related and I think >> > can overwrite the stack as well. >> >> I'm no expert in this code but when writing the patch I somehow >> convinced myself that a single unsigned long is always enough. I think >> that for cluster mode 'bitmap' needs 64-bits (and it is *not* a >> vcpu_bitmap, we need to convert). I may be completely wrong of course >> but in any case this is a different issue. In ioapic_write_indirect() we >> have 'vcpu_bitmap' which should certainly be longer than 64 bits. > > > This code which I mentioned in 'other callers' as far as I see is not IOAPIC related. > For regular local APIC all bets are off, any vCPU and apic ID are possible > (xapic I think limits apic id to 255 but x2apic doesn't). > > I strongly suspect that this code can overflow as well. I've probably missed something but I don't see how kvm_apic_map_get_dest_lapic() can set bits above 64 in 'bitmap'. If it can, then we have a problem indeed. -- Vitaly