Received: by 2002:a25:ab43:0:0:0:0:0 with SMTP id u61csp4578095ybi; Mon, 3 Jun 2019 13:27:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqyj8iFUBWfW6+w29TrJf1eom6dn8bIZRjFgaq2PNUY8EHKkDkbFBvubkeTQL7PFJFId/I1f X-Received: by 2002:a17:90a:fa07:: with SMTP id cm7mr11372888pjb.115.1559593627013; Mon, 03 Jun 2019 13:27:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559593627; cv=none; d=google.com; s=arc-20160816; b=yEAcmZ0xn9xJBkGHi1Kd0DUBVf51W2pcE/jGSchNo/taEouOx9YXm+oCVFi1x40dwx 3kDzJcruiu4QKEn3f643KIrRVyWINkFQwdMj5YL76xSUzYrz4m91M+cfBNnRcpNuqB4H 9TF4eAkSs0L43BD9AY36RXG6ggqrcmfOHT3qdovnLYukOl/1WyadRpmQemuOVCZFtmSU 5j9CJHgDK4gkkdAcjCnybmQdkczKXrVb0DsGXAdkF1VAlBpTS+Z474Wv3zzojtIcFJm2 VHlN4Qb24PZoD9mKydRU2OG/zyU8hJwRl+NyL8sSmXpuV3zHKlA44LP/j2Xd8nItNi4h ro7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:organization:message-id:date:subject:cc:to :from; bh=bO6FmhwbiToR77cYGbF3HMre8R37UzFndDvSFZb36wY=; b=iHAMpag7RC0P19JKDu4oJLmyH9w69SXQwkwKURlEnbU9gb2DY0093t3XcyLJmIQ8Uz ikwAF+wftVBaT5ePnArINg2a7Y7Qgfv4XrbSseyqgDAp5NmF0Bu9IMZ6VYfwN/JNJsY7 1f9QC2hvRqDtIXsdVifs5uxoUW0PDHID8u8J5W/qF/WsQ1xk14QhQulPoiXj+M13w/e3 0OunTqsDSffb/xtl3h/WizEJevBMqF5V4W3BFAbvgJpzaTRc3869wMaqzaXfA4Qu8g6b 1wPFuOEb/RhsyMW6/+Z8Yn5IsrHWO2/Llf5t3g1FyOzDFqcozc7H7JqPvF5iHGba675O eAvg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c17si4369544plo.46.2019.06.03.13.26.50; Mon, 03 Jun 2019 13:27:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726824AbfFCUYr (ORCPT + 99 others); Mon, 3 Jun 2019 16:24:47 -0400 Received: from mx1.redhat.com ([209.132.183.28]:51268 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725956AbfFCUYq (ORCPT ); Mon, 3 Jun 2019 16:24:46 -0400 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 697D9306E33B; Mon, 3 Jun 2019 20:24:39 +0000 (UTC) Received: from x2.localnet (ovpn-122-112.rdu2.redhat.com [10.10.122.112]) by smtp.corp.redhat.com (Postfix) with ESMTP id DC03E19936; Mon, 3 Jun 2019 20:24:23 +0000 (UTC) From: Steve Grubb To: Paul Moore Cc: Richard Guy Briggs , Tycho Andersen , "Serge E. Hallyn" , containers@lists.linux-foundation.org, linux-api@vger.kernel.org, Linux-Audit Mailing List , linux-fsdevel@vger.kernel.org, LKML , netdev@vger.kernel.org, netfilter-devel@vger.kernel.org, omosnace@redhat.com, dhowells@redhat.com, simo@redhat.com, Eric Paris , ebiederm@xmission.com, nhorman@tuxdriver.com Subject: Re: [PATCH ghak90 V6 02/10] audit: add container id Date: Mon, 03 Jun 2019 16:24:23 -0400 Message-ID: <97478582.yP93vGJyqj@x2> Organization: Red Hat In-Reply-To: References: <20190529145742.GA8959@cisco> <20190531002058.tsddah4edcazkuzs@madcap2.tricolour.ca> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.49]); Mon, 03 Jun 2019 20:24:46 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello Paul, I am curious about this. We seemed to be close to getting this patch pulled in. A lot of people are waiting for it. Can you summarize what you think the patches need and who we think needs to do it? I'm lost. Does LXC people need to propose something? Does Richard? Someone else? Who's got the ball? Thank, -Steve On Friday, May 31, 2019 8:44:45 AM EDT Paul Moore wrote: > On Thu, May 30, 2019 at 8:21 PM Richard Guy Briggs wrote: > > On 2019-05-30 19:26, Paul Moore wrote: > > > On Thu, May 30, 2019 at 5:29 PM Tycho Andersen wrote: > > > > On Thu, May 30, 2019 at 03:29:32PM -0400, Paul Moore wrote: > > > > > [REMINDER: It is an "*audit* container ID" and not a general > > > > > "container ID" ;) Smiley aside, I'm not kidding about that part.] > > > > > > > > This sort of seems like a distinction without a difference; > > > > presumably > > > > audit is going to want to differentiate between everything that > > > > people > > > > in userspace call a container. So you'll have to support all this > > > > insanity anyway, even if it's "not a container ID". > > > > > > That's not quite right. Audit doesn't care about what a container is, > > > or is not, it also doesn't care if the "audit container ID" actually > > > matches the ID used by the container engine in userspace and I think > > > that is a very important line to draw. Audit is simply given a value > > > which it calls the "audit container ID", it ensures that the value is > > > inherited appropriately (e.g. children inherit their parent's audit > > > container ID), and it uses the value in audit records to provide some > > > additional context for log analysis. The distinction isn't limited to > > > the value itself, but also to how it is used; it is an "audit > > > container ID" and not a "container ID" because this value is > > > exclusively for use by the audit subsystem. We are very intentionally > > > not adding a generic container ID to the kernel. If the kernel does > > > ever grow a general purpose container ID we will be one of the first > > > ones in line to make use of it, but we are not going to be the ones to > > > generically add containers to the kernel. Enough people already hate > > > audit ;) > > > > > > > > I'm not interested in supporting/merging something that isn't > > > > > useful; > > > > > if this doesn't work for your use case then we need to figure out > > > > > what > > > > > would work. It sounds like nested containers are much more common > > > > > in > > > > > the lxc world, can you elaborate a bit more on this? > > > > > > > > > > As far as the possible solutions you mention above, I'm not sure I > > > > > like the per-userns audit container IDs, I'd much rather just emit > > > > > the > > > > > necessary tracking information via the audit record stream and let > > > > > the > > > > > log analysis tools figure it out. However, the bigger question is > > > > > how > > > > > to limit (re)setting the audit container ID when you are in a > > > > > non-init > > > > > userns. For reasons already mentioned, using capable() is a non > > > > > starter for everything but the initial userns, and using > > > > > ns_capable() > > > > > is equally poor as it essentially allows any userns the ability to > > > > > munge it's audit container ID (obviously not good). It appears we > > > > > need a different method for controlling access to the audit > > > > > container > > > > > ID. > > > > > > > > One option would be to make it a string, and have it be append only. > > > > That should be safe with no checks. > > > > > > > > I know there was a long thread about what type to make this thing. I > > > > think you could accomplish the append-only-ness with a u64 if you had > > > > some rule about only allowing setting lower order bits than those > > > > that > > > > are already set. With 4 bits for simplicity: > > > > > > > > 1100 # initial container id > > > > 1100 -> 1011 # not allowed > > > > 1100 -> 1101 # allowed, but now 1101 is set in stone since there are > > > > > > > > # no lower order bits left > > > > > > > > There are probably fancier ways to do it if you actually understand > > > > math :) > > > > > > ;) > > > > > > > Since userns nesting is limited to 32 levels (right now, IIRC), and > > > > you have 64 bits, this might be reasonable. You could just teach > > > > container engines to use the first say N bits for themselves, with a > > > > 1 > > > > bit for the barrier at the end. > > > > > > I like the creativity, but I worry that at some point these > > > limitations are going to be raised (limits have a funny way of doing > > > that over time) and we will be in trouble. I say "trouble" because I > > > want to be able to quickly do an audit container ID comparison and > > > we're going to pay a penalty for these larger values (we'll need this > > > when we add multiple auditd support and the requisite record routing). > > > > > > Thinking about this makes me also realize we probably need to think a > > > bit longer about audit container ID conflicts between orchestrators. > > > Right now we just take the value that is given to us by the > > > orchestrator, but if we want to allow multiple container orchestrators > > > to work without some form of cooperation in userspace (I think we have > > > to assume the orchestrators will not talk to each other) we likely > > > need to have some way to block reuse of an audit container ID. We > > > would either need to prevent the orchestrator from explicitly setting > > > an audit container ID to a currently in use value, or instead generate > > > the audit container ID in the kernel upon an event triggered by the > > > orchestrator (e.g. a write to a /proc file). I suspect we should > > > start looking at the idr code, I think we will need to make use of it. > > > > My first reaction to using the IDR code is that once an idr is given up, > > it can be reused. I suppose we request IDRs and then never give them up > > to avoid reuse... > > I'm not sure we ever what to guarantee that an audit container ID > won't be reused during the lifetime of the system, it is a discrete > integer after all. What I think we do want to guarantee is that we > won't allow an unintentional audit container ID collision between > different orchestrators; if a single orchestrator wants to reuse an > audit container ID then that is its choice. > > > I already had some ideas of preventing an existing ID from being reused, > > Cool. I only made the idr suggestion since it is used for PIDs and > solves a very similar problem. > > > but that makes the practice of some container engines injecting > > processes into existing containers difficult if not impossible. > > Yes, we'll need some provision to indicate which orchestrator > "controls" that particular audit container ID, and allow that > orchestrator to reuse that particular audit container ID (until all > those containers disappear and the audit container ID is given back to > the pool).