Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752698AbdL0Wu2 (ORCPT ); Wed, 27 Dec 2017 17:50:28 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:36934 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752051AbdL0WuZ (ORCPT ); Wed, 27 Dec 2017 17:50:25 -0500 Subject: Re: [RFC PATCH bpf-next v2 4/4] error-injection: Support fault injection framework To: Masami Hiramatsu , Alexei Starovoitov References: <151427438796.32561.4235654585430455286.stgit@devbox> <151427450538.32561.2776225740675148782.stgit@devbox> <20171227021255.sahkcpgyzohl5brs@ast-mbp> <20171227170910.5ac1074bc86341f194130119@kernel.org> CC: Josef Bacik , , , , , , , , , , , Josef Bacik , Akinobu Mita From: Alexei Starovoitov Message-ID: <87e5e909-593d-7fd6-c7bb-714c1e3022a0@fb.com> Date: Wed, 27 Dec 2017 14:49:46 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20171227170910.5ac1074bc86341f194130119@kernel.org> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [2620:10d:c090:180::27b8] X-ClientProxiedBy: MWHPR12CA0065.namprd12.prod.outlook.com (2603:10b6:300:103::27) To BL2PR15MB0961.namprd15.prod.outlook.com (2603:10b6:201:15::23) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: dfb660dc-5181-4ecc-ea7e-08d54d7c24da X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(5600026)(4604075)(2017052603307)(7153060);SRVR:BL2PR15MB0961; X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0961;3:X/dCGcQaE1ZBrPA9uSYRTOPqvWs7YlRusoND1flfJ6LqkwQJKyHXAkwLGwxrhFftmxJXopUUEoxAWTBXsVn42p/CV6JC0lF7CorOK/xWyEaYmZoVIGOQ6F94SpT/P8T6FcocEpCAQFlQIHZnAIOAMSrRMJAhi8ZJ5z9xnb58uuEaWUiPwXRy4RzR16JZAXiDrjnFBdxr35hFi2yjJOX+JJdFb/aS2CbLdOWBIp8m4dE1dDR/5vZHkSTFIxZqmGRr;25:jiMK/wBkaceEde6TohQg8LnRdGA0MM5kpKJxs+TSzxzK5F6Ok7NDizGMddpYSoooYUokb6jHP2V6ZpCZT9zJG6kx7WlEzMPG3DGANATw09zeWFzEkmF9prGWekri+4dJHzT/TVMdGkhmLHYyX8ju9L4YcPNRV6kxF1wMzwq3ogNskiGsF4HpANs6n+wYyAiJSrWTlkbdg8Ot+wbBpP+BLsSC7b13wul7dKd8s7lAXtgBztxM41j2WEDod8S9yv7X+ME6E+dsrtyintpWbNudW8nvJcDrdpodHEutYF3+CMKbjjlKmO9xbEsKdPP6opT3JDGYNLrYwNQh7BsAEJ60JoQn4KiZFK0teGG3j0aIoAs=;31:tAiue/Ns0YRS9L+1uZqFBioxzy0ggjURHuGhLoWaJk641K2MHRCz9/AkYXfljNeBSsVfWcgfjiwVsNOhzg2rwjkxOUS1XZyWbi/svLg/2FMEXzKGC699o9KZemhpJxDoFs2twjOSetTX4tmMNsJl0HPmCfmgc1GYLB7C66D5xHvuAjwedRTChlTZgig4a9GAXXJh68l9lkn9rokhNhMHpbGHTUp7M4P3kxlJWlC14I8= X-MS-TrafficTypeDiagnostic: BL2PR15MB0961: X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0961;20:xRSuATZf/JOBmftYO//XE6RA0ZtsKs4RPJ8f5LELIB7sHSx4Dyl0ya3jhaB/gqFbbKaXXTZPRw5xwVJFu0qO4ZAWFHNvvMnT5DovNxyYLTvXdQRPl5FhrBP22upFw4ISEE89+DKt3cj3s2OusT1r015edz2/G1jw+96iPS2qBGAozvIEeGug7/nlwizrxrLm0SaP7hNr58OcPswZNIFV8WQ9T8SWxyBpJDgoe2KGue5Yz6hKDvTTJ8W52RS7i96rFeM1ZTNRAAJPGwNm08ynxtKGv+6Hpvnpyf2v7GU4jrMxy2WuUZgg6bplF64o9skkcZjjxNzCcwkfpYwLvy7/8AiOHCve6BEBDR4sw9dJYqn4abYXfpGuEMUy3tSshtpbj80hNKF6PQwrNDGHCn9yGrwfb2IXFPfKJcMhXi5ad6aX+xKA19Rs4mZ9khEBDUwTmkCHGXZUCYZoDKwbe2KXfgORK6758ums+ivUhXPMO70Wp7n/aYnH/RZosplYhwhv;4:0mhtaT110u/Agb0FQZw1pv3aps5/FjFcQqFa8ZYN1vUqv2SUeP6orVwHpvteLAzjkkzWejDBVrADDKhBM6Aj8z1aKjtuv+AgNbVRnKthCgPW7YiOhU0WCDsyXLXWC4/gLzDyLS9Ik0pcDHpmILxwCBCxWtSGDsedi2JCRCOHCx0DeykkP8d5eHX1v7PFbuuBAGXWBMTR6eVJ7w3kOaqHblNfI/CTtLQ/v2j3J3T/Rwt+R2ukdRsvvfos1Ak16wzp84+i/XGpK9JEFGMMj3pMLw== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(6040470)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(93001095)(3231023)(11241501184)(944501075)(6041268)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123562045)(6072148)(201708071742011);SRVR:BL2PR15MB0961;BCL:0;PCL:0;RULEID:(100000803101)(100110400095);SRVR:BL2PR15MB0961; X-Forefront-PRVS: 0534947130 X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10019020)(376002)(39380400002)(396003)(366004)(39860400002)(346002)(24454002)(189003)(199004)(2906002)(36756003)(6666003)(58126008)(54906003)(93886005)(4326008)(25786009)(110136005)(31696002)(59450400001)(386003)(86362001)(53546011)(106356001)(2950100002)(83506002)(6246003)(8936002)(39060400002)(230700001)(50466002)(97736004)(5660300001)(65956001)(65806001)(47776003)(316002)(68736007)(8676002)(81156014)(81166006)(64126003)(7416002)(478600001)(6486002)(31686004)(6116002)(105586002)(65826007)(67846002)(53936002)(23746002)(305945005)(7736002)(52396003)(1706002)(229853002)(52116002)(76176011)(217873001)(42262002);DIR:OUT;SFP:1102;SCL:1;SRVR:BL2PR15MB0961;H:[IPv6:2620:10d:c081:1132::1039];FPR:;SPF:None;PTR:InfoNoRecords;MX:1;A:1;LANG:en; X-Microsoft-Exchange-Diagnostics: =?Windows-1252?Q?1;BL2PR15MB0961;23:DY4/yK/oly6lDmnsGjNupsvNLOl8WLZjigMCV?= =?Windows-1252?Q?XQR88Kf/ZRxNeEZqD183fis2svlrec5/EjlQERpidZuIJmbEjrP2cJo8?= =?Windows-1252?Q?P+vRbrqwdISB7tn1YIslMipnihqVcRupujSvpXC2o5FxJZz/OGTJnHGe?= =?Windows-1252?Q?n59AR/BwmZHBMg2nyuxhFPYEIqvaz/92/YT8gm6MVTXE/adXdptU9tQM?= =?Windows-1252?Q?ZG4n0h1oL8KR185NNmpvlsVBsPyhQobyv2ruHzCVcZAnbHK2oZ1NwjVZ?= =?Windows-1252?Q?loo5wvYt2twyJ5ETj3pn8eArEjY5EAPl2yIwMnNoE9KDpvjLqE32vqUS?= =?Windows-1252?Q?Iff65ya5u4sq50Uw8tHIQVLwIt9lqTaD2dLJ0+XABjNxxEatr0i/fIB7?= =?Windows-1252?Q?CEcxorx1/0QXByEKNqRKVR5QWuDH1eHeCyrNGyKqlayDZMrfAYd38IgO?= =?Windows-1252?Q?k4Z1FISPi6FgV3enMJ1fuIOtNumvPiVCrKL082x0dRJrwSFKEvFViKVX?= =?Windows-1252?Q?Cco69t4yZ8oNXJVbEKoFX0h84GPz4apWlya2CenTu6Hy8awX6oB7NJJF?= =?Windows-1252?Q?YKpjADOXPAT43JCQZsgoDqQXBp9/tnCCtVd6JmrrKZOmBRp1O0zIt4Tq?= =?Windows-1252?Q?IFrEqufCgGXru3KfqbGQQHCvyH0hQOxRrUdDnCDGWUSrN+8VtfasSkBn?= =?Windows-1252?Q?VwtiPIzLak4VxTS8dAsR22vKM7gLVvFjNM+n3ObUhAdMjWEHIXw/4XVg?= =?Windows-1252?Q?0yxbOqMyPaBXPN/E1clNnueg5jXi0qhsxvGblrDL5jGJyKsTS/Oq82XW?= =?Windows-1252?Q?i1Nt1E+NN1y2VY3gMQto7YfHwFVt0DJ/cyitMtmUZzoAph4t6aY3WkVj?= =?Windows-1252?Q?wlBgrtL9zKYXQvkIn4kQ14C4hGbRodBp/H51MFIpvmblB3nHEvbtkyKe?= =?Windows-1252?Q?hCO3PpcUjsUyYWn39pOoy1AsqScdv/OstVQUD6qh0hnEV+Q+HpNWyQbx?= =?Windows-1252?Q?jfnvzAzVDr5leZ0s/+8TDAauEOiyZZxvR7KfTTu8Bbm1M6eCJIAhRWpn?= =?Windows-1252?Q?ID+cUtSdCa+VLbxBQ9YEj6q3zGzV7LBC8TD+y+As1w+e52kSreWTLFA3?= =?Windows-1252?Q?/UGX0iYBtnPKfsEB85BLZy+vt19mkZzciyV6LBvpEW2dd5jOIlid1wex?= =?Windows-1252?Q?1AO4s5KRrcrUW3YnMWozoniLnIkOVoekzxaH2wIQ09nB/PblOwpBXeKp?= =?Windows-1252?Q?iLP4yGgZcMIFRJxuVtoYElwGtXftkU4jC5u8AHVao8iprnKkbW/QxYIB?= =?Windows-1252?Q?cO5pqTekjimSPTTPsXlZLkIsL8EELcGubaWa3zVlsUqa7PfsciGXqaiF?= =?Windows-1252?Q?8itZmxc3zqQ9EpXRSDEp3yFv2IURYttndU1+8XeaAO9R06KimrMmL+x1?= =?Windows-1252?Q?BEL+jk8gSezyzyZ8/apDOAIxSRu+PVLxMv9egbI+0pdi5OC14ntO5hTT?= =?Windows-1252?Q?PwjUF5itvaGGuaN0g9B1AtkqXPhhBGuipGuuR1Uu5vMaFi5Pw=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0961;6:tkmDRzytarMmwYCafe7XwGiBxVHo2QYfTCBcXrEKkpR8ntGKFbxoJCX6a9pcRE6ys4H8cVk/HrOWlNhjfcwJvOPFWz4hGVG9YA4Kq5iCyfSxOiZ/5IWg0U6Wz5QnX3IvlE2T0xmuziQ7cOTBU4Bx7IUKfHmdWwZE9aBTEt28Hr3Wy0OYmfeytSjwvZ9w9WXsxg+EpB02k/b3J1oCENpA2rW7sO+n93G9MomZPczoGnPzLefFBH/b9LQpwycSLf3MvobgiYkZm1rTcKhETEd/C3oylEnLFi/6+G536lVrN8SxnOOe3d7amMYlHc3cFc0lqqAhxlPm+7HXjmQPT4iOs7cQTqE51J0BFcHlYC1G13U=;5:T/pjkpz8Vol6pJN8P4dn9xRc7htZYT5YMZvNW0nw7Hpl7TASI0L2FFQmTcb63CndM3Tj2M/qSwnexgXnOmaJDE/lx48r6ZgTR5wW793tbxa/VRKHa3hVczhSILE2Mv8bhgAY6KAGhDejR4czlXfc17tQZR3MruPNdWKnLgq6Hgk=;24:PEbT8vyGZBma74+tDgJhs8qXqMe6g/SGwdsowNfOKay3aJDbSyBcx6vBxBeh3hNkOFaHb6UUHD3aBEa4LfdxDSsSrRv2fNz2hr6EU28uB8Y=;7:FSkS4O6sM7v5D3Wc5ttpc8Xxq83JQ8dhGW4PeVvCIcyaViR7nKmNbBUZvR4E8aGylyJDzCViH1HItWnO5y4ncRl3fQLjnMDrCtYk9ubAHRUPDPKxIh0JsM5QAnMjzDUBHWaQsNHDBP0KxzQOVi+YL6zviBYdySKmNUXo7sk/LfsLe34wRid3kmgkQCXYo5J1d0bKbKWm8kmtBjDXRlzQGmYTmHxPBnZEkn09DtN/+2F6sZHgsj5hEuXvB3UOFm0Q SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;BL2PR15MB0961;20:BYMzwtnBb5yR1RHAaIqQCNI/5jHtHCtZ7Eu33RQ49rTwuF7V3FZIUNIzSa4QyiFHYXa+XGqFptWdDV1btNI/CNTd9XTheUSAZjiXZtrEy+qQ555lGQTye0QRDBF3mJImzf3v5eIrhzy9n5Y5QiQK7qIHPx2Gn/jTivOUvpx6tLs= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Dec 2017 22:49:51.1776 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dfb660dc-5181-4ecc-ea7e-08d54d7c24da X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL2PR15MB0961 X-OriginatorOrg: fb.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-12-27_16:,, signatures=0 X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2329 Lines: 54 On 12/27/17 12:09 AM, Masami Hiramatsu wrote: > On Tue, 26 Dec 2017 18:12:56 -0800 > Alexei Starovoitov wrote: > >> On Tue, Dec 26, 2017 at 04:48:25PM +0900, Masami Hiramatsu wrote: >>> Support in-kernel fault-injection framework via debugfs. >>> This allows you to inject a conditional error to specified >>> function using debugfs interfaces. >>> >>> Signed-off-by: Masami Hiramatsu >>> --- >>> Documentation/fault-injection/fault-injection.txt | 5 + >>> kernel/Makefile | 1 >>> kernel/fail_function.c | 169 +++++++++++++++++++++ >>> lib/Kconfig.debug | 10 + >>> 4 files changed, 185 insertions(+) >>> create mode 100644 kernel/fail_function.c >>> >>> diff --git a/Documentation/fault-injection/fault-injection.txt b/Documentation/fault-injection/fault-injection.txt >>> index 918972babcd8..6243a588dd71 100644 >>> --- a/Documentation/fault-injection/fault-injection.txt >>> +++ b/Documentation/fault-injection/fault-injection.txt >>> @@ -30,6 +30,11 @@ o fail_mmc_request >>> injects MMC data errors on devices permitted by setting >>> debugfs entries under /sys/kernel/debug/mmc0/fail_mmc_request >>> >>> +o fail_function >>> + >>> + injects error return on specific functions by setting debugfs entries >>> + under /sys/kernel/debug/fail_function. No boot option supported. >> >> I like it. >> Could you document it a bit better? > > Yes, I will do in next series. > >> In particular retval is configurable, but without an example no one >> will be able to figure out how to use it. > > Ah, right. BTW, as I pointed in the covermail, should we store the > expected error value range into the injectable list? e.g. > > ALLOW_ERROR_INJECTION(open_ctree, -1, -MAX_ERRNO) > > And provide APIs to check/get it. I'm afraid such check would be too costly. Right now we have only two functions marked but I expect hundreds more will be added in the near future as soon as developers realize the potential of such error injection. All of ALLOW_ERROR_INJECTION marks add 8 byte overhead each to .data. Multiple by 1k and we have 8k of data spent on marks. If we add max/min range marks that doubles it for very little use. I think marking function only is enough.