Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Message-ID: <2c8e61de-54da-44da-3a7b-b95eabfb29f2@ddn.com>
Date:   Fri, 22 Apr 2022 17:20:11 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Subject: Re: [PATCH 1/1] FUSE: Allow parallel direct writes on the same file
Content-Language: en-US
To:     Miklos Szeredi <miklos@szeredi.hu>,
        Dharmendra Hans <dharamhans87@gmail.com>
Cc:     linux-fsdevel@vger.kernel.org,
        fuse-devel <fuse-devel@lists.sourceforge.net>,
        linux-kernel@vger.kernel.org, Dharmendra Singh <dsingh@ddn.com>
References: <20220408061809.12324-1-dharamhans87@gmail.com>
 <20220408061809.12324-2-dharamhans87@gmail.com>
 <CAJfpegvU+y+WRhWrgWfc_7O5BdVi=N+3RUuVdBFFoyYVr9MDeg@mail.gmail.com>
 <CACUYsyGiNgbyoxWWdXm0z73B7QfnPGU2gYanDNSQqmq5_rnrhw@mail.gmail.com>
 <CAJfpegsZF4D-sshMK0C=jSECskyQRAgA_1hKD9ytsHKvmXoBeA@mail.gmail.com>
From:   Bernd Schubert <bschubert@ddn.com>
In-Reply-To: <CAJfpegsZF4D-sshMK0C=jSECskyQRAgA_1hKD9ytsHKvmXoBeA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RTNBMk1HMUJFUURjZDkvTWt1VkFIME1SRlFFZjB0a0pYSXZpaEdpUWUwbHpa?=
 =?utf-8?B?OVhkdW9BQXhXcVdnSzdUaGtGVDNsTFVHNEtuZTR2VXgxTXJPWjdUNzBVKzBl?=
 =?utf-8?B?dUd0RXFjL3Q5T1d5RG1tOVp4NExWeGtud1JhUkRKTUVhWDFPWmV4R2RzcUZS?=
 =?utf-8?B?cnYwMEwyaHlGQ1BvLzJLKzIxN1hvbnFwTlpyZzJHYkVPdVhoS1ltaEU0TUxq?=
 =?utf-8?B?b3lLQ3RmWVFUNHFteWRxMDcrYm80VW5nelh2S2xYR1VObmxoeC9VcEVlZmpC?=
 =?utf-8?B?UGQ0dXRNTjFTSHZqdXRFTDQwUTZHN2pkeGRhUFBiYSthczBhK1ltb3I0ck9G?=
 =?utf-8?B?ZEZ3U2pMbDFQbFlWVWsrZUJvVmRRTnpHRjdGVFhjU0dZM1Jpazk2dG5lcXBw?=
 =?utf-8?B?OWw2NWMxWUp2OWt2QWJxWnBnQ3UrWTE3UTExcnN1cUttVThFTy9hS01pTWxs?=
 =?utf-8?B?V0lSdFJuKy9sdFh3TVJ0aW9KZlkxTWtRT2dpMEhTdFlsZXh4akRydjdwQ2hD?=
 =?utf-8?B?VFZ0OUp0VXZFQWhlQlAvUXB5M2kzeW1mb3VDZnJKSk9hSWVKQTkyUmp3MzN5?=
 =?utf-8?B?R1FDRWNLSkRHY2pIS0c3dExWM1Rwa3QyWkVnTDN4UTdsbWhKOURSRnN4VGRH?=
 =?utf-8?B?N1dhRjNqN1F0WmZGNllQNFJSTzlqTFJMb04zNzh5VUxWWUxwcklDUVlBcmZL?=
 =?utf-8?B?QXRIKzcxa2thVldrNDVLc1VjcUlGcnU1U21lMVpUU3RLRUJkNWhjN2ZLMlVm?=
 =?utf-8?B?OEhQSTl4bEErU3RzYmRDZFZIblNtTmd0UElsZndkUklaSnVLVVJSOWdpUG1U?=
 =?utf-8?B?VXRxT0tEUVVaR0lXeWJLdmtFd0t6T0d2R2E2YWsxT0xBQWdHaGw3a1FFcmFz?=
 =?utf-8?B?cXd3OVNOdm5PNHJ1bkN4Z3c1RFZCRGdGWkxaQTdaVnJvK2x0cEJZcEhMK3lZ?=
 =?utf-8?B?TCtXdTd2OVdWY2pYVWQ1bHFZN2x2RzdwU3JEL042aTY0K2RqYlBoQTBES21W?=
 =?utf-8?B?N0VEOG9KTm91WXBNNzRHajNWMzJVbjBqdDNWelF4d1ltcUEvaElma3V2Ylor?=
 =?utf-8?B?QWpTVnZjVms0T1gvSExWRnRNZHJLWmxxL3Y3Y2llOTlKRDlNS0lKSzM1Yldp?=
 =?utf-8?B?Q2lBaDJxaHQrOWIwb0hlTWFiSXczTDNyZjJITndRcGprNTRHd3dDR1ROdTcz?=
 =?utf-8?B?bXlra0VSUkVUUzBCZ2l2TGRwYnVOYWNDM1dtRlhpTml4cDMyS1BrQnZ4azJp?=
 =?utf-8?B?c1l2T0U4anU0SUx6STVQYnBxcXVkWUhoSDhMVUFZVlBjU2VOSEFSK2lvYWNy?=
 =?utf-8?B?V0k3RmpBeGtRenIrSUZzbHNuWjJad1RmTjJsQncwczcrMWQyek0vMXd1TGNw?=
 =?utf-8?B?L1RWL3ArK09sZno3N251L3V1UDc0VFJwYnBKK3JqeGpmdTFSY0xWQm1uV1dn?=
 =?utf-8?B?UmNQK1dzM1U4QkRFOVo3VE5tQzlvZGZXb09DZTBKZDMrQ0Q4Y3Fjem00MTRV?=
 =?utf-8?B?TEdLc0tmcGhzQjdWMGxFME04SnNLYTJUWGlJTEhraGRFTUdYcS9lWnF1b1Zp?=
 =?utf-8?B?QlQ4T3FZSjNXa0J6NEI0YWhGcUc5UXB3S0hXSEpiS3FtVnpYZCswMXdoa2VO?=
 =?utf-8?B?blF0ck1qZWZiVnFuSGZ2N2FpdUJqdUtPbHZVendqRlpsbXlyWkZKdS9yL29L?=
 =?utf-8?B?dUt4N0FlOEUrTlNBaTB2amdLT3dOU3dCSGZXVDluV0QzRGZmc0l3bkwwMjg3?=
 =?utf-8?B?NVNLL3dUQVRucjlKT3I1V0ZpNUZ4NUlib0pDWFdJQ0lRckluTXFCMkFGMzkz?=
 =?utf-8?B?QUEzamxKQ0haZmdQR08wUWFOQlVuanRGb1h4WFlOVy94bExtZzNMdnNkODMy?=
 =?utf-8?B?RlZXazB4VjhPYjdyTHBQTWllVnpXUVBpTTZ4RlRtSEN1MXRQNWhyclJTMGVr?=
 =?utf-8?B?ZEN1MHJGTHl4a2pRWFRVVi90R1NtSXE5OS96OHZtQzN3MU0yR2Rta29hdmRs?=
 =?utf-8?B?Q1huSmV2ZHR2WDkreXRjUTFsZm05WFVybFhYaG5Dd1BZak53dnlCMDRXKzkx?=
 =?utf-8?B?VDEwdU1iNDkzZ0tzQjQxMWo1dUFMWUV6Ri9Ndkl4ZFRPM0o2OE8yNEl6NW5I?=
 =?utf-8?B?ZkNzUnMydUkzL1MxV0ZLWU93bjBxa0JhTGh2QnVPWWFuR2ZMTEFyTEcwbmdt?=
 =?utf-8?B?V2dvcEZsZS9WczVzSmFJY0k1RDNVeXpTQ2tWVkhBbXhMYmFCYlpWWHV4R3Zv?=
 =?utf-8?B?d2FWSnBYYlNaL1JjS0MvM2J3N0tla29NejhlSDlpbGY5NkxteTlzSlBxY2Vs?=
 =?utf-8?B?T0RkSlFZVGVYMmdvQUJ5TDFRc3paQlRFOUNELy9sM0hHMjZxYlZ2dz09?=
X-OriginatorOrg: ddn.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a5d415c1-6d5d-46b6-415c-08da24739b60
X-MS-Exchange-CrossTenant-AuthSource: DM5PR1901MB2037.namprd19.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 15:20:17.7336
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 753b6e26-6fd3-43e6-8248-3f1735d59bb4
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: U+lvrLdWJJyRKSaiHavLHuLecPFOF4cNmyp4FegKbfM7j5Wv5RLWbPfWhh+pq6cjsCQHcEhO2RVzPam+IUSn1Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR19MB5799
X-Spam-Status: No, score=-3.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 4/22/22 16:48, Miklos Szeredi wrote:
> On Fri, 22 Apr 2022 at 16:30, Dharmendra Hans <dharamhans87@gmail.com> wrote:
>>
>> On Thu, Apr 21, 2022 at 8:52 PM Miklos Szeredi <miklos@szeredi.hu> wrote:
>>>
>>> On Fri, 8 Apr 2022 at 08:18, Dharmendra Singh <dharamhans87@gmail.com> wrote:
>>>>
> 
> That's true, but I still worry...  Does your workload include
> non-append extending writes?  Seems to me making those run in parallel
> is asking for trouble.

Our main use case is MPIIO for now and I don't think it first sets the 
file size and would then write to these sparse files. Fixing all the 
different MPI implementations including closed source stacks is probably 
out of question.
Given that MPIIO also supports direct calls into its stack we also do 
support that for some MPIs, but not all stacks. Direct calls bypassing 
the vfs also haas  it's own issues, including security. So it would be 
really great if we could find a way to avoid the inode lock.

Would you mind to share what you worry about in detail?


> 
>> If we agreed, I  would be sending the updated patch shortly.
>> (Also please take a look on other patches raised by me for atomic-open,  these
>>   patches are pending since couple of weeks)
> 
> I'm looking at that currently.

Thank you! There are two more optimizations in the same area, but these 
require VFS changes - let's first get the 'easy' things done...


Thanks,
Bernd