Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <0f97539a-439f-d584-9ba3-f4bd5a302bc0@kernel.org>
Date:   Wed, 9 Mar 2022 19:57:03 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.6.2
Subject: Re: IPv4 saddr do not match with selected output device in double
 default gateways scene
Content-Language: en-US
To:     "Ziyang Xuan (William)" <william.xuanziyang@huawei.com>,
        David Miller <davem@davemloft.net>, yoshfuji@linux-ipv6.org,
        Jakub Kicinski <kuba@kernel.org>,
        netdev <netdev@vger.kernel.org>, ja@ssi.bg
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
References: <58c15089-f1c7-675e-db4b-b6dfdad4b497@huawei.com>
From:   David Ahern <dsahern@kernel.org>
In-Reply-To: <58c15089-f1c7-675e-db4b-b6dfdad4b497@huawei.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 3/7/22 11:41 PM, Ziyang Xuan (William) wrote:
> Create VLAN devices and add default gateways with following commands:
> 
> # ip link add link eth2 dev eth2.71 type vlan id 71
> # ip link add link eth2 dev eth2.72 type vlan id 72
> # ip addr add 192.168.71.41/24 dev eth2.71
> # ip addr add 192.168.72.41/24 dev eth2.72
> # ip link set eth2.71 up
> # ip link set eth2.72 up
> # route add -net default gw 192.168.71.1 dev eth2.71
> # route add -net default gw 192.168.72.1 dev eth2.72
> 

...

> We can find that IPv4 saddr "192.168.72.41" do not match with selected VLAN device "eth2.71".
> 
> I tracked the related processes, and found that user space program uses connect() firstly, then sends UDP packet.
> 

...

> Deep tracking, it because fa->fa_default has changed in fib_select_default() after first __ip_route_output_key() process,
> and a new fib_nh is selected in fib_select_default() within the second __ip_route_output_key() process but not update flowi4.
> So the phenomenon described at the beginning happens.
> 
> Does it a kernel bug or a user problem? If it is a kernel bug, is there any good solution?

That is a known problem with multipath routes.