Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <56cd0dde-600d-1bb0-1555-e66de8c37236@intel.com>
Date:   Fri, 18 Mar 2022 16:47:58 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Subject: Re: [PATCH] ixgbe: Manual AN-37 for troublesome link partners for
 X550 SFI
Content-Language: en-US
To:     Jeff Daly <jeffd@silicom-usa.com>,
        <intel-wired-lan@lists.osuosl.org>
CC:     Jesse Brandeburg <jesse.brandeburg@intel.com>,
        "David S. Miller" <davem@davemloft.net>,
        Jakub Kicinski <kuba@kernel.org>,
        "open list:NETWORKING DRIVERS" <netdev@vger.kernel.org>,
        open list <linux-kernel@vger.kernel.org>
References: <20220316192710.9947-1-jeffd@silicom-usa.com>
From:   Tony Nguyen <anthony.l.nguyen@intel.com>
In-Reply-To: <20220316192710.9947-1-jeffd@silicom-usa.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RFlaSGozRnpSSTJEN01JbUtRTVZwSTFQSkIxcnpUWXhsYnJpbjBNUWhIOVky?=
 =?utf-8?B?ZjZ5SnA5WlVqazFIaFZVMmErY1VwQ2trUENZNU0vWGpKNndyNFJHUlRlb2V3?=
 =?utf-8?B?MG1rbEFhTS9RL0VuNlNwc2tLZFZrMDgyVlZjRTdoYldtZGRIWXlEdnhzM3RN?=
 =?utf-8?B?dlBqbmRSWTZaU210OWNyNUxWSzZLODBhdE9qRlA1YWJ6UWdHYnBoTWVxRDhV?=
 =?utf-8?B?NFRCZXZUSDRKdEIxTjVHRFF3MDR5cHJzNlRuVWZHSTl0MDJRcks1akdpdnRX?=
 =?utf-8?B?LzIzbUh6STEzWUhPeTR3eWcwN0hFMzAzeDZtQ2ovMFdaM0YwOTlJWnZ0MnJS?=
 =?utf-8?B?NllINXJyUy9kNHNpUFViSlBDNHFOTE84TXU5VjZlVThVdnlZK1pIc1o0NkdS?=
 =?utf-8?B?blhNWjZNeHJjL3ZvNlF1dG44eFBBZlI5YVdyK0NuNzZvSnBQRGZxT0lrSXRJ?=
 =?utf-8?B?TEgzTVE5WDlVdzZMN0pESTExQTdFdWtxQmlRSU9KQ2xhRXpwTEI2Z3ZocVpR?=
 =?utf-8?B?ZXhTUGtjTEtCTXNsRWR1NGFwUEZjcFRzRzBXMzBHS0NkN0wxQlBKUGJOb0Y0?=
 =?utf-8?B?WmcrcUhIWWgzRHR6QjJldGFsbm82dnk5eEdUaXJDb3BvQjk3dEc3QURBR1lQ?=
 =?utf-8?B?ci9tcHZodEJuWWF4UDRCUjBhK250NXF1YUpxQW5aS2hxQ04wTDZiK3IyMW96?=
 =?utf-8?B?MDJ6aXVWcUZreTd2QmlycGk1RVBGMnU2MHdXbE4zZGRyaGloNnZoSmxJV2dY?=
 =?utf-8?B?b0lSNEZhQ0JMallXM2ZrUWlBSndRbVhxTlRCMTRhSEhzZ3laVlpDejlmeFdD?=
 =?utf-8?B?aW1KS3o2aVhNWjhHVFkyeWdZa1FocGtUcGVGbXE2Wis1MUFvT0ZFZHpCNVdL?=
 =?utf-8?B?VExuaHo2dHdxSUJ0b0RoT2R2SkRxY21MdFR6akdiNTRvcGxaalo4MG5Wak94?=
 =?utf-8?B?Z1FhaU1Wb2xNVmxYYWtyVjJDQ3g5aSt5Z09YSHd5WGtPalNiWS9PcW5iRHhR?=
 =?utf-8?B?eXROeDN1Mys0SkFaZXR0c2N5Q0s3dWNETGJLM1lWaTB2bVdPK3VvREZDMS9y?=
 =?utf-8?B?Z0toekVrNWxEaENFYVpnbWhhUURGb1VRT1VsaFd6NUVDN0VnNXYvbTFTeXg4?=
 =?utf-8?B?ZFZETFR0U1BmYlEwNWN1V002MmZLM3R1WXdwc05TRzdBMkUwemlqZThmTk9E?=
 =?utf-8?B?dXBGemprOHJmcDdjV254ekdlNHFOTFcrR1dLVlpTUDBydjJwMTlqU0t3MzFs?=
 =?utf-8?B?UThHaE5BRmRsWUtVdjhDeVVuRVE1bzR3OFNSTyt1M1RaNWJ5aGhlUko2YnRt?=
 =?utf-8?B?SWpySU53VUk0anFQbUxIR2FHUFJlWVpYeGxLUE1sOTUwS1hTRWdxS1MyNjBW?=
 =?utf-8?B?K1cvcXJUa2R6QTJQdXpUL1BhTVo0L0lsNnQ3SlJBcy95MUdRcHJ5S1ZnOUxl?=
 =?utf-8?B?NVZNWEtQQldpYThub3llM0RqT1ZNckV2SVZ6c2tZbm1mcHlWRUtTbzRFbVRp?=
 =?utf-8?B?N2Y3K0N0b3VoNGp5VmVkWnNpRUw0U1JGN1BralZrRlZScVZkNzBSbkhqQVdV?=
 =?utf-8?B?dTRsbHQ2ZXA0RlVJWUdsMFpFRGZjK0VtMC8vV1V4SmljUFUrV0lQZGh1Lyt5?=
 =?utf-8?B?TFkwNzZkdW1QTlpVYU0vZzhPZVFSVjAwYmR6WHdqWEszUGh1NWYzL25aODd0?=
 =?utf-8?B?eUkvQVdWbDF3MVd3Nk1Wd0o2azZYOE03cW41cWtQbSs3by9sWFdlRmxSVjJ6?=
 =?utf-8?B?c2svUWNrMjBjcEJGTHB5REZFK1ZoWXpkWWNXODdIZG5iRVhQa3ZDU2VuZWc4?=
 =?utf-8?B?T0Z6VnRDM0krbFVabEJ5YnRGYnpreWZ6VWRSbVNqRTBPMWllZ09GemdLY2dX?=
 =?utf-8?B?eVpaeE05TzlOWS82cGNxUFBaSjZkMXdEbHFUUmRLdEVrOGdrNDY5eDVOKzh6?=
 =?utf-8?B?SjlqM2R0T3ZUdGVkTEVPSENLZElybHU2S3lFMEdiVnVlYmhydFd6NlJpc2Zv?=
 =?utf-8?B?UE5qRUxwOVVnPT0=?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 7a6a6141-71f5-44c1-5072-08da0939bd10
X-MS-Exchange-CrossTenant-AuthSource: SN6PR11MB3229.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Mar 2022 23:48:01.7970
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: HdNxyXwAdVvb5peRBd1bgwU4LtsT1YpLDW2pCq1IxDNbZwrR0hyQ+/TvrdyZ4CzIKwkb4/yAG05VVjX5EcoGRFPuhfIBRGOkPknJT+uHvQM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1101MB2212
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-8.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 3/16/2022 12:27 PM, Jeff Daly wrote:
> Some (Juniper MX5) SFP link partners exhibit a disinclination to
> autonegotiate with X550 configured in SFI mode.  This patch enables
> a manual AN-37 restart to work around the problem.

Hi Jeff,

I talked to the ixgbe team about this and we need a bit more time to 
look this over. Will keep you updated.

> Signed-off-by: Jeff Daly <jeffd@silicom-usa.com>
> ---
>   drivers/net/ethernet/intel/ixgbe/ixgbe_type.h |  3 ++
>   drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c | 50 +++++++++++++++++++
>   2 files changed, 53 insertions(+)
>
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
> index 2647937f7f4d..dc8a259fda5f 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
> @@ -3705,7 +3705,9 @@ struct ixgbe_info {
>   #define IXGBE_KRM_LINK_S1(P)		((P) ? 0x8200 : 0x4200)
>   #define IXGBE_KRM_LINK_CTRL_1(P)	((P) ? 0x820C : 0x420C)
>   #define IXGBE_KRM_AN_CNTL_1(P)		((P) ? 0x822C : 0x422C)
> +#define IXGBE_KRM_AN_CNTL_4(P)		((P) ? 0x8238 : 0x4238)
>   #define IXGBE_KRM_AN_CNTL_8(P)		((P) ? 0x8248 : 0x4248)
> +#define IXGBE_KRM_PCS_KX_AN(P)		((P) ? 0x9918 : 0x5918)
>   #define IXGBE_KRM_SGMII_CTRL(P)		((P) ? 0x82A0 : 0x42A0)
>   #define IXGBE_KRM_LP_BASE_PAGE_HIGH(P)	((P) ? 0x836C : 0x436C)
>   #define IXGBE_KRM_DSP_TXFFE_STATE_4(P)	((P) ? 0x8634 : 0x4634)
> @@ -3715,6 +3717,7 @@ struct ixgbe_info {
>   #define IXGBE_KRM_PMD_FLX_MASK_ST20(P)	((P) ? 0x9054 : 0x5054)
>   #define IXGBE_KRM_TX_COEFF_CTRL_1(P)	((P) ? 0x9520 : 0x5520)
>   #define IXGBE_KRM_RX_ANA_CTL(P)		((P) ? 0x9A00 : 0x5A00)
> +#define IXGBE_KRM_FLX_TMRS_CTRL_ST31(P)	((P) ? 0x9180 : 0x5180)
>   
>   #define IXGBE_KRM_PMD_FLX_MASK_ST20_SFI_10G_DA		~(0x3 << 20)
>   #define IXGBE_KRM_PMD_FLX_MASK_ST20_SFI_10G_SR		BIT(20)
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c
> index e4b50c7781ff..f48a422ae83f 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c
> @@ -1725,6 +1725,56 @@ static s32 ixgbe_setup_sfi_x550a(struct ixgbe_hw *hw, ixgbe_link_speed *speed)
>   				IXGBE_KRM_PMD_FLX_MASK_ST20(hw->bus.lan_id),
>   				IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val);
>   
> +	/* change mode enforcement rules to hybrid */
> +	status = mac->ops.read_iosf_sb_reg(hw,
> +				IXGBE_KRM_FLX_TMRS_CTRL_ST31(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, &reg_val);
> +	reg_val |= 0x0400;
> +
> +	status = mac->ops.write_iosf_sb_reg(hw,
> +				IXGBE_KRM_FLX_TMRS_CTRL_ST31(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val);

I don't see a need for all the status assignments, they're not being 
used before being overwritten.

Thanks,

Tony

> +	/* manually control the config */
> +	status = mac->ops.read_iosf_sb_reg(hw,
> +				IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, &reg_val);
> +	reg_val |= 0x20002240;
> +
> +	status = mac->ops.write_iosf_sb_reg(hw,
> +				IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val);
> +
> +	/* move the AN base page values */
> +	status = mac->ops.read_iosf_sb_reg(hw,
> +				IXGBE_KRM_PCS_KX_AN(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, &reg_val);
> +	reg_val |= 0x1;
> +
> +	status = mac->ops.write_iosf_sb_reg(hw,
> +				IXGBE_KRM_PCS_KX_AN(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val);
> +
> +	/* set the AN37 over CB mode */
> +	status = mac->ops.read_iosf_sb_reg(hw,
> +				IXGBE_KRM_AN_CNTL_4(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, &reg_val);
> +	reg_val |= 0x20000000;
> +
> +	status = mac->ops.write_iosf_sb_reg(hw,
> +				IXGBE_KRM_AN_CNTL_4(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val);
> +
> +	/* restart AN manually */
> +	status = mac->ops.read_iosf_sb_reg(hw,
> +				IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, &reg_val);
> +	reg_val |= IXGBE_KRM_LINK_CTRL_1_TETH_AN_RESTART;
> +
> +	status = mac->ops.write_iosf_sb_reg(hw,
> +				IXGBE_KRM_LINK_CTRL_1(hw->bus.lan_id),
> +				IXGBE_SB_IOSF_TARGET_KR_PHY, reg_val);
> +
>   	/* Toggle port SW reset by AN reset. */
>   	status = ixgbe_restart_an_internal_phy_x550em(hw);
>