Received-SPF: pass (google.com: domain of linux-kernel+bounces-52673-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Resent-From: Andy Shevchenko <andriy.shevchenko@intel.com>
Resent-Date: Mon, 5 Feb 2024 14:54:15 +0200
Resent-Message-ID: <ZcDad7MYTNp2GaAS@smile.fi.intel.com>
Resent-To: howardyen@google.com, hch@lst.de, m.szyprowski@samsung.com,
	robin.murphy@arm.com, gregkh@linuxfoundation.org, rafael@kernel.org,
	linux-kernel@vger.kernel.org
Date: Mon, 5 Feb 2024 14:08:00 +0200
From: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
To: Howard Yen <howardyen@google.com>
Cc: hch@lst.de, m.szyprowski@samsung.com, robin.murphy@arm.com,
	gregkh@linuxfoundation.org, rafael@kernel.org, broonie@kernel.org,
	james@equiv.tech, james.clark@arm.com, masahiroy@kernel.org,
	linux-kernel@vger.kernel.org, iommu@lists.linux.dev
Subject: Re: [PATCH v2] dma-coherent: add support for multi coherent rmems
 per dev
Message-ID: <ZcDPn5eLvMwpb7N1@smile.fi.intel.com>
References: <20240205072312.2342188-1-howardyen@google.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240205072312.2342188-1-howardyen@google.com>
Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo

On Mon, Feb 05, 2024 at 07:23:00AM +0000, Howard Yen wrote:
> Add support for multiple coherent rmems per device. This patch replaces
> original dma_mem with dma_mems list in device structure to store multiple
> rmems.
> 
> These multiple rmems can be assigned to the device one by one by
> of_reserved_mem_device_init_by_idx() with the memory-region
> declaration in device tree as below and store the rmem to the dma_mems
> list.
> 
> 	device1@0 {
> 		...
> 		memory-region = <&reserved_mem0>, <&reserved_mem1>;
> 		...
> 	};
> 
> When driver tries to allocate memory from the rmems, looks for the first
> available rmem and allocates the memory from this rmem.
> 
> Then if driver removed, of_reserved_mem_device_release() needs to be
> invoked to release all the rmems assigned to the device.

..

> --- a/kernel/dma/coherent.c
> +++ b/kernel/dma/coherent.c
> @@ -18,15 +18,9 @@ struct dma_coherent_mem {
>  	unsigned long	*bitmap;
>  	spinlock_t	spinlock;
>  	bool		use_dev_dma_pfn_offset;
> +	struct list_head	node;

Have you run `pahole`? Here I see wasted bytes for nothing.

>  };

..

>  void dma_release_coherent_memory(struct device *dev)
>  {
> +	struct dma_coherent_mem *mem_tmp, *q;
> +
>  	if (dev) {

While at it, perhaps

	if (!dev)
		return;

?

> -		_dma_release_coherent_memory(dev->dma_mem);
> -		dev->dma_mem = NULL;
> +		list_for_each_entry_safe(mem_tmp, q, &dev->dma_mems, node) {
> +			list_del_init(&mem_tmp->node);
> +			_dma_release_coherent_memory(mem_tmp);
> +		}
>  	}
>  }

..

>  int dma_release_from_dev_coherent(struct device *dev, int order, void *vaddr)
>  {
> -	struct dma_coherent_mem *mem = dev_get_coherent_memory(dev);
> +	struct dma_coherent_mem *mem_tmp;
> +	int ret = 0;

'ret' (1)

> -	return __dma_release_from_coherent(mem, order, vaddr);
> +	list_for_each_entry(mem_tmp, &dev->dma_mems, node) {
> +		ret = __dma_release_from_coherent(mem_tmp, order, vaddr);
> +		if (ret == 1)
> +			break;
> +	}
> +
> +	return ret;
>  }

..

>  int dma_mmap_from_dev_coherent(struct device *dev, struct vm_area_struct *vma,
>  			   void *vaddr, size_t size, int *ret)
>  {
> -	struct dma_coherent_mem *mem = dev_get_coherent_memory(dev);
> +	struct dma_coherent_mem *mem_tmp;
> +	int retval = 0;

'retval' (2)

Can we be consistent, please? (See (1) and (2) above.)

> +	list_for_each_entry(mem_tmp, &dev->dma_mems, node) {
> +		retval = __dma_mmap_from_coherent(mem_tmp, vma, vaddr, size, ret);
> +		if (retval == 1)
> +			break;
> +	}
>  
> -	return __dma_mmap_from_coherent(mem, vma, vaddr, size, ret);
> +	return retval;
>  }

..

>  static void rmem_dma_device_release(struct reserved_mem *rmem,
>  				    struct device *dev)
>  {
> -	if (dev)
> -		dev->dma_mem = NULL;
> +	struct dma_coherent_mem *mem_tmp, *q;

> +	if (dev) {

As per above, esp. taking into account that you touch this line. With proposed
modification you won't need to.

> +		list_for_each_entry_safe(mem_tmp, q, &dev->dma_mems, node) {
> +			if (mem_tmp == rmem->priv) {
> +				list_del_init(&mem_tmp->node);
> +				break;
> +			}
> +		}
> +	}
>  }

Better question, do we really need the dev check (at least in static functions)
or it can be ommitted?

-- 
With Best Regards,
Andy Shevchenko