2023-10-31 13:13:51

by Zhuohao Bai

Subject: [PATCH v1 0/2] _rpc_dtablesize: decrease to fix excessive memory usage

In the client code, the function _rpc_dtablesize() is used to determine the
memory allocation for the __svc_xports array.

However, some operating systems (including the recent Manjaro OS) can have
_SC_OPEN_MAX values as high as 1073741816, which can cause the __svc_xports
array to become too large. This results in the process being killed.

There is a limit to the maximum number of files. To avoid this problem, a
possible solution is to set the size to the lesser of 1024 and this value to
ensure that the array space for open files is not too large, thus preventing
the process from terminating.

Also discovered that some users have taken action on the issue. It is necessary
to address this for all users. Ultimately, we determined that adjusting the
size value of _rpc_dtablesize() and streamlining the existing user code would
be the most effective solution.


---
Changes in v1:
Clean up the existing code in user

---
Links:
RFC:https://lore.kernel.org/linux-nfs/[email protected]/T/#u


Zhuohao Bai (2):
_rpc_dtablesize: Decrease the value of size.
_rpc_dtablesize: Cleaning up the existing code

src/rpc_dtablesize.c | 2 ++
src/svc.c | 2 --
2 files changed, 2 insertions(+), 2 deletions(-)

--
2.25.1

2023-10-31 13:14:00

by Zhuohao Bai

Subject: [PATCH v1 1/2] _rpc_dtablesize: Decrease the value of size.

To fix the bug caused by a Size value that is too large and leads to an array
that requires excessive memory, which subsequently results in the failure of
rpcbind to start properly.

Signed-off-by: Zhuohao Bai <[email protected]>
---
src/rpc_dtablesize.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/rpc_dtablesize.c b/src/rpc_dtablesize.c
index bce97e8..12f80c1 100644
--- a/src/rpc_dtablesize.c
+++ b/src/rpc_dtablesize.c
@@ -41,7 +41,7 @@ _rpc_dtablesize(void)
static int size;

if (size == 0) {
- size = sysconf(_SC_OPEN_MAX);
+ size = min(1024, sysconf(_SC_OPEN_MAX));
}
return (size);
}
--
2.25.1

2023-10-31 13:14:08

by Zhuohao Bai

Subject: [PATCH v1 2/2] _rpc_dtablesize: Cleaning up the existing code

Some users have already taken steps to address this issue. To prevent
duplication, we need to remove the modified code and modify the rpc_dtablesize
function.

Signed-off-by: Zhuohao Bai <[email protected]>
---
src/rpc_dtablesize.c | 4 +++-
src/svc.c | 2 --
2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/src/rpc_dtablesize.c b/src/rpc_dtablesize.c
index 12f80c1..e88698f 100644
--- a/src/rpc_dtablesize.c
+++ b/src/rpc_dtablesize.c
@@ -41,7 +41,9 @@ _rpc_dtablesize(void)
static int size;

if (size == 0) {
- size = min(1024, sysconf(_SC_OPEN_MAX));
+ size = sysconf(_SC_OPEN_MAX);
+ if (size > FD_SETSIZE)
+ size = FD_SETSIZE;
}
return (size);
}
diff --git a/src/svc.c b/src/svc.c
index 3a8709f..9b932a5 100644
--- a/src/svc.c
+++ b/src/svc.c
@@ -657,8 +657,6 @@ svc_getreqset (readfds)
assert (readfds != NULL);

setsize = _rpc_dtablesize ();
- if (setsize > FD_SETSIZE)
- setsize = FD_SETSIZE;
maskp = readfds->fds_bits;
for (sock = 0; sock < setsize; sock += NFDBITS)
{
--
2.25.1

2024-01-05 16:54:15

by Steve Dickson

Subject: Re: [PATCH v1 0/2] _rpc_dtablesize: decrease to fix excessive memory usage

On 10/31/23 9:13 AM, Zhuohao Bai wrote:
> In the client code, the function _rpc_dtablesize() is used to determine the
> memory allocation for the __svc_xports array.
>
> However, some operating systems (including the recent Manjaro OS) can have
> _SC_OPEN_MAX values as high as 1073741816, which can cause the __svc_xports
> array to become too large. This results in the process being killed.
>
> There is a limit to the maximum number of files. To avoid this problem, a
> possible solution is to set the size to the lesser of 1024 and this value to
> ensure that the array space for open files is not too large, thus preventing
> the process from terminating.
>
> Also discovered that some users have taken action on the issue. It is necessary
> to address this for all users. Ultimately, we determined that adjusting the
> size value of _rpc_dtablesize() and streamlining the existing user code would
> be the most effective solution.
>
>
> ---
> Changes in v1:
> Clean up the existing code in user
>
> ---
> Links:
> RFC:https://lore.kernel.org/linux-nfs/[email protected]/T/#u
>
>
> Zhuohao Bai (2):
> _rpc_dtablesize: Decrease the value of size.
> _rpc_dtablesize: Cleaning up the existing code
>
> src/rpc_dtablesize.c | 2 ++
> src/svc.c | 2 --
> 2 files changed, 2 insertions(+), 2 deletions(-)
>
Both Committed... (tag: libtirpc-1-3-5-rc2)

steved.