2021-02-22 09:35:16

by Michael J. Baars

[permalink] [raw]
Subject: problems with memory allocation and the alignment check

Hi,

I just wrote this little program to demonstrate a possible flaw in both malloc and calloc.

If I allocate a the simplest memory region from main(), one out of three optimization flags fail.
If I allocate the same region from a function, three out of three optimization flags fail.

Does someone know if this really is a flaw, and if so, is it a gcc or a kernel flaw?

Regards,
Mischa.


Attachments:
main.c (862.00 B)
makefile (102.00 B)
Download all attachments

2021-02-22 09:44:54

by Andrew Pinski

[permalink] [raw]
Subject: Re: problems with memory allocation and the alignment check

On Mon, Feb 22, 2021 at 1:37 AM Michael J. Baars
<[email protected]> wrote:
>
> On Mon, 2021-02-22 at 01:29 -0800, Andrew Pinski wrote:
> > On Mon, Feb 22, 2021 at 1:17 AM Michael J. Baars
> > <[email protected]> wrote:
> > > Hi,
> > >
> > > I just wrote this little program to demonstrate a possible flaw in both malloc and calloc.
> > >
> > > If I allocate a the simplest memory region from main(), one out of three optimization flags fail.
> > > If I allocate the same region from a function, three out of three optimization flags fail.
> > >
> > > Does someone know if this really is a flaw, and if so, is it a gcc or a kernel flaw?
> >
> > There is no flaw. GCC (kernel, glibc) all assume unaligned accesses
> > on x86 will not cause an exception.
>
> Is this just an assumption or more like a fact? I agree with you that byte aligned is more or less the same as unaligned.

It is an assumption that is even made inside GCC. You can modify GCC
not to assume that but you need to recompile all libraries and even
check the assembly code that is included with most programs.
Why are you enabling the alignment access check anyways? What are you
trying to do?
If you are looking into a performance issue with unaligned accesses,
may I suggest you look into perf to see if you can see unaligned
accesses?

Thanks,
Andrew

>
> >
> > Thanks,
> > Andrew
> >
> > > Regards,
> > > Mischa.
>

2021-02-22 10:11:35

by Michael J. Baars

[permalink] [raw]
Subject: Re: problems with memory allocation and the alignment check

On Mon, 2021-02-22 at 01:41 -0800, Andrew Pinski wrote:
> On Mon, Feb 22, 2021 at 1:37 AM Michael J. Baars
> <[email protected]> wrote:
> > On Mon, 2021-02-22 at 01:29 -0800, Andrew Pinski wrote:
> > > On Mon, Feb 22, 2021 at 1:17 AM Michael J. Baars
> > > <[email protected]> wrote:
> > > > Hi,
> > > >
> > > > I just wrote this little program to demonstrate a possible flaw in both malloc and calloc.
> > > >
> > > > If I allocate a the simplest memory region from main(), one out of three optimization flags fail.
> > > > If I allocate the same region from a function, three out of three optimization flags fail.
> > > >
> > > > Does someone know if this really is a flaw, and if so, is it a gcc or a kernel flaw?
> > >
> > > There is no flaw. GCC (kernel, glibc) all assume unaligned accesses
> > > on x86 will not cause an exception.
> >
> > Is this just an assumption or more like a fact? I agree with you that byte aligned is more or less the same as unaligned.
>
> It is an assumption that is even made inside GCC. You can modify GCC
> not to assume that but you need to recompile all libraries and even
> check the assembly code that is included with most programs.
> Why are you enabling the alignment access check anyways? What are you
> trying to do?

I'm writing an algorithm to compress and encrypt data for use in cluster computing. While trying to verify my own code, I stumbled upon this. Just like that...

And yes, as always performance certainly is an issue :) Correctness of the code even more.

> If you are looking into a performance issue with unaligned accesses,
> may I suggest you look into perf to see if you can see unaligned
> accesses?

So it's more like a joke than a flaw? I noticed it earlier with printf too, and traced it back to the strlen kernel assembly code where a lot of sse code is
used that expects memory to be aligned at the 16 byte boundary. Most character strings that come with printf as an argument, apparently aren't.

I'll have a look at perf.

>
> Thanks,
> Andrew
>
> > > Thanks,
> > > Andrew
> > >
> > > > Regards,
> > > > Mischa.



2021-02-24 12:57:44

by Michael J. Baars

[permalink] [raw]
Subject: Re: problems with memory allocation and the alignment check

On Mon, 2021-02-22 at 01:41 -0800, Andrew Pinski wrote:
> On Mon, Feb 22, 2021 at 1:37 AM Michael J. Baars
> <[email protected]> wrote:
> > On Mon, 2021-02-22 at 01:29 -0800, Andrew Pinski wrote:
> > > On Mon, Feb 22, 2021 at 1:17 AM Michael J. Baars
> > > <[email protected]> wrote:
> > > > Hi,
> > > >
> > > > I just wrote this little program to demonstrate a possible flaw in both malloc and calloc.
> > > >
> > > > If I allocate a the simplest memory region from main(), one out of three optimization flags fail.
> > > > If I allocate the same region from a function, three out of three optimization flags fail.
> > > >
> > > > Does someone know if this really is a flaw, and if so, is it a gcc or a kernel flaw?
> > >
> > > There is no flaw. GCC (kernel, glibc) all assume unaligned accesses
> > > on x86 will not cause an exception.
> >
> > Is this just an assumption or more like a fact? I agree with you that byte aligned is more or less the same as unaligned.
>
> It is an assumption that is even made inside GCC. You can modify GCC
> not to assume that but you need to recompile all libraries and even
> check the assembly code that is included with most programs.
> Why are you enabling the alignment access check anyways? What are you
> trying to do?
> If you are looking into a performance issue with unaligned accesses,
> may I suggest you look into perf to see if you can see unaligned
> accesses?

Next to performance and correctness, I always try to keep in mind that every clock cycle will eventually end up on the energy bill, to avoid that computers cost
ten times more on the energy bill then they do in the store.

If you look at the power consumption of the Playstation 1 vs that of the Playstation 3 for example, you will see that the Playstation 1 uses (10 W / 240 V
= 0.041666667 A max, while the Playstation 3 consumes 240 V * 1.7 A = 408 W. More than 40 times as much energy!!!

Code and style always go hand in hand. Try to keep you code as sleek as possible and you will see that even an old computer can do a lot more than you ever
thought possible :)

Thanks,
Mischa.

> Thanks,
> Andrew
>
> > > Thanks,
> > > Andrew
> > >
> > > > Regards,
> > > > Mischa.


Attachments:
compression.c (2.78 kB)
compression.h (288.00 B)
main.c (687.00 B)
makefile (153.00 B)
Download all attachments