-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
I would like to suggest a new attribute for inodes in linux filesystems
to record the last execution time of files.
The problem:
If a linux installation gets older and older, more and more programs get
installed over time. Mostly to just test them for a particular problem
and often the deinstallation is forgotten. To find out which packages
are not used for a long time is currently quite impossible. The user may
use program X which will run but not depend on program Y as a subprocess
for example.
The solution:
I suggest a new inode attribute called xtime, which is like atime, but
will only be updated when a file is executed. This would allow tracking
of unused binaries and could be used with some clever algorithms in the
cleanup program to find unused packages for removal or other cleanup
purposes.
It would also add an additional information in forensic analysis of
hacked systems btw.
Please CC me, I'm not on the list.
kindly regards
Daniel Poelzleithner
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iQIcBAEBAgAGBQJLKS2ZAAoJEFYpgV2RoepczcwQAMEyN63uJd0nW5YMgVn2sxL6
DcHJVnKjxyN31FDJhJcQynn9yvHhFmOMFNQdzKoa4AYwJxBnwlhqtflUt+lkucFo
Bw9WylnKH48tbh4jOpSb6yD7aZd23MIAcGZtF1/cKR7l+UvMpxWhBAKtulJ5mAzm
xKc4M5ItjyAO+mArdgpUO/3hsFUHvG6ufN487njHiLDMDrNc6HVg8nsxVsZvZOXf
7AioGb+M08lkqoG6TLTWXttx9jrKTFJY0/cubMpb3VmCEdFfU1GdQ9eUPCAkPxcJ
zDFrCG2d9Q/mYGuOUWW7kOE8IMPciNWoWuT/BhFD3mW9V1up37hUug7M5TT+QCC3
ypU9Go5eLVtSJbLEUHXR1QOg/vOVCfWTBioSX0tBJTcy2YkNrp7kgQOFgOQtkTKO
GgIN49Y0qiHvyVjy/G40d1rkso312aWpYJqtsDgG78yKW0nPMOUej0p5BTrAvIHu
uR2dzm92SHyt+llGj/I+K6MACrt10iGqmjIEFov1bXiTAyuUbnoKVCD+0jvi1PqV
029Fy8O8ESjmkNlssrNT7DNYPEDoV8Sn5HZbtJr7nxdhefCSn6J7euOGJdiTHfM4
E0mwcGsSGsLaF+XV1tg50RWbIOXFw1fdtbvLvdKbgM6PXxhvSBJBOS2qVIAyqf2u
O4uD9rjGwvmMYSv8HvuD
=EddF
-----END PGP SIGNATURE-----
On Wed, Dec 16, 2009 at 07:57:30PM +0100, Daniel Poelzleithner wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I would like to suggest a new attribute for inodes in linux filesystems
> to record the last execution time of files.
>
> The problem:
>
> If a linux installation gets older and older, more and more programs get
> installed over time. Mostly to just test them for a particular problem
> and often the deinstallation is forgotten. To find out which packages
> are not used for a long time is currently quite impossible. The user may
> use program X which will run but not depend on program Y as a subprocess
> for example.
>
> The solution:
>
> I suggest a new inode attribute called xtime, which is like atime, but
> will only be updated when a file is executed. This would allow tracking
> of unused binaries and could be used with some clever algorithms in the
> cleanup program to find unused packages for removal or other cleanup
> purposes.
> It would also add an additional information in forensic analysis of
> hacked systems btw.
Given the existence of noatime and other related mount options
designed to mitigate the performance penalty related to atime,
adding xtime doesn't strike me as a particularly good idea.
I suspect there are easier ways to track when executables are
executed.
Daniel Poelzleithner wrote:
> The problem:
>
> If a linux installation gets older and older, more and more programs get
> installed over time. Mostly to just test them for a particular problem
> and often the deinstallation is forgotten. To find out which packages
> are not used for a long time is currently quite impossible. The user may
> use program X which will run but not depend on program Y as a subprocess
> for example.
>
> The solution:
Perhaps something like the "Debian package Popularity Contest":
http://popcon.debian.org/
...
The popularity contest project is an attempt to map the usage of Debian
packages. This site publishes the statistics gathered from report sent
by users of the popularity-contest package. This package sends every
week the list of packages installed and the access time of relevant
files to the server via email. Every day the server anonymizes the
result and publishes this survey. For more information, read the README
and the FAQ.
...
--
Jeffrey Hundstad
On Wed, Dec 16, 2009 at 1:57 PM, Daniel Poelzleithner <[email protected]> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I would like to suggest a new attribute for inodes in linux filesystems
> to record the last execution time of files.
>
> The problem:
>
> If a linux installation gets older and older, more and more programs get
> installed over time. Mostly to just test them for a particular problem
> and often the deinstallation is forgotten. To find out which packages
> are not used for a long time is currently quite impossible. The user may
> use program X which will run but not depend on program Y as a subprocess
> for example.
>
> The solution:
>
> I suggest a new inode attribute called xtime, which is like atime, but
> will only be updated when a file is executed. This would allow tracking
> of unused binaries and could be used with some clever algorithms in the
> cleanup program to find unused packages for removal or other cleanup
> purposes.
> It would also add an additional information in forensic analysis of
> hacked systems btw.
Why isn't atime sufficient? atime is updated when programs are
executed, and it's uncommon for executable files to be accessed
without being executed; for the purposes of determining whether a
program is needed, it ought to be good enough.
Note that many modern distros use relatime to reduce the (significant)
overhead of atime; this may account for why you may not be seeing
atime update for executables; try mounting with -o strictatime if you
want to see it at work, but beware, as this will bring with it a major
performance hit - which your proposal would as well :)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bryan Donlan wrote:
> Why isn't atime sufficient? atime is updated when programs are
> executed, and it's uncommon for executable files to be accessed
> without being executed; for the purposes of determining whether a
> program is needed, it ought to be good enough.
Why atime is not useable is for example that full system backup would
distroy all data. Thinking about grepping for a string in /usr or
/usr/bin because you are searching for something will update all atime
stamps and therefore cause dataloss for this purpose.
> Note that many modern distros use relatime to reduce the (significant)
> overhead of atime; this may account for why you may not be seeing
> atime update for executables; try mounting with -o strictatime if you
> want to see it at work, but beware, as this will bring with it a major
> performance hit - which your proposal would as well :)
The performance impact is one reason of course. As executing is much
more seldom the impact of updating the xtime should be mild enough that
it could be done by default. I don't mount strictatime for this reason, too.
kindly regards
Daniel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iQIcBAEBAgAGBQJLK0kdAAoJEFYpgV2Roepc35AP/RkT6Q0Nz0qseGboEFYOfwju
NDsyJT2Thw/bm/zwo024TwnyoBOA2o9rGwmIuppsOGijpyuSSwJXVUcnPXoC/Gto
Eb3eMud41qZewFgIRNkur7torWG2k59GmY9AgmKgYbm8L1O81k6Zeesl3aY0SedJ
gF690R7Ktn+Cb+9xMq3g2ip8Vu7TegHHS0L9UtD0zaVRlmqBf1cfZBWpKZJ1ndyG
15/pUBdYoIJavMxgRAbpcpSZfBEJlW1kJpELeIyBCglYkh8j2W5mJVi0PcwUtEzp
cdgVNjQwpqUicYsY7WbG469m2jd7oPl6hCKPJ1oWncaVUyddubU9JHYRAAgTYvMT
fBb/83yxQ6H9DFfwHk7dRGLrko8vhZ3GiDJIYmbWbR5XHa+qr4lwoX1M3NTAp62B
j15hy0Ap2cu58tFhMFoh9pOPY95b5Xs7hVlBcqm2CqSwIOcmO/3f8Vmw1RLlG/F+
0XZ692nmYFPrnKqQAM6mQXuKUI4QGPLlaD2lSyZf3czXD1kTt01SQSvZ7stI3aA5
hEgYDm+zJHR76W6Wr5JXISQMSCLOiYf5g0gRLF2TIXz5VL2W3NOex+vCnBoEPmXt
hQ6t3x1YWrhE35VV2r4zhxUz+/9wGEncGGOwwdBPmegVW/NrxAwjsa7hPdddG5+4
V9RsTox7UQNQYaxk3PpR
=ixfn
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Jeffrey Hundstad wrote:
>> If a linux installation gets older and older, more and more programs get
>> installed over time. Mostly to just test them for a particular problem
>> and often the deinstallation is forgotten. To find out which packages
>> are not used for a long time is currently quite impossible. The user may
>> use program X which will run but not depend on program Y as a subprocess
>> for example.
>>
>> The solution:
>
>
> Perhaps something like the "Debian package Popularity Contest":
> http://popcon.debian.org/
I think you missunderstood my intension. I wasn't thinking about a
distribution but the installation on a system. All my systems grow in
installed packages over time, i think that is something every hard core
user experiences.
You test 4-5 programs for a new task you have to do, but over time you
only use 1-2 of them and simply forget you installed some more programs
for this.
With xtime there is a very clean way to find which packages are
installed and are never used for a long time and can therefore suggested
to the user to be uninstalled. The cleanup script of course will highly
depend on the package system used and how the packages are organized and
named.
kindly regards
Daniel
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iQIcBAEBAgAGBQJLK0scAAoJEFYpgV2Roepc3F8QAItCR2ma5c5Il26YqZlR7m/j
eFTObrN1RGPTVs3kg0E5DdlBMwg2DyUudrr0JSkfkf6x7Hc0ar39WVM2+LDcT1d/
gV8T8CMONNsqBwfPDfxkkKWzXjVvD48HdwLMLhcbzRHL4KoZJw0Fj3MQfipu5Xe+
qP6RIShyj7zLaUkjsY1p2LP+b/euJ3jpzntvZA/I0/Q/rXQ+2eEHYzelFeyPCgqT
e4zjm7RKBuukQOtvS0Yiys3zyW0lFszfSwLMnWUM8V3vZBjUsVeuULlqmwWgcQhl
cKMzz4lCpkFUZen1CKsw64m8LVWH5MryfEJ+wl8UkgtwBJpKENIrkMRFgjXaIMLQ
4n2rLjbLNwdzJsCA6HdGCqD5Vq7PZtMnWf4n3Fn1NfLGcDN+fScKW9R+SLfEwM8D
i213qvL40jM8nFRxZ3bCjwM9iMKWObF96i8SEZ+q1ig4BzjJXYt3Va1HL31KL9v2
CrDa0w5R04p/IIp5vl6OmXryS1Nsj8kidL433z2WOttrr6tQPnlk1IjPTzoU9zUw
VZcuPck1SNAkLpqeCMK4n5lGeVsfAsiiNRRcZXyOkyqSJimd/BxFmLPmNaf7HYrd
V8oUDUO8NP7Sx9bPCxlmZKmuQWXqm+Eq9bm6vrl179Jy4nd1+UTfRL1twSDuIsnm
nAxFTQ2Ea8Vm2gE/IHvE
=DOSZ
-----END PGP SIGNATURE-----
On Fri, Dec 18, 2009 at 10:27:57AM +0100, Daniel Poelzleithner wrote:
> I think you missunderstood my intension. I wasn't thinking about a
> distribution but the installation on a system. All my systems grow in
> installed packages over time, i think that is something every hard core
> user experiences.
> You test 4-5 programs for a new task you have to do, but over time you
> only use 1-2 of them and simply forget you installed some more programs
> for this.
> With xtime there is a very clean way to find which packages are
> installed and are never used for a long time and can therefore suggested
> to the user to be uninstalled. The cleanup script of course will highly
> depend on the package system used and how the packages are organized and
> named.
This sounds like something a distro might want to do; and as such,
I'll note that if such a feature is *really* wanted, it's something
that can be done in user space --- for example, by adding a hack to
/lib/ld-linux.so which sends a ping-o-gram to a daemon that then
updates the relevant database. That way you don't penalize every
single inode for something that is only needed for some inodes
(namely, the executables).
An extended attribute might also do, although in that case I'd
strongly suggest that the timestamp field only be updated if the last
execute time is over 24 hours old. That way you're not constantly
updating the file system for very frequently updated executables such
as "/bin/ls". Do we really need to know whether /bin/ls was executed
24 seconds ago as opposed to 30 minutes ago?
- Ted
Daniel Poelzleithner <[email protected]> writes:
> I would like to suggest a new attribute for inodes in linux filesystems
> to record the last execution time of files.
>
> The problem:
>
> If a linux installation gets older and older, more and more programs get
> installed over time. Mostly to just test them for a particular problem
> and often the deinstallation is forgotten. To find out which packages
> are not used for a long time is currently quite impossible. The user may
> use program X which will run but not depend on program Y as a subprocess
> for example.
>
> The solution:
>
> I suggest a new inode attribute called xtime, which is like atime, but
> will only be updated when a file is executed. This would allow tracking
> of unused binaries and could be used with some clever algorithms in the
> cleanup program to find unused packages for removal or other cleanup
> purposes.
As I've never seen "cat /bin/sh" or similar done on an executable binary
by sane people, and I imagine that most sane back-up programs wouldn't
touch atime, I suspect that what you want to do can be done by checking
atime instead.
Junio C Hamano wrote:
> As I've never seen "cat /bin/sh" or similar done on an executable binary
> by sane people
How about 'strings /bin/sh' then?
Or maybe 'objdump -x /bin/sh'?
Or 'grep "something" /directory/containing/shell/scripts/*'?
But maybe I am insane.
Cheers,
FJP