2002-10-03 11:09:40

by jbradford

[permalink] [raw]
Subject: RAID backup

Might it not be a good idea to DD the raw contents of each disk to a tape drive, just incase you fubar the array? It would be time consuming, but at least you could restore your data in the event that it gets corrupted.

John.


2002-10-03 11:11:16

by Roy Sigurd Karlsbakk

[permalink] [raw]
Subject: Re: RAID backup

On Thursday 03 October 2002 13:20, [email protected] wrote:
> Might it not be a good idea to DD the raw contents of each disk to a tape
> drive, just incase you fubar the array? It would be time consuming, but at
> least you could restore your data in the event that it gets corrupted.

er

16 120GB disks?
--
Roy Sigurd Karlsbakk, Datavaktmester
ProntoTV AS - http://www.pronto.tv/
Tel: +47 9801 3356

Computers are like air conditioners.
They stop working when you open Windows.

2002-10-03 11:12:10

by Roy Sigurd Karlsbakk

[permalink] [raw]
Subject: Re: RAID backup

sorry

wrong answer
wrong mail
oops

On Thursday 03 October 2002 13:20, [email protected] wrote:
> Might it not be a good idea to DD the raw contents of each disk to a tape
> drive, just incase you fubar the array? It would be time consuming, but at
> least you could restore your data in the event that it gets corrupted.
>
> John.

--
Roy Sigurd Karlsbakk, Datavaktmester
ProntoTV AS - http://www.pronto.tv/
Tel: +47 9801 3356

Computers are like air conditioners.
They stop working when you open Windows.

2002-10-03 12:32:06

by jbradford

[permalink] [raw]
Subject: Re: RAID backup

> > Might it not be a good idea to DD the raw contents of each disk to a tape
> > drive, just incase you fubar the array? It would be time consuming, but at
> > least you could restore your data in the event that it gets corrupted.
>
> er
>
> 16 120GB disks?

A fast, large tape backup :-)

Seriously, if this is crucial data, (E.G. solution to Fermat's last theorum), I was just pointing out that you can backup, even without mounting the disks.

John.

2002-10-03 19:58:23

by Kanoalani Withington

[permalink] [raw]
Subject: Re: RAID backup

I have to pipe in here and agree that the idea of using a disk array
alone for backups is not a sound idea. Sure, backing up 2Tb to an old
exabyte drive isn't going to work, if you really have that much data you
need some more modern equipment.

Essentially I believe the idea of a redundant array sounds safer than it
really is in practice, especially when dealing with very large arrays
and with level 5 arrays. The reasons why this is so are manifold,
suffice to say that a few years of actually using such devices shows
that they have much more potential for catastrophic failure and latent
failure (you don't know it's broken until you go to use it and find out
it's broken) than a well designed tape archive or backup.

Not that disk to disk backups are a completely bad idea. In my
experience a combination works best. For example, automatic backups to
reserved disks or disk arrays on remote systems every night, but once a
week tape snapshots of that data. It's a lot of tapes but over time it
will prove to be worthwhile. If the data volume is too high, simple
backup scripts that write every file only once (essentially an archive)
to tape to make it more practical.

-Kanoa


Roy Sigurd Karlsbakk wrote:

>On Thursday 03 October 2002 13:20, [email protected] wrote:
>
>>Might it not be a good idea to DD the raw contents of each disk to a tape
>>drive, just incase you fubar the array? It would be time consuming, but at
>>least you could restore your data in the event that it gets corrupted.
>>
>
>er
>
>16 120GB disks?
>


2002-10-03 23:55:13

by Eff Norwood

[permalink] [raw]
Subject: RE: RAID backup

> I have to pipe in here and agree that the idea of using a disk array
> alone for backups is not a sound idea. Sure, backing up 2Tb to an old
> exabyte drive isn't going to work, if you really have that much data you
> need some more modern equipment.

I've worked in the storage industry for years and surprisingly more and more
organizations *are* moving to disk only backup strategies with a lot of good
reasons for doing so. Large companies are replacing 100% of their tape
infrastructure with multiple large yet inexpensive disk array's. Disk backup
infrastructure prices are now equal to or lower than tape backup
infrastructure prices on initial acquisition, and significantly lower if you
factor in ROI and TCO - not to mention huge backup/restore time savings and
potential disaster recovery possibilities.

> Essentially I believe the idea of a redundant array sounds safer than it
> really is in practice, especially when dealing with very large arrays
> and with level 5 arrays. The reasons why this is so are manifold,
> suffice to say that a few years of actually using such devices shows
> that they have much more potential for catastrophic failure and latent
> failure (you don't know it's broken until you go to use it and find out
> it's broken) than a well designed tape archive or backup.

With multiple inexpensive large disk arrays from companies like Network
Appliance (NearStor) and Exstor (T-2120) organizations are asynchronously
mirroring their data to geographically distant locations to prevent single
points of failure with their arrays. As you suggest, a single array can fail
and lose data. From a tape perspective, a tape in a backup set can fail as
well-potentially presenting the same catastrophic situation of not being
able to recover. Just as organizations have multiple sets of tapes on a
rotation schedule, organizations now often have multiple large near-line
storage arrays with a data rotation schedule.

> Not that disk to disk backups are a completely bad idea. In my
> experience a combination works best. For example, automatic backups to
> reserved disks or disk arrays on remote systems every night, but once a
> week tape snapshots of that data. It's a lot of tapes but over time it
> will prove to be worthwhile. If the data volume is too high, simple
> backup scripts that write every file only once (essentially an archive)
> to tape to make it more practical.

Classical Hierarchical Storage Management (HSM) as you describe here is
being undermined by the fact that these large IDE based disk arrays are
equally or less expensive than tapes. In addition, HSM software costs
something (even if you write it yourself) on top of the tape infrastructure.
One customer of ours was quoted 40K per TB of HSM *software* alone. Well
designed tape replacement disk systems can cost 25K per TB or less and have
many advantages over tape. Online data, faster data access, and disaster
recovery just to name a few. Further, backup software vendors like Veritas
and Legato now recognize disks as a viable tape replacement and are
incorporating that fact into their software.

If tape is absolutely required, the infrastructure required per TB of taped
data can be significantly smaller and the backup window can be massively
extended - from hours to weeks with a large near-line disk approach. The
tape/library cost component in this way is significantly reduced.

For purposes of full disclosure, I work at Exstor and we provide tape
replacement solutions as well as high performance NAS solutions for the
enterprise.

Regards,

Eff Norwood


2002-10-04 07:55:32

by Lars Marowsky-Bree

[permalink] [raw]
Subject: Re: RAID backup

On 2002-10-03T16:59:57,
Effrem Norwood <[email protected]> said:

> With multiple inexpensive large disk arrays from companies like Network
> Appliance (NearStor) and Exstor (T-2120) organizations are asynchronously
> mirroring their data to geographically distant locations to prevent single
> points of failure with their arrays.

Let's just point out that Linux can do that too with drbd.

(I wonder if that will stay a separate module or whether it'll become a EVMS
plugin ;-)


Sincerely,
Lars Marowsky-Br?e <[email protected]>

--
Principal Squirrel
Research and Development, SuSE Linux AG

``Immortality is an adequate definition of high availability for me.''
--- Gregory F. Pfister

2002-10-04 10:22:01

by Illtud Daniel

[permalink] [raw]
Subject: Re: RAID backup

Effrem Norwood wrote:

> In addition, HSM software costs
> something (even if you write it yourself) on top of the tape infrastructure.
> One customer of ours was quoted 40K per TB of HSM *software* alone.

I've got 25TB uncompressed of HSM here, and it's cost us (ex VAT)
roughly:

?10k for the first 1.6TB (on NT, DLT library)
?18k for the next 6.0TB (on NT, LTO library)
?45k for the next 18.0TB (on Solaris, LTO Library)

...for the software licencing alone. Plus about 10% pa. in support
costs.
You're looking at about 50-60% of the library cost for the HSM software
to manage it (tapes are another thing). Is HSM really that difficult?

It really is a racket, but it's not so much compared with the
cost of re-producing the data (mainly digitized collections).
I'd be happier about it if they were more reliable (libs and s/w).
Disk arrays, on the other hand, would cost us a fortune in
upgrading the cooling - we've had to do this once just because of
the 3-4 TB of online storage we've got, and adding huge exchangers
(and associated pipes) isn't something I want to do much of.

Oh, and having spent much of last night and this morning dealing
with multiple SCSI disk failures, and having seen about 5% of
ours fail in a year, I'm rapidly seeing the light on IDE.

--
Illtud Daniel [email protected]
Uwch Ddadansoddwr Systemau Senior Systems Analyst
Llyfrgell Genedlaethol Cymru National Library of Wales
Yn siarad drosof fy hun, nid LlGC - Speaking personally, not for NLW

2002-10-04 11:05:49

by jbradford

[permalink] [raw]
Subject: Re: RAID backup

> > In addition, HSM software costs
> > something (even if you write it yourself) on top of the tape infrastructure.
> > One customer of ours was quoted 40K per TB of HSM *software* alone.
>
> I've got 25TB uncompressed of HSM here, and it's cost us (ex VAT)
> roughly:
>
> ?10k for the first 1.6TB (on NT, DLT library)
> ?18k for the next 6.0TB (on NT, LTO library)
> ?45k for the next 18.0TB (on Solaris, LTO Library)
>
> ..for the software licencing alone. Plus about 10% pa. in support
> costs.
> You're looking at about 50-60% of the library cost for the HSM software
> to manage it (tapes are another thing). Is HSM really that difficult?
>
> It really is a racket, but it's not so much compared with the
> cost of re-producing the data (mainly digitized collections).
> I'd be happier about it if they were more reliable (libs and s/w).
> Disk arrays, on the other hand, would cost us a fortune in
> upgrading the cooling - we've had to do this once just because of
> the 3-4 TB of online storage we've got, and adding huge exchangers
> (and associated pipes) isn't something I want to do much of.
>
> Oh, and having spent much of last night and this morning dealing
> with multiple SCSI disk failures, and having seen about 5% of
> ours fail in a year, I'm rapidly seeing the light on IDE.

This is rapidly becoming off topic, especially for the kernel dev list, which is why I originally just posted the following info to the cc'ed people, but since it might be of interest, I'm posting it to the lists:

Sony GY-8240-DTF2 tape drive, picture of it, with some info here:

http://www.tomtec.co.jp/english/tape_hisped.html

this uses similar technology to that which is used in their Digital Betacam(tm) studio VCRs, and stores 200 Gigs uncompressed on one large tape, (60 on a small tape).

Data rate, 24 MB/s.

You've got to have a lot of data for that to be insufficient, and it would be my backup system of choice, if I needed the capcity, (I currently use punched Myler tape :-) ).

John.

2002-10-04 11:15:49

by Alvin Oga

[permalink] [raw]
Subject: Re: RAID backup


hi ya

we can build an 8-drive ( 120GB at $200ea ) or ( 160GB at $300>? each )
1U box...about 0.960 - 1.28 TB each backup server ( 1U ) for under $2,500 in parts
+ cost of raid setup/testing is up to the user
- am thinking the 1.6TB of storage for 10K lira(?) is too much

i prefer disks to backup data.. so that its always a semi-warm backup
( tapes have always been way tooo slow to find a file and to restore

have fun
alvin
http://www.Linux-1U.net

On Fri, 4 Oct 2002, Illtud Daniel wrote:

> Effrem Norwood wrote:
>
> > In addition, HSM software costs
> > something (even if you write it yourself) on top of the tape infrastructure.
> > One customer of ours was quoted 40K per TB of HSM *software* alone.
>
> I've got 25TB uncompressed of HSM here, and it's cost us (ex VAT)
> roughly:
>
> ?10k for the first 1.6TB (on NT, DLT library)
> ?18k for the next 6.0TB (on NT, LTO library)
> ?45k for the next 18.0TB (on Solaris, LTO Library)
>
>

2002-10-04 12:40:41

by Alan

[permalink] [raw]
Subject: Re: RAID backup

On Fri, 2002-10-04 at 12:20, Alvin Oga wrote:
> we can build an 8-drive ( 120GB at $200ea ) or ( 160GB at $300>? each )
> 1U box...about 0.960 - 1.28 TB each backup server ( 1U ) for under $2,500 in parts
> + cost of raid setup/testing is up to the user
> - am thinking the 1.6TB of storage for 10K lira(?) is too much
>
> i prefer disks to backup data.. so that its always a semi-warm backup
> ( tapes have always been way tooo slow to find a file and to restore

The problem with disks is you still have to archive them somewhere, and
they are bulky. I also dont know what studies are available on the
degradation of stored disk media over time.

Capacity is not a problem, 3ware do a 12 channel sata card, with maxtor
drives that comes in at 320x12 = 3.5Tb

2002-10-04 12:47:32

by Mr. James W. Laferriere

[permalink] [raw]
Subject: Re: RAID backup


Hello Alan , I do not know about modern disk media . But I had a
Seagate st501 drive that sat shelved for 5 years & put it back
into the micro-pdp11/23 & booted fine . I ran that for ~ another
year for a small project I was involved in . I would hope that
more modern media would have better shelf life than that even ?-)
Twyl , JimL

On 4 Oct 2002, Alan Cox wrote:
> On Fri, 2002-10-04 at 12:20, Alvin Oga wrote:
> > we can build an 8-drive ( 120GB at $200ea ) or ( 160GB at $300>? each )
> > 1U box...about 0.960 - 1.28 TB each backup server ( 1U ) for under $2,500 in parts
> > + cost of raid setup/testing is up to the user
> > - am thinking the 1.6TB of storage for 10K lira(?) is too much
> > i prefer disks to backup data.. so that its always a semi-warm backup
> > ( tapes have always been way tooo slow to find a file and to restore

> The problem with disks is you still have to archive them somewhere, and
> they are bulky. I also dont know what studies are available on the
> degradation of stored disk media over time.

> Capacity is not a problem, 3ware do a 12 channel sata card, with maxtor
> drives that comes in at 320x12 = 3.5Tb

+------------------------------------------------------------------+
| James W. Laferriere | System Techniques | Give me VMS |
| Network Engineer | P.O. Box 854 | Give me Linux |
| [email protected] | Coudersport PA 16915 | only on AXP |
+------------------------------------------------------------------+

2002-10-04 13:18:48

by Dave Gilbert (Home)

[permalink] [raw]
Subject: Re: RAID backup

* Alan Cox ([email protected]) wrote:
>
> The problem with disks is you still have to archive them somewhere, and
> they are bulky. I also dont know what studies are available on the
> degradation of stored disk media over time.

Not sure about that; DLT tapes are pretty bulky themselves; I think the
difference between say a set of 4 DLT tapes and a single Maxtor 320 in
caddy would be minimal. As for stored media, I think Maxtor are quoting
1M hours MTTF - (I hate to think how you measure such a figure) - for
the 320G, and that is probably longer than I'd trust either the tape or
the drive to survive.

> Capacity is not a problem, 3ware do a 12 channel sata card, with maxtor
> drives that comes in at 320x12 = 3.5Tb

Well to me there are two questions:
1) Price with caddy/drive - especially when you need to have
multiple backup sets.

2) Linux serial/ata working reliably with hot swapping.

If both those came out on the right side then I'd happily swap to discs.

Dave
---------------- Have a happy GNU millennium! ----------------------
/ Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \
\ gro.gilbert @ treblig.org | MIPS,x86,ARM, SPARC and HP-PA | In Hex /
\ _________________________|_____ http://www.treblig.org |_______/

2002-10-04 14:02:28

by Russell King

[permalink] [raw]
Subject: Re: RAID backup

On Fri, Oct 04, 2002 at 02:24:19PM +0100, Dr. David Alan Gilbert wrote:
> Not sure about that; DLT tapes are pretty bulky themselves; I think the
> difference between say a set of 4 DLT tapes and a single Maxtor 320 in
> caddy would be minimal.

The 4 DLT tapes would take up twice the room.

> As for stored media, I think Maxtor are quoting
> 1M hours MTTF - (I hate to think how you measure such a figure) - for
> the 320G, and that is probably longer than I'd trust either the tape or
> the drive to survive.

However, drive in caddy or no caddy, the accidental drop test would
probably be more favourable to the DLT tape surviving over the drive.
Physical accidents do happen.

--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html

2002-10-04 14:27:02

by Luca Berra

[permalink] [raw]
Subject: Re: RAID backup

Dr. David Alan Gilbert wrote:
> * Alan Cox ([email protected]) wrote:
>
>>The problem with disks is you still have to archive them somewhere, and
>>they are bulky. I also dont know what studies are available on the
>>degradation of stored disk media over time.
>
>
> Not sure about that; DLT tapes are pretty bulky themselves; I think the
> difference between say a set of 4 DLT tapes and a single Maxtor 320 in
> caddy would be minimal. As for stored media, I think Maxtor are quoting
> 1M hours MTTF - (I hate to think how you measure such a figure) - for
> the 320G, and that is probably longer than I'd trust either the tape or
> the drive to survive.

i DO seriously doubt that this figure includes removing the drive,
stuffing it in a siutcase or similar, loading on a truck/car/bike and
unloading at a remote site.

Regards,
L.

--
Luca Berra -- [email protected]
Service Delivery Manager
Communication Media & Services S.r.l.
Via A. Modigliani 1 - MILANO


2002-10-04 15:09:52

by Richard B. Johnson

[permalink] [raw]
Subject: Re: RAID backup

On Fri, 4 Oct 2002, Luca Berra wrote:

> Dr. David Alan Gilbert wrote:
> > * Alan Cox ([email protected]) wrote:
> >
> >>The problem with disks is you still have to archive them somewhere, and
> >>they are bulky. I also dont know what studies are available on the
> >>degradation of stored disk media over time.
> >
> >
> > Not sure about that; DLT tapes are pretty bulky themselves; I think the
> > difference between say a set of 4 DLT tapes and a single Maxtor 320 in
> > caddy would be minimal. As for stored media, I think Maxtor are quoting
> > 1M hours MTTF - (I hate to think how you measure such a figure) - for
> > the 320G, and that is probably longer than I'd trust either the tape or
> > the drive to survive.
>
> i DO seriously doubt that this figure includes removing the drive,
> stuffing it in a siutcase or similar, loading on a truck/car/bike and
> unloading at a remote site.
>
> Regards,
> L.

Fire-wire 80 GB external drives work fine for this (Maxtor and others).
Keep them in a cool, not too dry (30 - 50 % RH), area. After a year or
two, they may take several power-cycles to start them up. You should
store them on an edge. This helps keep lube from weeping from the
sintered-bronze (Oilite) bearing, onto the platters.

Electronics that contain electrolytic capacitors shouldn't be stored
where the RH is below 10 % (either should tapes). The platters of
these disk-drives are in an environment where air-pressure can equalize
so they are not really "sealed". Instead, any air entering gets filtered.
Therefore, keep them away from things that leak nasty vapors like
batteries, paint, and fuels.

I've tried for years to find tape-drives that can reliably restore
backed-up data. Once you get to multi- Gigabyte drives, you can just
forget it. The data reliability necessary to read through 80 or more
gigibytes of data (serially) with no errors, to get to the file you need,
is just not available in any drive I've tested including the newest Sony
DLT.

The Linux SCSI tape driver will not attempt to read past a "permanent"
error. In principle, it could find the next tape-mark, then continue,
but it doesn't. Once an error occurs that the drive can't electronically
correct, that's all she wrote. It's not like a disk where you can just
tell some software layer to never read a bad "sector" again.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
The US military has given us many words, FUBAR, SNAFU, now ENRON.
Yes, top management were graduates of West Point and Annapolis.

2002-10-04 16:57:49

by Cress, Andrew R

[permalink] [raw]
Subject: RE: RAID backup

Not that IDE disks would be more reliable, just that you couldn't see the
disk errors until it was too late. :-)

However, given the IDE disk prices, you can afford to replace them more
often.

Also, with SCSI drives, some people see sense errors, and correctable media
defects, and mistakenly assume it is a 'disk failure'.

Andy

-----Original Message-----
From: Illtud Daniel [mailto:[email protected]]
Sent: Friday, October 04, 2002 6:25 AM

[snip]

Oh, and having spent much of last night and this morning dealing
with multiple SCSI disk failures, and having seen about 5% of
ours fail in a year, I'm rapidly seeing the light on IDE.

--
Illtud Daniel [email protected]
Uwch Ddadansoddwr Systemau Senior Systems Analyst
Llyfrgell Genedlaethol Cymru National Library of Wales
Yn siarad drosof fy hun, nid LlGC - Speaking personally, not for NLW

2002-10-04 15:22:33

by Herman Oosthuysen

[permalink] [raw]
Subject: Re: RAID backup

Howdy,

The MTTF number is calculated during a process intended to identify
weaknesses in the design of electronic equipment. Depending on how you
model the failure modes and who's failure tables you use, the MTTF
number will vary enormously.

The important thing is not the number per se, but rather the quality
review process associated with the calculation effort.

Unfortunately, the MTTF number became a marketing fad, with the result
that some companies will calculate it, without doing any quality
reviews, purely for marketing purposes. There is no way to tell what
they did. The MTTF number is consequently totally meaningless by itself.

However, judging by my own experience, Maxtor drives are quite reliable
and should last 3 years or more when in use and just about indefinitely
when in storage.

Self demagnetization used to be a problem of magnetic media and some
components such as capacitors used to deteriorate with age, but I think
that those problems have been solved decades ago, so equipment in clean
and dry storage should last almost forever.

The important thing to remember with disks and tapes is that they will
eventually fail. When that will happen is anybody's guess, but you have
to plan for the eventuality; you can't just sit on your hands and hope
for the best and the backup measures that you implement, should be
commensurate with the value of the data.

Cheers,

Herman
http://www.AerospaceSoftware.com

Luca Berra wrote:
> Dr. David Alan Gilbert wrote:
>
>> * Alan Cox ([email protected]) wrote:
>>
>>> The problem with disks is you still have to archive them somewhere, and
>>> they are bulky. I also dont know what studies are available on the
>>> degradation of stored disk media over time.
>>
>>
>>
>> Not sure about that; DLT tapes are pretty bulky themselves; I think the
>> difference between say a set of 4 DLT tapes and a single Maxtor 320 in
>> caddy would be minimal. As for stored media, I think Maxtor are quoting
>> 1M hours MTTF - (I hate to think how you measure such a figure) - for
>> the 320G, and that is probably longer than I'd trust either the tape or
>> the drive to survive.
>
>
> i DO seriously doubt that this figure includes removing the drive,
> stuffing it in a siutcase or similar, loading on a truck/car/bike and
> unloading at a remote site.
>
> Regards,
> L.
>

--

------------------------------------------------------------------------
Herman Oosthuysen
B.Eng.(E), Member of IEEE
Wireless Networks Inc.
http://www.WirelessNetworksInc.com
E-mail: [email protected]
Phone: 1.403.569-5687, Fax: 1.403.235-3964
------------------------------------------------------------------------


2002-10-04 17:10:23

by Dave Gilbert (Home)

[permalink] [raw]
Subject: Re: RAID backup

* Russell King ([email protected]) wrote:

> However, drive in caddy or no caddy, the accidental drop test would
> probably be more favourable to the DLT tape surviving over the drive.
> Physical accidents do happen.

While I guess a good caddy might put a lump of rubber in to help. But
there again I'm not sure if I have that much confidence in the DLT tapes
either; the instructions in the insert for HP DLT tapes tell you to
rattle them and listen to hear if you can hear anything loose
before putting them into your drive!

Anyway, from the side of data integrity the drop test doesn't worry me -
for critical data I have a lot more than one backup; and users perform
an important part of the backup test system by regularly deleting files
to be restored.

Dave

---------------- Have a happy GNU millennium! ----------------------
/ Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \
\ gro.gilbert @ treblig.org | MIPS,x86,ARM, SPARC and HP-PA | In Hex /
\ _________________________|_____ http://www.treblig.org |_______/

2002-10-04 16:05:53

by Russell King

[permalink] [raw]
Subject: Re: RAID backup

On Fri, Oct 04, 2002 at 09:31:26AM -0600, Herman Oosthuysen wrote:
> Self demagnetization used to be a problem of magnetic media and some
> components such as capacitors used to deteriorate with age, but I think
> that those problems have been solved decades ago, so equipment in clean
> and dry storage should last almost forever.

You missed "stable temperature" as well. Some capacitors still
"dry out" with age and heat, even with todays technology.

--
Russell King ([email protected]) The developer of ARM Linux
http://www.arm.linux.org.uk/personal/aboutme.html

2002-10-04 18:53:59

by Kanoalani Withington

[permalink] [raw]
Subject: Re: RAID backup



Illtud Daniel wrote:

>Effrem Norwood wrote:
>
>>In addition, HSM software costs
>>something (even if you write it yourself) on top of the tape infrastructure.
>>One customer of ours was quoted 40K per TB of HSM *software* alone.
>>
>
>I've got 25TB uncompressed of HSM here, and it's cost us (ex VAT)
>roughly:
>
>?10k for the first 1.6TB (on NT, DLT library)
>?18k for the next 6.0TB (on NT, LTO library)
>?45k for the next 18.0TB (on Solaris, LTO Library)
>
>...for the software licencing alone. Plus about 10% pa. in support
>costs.
>You're looking at about 50-60% of the library cost for the HSM software
>to manage it (tapes are another thing). Is HSM really that difficult?
>
>It really is a racket, but it's not so much compared with the
>cost of re-producing the data (mainly digitized collections).
>I'd be happier about it if they were more reliable (libs and s/w).
>Disk arrays, on the other hand, would cost us a fortune in
>upgrading the cooling - we've had to do this once just because of
>the 3-4 TB of online storage we've got, and adding huge exchangers
>(and associated pipes) isn't something I want to do much of.
>
I agree it's a total racket. I've spent an appalling amount of money on
this stuff over the years considering how simple it is. Last year I
finally built mtx, the open source tape library driver, and wrote my own
software in tcl scripts for a new archiving system. It really is that
simple, I don't know how they can charge so much for thier software,
especially when some it is junk to begin with.

-Kanoa

>
>
>Oh, and having spent much of last night and this morning dealing
>with multiple SCSI disk failures, and having seen about 5% of
>ours fail in a year, I'm rapidly seeing the light on IDE.
>


2002-10-04 18:41:21

by Herman Oosthuysen

[permalink] [raw]
Subject: Re: RAID backup

Yep, a constant lowish temperature will also help a little, but I
honestly think that in storage, a disk drive bearing will seize due to
corrosion and lack of movement, before the electronics will fail. You
can prevent bearing (and capacitor) problems by starting the drive up
every 6 months for a while though.

The effects of long term storage is experienced with military equipment,
which frequently have to stay in storage for 40 or 50 years without ever
really being used. The stuff in storage has to be cycled through the
workshops to keep them operational, which is part of the high cost of peace.

If disk drives are used for backup purposes, then I would suggest that
they are rotated, so that they all remain in occational use, which
should help to keep them alive. Of course, the first thing to fail,
would be the connector of the removable drive bay and there is nothing
you can do about that, except to make sure that the connectors are gold
plated to begin with.

I don't think that people have to worry too much about transportation
vibration and shock to/from an off-site storage facility. These things
are quite rugged when not spinning, so if you transport them in
styrofoam boxes and don't drop them on the floor, they should be OK.

BTW, putting a drive in storage for 40 or 50 years is not recommended
for another reason: Obsolecense. In 40 or 50 years, you probably won't
have a computer that can use these drives anymore! So, the only way to
keep data long term, is to rotate the media continuously and upgrade as
new technology is introduced.

Cheers,

Herman
http://www.AerospaceSoftware.com

Russell King wrote:
> On Fri, Oct 04, 2002 at 09:31:26AM -0600, Herman Oosthuysen wrote:
>
>>Self demagnetization used to be a problem of magnetic media and some
>>components such as capacitors used to deteriorate with age, but I think
>>that those problems have been solved decades ago, so equipment in clean
>>and dry storage should last almost forever.
>
>
> You missed "stable temperature" as well. Some capacitors still
> "dry out" with age and heat, even with todays technology.
>

--

------------------------------------------------------------------------
Herman Oosthuysen
B.Eng.(E), Member of IEEE
Wireless Networks Inc.
http://www.WirelessNetworksInc.com
E-mail: [email protected]
Phone: 1.403.569-5687, Fax: 1.403.235-3964
------------------------------------------------------------------------


2002-10-04 21:46:13

by Alvin Oga

[permalink] [raw]
Subject: Re: RAID backup - mtx w/ tcl



hi ya kanoalani

are ya willing to release that mtx code ??
( well more like where can i find it )

i'd like to add it to the collection
http://www.Linux-Backup.net/app.gwif.html

thanx
alvin

On Fri, 4 Oct 2002, Kanoalani Withington wrote:

>
...

> I agree it's a total racket. I've spent an appalling amount of money on
> this stuff over the years considering how simple it is. Last year I
> finally built mtx, the open source tape library driver, and wrote my own
> software in tcl scripts for a new archiving system. It really is that
> simple, I don't know how they can charge so much for thier software,
> especially when some it is junk to begin with.
>
> -Kanoa
>
> >

2002-10-04 21:33:53

by Alvin Oga

[permalink] [raw]
Subject: Re: RAID backup - media


hi ya alan

yuppers...

those 320GB disks are not cheap?? but are floating around

- i use backups for "warm swap".... and dont care about it over time sorta
thing ... need to be able to keep TBs of data online as fast as possible
( few minutes ) to restore a dead/hacked box..

- for "fast turn around"... tapes have always been dayz of effort to
restore and too much of a headache to keep the tapes and heads clean
and test it more rigorously than having to test disks

- you already implicitly trusts disks to hold your data .. till the disk
dies ... hopefully due to ball bearing/lubricant/heat failure etc

fun stuff

have fun
alvin

On 4 Oct 2002, Alan Cox wrote:

> On Fri, 2002-10-04 at 12:20, Alvin Oga wrote:
> > we can build an 8-drive ( 120GB at $200ea ) or ( 160GB at $300>? each )
> > 1U box...about 0.960 - 1.28 TB each backup server ( 1U ) for under $2,500 in parts
> > + cost of raid setup/testing is up to the user
> > - am thinking the 1.6TB of storage for 10K lira(?) is too much
> >
> > i prefer disks to backup data.. so that its always a semi-warm backup
> > ( tapes have always been way tooo slow to find a file and to restore
>
> The problem with disks is you still have to archive them somewhere, and
> they are bulky. I also dont know what studies are available on the
> degradation of stored disk media over time.
>
> Capacity is not a problem, 3ware do a 12 channel sata card, with maxtor
> drives that comes in at 320x12 = 3.5Tb
>

2002-10-04 21:41:32

by Alvin Oga

[permalink] [raw]
Subject: Re: RAID backup


hi ya

mroe fun stuff

"drop tests" ... that person that tend to have "accidents" should be
kept outof the lab and away from hardware
( ceo/cfo/managers dont wanna spend un-necessary $2K - $10k
( because somebody dropped it ... or dropped a screw in a running
( system

on the other hand... that clumbsy tech/engineer is purrfect ina
testing lab or qa lab where they do require drop tests for "submarine"
applications

have fun
alvin

- i never dropped a disks in 20 years... or mb or anything
importnat/expensive

- dropped lots of screws, screwdrivers into "turned off systems" while
working on it

- dropped tape drives too and sometimes the tape drives itself ejects the
tapes too too fast
- clumbsy me for not being able to catch the 'flying tape" as
it ejects

- a months worth of daily tapes ( we keep 30 days worth of tape )
takes up as much room as a 1U box..

- a 1U box can hold about 3 months worth of compressed data
in our environment ... ( 3:1 compression on the average )


On Fri, 4 Oct 2002, Russell King wrote:

> On Fri, Oct 04, 2002 at 02:24:19PM +0100, Dr. David Alan Gilbert wrote:
> > Not sure about that; DLT tapes are pretty bulky themselves; I think the
> > difference between say a set of 4 DLT tapes and a single Maxtor 320 in
> > caddy would be minimal.
>
> The 4 DLT tapes would take up twice the room.
>
> > As for stored media, I think Maxtor are quoting
> > 1M hours MTTF - (I hate to think how you measure such a figure) - for
> > the 320G, and that is probably longer than I'd trust either the tape or
> > the drive to survive.
>
> However, drive in caddy or no caddy, the accidental drop test would
> probably be more favourable to the DLT tape surviving over the drive.
> Physical accidents do happen.
>
> --
> Russell King ([email protected]) The developer of ARM Linux
> http://www.arm.linux.org.uk/personal/aboutme.html
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

2002-10-04 21:54:46

by Eff Norwood

[permalink] [raw]
Subject: RE: RAID backup - mtx w/ tcl

I was going to ask the same thing. That would be great.

Thanks,

Eff Norwood

> hi ya kanoalani
>
> are ya willing to release that mtx code ??
> ( well more like where can i find it )
>
> i'd like to add it to the collection
> http://www.Linux-Backup.net/app.gwif.html
>
> thanx
> alvin
>
> On Fri, 4 Oct 2002, Kanoalani Withington wrote:
>
> >
> ...
>
> > I agree it's a total racket. I've spent an appalling amount of money on
> > this stuff over the years considering how simple it is. Last year I
> > finally built mtx, the open source tape library driver, and
> wrote my own
> > software in tcl scripts for a new archiving system. It really is that
> > simple, I don't know how they can charge so much for thier software,
> > especially when some it is junk to begin with.
> >
> > -Kanoa
> >
> > >
>
>
>


2002-10-05 03:07:52

by Chris Adams

[permalink] [raw]
Subject: Re: RAID backup

Once upon a time, Dr. David Alan Gilbert <[email protected]> said:
>As for stored media, I think Maxtor are quoting
>1M hours MTTF - (I hate to think how you measure such a figure) - for
>the 320G, and that is probably longer than I'd trust either the tape or
>the drive to survive.

DLT tapes are rated for a 30 year shelf life - I would not expect the
shelf life of a hard drive to be more than a few years due to
"stiction".
--
Chris Adams <[email protected]>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.