I use bzip2 as well. In addition, I generally store md5sums of the
images (before and after compression) because it's quicker to check the
hash for validity than load up the images - but this may be overkill.
Pete
Graeme Winter wrote:
> Dear Eugene,
>
> Personally I have a habit of using bzip2 for archival of data.
> Negative points: very slow. Positive points: universally supported,
> lossless. I have lots of data. To be honest most of it I keep in the
> native format.
>
> I expect to see plenty of comments of lossless vs. lossy compression now :o)
>
> N.B. well processed unmerged raw-from-integration .x, INTEGRATE.HKL,
> mosflm mtz represents pretty good lossy compression.
>
> Long term storage: depends on your definition of long term, which will
> also depend on what you want to do with them... I would guess
> useful-long-term would correspond to ~ 10 years to ~ 20 years tops. In
> the past I have written data to tape which I have never attempted to
> recover. Everything I now have on central file servers (raid systems)
> on local fast drives and local cheap USB drives which sit unplugged on
> my desk. Local fast drives fail (more frequently than the
> manufacturers would admit), raids fail (less frequently but more
> catastrophically), local cheap USB drives fail. I bank on them not all
> failing at the same time :o) esp. the ones I leave unplugged.
>
> The cheap drives store about 2TB in something the size of a paperback
> book and are relatively universal with USB connections. But they will
> fail one day. You weigh up how sad you will be at the loss against the
> cost of being more certain of not losing the data...
>
> Cheerio,
>
> Graeme
>
>
>
>
> On 11 March 2013 08:39, Eugene Osipov <[log in to unmask]> wrote:
>> Hello everyone,
>>
>> I have a couple of questions about images compression and storage:
>> 1)do someone use it in routine work and does it works well for them?
>> 2)I found this page by google:
>> http://bl831.als.lbl.gov/~jamesh/lossy_compression/
>> And want to ask about reports of usage of this program (of course if someone
>> already uses it)
>> 3)Is there any advice for long-term diffraction image storage?
>> Thank you for your attention,
>>
>> --
>> Eugene Osipov
>> Junior Research Scientist
>> Laboratory of Enzyme Engineering
>> A.N. Bach Institute of Biochemistry
>> Russian Academy of Sciences
>> Leninsky pr. 33, 119071 Moscow, Russia
>> e-mail: [log in to unmask]
|