 |
 |
Why not just upload it to proteindiffraction.org ? Or the SBGrid data bank (https://data.sbgrid.org/) ? Or both for "redundancy" ?
Yes, I did once do some calculations on what it would take to preserve data for tens of thousands of years, and the only proven storage medium for that timescale is clay tablets. Assuming 1 mm^3 is all you need to store one bit it comes to about $3000/GB.
Hard drives, however, are now down to $33/TB, which is comparable to a box of pipette tips, and takes up less space. LTO-6 tapes are $3/TB. So the cost of storage I don't think is any real burden, its the cost of managing that storage. If you buy a box of 12 TB bare drives, then you need to spend a lot of time and effort getting your data onto them, and then wondering if they will still work after a few years. Modern drives are much more reliable than they used to be, but maybe you want two copies? Or a parity disk? What you pay for when you buy a NAS, particularly a high-end NAS like NetApp is the cost and quality of management. Rolled into the price of the product is not just redundant bits and the wires to connect them, but a team of people who get paid to make sure your data are always safe and available.
The question then always comes down to cost/benefit. What is the consequence of data loss? What is the probability of data loss? And are you feeling lucky?
A few years ago I got a panicked email from a user whom I will not name, but this user had just been "Rupp-ed". As in Bernhard had found a deposit of theirs that look a lot like a fake structure, and asked about it. This deposition had been made ten years earlier, the student who did it had left science, and could not be reached. This left the PI holding the bag. Turns out the student had made a mistake and deposited Fcalc instead of Fobs. But how do you prove that? This user was VERY happy to find out that I still had their images on DVD. I was able to restore them and re-process them in about an hour.
Lucky? Perhaps. Not every beamline at every synchrotron backs up data, and not every DVD I've written can be read back. About 3000 images are still unrecoverable from those days. On the other hand, there are other beamlines who make a point of destroying any traces of user data as part of their data protection plan. Most, I think, are middle-of-the-road with a data retention policy like "we'll do what we can, but can't promise anything". Even at the same synchrotron policies can vary from beamline to beamline. So again: do you feel lucky? Do you?
-James Holton
MAD Scientist
On 7/13/2018 2:30 AM, Sergei Strelkov wrote:
Dear All,
I believe this question may be of some interest.
In the past, we always stored all raw data ever collected by the lab.
With the recent advances, such as
(a) automated/on-the-fly processing offered by some (European) synchrotrons, and
(b) an ongoing discussion on centralized raw data archiving,
I wonder if it is time to revise the strict policy of keeping all data
(before we invest in a new NAS system... )
Best wishes,
Sergei
Prof. Sergei V. Strelkov Laboratory for Biocrystallography Department of Pharmaceutical Sciences, KU Leuven
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1= CCP4BB&A=1
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1= CCP4BB&A=1
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1