 |
 |
For the record, the amount of disk storage space per unit cost has doubled every 14 months for the last 30 years. It's an exponential relationship: www.mkomo.com/cost-per-gigabyte So data generated at a very high rate today, will be trivial to store in the near future. That's not to say it is cost free, of course ... but exponentially approaching free. I worked at a Supercomputing facility for 7 years. At that time whole rooms were filled with state-of-the-art tape archive robots that could hold an unimaginable amount of data: a whole terabyte. Today, of course, that same volume costs under 100 USD with much faster I/O ... and I have personal copies of everything I generated (even digitized, uncompressed analog video). To keep data backed up and online, of course costs something, but distributed/cloud computing is also changing that picture dramatically. I am curious to know: those who have Pilatus 6M, for example. How much data do you generate in a year? I suspect this is limited by beam intensity ... at the moment. Richard On Oct 18, 2011, at 6:52 AM, Chris Morris wrote: > Some crystals are hard to make, so storing all the data the best way to get reproducibility. On the other hand, no one needs more images of lysozyme. So using the same standard for every deposition doesn't sound right. > > The discussion should be held on the basis of overall cost to the research budget - not on the assumption that some costs can be externalised. It is too easy to say "you should store the images, in case I want to reprocess them sometime". IT isn't free, nor is it always cheaper than the associated experimental work. The key comparison is: > > Cost of growing new crystals + cost of beam line time > > With: > > Cost of storing images * probability of processing them again > > At present, detectors are improving more quickly than processing software. Sample preparation methods are also improving. These forces both press downward the probability that a particular image will ever be reprocessed. > > regards, > Chris