“How do you handle terabytes of data?” asks Nature Methods’ Chief Editor Veronique Kiermer at Methagora, the journal’s blog. That, she says, “is a question that more and more investigators must face, on a weekly basis” — and is also the topic of the journal’s July Editorial (Nature Methods 5, 577; 2008).
Kiermer notes that a single experimental run of light-sheet fluorescence microscopy can generate so much raw data that handling and storage become a challenge. The ‘1000 Genomes’ project asks participants to save raw sequencing data in public archives that are being developed in the United States and Europe.
There is no one-size-fits-all solution for the raw data challenge. The solutions depend on the number of users, the fraction of the experimental cost that data storage represents and the intrinsic value of raw data compared with processed data. Even within one technology, these parameters evolve with time and vary with applications. But, the Editorial notes, “saving only processed data is not recommended for any field”.
Nature 454, viii; 17 July 2008

*Owing to a technical error, the wrong entry for the 17 July Blogosphere was previously posted. This post is the correct one.