Since the Biology of Genomes meeting in early May, a tempest has been brewing. It is only in this last week that this tempest has gathered enough strength that it could no longer be contained by those who have chosen to stir it up. The esteemed Daniel MacArthur blogged and tweeted from the conference. This apparently caught the attention of the conference organizers and GenomeWeb. As journalists, the folks at GenomeWeb are required to follow CSHL’s media rules which require that journalists get the permission of a speaker before publishing any information from her talk. GenomeWeb saw a double standard when comparing what Daniel was allowed to do and what they were allowed to do. They then contacted CSHL. The initial write-up of the gathering storm in Science Insider characterized this contact as complaining. GenomeWeb characterized it as asking CSHL for clarification of their policy (in a comment on a response posted by Daniel in his blog, Genetic Future). Of course this attempt to, in effect, censor has only served to bring more attention to Daniel’s blog (the so-called Streisand effect), and has resulted in a number of responses from other bloggers like Anthony Fejes, DrugMonkey, and even GenomeWeb’s Daily Scan, comments (some quite passionate) on the Science Insider story, Daniel’s response, and FriendFeed, as well as a couple well-reasoned pieces on where the policy should head from here by Ed Yong and Andrew Maynard. Daniel himself provides a nice summary of it all in a follow-up post. With all that sound and fury, there is not much to add on the subject other than to say I suppose I am lucky that the 500 or so emails I had to pore through each night after the meeting ended at 10:30 or 11 p.m. prevented me from posting any commentary during the meeting (well, the emails plus the fact that I knew Daniel would do a better job than me).

Taking a step back, there is a larger double standard at play here than the distinction between professional journalists and peddlers of new media. Many of the conclusions around whether CSHL is right in restricting any type of journalist focus on the type of conference and the expectations that type of conference creates in the minds of the presenters. At a private, invitation-only conference, no publishing. At a breaking results conference like Biology of Genomes, get permission. At an open conference, anything goes. So then one might ask: why aren’t all conferences open? The whole notion that presenting something at a conference that has some understanding of respecting others’ unpublished work is a bit ridiculous (this point has been made by others, along with the fact that Biology of Genomes is over-subscribed every year; getting people in the door is not a problem). But I am not even going to debate that point. The more interesting question is: why aren’t all data and research released rapidly and freely available? Since the Bermuda Principles were agreed to in 1996, all genome sequencing centers have submitted their data, from raw sequence data to finished sequence to assemblies to annotation, to public repositories as quickly after generation as possible. These principles were reinforced by the Fort Lauderdale agreement in 2003 which added a provision that protected the production centers’ right to first publication. But as we have seen recently, that provision of the Fort Lauderdale agreement is not always enforced. As sequencing has moved into medical applications, the sequencing centers have taken great pains to release human sequence data in a responsible manner, but still rapidly. What’s more, they now also release the detected variants fully annotated and correlated with phenotypic information in protected access databases available to any researcher. As data that requires more and more analysis and significant human curation are made rapidly available well before publication, the production centers become ever more vulnerable to getting “scooped” on their hard won findings.

As Church and Hillier properly conclude in the above referenced article

Sequence data are now easier to produce, but decisions about timelines for data release, publication, and ownership and standards for assembly comparison and quality assessment, as well as the tools for managing and displaying these data, need considerable attention in order to best serve the entire community. (Emphasis mine)

This conclusion begets many questions. If the rapid release described in the Bermuda Principles still holds true, why does it only apply to large-scale sequencing centers? Many researchers are generating more sequence in a month than the Human Genome Project was able to produce in a year. As they continue to be allowed to perform pre-publication (as opposed to post-generation) data submission, why are they not being held to the same standard as the large-scale sequencing centers?

Stepping back further, does dumping all of those data, literally terabytes and terabytes, into public nucleotide repositories like the SRA and ERA as soon as it is generated still make sense? Who has the bandwidth to download and use it all? Mainly only those centers that are submitting it. For human data, a single instrument run contains enough data to identify an individual. Should there not be at least some provisions in place to allow data generators to properly assess and quality control their data?

The human reference has been published (with a recent update to GRCh37). The blueprint exists. Thus, many of the reasons underlying the conclusions of the Bermuda Principles are no longer applicable. So should those open access principles be applied more widely to other areas of biology and science at large or should they no longer apply to sequence data from a genome for which a reference exists? It is time to rethink the current policies and begin to apply them to all sequence generators. And people are doing just that. The double standard must end.

Several people have asked for an electronic copy of my poster, Maximizing Utility of Genome Sequence Data (pdf) (posted on the Internet Archive). As is hopefully clear from the poster, in addition to high-throughput sequencing, we now have high-throughput sequence analysis. After listening to Lynda Chin‘s talk on the first evening of the conference, which described the arduous process of translating a single putative cancer driver mutation to its function in the cell, one can’t help but feel we are just kicking the can down the road here. The alleviation of one bottleneck just creates another. This was the case with the PC, where after CPUs became faster and faster, other components, e.g., memory, network, and disk I/O, became bottlenecks. This has also been the case with high-throughput production sequencing. You buy more sequencers, you need more disk, then need more CPUs to analyze all the data, and then you need to upgrade your network to move all the data around. Now in genomics, we have a situation where we are able to generate lots of data and lots of variants which may play a role in cancer. How will we be able to determine the function of all these variants? What technologies are on the horizon that will enable high-throughput functional genomics?

Advances in DNA sequencing technologies over the past few years have led to data generation and processing rates that far outpace Moore’s Law and storage capacity improvements. As a result, there will come a time when one will no longer be able to “throw more money” at the problems presented by DNA sequencing, i.e., researchers will not be able to keep pace with data generation by purchasing more and more storage and computational nodes. Proposed sequencing platform improvements and the rapid rate of adoption of these technologies by labs large and small will only hasten the time when the old solutions will no longer apply. The history of freely shared sequence data through the NCBI and EBI Trace Archives transform the very difficult problem of massive sequence data generation into a problem of data generation and data sharing on a scale heretofore unimaginable. Over the last year, several organizations, e.g., MGED, NCI, Illumina, 1000 Genomes DCC, and NHGRI, have convened meetings to discuss the problems presented by the massive amounts of data generated by next-generation sequencing technologies. As prologue, brief overviews of these meetings will be presented along with approaches to dealing with massive data generation rates from other disciplines, e.g., high energy physics and high-resolution medical imaging. The Genome Center at Washington University in St. Louis, due to its large-scale sequencing operation and whole-genome analysis capabilities, experiences the difficulties presented by massively-parallel sequencing platforms acutely. To address the many challenges presented by the scale of data generation and requisite analysis, we have developed a multidisciplinary approach involving experts in biology, genomics, bioinformatics, computer science, information technology, and engineering. The resulting approach involves many techniques including intelligent compression and data reduction, data aging, archiving, parallelization, fault-tolerant workflows, scalable software frameworks, and multivariate/multi-genome visualization and comparison, which leverage and extend our laboratory information management system. This approach and its application to the sequencing and analysis of cancer samples will be presented.

It’s a lot to cover in 4 ft × 4 ft, but I’ll do my best. If you are going to be at Cold Spring Harbor, stop by and say hello.

Column-oriented database architecture is not a new idea, but it does seem well suited for the SRA. In traditional relational databases, a table has several columns of related items. For example, a “person” table might have a column for “first_name”, a column for “last_name”, and a column to hold a unique identifier (id), typically an integer. When you add an entry to the database, you add a row in the table. In the person example, you might add “Jane” in the first_name column, “Doe” in the last_name column, and a unique number in the id column. In a column-based database, you basically translate each column into its own two column table, one column for the data and one for the unique identifier. So in the person example you would still have two “columns”, but the columns would each store the appropriate name (first or last) and the id associated with that name. This architecture performs very well when you only retrieve one type of data at a time. Other advantages include that each column is fully indexed, each column compresses well (since each column contains only one type of data), and adding new data types (columns) is easy. One disadvantage is that every time you want to retrieve more than one data type, a “join” is required. So, for example, if you want to retrieve the sequence base calls and quality values for some number of reads, you will need to find that read separately in the base call column and the base quality column. All in all, it seems the advantages outweigh the disadvantages, especially considering the likely use cases for the data in the SRA.

The best part about their design is that they are planning to release the source code used to create and maintain the SRA freely (when we spoke, they had not settled on a free/open-source license). If they actually follow through with this (which they said would take a few months to remove NCBI-specific code), it would be of great benefit to researchers working with this data as the SRA format is likely more efficient for analyzing this data while the SRF is more efficient for transferring the data. If they don’t follow through, someone else will probably fill the gap as the column-oriented architecture seems to be the right idea and its implementation need not be difficult.

You can find out more detailed information about the SRA format from Valex’s web site which includes a link to documentation in PDF format and the presentation I saw in PowerPoint format.

After the data flow meeting was a meeting of the 1000 Genomes Steering Committee. The day and a half that ensued was filled with a lot of lively discussion. When all was said and done, one thing was clear: there are a lot of questions that need to be answered. The analysis group presented convincing results from simulations that indicated 2× coverage in a large number of individual genomes (Pilot 1) is probably not sufficient to detect the rare variants the project is going after (present in 1-2% of the population). The simulations indicated that the power of the study to detect such variants (at a constant cost, i.e., constant total amount of sequence generated) would be greatly enhanced by sequencing half as many people at 4× coverage. There was no firm decision on how to change the pilot (if at all), but going forward it is likely that some of the individuals in Pilot 1 will be sequenced up to 4× or even 8×. Thus, while the project may be named 1000 Genomes, exactly how many genomes we are going to sequence is yet to be determined.

Another issue that arose was the rapid development of the massively parallel sequencing technologies. These platforms (454 FLX, Illumina Genome Analyzer, and AB SOLiD) increase their throughput, improve their data quality, improve analysis software, etc. several times each year. Such dynamic platforms make the development of tools to analyze their data, e.g., align the data to a reference genome and detect variants, very difficult. The right platforms and tools today may not be the best next month or next year when the main project gets underway. This causes two major needs to come to the fore. First, experimental design will not end when the project starts. The experiment will need to be adjusted as capabilities and capacities change. Second, we will not only have to continually develop and refine tools throughout the project, we will need to develop frameworks to continually test and compare the tools that are available. It’s always fun to hit a moving target.

The meeting also discussed the ethical, legal, and social implications (ELSI) of the project. This discussion largely focused on which populations to sample for the project. Should we deepen our knowledge of individuals of Central European, African, and East Asian ancestry to aid in methodology development? Or should we broaden our knowledge of overall human variation by including fewer individuals from a larger number of populations? To be determined…