In the long awaited follow up to Breakin’ and Breakin’ 2, Ken Chen has released BreakDancer. As described in his Nature Methods article and a recent Genome Technology article, BreakDancer is not so much a movie as it is a bioinformatics program that can detect structural variation (insertions, deletions, inversions, and translocations) in genomes using paired-end read data. It can be used to detect structural variation in individual genomes, pools of genomes (like the low-coverage 1000 Genomes data), and tumor/normal samples with greater sensitivity and specificity than other structural variation detection algorithms.

President Obama by Glenn “‘President Obama has a deep-seated hatred of white people’ and 75 seconds later ‘I’m not saying President Obama doesn’t like white people’” Beck and me by neoprene nancy (not to be confused with Polythene Pam) in a comment on my recent post Another rich white guy sequences own genome about Stephen Quake sequencing his own genome. The entirety of her comment is “Another blogging guy inadvertently reveals racism.” I truly find it a strange and discomforting thing that people so quickly and easily throw out accusations of racism. It is almost as though racists are adopting the approach taken by hip hop artists with the N-word: adopt the word’s use to the point of overuse to diminish its derogatory connotation. Hopefully neoprene nancy was joking. In case she was not, let’s take the accusation apart. Since she does not call me socialist or sexist and herself uses the term “guy”, I assume she is only concerned with my reference to “white” and not “rich” or “guy”. Perhaps, like Glenn Beck would have us believe about President Obama, I am a self loather and have a deep-seated hatred of white people (for those of you wondering, you can see a picture of me). Or perhaps my dissatisfaction with the selection of another healthy, white male for whole genome sequencing has to do with the fact that it will provide essentially no scientific value. This is likely why the sequence analysis was so cursory and why it got published as a letter in Nature Biotechnology rather than an article in a top-tier journal: it is essentially just a proof of concept for the Helicos platform. While knowing his sequence has provided some health insights for Stephen Quake (see Quake Traits), it does not advance our understanding of the relationship between genotype (DNA) and phenotype (traits). It does not add to our understanding of natural human variation beyond that provided by the 1000 Genomes Project. It does not help us to understand variants associated with cancer or other common diseases. It does not help us to interpret the biological role of conserved sequences in the genome that are not in genes. In short, it is a novelty and should be called out for what it is, another rich, white guy sequencing his own genome.

By now it is well know that scientists are horrible people. It is clear from reports like that found last month in PLoS ONE that all scientists (OK, well more than 72%) are lying data falsifiers with questionable morals. This week we find out from the (n)ever insightful Sharon Begley of Newsweek that researchers are withholding life-saving information to further their careers. The selfish beasts! Just think of all the children whose lives could be saved if it weren’t for these terrible scientists. Certainly the few anecdotes she presents are indicative of the entire reality faced by scientists and medical practitioners and not just sour grapes from researchers with an inflated sense of the importance of their research. Certainly these “top tier” journals are publishing complete rubbish instead of these seminal papers. Everyone knows that tenure decision are based solely on the number of publications a PI has in Nature. It has nothing to do with how much grant money (and therefore overhead to the institution) the PI brings in.

Of course these mass murderers must be stopped and Ms. Begley says the new head of NIH is just the one to do it. As many readers of this blog will undoubtedly know, Francis Collins, former head of the National Human Genome Research Institute (NHGRI), has been rumored to be on the short list for NIH director. Now we must all ask ourselves, would Sharon Begley approve? With his penchant for pushing big projects like the Human Genome Project and the 1000 Genomes Project that have publications in prestigious journals, can he be trusted? Never mind his role in pushing for early release of human sequence data and driving GINA through the U.S. Congress. Prestige-hounds like Dr. Collins must be stopped!

(Yes, I’m being sarcastic. And, yes, I know publications and grant money are linked, but fairly weakly. Lots more people get grant money than are published in “top tier” journals. As Randy Newman sings, “it’s money that matters”.)

Of course, even though they were just three “pilot” projects, they still generated a lot of data and new variants. According to Paul Flicek of EBI, over 3.6 Tb of sequence in over 95 billion sequence reads have been submitted and made available on the 1000 Genomes FTP site. For some coverage of the new variants that have been found, check out this Times Online article and the GenomeWeb coverage of Gonçalo Abecasis’s talk at the Biology of Genomes meeting a few weeks ago.

Advances in DNA sequencing technologies over the past few years have led to data generation and processing rates that far outpace Moore’s Law and storage capacity improvements. As a result, there will come a time when one will no longer be able to “throw more money” at the problems presented by DNA sequencing, i.e., researchers will not be able to keep pace with data generation by purchasing more and more storage and computational nodes. Proposed sequencing platform improvements and the rapid rate of adoption of these technologies by labs large and small will only hasten the time when the old solutions will no longer apply. The history of freely shared sequence data through the NCBI and EBI Trace Archives transform the very difficult problem of massive sequence data generation into a problem of data generation and data sharing on a scale heretofore unimaginable. Over the last year, several organizations, e.g., MGED, NCI, Illumina, 1000 Genomes DCC, and NHGRI, have convened meetings to discuss the problems presented by the massive amounts of data generated by next-generation sequencing technologies. As prologue, brief overviews of these meetings will be presented along with approaches to dealing with massive data generation rates from other disciplines, e.g., high energy physics and high-resolution medical imaging. The Genome Center at Washington University in St. Louis, due to its large-scale sequencing operation and whole-genome analysis capabilities, experiences the difficulties presented by massively-parallel sequencing platforms acutely. To address the many challenges presented by the scale of data generation and requisite analysis, we have developed a multidisciplinary approach involving experts in biology, genomics, bioinformatics, computer science, information technology, and engineering. The resulting approach involves many techniques including intelligent compression and data reduction, data aging, archiving, parallelization, fault-tolerant workflows, scalable software frameworks, and multivariate/multi-genome visualization and comparison, which leverage and extend our laboratory information management system. This approach and its application to the sequencing and analysis of cancer samples will be presented.

It’s a lot to cover in 4 ft × 4 ft, but I’ll do my best. If you are going to be at Cold Spring Harbor, stop by and say hello.

I hope you enjoy the video. You may need to turn up the volume on your computer to hear it. I’ll post more about the meeting soon.

Dear All,

I’m pleased to provide everyone a stocking stuffer in the form of the first release of data from the 1000 Genomes project.

The preliminary list of SNPs for 4 of the high coverage individuals are now available on the EBI and NCBI 1000 Genomes FTP sites. Instructions on how to access the data can be found at http://www.1000genomes.org.

In addition, we have created a project specific genome browser to allow the data to be visualised in the context of genome annotations and data from other projects including the Venter and Watson genomes. The browser is based on the Ensembl platform and is available at http://browser.1000genomes.org. We will be making updates to the browser throughout January to ensure the 1000 Genomes data is visible by default and is easy to find (SNP tracks can now be found on the “Features” menu). I welcome any comments, questions or suggestions that that you have about the workings of the browser.

A long list of people worked very hard to get this done and any attempt to mention people will certainly miss some. However, I would like to specifically acknowledge Tom Blackwell, Goncalo Abecasis, Fiona Hyland, Zam Iqbal, Laura Clarke, Eugene Kulesha, Yuan Chen, Stephen Keenan, Fiona Cunningham, Justin Paschall, Martin Shumway,
Hoda Kouri and Steve Sherry.

All the very best for the holiday season.

Paul Flicek

Obviously, the three 1000 Genomes pilot projects have been a massive undertaking that has strained not only the production centers, but the IT and informatics infrastructures of the production and analysis centers. To date, over 3.8 terabases (3.8×10¹² or 3.8 trillion bases which is equivalent to about 1270 haploid human genomes) have been submitted as part of these pilot projects. The average SRF file submitted to the NCBI SRA stored 50 bytes of information per base; so the amount of data submitted so far is nearly 200 TB! At current broadband rates in the United States, it would take nearly 10 years to download all of this data (those still using 1600 baud modems may want to request they ship you the data on hard drives). Did I mention these are just the pilot projects?

The SRA is currently accepting 454 data in standard flowgram format (SFF) and Solexa in SRF format. Soon 454 and AB SOLiD will support the SRF format and submissions will commence in that format for those platforms. The SFF format contains the flowgrams (intensity per cycle at each spot), base calls, and base quality values. In other words, the SFF is very similar to the SCF format used for capillary sequencing data (except flowgrams are discrete whereas chromatograms are continuous). Also, NCBI (as recently discussed) has developed their own storage format for massively parallel sequencing data that they will also be accepting as a submission format within the next few months.

So what is an SRF? Well, it is basically just a container format, i.e., what you store in it is up to the implementation. Thus far, SRF has only been implemented for Illumina/Solexa data; so the rest of this post is specific to that platform and the data types that its implementation of the SRF format contains. The Solexa SRF implementation was done largely by James Bonfield at Sanger and is distributed as part of the io_lib package (now distributed separately from the Staden package). I would imagine that the SOLiD implementation will be very similar to the Solexa implementation. The 454 implementation will likely be very similar to the SFF already in wide use.

For the 1000 Genomes pilot projects, the 1000 Genomes Data Collection Center (DCC) is asking that we submit the “raw”, “processed”, and “base” data for each spot. Raw data are the intensity values (int) and noise (nse) values. Processed data are the processed intensity values (sig2) and four-channel quality values (prb). Base data are the base calls (the quality value is gotten from the prb for the called base). This results in about 50 bytes per base for the SRF. Compared to 2 bits per base, the minimum possible for DNA’s four letter alphabet, this is a 200-fold increase. So not only do these instruments generate a lot more data, we are storing more information per base now too. The average submission for an Solexa run is about 100 GB.

Why store all this extra information? Essentially, people do not trust/believe the data at this point. The quality values provided by these pipelines are not as reliable as those generated for capillary sequence data. Some people want the raw data so that they can develop and improve base calling/quality algorithms. Clearly you would not need all the 1000 Genomes data to develop such algorithms (although the technology changes at such a rate that you would likely want some rolling subset of the latest runs). Others want the raw data because they think they may want to go back and re-analyze data when better algorithms become available. For a wide variety of reasons (disk space, computational cost, network bandwidth, keeping pace with newly generated data), I doubt any such massive re-analysis will ever take place.