Challenges engage wider, diverse communities to competitively solve a specific problem in a given time period. The concept fosters collaboration between scientists through shared data and approaches.
The past two decades have seen an amazing growth in the ability to generate genomic data. However, with a few exceptions, acquisition of this type of data has so far failed to generate significant improvements in the treatment of human diseases. The Challenge concept engages the statistical, machine learning and computational biology communities to improve the analysis of clinical genomics data and fosters collaboration between scientists though shared data and approaches.
This approach has been used successfully in other research fields, and for the past 6 years the DREAM project has organized a series of challenges in Systems Biology intended to advance the field by organizing multiple groups to attack the same problems from different angles.
Here at Sage Bionetworks, we partnered up with DREAM to run our first Challenge – the Breast Cancer Prognosis Challenge (BCC) in 2012. This computational Challenge demonstrated what can happen when a Big Data biology project runs off of an open source data-sharing workspace like Synapse and succeeds in achieving a high level of participation from the world’s best data scientists. The BCC invited citizens and scientists alike to try their hand at building a computational model that would accurately predict breast cancer survival: the Challenge ended up attracting 350 participants from more than 35 countries who submitted a total of 1,700 models over a three-month period of time. The winning model’s predictive accuracy for breast cancer survival outperformed the best 60 models of a pre-competition group of expert programmers and also beat out the currently available best-in-class methodologies. And the winning team received the right to submit a pre-approved article about their winning model to Science Translational Medicine.
Based on the success of the BCC, Sage Bionetworks and DREAM merged in early 2013 in order to run open science computational Challenges that foster the broader collaboration of the research community and that provide a meaningful impact to both discovery and clinical research. We believe Challenges must be open source and encourage code-sharing to forge innovative computational models. And they should be run on a standardized and shared computational infrastructure like Synapse to enable participants to use code submitted by others in their own model building.
In our first year running DREAM Challenges together, we ran three DREAM8 Challenges in Summer, 2013 that attracted more than 600 participants working together in 253 teams and submitting more than 2000 predictions for scoring. Then at the tail end of DREAM8 in Dec 2013, we opened three DREAM8.5 Challenges (more on these below).
- The Alzheimer’s Disease (AD) Big Data DREAM Challenge #1: Predict the best biomarkers for early AD-related cognitive decline, and the mismatch between high amyloid levels and cognitive decline.
- The ICGC-TCGA DREAM Somatic Mutation Calling Challenge: Predict cancer-associated mutations from whole-genome sequencing data.
- The Rheumatoid Arthritis Responder Challenge: Predict which patients will not respond to anti-TNF therapy.
The DREAM8.5 Challenges tackle important questions related to cancer, rheumatoid arthritis and Alzheimer’s Disease. In our planning of DREAM8.5, we realized that the traditional DREAM schedule of three-month long Challenges opening in the spring and completing in the fall seems insufficient to process the massive amount of data being generated in disease areas like Alzheimer’s, arthritis and cancer. Therefore, the 8.5 “Challenge season” will extend until Summer, 2014, with best performers invited to co-author a Challenge-specific paper for submission to a scientific journal and to present their winning Challenge model at the 2014DREAM conference (date and location to be determined).
We are excited to journey forward maintaining DREAM’s high level of excellence in systems biology while also welcoming new DREAMers to help us innovate and extend the wisdom of the crowds into new arenas of human health.
To read more about the DREAM8 results or register for a DREAM8.5 Challenge, please go to https://www.synapse.org/#!Challenges:DREAM8.5