FLICKR, SHAURYParticipants in a Department of Defense competition aimed at improving detection of bioterrorism threats using DNA sequencing analysis say that the contest was poorly organized and scored, ScienceInsider reported.
Researchers were offered a $1 million prize for identifying the genetic code of individual species within masses of raw DNA sequence data, with the ultimate goal of creating better tools for detecting dangerous organisms. In the end, 103 groups or individuals submitted work. Of those, three teams have made the final round. However, participants complain that the rules of the contest and its scoring encouraged researchers to manipulate programs to fit opaque scoring rubrics.
Participants were scored on a 0-to-100 scale and were allowed to submit their programs multiple times to check their scores. “We spent the last month trying to get the [scoring] algorithm to tell us that we’ve done well, instead of actually doing the proper...
Another participant, Derrick Wood of the University of Maryland, College Park, who did not make the final round, complained that instructions were unclear. He was penalized for failing to mention the strains of some species he identified, while for other species, naming the strain did not appear to garner him points.
The Department of Defense pushed back the deadline multiple time and changed contest rules because no contestants had yet qualified to move on to the next round. Then, in the final week of the contest, participants were encouraged to start collaborating and mixing and matching their algorithms, leading to a mad rush for alliances.
“In terms of the goal of the contest as I understood it, I think it’s unlikely that the best algorithm will win,” Steven Salzberg of Johns Hopkins University, whose lab members participated in the contest, told ScienceInsider.
Interested in reading more?
The Scientist ARCHIVES
Become a Member of
