The deadline for entries to the OMOP Cup has passed.  The leaderboard shows unofficial results, and we will be working to determine the winners of each competition in the days to come.  Thanks very much to everyone who participated, and stay tuned for more information about OMOP's next steps.

The competition is closing at midnight Eastern daylight time tonight.  If you have any problems submitting, email omopcup@gmail.com and we'll work to remedy the situation.

Recently, a few contestants have identified some inconsistencies in our competition documentation and scoring system. After further exploration, we did discover minor issues that required modification. The scoring algorithm now properly excludes the drug-condition pairs in the OMOP_TRUE_RELATIONSHIPS.txt file. There was a typo in the formula for scoring Challenge 2 submissions that has been corrected. The proper calculation is a simple arithmetic mean for the mean average precision across years. The current scoring is consistent with the documentation provided.

In fairness for the competition, we have rescored all submissions and refreshed the leaderboard based on the corrected scoring system. Note, the leaderboard displays the score to the 7th decimal place, but the official score is based on the full precision calculated by the scoring algorithm. To be fully transparent, we have also updated the documentation (Rules , Challenge 1 , Challenge 2) and posted the source code used to grade the submissions, so everyone has a complete understanding of how the OMOP Cup will be evaluated from now until the end of the contest. We apologize for any inconvenience this has caused. If you have any questions, please contact us at omopcup@gmail.com .

The submission system is now reopen to accept submissions to be eligible for the Grand Prize. Thanks again to everyone who submitted for the Progress Prize, and good luck to everyone in the months ahead. The Grand Prize, with $15,000 in prize money, closes March 31, 2010.

After two months of entries, the Observational Medical Outcomes Partnership’s OMOP Cup has reached its first milestone with the announcement of the Progress Prize. Four awards totaling $5,000 were given out to participants who came up with the highest-performing methods that improve the state of the art in identifying adverse drug reactions in medical records. The competition brought together competitors from epidemiology, drug safety, statistics, and machine learning in a cross-disciplinary challenge. The Progress Prizes are only the first part of the OMOP Cup, which culminates with $15,000 in additional prizes in March 2010.

The OMOP Cup has two related challenges. Challenge 1 explores how well the method works when provided an entire longitudinal dataset, and the goal is to accurately classify which drugs are associated with which outcomes. The winners of the Progress Prize for the first challenge are David Vogel and Eric Gottschalk of Data Mining Solutions, with $2,500 in prize money. Robin Sabhnani of Carnegie Mellon University won $1,000 for second place. Challenge 2 evaluates the timeliness of detection of drug-event associations by having your methods run against data sequentially as it accumulates over time. Robin Sabhnani was in first place, worth $1,000. Lisa Friedland of the University of Massachusetts Amherst took home $500 for second place.

The Observational Medical Outcomes Partnership (http://omop.fnih.org) is a public-private partnership designed to help improve the monitoring of drugs for safety. The partnership began in late 2008 and is conducting a two-year research initiative to determine the contribution and utility of using existing healthcare databases to identify and evaluate safety issues of drugs already on the market. The new data sources have now become available but algorithmic progress has lagged. OMOP is conducting research on the feasibility and performance of a wide range of algorithms and statistical methods. We are also seeking new methods and the OMOP Cup is one of several efforts intended to stimulate interest in observational methods development.

We provided a large simulated dataset that resembles observational data that can be extracted from insurance claims or electronic medical records. The task is to identify relationships in the data between drugs and medical outcomes (adverse events). The goal is to develop methods that correctly identify true drug-event associations while minimizing false positive findings. Methods are evaluated by how closely they predict the known relationships that exist in the data.

The period for progress prize entries is now closed. We are currently in the process of analyzing and double-checking the results. The official announcement of winners will take place on December 5. The grand prize will open soon. Thanks very much to everyone who participated for your hard work.

The deadline for the progress prize is today, November 30, at 11:59pm Eastern time (5am GMT December 1). At that time, the submissions will close while we begin to take a look at the entries. The submissions will then reopen for the second phase of the competition, leading up to the grand prize in March.

Just a quick reminder that the deadline for OMOP Cup progress prize submissions is just 10 days away! You can submit one entry per day for each challenge. A total of $5000 is up for grabs for the progress prize.

I’ve added an explanation of some new benchmarks in the Methods Discussion section. As a reminder, these entries are produced by OMOP and not eligible for the competition.

A slight clarification has been made to the rules regarding the licensing of items submitted to the competition. OMOP intends to release all competition documents and results under the Apache public license at the close of the competition.

Welcome the the OMOP Cup. Registration is now open, and we have details and data for the challenges available.