CADDementia Banner

Outline

  • Team registration

  • Participants need to register a team via the challenge web site.

  • Anonymous registration is not allowed; you must use your real name, affiliation and email address.

  • Participants from institutions that contributed data to this challenge who do have access to (a part of) the provided data can participate in this challenge. Their results will be evaluated but cannot be included in the final ranking. The data contributing centers are: Erasmus MC, VUmc, and the University of Porto.

  • Data confidentiality

  • To finalize your registration and to get access to the data, you should email 1) a brief description of the proposed algorithm and 2) a signed agreement of data usage to bigr-caddementia@lists.erasmusmc.nl.

  • Teams are allowed to propose multiple algorithms with a maximum of five only if it is clear from the descriptions that these are distinctly different.

  • See below for the rules on data confidentiality.

  • Data download

  • Train and test data can be downloaded from XNAT.

  • The data exist of MRI data sets, demographic information and associated diagnostic labels of the training data.

  • New participants will get access to the 30 training data images. Participants can use these data to optimize their algorithm. After submission of a full methods paper, participants will receive access to the XNAT project with all data to prepare their challenge submission.

  • In addition to the provided data, methods can be trained and tuned on any suitable data (i.e. from the ADNI database).

  • Submitting paper

  • Every team writes a paper about one or more algorithms. This paper can be uploaded on this web site.

  • The paper should describe the used methods including a proper introduction and discussion. It should be formatted as a standard "MICCAI-style" paper describing the proposed algorithm. The maximum number of pages is 10.

  • The following items should be detailed in the workshop paper: 1) description of the used training data, 2) computation time per subject, 3) whether the algorithm is automatic, semi-automatic or mainly manual, and 4) results on the training data.

  • For every submission, this paper with the full description of the algorithm is required. If this description should still remain confidential (because the work is being submitted to a journal for publication for example), we can postpone the publication of the full description on the CADDementia website for a maximum of 1 year after submission of the classification results. Once the journal paper is published, we can also add a link to it.

  • More details on the paper requirements can be found on the Submit page. 

  • After submission of the paper, teams receive access to the full dataset.

  • Submitting classification outcomes

  • Teams submit classification outcomes on the test set via this web site

  • For participants to evaluate their training set results with the described measures, the evaluation scritps (Python) are made available.

  • Results

  • The results of all participating teams and the papers describing their algorithms will become available from this web site.

  • Reuse of proposed algorithms

  • For enabling reuse of the algorithms proposed in this challenge, participants are encouraged to write a practical guide on how to apply their algorithm, where possible including an executable of their method.

  • These practical guides are located on the Wiki. You will receive a login for this Wiki when you submit your results.

Data usage agreement

  • MRI data sets, demographic information and associated diagnostic labels of the training data sets provided by the organizers may not be given or distributed under any circumstances to persons other than the participant and the team as mentioned in the agreement of data usage.
  • The data of the evaluation framework may only be used for the evaluation of methods for computer-aided diagnosis dementia through this challenge.
  • When submitting results to CADDementia, you agree with presentation of these results on any medium including this web site and journal publications.
  • When publishing the obtained results, you have to apply to the following publication guidelines:
  • Include the following citation in the methods section of the manuscript: E.E. Bron, M. Smits, W.M. van der Flier, et al. Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia Challenge, NeuroImage, 2015
  • Acknowledge CADDementia in the acknowledgement section of the manuscript using language similar to: "*Data used in the preparation of this article were obtained from the CADDementia challenge (http://caddementia.grand-challenge.org)". *
  • Submit the manuscript to the CADDementia organizers prior to submitting to a journal. The organizers will check whether the data usage agreement is satisfied. The CADDementia organizers will maintain confidentiality of the manuscript and will complete the review within 2 weeks.
  • Notify the organizers if a manuscript is accepted for publication, allowing the organizers to keep track of all papers. The CADDementia-related papers are listed on this web site.