The "Benchmarking Open Problems in Single Cell Analysis" workshop aims to address critical challenges in the field by fostering community engagement in the development of robust benchmarks. This 90-minute session will introduce participants to the core mission of the Open Problems in Single-Cell Analysis, followed by an interactive session where we will build a new benchmark from scratch. As part of this tutorial, participants not only learn about the technical aspects of setting up a benchmark within the Open Problems framework, but will also learn about essential best practices in benchmarking computational methods.
Content
The workshop will commence with a 20-minute introduction, outlining the goals and importance of the Open Problems initiative in single-cell analysis. This segment will focus on the role of benchmarking in improving reproducibility and reliability in computational methods.
Following the introduction, the 70-minute interactive session will guide participants through the process of designing and developing a benchmark.
- Problem identification
- Experimental design
- Dataset selection and processing
- Metric selection and implementation
- Implement methods
- Implement control methods
- Essential best practices for benchmarking computational methods
By the end of the workshop, participants will have practical experience in creating benchmarks and an understanding of best practices that can be applied to their research projects.
Update
The inaugural scverse conference was a great success! Thanks to all the participants for their enthusiasm and engagement.
