Getting started

To get started with contributing to OpenProblems, you’ll need to fork and clone the OpenProblems repository to your local machine.

Step 1: Create a fork

Go to the OpenProblems repository at https://github.com/openproblems-bio/openproblems-v2 and click on the “Fork” button in the top right corner of the page.

This will create a copy of the repository under your GitHub account.

Tip

While you’re at it, why not click the “Star” button as well?

Step 2: Clone the repository

To clone this forked repository to your local machine, copy the URL of the forked repository by clicking the green “Code” button and selecting HTTPS or SSH.

In your terminal or command prompt, navigate to the directory where you want to clone the repository and enter the following command:

git clone <forked repository URL> openproblems-v2
cd openproblems-v2

This will download a copy of the repository to your local machine. You can now make changes to the code, add new functionality, and commit your changes.

Step 3: Download test resources

You will also need to download the test resources by running the following command. These are needed for testing the existing components and can be used for developing new unit tests. From the repository root, run:

viash run src/common/sync_test_resources/config.vsh.yaml

Output

Completed 256.0 KiB/75.3 MiB (1.8 MiB/s) with 73 file(s) remaining
Completed 256.1 KiB/75.3 MiB (1.8 MiB/s) with 73 file(s) remaining
download: s3://openproblems-data/resources_test/batch_integration/pancreas/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/batch_integration/pancreas/state.yaml
Completed 256.1 KiB/75.3 MiB (1.8 MiB/s) with 72 file(s) remaining
Completed 512.1 KiB/75.3 MiB (3.5 MiB/s) with 72 file(s) remaining
Completed 768.1 KiB/75.3 MiB (5.2 MiB/s) with 72 file(s) remaining
Completed 1.0 MiB/75.3 MiB (6.9 MiB/s) with 72 file(s) remaining  
Completed 1.3 MiB/75.3 MiB (8.6 MiB/s) with 72 file(s) remaining  
Completed 1.3 MiB/75.3 MiB (8.8 MiB/s) with 72 file(s) remaining  
download: s3://openproblems-data/resources_test/batch_integration/pancreas/integrated_graph.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/batch_integration/pancreas/integrated_graph.h5ad
Completed 1.3 MiB/75.3 MiB (8.8 MiB/s) with 71 file(s) remaining
Completed 1.5 MiB/75.3 MiB (10.3 MiB/s) with 71 file(s) remaining
Completed 1.8 MiB/75.3 MiB (11.8 MiB/s) with 71 file(s) remaining
Completed 2.0 MiB/75.3 MiB (13.0 MiB/s) with 71 file(s) remaining
Completed 2.0 MiB/75.3 MiB (12.8 MiB/s) with 71 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/dataset_meta.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/dataset_meta.yaml
Completed 2.0 MiB/75.3 MiB (12.8 MiB/s) with 70 file(s) remaining
Completed 2.3 MiB/75.3 MiB (14.2 MiB/s) with 70 file(s) remaining
Completed 2.5 MiB/75.3 MiB (15.6 MiB/s) with 70 file(s) remaining
Completed 2.8 MiB/75.3 MiB (17.0 MiB/s) with 70 file(s) remaining
Completed 2.8 MiB/75.3 MiB (17.2 MiB/s) with 70 file(s) remaining
download: s3://openproblems-data/resources_test/batch_integration/pancreas/integrated_feature.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/batch_integration/pancreas/integrated_feature.h5ad
Completed 2.8 MiB/75.3 MiB (17.2 MiB/s) with 69 file(s) remaining
Completed 3.1 MiB/75.3 MiB (16.9 MiB/s) with 69 file(s) remaining
Completed 3.3 MiB/75.3 MiB (18.2 MiB/s) with 69 file(s) remaining
Completed 3.6 MiB/75.3 MiB (19.4 MiB/s) with 69 file(s) remaining
Completed 3.8 MiB/75.3 MiB (20.7 MiB/s) with 69 file(s) remaining
Completed 4.1 MiB/75.3 MiB (21.9 MiB/s) with 69 file(s) remaining
Completed 4.3 MiB/75.3 MiB (22.8 MiB/s) with 69 file(s) remaining
Completed 4.5 MiB/75.3 MiB (23.6 MiB/s) with 69 file(s) remaining
download: s3://openproblems-data/resources_test/batch_integration/pancreas/integrated_embedding.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/batch_integration/pancreas/integrated_embedding.h5ad
Completed 4.5 MiB/75.3 MiB (23.6 MiB/s) with 68 file(s) remaining
Completed 4.6 MiB/75.3 MiB (23.8 MiB/s) with 68 file(s) remaining
download: s3://openproblems-data/resources_test/batch_integration/pancreas/dataset.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/batch_integration/pancreas/dataset.h5ad
Completed 4.6 MiB/75.3 MiB (23.8 MiB/s) with 67 file(s) remaining
Completed 4.8 MiB/75.3 MiB (24.0 MiB/s) with 67 file(s) remaining
download: s3://openproblems-data/resources_test/common/bmmc_multiome_starter/dataset_rna.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/bmmc_multiome_starter/dataset_rna.h5ad
Completed 4.8 MiB/75.3 MiB (24.0 MiB/s) with 66 file(s) remaining
Completed 5.0 MiB/75.3 MiB (24.8 MiB/s) with 66 file(s) remaining
Completed 5.3 MiB/75.3 MiB (25.9 MiB/s) with 66 file(s) remaining
Completed 5.5 MiB/75.3 MiB (27.0 MiB/s) with 66 file(s) remaining
Completed 5.8 MiB/75.3 MiB (28.1 MiB/s) with 66 file(s) remaining
Completed 6.0 MiB/75.3 MiB (29.2 MiB/s) with 66 file(s) remaining
Completed 6.3 MiB/75.3 MiB (30.3 MiB/s) with 66 file(s) remaining
Completed 6.5 MiB/75.3 MiB (31.1 MiB/s) with 66 file(s) remaining
Completed 6.8 MiB/75.3 MiB (32.1 MiB/s) with 66 file(s) remaining
Completed 7.0 MiB/75.3 MiB (33.2 MiB/s) with 66 file(s) remaining
Completed 7.3 MiB/75.3 MiB (34.3 MiB/s) with 66 file(s) remaining
Completed 7.3 MiB/75.3 MiB (34.0 MiB/s) with 66 file(s) remaining
Completed 7.3 MiB/75.3 MiB (34.0 MiB/s) with 66 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/state.yaml
Completed 7.3 MiB/75.3 MiB (34.0 MiB/s) with 65 file(s) remaining
Completed 7.5 MiB/75.3 MiB (34.6 MiB/s) with 65 file(s) remaining
download: s3://openproblems-data/resources_test/common/bmmc_cite_starter/dataset_rna.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/bmmc_cite_starter/dataset_rna.h5ad
Completed 7.5 MiB/75.3 MiB (34.6 MiB/s) with 64 file(s) remaining
Completed 7.8 MiB/75.3 MiB (34.9 MiB/s) with 64 file(s) remaining
Completed 8.0 MiB/75.3 MiB (35.9 MiB/s) with 64 file(s) remaining
Completed 8.3 MiB/75.3 MiB (36.9 MiB/s) with 64 file(s) remaining
Completed 8.5 MiB/75.3 MiB (37.6 MiB/s) with 64 file(s) remaining
Completed 8.8 MiB/75.3 MiB (38.6 MiB/s) with 64 file(s) remaining
Completed 9.0 MiB/75.3 MiB (39.4 MiB/s) with 64 file(s) remaining
Completed 9.3 MiB/75.3 MiB (40.3 MiB/s) with 64 file(s) remaining
Completed 9.5 MiB/75.3 MiB (41.2 MiB/s) with 64 file(s) remaining
Completed 9.8 MiB/75.3 MiB (41.9 MiB/s) with 64 file(s) remaining
Completed 10.0 MiB/75.3 MiB (42.9 MiB/s) with 64 file(s) remaining
Completed 10.3 MiB/75.3 MiB (43.8 MiB/s) with 64 file(s) remaining
Completed 10.4 MiB/75.3 MiB (43.9 MiB/s) with 64 file(s) remaining
Completed 10.7 MiB/75.3 MiB (44.8 MiB/s) with 64 file(s) remaining
Completed 10.9 MiB/75.3 MiB (45.8 MiB/s) with 64 file(s) remaining
Completed 11.2 MiB/75.3 MiB (46.5 MiB/s) with 64 file(s) remaining
download: s3://openproblems-data/resources_test/batch_integration/pancreas/solution.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/batch_integration/pancreas/solution.h5ad
Completed 11.2 MiB/75.3 MiB (46.5 MiB/s) with 63 file(s) remaining
Completed 11.4 MiB/75.3 MiB (47.2 MiB/s) with 63 file(s) remaining
Completed 11.5 MiB/75.3 MiB (47.3 MiB/s) with 63 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/dataset.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/dataset.h5ad
Completed 11.5 MiB/75.3 MiB (47.3 MiB/s) with 62 file(s) remaining
Completed 11.8 MiB/75.3 MiB (48.0 MiB/s) with 62 file(s) remaining
Completed 12.0 MiB/75.3 MiB (48.9 MiB/s) with 62 file(s) remaining
Completed 12.3 MiB/75.3 MiB (49.0 MiB/s) with 62 file(s) remaining
Completed 12.4 MiB/75.3 MiB (49.3 MiB/s) with 62 file(s) remaining
Completed 12.6 MiB/75.3 MiB (50.1 MiB/s) with 62 file(s) remaining
download: s3://openproblems-data/resources_test/common/bmmc_cite_starter/dataset_adt.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/bmmc_cite_starter/dataset_adt.h5ad
Completed 12.6 MiB/75.3 MiB (50.1 MiB/s) with 61 file(s) remaining
Completed 12.9 MiB/75.3 MiB (50.6 MiB/s) with 61 file(s) remaining
Completed 13.1 MiB/75.3 MiB (51.4 MiB/s) with 61 file(s) remaining
Completed 13.4 MiB/75.3 MiB (52.3 MiB/s) with 61 file(s) remaining
Completed 13.6 MiB/75.3 MiB (53.0 MiB/s) with 61 file(s) remaining
Completed 13.9 MiB/75.3 MiB (53.5 MiB/s) with 61 file(s) remaining
Completed 14.1 MiB/75.3 MiB (54.4 MiB/s) with 61 file(s) remaining
Completed 14.4 MiB/75.3 MiB (55.3 MiB/s) with 61 file(s) remaining
Completed 14.6 MiB/75.3 MiB (55.9 MiB/s) with 61 file(s) remaining
Completed 14.9 MiB/75.3 MiB (56.7 MiB/s) with 61 file(s) remaining
Completed 15.1 MiB/75.3 MiB (57.4 MiB/s) with 61 file(s) remaining
Completed 15.4 MiB/75.3 MiB (58.3 MiB/s) with 61 file(s) remaining
Completed 15.6 MiB/75.3 MiB (59.0 MiB/s) with 61 file(s) remaining
Completed 15.9 MiB/75.3 MiB (59.7 MiB/s) with 61 file(s) remaining
Completed 16.1 MiB/75.3 MiB (60.5 MiB/s) with 61 file(s) remaining
Completed 16.4 MiB/75.3 MiB (61.3 MiB/s) with 61 file(s) remaining
Completed 16.4 MiB/75.3 MiB (61.5 MiB/s) with 61 file(s) remaining
Completed 16.7 MiB/75.3 MiB (62.3 MiB/s) with 61 file(s) remaining
Completed 16.9 MiB/75.3 MiB (62.8 MiB/s) with 61 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/raw.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/raw.h5ad
Completed 16.9 MiB/75.3 MiB (62.8 MiB/s) with 60 file(s) remaining
Completed 16.9 MiB/75.3 MiB (62.1 MiB/s) with 60 file(s) remaining
download: s3://openproblems-data/resources_test/common/task_metadata/method_info.json to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/task_metadata/method_info.json
Completed 16.9 MiB/75.3 MiB (62.1 MiB/s) with 59 file(s) remaining
Completed 16.9 MiB/75.3 MiB (61.4 MiB/s) with 59 file(s) remaining
Completed 17.2 MiB/75.3 MiB (62.2 MiB/s) with 59 file(s) remaining
download: s3://openproblems-data/resources_test/common/scicar_cell_lines/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/scicar_cell_lines/state.yaml
Completed 17.2 MiB/75.3 MiB (62.2 MiB/s) with 58 file(s) remaining
Completed 17.4 MiB/75.3 MiB (62.0 MiB/s) with 58 file(s) remaining
Completed 17.6 MiB/75.3 MiB (62.5 MiB/s) with 58 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/hvg.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/hvg.h5ad
Completed 17.6 MiB/75.3 MiB (62.5 MiB/s) with 57 file(s) remaining
Completed 17.9 MiB/75.3 MiB (62.9 MiB/s) with 57 file(s) remaining
Completed 18.1 MiB/75.3 MiB (63.8 MiB/s) with 57 file(s) remaining
Completed 18.4 MiB/75.3 MiB (64.2 MiB/s) with 57 file(s) remaining
Completed 18.6 MiB/75.3 MiB (64.9 MiB/s) with 57 file(s) remaining
Completed 18.9 MiB/75.3 MiB (65.5 MiB/s) with 57 file(s) remaining
Completed 19.1 MiB/75.3 MiB (66.2 MiB/s) with 57 file(s) remaining
Completed 19.2 MiB/75.3 MiB (66.4 MiB/s) with 57 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/normalized.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/normalized.h5ad
Completed 19.2 MiB/75.3 MiB (66.4 MiB/s) with 56 file(s) remaining
Completed 19.4 MiB/75.3 MiB (66.9 MiB/s) with 56 file(s) remaining
Completed 19.7 MiB/75.3 MiB (67.6 MiB/s) with 56 file(s) remaining
Completed 19.9 MiB/75.3 MiB (67.6 MiB/s) with 56 file(s) remaining
Completed 20.1 MiB/75.3 MiB (68.4 MiB/s) with 56 file(s) remaining
Completed 20.4 MiB/75.3 MiB (69.1 MiB/s) with 56 file(s) remaining
Completed 20.4 MiB/75.3 MiB (69.0 MiB/s) with 56 file(s) remaining
download: s3://openproblems-data/resources_test/common/task_metadata/input_git_sha.json to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/task_metadata/input_git_sha.json
Completed 20.4 MiB/75.3 MiB (69.0 MiB/s) with 55 file(s) remaining
download: s3://openproblems-data/resources_test/common/scicar_cell_lines/dataset_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/scicar_cell_lines/dataset_mod2.h5ad
Completed 20.4 MiB/75.3 MiB (69.0 MiB/s) with 54 file(s) remaining
Completed 20.4 MiB/75.3 MiB (68.1 MiB/s) with 54 file(s) remaining
download: s3://openproblems-data/resources_test/denoising/pancreas/score.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/denoising/pancreas/score.h5ad
Completed 20.4 MiB/75.3 MiB (68.1 MiB/s) with 53 file(s) remaining
Completed 20.7 MiB/75.3 MiB (68.5 MiB/s) with 53 file(s) remaining
Completed 20.9 MiB/75.3 MiB (69.1 MiB/s) with 53 file(s) remaining
Completed 21.2 MiB/75.3 MiB (69.7 MiB/s) with 53 file(s) remaining
Completed 21.4 MiB/75.3 MiB (70.3 MiB/s) with 53 file(s) remaining
Completed 21.4 MiB/75.3 MiB (69.8 MiB/s) with 53 file(s) remaining
download: s3://openproblems-data/resources_test/denoising/pancreas/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/denoising/pancreas/state.yaml
Completed 21.4 MiB/75.3 MiB (69.8 MiB/s) with 52 file(s) remaining
Completed 21.7 MiB/75.3 MiB (70.0 MiB/s) with 52 file(s) remaining
Completed 21.9 MiB/75.3 MiB (70.8 MiB/s) with 52 file(s) remaining
Completed 22.2 MiB/75.3 MiB (71.3 MiB/s) with 52 file(s) remaining
Completed 22.4 MiB/75.3 MiB (71.9 MiB/s) with 52 file(s) remaining
Completed 22.7 MiB/75.3 MiB (72.6 MiB/s) with 52 file(s) remaining
Completed 22.9 MiB/75.3 MiB (73.3 MiB/s) with 52 file(s) remaining
Completed 23.2 MiB/75.3 MiB (73.7 MiB/s) with 52 file(s) remaining
Completed 23.4 MiB/75.3 MiB (74.4 MiB/s) with 52 file(s) remaining
Completed 23.6 MiB/75.3 MiB (74.8 MiB/s) with 52 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/pca.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/pca.h5ad
Completed 23.6 MiB/75.3 MiB (74.8 MiB/s) with 51 file(s) remaining
Completed 23.6 MiB/75.3 MiB (74.1 MiB/s) with 51 file(s) remaining
Completed 23.6 MiB/75.3 MiB (73.9 MiB/s) with 51 file(s) remaining
download: s3://openproblems-data/resources_test/dimensionality_reduction/pancreas/score.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/dimensionality_reduction/pancreas/score.h5ad
Completed 23.6 MiB/75.3 MiB (73.9 MiB/s) with 50 file(s) remaining
Completed 23.9 MiB/75.3 MiB (74.4 MiB/s) with 50 file(s) remaining
download: s3://openproblems-data/resources_test/dimensionality_reduction/pancreas/embedding.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/dimensionality_reduction/pancreas/embedding.h5ad
Completed 23.9 MiB/75.3 MiB (74.4 MiB/s) with 49 file(s) remaining
Completed 24.1 MiB/75.3 MiB (74.4 MiB/s) with 49 file(s) remaining
Completed 24.3 MiB/75.3 MiB (74.5 MiB/s) with 49 file(s) remaining
Completed 24.5 MiB/75.3 MiB (74.9 MiB/s) with 49 file(s) remaining
Completed 24.8 MiB/75.3 MiB (75.5 MiB/s) with 49 file(s) remaining
Completed 25.0 MiB/75.3 MiB (76.1 MiB/s) with 49 file(s) remaining
Completed 25.3 MiB/75.3 MiB (76.7 MiB/s) with 49 file(s) remaining
Completed 25.5 MiB/75.3 MiB (77.3 MiB/s) with 49 file(s) remaining
Completed 25.8 MiB/75.3 MiB (77.7 MiB/s) with 49 file(s) remaining
Completed 26.0 MiB/75.3 MiB (78.4 MiB/s) with 49 file(s) remaining
Completed 26.0 MiB/75.3 MiB (78.2 MiB/s) with 49 file(s) remaining
Completed 26.3 MiB/75.3 MiB (78.8 MiB/s) with 49 file(s) remaining
Completed 26.5 MiB/75.3 MiB (79.4 MiB/s) with 49 file(s) remaining
download: s3://openproblems-data/resources_test/dimensionality_reduction/pancreas/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/dimensionality_reduction/pancreas/state.yaml
Completed 26.5 MiB/75.3 MiB (79.4 MiB/s) with 48 file(s) remaining
Completed 26.6 MiB/75.3 MiB (79.4 MiB/s) with 48 file(s) remaining
download: s3://openproblems-data/resources_test/common/pancreas/knn.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/pancreas/knn.h5ad
Completed 26.6 MiB/75.3 MiB (79.4 MiB/s) with 47 file(s) remaining
Completed 26.9 MiB/75.3 MiB (79.7 MiB/s) with 47 file(s) remaining
Completed 27.1 MiB/75.3 MiB (80.2 MiB/s) with 47 file(s) remaining
Completed 27.1 MiB/75.3 MiB (79.8 MiB/s) with 47 file(s) remaining
Completed 27.4 MiB/75.3 MiB (80.5 MiB/s) with 47 file(s) remaining
download: s3://openproblems-data/resources_test/common/scicar_cell_lines/dataset_metadata_mod2.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/scicar_cell_lines/dataset_metadata_mod2.yaml
Completed 27.4 MiB/75.3 MiB (80.5 MiB/s) with 46 file(s) remaining
Completed 27.6 MiB/75.3 MiB (80.8 MiB/s) with 46 file(s) remaining
Completed 27.9 MiB/75.3 MiB (81.4 MiB/s) with 46 file(s) remaining
Completed 28.1 MiB/75.3 MiB (81.9 MiB/s) with 46 file(s) remaining
Completed 28.4 MiB/75.3 MiB (82.4 MiB/s) with 46 file(s) remaining
Completed 28.6 MiB/75.3 MiB (83.0 MiB/s) with 46 file(s) remaining
Completed 28.9 MiB/75.3 MiB (83.6 MiB/s) with 46 file(s) remaining
Completed 29.1 MiB/75.3 MiB (83.8 MiB/s) with 46 file(s) remaining
Completed 29.4 MiB/75.3 MiB (84.4 MiB/s) with 46 file(s) remaining
Completed 29.6 MiB/75.3 MiB (84.7 MiB/s) with 46 file(s) remaining
Completed 29.9 MiB/75.3 MiB (85.2 MiB/s) with 46 file(s) remaining
Completed 30.1 MiB/75.3 MiB (85.7 MiB/s) with 46 file(s) remaining
Completed 30.4 MiB/75.3 MiB (86.2 MiB/s) with 46 file(s) remaining
Completed 30.6 MiB/75.3 MiB (86.7 MiB/s) with 46 file(s) remaining
Completed 30.9 MiB/75.3 MiB (87.3 MiB/s) with 46 file(s) remaining
download: s3://openproblems-data/resources_test/common/bmmc_multiome_starter/dataset_atac.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/bmmc_multiome_starter/dataset_atac.h5ad
Completed 30.9 MiB/75.3 MiB (87.3 MiB/s) with 45 file(s) remaining
Completed 31.1 MiB/75.3 MiB (87.0 MiB/s) with 45 file(s) remaining
Completed 31.1 MiB/75.3 MiB (86.9 MiB/s) with 45 file(s) remaining
download: s3://openproblems-data/resources_test/denoising/pancreas/test.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/denoising/pancreas/test.h5ad
Completed 31.1 MiB/75.3 MiB (86.9 MiB/s) with 44 file(s) remaining
Completed 31.3 MiB/75.3 MiB (85.9 MiB/s) with 44 file(s) remaining
download: s3://openproblems-data/resources_test/label_projection/pancreas/prediction.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/label_projection/pancreas/prediction.h5ad
Completed 31.3 MiB/75.3 MiB (85.9 MiB/s) with 43 file(s) remaining
Completed 31.6 MiB/75.3 MiB (86.1 MiB/s) with 43 file(s) remaining
Completed 31.8 MiB/75.3 MiB (86.7 MiB/s) with 43 file(s) remaining
Completed 32.1 MiB/75.3 MiB (87.0 MiB/s) with 43 file(s) remaining
Completed 32.3 MiB/75.3 MiB (87.6 MiB/s) with 43 file(s) remaining
Completed 32.4 MiB/75.3 MiB (87.7 MiB/s) with 43 file(s) remaining
download: s3://openproblems-data/resources_test/denoising/pancreas/train.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/denoising/pancreas/train.h5ad
Completed 32.4 MiB/75.3 MiB (87.7 MiB/s) with 42 file(s) remaining
Completed 32.6 MiB/75.3 MiB (87.8 MiB/s) with 42 file(s) remaining
Completed 32.6 MiB/75.3 MiB (87.1 MiB/s) with 42 file(s) remaining
download: s3://openproblems-data/resources_test/common/scicar_cell_lines/dataset_metadata_mod1.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/scicar_cell_lines/dataset_metadata_mod1.yaml
Completed 32.6 MiB/75.3 MiB (87.1 MiB/s) with 41 file(s) remaining
Completed 32.9 MiB/75.3 MiB (87.3 MiB/s) with 41 file(s) remaining
Completed 33.1 MiB/75.3 MiB (87.8 MiB/s) with 41 file(s) remaining
Completed 33.4 MiB/75.3 MiB (88.4 MiB/s) with 41 file(s) remaining
Completed 33.5 MiB/75.3 MiB (88.5 MiB/s) with 41 file(s) remaining
download: s3://openproblems-data/resources_test/dimensionality_reduction/pancreas/dataset.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/dimensionality_reduction/pancreas/dataset.h5ad
Completed 33.5 MiB/75.3 MiB (88.5 MiB/s) with 40 file(s) remaining
Completed 33.7 MiB/75.3 MiB (88.6 MiB/s) with 40 file(s) remaining
Completed 34.0 MiB/75.3 MiB (89.1 MiB/s) with 40 file(s) remaining
Completed 34.2 MiB/75.3 MiB (89.6 MiB/s) with 40 file(s) remaining
Completed 34.5 MiB/75.3 MiB (90.1 MiB/s) with 40 file(s) remaining
Completed 34.7 MiB/75.3 MiB (90.6 MiB/s) with 40 file(s) remaining
download: s3://openproblems-data/resources_test/denoising/pancreas/denoised.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/denoising/pancreas/denoised.h5ad
Completed 34.7 MiB/75.3 MiB (90.6 MiB/s) with 39 file(s) remaining
Completed 34.7 MiB/75.3 MiB (89.5 MiB/s) with 39 file(s) remaining
download: s3://openproblems-data/resources_test/label_projection/pancreas/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/label_projection/pancreas/state.yaml
Completed 34.7 MiB/75.3 MiB (89.5 MiB/s) with 38 file(s) remaining
Completed 34.9 MiB/75.3 MiB (88.1 MiB/s) with 38 file(s) remaining
download: s3://openproblems-data/resources_test/label_projection/pancreas/test.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/label_projection/pancreas/test.h5ad
Completed 34.9 MiB/75.3 MiB (88.1 MiB/s) with 37 file(s) remaining
Completed 35.1 MiB/75.3 MiB (88.2 MiB/s) with 37 file(s) remaining
download: s3://openproblems-data/resources_test/label_projection/pancreas/score.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/label_projection/pancreas/score.h5ad
Completed 35.1 MiB/75.3 MiB (88.2 MiB/s) with 36 file(s) remaining
Completed 35.4 MiB/75.3 MiB (87.1 MiB/s) with 36 file(s) remaining
Completed 35.6 MiB/75.3 MiB (87.2 MiB/s) with 36 file(s) remaining
Completed 35.6 MiB/75.3 MiB (87.2 MiB/s) with 36 file(s) remaining
download: s3://openproblems-data/resources_test/match_modalities/scicar_cell_lines/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/match_modalities/scicar_cell_lines/state.yaml
Completed 35.6 MiB/75.3 MiB (87.2 MiB/s) with 35 file(s) remaining
Completed 35.9 MiB/75.3 MiB (87.4 MiB/s) with 35 file(s) remaining
Completed 35.9 MiB/75.3 MiB (87.4 MiB/s) with 35 file(s) remaining
download: s3://openproblems-data/resources_test/match_modalities/scicar_cell_lines/integrated_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/match_modalities/scicar_cell_lines/integrated_mod2.h5ad
Completed 35.9 MiB/75.3 MiB (87.4 MiB/s) with 34 file(s) remaining
Completed 36.0 MiB/75.3 MiB (86.5 MiB/s) with 34 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter/score.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter/score.h5ad
Completed 36.0 MiB/75.3 MiB (86.5 MiB/s) with 33 file(s) remaining
Completed 36.0 MiB/75.3 MiB (83.9 MiB/s) with 33 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter/state.yaml to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter/state.yaml
Completed 36.0 MiB/75.3 MiB (83.9 MiB/s) with 32 file(s) remaining
Completed 36.2 MiB/75.3 MiB (83.8 MiB/s) with 32 file(s) remaining
Completed 36.5 MiB/75.3 MiB (84.1 MiB/s) with 32 file(s) remaining
Completed 36.7 MiB/75.3 MiB (84.4 MiB/s) with 32 file(s) remaining
Completed 37.0 MiB/75.3 MiB (84.8 MiB/s) with 32 file(s) remaining
Completed 37.1 MiB/75.3 MiB (85.0 MiB/s) with 32 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter/prediction.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter/prediction.h5ad
Completed 37.1 MiB/75.3 MiB (85.0 MiB/s) with 31 file(s) remaining
Completed 37.4 MiB/75.3 MiB (84.4 MiB/s) with 31 file(s) remaining
download: s3://openproblems-data/resources_test/label_projection/pancreas/solution.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/label_projection/pancreas/solution.h5ad
Completed 37.4 MiB/75.3 MiB (84.4 MiB/s) with 30 file(s) remaining
Completed 37.6 MiB/75.3 MiB (83.7 MiB/s) with 30 file(s) remaining
Completed 37.9 MiB/75.3 MiB (84.2 MiB/s) with 30 file(s) remaining
Completed 38.1 MiB/75.3 MiB (84.5 MiB/s) with 30 file(s) remaining
Completed 38.4 MiB/75.3 MiB (84.9 MiB/s) with 30 file(s) remaining
Completed 38.6 MiB/75.3 MiB (85.4 MiB/s) with 30 file(s) remaining
Completed 38.6 MiB/75.3 MiB (85.1 MiB/s) with 30 file(s) remaining
Completed 38.9 MiB/75.3 MiB (85.0 MiB/s) with 30 file(s) remaining
Completed 39.1 MiB/75.3 MiB (85.5 MiB/s) with 30 file(s) remaining
Completed 39.1 MiB/75.3 MiB (85.5 MiB/s) with 30 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter/test_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter/test_mod1.h5ad
Completed 39.1 MiB/75.3 MiB (85.5 MiB/s) with 29 file(s) remaining
Completed 39.4 MiB/75.3 MiB (85.6 MiB/s) with 29 file(s) remaining
download: s3://openproblems-data/resources_test/label_projection/pancreas/train.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/label_projection/pancreas/train.h5ad
Completed 39.4 MiB/75.3 MiB (85.6 MiB/s) with 28 file(s) remaining
Completed 39.6 MiB/75.3 MiB (85.8 MiB/s) with 28 file(s) remaining
Completed 39.9 MiB/75.3 MiB (86.0 MiB/s) with 28 file(s) remaining
Completed 40.1 MiB/75.3 MiB (86.5 MiB/s) with 28 file(s) remaining
Completed 40.4 MiB/75.3 MiB (86.6 MiB/s) with 28 file(s) remaining
Completed 40.6 MiB/75.3 MiB (86.9 MiB/s) with 28 file(s) remaining
Completed 40.7 MiB/75.3 MiB (87.0 MiB/s) with 28 file(s) remaining
download: s3://openproblems-data/resources_test/dimensionality_reduction/pancreas/solution.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/dimensionality_reduction/pancreas/solution.h5ad
Completed 40.7 MiB/75.3 MiB (87.0 MiB/s) with 27 file(s) remaining
Completed 40.8 MiB/75.3 MiB (86.0 MiB/s) with 27 file(s) remaining
Completed 41.0 MiB/75.3 MiB (86.4 MiB/s) with 27 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter/train_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter/train_mod1.h5ad
Completed 41.0 MiB/75.3 MiB (86.4 MiB/s) with 26 file(s) remaining
Completed 41.3 MiB/75.3 MiB (86.7 MiB/s) with 26 file(s) remaining
Completed 41.5 MiB/75.3 MiB (86.9 MiB/s) with 26 file(s) remaining
Completed 41.8 MiB/75.3 MiB (87.3 MiB/s) with 26 file(s) remaining
Completed 42.0 MiB/75.3 MiB (87.7 MiB/s) with 26 file(s) remaining
Completed 42.3 MiB/75.3 MiB (88.2 MiB/s) with 26 file(s) remaining
Completed 42.5 MiB/75.3 MiB (88.6 MiB/s) with 26 file(s) remaining
Completed 42.6 MiB/75.3 MiB (88.7 MiB/s) with 26 file(s) remaining
Completed 42.7 MiB/75.3 MiB (88.6 MiB/s) with 26 file(s) remaining
download: s3://openproblems-data/resources_test/match_modalities/scicar_cell_lines/solution_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/match_modalities/scicar_cell_lines/solution_mod2.h5ad
Completed 42.7 MiB/75.3 MiB (88.6 MiB/s) with 25 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter/test_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter/test_mod2.h5ad
Completed 42.7 MiB/75.3 MiB (88.6 MiB/s) with 24 file(s) remaining
Completed 42.9 MiB/75.3 MiB (88.5 MiB/s) with 24 file(s) remaining
Completed 43.2 MiB/75.3 MiB (88.6 MiB/s) with 24 file(s) remaining
Completed 43.4 MiB/75.3 MiB (88.9 MiB/s) with 24 file(s) remaining
Completed 43.7 MiB/75.3 MiB (89.3 MiB/s) with 24 file(s) remaining
Completed 43.9 MiB/75.3 MiB (89.7 MiB/s) with 24 file(s) remaining
Completed 44.2 MiB/75.3 MiB (90.1 MiB/s) with 24 file(s) remaining
Completed 44.4 MiB/75.3 MiB (90.5 MiB/s) with 24 file(s) remaining
Completed 44.4 MiB/75.3 MiB (90.5 MiB/s) with 24 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter/train_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter/train_mod2.h5ad
Completed 44.4 MiB/75.3 MiB (90.5 MiB/s) with 23 file(s) remaining
Completed 44.4 MiB/75.3 MiB (89.9 MiB/s) with 23 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter_swapped/score.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter_swapped/score.h5ad
Completed 44.4 MiB/75.3 MiB (89.9 MiB/s) with 22 file(s) remaining
Completed 44.7 MiB/75.3 MiB (90.0 MiB/s) with 22 file(s) remaining
Completed 44.9 MiB/75.3 MiB (90.5 MiB/s) with 22 file(s) remaining
Completed 45.0 MiB/75.3 MiB (90.5 MiB/s) with 22 file(s) remaining
download: s3://openproblems-data/resources_test/match_modalities/scicar_cell_lines/integrated_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/match_modalities/scicar_cell_lines/integrated_mod1.h5ad
Completed 45.0 MiB/75.3 MiB (90.5 MiB/s) with 21 file(s) remaining
Completed 45.2 MiB/75.3 MiB (90.6 MiB/s) with 21 file(s) remaining
Completed 45.5 MiB/75.3 MiB (91.0 MiB/s) with 21 file(s) remaining
Completed 45.7 MiB/75.3 MiB (91.4 MiB/s) with 21 file(s) remaining
Completed 46.0 MiB/75.3 MiB (91.8 MiB/s) with 21 file(s) remaining
Completed 46.2 MiB/75.3 MiB (92.1 MiB/s) with 21 file(s) remaining
Completed 46.5 MiB/75.3 MiB (92.5 MiB/s) with 21 file(s) remaining
Completed 46.7 MiB/75.3 MiB (92.8 MiB/s) with 21 file(s) remaining
Completed 47.0 MiB/75.3 MiB (93.2 MiB/s) with 21 file(s) remaining
Completed 47.2 MiB/75.3 MiB (93.6 MiB/s) with 21 file(s) remaining
Completed 47.5 MiB/75.3 MiB (93.9 MiB/s) with 21 file(s) remaining
Completed 47.7 MiB/75.3 MiB (94.2 MiB/s) with 21 file(s) remaining
Completed 48.0 MiB/75.3 MiB (94.5 MiB/s) with 21 file(s) remaining
Completed 48.2 MiB/75.3 MiB (94.8 MiB/s) with 21 file(s) remaining
download: s3://openproblems-data/resources_test/match_modalities/scicar_cell_lines/solution_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/match_modalities/scicar_cell_lines/solution_mod1.h5ad
Completed 48.2 MiB/75.3 MiB (94.8 MiB/s) with 20 file(s) remaining
Completed 48.5 MiB/75.3 MiB (95.0 MiB/s) with 20 file(s) remaining
Completed 48.7 MiB/75.3 MiB (95.4 MiB/s) with 20 file(s) remaining
Completed 49.0 MiB/75.3 MiB (95.7 MiB/s) with 20 file(s) remaining
Completed 49.2 MiB/75.3 MiB (96.1 MiB/s) with 20 file(s) remaining
Completed 49.5 MiB/75.3 MiB (96.5 MiB/s) with 20 file(s) remaining
Completed 49.7 MiB/75.3 MiB (95.0 MiB/s) with 20 file(s) remaining
Completed 49.7 MiB/75.3 MiB (94.9 MiB/s) with 20 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter_swapped/test_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter_swapped/test_mod2.h5ad
Completed 49.7 MiB/75.3 MiB (94.9 MiB/s) with 19 file(s) remaining
Completed 50.0 MiB/75.3 MiB (94.9 MiB/s) with 19 file(s) remaining
Completed 50.0 MiB/75.3 MiB (94.8 MiB/s) with 19 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter_swapped/prediction.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter_swapped/prediction.h5ad
Completed 50.0 MiB/75.3 MiB (94.8 MiB/s) with 18 file(s) remaining
Completed 50.3 MiB/75.3 MiB (94.7 MiB/s) with 18 file(s) remaining
Completed 50.5 MiB/75.3 MiB (94.1 MiB/s) with 18 file(s) remaining
Completed 50.8 MiB/75.3 MiB (94.1 MiB/s) with 18 file(s) remaining
Completed 51.0 MiB/75.3 MiB (94.4 MiB/s) with 18 file(s) remaining
Completed 51.3 MiB/75.3 MiB (94.8 MiB/s) with 18 file(s) remaining
Completed 51.5 MiB/75.3 MiB (95.0 MiB/s) with 18 file(s) remaining
Completed 51.5 MiB/75.3 MiB (95.0 MiB/s) with 18 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter_swapped/train_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter_swapped/train_mod1.h5ad
Completed 51.5 MiB/75.3 MiB (95.0 MiB/s) with 17 file(s) remaining
Completed 51.5 MiB/75.3 MiB (94.4 MiB/s) with 17 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter_swapped/score.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter_swapped/score.h5ad
Completed 51.5 MiB/75.3 MiB (94.4 MiB/s) with 16 file(s) remaining
Completed 51.7 MiB/75.3 MiB (94.3 MiB/s) with 16 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter/prediction.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter/prediction.h5ad
Completed 51.7 MiB/75.3 MiB (94.3 MiB/s) with 15 file(s) remaining
Completed 51.9 MiB/75.3 MiB (93.4 MiB/s) with 15 file(s) remaining
Completed 52.0 MiB/75.3 MiB (93.5 MiB/s) with 15 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter_swapped/train_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter_swapped/train_mod2.h5ad
Completed 52.0 MiB/75.3 MiB (93.5 MiB/s) with 14 file(s) remaining
Completed 52.2 MiB/75.3 MiB (93.7 MiB/s) with 14 file(s) remaining
Completed 52.3 MiB/75.3 MiB (93.7 MiB/s) with 14 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter/train_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter/train_mod2.h5ad
Completed 52.3 MiB/75.3 MiB (93.7 MiB/s) with 13 file(s) remaining
Completed 52.5 MiB/75.3 MiB (93.7 MiB/s) with 13 file(s) remaining
Completed 52.8 MiB/75.3 MiB (94.0 MiB/s) with 13 file(s) remaining
Completed 53.0 MiB/75.3 MiB (94.3 MiB/s) with 13 file(s) remaining
Completed 53.1 MiB/75.3 MiB (94.3 MiB/s) with 13 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_cite_starter_swapped/test_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_cite_starter_swapped/test_mod1.h5ad
Completed 53.1 MiB/75.3 MiB (94.3 MiB/s) with 12 file(s) remaining
Completed 53.1 MiB/75.3 MiB (93.8 MiB/s) with 12 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter/score.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter/score.h5ad
Completed 53.1 MiB/75.3 MiB (93.8 MiB/s) with 11 file(s) remaining
Completed 53.3 MiB/75.3 MiB (93.1 MiB/s) with 11 file(s) remaining
Completed 53.6 MiB/75.3 MiB (92.6 MiB/s) with 11 file(s) remaining
Completed 53.8 MiB/75.3 MiB (92.1 MiB/s) with 11 file(s) remaining
download: s3://openproblems-data/resources_test/match_modalities/scicar_cell_lines/dataset_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/match_modalities/scicar_cell_lines/dataset_mod1.h5ad
Completed 53.8 MiB/75.3 MiB (92.1 MiB/s) with 10 file(s) remaining
Completed 54.0 MiB/75.3 MiB (91.5 MiB/s) with 10 file(s) remaining
Completed 54.3 MiB/75.3 MiB (91.8 MiB/s) with 10 file(s) remaining
Completed 54.5 MiB/75.3 MiB (92.0 MiB/s) with 10 file(s) remaining
Completed 54.8 MiB/75.3 MiB (92.3 MiB/s) with 10 file(s) remaining
Completed 55.0 MiB/75.3 MiB (92.6 MiB/s) with 10 file(s) remaining
Completed 55.3 MiB/75.3 MiB (92.9 MiB/s) with 10 file(s) remaining
Completed 55.5 MiB/75.3 MiB (93.2 MiB/s) with 10 file(s) remaining
Completed 55.8 MiB/75.3 MiB (93.4 MiB/s) with 10 file(s) remaining
Completed 56.0 MiB/75.3 MiB (93.6 MiB/s) with 10 file(s) remaining
Completed 56.3 MiB/75.3 MiB (93.9 MiB/s) with 10 file(s) remaining
Completed 56.5 MiB/75.3 MiB (94.0 MiB/s) with 10 file(s) remaining
Completed 56.7 MiB/75.3 MiB (94.2 MiB/s) with 10 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter_swapped/test_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter_swapped/test_mod1.h5ad
Completed 56.7 MiB/75.3 MiB (94.2 MiB/s) with 9 file(s) remaining
Completed 56.9 MiB/75.3 MiB (94.3 MiB/s) with 9 file(s) remaining
Completed 57.2 MiB/75.3 MiB (94.1 MiB/s) with 9 file(s) remaining
Completed 57.4 MiB/75.3 MiB (94.4 MiB/s) with 9 file(s) remaining
Completed 57.7 MiB/75.3 MiB (94.6 MiB/s) with 9 file(s) remaining
Completed 57.9 MiB/75.3 MiB (94.9 MiB/s) with 9 file(s) remaining
Completed 58.0 MiB/75.3 MiB (95.0 MiB/s) with 9 file(s) remaining
download: s3://openproblems-data/resources_test/match_modalities/scicar_cell_lines/dataset_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/match_modalities/scicar_cell_lines/dataset_mod2.h5ad
Completed 58.0 MiB/75.3 MiB (95.0 MiB/s) with 8 file(s) remaining
Completed 58.3 MiB/75.3 MiB (95.2 MiB/s) with 8 file(s) remaining
Completed 58.5 MiB/75.3 MiB (95.2 MiB/s) with 8 file(s) remaining
Completed 58.8 MiB/75.3 MiB (95.2 MiB/s) with 8 file(s) remaining
Completed 59.0 MiB/75.3 MiB (95.2 MiB/s) with 8 file(s) remaining
Completed 59.1 MiB/75.3 MiB (95.1 MiB/s) with 8 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter_swapped/train_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter_swapped/train_mod2.h5ad
Completed 59.1 MiB/75.3 MiB (95.1 MiB/s) with 7 file(s) remaining
Completed 59.3 MiB/75.3 MiB (94.2 MiB/s) with 7 file(s) remaining
Completed 59.5 MiB/75.3 MiB (94.3 MiB/s) with 7 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter/test_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter/test_mod2.h5ad
Completed 59.5 MiB/75.3 MiB (94.3 MiB/s) with 6 file(s) remaining
Completed 59.7 MiB/75.3 MiB (94.0 MiB/s) with 6 file(s) remaining
Completed 60.0 MiB/75.3 MiB (94.1 MiB/s) with 6 file(s) remaining
Completed 60.2 MiB/75.3 MiB (94.4 MiB/s) with 6 file(s) remaining
Completed 60.5 MiB/75.3 MiB (94.5 MiB/s) with 6 file(s) remaining
Completed 60.7 MiB/75.3 MiB (94.7 MiB/s) with 6 file(s) remaining
Completed 61.0 MiB/75.3 MiB (95.0 MiB/s) with 6 file(s) remaining
Completed 61.2 MiB/75.3 MiB (95.1 MiB/s) with 6 file(s) remaining
Completed 61.5 MiB/75.3 MiB (95.4 MiB/s) with 6 file(s) remaining
Completed 61.7 MiB/75.3 MiB (95.7 MiB/s) with 6 file(s) remaining
Completed 62.0 MiB/75.3 MiB (95.9 MiB/s) with 6 file(s) remaining
Completed 62.2 MiB/75.3 MiB (96.1 MiB/s) with 6 file(s) remaining
Completed 62.5 MiB/75.3 MiB (96.3 MiB/s) with 6 file(s) remaining
Completed 62.7 MiB/75.3 MiB (96.6 MiB/s) with 6 file(s) remaining
Completed 63.0 MiB/75.3 MiB (96.7 MiB/s) with 6 file(s) remaining
Completed 63.2 MiB/75.3 MiB (97.0 MiB/s) with 6 file(s) remaining
Completed 63.5 MiB/75.3 MiB (97.2 MiB/s) with 6 file(s) remaining
Completed 63.7 MiB/75.3 MiB (97.5 MiB/s) with 6 file(s) remaining
Completed 64.0 MiB/75.3 MiB (97.8 MiB/s) with 6 file(s) remaining
Completed 64.2 MiB/75.3 MiB (98.1 MiB/s) with 6 file(s) remaining
Completed 64.5 MiB/75.3 MiB (98.1 MiB/s) with 6 file(s) remaining
Completed 64.7 MiB/75.3 MiB (98.1 MiB/s) with 6 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter_swapped/prediction.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter_swapped/prediction.h5ad
Completed 64.7 MiB/75.3 MiB (98.1 MiB/s) with 5 file(s) remaining
Completed 65.0 MiB/75.3 MiB (98.3 MiB/s) with 5 file(s) remaining
Completed 65.2 MiB/75.3 MiB (98.6 MiB/s) with 5 file(s) remaining
Completed 65.5 MiB/75.3 MiB (98.8 MiB/s) with 5 file(s) remaining
Completed 65.7 MiB/75.3 MiB (98.7 MiB/s) with 5 file(s) remaining
Completed 66.0 MiB/75.3 MiB (98.6 MiB/s) with 5 file(s) remaining
Completed 66.2 MiB/75.3 MiB (98.9 MiB/s) with 5 file(s) remaining
Completed 66.5 MiB/75.3 MiB (99.1 MiB/s) with 5 file(s) remaining
Completed 66.7 MiB/75.3 MiB (99.4 MiB/s) with 5 file(s) remaining
Completed 67.0 MiB/75.3 MiB (99.5 MiB/s) with 5 file(s) remaining
Completed 67.2 MiB/75.3 MiB (99.8 MiB/s) with 5 file(s) remaining
Completed 67.5 MiB/75.3 MiB (100.1 MiB/s) with 5 file(s) remaining
Completed 67.7 MiB/75.3 MiB (100.3 MiB/s) with 5 file(s) remaining
Completed 68.0 MiB/75.3 MiB (100.6 MiB/s) with 5 file(s) remaining
Completed 68.2 MiB/75.3 MiB (100.9 MiB/s) with 5 file(s) remaining
Completed 68.5 MiB/75.3 MiB (101.1 MiB/s) with 5 file(s) remaining
Completed 68.7 MiB/75.3 MiB (101.3 MiB/s) with 5 file(s) remaining
Completed 69.0 MiB/75.3 MiB (101.7 MiB/s) with 5 file(s) remaining
Completed 69.2 MiB/75.3 MiB (101.9 MiB/s) with 5 file(s) remaining
Completed 69.5 MiB/75.3 MiB (102.1 MiB/s) with 5 file(s) remaining
Completed 69.5 MiB/75.3 MiB (102.2 MiB/s) with 5 file(s) remaining
Completed 69.8 MiB/75.3 MiB (102.4 MiB/s) with 5 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter/train_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter/train_mod1.h5ad
Completed 69.8 MiB/75.3 MiB (102.4 MiB/s) with 4 file(s) remaining
Completed 70.0 MiB/75.3 MiB (102.5 MiB/s) with 4 file(s) remaining
Completed 70.3 MiB/75.3 MiB (102.7 MiB/s) with 4 file(s) remaining
Completed 70.5 MiB/75.3 MiB (102.9 MiB/s) with 4 file(s) remaining
Completed 70.8 MiB/75.3 MiB (103.2 MiB/s) with 4 file(s) remaining
Completed 71.0 MiB/75.3 MiB (103.3 MiB/s) with 4 file(s) remaining
Completed 71.3 MiB/75.3 MiB (103.5 MiB/s) with 4 file(s) remaining
Completed 71.5 MiB/75.3 MiB (103.7 MiB/s) with 4 file(s) remaining
Completed 71.8 MiB/75.3 MiB (103.8 MiB/s) with 4 file(s) remaining
Completed 72.0 MiB/75.3 MiB (104.1 MiB/s) with 4 file(s) remaining
Completed 72.3 MiB/75.3 MiB (102.9 MiB/s) with 4 file(s) remaining
Completed 72.5 MiB/75.3 MiB (103.2 MiB/s) with 4 file(s) remaining
Completed 72.8 MiB/75.3 MiB (103.3 MiB/s) with 4 file(s) remaining
Completed 73.0 MiB/75.3 MiB (103.6 MiB/s) with 4 file(s) remaining
Completed 73.3 MiB/75.3 MiB (103.8 MiB/s) with 4 file(s) remaining
Completed 73.4 MiB/75.3 MiB (103.9 MiB/s) with 4 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter_swapped/test_mod2.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter_swapped/test_mod2.h5ad
Completed 73.4 MiB/75.3 MiB (103.9 MiB/s) with 3 file(s) remaining
Completed 73.7 MiB/75.3 MiB (103.7 MiB/s) with 3 file(s) remaining
Completed 73.7 MiB/75.3 MiB (103.7 MiB/s) with 3 file(s) remaining
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter_swapped/train_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter_swapped/train_mod1.h5ad
Completed 73.7 MiB/75.3 MiB (103.7 MiB/s) with 2 file(s) remaining
Completed 74.0 MiB/75.3 MiB (95.0 MiB/s) with 2 file(s) remaining 
Completed 74.2 MiB/75.3 MiB (95.1 MiB/s) with 2 file(s) remaining 
Completed 74.5 MiB/75.3 MiB (95.3 MiB/s) with 2 file(s) remaining 
Completed 74.7 MiB/75.3 MiB (95.5 MiB/s) with 2 file(s) remaining 
Completed 75.0 MiB/75.3 MiB (95.7 MiB/s) with 2 file(s) remaining 
Completed 75.1 MiB/75.3 MiB (95.8 MiB/s) with 2 file(s) remaining 
download: s3://openproblems-data/resources_test/predict_modality/bmmc_multiome_starter/test_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/predict_modality/bmmc_multiome_starter/test_mod1.h5ad
Completed 75.1 MiB/75.3 MiB (95.8 MiB/s) with 1 file(s) remaining
Completed 75.3 MiB/75.3 MiB (93.8 MiB/s) with 1 file(s) remaining
download: s3://openproblems-data/resources_test/common/scicar_cell_lines/dataset_mod1.h5ad to ../viash_automount/home/runner/work/website/website/_openproblems-v2/resources_test/common/scicar_cell_lines/dataset_mod1.h5ad

The test resources are stored in the resources_test directory.

Ready, set, go!

That’s it! Now you should be able to test whether the existing components work as expected and then start adding functionality to the pipeline. Running an existing component is as simple as running a command in your terminal. Using test data as input, you can try this out immediately.

Run a component

Use the viash run command to run a Viash component. Everything after the -- separator counts as the arguments of the component itself. In this case, the knn component has an --input_train and --input_test argument to which the test resources are passed.

viash run src/tasks/label_projection/methods/knn/config.vsh.yaml -- \
  --input_train resources_test/label_projection/pancreas/train.h5ad \
  --input_test resources_test/label_projection/pancreas/test.h5ad \
  --output output.h5ad

Output

[notice] Checking if Docker image is available at 'ghcr.io/openproblems-bio/label_projection/methods/knn:dev'
[warning] Could not pull from 'ghcr.io/openproblems-bio/label_projection/methods/knn:dev'. Docker image doesn't exist or is not accessible.
[notice] Building container 'ghcr.io/openproblems-bio/label_projection/methods/knn:dev' with Dockerfile
Load input data
Fit to train data
Predict on test data
Write output to file

Testing components

Testing components is an important part of the development process. Each tasks comes with pre-defined unit tests that can be run using the viash test command.

viash test src/tasks/label_projection/methods/knn/config.vsh.yaml

Output

Running tests in temporary directory: '/tmp/viash_test_knn15801522474527101908'
====================================================================
+/tmp/viash_test_knn15801522474527101908/build_executable/knn ---verbosity 6 ---setup cachedbuild
[notice] Building container 'ghcr.io/openproblems-bio/label_projection/methods/knn:test' with Dockerfile
[info] Running 'docker build -t ghcr.io/openproblems-bio/label_projection/methods/knn:test /tmp/viash_test_knn15801522474527101908/build_executable -f /tmp/viash_test_knn15801522474527101908/build_executable/tmp/dockerbuild-knn-PZrzob/Dockerfile'
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 608B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for ghcr.io/openproblems-bio/base_python:1.0.2
#3 DONE 0.0s

#4 [1/2] FROM ghcr.io/openproblems-bio/base_python:1.0.2@sha256:eafa661ef06e6de0e1fd1f0f22a5e735983b02280f57c44202f05bff63e8047c
#4 DONE 0.0s

#5 [2/2] RUN pip install --upgrade pip &&   pip install --upgrade --no-cache-dir "scikit-learn" "jsonschema"
#5 CACHED

#6 exporting to image
#6 exporting layers done
#6 writing image sha256:0a8c3db62c855724c2adcc54ce4a001c7ea5f8f880f92e4380cf31c6dc662048 done
#6 naming to ghcr.io/openproblems-bio/label_projection/methods/knn:test done
#6 DONE 0.0s
====================================================================
+/tmp/viash_test_knn15801522474527101908/test_check_method_config/test_executable
Load config data
Check general fields
Check info fields
Check platform fields
All checks succeeded!
====================================================================
+/tmp/viash_test_knn15801522474527101908/test_run_and_check_adata/test_executable
>> Checking whether input files exist
>> Running script as test
Load input data
Fit to train data
Predict on test data
Write output to file
>> Checking whether output file exists
>> Reading h5ad files and checking formats
Reading and checking input_train
  AnnData object with n_obs × n_vars = 400 × 500
    obs: 'label', 'batch'
    var: 'hvg', 'hvg_score'
    uns: 'dataset_id', 'normalization_id'
    obsm: 'X_pca'
    layers: 'counts', 'normalized'
Reading and checking input_test
  AnnData object with n_obs × n_vars = 100 × 500
    obs: 'batch'
    var: 'hvg', 'hvg_score'
    uns: 'dataset_id', 'normalization_id'
    obsm: 'X_pca'
    layers: 'counts', 'normalized'
Reading and checking output
  AnnData object with n_obs × n_vars = 100 × 500
    obs: 'batch', 'label_pred'
    var: 'hvg', 'hvg_score'
    uns: 'dataset_id', 'method_id', 'normalization_id'
    obsm: 'X_pca'
    layers: 'counts', 'normalized'
All checks succeeded!
====================================================================
SUCCESS! All 2 out of 2 test scripts succeeded!
Cleaning up temporary directory

If you want to run the unit tests for all of the components of a task, you can use the viash ns test command.

viash ns test --query label_projection --parallel --platform docker

Output

             namespace        functionality             platform            test_name exit_code duration               result
label_projection/methods  logistic_regression               docker                start                                        
label_projection/methods               scanvi               docker                start                                        
label_projection/methods                  knn               docker                start                                        
label_projection/methods                  mlp               docker                start                                        
label_projection/metrics             accuracy               docker                start                                        
label_projection/metrics                   f1               docker                start
label_projection/methods  logistic_regression               docker     build_executable         0        4              SUCCESS
label_projection/methods  logistic_regression               docker      generic_test.py         0        9              SUCCESS
label_projection/metrics                   f1               docker     build_executable         0        7              SUCCESS
label_projection/metrics                   f1               docker      format_check.py         0        8              SUCCESS
label_projection/metrics             accuracy               docker     build_executable         0        8              SUCCESS
label_projection/metrics             accuracy               docker      format_check.py         0        7              SUCCESS
...