pipeline runner for snakemake

The architecture is like this

image

Version info

1.0.0

• docker
• python3.11
• snakemake 7.32.2
• singularity 4.0.2

1.1.0

caution : can't use sge itself in docker. you should do settings for sge and nfs.

• same tools with 1.0.0
• sun grid engine(SGE) 8.1.9

This is may limitation

Test for running snakemake on single server is completed but on the cluster could failed that is not tested.

Usage

You should run docker with --previleged option because of singularity in snakemake for using container

example

docker run --privileged -it -v /data08:/data08 -v /TBI:/TBI shinejh0528/pipeline_runner:snakemake_1.0.0 /bin/bash

To use container, use --use-singularity --singularity-args option.

The snakemake command is like

snakemake \
    --jobs 150 --local-cores 8 \
    --use-singularity \
    --singularity-args '--bind /data08,/TBI' \
    --printshellcmds \
    --snakefile ${pplrep}/workflow/${anlmd}.snakefile \
    --configfile ${workhome}/config.yaml \
    --directory ${workhome} \
    --config 'prjtid=${prjtid}' \
             'anlid=${anlid}' \
             'atmid=${atmid}' \
             'pipedir=${pplrep}' \
             'anlmd=${anlmd}' \
             'vqsrmd=${vqsrmd}' \
             'ref=${pplrep}/config/ref.${refid}.yaml' \
             'kit=${pplrep}/config/kit.${kitid}.yaml' \
             'param=${pplrep}/config/param.${anlmd}.yaml' \
             'workdir=${workhome}' \
             'conf_cnv=${workhome}/config.cnv.yaml' \
    --notemp \
    --latency-wait=120 \
    --max-jobs-per-second 5 \
    --keep-target-files \
    --keep-going"
#    --cluster-config ${pplrep}/config/cluster.${anlmd}.json \
#    --cluster 'qsub -V \
#                    -o ${workhome}/{cluster.output} \
#                    -e ${workhome}/{cluster.error} \
#                    -pe smp {cluster.threads} -N {cluster.jobName} -S /bin/bash -q wine.q'"

If you wan to constitute snakemake with cluster (server), then confer to this link that is snakemake template and You can also check how to set container on config.

[***]