Snakemake make use of cluster engines

https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html#cluster-configuration-deprecated

https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles

Threads

When Snakemake is executed with fewer cores, the number of threads will be adjusted, i.e. threads = min(threads, cores) with cores being the number of cores specified at the command line (option --cores).

图片alt

sankemake logs analysis

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cluster nodes: 10
Conda environments: ignored
Job stats:
job                    count    min threads    max threads
-------------------  -------  -------------  -------------
fastp_QC                  45              8              8
featureCounts_count        1             24             24
hisat2_align              55             24             24
total                    101              8             24

Select jobs to execute...

[Tue Aug 16 20:54:18 2022]
rule hisat2_align:
    input: results/trimmed/H3_1HIP-HH_R1.fastq.gz, results/trimmed/H3_1HIP-HH_R2.fastq.gz, resources/hisat2_genome
    output: results/hisat2/H3_1HIP-HH/hisat2.sam
    log: logs/hisat2/H3_1HIP-HH.log
    jobid: 71
    reason: Missing output files: results/hisat2/H3_1HIP-HH/hisat2.sam
    wildcards: sample=H3_1HIP, unit=HH
    threads: 24
    resources: mem_mb=16799, disk_mb=16799, tmpdir=/tmp


            hisat2 -p 24                 -x resources/hisat2_genome/genome                 -1 results/trimmed/H3_1HIP-HH_R1.fastq.gz                 -2 results/trimmed/H3_1HIP-HH_R2.fastq.gz                 -S results/hisat2/H3_1HIP-HH/hisat2.sam > logs/hisat2/H3_1HIP-HH.log 2>&1
        
Submitted job 71 with external jobid '2224'.

[Tue Aug 16 20:54:18 2022]
rule fastp_QC:
    input: /data2/mrna/zbb_mrna/220613-40/220613-40_1.fq.gz, /data2/mrna/zbb_mrna/220613-40/220613-40_2.fq.gz
    output: results/trimmed/DL3_1HYP-LSY_R1.fastq.gz, results/trimmed/DL3_1HYP-LSY_R2.fastq.gz
    jobid: 38
    reason: Missing output files: results/trimmed/DL3_1HYP-LSY_R1.fastq.gz, results/trimmed/DL3_1HYP-LSY_R2.fastq.gz
    wildcards: sample=DL3_1HYP, unit=LSY
    threads: 8
    resources: mem_mb=17053, disk_mb=17053, tmpdir=/tmp


            fastp -i /data2/mrna/zbb_mrna/220613-40/220613-40_1.fq.gz -o  results/trimmed/DL3_1HYP-LSY_R1.fastq.gz                  -I  /data2/mrna/zbb_mrna/220613-40/220613-40_2.fq.gz  -O  results/trimmed/DL3_1HYP-LSY_R2.fastq.gz                 -Q --thread=8 --length_required=50 --n_base_limit=2 --compression=6
         
Submitted job 38 with external jobid '2225'.

[Tue Aug 16 21:21:37 2022]
Finished job 117.
1 of 101 steps (1%) done
Select jobs to execute...

Error

[Tue Aug 16 23:04:56 2022]
Error in rule hisat2_align:
    jobid: 85
    output: results/hisat2/H8_2HYP-HY/hisat2.sam
    log: logs/hisat2/H8_2HYP-HY.log (check log file(s) for error message)
    shell:
        
            hisat2 -p 24                 -x resources/hisat2_genome/genome                 -1 results/trimmed/H8_2HYP-HY_R1.fastq.gz                 -2 results/trimmed/H8_2HYP-HY_R2.fastq.gz                 -S results/hisat2/H8_2HYP-HY/hisat2.sam > logs/hisat2/H8_2HYP-HY.log 2>&1
        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
    cluster_jobid: 2251

Error executing rule hisat2_align on cluster (jobid: 85, external: 2251, jobscript: /data3/cluster/workspace/mouse_brain/.snakemake/tmp.w5920d1q/snakejob.hisat2_align.85.sh). For error details see the cluster log and the log files of the involved rule(s).