Проблема Nextflow Docker - проблема с разрешением в Docker? Не удается запустить контейнер pomoxis

Я новичок в контейнерах nextflow и docker. Я пытаюсь denovo собрать некоторые чтения и сопоставить чтения с эталонным геномом, но продолжаю получать следующую ошибку:

shaun@shaun-HP-Z6-G4-Workstation:~/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble$ nextflow pomoxis_map_assemble_nextflow.nf
N E X T F L O W  ~  version 22.04.5
Launching `pomoxis_map_assemble_nextflow.nf` [goofy_hoover] DSL1 - revision: 6e4be1e0bd
executor >  local (1)
executor >  local (1)
[cf/bc2b69] process > pomoxis (1) [100%] 1 of 1, failed: 1 ✘
WARN: There's no process matching config selector: fastqc
WARN: There's no process matching config selector: porechop
WARN: There's no process matching config selector: bioawk
WARN: There's no process matching config selector: fastqconvert
WARN: There's no process matching config selector: blast_raw
Error executing process > 'pomoxis (1)'

Caused by:
  Process `pomoxis (1)` terminated with an error exit status (127)

Command executed:

  mini_align -i output.fastq -r /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/ref/*.fasta -o results -p > output_test_final.fa

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.sh: line 2: mini_align: command not found

Work dir:
  /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/work/cf/bc2b696aea0863d76c2c9221315c39

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

ПРЕДУПРЕЖДЕНИЕ связано с тем, что я использую главный файл конфигурации для всех своих сценариев .nf.

Вот лог-файл вышеуказанной ошибки:

Oct.-12 10:33:04.759 [main] DEBUG nextflow.cli.Launcher - $> nextflow pomoxis_map_assemble_nextflow.nf
Oct.-12 10:33:04.822 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 22.04.5
Oct.-12 10:33:04.837 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/nextflow.config
Oct.-12 10:33:04.838 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/nextflow.config
Oct.-12 10:33:04.856 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`
Oct.-12 10:33:05.414 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=1 by probing script field
Oct.-12 10:33:05.428 [main] INFO  nextflow.cli.CmdRun - Launching `pomoxis_map_assemble_nextflow.nf` [goofy_hoover] DSL1 - revision: 6e4be1e0bd
Oct.-12 10:33:05.439 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; plugins-dir=/home/shaun/.nextflow/plugins; core-plugins: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected]
Oct.-12 10:33:05.440 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Oct.-12 10:33:05.449 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Oct.-12 10:33:05.449 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Oct.-12 10:33:05.452 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Oct.-12 10:33:05.478 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Oct.-12 10:33:05.527 [main] DEBUG nextflow.Session - Session uuid: ddab5124-cc54-4c39-840d-6cc1bf664773
Oct.-12 10:33:05.527 [main] DEBUG nextflow.Session - Run name: goofy_hoover
Oct.-12 10:33:05.527 [main] DEBUG nextflow.Session - Executor pool size: 16
Oct.-12 10:33:05.547 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 22.04.5 build 5708
  Created: 15-07-2022 16:09 UTC (16-07-2022 01:39 ACDT)
  System: Linux 5.13.0-52-generic
  Runtime: Groovy 3.0.10 on OpenJDK 64-Bit Server VM 11.0.15+10-Ubuntu-0ubuntu0.21.10.1
  Encoding: UTF-8 (UTF-8)
  Process: 665521@shaun-HP-Z6-G4-Workstation [127.0.1.1]
  CPUs: 16 - Mem: 62.5 GB (32.1 GB) - Swap: 2 GB (2 GB)
Oct.-12 10:33:05.564 [main] DEBUG nextflow.Session - Work-dir: /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/work [ext2/ext3]
Oct.-12 10:33:05.564 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/bin
Oct.-12 10:33:05.573 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Oct.-12 10:33:05.583 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Oct.-12 10:33:05.605 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Oct.-12 10:33:05.613 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 17; maxThreads: 1000
Oct.-12 10:33:05.700 [main] DEBUG nextflow.Session - Session start invoked
Oct.-12 10:33:05.706 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/pipeline_trace.txt
Oct.-12 10:33:05.921 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Oct.-12 10:33:05.969 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/; pattern: *.fastq; options: [:]
Oct.-12 10:33:05.998 [PathVisitor-1] DEBUG nextflow.file.PathVisitor - files for syntax: glob; folder: /home/shaun/nextflow_pipelines/pipelines/pomoxis_map_assemble/ref/; pattern: *.fasta; options: [:]
Oct.-12 10:33:06.048 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Oct.-12 10:33:06.048 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Oct.-12 10:33:06.052 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Oct.-12 10:33:06.056 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=16; memory=62.5 GB; capacity=16; pollInterval=100ms; dumpInterval=5m
Oct.-12 10:33:06.121 [main] DEBUG nextflow.Session - Workflow process names [dsl1]: pomoxis
Oct.-12 10:33:06.133 [main] WARN  nextflow.Session - There's no process matching config selector: fastqc
Oct.-12 10:33:06.134 [main] WARN  nextflow.Session - There's no process matching config selector: porechop
Oct.-12 10:33:06.135 [main] WARN  nextflow.Session - There's no process matching config selector: bioawk
Oct.-12 10:33:06.135 [main] WARN  nextflow.Session - There's no process matching config selector: fastqconvert
Oct.-12 10:33:06.135 [main] WARN  nextflow.Session - There's no process matching config selector: blast_raw
Oct.-12 10:33:06.135 [main] DEBUG nextflow.script.ScriptRunner - > Await termination 
Oct.-12 10:33:06.135 [main] DEBUG nextflow.Session - Session await
Oct.-12 10:33:06.344 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Oct.-12 10:33:06.349 [Task submitter] INFO  nextflow.Session - [cf/bc2b69] Submitted process > pomoxis (1)
Oct.-12 10:33:06.457 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: pomoxis (1); status: COMPLETED; exit: 127; error: -; workDir: /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/work/cf/bc2b696aea0863d76c2c9221315c39]
Oct.-12 10:33:06.491 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'pomoxis (1)'

Caused by:
  Process `pomoxis (1)` terminated with an error exit status (127)

Command executed:

  mini_align -i output.fastq -r /home/shaun/nextflow_pipelines/pipelines/pomoxis_map_assemble/ref/*.fasta -o results -p > output_test_final.fa

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.sh: line 2: mini_align: command not found

Work dir:
  /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/work/cf/bc2b696aea0863d76c2c9221315c39

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
Oct.-12 10:33:06.499 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `pomoxis (1)` terminated with an error exit status (127)
Oct.-12 10:33:06.503 [main] DEBUG nextflow.Session - Session await > all process finished
Oct.-12 10:33:06.520 [main] DEBUG nextflow.Session - Session await > all barriers passed
Oct.-12 10:33:06.540 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=1.2s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=16; peakMemory=60 GB; ]
Oct.-12 10:33:06.540 [main] DEBUG nextflow.trace.TraceFileObserver - Flow completing -- flushing trace file
Oct.-12 10:33:06.715 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Oct.-12 10:33:06.728 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

У меня такая же ошибка, когда я пытаюсь использовать pomoxis для devno сборки чтения:

shaun@shaun-HP-Z6-G4-Workstation:~/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble$ nextflow pomoxis_denovo_nextflow.nf
N E X T F L O W  ~  version 22.04.5
Launching `pomoxis_denovo_nextflow.nf` [intergalactic_sammet] DSL1 - revision: 1927bae9ab
executor >  local (1)
[fb/3868bc] process > pomoxis (1) [100%] 1 of 1, failed: 1 ✘
WARN: There's no process matching config selector: fastqc
WARN: There's no process matching config selector: porechop
WARN: There's no process matching config selector: bioawk
WARN: There's no process matching config selector: fastqconvert
WARN: There's no process matching config selector: blast_raw
Error executing process > 'pomoxis (1)'

Caused by:
  Process `pomoxis (1)` terminated with an error exit status (127)

Command executed:

  mini_assemble -i output.fastq -o results -p > output_test_final.fa

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.sh: line 2: mini_assemble: command not found

Work dir:
  /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_map_assemble/work/fb/3868bc97cabfa0d2934f4344c13c89

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

Это результат для .command.sh

#!/bin/bash -ue
mini_assemble -i output.fastq -o results -p > output_test_final.fa

Я использовал флаги для помоксиса отсюда --> https://nanoporetech.github.io/pomoxis/programs.html

Ниже приведены два файла .nf:

#!/usr/bin/env nextflow

//data_location

params.outdir = './results'
params.in = "$PWD/*.fastq"
datasetA = Channel.fromPath(params.in)
params.ref = "$PWD/ref/*.fasta"
datasetB = Channel.fromPath(params.ref)


//map and assemble

process pomoxis {

    publishDir "${params.outdir}", mode:'copy'

    input:
    path (c) from datasetA

    output:
    path "${c.baseName}_test_final.fa" into mapped_ch
  
    script:
    """
    mini_align -i $c -r $params.ref -o results -p > ${c.baseName}_test_final.fa 
    """
}

#!/usr/bin/env nextflow

//data_location

params.outdir = './results'
params.in = "$PWD/*.fastq"
dataset = Channel.fromPath(params.in)


//devno only

process pomoxis {
    
    publishDir "${params.outdir}", mode:'copy'
        
    input:
    path (c) from dataset

    output:
    path "${c.baseName}_test_final.fa" into mapped_ch
  
    script:
    """
    mini_assemble -i $c -o results -p > ${c.baseName}_test_final.fa 
    """
}

Вот файл конфигурации, я пробовал несколько контейнеров докеров из концентратора докеров, но получаю одно и то же сообщение об ошибке.

//resume = true

process {
            cpus = 16
            accelerator = 'Quadro-RTX-5000'
              memory = 60. GB
}

trace {
        enabled = true
        file = 'pipeline_trace.txt'
        fields = 'task_id,hash,process,name,status,exit,module,container,cpus,time,disk,memory,attempt,submit,start,complete,duration,realtime,queue,%cpu,%mem'
}

docker {
            enabled = true
            temp = 'auto'
            runOption = '--user root' 
}

params {
            nt_db_20221011 = '/home/shaun/blast/nt_db_20221011/nt'
}

process {
            withName:fastqc      {container = 'staphb/fastqc:latest'                                               }
            withName:porechop    {container = 'quay.io/biocontainers/porechop:0.2.3_seqan2.1.1--py36h2d50403_3'    }
            withName:bioawk      {container = 'wslhbio/bioawk:1.0-wslh-signed'                                     }
            withName:fastqconvert{container = 'staphb/seqtk:1.3'                                                   }
            withName:blast_raw   {container = 'staphb/blast:2.13.0'                                                }
            withname:pomoxis    {container = 'dpirdmk/pomoxis:0.1.11'                          }
}

Обновлять:

После исправления синтаксической ошибки я изменил контейнер, как было предложено, и добавил функцию -h в раздел сценария. Ниже приводится вывод из этого:


shaun@shaun-HP-Z6-G4-Workstation:~/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_align$ nextflow pomoxis_align_nextflow.nf
N E X T F L O W  ~  version 22.04.5
Launching `pomoxis_align_nextflow.nf` [voluminous_solvay] DSL1 - revision: 4dca1d1d46
executor >  local (1)
[55/4dad0b] process > pomoxis (1) [  0%] 0 of 1
WARN: There's no process matching config selector: fastqc
WARN: There's no process matching config selector: porechop
WARN: There's no process matching config selector: bioawk
WARN: There's no process matching config selector: fastqconvert
WARN: There's no process matching config selector: blast_raw
Error executing process > 'pomoxis (1)'

Caused by:
  Missing output file(s) `output` expected by process `pomoxis (1)`

Command executed:

  mini_align -h

Command exit status:
  0

Command output:
  (empty)
executor >  local (1)
[55/4dad0b] process > pomoxis (1) [100%] 1 of 1, failed: 1 ✘
WARN: There's no process matching config selector: fastqc
WARN: There's no process matching config selector: porechop
WARN: There's no process matching config selector: bioawk
WARN: There's no process matching config selector: fastqconvert
WARN: There's no process matching config selector: blast_raw
Error executing process > 'pomoxis (1)'

Caused by:
  Missing output file(s) `output` expected by process `pomoxis (1)`

Command executed:

  mini_align -h

Command exit status:
  0

Command output:
  (empty)

Command error:
  mini_align [-h] -r <reference> -i <fastq>

  Align fastq/a formatted reads to a genome using minimap2.

      -h  show this help text.
      -r  reference, should be a fasta file. If correspondng minimap indices
          do not exist they will be created. (required).
      -i  fastq/a input reads (required).
      -I  split index every ~NUM input bases (default: 16G, this is larger
          than the usual minimap2 default).
      -d  set the minimap2 preset, e.g. map-ont, asm5, asm10, asm20 [default: map-ont]
      -f  force recreation of index file.
      -a  aggressively extend gaps (sets -A1 -B2 -O2 -E1 for minimap2).
      -P  filter to only primary alignments (i.e. run samtools view -F 2308).
          Deprecated: this filter is now default and can be disabled with -A.
      -y  filter to primary and supplementary alignments (i.e. run samtools view -F 260)
      -A  do not filter alignments, output all.
      -n  sort bam by read name.
      -c  chunk size. Input reads/contigs will be broken into chunks
          prior to alignment.
      -t  alignment threads (default: 1).
      -p  output file prefix (default: reads).
      -m  fill MD tag.
      -s  fill cs(=long) tag.
      -X  only create reference index files.
      -x  log all commands before running.
      -M  match score
      -S  mismatch score
      -O  open gap penalty
      -E  extend gap penalty.

Work dir:
  /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_align/work/55/4dad0b1df7105e31362f58b0c67f1f

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

Хотя есть ошибка, я предполагаю, что это сработало, поскольку я вижу меню справки?

Затем я добавил флаги в раздел сценария .nf согласно веб-сайту Pomoxis.

#!/usr/bin/env nextflow

//data_location

params.outdir = './results'
params.in = "$PWD/*.fastq"
datasetA = Channel.fromPath(params.in)
params.ref = "$PWD/SLCMV.fasta"
datasetB = Channel.fromPath(params.ref)


//map and assemble; input fastq only
//output; pomoxis is very particular about names, minimap can out put 
//.sam or .paf 

process pomoxis {

    publishDir "${params.outdir}", mode:'copy'
        
    input:
    path (c) from datasetA
    path (d) from datasetB

    output:
    path "${c.simpleName}" into mapped_ch
  
    script:
    """
    mini_align -r $d -i $c -p ${c.simpleName}
    """
}

Результаты ниже:

shaun@shaun-HP-Z6-G4-Workstation:~/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_align$ nextflow pomoxis_align_nextflow.nf
N E X T F L O W  ~  version 22.04.5
Launching `pomoxis_align_nextflow.nf` [determined_bassi] DSL1 - revision: 1ad3e75c1b
executor >  local (1)
[50/860f9f] process > pomoxis (1) [  0%] 0 of 1
WARN: There's no process matching config selector: fastqc
WARN: There's no process matching config selector: porechop
WARN: There's no process matching config selector: bioawk
WARN: There's no process matching config selector: fastqconvert
WARN: There's no process matching config selector: blast_raw
Error executing process > 'pomoxis (1)'

Caused by:
  Missing output file(s) `output` expected by process `pomoxis (1)`
executor >  local (1)
[50/860f9f] process > pomoxis (1) [100%] 1 of 1, failed: 1 ✘
WARN: There's no process matching config selector: fastqc
WARN: There's no process matching config selector: porechop
WARN: There's no process matching config selector: bioawk
WARN: There's no process matching config selector: fastqconvert
WARN: There's no process matching config selector: blast_raw
Error executing process > 'pomoxis (1)'

Caused by:
  Missing output file(s) `output` expected by process `pomoxis (1)`

Command executed:

  mini_align -r SLCMV.fasta -i output.fastq -p output

Command exit status:
  0

Command output:
  (empty)

Command error:
  Creating fai index file SLCMV.fasta.fai
  Creating mmi index file SLCMV.fasta.map-ont.mmi
  [M::mm_idx_gen::0.003*2.10] collected minimizers
  [M::mm_idx_gen::0.005*2.42] sorted minimizers
  [M::main::0.009*1.77] loaded/built the index for 1 target sequence(s)
  [M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 1
  [M::mm_idx_stat::0.009*1.74] distinct minimizers: 522 (100.00% are singletons); average occurrences: 1.000; average spacing: 5.282; total length: 2757
  [M::main] Version: 2.24-r1122
  [M::main] CMD: minimap2 -I 16G -x map-ont -d SLCMV.fasta.map-ont.mmi SLCMV.fasta
  [M::main] Real time: 0.010 sec; CPU: 0.017 sec; Peak RSS: 0.003 GB
  [M::main::0.005*1.39] loaded/built the index for 1 target sequence(s)
  [M::mm_mapopt_update::0.005*1.37] mid_occ = 10
  [M::mm_idx_stat] kmer size: 15; skip: 10; is_hpc: 0; #seq: 1
  [M::mm_idx_stat::0.005*1.35] distinct minimizers: 522 (100.00% are singletons); average occurrences: 1.000; average spacing: 5.282; total length: 2757
  [M::worker_pipeline::9.159*1.00] mapped 87369 sequences
  [M::main] Version: 2.24-r1122
  [M::main] CMD: minimap2 -x map-ont --secondary=no -L -t 1 -a SLCMV.fasta.map-ont.mmi output.fastq
  [M::main] Real time: 9.182 sec; CPU: 9.145 sec; Peak RSS: 0.299 GB

Work dir:
  /home/shaun/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_align/work/50/860f9ffe37f7a17a4d5e216f67b512

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

Я не знаю, как это исправить, я пытался изменить имя вывода, но это не помогло.

интересно, когда я запускаю mini_assemble .nf:

#!/usr/bin/env nextflow

//data_location

params.outdir = './results'
params.in = "$PWD/*.fastq"
dataset = Channel.fromPath(params.in)

//devno only; input is fastq
//output; pomoxis is very particular about names, the name that it out puts is
//prefix_test_final.fa where prefic can be anything and it will add _test_final.fa
//to the end of the file


process pomoxis {

    tag "$c"
    publishDir "${params.outdir}", mode:'copy'
        
    input:
    path (c) from dataset

    output:
    path ("${c.simpleName}_test_final.fa") into mapped_ch
  
    script:
    """
    mini_assemble -i $c -p ${c.simpleName}
    """ 
}

Я получаю следующую ошибку:

shaun@shaun-HP-Z6-G4-Workstation:~/nextflow_pipelines/pipelines/WA_MomicaKehoe/Asad/nf_pipeline/pomoxis_assemble$ nextflow pomoxis_assemble_nextflow.nf
N E X T F L O W  ~  version 22.04.5
Launching `pomoxis_assemble_nextflow.nf` [intergalactic_fermi] DSL1 - revision: ce7a466307
executor >  local (1)
[04/28f5e6] process > pomoxis (output.fastq) [  0%] 0 of 1


Error executing process > 'pomoxis (output.fastq)'

Caused by:
  Missing output file(s) `output_test_final.fa` expected by process `pomoxis (output.fastq)`

Command executed:

  mini_assemble -i output.fastq -p output

Command exit status:
  0

Command output:
  Copying FASTX input to workspace: output.fastq > assm/output.fa.gz
  Skipped adapter trimming.
  Skipped pre-assembly correction.
  Overlapping reads...
  Assembling graph...
  Running racon read shuffle 1...
  Running round 1 consensus...
  Running round 2 consensus...
  Running round 3 consensus...
  Running round 4 consensus...
  Waiting for cleanup.
  Final assembly written to assm/output_final.fa. Have a nice day.

Command error:
  [M::mm_idx_stat::0.065*1.04] distinct minimizers: 69519 (92.83% are singletons); average occurrences: 1.101; average spacing: 5.337; total length: 408518
  [M::worker_pipeline::9.872*0.97] mapped 87369 sequences
  [M::main] Version: 2.24-r1122
  [M::main] CMD: minimap2 -L -K 500M -t 1 racon_1_3.fa.gz output.fa.gz
  [M::main] Real time: 9.880 sec; CPU: 9.620 sec; Peak RSS: 0.165 GB
  [racon::Polisher::initialize] loaded target sequences 0.011974 s
  [racon::Polisher::initialize] loaded sequences 2.047803 s
  [racon::Polisher::initialize] loaded overlaps 0.106485 s
  [racon::Polisher::initialize] aligning overlaps [=>                  ] 1.252726 s
  [racon::Polisher::initialize] aligning overlaps [==>                 ] 1.827141 s
  [racon::Polisher::initialize] aligning overlaps [===>                ] 2.413558 s
  [racon::Polisher::initialize] aligning overlaps [====>               ] 2.974846 s
  [racon::Polisher::initialize] aligning overlaps [=====>              ] 3.609222 s
  [racon::Polisher::initialize] aligning overlaps [======>             ] 4.263570 s
  [racon::Polisher::initialize] aligning overlaps [=======>            ] 4.934294 s
  [racon::Polisher::initialize] aligning overlaps [========>           ] 5.554714 s
  [racon::Polisher::initialize] aligning overlaps [=========>          ] 6.164404 s
  [racon::Polisher::initialize] aligning overlaps [==========>         ] 6.746845 s
executor >  local (1)
[04/28f5e6] process > pomoxis (output.fastq) [100%] 1 of 1, failed: 1 ✘
Error executing process > 'pomoxis (output.fastq)'

Caused by:
  Missing output file(s) `output_test_final.fa` expected by process `pomoxis (output.fastq)`

Command executed:

  mini_assemble -i output.fastq -p output

Command exit status:
  0

Command output:
  Copying FASTX input to workspace: output.fastq > assm/output.fa.gz
  Skipped adapter trimming.
  Skipped pre-assembly correction.
  Overlapping reads...
  Assembling graph...
  Running racon read shuffle 1...
  Running round 1 consensus...
  Running round 2 consensus...
  Running round 3 consensus...
  Running round 4 consensus...
  Waiting for cleanup.
  Final assembly written to assm/output_final.fa. Have a nice day.

Command error:
  [M::mm_idx_stat::0.065*1.04] distinct minimizers: 69519 (92.83% are singletons); average occurrences: 1.101; average spacing: 5.337; total length: 408518
  [M::worker_pipeline::9.872*0.97] mapped 87369 sequences
  [M::main] Version: 2.24-r1122
  [M::main] CMD: minimap2 -L -K 500M -t 1 racon_1_3.fa.gz output.fa.gz
  [M::main] Real time: 9.880 sec; CPU: 9.620 sec; Peak RSS: 0.165 GB
  [racon::Polisher::initialize] loaded target sequences 0.011974 s
  [racon::Polisher::initialize] loaded sequences 2.047803 s
  [racon::Polisher::initialize] loaded overlaps 0.106485 s
  [racon::Polisher::initialize] aligning overlaps [=>                  ] 1.252726 s
  [racon::Polisher::initialize] aligning overlaps [==>                 ] 1.827141 s
  [racon::Polisher::initialize] aligning overlaps [===>                ] 2.413558 s
  [racon::Polisher::initialize] aligning overlaps [====>               ] 2.974846 s
  [racon::Polisher::initialize] aligning overlaps [=====>              ] 3.609222 s
  [racon::Polisher::initialize] aligning overlaps [======>             ] 4.263570 s
  [racon::Polisher::initialize] aligning overlaps [=======>            ] 4.934294 s
  [racon::Polisher::initialize] aligning overlaps [========>           ] 5.554714 s
  [racon::Polisher::initialize] aligning overlaps [=========>          ] 6.164404 s
  [racon::Polisher::initialize] aligning overlaps [==========>         ] 6.746845 s
  [racon::Polisher::initialize] aligning overlaps [===========>        ] 7.282895 s
  [racon::Polisher::initialize] aligning overlaps [============>       ] 7.878856 s
  [racon::Polisher::initialize] aligning overlaps [=============>      ] 8.459672 s
  [racon::Polisher::initialize] aligning overlaps [==============>     ] 8.984196 s
  [racon::Polisher::initialize] aligning overlaps [===============>    ] 9.558089 s
  [racon::Polisher::initialize] aligning overlaps [================>   ] 10.072586 s
  [racon::Polisher::initialize] aligning overlaps [=================>  ] 10.648603 s
  [racon::Polisher::initialize] aligning overlaps [==================> ] 11.220553 s
  [racon::Polisher::initialize] aligning overlaps [===================>] 11.753881 s
  [racon::Polisher::initialize] aligning overlaps [====================] 12.364460 s
  [racon::Polisher::initialize] transformed data into windows 0.018851 s
  [racon::Polisher::polish] generating consensus [=>                  ] 0.515508 s
  [racon::Polisher::polish] generating consensus [==>                 ] 1.077083 s
  [racon::Polisher::polish] generating consensus [===>                ] 1.561330 s
  [racon::Polisher::polish] generating consensus [====>               ] 1.967063 s
  [racon::Polisher::polish] generating consensus [=====>              ] 2.574379 s
  [racon::Polisher::polish] generating consensus [======>             ] 3.037187 s
  [racon::Polisher::polish] generating consensus [=======>            ] 3.740370 s
  [racon::Polisher::polish] generating consensus [========>           ] 20.112573 s
  [racon::Polisher::polish] generating consensus [=========>          ] 46.718507 s
  [racon::Polisher::polish] generating consensus [==========>         ] 48.913841 s
  [racon::Polisher::polish] generating consensus [===========>        ] 52.859126 s
  [racon::Polisher::polish] generating consensus [============>       ] 57.495382 s
  [racon::Polisher::polish] generating consensus [=============>      ] 74.111321 s
  [racon::Polisher::polish] generating consensus [==============>     ] 79.382051 s
  [racon::Polisher::polish] generating consensus [===============>    ] 113.129291 s
  [racon::Polisher::polish] generating consensus [================>   ] 115.971094 s
  [racon::Polisher::polish] generating consensus [=================>  ] 123.322513 s
  [racon::Polisher::polish] generating consensus [==================> ] 128.155934 s
  [racon::Polisher::polish] generating consensus [===================>] 130.509134 s
  [racon::Polisher::polish] generating consensus [====================] 131.095442 s
  [racon::Polisher::] total = 145.794273 s

Work dir:
  /home/shaun/nextflow_pipelines/pipelines/WA_MomicaKehoe/Asad/nf_pipeline/pomoxis_assemble/work/04/28f5e6eed13d3fc986739e287594d9

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

Однако файл, который мне нужен, находится в:

shaun@shaun-HP-Z6-G4-Workstation:~/nextflow_pipelines/pipelines/nf_pipeline/pomoxis_assemble/work/04/28f5e6eed13d3fc986739e287594d9/assm$ cat output_final.fa

Когда я просматриваю этот файл, я вижу, что в файле есть контиги, как и ожидалось.

Я вижу, что pomoxis ожидает output_test_final.fa, который действительно находится в рабочем каталоге, но pomoxis не может его увидеть или использовать?

Есть ли у вас какие-либо предложения о том, почему эти ошибки происходят? Я ценю ваше время.

Я попытался добавить содержимое удаленного ответа, но достиг предела в 30000 символов. Я удалил часть об установке/настройке Docker, чтобы она подходила. Пожалуйста, рассмотрите возможность пересмотра вашего вопроса, если это необходимо.

Steve 13.10.2022 17:30
Развертывание модели машинного обучения с помощью Flask - Angular в Kubernetes
Развертывание модели машинного обучения с помощью Flask - Angular в Kubernetes
Kubernetes - это портативная, расширяемая платформа с открытым исходным кодом для управления контейнерными рабочими нагрузками и сервисами, которая...
Как создать PHP Image с нуля
Как создать PHP Image с нуля
Сегодня мы создадим PHP Image from Scratch для того, чтобы легко развернуть базовые PHP-приложения. Пожалуйста, имейте в виду, что это разработка для...
1
1
112
1
Перейти к ответу Данный вопрос помечен как решенный

Ответы 1

Ответ принят как подходящий
Command error:
  .command.sh: line 2: mini_assemble: command not found

Вы получите эту ошибку, когда попытаетесь запустить команду вне контейнера, а исполняемый файл не существует в вашем $PATH. Обычно мы видим это, когда забываем добавить docker.enabled = true (или singularity.enabled = true при использовании Singualarity) в наш nextflow.config . Однако в данном случае это просто вызвано опечаткой в ​​вашем файле конфигурации. Селектор конфигурации withName должен иметь верблюжий регистр. Вы также получите сообщение об ошибке «команда не найдена» при использовании выбранного контейнера. Если вместо этого использовать биоконтейнер помоксиса, то мы получим ожидаемый результат. Например:

Содержимое nextflow.config:

docker {
    enabled = true
}

process {

    withName: pomoxis {
        container = 'quay.io/biocontainers/pomoxis:0.3.10--pyhdfd78af_0'
    }
}

Содержимое map_and_assembly.nf:

process pomoxis {

    """
    mini_align -h
    """
}

Результаты:

$ nextflow run map_and_assembly.nf -dsl1
N E X T F L O W  ~  version 22.04.4
Launching `map_and_assembly.nf` [loving_thompson] DSL1 - revision: 6efbc53bf8
executor >  local (1)
[22/23c938] process > pomoxis [100%] 1 of 1 ✔

Completed at: 13-Oct-2022 01:05:27
Duration    : 2m 46s
CPU hours   : (a few seconds)
Succeeded   : 1
$ cat work/22/23c938ca9312037ee10087ba5d48a2/.command.err 
Unable to find image 'quay.io/biocontainers/pomoxis:0.3.10--pyhdfd78af_0' locally
0.3.10--pyhdfd78af_0: Pulling from biocontainers/pomoxis
c1a16a04cedd: Already exists
4ca545ee6d5d: Already exists
af0d0c971daf: Pulling fs layer
af0d0c971daf: Verifying Checksum
af0d0c971daf: Download complete
af0d0c971daf: Pull complete
Digest: sha256:b42d95b742be3dc8333f57892c4aa2cc5cd739e796b33c7f310696856dcdea4d
Status: Downloaded newer image for quay.io/biocontainers/pomoxis:0.3.10--pyhdfd78af_0
mini_align [-h] -r <reference> -i <fastq>

Align fastq/a formatted reads to a genome using minimap2.

    -h  show this help text.
    -r  reference, should be a fasta file. If correspondng minimap indices
        do not exist they will be created. (required).
    -i  fastq/a input reads (required).
    -I  split index every ~NUM input bases (default: 16G, this is larger
        than the usual minimap2 default).
    -d  set the minimap2 preset, e.g. map-ont, asm5, asm10, asm20 [default: map-ont]
    -f  force recreation of index file.
    -a  aggressively extend gaps (sets -A1 -B2 -O2 -E1 for minimap2).
    -P  filter to only primary alignments (i.e. run samtools view -F 2308).
        Deprecated: this filter is now default and can be disabled with -A.
    -y  filter to primary and supplementary alignments (i.e. run samtools view -F 260)
    -A  do not filter alignments, output all.
    -n  sort bam by read name.
    -c  chunk size. Input reads/contigs will be broken into chunks
        prior to alignment.
    -t  alignment threads (default: 1).
    -p  output file prefix (default: reads).
    -m  fill MD tag.
    -s  fill cs(=long) tag.
    -X  only create reference index files.
    -x  log all commands before running.
    -M  match score 
    -S  mismatch score
    -O  open gap penalty
    -E  extend gap penalty.

Проблема с использованием контейнера dpirdmk/pomoxis:0.1.11 заключается в том, что он указывает альтернативную точку входа, что затрудняет использование «из коробки» без изменения выполняемой команды или предоставления некоторой дополнительной конфигурации Docker:

$ docker run --rm dpirdmk/pomoxis:0.1.11 bash -c 'cat /init.sh'
#!/bin/bash
. /apps/pomoxis/venv/bin/activate
exec "$@"

Ответ на уточняющие вопросы:

Вы получаете ошибку Missing output file(s) `output` expected by process, когда команда завершается успешно (т. е. статус выхода равен нулю), но Nextflow (не pomoxis) не может найти один или несколько выходных файлов в рабочем каталоге, как указано в объявлении вывода. Я думаю, что вы хотите следующее. Я взял на себя смелость переименовать некоторые переменные, чтобы было немного понятнее. Код ниже, конечно, не проверен:

params.outdir = './results'

params.ref_fasta = "SLCMV.fasta"
params.reads = "*.fastq"

reads = Channel.fromPath( params.reads )
ref_fasta = file( params.ref_fasta )


process mini_align {

    tag { fastq.name }

    publishDir "${params.outdir}/mini_align", mode:'copy'

    input:
    path fastq from reads
    path ref_fasta

    output:
    path "${fastq.simpleName}.bam{,.bai}" into mapped_ch

    script:
    """
    mini_align \\
        -r "${ref_fasta}" \\
        -i "${fastq}" \\
        -p "${fastq.simpleName}"
    """
}
params.outdir = './results'

params.reads = "*.fastq"

reads = Channel.fromPath( params.reads )


process mini_assemble {

    tag { fastq.name }

    publishDir "${params.outdir}/mini_assemble", mode:'copy'

    input:
    path fastq from reads

    output:
    path "assm/${fastq.simpleName}_final.fa" into mapped_ch

    script:
    """
    mini_assemble \\
        -i "${fastq}" \\
        -p "${fastq.simpleName}"
    """
}

Спасибо, Стив, я последовал вашим советам выше. Они исправили первоначальные ошибки, хотя есть еще несколько, если у вас есть время. Я добавил их в качестве ответа.

naqs 13.10.2022 05:52

@naqs Пожалуйста, смотрите мой обновленный ответ выше. Ваше здоровье.

Steve 13.10.2022 17:33

Привет, Стив, спасибо за тонкие, но важные изменения! Я следовал приведенным выше сценариям, и теперь он работает для меня. Все еще читаю документы, пытаясь обернуть голову вокруг этого материала. Я ценю ваше время и усилия, чтобы помочь мне.

naqs 14.10.2022 04:29

Другие вопросы по теме