qiime执行multiple_join_paired_ends.py 报错Cannot find fastq-join.

centos 6.6 通过 conda 安装 qiime后, 在执行multiple_join_paired_ends.py 时, 报错:

burrito.util.ApplicationNotFoundError: Cannot find fastq-join. Is it installed? Is it in your path?

这是因为系统里没有 fastq-join

安装 fastq-join 有两种方式:

1. 安装 ea-utils.

https://github.com/ExpressionAnalysis/ea-utils/blob/wiki/Compiling.md

ea-utils 就是 这个博客提到的 : QIIME组装paired-end reads的两个实用工具

fastq-join - Erik Aronesty, 2011. ea-utils : “Command-line tools for processing biological sequencing data” ( http://code.google.com/p/ea-utils)

其实上面博客的内容也是在qiime官网里介绍的:

http://qiime.org/scripts/join_paired_ends.html

Currently, there are two methods that can be selected by the user to join paired-end data:

fastq-join - Erik Aronesty, 2011. ea-utils : “Command-line tools for processing biological sequencing data” ( http://code.google.com/p/ea-utils)

现在没在code.google.com上了,保存在 github上.

从github上clone, 在安装必须的库:

sudo yum install gsl-devel

之后, make,结果就出错了:

$make

prove -j 4 t

===( 2;0 1/? 1/? 0/? )=========================================Can't locate object method "is_passing" via package "Test::Builder" at t/test-prep.pl line 72.

END failed--call queue aborted.

# Looks like your test exited with 22 just after 18.

t/join.t ... Dubious, test returned 22 (wstat 5632, 0x1600)

All 18 subtests passed

===( 45;1 27/? 0/? )=============================================Can't locate object method "is_passing" via package "Test::Builder" at t/test-prep.pl line 72.

END failed--call queue aborted.

# Looks like your test exited with 22 just after 28.

t/mcf.t .... Dubious, test returned 22 (wstat 5632, 0x1600)

All 28 subtests passed

t/multx.t .. 1/? Can't locate object method "is_passing" via package "Test::Builder" at t/test-prep.pl line 72.

END failed--call queue aborted.

# Looks like your test exited with 22 just after 12.

t/multx.t .. Dubious, test returned 22 (wstat 5632, 0x1600)

All 12 subtests passed

Test Summary Report

-------------------

t/join.t (Wstat: 5632 Tests: 18 Failed: 0)

Non-zero exit status: 22

t/mcf.t (Wstat: 5632 Tests: 28 Failed: 0)

Non-zero exit status: 22

t/multx.t (Wstat: 5632 Tests: 12 Failed: 0)

Non-zero exit status: 22

Files=3, Tests=58, 4 wallclock secs ( 0.02 usr 0.01 sys + 0.96 cusr 3.81 csys = 4.80 CPU)

Result: FAIL

make: *** [check] Error 1

在这一步,实际已经完成了编译. 但是被test报错信息所误导.

于是进行了第二次尝试:

方法2. 单独安装 fastq-join

https://github.com/brwnj/fastq-join

git clone https://github.com/brwnj/fastq-join

cd fastq-join

make

到这里,就已经发现, 可以编译出来, 并且回去看了之前的编译, 也是可以正常编译出来bin

既然方法1 可以正常编译出来,就把他们make install 到 自定义路径下 (第三方工具软件统一放到 /usr/local/bin/下):

sudo make install BINDIR=/usr/local/bin/ea-utils

然后试用:

PATH=/usr/local/bin/ea-utils/:/usr/local/bin/conda/miniconda/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin

multiple_join_paired_ends.py -i input_data/ -o output_data/

试验成功后, 又返回去看之前的报错. 

其实到这里问题已经解决了,但是因为没碰到过这样的错误.所以要搞清楚.

在方法2 编译后的 test文件夹 , 尝试

$perl test-prep.pl

Can't locate object method "is_passing" via package "Test::Builder" at test-prep.pl line 72.

END failed--call queue aborted.

和方法1在make报错的一样.

看来是因为方法2的Makefile没有加入test,所以在编译时没报错. 查看Makefile:

$cat Makefile

CC=g++

CFLAGS=-O3 -I.

all: fastq-join

%: %.cpp fastq-lib.cpp fastq-lib.h

$(CC) $(CFLAGS) $< fastq-lib.cpp -o $@

check:

prove -j 4 tests

clean:

rm -rf fastq-join tests/tmp/join*

到这里,就已经很清楚,是因为 Can't locate object method "is_passing" via package "Test::Builder"导致的.

---

解决 Can't locate object method "is_passing" via package "Test::Builder" 问题.

在下面知道的是Test模块的版本问题:

I got this too with Test::More version 0.8, but had better luck with Test::More version 0.93 . – mob Oct 2 '09 at 15:12

https://stackoverflow.com/questions/1506674/can-i-tell-if-all-tests-passed-under-perls-testmore

于是看Test::Builder的版本:

$perl -MTest::Builder -e 'print Test::Builder->VERSION."\n"'

0.92

在notebook上编译和测试都没问题. 所以可以对比notebook的:

perl -MTest::Builder -e 'print Test::Builder->VERSION."\n" '

0.98

于是可以确定是 Test::Builder 的版本导致的 Can't locate object method "is_passing" via package "Test::Builder"问题.

https://metacpan.org/pod/Test::Builder 看 Changes

https://metacpan.org/changes/distribution/Test-Simple

可以看到changelogs , 0.89_01 2009-Jun-23 进行了一下升级, 但在Test-Simple 0.90 2009-Jul-2 没使用, 直到Test-Simple 0.93_01 2009-Jul-20 才加进去:

Test-Simple 0.94 2009-Sep-2 11:17-08:00 PDT

Releasing 0.93_01 as stable.

Test-Simple 0.93_01 2009-Jul-20 09:51-08:00 PDT

Bug Fixes

* Make sure that subtest works with Test:: modules which call

Test::Builder->new at the top of their code. (Ovid)

Other

* subtest() returns!

Test-Simple 0.92 2009-Jul-3 11:08-08:00 PDT

Test Fixes

* Silence noise on VMS in exit.t (Craig Berry)

* Skip Builder/fork_with_new_stdout.t on systems without fork (Craig Berry)

Test-Simple 0.90 2009-Jul-2 13:18-08:00 PDT

Docs

* Note the IO::Stringy license in our copy of it.

[test-more.googlecode.com 47]

Other

* This is a stable release for 5.10.1. It does not include

the subtest() work in 0.89_01.

Test-Simple 0.89_01 2009-Jun-23 15:13-08:00 EDT

New Features

* subtest() allows you to run more tests in their own plan.

(Thanks Ovid!)

* Test::Builder->is_passing() will let you check if the test is

currently passing.

Docs

* Finally added a note about the "Wide character in print" warning and

how to work around it.

Test Fixes

* Small fixes for integration with the Perl core

[bleadperl eaa0815147e13cd4ab5b3d6ca8f26544a9f0c3b4]

* exit code tests could be effected by errno when PERLIO=stdio

[bleadperl c76230386fc5e6fba9fdbeab473abbf4f4adcbe3]

---

接下来问题就简单了,下载没有问题的Test::Builder模块源码,编译后替换.

在 https://metacpan.org/release/MSCHWERN/Test-Simple-0.98 的TOOLS 下载 Download (104.74Kb)

然后升级模块:

perl Makefile.PL

Checking if your kit is complete...

Looks good

Writing Makefile for Test::Simple

$make

cp lib/Test/Builder/Tester/Color.pm blib/lib/Test/Builder/Tester/Color.pm

cp lib/Test/Simple.pm blib/lib/Test/Simple.pm

cp lib/Test/Builder.pm blib/lib/Test/Builder.pm

cp lib/Test/Builder/IO/Scalar.pm blib/lib/Test/Builder/IO/Scalar.pm

cp lib/Test/More.pm blib/lib/Test/More.pm

cp lib/Test/Builder/Tester.pm blib/lib/Test/Builder/Tester.pm

cp lib/Test/Builder/Module.pm blib/lib/Test/Builder/Module.pm

cp lib/Test/Tutorial.pod blib/lib/Test/Tutorial.pod

Manifying blib/man3/Test::Builder::Tester::Color.3pm

Manifying blib/man3/Test::Builder.3pm

Manifying blib/man3/Test::Simple.3pm

Manifying blib/man3/Test::Builder::IO::Scalar.3pm

Manifying blib/man3/Test::More.3pm

Manifying blib/man3/Test::Builder::Module.3pm

Manifying blib/man3/Test::Builder::Tester.3pm

Manifying blib/man3/Test::Tutorial.3pm

make test

PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/*/*.t

t/00compile.t ........................... ok

t/00test_harness_check.t ................ ok

t/bad_plan.t ............................ ok

t/bail_out.t ............................ ok

t/BEGIN_require_ok.t .................... ok

t/BEGIN_use_ok.t ........................ ok

t/buffer.t .............................. ok

t/Builder/Builder.t ..................... ok

t/Builder/carp.t ........................ ok

t/Builder/create.t ...................... ok

t/Builder/current_test.t ................ ok

t/Builder/current_test_without_plan.t ... ok

t/Builder/details.t ..................... ok

t/Builder/done_testing.t ................ ok

t/Builder/done_testing_double.t ......... ok

t/Builder/done_testing_plan_mismatch.t .. ok

t/Builder/done_testing_with_no_plan.t ... ok

t/Builder/done_testing_with_number.t .... ok

t/Builder/done_testing_with_plan.t ...... ok

t/Builder/fork_with_new_stdout.t ........ ok

t/Builder/has_plan.t .................... ok

t/Builder/has_plan2.t ................... ok

t/Builder/is_fh.t ....................... ok

t/Builder/is_passing.t .................. ok

t/Builder/maybe_regex.t ................. ok

t/Builder/no_diag.t ..................... ok

t/Builder/no_ending.t ................... ok

t/Builder/no_header.t ................... ok

t/Builder/no_plan_at_all.t .............. ok

t/Builder/ok_obj.t ...................... ok

t/Builder/output.t ...................... ok

t/Builder/reset.t ....................... ok

t/Builder/reset_outputs.t ............... ok

t/Builder/try.t ......................... ok

t/c_flag.t .............................. ok

t/circular_data.t ....................... ok

t/cmp_ok.t .............................. ok

t/dependents.t .......................... skipped: Dependents only tested when releasing

t/diag.t ................................ ok

t/died.t ................................ ok

t/dont_overwrite_die_handler.t .......... ok

t/eq_set.t .............................. ok

t/exit.t ................................ ok

t/explain.t ............................. ok

t/extra.t ............................... ok

t/extra_one.t ........................... ok

t/fail-like.t ........................... ok

t/fail-more.t ........................... ok

t/fail.t ................................ ok

t/fail_one.t ............................ ok

t/filehandles.t ......................... ok

t/fork.t ................................ ok

t/harness_active.t ...................... ok

t/import.t .............................. ok

t/is_deeply_dne_bug.t ................... ok

t/is_deeply_fail.t ...................... ok

t/is_deeply_with_threads.t .............. skipped: many perls have broken threads. Enable with AUTHOR_TESTING.

t/missing.t ............................. ok

t/More.t ................................ ok

t/new_ok.t .............................. ok

t/no_plan.t ............................. ok

t/no_tests.t ............................ ok

t/note.t ................................ ok

t/overload.t ............................ ok

t/overload_threads.t .................... ok

t/plan.t ................................ ok

t/plan_bad.t ............................ ok

t/plan_is_noplan.t ...................... ok

t/plan_no_plan.t ........................ ok

t/plan_shouldnt_import.t ................ ok

t/plan_skip_all.t ....................... skipped: Just testing plan & skip_all

t/pod-coverage.t ........................ skipped: Test::Pod::Coverage 1.08 required for testing POD coverage

t/pod.t ................................. skipped: Test::Pod 1.00 required for testing POD

t/require_ok.t .......................... ok

t/simple.t .............................. ok

t/Simple/load.t ......................... ok

t/skip.t ................................ ok

t/skipall.t ............................. ok

t/subtest/args.t ........................ ok

t/subtest/basic.t ....................... ok

t/subtest/die.t ......................... ok

t/subtest/do.t .......................... ok

t/subtest/exceptions.t .................. ok

t/subtest/fork.t ........................ ok

t/subtest/implicit_done.t ............... ok

t/subtest/line_numbers.t ................ ok

t/subtest/plan.t ........................ ok

t/subtest/predicate.t ................... ok

t/subtest/singleton.t ................... ok

t/subtest/todo.t ........................ ok

t/subtest/wstat.t ....................... ok

t/tbm_doesnt_set_exported_to.t .......... ok

t/Tester/tbt_01basic.t .................. ok

t/Tester/tbt_02fhrestore.t .............. ok

t/Tester/tbt_03die.t .................... ok

t/Tester/tbt_04line_num.t ............... ok

t/Tester/tbt_05faildiag.t ............... ok

t/Tester/tbt_06errormess.t .............. ok

t/Tester/tbt_07args.t ................... ok

t/thread_taint.t ........................ ok

t/threads.t ............................. ok

t/todo.t ................................ ok

t/undef.t ............................... ok

t/use_ok.t .............................. ok

t/useing.t .............................. ok

t/utf8.t ................................ ok

t/versions.t ............................ ok

All tests successful.

Files=107, Tests=1168, 5 wallclock secs ( 0.24 usr 0.09 sys + 3.71 cusr 1.02 csys = 5.06 CPU)

Result: PASS

sudo make install

Installing /usr/share/perl5/Test/Simple.pm

Installing /usr/share/perl5/Test/Builder.pm

Installing /usr/share/perl5/Test/Tutorial.pod

Installing /usr/share/perl5/Test/More.pm

Installing /usr/share/perl5/Test/Builder/Tester.pm

Installing /usr/share/perl5/Test/Builder/Module.pm

Installing /usr/share/perl5/Test/Builder/Tester/Color.pm

Installing /usr/share/perl5/Test/Builder/IO/Scalar.pm

Installing /usr/share/man/man3/Test::Builder::Tester::Color.3pm

Installing /usr/share/man/man3/Test::Builder::Tester.3pm

Installing /usr/share/man/man3/Test::Builder::Module.3pm

Installing /usr/share/man/man3/Test::Tutorial.3pm

Installing /usr/share/man/man3/Test::Simple.3pm

Installing /usr/share/man/man3/Test::More.3pm

Installing /usr/share/man/man3/Test::Builder.3pm

Installing /usr/share/man/man3/Test::Builder::IO::Scalar.3pm

Appending installation info to /usr/lib64/perl5/perllocal.pod

最后进行测试,成功:

$perl test-prep.pl

# removing tmp/test-prep.pl.aB7uX

返回编译 ea-utils , 当然也没问题了:

$make

prove -j 4 t

===( 2;0 1/? 1/? 0/? )=========================================# removing tmp/join.t.JyHL6

t/join.t ... ok

===( 36;0 18/? 0/? )=============================================# removing tmp/mcf.t.ep9D0

t/mcf.t .... ok

t/multx.t .. 1/? # removing tmp/multx.t.xwXQY

t/multx.t .. ok

All tests successful.

Files=3, Tests=58, 2 wallclock secs ( 0.02 usr 0.00 sys + 0.97 cusr 1.91 csys = 2.90 CPU)

Result: PASS

总结:

1.碰到问题后,先看看生成的结果和之前有没有区别.这次就是没看,才耗费了这么长时间.

2.及时是官方库,也有版本不一致导致的某版本号之后几个版本没有集成之前的内容的情况.