mpiblast 的安装与运行

Installation for Unix/Linux/BSD

If you are using Linux on Intel or AMD hardware, you may be able to find a pre-compiled RPM of mpiBLAST from Scalable Informatics. Otherwise, you will have to download and compile mpiBLAST from source code.

There are five steps to installing mpiBLAST from source.

1.Install MPI (if not already installed)

2.Download and extract mpiBLAST and the matching NCBI Toolbox release

3.Patch and compile the NCBI Toolbox

https://www.360docs.net/doc/f011492273.html,pile and install mpiBLAST

5.Configure mpiBLAST by editing the ~/.ncbirc file

Each of these steps are described in more detail below.

1) MPI installation

Two popular free versions of MPI are MPICH and LAM/MPI. A successful installation of either is sufficient for mpiBLAST to work. Many OS vendors bundle either MPICH or LAM-MPI with the system, making an installation from source unnecessary.

2) Download and extract mpiBLAST and the matching NCBI Toolbox release

2a) Download mpiBLAST from Downloads.Archive.html.html.

wget https://www.360docs.net/doc/f011492273.html,/downloads/files/mpiBLAST-x.y.z.tgz

or

wget https://www.360docs.net/doc/f011492273.html,/downloads/files/mpiBLAST-x.y.z.tbz

2b)Extract mpiBLAST

tar -xvzf mpiblast-x.y.z.tgz

or

tar -xvjf mpiblast-x.y.z.tbz

2c) Download the NCBI Toolbox from

ftp://https://www.360docs.net/doc/f011492273.html,/toolbox/ncbi_tools/. (Note: mpiBLAST-1.4.0 requires the October 2004 release.)

wget ftp://https://www.360docs.net/doc/f011492273.html,/toolbox/ncbi_tools/old/20041020/ncbi.tar.gz

2d) Extract the NCBI Toolbox

tar -xvzf ncbi.tar.gz

3) Patch and compile the NCBI Toolbox

In order to provide exact e-value statistics, mpiBLAST requires slight modifications to the NCBI Toolbox. We have provided a patch that modifies the NCBI Toolbox accordingly. Once patched, the toolbox must be re-compiled prior to compiling mpiBLAST. The e-value patch is required for mpiBLAST to work properly.

3a) Change to the directory containing the NCBI Toolbox

cd /path/to/ncbi

(most likely the command will simply be cd ncbi)

3b)Apply the patch distributed with mpiBLAST

patch -p1 < /path/to/mpiBLAST-x.y.z/ncbi_Oct2004_evalue.patch

Output should look like:

patching file tools/blast.c

patching file tools/blastdef.h

patching file tools/blastkar.c

patching file tools/blastutl.c

3c)Build the NCBI Toolbox

cd /path/to/ncbi

ncbi/make/makedis.csh

The NCBI toolbox should build successfully, but in the case that it does not build correctly, please return to step 2c, and try again but skip steps 3a and 3b (i.e. you will perform steps 2c, 2d, and then 3c). The goal is to determine whether the NCBI Toolbox can build successfully without the mpiBLAST e-value patch. If the Toolbox build fails, refer to ncbi/README for detailed instructions on building the Toolbox. If the Toolbox build succeeds without the patch, please send an email to the mpiblast mailing lists informing us of the problem.

如何最后报库不匹配的错误，可能是需要的库在lib64目录下，而去到lib目录下去寻找，修改一下相应的变量就可以了。

4) Compile and install mpiBLAST

4a) Change to the directory containing mpiBLAST

cd /path/to/mpiBLAST-x.y.z

(most likely the command will simply be cd mpiBLAST-x.y.z)

4b) Configure mpiBLAST

./configure --with-ncbi=/full/path/to/ncbi

(it is necessary to provide the full path to the ncbi directory)

Useful extra options to 'configure' in addition to the standard configure flags:

?--prefix=/path/to/installation/directory

Specifies the location where mpiBLAST should be installed. (Default is /usr/local/)

?--enable-mpe-timing

Causes mpiblast to use MPE for profiling.

?--with-mpe=/path/to/mpe

Specifies location of MPE headers and libraries to use.

?--with-mpi=/path/to/mpi

Specifies location of MPI headers and libraries.

?--enable-MPI_Alloc_mem

Causes mpiblast to use MPI_Alloc_mem() for allocating memory. Improves performance for some network interconnects and MPI implementations. This is not compatible with mpich-1.x.

4c) Build mpiBLAST

make

4d) Install mpiBLAST

make install

(If the --prefix configure option wasn't used then mpiBLAST will be placed in /usr/local and root priveleges will probably be required to install it)

5) Configure mpiBLAST by editing the ~/.ncbirc file

Before running mpiBLAST, it is necessary to configure the shared and local storage paths that each node will use to access the database. A shared storage path is usually a path to a directory residing on a file server, such as NFS, AFS, or samba. The local storage path is typically a subdirectory within the /tmp directory, e.g. /tmp/mpiblast. As worker nodes search the database, they will copy fragments to the local storage directory. During subsequent searches of the same database, the fragments will already reside in local storage and thus will not need to be copied. Note that diskless nodes can be supported by setting the local storage path to be the same as the shared storage path. To configure mpiBLAST create a .ncbirc file in your home directory that looks like:

[NCBI]

Data=/path/to/shared/storage/data

[BLAST]

BLASTDB=/path/to/shared/storage

BLASTMAT=/path/to/shared/storage/data

[mpiBLAST]

Shared=/path/to/shared/storage

Local=/path/to/local/storage

Shared变量设置为nfs共享的目录，在mpiformatdb命令执行后，将自动把处理后的数据放置到这个目录下；而Local目录则是每个节点上的处理器从Shared目录拷贝自己要处理的那一部分数据到本地；可以设置为/tmp/mpiblast

The Data variable gives the location of the NCBI Data directory containing BLOSUM and PAM scoring matrices, among other things. The scoring matrix files are necessary for any type of protein BLAST search and should be accessible by all cluster nodes. The BLASTMAT variable also specifies the path to the scoring matrices, and will usually be identical to the Data variable. The BLASTDB variable tells standard NCBI blastall (not mpiBLAST) where to find BLAST databases. As previously mentioned, the Shared and Local variables give the shared and local database paths, respectively. By setting BLASTDB to the same path as Shared, it is possible for NCBI blastall to share the same databases that mpiBLAST uses. In such a configuration, be sure to format all databases with mpiformatdb rather than formatdb.

Installation for Windows

1.Copy mpiblast.exe and mpiformatdb.exe to a shared storage directory

2.Create the mpiblast configuration file (see below)

3.Get the NCBI data distribution and copy its contents to your shared

(database) storage directory

4.Make sure mpich2 is installed

5.If your Windows machines are running Windows XP SP2 or have third party

firewall software installed, be sure to configure it to allow the mpich2 daemon smpd access to the internet. By default it listens on tcp ports 8675 and 8676 and possibly others.

6.mpiBLAST is ready to run

Note: Building mpiBLAST from source under Windows requires MS Visual Studio 7. Also, the function BlastScoreBlkMatFill() in the file blastkar.c of the NCBI toolkit must be modified to search the environment for paths to scoring

matrices.

Creating the config file

On Windows, mpiBLAST (and NCBI's blastall) requires a ncbi.ini configuration file created in the Windows directory, usually C:\Windows or C:\WINNT. The format of ncbi.ini is similar to .ncbirc under Unix. For example:

[NCBI]

[BLAST]

[mpiBLAST]

Shared=/path/to/shared/storage

Local=/path/to/local/storage

The mpiBLAST web interface

Scripts have been included that allow mpiBLAST to interface with the NCBI web front-end available at

ftp://https://www.360docs.net/doc/f011492273.html,/blast/executables/LATEST-WWWBLAST

The scripts are designed to queue jobs submitted through the web front-end into a PBS system. The interface between the NCBI HTML pages and BLAST is a file called blast.cgi. mpiBLAST includes a replacement blast.cgi that calls mpiBLAST's WWWBlastWrap.pl job submission script. blast.cgi and WWWBlastWrap.pl should be copied into wwwblast's directory. You will have to manually edit WWWBlastWrap.pl to adapt it to your system.

In order to perform a search with mpiBLAST, the target BLAST database must first be formatted and segmented using mpiformatdb. Then, mpirun can be used to execute mpiblast in parallel on several cluster nodes.

Formatting a database

Before processing blast queries the sequence database must be formatted with mpiformatdb. The command line syntax looks like this:

mpiformatdb -N 25 -i nt //在数据库放置的目录下执行，或者指定数据库的路径

mpiformatdb -N 8 -i Human.fasta //对于Human.fasta数据库

The above command would format the nt database into 25 fragments, ideally for 25 worker nodes. mpiformatdb accepts the same command line options as NCBI's formatdb. See the README.formatdb file that comes with the NCBI BLAST distribution for more details.

mpiformatdb reads the ~/.ncbirc file and creates the formatted database fragments in the shared storage directory.

会自动把处理后的数据放置到Shared设置的目录下；

Querying the database

mpiblast command line syntax is nearly identical to NCBI's blastall program. See the README.bls file included in the BLAST distribution for details. Running a query on 25 nodes would look like:

mpirun -np 27 mpiblast --config-file=/path/to/mpiblast.conf -p blastn -d nt -i blast_query.fas -o blast_results.txt

mpirun -np 10 /root/mpiblast/bin/mpiblast -p blastp -i MM.fa -d Human.fasta -o blast_results.txt //如果不在当前目录，需要指定MM.fa

的路径

mpirun -np 8 -machinefile ./ma

/annotation/software/mpiblast/bin/mpiblast -p blastp -i month.aa -d F.joh_aa.fasta -o blast_result.txt

The above command would query the sequences in blast_query.fas against the nt database and write out results to the blast_results.txt file in the current working directory. The --config-file argument is optional a nd specifies the location of mpiblast.conf. To get the best performance, it is important to start at least one more process than the number of processors in the cluster because one of the mpiBLAST processes is dedicated to scheduling, which is not CPU-intensive. Furthermore, mpiBLAST needs at least 3 processes to perform a search. One process performs file output and another schedules search tasks, while any additional processes actually perform search tasks.

Extra options to mpiblast

?--debug[=filename]

Produces verbose debugging output for each node, optionally logs the output to a file

?--scheduler-rank=

Set the scheduler process' MPI Rank (default is 1). Because the scheduler uses very little CPU it can be useful to force the

scheduler to run on the same physical machine as the writer (rank

0).

?--altschul-reference

Print the Altschul. et. al. 1997 paper reference instead of the mpiBLAST paper reference. With this option mpiblast output is

nearly identical to NCBI-BLAST output.

?--removedb

Removes the local copy of the database from each node before

terminating execution

?--copy-via=[cp|rcp|scp|mpi|none]

Sets the method of copying files that each worker will use. Default = "cp"

o cp : use standard file system "cp" command. Additional option is --concurrent.

o rcp : use rsh "rcp" command. Additonal option is

--concurrent.

o scp : use ssh "scp" command. Additional option is

--concurrent.

o mpi : use MPI_Send/MPI_Recv to copy files. Additional option is --mpi-size.

o none : do not copy files, instead use shared storage as local storage

?--concurrent=

set the number of concurrent accesses to shared storage. Default = 1

?--mpi-size=

in bytes, set the maximum buffer size that MPI will use to send data when transferring files. Default = 65536

?--lock[=on|off]

set whether file locking should be used to manage local fragment lists. Defaults to off. When --concurrency > 1 defaults to on ?--disable-mpi-db

When set, the writer will use the database on shared storage for sequence lookup. Can drastically reduce overhead for some blastn searches.

?--nice=

Under unix, sets the nice value for each mpiblast process.

?--resume-run

Experimental. When set, mpiblast will read the output file and

attempt to continue a previously aborted run where it left off ?--version

print the mpiBLAST version

Removing a database

The --removedb command line option will cause mpiBLAST to do all work in a temporary directory that will get removed from each node's local storage directory upon successful termination. For example:

mpirun -np 16 mpiblast -p blastx -d yeast.aa -i ech_10k.fas -o results.txt --removedb

The above command would perform a 16 node (14 worker) search of the yeast.aa database, writing the output to results.txt. Upon completion, worker nodes would delete the yeast.aa database fragments from their local storage.

Databases can also be removed without performing a search in the following manner:

mpirun -np 16 mpiblast_cleanup