Making LSF job submitting nicer

Submitted by Matias on Fri, 05/16/2008 - 14:46

I use bsub often enough to hate specifying the -R,-o,-n and -q switches and stdout and stderr capturing every time. I'm sharing here a little goodie with you that takes away some of the trouble: nsub (nice sub)!

#!/bin/bash
 
if [ $# -lt 3 ]; then
  echo "USAGE: nsub jobname cmd"
fi
 
bsub -o"${1}.bsub.out" \
-J"${1}" \
-qlong \
-R"$RES" \
-n4 \
"${2} ${@:3} > ${1}.log 2> ${1}.err"

You also need to specify default resource usage in an environment variable $RES (put this in your .profile if you're using bash):

RES="select[type==X86_64 && \
ncpus>=4 && mem>= 700] \
span[hosts=1] \
rusage[mem=700]"
export RES

Here's an example showing how to use it:

for f in `ls *.bzip.fasta`; do 
nsub nminfer "$f" -numMotifs 1 \
-minLength 3 -maxLength 15 \
-backgroundModel human.100bg.500seq.msg.xml \
-seqs "$f" \
-out "${f}.xms"; done

This outputs the following files without me having to worry about output or error log naming or the bsub output file naming:

ls -l *.err *.log *.bsub.out
-rw-r--r--  1 mp4 uucp   4483 May 16 14:35 human.100bp.500seq.0.05spike.bzip.fasta.bsub.out
-rw-r--r--  1 mp4 uucp      0 May 16 14:19 human.100bp.500seq.0.05spike.bzip.fasta.err
-rw-r--r--  1 mp4 uucp 140085 May 16 14:35 human.100bp.500seq.0.05spike.bzip.fasta.log
-rw-r--r--  1 mp4 uucp   4466 May 16 14:35 human.100bp.500seq.0.1spike.bzip.fasta.bsub.out
-rw-r--r--  1 mp4 uucp      0 May 16 14:19 human.100bp.500seq.0.1spike.bzip.fasta.err
-rw-r--r--  1 mp4 uucp 143839 May 16 14:35 human.100bp.500seq.0.1spike.bzip.fasta.log
-rw-r--r--  1 mp4 uucp   4466 May 16 14:35 human.100bp.500seq.0.2spike.bzip.fasta.bsub.out
-rw-r--r--  1 mp4 uucp      0 May 16 14:19 human.100bp.500seq.0.2spike.bzip.fasta.err
-rw-r--r--  1 mp4 uucp 141419 May 16 14:35 human.100bp.500seq.0.2spike.bzip.fasta.log
-rw-r--r--  1 mp4 uucp   4466 May 16 14:35 human.100bp.500seq.0.3spike.bzip.fasta.bsub.out
-rw-r--r--  1 mp4 uucp      0 May 16 14:19 human.100bp.500seq.0.3spike.bzip.fasta.err
-rw-r--r--  1 mp4 uucp 138307 May 16 14:35 human.100bp.500seq.0.3spike.bzip.fasta.log
-rw-r--r--  1 mp4 uucp   4466 May 16 14:35 human.100bp.500seq.0.4spike.bzip.fasta.bsub.out
-rw-r--r--  1 mp4 uucp      0 May 16 14:19 human.100bp.500seq.0.4spike.bzip.fasta.err
-rw-r--r--  1 mp4 uucp 141111 May 16 14:35 human.100bp.500seq.0.4spike.bzip.fasta.log
-rw-r--r--  1 mp4 uucp   4466 May 16 14:35 human.100bp.500seq.0.5spike.bzip.fasta.bsub.out
-rw-r--r--  1 mp4 uucp      0 May 16 14:19 human.100bp.500seq.0.5spike.bzip.fasta.err
-rw-r--r--  1 mp4 uucp 138433 May 16 14:35 human.100bp.500seq.0.5spike.bzip.fasta.log