[Beowulf] Gridengine and bash + Modules

[Brian Smith]

On Mon, 2010-01-18 at 19:53 +0000, madskaddie at gmail.com wrote:
> 2010/1/18 Brian Smith <brs at usf.edu>:
> > Ah, the RedHat-isms that we take for granted... hah!  I forgot that the
> > default ~/.bashrc I push out to everyone sources /etc/bashrc by default.
> > What distro are you using?
> >
> 
> Debian lenny
> 
> > There's also this bit of goodness from the man page:
> >
> > "Bash attempts to determine when it is being run with its standard input
> > connected  to a a network connection, as if by the remote shell daemon,
> > usually rshd, or the secure shell daemon sshd.
> >
> 
> The Debian bash man page doesn't say the word "sshd" (only "rshd"),
> and I'm using ssh as the remote shell, so it may be the case (weird,
> but possible).
> 
> (...)
> 
> >
> > I wonder if sge_shepherd doesn't, in fact, trick shells into behaving
> > this way... I know I'm not using BASH_ENV and my modules environment
> > works correctly.
> >
> 
> Just to be sure we aren't missing something: you can load a module
> inside the submit job, correct?
> 
> Case 1:
> 
>  - module load something
>  - qsub job.sh
>  - cat job.sh
> #!/bin/bash
> #(sge config stuff)
> 
> mpirun ...
> 
> #EOF

(Unless you are doing qsub -V)
This should be 
- module initadd something
- qsub job.sh
...

Remember, you need a "module load null" line in ~/.bashrc or, in my
case, ~/.modules.  This makes sure the module is loaded when bash
starts.

> Case 2 (what I pretend):
>  - qsub job.sh
>  - cat job.sh
> #!/bin/bash
> #(sge config stuff)
> 
> module add something
> mpirun ...
> 
> #EOF
> 
> 
> 
> 
> >
> > -Brian
> >
> > --
> > Brian Smith
> > Senior Systems Administrator
> > IT Research Computing, University of South Florida
> > 4202 E. Fowler Ave. ENB308
> > Office Phone: +1 813 974-1467
> > Organization URL: http://rc.usf.edu
> >
> >
> > On Mon, 2010-01-18 at 14:38 +0000, madskaddie at gmail.com wrote:
> >> 2010/1/16 Brian Smith <brs at usf.edu>:
> >> > I'm using this in our environment.  I've simply added the Modules
> >> > environment code to /etc/bashrc and /etc/csh.cshrc on all nodes (I use
> >> > puppet to manage everything, so this is easy).  This ensures that
> >> > Modules is properly integrated with your environment regardless of
> >> > whether you are using an interactive or non-interactive invocation of
> >> > these shells.  This works for SGE (I'm on 6.2u4, ATM)
> >> >
> >>
> >> But it seems that gridengine spawns like "bash script_name" so no rc
> >> files are read. Reading bash manpage, I found the BASH_ENV environment
> >> variable:
> >>
> >> """
> >> When  bash  is  started non-interactively, to run a shell script, for
> >> example, it looks for the variable BASH_ENV in the environment,
> >> expands its value if it appears there, and uses the expanded value as
> >> the name of a file to read and execute.  Bash behaves as if the
> >> following command were executed:
> >>               if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
> >> but the value of the PATH variable is not used to search for the file name.
> >> """
> >> (bash manpage)
> >>
> >> Right now I'm setting this variable and with the "-V" job submission
> >> flag it's working well (it does not work correctly without it)
> >>
> >> Gil
> >>
> >>
> >
> >
> 
> 
>