[Beowulf] Gridengine and bash + Modules

[madskaddie at gmail.com]

2010/1/18 Brian Smith <brs at usf.edu>:
> Ah, the RedHat-isms that we take for granted... hah!  I forgot that the
> default ~/.bashrc I push out to everyone sources /etc/bashrc by default.
> What distro are you using?
>

Debian lenny

> There's also this bit of goodness from the man page:
>
> "Bash attempts to determine when it is being run with its standard input
> connected  to a a network connection, as if by the remote shell daemon,
> usually rshd, or the secure shell daemon sshd.
>

The Debian bash man page doesn't say the word "sshd" (only "rshd"),
and I'm using ssh as the remote shell, so it may be the case (weird,
but possible).

(...)

>
> I wonder if sge_shepherd doesn't, in fact, trick shells into behaving
> this way... I know I'm not using BASH_ENV and my modules environment
> works correctly.
>

Just to be sure we aren't missing something: you can load a module
inside the submit job, correct?

Case 1:

 - module load something
 - qsub job.sh
 - cat job.sh
#!/bin/bash
#(sge config stuff)

mpirun ...

#EOF

Case 2 (what I pretend):
 - qsub job.sh
 - cat job.sh
#!/bin/bash
#(sge config stuff)

module add something
mpirun ...

#EOF




>
> -Brian
>
> --
> Brian Smith
> Senior Systems Administrator
> IT Research Computing, University of South Florida
> 4202 E. Fowler Ave. ENB308
> Office Phone: +1 813 974-1467
> Organization URL: http://rc.usf.edu
>
>
> On Mon, 2010-01-18 at 14:38 +0000, madskaddie at gmail.com wrote:
>> 2010/1/16 Brian Smith <brs at usf.edu>:
>> > I'm using this in our environment.  I've simply added the Modules
>> > environment code to /etc/bashrc and /etc/csh.cshrc on all nodes (I use
>> > puppet to manage everything, so this is easy).  This ensures that
>> > Modules is properly integrated with your environment regardless of
>> > whether you are using an interactive or non-interactive invocation of
>> > these shells.  This works for SGE (I'm on 6.2u4, ATM)
>> >
>>
>> But it seems that gridengine spawns like "bash script_name" so no rc
>> files are read. Reading bash manpage, I found the BASH_ENV environment
>> variable:
>>
>> """
>> When  bash  is  started non-interactively, to run a shell script, for
>> example, it looks for the variable BASH_ENV in the environment,
>> expands its value if it appears there, and uses the expanded value as
>> the name of a file to read and execute.  Bash behaves as if the
>> following command were executed:
>>               if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
>> but the value of the PATH variable is not used to search for the file name.
>> """
>> (bash manpage)
>>
>> Right now I'm setting this variable and with the "-V" job submission
>> flag it's working well (it does not work correctly without it)
>>
>> Gil
>>
>>
>
>



-- 
"
It can't continue forever. The nature of exponentials is that you push
them out and eventually disaster happens.
"
Gordon Moore  (Intel co-founder and author of the Moore's law)