[Beowulf] any creative ways to crash Linux?: does a shared NIC IMPI always remain responsive?

[Rahul Nabar]

On Mon, Oct 26, 2009 at 8:11 AM, Bogdan Costescu <bcostescu at gmail.com> wrote:
> On Sat, Oct 24, 2009 at 11:13 PM, Rahul Nabar <rpnabar at gmail.com> wrote:
>> What surprised me was that even if I take down my eth interface with a
>> ifdown the IPMI still works. How does it do that ?
>
> The IPMI traffic is IP (UDP) based and by inspecting the IP header one
> can make a difference between packets with the same MAC and different
> IPs.

Actually, the MAC is different too. I have one NIC but it responds to
two MACs. I guess one is transparent to the OS and the other is
handled by the BMC.



> taken down, it's the Linux networking stack that doesn't see any
> packet coming in, however the BMC's network stack will still be
> active. That's the whole point of the BMC being a separate entity from
> the main system, so that its functionality remains undisturbed when
> something bad happens to the main system.

I see. So I assume the BMC's network stack is something that's
hardware or firmware implemented. It's funny that in spite of this the
IPMI gets hung sometimes (like Gerry says in his reply). I guess I can
just attribute that to bad firmware coding in the BMC.


>> Another mysterious observation was this: Whenever I took eth down via the OS there is a latent period when the IPMI stops
>> responding but then somehow it magically resurrects itself and starts working again.
>
> Without claiming that this is the best explanation: it's possible that
> the Linux driver talks to the hardware and takes down the link at the
> physical level. The BMC driver then detects this and brings the link
> back up so that it can continue to receive the IPMI packets.

You are probably right. THe explanation sounds reasonable to me. A
similar observation is for accessing the BIOS as well. The BMC stack
is not responsive right from the power-up. It does become responsive
for a bit but then the system drags it down (maybe when the BIOS hands
over to PXE). If I manage to "ipmitool sol activate" within this
correct window then I am able to see the BIOS. But that's pretty much
trial and error.

-- 
Rahul