[Beowulf] any creative ways to crash Linux?: does a shared NIC IMPI always remain responsive?

[Bogdan Costescu]

On Sat, Oct 24, 2009 at 11:13 PM, Rahul Nabar <rpnabar at gmail.com> wrote:
> What surprised me was that even if I take down my eth interface with a
> ifdown the IPMI still works. How does it do that ?

The IPMI traffic is IP (UDP) based and by inspecting the IP header one
can make a difference between packets with the same MAC and different
IPs.

> That's why I am a bit surprised that my IPMI I/P continues to respond to the pings even after the primary I/P is dead.

Generally speaking, an ARP or IP reply comes from the networking stack
- if the port is ifdown-ed, the stack doesn't see any packets coming
in and has no reason to send a reply. When the primary (system) IP is
taken down, it's the Linux networking stack that doesn't see any
packet coming in, however the BMC's network stack will still be
active. That's the whole point of the BMC being a separate entity from
the main system, so that its functionality remains undisturbed when
something bad happens to the main system.

> Another mysterious observation was this: Whenever I took eth down via the OS there is a latent period when the IPMI stops
> responding but then somehow it magically resurrects itself and starts working again.

Without claiming that this is the best explanation: it's possible that
the Linux driver talks to the hardware and takes down the link at the
physical level. The BMC driver then detects this and brings the link
back up so that it can continue to receive the IPMI packets.

--
Bogdan