[Beowulf] Have machine, will compute: ESXi or bare metal?

[Skylar Thompson]

On Tue, Feb 11, 2020 at 06:25:24AM +0800, Benson Muite wrote:
> 
> 
> On Tue, Feb 11, 2020, at 9:31 AM, Skylar Thompson wrote:
> > On Sun, Feb 09, 2020 at 10:46:05PM -0800, Chris Samuel wrote:
> > > On 9/2/20 10:36 pm, Benson Muite wrote:
> > > 
> > > > Take a look at the bootable cluster CD here:
> > > > http://www.littlefe.net/
> > > 
> > > From what I can see BCCD hasn't been updated for just over 5 years, and the
> > > last email on their developer list was Feb 2018, so it's likely a little out
> > > of date now.
> > > 
> > > http://bccd.net/downloads
> > > 
> > > http://bccd.net/pipermail/bccd-developers/
> > > 
> > > On the other hand their TRAC does list some ticket updates a few months ago,
> > > so perhaps there are things going on but Skylar needs more hands?
> > > 
> > > https://cluster.earlham.edu/trac/bccd-ng/report/1?sort=created&asc=0&page=1
> > 
> > Wow, I had no idea people on the Beowulf list were still thinking of BCCD.
> > :)
> > 
> > I've been working on a major BCCD update for a while now (modern Debian,
> > better node auto-detection) but a combination of life interference and a
> > shift in focus for the project to curriculum development has slowed me
> > down.
> > 
> > At the end of the day, BCCD has three has three main goals:
> > 
> > 1. Non-destructive in its default mode
> > 2. Simple ("just press enter")
> > 3. Ready with pedagogically-useful ("validated, verified, and accredited")
> > curriculum modules
> > 
> > One thing I'm hoping can come out of this major update is to decouple the BCCD
> > from the underlying distribution, since that's been a barrier for some
> > people in using BCCD. That's just an aspiration right now, but we'll see
> > where it goes.
> 
> Can you give more details on how you expect to decouple it?

There's a couple things that are tightly integrated with the init process
now:

* Network setup - prompts the user for information (which NIC to run on,
  whether to use DHCP to assign addresses, etc.), detects other BCCD
  systems on the network, etc.

* SSH public key broadcast (pkbcast) - needs to run after a user logs in to
  ensure that authorized_keys is setup on other participating systems.

The network setup in particular is a challenge in the systemd world, since
getting STDIN from systemd-invoked processes is not trivial. We've also had
some users wanting better integration between networking and desktop
applications, which pushed me to try to make use of the existing
systemd/networkd toolchain rather than rolling our own tooling.

Right now the challenge is the node auto-detection, though I'm hoping that
we might be able to use mDNS and zeroconf-assigned addressing rather than
depending on custom DHCP tags which have been problematic in the past.
zeroconf might also mitigate the biggest problem we have in workshops:
someone jumping the gun and starting up a head node before we're ready to
go.

While I'm not the biggest fan of systemd, it does have the potential to
allow us to get away from custom scripts and use functionality common
across more than one distribution.

-- 
Skylar