[Beowulf] delayed savings time crashes

Florent Calvayrac florent.calvayrac at univ-lemans.fr
Wed Apr 12 12:12:16 PDT 2006

David Kewley wrote:

>The reboots were due to a City of Pasadena power glitch at 9:17 that 
>morning. :)  It was raining, and a 34kV city feeder line that runs between 
>the generating plant at the entrance of the 110 and a substation at Del Mar 
>& Los Robles faulted.  The responsible breaker took 13 cycles to break, 
>during which time the single-phase voltage seen at Caltech dropped to about 
>induce massive 12Hz oscillations on the room's power lines.
>As for the time glitch, that is probably induced by the fact that Daylight 
>Savings Time changes only take place on the "system" clock, and in a 
>standard Red Hat system those changes only get synced to the hardware clock 
>upon a clean shutdown.  So if your machine crashes after a DST change, then 
>upon bootup syslogd gets its time from the hardware clock, which is wrong.  
>The system clock is only corrected later in the bootup sequence, when ntpd 
>starts.  The best solution is probably to set the hardware clock to UCT 
>rather than local time.  UCT doesn't undergo step changes like most 
>timezones in the U.S. do, so the compensation for DST happens dynamically 
>in software, rather than requiring a hardware clock change.
Why don't you use a line conditioner ? It's much cheaper than a similar 
UPS ($1000 for 10kW),  many UPS dont guarantee voltage and waveform excepted
during clean power cuts, anyway.
A line conditioner is very handy in time of brownouts like during August 
We have a Salicru one on our cluster  and have a much better MTBF on our 
compute nodes in comparison
with the ones of a computing  room nearby.

Besides, I was confronted with the same problem about daylight saving time.
Just added a

clock -w

after the NTP synchronization in cron.daily so that the time would be 
corrected automatically
on the hardware clock.  (which is automatic on Windows btw).

More information about the Beowulf mailing list