<div dir="ltr">Paul,<div><br></div><div>How big are your ldiskfs volumes? What type of underlying hardware are they? Running e2fsck (ldiskfs aware) is wise and can be done in parallel. It could be within a couple of days, the time all depends on the size and underlying hardware. </div><div><br></div><div>Going from 2.5.34 to 2.10.4 is a significant jump. I would be sure there isn't a step upgrade advised. I know there has been step upgrades in the past, not sure about going to/from these two versions. </div><div><br></div><div>--Jeff</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 23, 2018 at 10:34 AM, Paul Edmon <span dir="ltr"><<a href="mailto:pedmon@cfa.harvard.edu" target="_blank">pedmon@cfa.harvard.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    <p>Yeah we've found out firsthand that its problematic as we have
      been seeing issues :).  Hence the urge to upgrade.</p>
    <p>We've begun exploring this but we wanted to reach out to other
      people who may have gone through the same thing to get their
      thoughts.  We also need to figure out how significant an outage
      this will be.  As if it takes a day or two of full outage to do
      the upgrade that is more acceptable than a week.  We also wanted
      to know if people had experienced data loss/corruption in the
      process and any other kinks.</p>
    <p>We were planning on playing around on VM's to test the upgrade
      path before committing to upgrading our larger systems.  One of
      the questions we had though was if we needed to run e2fsck
      before/after the upgrade as that could add significant time to the
      outage for that to complete.</p><span class="HOEnZb"><font color="#888888">
    <p>-Paul Edmon-<br>
    </p></font></span><div><div class="h5">
    <br>
    <div class="m_8894130196693404536moz-cite-prefix">On 07/23/2018 01:18 PM, Jeff Johnson
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">You're running 2.10.4 clients against 2.5.34
        servers? I believe there are notable lnet attrs that don't exist
        in 2.5.34. Maybe a Whamcloud wiz might chime in but I think that
        version mismatch might be problematic. 
        <div><br>
        </div>
        <div>You can do a testbed upgrade to test taking a ldiskfs
          volume from 2.5.34 to 2.10.4, just to be conservative. </div>
        <div><br>
        </div>
        <div>--Jeff</div>
        <div><br>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Mon, Jul 23, 2018 at 10:05 AM, Paul
          Edmon <span dir="ltr"><<a href="mailto:pedmon@cfa.harvard.edu" target="_blank">pedmon@cfa.harvard.edu</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div bgcolor="#FFFFFF" text="#000000">
              <p>My apologies I meant 2.5.34 not 2.6.34.  We'd like to
                get up to 2.10.4 which is what our clients are running. 
                Recently we upgraded our cluster to CentOS7 which
                necessitated the client upgrade.  Our storage servers
                though stayed behind on 2.5.34.</p>
              <span class="m_8894130196693404536HOEnZb"><font color="#888888">
                  <p>-Paul Edmon-<br>
                  </p>
                </font></span>
              <div>
                <div class="m_8894130196693404536h5"> <br>
                  <div class="m_8894130196693404536m_1100149130740236357moz-cite-prefix">On
                    07/23/2018 01:00 PM, Jeff Johnson wrote:<br>
                  </div>
                  <blockquote type="cite">
                    <div dir="ltr">Paul,
                      <div><br>
                      </div>
                      <div>2.6.34 is a kernel version. What version of
                        Lustre are you at now? Some updates are easier
                        than others.</div>
                      <div><br>
                      </div>
                      <div>--Jeff</div>
                    </div>
                    <div class="gmail_extra"><br>
                      <div class="gmail_quote">On Mon, Jul 23, 2018 at
                        8:59 AM, Paul Edmon <span dir="ltr"><<a href="mailto:pedmon@cfa.harvard.edu" target="_blank">pedmon@cfa.harvard.edu</a>></span>
                        wrote:<br>
                        <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">We have some old large
                          scale Lustre installs that are running 2.6.34
                          and we want to get these up to the latest
                          version of Lustre.  I was curious if people in
                          this group have any experience with doing this
                          and if they could share them.  How do you
                          handle upgrades like this?  How much time does
                          it take?  What are the pitfalls?  How do you
                          manage it with minimal customer interruption?
                          Should we just write off upgrading and stand
                          up new servers that are on the correct version
                          (in which case we need to transfer the several
                          PB's worth of data over to the new system)?<br>
                          <br>
                          Thanks for your wisdom.<br>
                          <br>
                          -Paul Edmon-<br>
                          <br>
                          ______________________________<wbr>_________________<br>
                          Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a>
                          sponsored by Penguin Computing<br>
                          To change your subscription (digest mode or
                          unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" rel="noreferrer" target="_blank">http://www.beowulf.org/mailman<wbr>/listinfo/beowulf</a><br>
                        </blockquote>
                      </div>
                      <br>
                      <br clear="all">
                      <div><br>
                      </div>
                      -- <br>
                      <div class="m_8894130196693404536m_1100149130740236357gmail_signature" data-smartmail="gmail_signature">
                        <div dir="ltr">
                          <div>
                            <div dir="ltr">------------------------------<br>
                              Jeff Johnson<br>
                              Co-Founder<br>
                              Aeon Computing<br>
                              <br>
                              <a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br>
                              <a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>
                              t: 858-412-3810 x1001   f: 858-412-3845<br>
                              m: 619-204-9061<br>
                              <br>
                              4170 Morena Boulevard, Suite C - San
                              Diego, CA 92117
                              <div><br>
                              </div>
                              <div>High-Performance Computing / Lustre
                                Filesystems / Scale-out Storage</div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                    <br>
                    <fieldset class="m_8894130196693404536m_1100149130740236357mimeAttachmentHeader"></fieldset>
                    <br>
                    <pre>______________________________<wbr>_________________
Beowulf mailing list, <a class="m_8894130196693404536m_1100149130740236357moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit <a class="m_8894130196693404536m_1100149130740236357moz-txt-link-freetext" href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/mailman<wbr>/listinfo/beowulf</a>
</pre>
                  </blockquote>
                  <br>
                </div>
              </div>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored
            by Penguin Computing<br>
            To change your subscription (digest mode or unsubscribe)
            visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" rel="noreferrer" target="_blank">http://www.beowulf.org/mailman<wbr>/listinfo/beowulf</a><br>
            <br>
          </blockquote>
        </div>
        <br>
        <br clear="all">
        <div><br>
        </div>
        -- <br>
        <div class="m_8894130196693404536gmail_signature" data-smartmail="gmail_signature">
          <div dir="ltr">
            <div>
              <div dir="ltr">------------------------------<br>
                Jeff Johnson<br>
                Co-Founder<br>
                Aeon Computing<br>
                <br>
                <a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br>
                <a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>
                t: 858-412-3810 x1001   f: 858-412-3845<br>
                m: 619-204-9061<br>
                <br>
                4170 Morena Boulevard, Suite C - San Diego, CA 92117
                <div><br>
                </div>
                <div>High-Performance Computing / Lustre Filesystems /
                  Scale-out Storage</div>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <fieldset class="m_8894130196693404536mimeAttachmentHeader"></fieldset>
      <br>
      <pre>______________________________<wbr>_________________
Beowulf mailing list, <a class="m_8894130196693404536moz-txt-link-abbreviated" href="mailto:Beowulf@beowulf.org" target="_blank">Beowulf@beowulf.org</a> sponsored by Penguin Computing
To change your subscription (digest mode or unsubscribe) visit <a class="m_8894130196693404536moz-txt-link-freetext" href="http://www.beowulf.org/mailman/listinfo/beowulf" target="_blank">http://www.beowulf.org/<wbr>mailman/listinfo/beowulf</a>
</pre>
    </blockquote>
    <br>
  </div></div></div>

<br>______________________________<wbr>_________________<br>
Beowulf mailing list, <a href="mailto:Beowulf@beowulf.org">Beowulf@beowulf.org</a> sponsored by Penguin Computing<br>
To change your subscription (digest mode or unsubscribe) visit <a href="http://www.beowulf.org/mailman/listinfo/beowulf" rel="noreferrer" target="_blank">http://www.beowulf.org/<wbr>mailman/listinfo/beowulf</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">------------------------------<br>Jeff Johnson<br>Co-Founder<br>Aeon Computing<br><br><a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br><a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>t: 858-412-3810 x1001   f: 858-412-3845<br>m: 619-204-9061<br><br>4170 Morena Boulevard, Suite C - San Diego, CA 92117<div><br></div><div>High-Performance Computing / Lustre Filesystems / Scale-out Storage</div></div></div></div></div>
</div>