Fwd: [Mauiusers] March 2003 Newsletter

[Ron Chen]

FYI,

 -Ron

--- help at supercluster.org wrote:
> Dear user,
> In today's newsletter we are announcing the
> availability of MAUI SCHEDULER 
> 3.2.6, continued rapid growth of MAUI usage, added
> resource manager 
> support and more.  
> 
> 
> News/Announcements:
> 
> 1.  Usage of MAUI SCHEDULER is growing significantly
> (Downloads exceed 
> 250% of previous year)
> 2.  MAUI SCHEDULER 3.2.6 (Alpha) is now available
> 3.  Added resource manager support (SGE 5.3, LSF
> 5.1, Loadleveler 3.1, 
> PBSPro 5.2) 
> 4.  Support for the Department of Energy's SSS
> standard (Scalable System 
> Software)
> 5.  New XML support on Client and Peer Services
> 6.  CLUSTER RESOURCES, INC. - Free consultation
> offer
> 
> Tips and Tricks:
> ·	How to use MAUI's Advanced Preemption Support
> 
> 
> Please support our MAUI awareness initiative and add
> a link to - 
> supercluster.org or clusterresources.com or add a
> web banner that shows 
> you're a proud user
> (www.supercluster.org/mauilink.html) 
> 
> 
>
*****************************************************************************
> Usage of MAUI SCHEDULER is growing significantly
> 
> There has been continued strong growth of MAUI
> SCHEDULER adoption, with 
> 2002 downloads reaching 250% of the previous year,
> and the growth 
> continues.  Web site hits last year were over 80,000
> hits per month, and 
> have already grown an additional 50% in just the
> first quarter of this 
> year.  Contributing to the growth has been
> significant feature 
> advancements, improved usability, and support for a
> broader range of 
> resource managers.  Also, your efforts to grow the
> community of 
> MAUI-knowledgeable experts have been significant and
> much appreciated.  
> Please feel free to suggest feature enhancements at
> any time by sending 
> and email to support at clusterresources.com, and thank
> you again for helping 
> to get the word out with your peers.
> 
> 
>
*****************************************************************************
> MAUI SCHEDULER 3.2.6 (Alpha) is now available
> 
> MAUI SCHEDULER 3.2.6 (Alpha) is now available for
> use.  It has no known 
> stability or compatibility issues and is already
> being used on over 100 
> systems.  To gain access to 3.2.6 please contact us
> at 
> help at supercluster.org.  We will email you a username
> and password and 
> then you can proceed to download it at
>  
> http://supercluster.org/downloads/maui/alpha.  
> 
>   This release adds support for the SSS 1.0
> Protocol, provides additional 
> resource manager interfaces, and utilizes XML for
> client and peer 
> interfaces (see sections below for additional
> details). Further, 3.2.6 
> adds 64 bit support as well as a feature known as
> 'Standalone Mode'.  
> Standalone Mode allows you to perform simple load
> balancing with MAUI 
> SCHEDULER without requiring a resource manager. 
> Using the COMPUTEHOST 
> parameter, sites can specify the list of nodes they
> want Maui to consider 
> and track.  The new mbal command allows a user to
> load balance either 
> logins or remote command execution.  If a command
> argument is specified, 
> MAUI will launch this command on the most lightly
> loaded node.  If no 
> command is specified, a login shell to that node
> will be launched.  
> 
> While partially available in 3.2.5, more advanced
> preemption features 
> have been added to MAUI SCHEDULER since our last
> newsletter (See Tips and 
> Tricks: How to use MAUI's Advanced Preemption
> Support).
> 
> 
>
*****************************************************************************
> Added resource manager support (SGE 5.3, LSF 5.1,
> Loadleveler 3.1, PBSPro 5.2)
> 
> In an effort to maintain compatibility with the
> latest of both open and 
> proprietary resource managers, we have added support
> for SGE 5.3, LSF 
> 5.3, Loadleveler 3.1 and PBSPro 5.2.  MAUI SCHEDULER
> now supports these 
> latest four additions plus OpenPBS 2.3, BProc, WIKI
> and SSS 1.0.  For 
> integration guides for SGE, PBSPro and others visit 
> http://supercluster.org/documentation. 
> 
> 
>
*****************************************************************************
> Support for the Department of Energy's SSS Protocol
> (Scalable System 
> Software)
> 
> MAUI SCHEDULER 3.2.6 adds support for SSS 1.0.  The
> SSS protocol is a 
> resource management protocol that provides
> standardized interfaces for 
> resource managers, queue managers, allocation
> managers, cluster 
> schedulers, grid schedulers, event management
> systems, and information 
> services.  Reference implementations of these tools
> have been and are 
> being developed which target the massive compute
> systems of the future.  
> Design criteria are being used to ensure efficient,
> secure, and flexible 
> control of clusters possessing in excess of 50,000
> processors.  SSS 
> Protocols and tools are being developed by the U.S.
> Department of Energy 
> via a collaboration of 8 research laboratories.
>  
> MAUI SCHEDULER 3.2.6 and higher can be configured to
> utilize resource 
> management tools conforming to the SSS protocol.  As
> part of this 
> initiative, there have also been significant
> enhancements to the 
> scheduler client and peer interfaces, the scheduling
> feature set, 
> security options, and scalability.  Maui/SSS
> installations are currently 
> being tested at a number of your favorite leading
> edge sites.  Further 
> details can be found on the Scalable Software
> Systems Homepage - 
> (http://www.scidac.org/ScalableSystems).
> 
> 
>
*****************************************************************************
> New XML support on Client and Peer Services 
> 
> MAUI SCHEDULER 3.2.6 and higher have added XML
> support for Client and 
> Peer Services.  For a subset of commands, the
> '--format=XML' command line 
> flag can be used to provide an XML response string. 
> This string may 
> present data in a format that is more amenable to
> use by peer services, 
> databases, and GUIs.  Clients at various stages of
> XML support now  
> include mshow (showq), showstate, showres, checknode
> , mjobctl (runjob, 
> setjobhold, setspri, canceljob), and mresctl
> (setres, releaseres).  
> 
> 
>
*****************************************************************************
> CLUSTER RESOURCES INC. - Free consultation offer
> 
> Call CLUSTER RESOURCES, INC. today (801) 873-3400
> and get a free 
> consultation on what MAUI SCHEDULER and SILVER can
> do for you and how 
> best to deploy it within your environment.  The
> Catch: It's Free, but we 
> will better understand your needs and you will
> better understand what we 
> can do for you and how our services can make you
> more successful.  Maybe 
> you'll realize how much money we can save you.
> 
> The developers of Maui and founders of
> Supercluster.org formed CLUSTER 
> RESOURCES INC., in an effort to provide customers
> access to professional 
> services supporting their use of MAUI SCHEDULER, &
> SILVER.  See 
> www.clusterresources.com for details.
> 
> 
>
*****************************************************************************
> Tips & Tricks:
> 
> How to use MAUI's Advanced Preemption Support
> 
> MAUI SCHEDULER has had limited preemption support
> since 3.0.7, but Maui 
> 3.2.6 has added important new preemption features. 
> Note:  Preemption 
> features are currently available for PBS,
> Loadleveler and LSF 
> environments.
>   
> Basic preemption of 3.0.7 and higher:
> It is almost always the case that sites possess
> workloads of varying 
> importance.  While it may be critical that some jobs
> obtain resources 
> immediately, other jobs are less 'response time'
> sensitive but have an 
> insatiable hunger for compute cycles, consuming
> every available cycle for 
> weeks, months or even years on end.  The preemption
> feature enables 
> systems to run low priority, preemptible jobs
> whenever something more 
> pressing is not running.  MAUI SCHEDULER makes this
> feature available 
> through its QoS-based preemption system.  Certain
> QoS's may be marked 
> with the flag PREEMPTOR, others with the flag
> PREEMPTEE.  With this 
> configuration, low priority, preemptee jobs can be
> started whenever idle 
> resources are available.  These jobs will be allowed
> to run until a 
> higher priority preemptor job arrives, at which
> point the preemptee job 
> will be checkpointed if possible and vacated.  This
> allows near immediate 
> resource access for the higher priority preemptor
> job. 
> 
> Using this approach, a cluster can maintain near
> 100% system utilization 
> while still delivering excellent turnaround time to
> the jobs of greatest 
> value.  Use of the preemption system need not be
> limited to controlling 
> low priority jobs.  Other uses include optimistic
> scheduling and 
> development job support. 
> 
> Example: 
> QOSCFG[high] PRIORITY=10000 QFLAGS=PREEMPTOR 
> QOSCFG[med] 
> QOSCFG[low]  QFLAGS=PREEMPTEE 
> 
> Advanced preemption of 3.2.6 and higher:
> MAUI SCHEDULER 3.2.6 and higher has preemption-based
> backfill and policy 
> violation based preemption.  The first feature,
> preemption-based 
> backfill, allows an organization to combine the
> familiar backfill 
> optimization policy with preemption.  The value in
> this is that it now 
> allows 'optimistic' scheduling. Normally backfill
> allows high priority 
> jobs to receive their 'earliest job start'
> guarantee, and allows lower 
> priority jobs to be backfilled into available
> resources as long as they 
> do not interfere with the 'earliest job start' of
> the high priority jobs.  
> Adding in the preemption element allows even more of
> the low priority 
> jobs to be run, since now they can be
> 'optimistically' started even if 
> they cannot be finished in a particular open slot of
> resources.  If the 
> low priority backfill job runs too long, it would
> simply be preempted, 
> and if possible, checkpointed at the partially
> completed point.  When 
> there was another available resource opening, the
> low priority job could 
> continue.  Preemption based backfill is not a
> panacea and may not improve 
> performance in all cases.  Each site should consider
> their workload and 
> environment closely before enabling this policy, so
> feel free to contact 
> us for a site-specific recommendation.
> 
> The second feature, 'policy violation based
> preemption', is actually 
> quite a useful feature.  This feature allows an
> organization to specify 
> per job policy based limitations, and then, if said
> limits are violated, 
> allow another job that is compliant to preempt the
> violating job.  This 
> allows an organization that has competing internal
> interests to manage 
> within a set of established policies in such a
> manner that violations do 
> not simply cancel a job or cause it to fail, but
> rather automatically 
> allow it to be preempted by compliant job requests,
> and then allow it to 
> continue once resources are available again.  This
> maintains fairness, 
> maximizes utilization, and adds the flexibility for
> violating jobs to run 
> on available resources when they don't compete with
> jobs that are in 
> compliance.  
> 
> Example:
> 
> # suspend jobs which violate memory usage limits and
> share resources
> # with other jobs
> RESOURCELIMITPOLICY MEM:BLOCKEDWORKLOADONLY:SUSPEND
> 
> # cancel jobs which exceed CPU targets for more than
> 10 minutes
> RESOURCELIMITPOLICY PROC:ALWAYS:CANCEL:00:10:00
> 
> Documentation on this policy and other 3.2.6
> features is available to all 
> registered visitors; to gain access to 3.2.6 as well
> as the supporting 
> documentation please contact us at
> help at supercluster.org 
> 
> 
>
*****************************************************************************
> Highlight/Did you Know:
> 
> Did you know supercluster.org/clusterresources.com
> reached the 1,000,000 
> hits/year mark in 2002, and already this year we are
> at a 50% higher run 
> rate, just part way into the first quarter.
> 
> ***** View online at 
> http://clusterresources.com/newsletter/march2003.htm
> *****
>
______________________________________________________
> Support Maui - link to supercluster.org or
> clusterresources.com using a 
> URL or the web banners found at
> www.supercluster.org/mauilink.html 
> 
> Contact Us:
> Tech. Support, Consulting, Custom Development
> sales at clusterresources.com
> Product Feedback help at supercluster.org
> 
> -- 
>
--------------------------------------------------------
> Supercluster Development Group
> Scheduling and Resource Management of Clusters and
> Grids
> Maui Home Page   - http://supercluster.org/maui
> Silver Home Page - http://supercluster.org/silver
> Documentation    -
> http://supercluster.org/documentation
> 
> 
> _______________________________________________
> mauiusers mailing list
> mauiusers at supercluster.org
> http://supercluster.org/mailman/listinfo/mauiusers


__________________________________________________
Do you Yahoo!?
Yahoo! Web Hosting - establish your business online
http://webhosting.yahoo.com