Get documents about "Priority Scheduler Overview
Document Sample
Priority Scheduler Overview
Teradata Development Division
August, 2002
The Tip of the Iceberg
• What Priority Scheduler is Not
• What Priority Scheduler is
• Architecture
• Implementation
02/21/2002 2
NCR Confidential
What Priority Scheduler is Not
• Hardware Segmentation Tool
– Priority Scheduler performs logical
segmentation, not physical segmentation
• Group Management Tool
– Priority Scheduler provides relative priority
• CPU Optimizer
– Priority Scheduler is not a band aid for poorly
written queries, it does not give more CPU
seconds per day
02/21/2002 3
NCR Confidential
What is Priority Scheduler
• Priority Scheduler is:
– A Workload Manager
– Based on relative workloads determined at the time of
execution
• What Priority Scheduler can do for you:
– Instituting better service for your more important
work
– Controlling resource sharing among different
applications
– Preventing aggressive queries from over-consuming at
the expense of other work
– Placing a ceiling on CPU usage for some applications
02/21/2002 4
NCR Confidential
Priority Scheduler is Prioritized
Queues not Percentages of CPU
Coach Business Class First Class
02/21/2002 5
NCR Confidential
Priority Scheduler Components
Resources RP0
RP1 RP4
Partition “Default”
Performance
L M H R L1 M1 H1 R1 L4 M4 H4 R4
Groups
L+ M+ H+ R+
Management
Performance 8am-8am 8am-11pm 11pm-6am 6am-8am schmon Tool
Periods AG1 AG20 AG21 AG22
Allocation AG1 AG20 AG21 AG22 AG30 AG31 AG32
Group 5 20 40 5 5 10 20
02/21/2002 6
NCR Confidential
Summary of Components
(For your reference)
• Resource Partition (RP): • Performance Period (PP)
– A high-level resource and – 1 to 5 per PG
priority grouping
– Links a PG to an
– May define up to 5 Allocation Group’s weight
– Partition 0 is provided and policy
– Makes possible changes in
• Performance Group (PG): priority weight/policy by
– Must define 8 (with values 0 time or resource usage
- 7) within each Partition
• Allocation Group (AG)
– Only PGs with values of
0,2,4,6 are for users (1, 3, – Carries the weight
5, 7 are used internally) – Determines the Policy and
– PG name matches to acctid Set Division Type
string on logon statement, – PGs may share the same
must be unique system-wide Allocation Groups
02/21/2002 7
NCR Confidential
Relative Weighting and Allocating
Resources
• Weights are distributed among
Resource Partitions
• RP0 100 RP2
L2 L M
• RP1 75
• RP2 25
• Weights are then allocated H
R1
among Allocation Groups
RP0
• Allocation Group Weights RP1
• RP0/L 5
• RP0/M 10 H1 R
• RP0/H 20
• RP0/R 40 M1
L1
• RP1/L 2
• RP1/M 10
• RP1/H 50 Example:
RP1/M1 = RP1 Relative % * AG Relative %
• RP1/R 100 = (75/(100+75+25)) * (10/(2+10+50+100))
• RP2/L 10 = 37% * 6%
= 2%
02/21/2002 8
NCR Confidential
What is Relative Weighting
% CPU ASSIGNED % CPU TARGETED % CPU CONSUMED
L
RP2 M L RP2
RP2 H
R L
H
H
RP1 R
R
• % CPU Consumed: The CPU actually allocated to
the active partitions and active groups
• Reflects targeted but unused CPU spilling over to
other partitions and groups
02/21/2002 9
NCR Confidential
Performance Groups
Resources
RP1
Partition
Users are assigned to a
Performance PG
L1 M1 H1 R1
Groups
Performance 8am-11pm 11pm-6am 6am-8am At Logon Users are
AG20 AG21 AG22 associated to an AG via
Periods
a PP
Allocation AG20 AG21 AG22
Group 20 40 5 Users execute at the
weight of the assigned
AG
02/21/2002 10
NCR Confidential
You Assign Weights to Groups of
Users
Allocation
User
Group Wgt
Sessions
%
Perf. Grp 1 Allocation
Perf.
‘$LDEV$’ Period
Group 1 9%
Weight = 5
Perf. Grp 2 Allocation
Perf.
‘$MDEV$’ Period
Group 2 18 %
Weight = 10
Perf. Grp 4 Perf Allocation
‘$RDEV$’ Period
Group 4 73 %
Weight = 40
• Sum the weights
• Each user session is assigned to an Allocation 5+10+40 = 55
Group at log on time • Calculate Relative Weights:
• Each Allocation Group has a targeted share of the L: 5/55 = 9%
system resources, called a Wgt % M: 10/55 = 18%
R: 40/75 = 73%
02/21/2002 11
NCR Confidential
Wgt % Combines Partition and
Allocation Group Weight
RP0 RP1
RP Assigned Weight = 100
Weights Weight = 50
AG Assigned
Weight 5 10 40 10
L M R X
Relative
67% 33%
Partition Wgt
Relative AG Wgt
9% 18% 73% 100%
within Partition
Wgt % 6% 12% 49% 33%
.67 *.9 =.6 .67 *.18 =.12 .67 *.73 =.49 .33 * 1.00=.33
02/21/2002 12
NCR Confidential
Performance Periods by CPU
Usage
Automatic Change Based on CPU Usage Since Session Logon
User logs on User reaches
100 Seconds
CPU accumulation
Time in Seconds 0 50 100 125
Performance Period 0 Performance Period 1
Usage: 100 Seconds Usage: 0 Seconds
Allocation Group 9 Allocation Group 33
Alloc Grp 9 Alloc Grp 33
Weight=20 Weight=5
Default Default
02/21/2002 13
NCR Confidential
Performance Periods by Time
• T is type of Time Performance Period 1
End-time: 1700 hours
Alloc Grp 9
Weight=60
• VALUE is the END Allocation Group 9 IMMEDIATE
TIME Performance Period 2 Alloc Grp 5
End-time: 2300 hours Weight=20
• Define Periods with Allocation Group 5 Default
Performance Group
Performance Period 3
• Up to four per End-time: 0700 hours
Alloc Grp 6
Weight=10
Performance Group Allocation Group 6 Absolute
02/21/2002 14
NCR Confidential
Allocation Group Policies
Unrestricted Policies Restricted Policies
DEF (DEFAULT) ABS (ABSOLUTE)
• Keeps track of past usage of each • The assigned weight of the
process allocation group becomes the
• Seeks out & adjusts over- or under- ceiling value
consumers • This ceiling is the same no matter
• May benefit complex queries or what other allocation groups are
mixed work active
• Is the default policy • Only when the allocation of CPU is
greater than the ABS % will the
ceiling have an impact
IMD (IMMEDIATE) REL (RELATIVE)
• All processes in the Allocation • Makes the Allocation Group Wgt %
Group are treated as equal value the ceiling on CPU
• Ignores uneven process usage • This ceiling will change based on
• Preferable for short, repetitive work which allocation groups are active
at the time
02/21/2002 15
NCR Confidential
Same Job Running under
Different Policies
All tests are run in M with the same assigned weight of 10
10 Concurrent Streams of Short 3-second Queries (Actual Tests)
800
Test 1 Test 2 Test 3 Test 4
700
600 719
500 585
Time in Seconds
400
300
200
100
89 87 10/70 10%
0
DEFAULT IMMEDIATE RELATIVE ABSOLUTE
100% of CPU 100% of CPU 14% of CPU * 10% of CPU
* Assume Only M, H and R active. Sum of weights is 70. The relative weight of M would be 10/70 or 14%.
02/21/2002 16
NCR Confidential
All Users Share the CPU Allocated
to the Allocation Group
Actual Test
40
39
30
Time in Seconds
20
18
10
11
7
0
1 $H User 5 $H Users 10 $H Users 20 $H Users
• The higher the number of active users in
the Allocation Group, the less CPU each
user will receive
02/21/2002 17
NCR Confidential
Dispersing Surplus Resources
Not used
RP0 - R 49% 10%
RP1 - X 33% 55%
RP0 - M 12% 20%
Surplus
RP0 - L 6% 15%
• When an Allocation Group cannot consume
all the resource it is entitled to, unused CPU
is shared based on Wgt %
02/21/2002 18
NCR Confidential
Example: Priority Scheduler
Aiding Response Time Consistency
Average Query Time for 100,000 Single Customer Reads
With and without High/Low Priorities (Actual Test)
0.25
Same
Priority
Average Time in Seconds
0.20
High vs
Low
0.15
0.10
0.05
0.00
0 Streams 5 Streams 10 Streams 20 Streams 30 Streams
Background Background Background Background Background
02/21/2002 19
NCR Confidential
SCHMON Utility
A UNIX-based Facility
• Priority Scheduler Facility (PSF) is
managed by issuing UNIX commands
• UNIX Root privileges are required
• Two key UNIX commands:
– schmon -d Displays the current PSF setting
– schmon -m Reports current resource usage by
group
• All changes in set up take place
immediately across all nodes
• No perceivable performance overhead
02/21/2002 20
NCR Confidential
Implementing Priority Scheduler
• Basic Steps
• Be Prepared
• Generalized Rules
02/21/2002 21
NCR Confidential
Basic Steps
• Understand the goals of using the ADW
– What workloads should have what priority at
what time (day, night, weekend)
• Set up Service Level Agreements
(SLAs)
• Understand the current performance
before making changes
• Take small steps and grow
02/21/2002 22
NCR Confidential
Be Prepared
• Collect data on the current performance
of the environment
– CPU and I/O by user, by hour (source
ampusage
– Overall system usage (source resusage)
– Current Query Wall clock time
• Collect data to understand the following
– Hourly CPU consumption by workload by day by
week
– Number of queries per hour per session
– Complexity of queries (CPU intensive versus
I/O) intensive.
• As stated so many times before: EXPLAIN, EXPLAIN
02/21/2002 23
NCR Confidential
Key Recommendations
1 There is rarely one way to implement Priority Scheduler
2 Understand you goals and current performance before
implementing Priority Scheduler
3 Priority Scheduler is not a band aid for poorly written queries
and locking bottlenecks
4 Try to minimize user activity in RP0, especially in the High and
Rush
5 Unless there is a good reason not to, keep RP0 the highest
weighted resource partition.
6 If consistent times are required for active queries (single and
few AMP), then assign RP weights such that there is at least a 4
to 1 ratio between the partition running response-sensitive work
and all other user partitions.
02/21/2002 24
NCR Confidential
Related docs
