Vista Growth Forecast
Developed for the UNC Vista Technical Working Group
TSI, DELTA, NC STATE UNIVERSITY 2007 Authored by: James R T Bossert, Manager, Technical Systems and Integration
Vista Growth Forecast
Developed for the UNC Vista Technical Working Group Executive Summary
The UNC Vista Technical Working Group has been tasked with projecting the growth of the UNC collaborative Vista Learning Management System implementation in order to inform UNC System planning efforts including future hardware purchases, systems staffing and budget planning. This report is based on four principal bodies of work: 1. A growth estimate through 2007 based on a linear regression forecast of 2006 usage data (pages 2-4). 2. Analysis of predicted future growth as estimated by each UNC member school (page 5). 3. Growth recommendations directly from Blackboard (page 6). 4. Polling other Universities that use Vista to help establish a current baseline for hardware implementation (pages 7-10). Analysis of current use shows that up to 10 application nodes are currently required in order to maintain optimal conditions during peak usage times. Results from the preliminary model show that up to 11 additional application nodes will be needed in order to maintain optimal conditions during peak usage times prior to the Fall 2007 Semester. Analysis of predicted growth by each of the collaborating UNC Schools shows that the number of sections and students using Vista will increase by approximately 173% between the Spring and Fall 2007 Semesters. A direct relationship between the number of sections and students and concurrent user sessions has not been established with the limited amount of historical data currently available. However, a conservative approach would be to plan on increasing the amount of required application nodes (up to 10) to accommodate a potential increase in concurrent user sessions of the same magnitude as the predicted growth in sections and users (up to 17). Blackboard publishes a minimum hardware requirements document which contains hardware configurations that will not scale to the collaborating UNC Schools current number of concurrent users, courses or students. Blackboard also has a services group that we can contract with to provide a customized hardware growth plan. Acquiring this service from Blackboard would provide a valuable comparison to results presented within this report. Analysis of the results from a poll of other Universities using Vista indicate that the collaborating UNC Schools have less hardware implemented with comparable course sections and student use than the reported average. Acquiring an additional 10 application nodes (for a total of 16) would bring us more in line with implementations at other Universities. A final conclusion will not be made in this report in order to allow VTWG members to make their own conclusions from the data presented herein. Please contact the author of this report for clarification or corrections to any materials contained in or referenced by this report.
Vista Growth Forecast | James R T Bossert | 2/27/2007
1
Statistical Analysis of Growth Method
The primary metric recommended by Blackboard to determine system usage is the number of Sessions per Application Node. TSI did not track this metric until January 2007, prior to that, different metrics were tracked (concurrent users and unique logins). Concurrent users (figure 1) have been tracked since October 2006, and unique logins (figure 2) since February 2006.
350 300 250 200 150 100 50 0 10/17 11/17 20000 15000 y = 1.022x - 39801 10000 5000 0 12/17 1/17 2/9 4/9 6/9 8/9 10/9 12/9 2/9 y = 15.87x - 61379
F IGURE 1: C ONCURRENT U SERS
F IGURE 2: U NIQUE L OGINS
Discussion posts read (figure 3) and content pages viewed (figure 4) were also added to the model to provide additional points of reference.
30000 25000 20000 15000 10000 5000 0 2/9 4/9 6/9 8/9 10/9 12/9 2/9 y = 21.01x - 81307 70000 60000 50000 40000 30000 20000 10000 0 2/9 4/9 6/9
y = 74.77x - 3E+06
8/9
10/9 12/9
2/9
F IGURE 3: D ISCUSSION P OSTS R EAD
F IGURE 4: C ONTENT P AGES V IEWED
Linear regression/best fit lines were modeled for each of the metrics and the resulting growth rate (slope) was averaged to find an average growth rate for the system. This average growth rate was then applied to the remainder of the year for the best data set (concurrent users). Because concurrent users and sessions are correlated (figure 5), a model can be created forecasting optimal user experience based on the number of concurrent users and Blackboard's sessions per application node recommendation.
Vista Growth Forecast | James R T Bossert | 2/27/2007
2
Result ts
Based on usage data as compiled so far during th Spring 200 Semester f all UNC S s o he 07 for Schools participati in the collaborative Vis implementation, a base ing sta eline estimate for the number of applica e ation nodes cur rrently requir was estab red blished (figure 6). This base e eline was then plotted ove time throug n er gh the rest o the calenda year at the estimated gr of ar rowth rate for median min nimum, media optimal, pe an eak minimum and peak opt timal usage (a currently d as defined by Bla ackboard).
3500 3000 2500 2000 1500 1000 500 0 0 100 200
Media an 6000 5000 4000 3000 2000 1000 0 400 1/23 1/30 2/6
Max ximum
R² = 0.648
300
2/13
2/20
2/27
F IGURE 5: C ONCURRENT U S O SERS & S ESSIONS S
F IGURE 6: D AILY M EDIAN S ESSIONS (200 N 07)
25 5
20 0 Vista Growth Forecast | James R T Bossert | 2/27/2007
15 5
10 0
5
0 2/12/2007 2/26/2007 3/12/2007 3/26/2007 5/21/2007 6/18/2007 7/16/2007 7/30/2007 8/13/2007 8/27/2007 9/10/2007 9/24/2007 10/8/2007 11/5/2007 4/23/2007 10/22/2007 11/19/2007 12/3/2007 6/4/2007 4/9/2007 5/7/2007 7/2/2007 12/17/2007 12/31/2007
median optimal n
median minim mum
pea optimal ak
peak minimum
F IGURE 7: E ST TIMATED A PPLIC CATION N ODE G ROWTH
3
Results | Continued
Analysis of current usage data shows that: • Between 6 and 10 application nodes are currently required in order to maintain minimum and optimal conditions respectively during peak usage times and between 4 and 7 application nodes to maintain minimum and optimal conditions respectively during regular usage times. In order to meet demand as predicted with the model described above, between 6 and 11 additional application nodes will be needed prior to the end of the Fall 2007 Semester (figure 7) to maintain minimum and optimal conditions respectively during peak usage times and between 7 and 11 additional application nodes to maintain minimum and optimal conditions respectively during regular usage times. Due to the nature of the increase in usage (which occurs mainly between semesters and not during semesters, see figures 1-4), the safest recommendation would be to have the desired amount of application nodes in place prior to the start of the Fall 2007 Semester.
•
•
Model Limitations
The absence of historical data for Sessions necessitated the need for correlation to data sets that had been tracked. The linear regression/best fit lines applied to the 4 data sets obtained at most a 32% correlation. The best data set (concurrent users) was only correlated to sessions at 65%. The statistical degree to which this model predicts future growth is therefore quite low. Session data will be kept in the future in order to more accurately project growth.
Vista Growth Forecast | James R T Bossert | 2/27/2007
4
Predic cted Grow wth as R Reported by UNC Member School s d r Predic ction Dat ta
Each of th collaborating UNC Scho (current NC State, UNC Charlo he ools tly otte, Wilming gton and Wes stern Carolina) were asked to predict usa for the up t age pcoming Sum mmer and Fall Semesters in order to aid n d growth pr rojections. Th raw data r he reported by e each school is shown in tab 1 and 2 b s bles below. School NCSU UNCC UNCW WCU Total Spring g 1095 5 565 5 90 0 264 4 2014 4 Summer r 292 2 178 8 560 0 125 5 1155 5 Fall 1692 1072 3245 600 6609 School NCSU UNCC UNCW WCU Total Sprin ng 1690 06 1480 00 225 54 333 33 3729 93 Summer 5364 4 2468 8 12000 0 1400 0 21232 2 Fall 24634 19883 12000 6307 62824
T ABLE 1: P RE EDICTED S ECTIO ONS R AW D ATA A
T ABLE 2: P R EDICTED S TUDE ENTS R AW D ATA A
Analys sis of th e Data
The sectio and studen growth dat submitted by each of th schools pre on nt ta he edicts a 191% (figure 8) an % nd 156% (figu 9) increas in sections and students respectively between the Spring and F 2007 ure se s s y e Fall Semesters The data su s. ubmitted by U UNCW was r removed from this average as an outlie (3606% and m e er d 532%). Av veraging the section and st s tudent growt rate yields a combined e th estimated gro owth rate of 173%.
7000 6000 5000 4000 3000 2000 1000 0 Spring Summ mer Fall 70000 0 60000 0 50000 0 40000 0 30000 0 20000 0 10000 0 0 Spring Sum mmer Fall
Vista Growth Forecast | James R T Bossert | 2/27/2007
WCU UNCW UNCC NCSU
WCU UNCW W UNCC NCSU
F IGURE 8: P REDICTED S EC CTION G ROWTH H
F IGURE 9 P REDICTED S T 9: TUDENT G ROW TH
alculate the p predicted grow for NC S wth State was to a average past semester to The formula used to ca o d 6 007) growth a apply the lower estimate and e semester (Fall 2005 to Fall 2006 and Spring 2006 to Spring 20 (130% for students, 14 for sectio to the Fa 2006 Seme r 46% ons) all ester numbers. There is n currently enough data to support a direct rela not any ationship betw ween the num mber of stude ents or section and the number of conc ns current user sessions. How wever, the sa afest conclusio to draw fr on rom this analys would be to plan for ap sis t pproximately a 173% incre ease in concurrent user se essions from t the Spring to Fall 2007 Sem mesters.
5
Growth Recommendations from Blackboard
The third method of estimating growth for future semesters is to ask Blackboard about hardware recommendations for the collaborating UNC Schools current user-base and to employ whatever means Blackboard has at their disposal for predicting growth.
Freely Available Documentation
After asking Blackboard directly for any documentation about hardware requirements, TSI received a standard 2 page Reference Hardware Requirements document. This document was designed for Campus Edition 6, but the Blackboard services representative assured TSI that it was applicable for Vista 4. The document describes the minimum hardware configuration for Windows, Linux and Sun Solaris configurations that support “up to 15,000 students, 1000 courses and up to 500 concurrent sessions”. In both cases, the hardware configuration is a dual processor application node and a dual processor database node all with 4 gigs of ram. Scaling these recommendations up to the collaborating UNC Schools current load would yield 4 dual processor application nodes for courses, about 3 dual processor application nodes for students and about 6 dual processor application nodes for concurrent sessions. These minimum recommendations do not scale to the level of use the collaborating UNC Schools are currently experiencing and expect in the future.
Custom Forecasting Services from Blackboard
The other option available from Blackboard is to contract with their services group to provide a hardware and growth recommendation based on an in-depth analysis by Blackboard of the collaborating UNC School’s current situation and plans for growth. This would presumably mirror methods used in creating this report, but would possibly benefit from the extra data that Blackboard has available on other Vista installations.
Vista Growth Forecast | James R T Bossert | 2/27/2007
6
Hardware Implementations at other Blackboard Vista Schools
In order to better establish a baseline for the collaborating UNC Schools current hardware requirements and to verify recommendations from Blackboard, current Vista Universities were polled via the primary systems administrators email list. Universities were asked to report on the following data points: • • • • • • • • Total Users Students Sections Average Concurrent Users Maximum Concurrent Users Application nodes Database processors Hardware configurations
Results
Results from the poll are shown in table 3 below.
School Texas State Tech College Monash University Iowa State University McGill University Purdue University Total Users Students Sections Avg Users Max Users App Nodes DB CPUs ‐ ‐ ‐ ‐ 62851 140033 ‐ 49000 101442 ‐ ‐ 25000 2600 ‐ ‐ ‐ ‐ 13800 1000 2000 2000 1500 11000 9592 ‐ 2800 4270 ‐ 2000 1500 1500 4000 ‐ 2100 1892 250 ‐ 5500 2700 2700 2500 8000 4000 3600 4143 2 20 6 9 12 52 20 9 16 4 8 4 8 8 24 8 8 9
Vista Growth Forecast | James R T Bossert | 2/27/2007
University of Georgia System UMass Online Texas A&M University Averages
T ABLE 3: B LACKBOARD S CHOOLS G ROWTH P OLL R ESULTS
The corresponding numbers from the Spring 2007 Semester for the collaborating UNC Schools are shown in table 4 below.
School UNC System Total Users Students Sections Avg Users Max Users App Nodes DB CPUs 207179 32207 7350 2080 3002 6 8 T ABLE 4: C OLLABORATING UNC S CHOOLS G ROWTH C OMPARABLES
Poll Limitations
The results from this poll must be viewed with caution. The response rate from the poll was quite low (8 Universities or Systems). Also, the methodology for generating the numbers was almost certainly different for each respondent. A follow up to this poll should utilize a web-based survey form with clear and explicit directions for calculating each data point.
7
Analys sis
Generally the collabor y, rating UNC S Schools have implemen nted less hard dware than the average reported by other scho using Vis with ools sta comparab usage. ble
2.00 1.50 1.00 Average
The collab borating UNC Schools hav implement C ve ted 0.50 s UNC Schools more app plication node per databas processor es se 0.00 than the r reported aver rage. TSI has strictly App Nodes per DB followed t general ad the dvice from Bl lackboard tha at CPU U there sho ould be a dedicated CPU in the database n e server for each applica r ation node. O Other F IGU 10: A PP N O DES VS . DB CP U S URE Universities don’t appe to be follo ear owing this rule d about this, ot ther Universit reported acceptable d ties d database load. One as strictly. When asked specifically a area for im mproving per rformance wit minimal ex th xpenditure wo ould therefor be to try in re ncreasing the number o application nodes withou increasing the size of th database se of ut he erver. The collab borating UNC Schools hav implement far fewer application n C ve ted nodes than ot ther Universit ties when com mpared to the number of s e sections and s students using Vista (figure 11 and 12) es ).
4.00 3.50 3.00 2.50 2.00 1.50 1.00 0.50 0.00 App A Nodes per 1000 s Sections
F IGUR E 11: A PP N OD ES
VS .
A Average U UNC Schools
App Nodes p 1000 per Studen nts
S ECTION S F IGU RE 12: A PP N O DES
VS .
S TUDEN NTS
The collab borating UNC Schools cur C rrently have 6 less than the average Blackboard r 68% n recommended d number o application nodes implem of mented verse concurrent users report by other Universities es t ted (figures 13 and 14). 3
5.00 4.00 3.00 2.00 1.00 0.00 Avg Avg A nimum Optimal Min
F IGURE 13: A VG A PP N ODE C OMPARISON ES N
2.50 2.00 1.50 A Average U UNC Schools 1.00 0.50 0.00 Peak Peak Minimum Optimal M
F IGURE 14 : P EAK A PP N O DES C OMPARISO ON
Average UNC Schools
Vista Growth Forecast | James R T Bossert | 2/27/2007
1.40 1.20 1.00 0.80 0.60 0.40 0.20 0.00
Average UNC Schools s
8
Conclusions
Results from the poll indicate that the collaborating UNC Schools have less hardware implemented than the average reported by other Universities with comparable course sections and student use. Acquiring an additional 10 application nodes (while maintaining our current database hardware) would bring us more in line with implementations at other Universities. These results should only be used in conjunction with other analysis due to the limitations documented above.
Vista Growth Forecast | James R T Bossert | 2/27/2007
9