201011-CS-CAS-Assessment by xiagong0815

VIEWS: 0 PAGES: 24

									                                            Program Level Outcome Assessment Report Instructions
                                                                  Applicable to the 2007/2008 Academic Year
                                                                         Version 2.0 edited: 9/10/2007
 Note: Because major changes have been made to the Program Level Outcome Assessment Report Template in 2007,
    we have highlighted major changes in the template instructions in yellow. We still recommend reading the full
        instructions, but expect this will better ensure that the new facets of the template are not overlooked.

Introduction
     The intent of this spreadsheet is to provide a means for each program to present the results of their assessment processes in a systematic fashion that is comparable to
     the reports of all other programs on campus.
     It is anticipated that the first use of this spreadsheet will be somewhat time consuming but that subsequent uses will involve a minimum of effort to accomplish the
     assessment. Please request assistance from the Assistant Provost for Academic Affairs in the Office of Academic Affairs if you have questions or require assistance in
     understanding these tables.
     It is assumed that there are within your program sufficient spreadsheet skills to edit the required tables. If you have trouble with any part of the spreadsheets please
     contact the Assistant Provost's office for assistance.

     To aid you in filling out this table, comments have been scattered throughout the spreadsheets. You can get rid of these by right clicking on the cell the comment is
     attached to then either hiding or deleting the comment. Note: unless you change your settings to print comments, they will not print with the rest of the document.

     The edited version of this spreadsheet for each program is to be submitted to the Assistant Provost by June 15th of each year. It is important to submit these materials
     on time so that the summer Assessment Peer Review Committee can complete its review of the institution's assessment progress in a timely manner.

                                                            Instructions                                                                    Editing Frequency

                                                Sheet tab: Mission & Context

                                                Table 1: Program Mission & Context

           Program Mission & Context
           Edit the information in the upper left hand corner identifying the program being reported on and the
           assessment coordinator (or faculty member responsible for the program's assessment). This information will
           automatically fill in on the other tabs of the report.
                                                                                                                                  When the mission or values of the program
           Enter the mission for your program. You can also list values for the program that are not related to measurable
                                                                                                                                change. Review this tab each year to determine
           student learning outcomes.
                                                                                                                               whether the mission and values are still accurate.

           If you would like to provide context to the peer review evaluators about your program or your assessment
           process (i.e. the program is new, or the program had fewer faculty than usual in the current academic year.)        Review this tab each year to determine whether
           This is an optional cell, which serves as a location for comments not related directly to the assessment of              the context narrative is still accurate.
           student learning outcomes.
                                                                                                                               Review this tab each year to determine whether
           Outcomes should be published in a student-friendly, accessible location, such as your catalog copy or program
                                                                                                                                the location of the published outcomes is still
           website. Note where your outcomes are located here.
                                                                                                                                                   accurate.




     Submitted Month/Day/Year                                    eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                       Instructions
                                            Program Level Outcome Assessment Report Instructions
                                                                   Applicable to the 2007/2008 Academic Year
                                                                          Version 2.0 edited: 9/10/2007
 Note: Because major changes have been made to the Program Level Outcome Assessment Report Template in 2007,
    we have highlighted major changes in the template instructions in yellow. We still recommend reading the full
        instructions, but expect this will better ensure that the new facets of the template are not overlooked.

Introduction
     The intent of this spreadsheet is to provide a means for each program to present the results of their assessment processes in a systematic fashion that is comparable to
     the reports of all other programs on campus.
     It is anticipated that the first use of this spreadsheet will be somewhat time consuming but that subsequent uses will involve a minimum of effort to accomplish the
     assessment. Please request assistance from the Assistant Provost for Academic Affairs in the Office of Academic Affairs if you have questions or require assistance in
     understanding these tables.
     It is assumed that there are within your program sufficient spreadsheet skills to edit the required tables. If you have trouble with any part of the spreadsheets please
     contact the Assistant Provost's office for assistance.

     To aid you in filling out this table, comments have been scattered throughout the spreadsheets. You can get rid of these by right clicking on the cell the comment is
     attached to then either hiding or deleting the comment. Note: unless you change your settings to print comments, they will not print with the rest of the document.

     The edited version of this spreadsheet for each program is to be submitted to the Assistant Provost by June 15th of each year. It is important to submit these materials
     on time so that the summer Assessment Peer Review Committee can complete its review of the institution's assessment progress in a timely manner.

                                                         Instructions                                                                       Editing Frequency
                                                Sheet tab: Outcomes Results

                                            Table 2: Assessment of Program Outcomes

           Published Program Outcomes:
           Edit this part of the table to match the program outcomes that you have currently listed in your program's
                                                                                                                               Edit when program outcomes in the assessment
           assessment plan. The plan and report should be updated whenever there are changes made to the outcomes
                                                                                                                               plan change. This should be done only with input
           or measures.
                                                                                                                                            from program faculty.
           You will probably need to add and delete rows here.
           Outcome Measures:
           Edit this part of the table to match the assessment measures that you have currently listed in your program's
                                                                                                                                   Edit when you add or delete assessment
           assessment plan. The plan and report should be updated whenever there are changes made to the outcomes
                                                                                                                               measures in your assessment plan. This should
           or measures.
                                                                                                                               be done only with input from the program faculty.
           You will probably need to add and delete columns here.
           Grade:
           This is the "grade" that your assessment measure would indicate for performance relative to the associated
           outcome. In some cases, this grade can be determined by some computational procedure that compares the
                                                                                                                                 Edit these EVERY YEAR. This is what your
           results of the assessment to some expected performance value. In many cases, this will be a judgment call by
                                                                                                                                 measures are telling you about performance.
           the program faculty based on what they are seeing in the data and/or experience that came out of the
           application of the indicated assessment process.
           The grades enter into the spreadsheet should meet the standard listed as follows:
           A = Honor grade; indicates comprehensive mastery of the outcome. Performance is exemplary. There is no
           way to improve performance on this outcome.
           B = Indicates a high level of performance in meeting the outcome. Performance is more than acceptable but is
           not exemplary.
           C = Indicates a satisfactory level of performance in meeting the outcome. Performance relative to this
           outcome meets minimum standards.
           D = Indicates that some level of performance has been achieved but that the level achieved is not acceptable.
           F = Indicates failure to achieve this outcome in any reasonable sense.
           NG = No Grade. The assessment measures are not used to measure the associated outcome.

           Plus and Minuses may be added to the grades to indicate degrees of accomplishment. Note: The high end of
           the grade table is an A, and the low end is an F. (There is no A+ or F-.)

           NOTE: A grade MUST be assigned to each cell or an error will result. All grades are upper case and
           without spaces before, within, or after.

           Be sure to leave grades as NG if a tool does not apply to a measure. This keeps the cell from factoring
           into the averages.
           For each grade, you should list the sample size ("n") of the results listed. For instance, if 20 student
           presentations were evaluated to determine a grade of B, the sample size would be 20.




     Submitted Month/Day/Year                                     eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                      Instructions
                                            Program Level Outcome Assessment Report Instructions
                                                                   Applicable to the 2007/2008 Academic Year
                                                                          Version 2.0 edited: 9/10/2007
 Note: Because major changes have been made to the Program Level Outcome Assessment Report Template in 2007,
    we have highlighted major changes in the template instructions in yellow. We still recommend reading the full
        instructions, but expect this will better ensure that the new facets of the template are not overlooked.

Introduction
     The intent of this spreadsheet is to provide a means for each program to present the results of their assessment processes in a systematic fashion that is comparable to
     the reports of all other programs on campus.
     It is anticipated that the first use of this spreadsheet will be somewhat time consuming but that subsequent uses will involve a minimum of effort to accomplish the
     assessment. Please request assistance from the Assistant Provost for Academic Affairs in the Office of Academic Affairs if you have questions or require assistance in
     understanding these tables.
     It is assumed that there are within your program sufficient spreadsheet skills to edit the required tables. If you have trouble with any part of the spreadsheets please
     contact the Assistant Provost's office for assistance.

     To aid you in filling out this table, comments have been scattered throughout the spreadsheets. You can get rid of these by right clicking on the cell the comment is
     attached to then either hiding or deleting the comment. Note: unless you change your settings to print comments, they will not print with the rest of the document.

     The edited version of this spreadsheet for each program is to be submitted to the Assistant Provost by June 15th of each year. It is important to submit these materials
     on time so that the summer Assessment Peer Review Committee can complete its review of the institution's assessment progress in a timely manner.

                                                            Instructions                                                                    Editing Frequency
           Values:
           The values entered here correspond to the grades entered on the line above. The scale is a 4.0 scale. The
           associated values are: A = 4.0, B = 3.0, C = 2.0, D = 1.0, F = 0.0. A PLUS adds 0.3 to the grade value. A
                                                                                                                              These are automatically computed values. There
           MINUS subtracts 0.3 from the grade value.
                                                                                                                                            is no need to edit.
           The scale is from A to F, with no value for A+ or F-.

           If you measure your outcomes in a way that makes it easier to enter values manually (such as performance on
           a qualifying professional exam), please be sure to update the letter grades that correspond to the entered
           values.

           Weights:
           It is recognized that not all measures are as significant as others when evaluating performance. The "weights"
           allow you to give different weight to each measure.

           In general, a "1" should be used for any measure that is a strong indication of performance on a particular
           outcome. A "0.5" weight should be used for measures that are indirect (such as surveys), have a low return
           rate, or are anecdotal in nature.
                                                                                                                              This should not change unless your assessment
           For example, if your assessment measure did not return good results for reasons such as low return, it did not     measures change or there is a change in opinion
           measure accurately for some reason, or any other reason that makes the results suspect or of lesser value,         of the program faculty concerning the applicability
           you should enter a 0.5 for that measurement.                                                                                        of the measure.

           If you have a high degree of confidence in the correlation between the results of the assessment and the
           associated outcome then enter a 1.0.

           Be sure to leave weights as NG if a tool does not apply to a measure. This keeps the cell from factoring into
           the averages.

           Current Year Results:
           These cells contain the weighted average of all the measure results for a given outcome. Particularly verify        These are automatically computed values. There
           that all measures are accounted for if you added to the list of measures. These weighted averages will be          is no need to edit unless columns were added that
           mapped to the university outcomes and matched with similar results from other programs.                                    do not appear in final computation.
           5 Year Trend Data:
           It is important to follow trends, particularly when working with the imprecise data of assessment. This part of
                                                                                                                                Edit EVERY YEAR, including the data from the
           the table should go back up to five years if you have such data available. You will need to enter these data
                                                                                                                                            prior assessments.
           from previous years; the cells do not automatically fill in previous years.

                                               Sheet Tab: Outcomes Analysis

                                        Table 3: Analysis of Outcome Assessment Results
           The purpose of this table is to provide analysis on the meaning of the results listed in Table 2. Outcomes and
           results will automatically fill in from Table 2.
                                                                                                                                Edit EVERY YEAR. This section interprets the
                                                                                                                                           current year's results.
           Your analysis should interpret the results, and discuss the significance of the program. What do the results
           really mean? Analysis should not focus on the tools or outcomes themselves.

           As an alternate, a word processing document can be provided (electronically) that accomplishes the same
           purpose. This table gives the program's interpretation of the results and explains the significance of the
           results. If a word processing document is used, please provide it in a format compatible with MS Word, and
           include all the aspects of the table (i.e. Outcome, Result, and Explanation of the Results).


     Submitted Month/Day/Year                                      eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                     Instructions
                                            Program Level Outcome Assessment Report Instructions
                                                                  Applicable to the 2007/2008 Academic Year
                                                                         Version 2.0 edited: 9/10/2007
 Note: Because major changes have been made to the Program Level Outcome Assessment Report Template in 2007,
    we have highlighted major changes in the template instructions in yellow. We still recommend reading the full
        instructions, but expect this will better ensure that the new facets of the template are not overlooked.

Introduction
     The intent of this spreadsheet is to provide a means for each program to present the results of their assessment processes in a systematic fashion that is comparable to
     the reports of all other programs on campus.
     It is anticipated that the first use of this spreadsheet will be somewhat time consuming but that subsequent uses will involve a minimum of effort to accomplish the
     assessment. Please request assistance from the Assistant Provost for Academic Affairs in the Office of Academic Affairs if you have questions or require assistance in
     understanding these tables.
     It is assumed that there are within your program sufficient spreadsheet skills to edit the required tables. If you have trouble with any part of the spreadsheets please
     contact the Assistant Provost's office for assistance.

     To aid you in filling out this table, comments have been scattered throughout the spreadsheets. You can get rid of these by right clicking on the cell the comment is
     attached to then either hiding or deleting the comment. Note: unless you change your settings to print comments, they will not print with the rest of the document.

     The edited version of this spreadsheet for each program is to be submitted to the Assistant Provost by June 15th of each year. It is important to submit these materials
     on time so that the summer Assessment Peer Review Committee can complete its review of the institution's assessment progress in a timely manner.

                                                            Instructions                                                                    Editing Frequency

                                      Sheet Tab: Current Year Recommendations

                            Table 4: Program Improvement Recommendations Based on Assessment
           This table is a critical part of the improvement process. After discussing the results of your assessments with
           your constituent groups, it is time to recommend improvements to the program. Recommendations need to be
           recorded and tracked.                                                                                                            Update EVERY YEAR.
           Record all improvement recommendations based on the current year's assessment. Identify the outcomes that
           the improvement recommendation is intended to enhance. Some rationale for the proposed action is
           suggested to tie the recommendation to the issues discussed in Table 3.

           Responding to popular demand, there are now two recommendation tables on this tab. The first table should
           reflect recommendations for program improvement, rather than for the assessment process. Examples of
           program improvement recommendations include adding tutors or course lab time to improve weak
           performance on a particular outcome.

           The second table can be used to make recommendations that are not programmatic, but are solely focused on
           the program assessment process. Examples of assessment process recommendations include changing an
           outcome, or adding a direct measure (such as performance on a course presentation) because of low
           response rate for existing surveys.

           State the resource implications associated with each suggestion. For instance, if you are adding tutors for
           students struggling with a given outcome, will a course fee be added to offset the cost?

           Also include all prior year recommendations that have not been previously completed. For clarity, it is good to
           note that these recommendations carry over from previous years.


                                     Sheet Tab: Previous Year Recommendations

                            Table 5: Program Improvement Recommendations Based on Assessment
           Record all improvement recommendations made in the previous year's assessment report. Note the action
           taken to address each of these recommendations. Describe the effect of these actions, if there has been                          Update EVERY YEAR.
           sufficient time to demonstrate results.
           Like the Current Year Recommendations tab, there are two recommendation tables on this tab. The first table
           should reflect previous year recommendations for program improvement, rather than for the assessment
           process. Examples of program improvement recommendations include adding tutors or course lab time to
           improve weak performance on a particular outcome.

           The second table can be used to show previous year recommendations that are not programmatic, but are
           solely focused on the program assessment process. Examples of assessment process recommendations
           include changing an outcome, or adding a direct measure (such as performance on a course presentation)
           because of low response rate for existing survey tools.




     Submitted Month/Day/Year                                    eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                       Instructions
Program Outcomes Assessment
                Program    BS/BA Computer Science
             Department    Mathematical Sciences
          College/School   College of Arts & Sciences
          Academic Year    2010

 Assessment Coordinator Kenrick Mock
                                                                                  TABLE 1
                                                                        PROGRAM MISSION AND CONTEXT
Use the space below to enter your program's mission, and provide general comments about the program that are not directly related to the assessment process. If there are factors
             you would like the peer review committee to be aware of that are not directly related to student learning outcomes, you can describe them in this space.

                                               You should also use the space below to note where the program outcomes are published.
                                                                                 Program Mission

The Computer Science program at the University of Alaska Anchorage is committed to the University’s mission to discover and
 disseminate knowledge through teaching, research, engagement, and creative expression. First, through undergraduate and
 graduate programs, we strive to teach our students the fundamental principles of computer science and important issues in
 computing so they may pursue advanced degrees or enter the workplace as productive, competent software development or
   information technology professionals. Second, the program seeks to further the profession of computer science through
    professional activities and public service within the community, state, nation, and society at large. Finally, the program
engages in and disseminates research to advance the development of computer science and provide innovative technological
                                        solutions to address the needs of modern society.
                                                                     Program Context and General Comments

The eight students that completed our assessment process this year were stronger than normal and exceeded national averages
  for our outcomes. Normally we have a couple of weak students but that was not the case this year. Performance was on par
 with the great graduating crop of 2004-2005. We are still investigating the use of a rubric to evaluate our capstone projects and
       will be adding additional assessment instruments to measure student achievement after graduation to satisfy ABET
                                                  accreditation requirements.
                                                                         Location of Published Outcomes


   The outcomes are available online at http://www.math.uaa.alaska.edu (look for the CS Program Outcomes in the lower left
                                                          corner).




  Submitted Month/Day/Year                                           eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                               Mission & Context
Program Outcomes Assessment
       Program                                    BS/BA Computer Science
     Department                                   Mathematical Sciences
    College/School                                College of Arts & Sciences
    Academic Year                                 2010
Assessment Coordinator                            Kenrick Mock
                                                                                             TABLE 2
                                                                               PROGRAM OUTCOMES ASSESSMENT RESULTS
                                                                                                                                                                   Current
                                                                                                                       Measures                                                                   5 Year Trend Data
                                                                                                                                                                    Year




                                                                                                           Computer Science




                                                                                                                                                                   Weighted Average
                                                                                                           Capstone Project




                                                                                                           ETS Field Test in
                                                                                                            Sample size for




                                                                                                            Sample size for




                                                                                                                                                 Sample size for




                                                                                                                                                                    automatically)
                                                                                                             Evaluations




                                                                                                                                   Exit Survey




                                                                                                                                                                      (Calculates



                                                                                                                                                                                      2006/2007

                                                                                                                                                                                                    2007/2008

                                                                                                                                                                                                                2008/2009

                                                                                                                                                                                                                            2009/2010

                                                                                                                                                                                                                                        2010/2011
                                                                                                              measure




                                                                                                              measure




                                                                                                                                                   measure
                                                                                                Grade B+                NG         C+                                    B             B             B          B-          B-           B
                                                     Demonstrate oral communications skills
                                              1                                                 Value 3.44         8              2.60                 4              3.16            3.29          3.2         2.78 2.94 3.16
                                                  consistent with a career in Computer Science.
                                                                                                Weight 1.00             NG        0.50
                                                                                                   Grade    B+          NG         C+                                    B            B+            B-           B          B-           B
       Published ProgramObjectives/Outcomes




                                                    Demonstrate written communication skills
                                              2                                                 Value 3.50         8              2.60                 4              3.20            3.33 2.97 3.29 2.92                               3.2
                                                  consistent with a career in Computer Science.
                                                                                                Weight 1.00             NG        0.50
                                                                                                   Grade     B          NG         B+                                    B            B+            B-           B          B+           B
                                                     Demonstrate abilities in critical thinking,
                                              3     problem solving and analysis skills, and       Value   3.16    8              3.40                 4              3.24            3.41 2.77 3.15 3.33 3.24
                                                                software design.
                                                                                                   Weight 1.00          NG        0.50
                                                                                                   Grade     B          A-           B                                  B+            B-            C+           B          B-          B+
                                                  Demonstrate abilities in software development
                                              4                                                 Value 3.25         8   3.80   8   3.00                 4              3.42            2.92 2.66 3.12 2.74 3.42
                                                             and implementation.
                                                                                                Weight 1.00            1.00       0.50
                                                     Understand and apply core concepts in    Grade NG                  B+         B+                                   B+            B-            C+          C+          B-          B+
                                                    Computer Science to the development or
                                                                                              Value                    3.60       3.60                                3.60            2.80          2.3         2.31 2.75               3.6
                                                    analysis of computer systems, including
                                              5                                                                               8                        4
                                                    algorithms, data structures, concepts of
                                                  programming languages, operating systems, Weight NG                  1.00       0.50
                                                  and computer organization and architecture.

                                                   Demonstrate an understanding of theoretical Grade NG                 B+           B                                  B+            C+            C+          C+          B-          B+
                                                   foundations of Computer Science including
                                              6                                                 Value                  3.60   8   3.00                 4              3.40            2.67 2.53 2.68 2.84                               3.4
                                                  discrete mathematics, algorithm analysis, and
                                                                  computability.                Weight NG              1.00       0.50
                                                                                                   Blank and shaded = Does not measure this published outcome.
                                                                                                   Weights: 0.5 = Measure gives anecdotal or indirect indication of published outcome, or has a limited sample size.
                                                                                                                1 = Measure provides direct, reliable information on the published outcome.




    Submitted Month/Day/Year                                                                        eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                                                                       Outc Results
Program Outcomes Assessment Discussion
      Program    BS/BA Computer Science
   Department    Mathematical Sciences
College/School   College of Arts & Sciences
Academic Year    2010
  Assessment
   Coordinator   Kenrick Mock
                                                                                                      TABLE 3
                                                                                 ANALYSIS OF OUTCOME ASSESSMENT RESULTS
                 Note: Alternatively, an electronic word-processing document compatible with MS Word may be substituted for this table.
                                 Outcome                    Result                                                               Explanation of the Results

                                                                        Outcomes and results automatically fill in from the previous tab.
                 Demonstrate oral communications skills
                                                                    The grade is determined by faculty evaluation of student project presentations and the student exit survey. Faculty were generally pleased with the oral
            1     consistent with a career in Computer         B
                                                                    presentations. Students rate our instruction almost a grade point lower than the faculty evaluation. This is consistent with evaluations from prior years.
                                Science.


                   Demonstrate written communication                The grade is determined by faculty evaluation of student project writeups and the student exit survey. Overall, faculty were pleased with the written
            2       skills consistent with a career in         B    reports with ratings of mostly A's and B's. Similar to outcome #1, students rated our instruction for written communication slightly lower than the faculty
                            Computer Science.                       evaluation.


                                                                    The grade is determined by faculty evaluation of student project writeups and the student exit survey. Overall, the students demonstrated success for
                 Demonstrate abilities in critical thinking,
                                                                    this outcome although there is room for improvement - some students made poor progress on their projects and were not able to demonstrate analysis
            3     problem solving and analysis skills,         B
                                                                    and problem solving skills. The software design was correspondingly weak. Most students independently learned new programming languages or
                         and software design.
                                                                    computing technologies.


                                                                    The grade is measured by all three assessment tools and is likely the most accurate. All students performed well on the ETS exam, indicating that they
                    Demonstrate abilities in software
            4                                                  B+   understand programming concepts. However, on the capstone project, some students only implemented a portion of their project which hurt their final
                    development and implementation.
                                                                    grade.

              Understand and apply core concepts in
              Computer Science to the development
            5    or analysis of computer systems,              B+   The grade is determined by the ETS field test and the exit survey. Students performed at the 90th percentile nationally, which is up from previous years.
               including algorithms, data structures,
               concepts of programming languages,
                 operating systems, and computer
                 Demonstrate an understanding of
                  theoretical foundations of Computer               The grade is determined by the ETS field test and the exit survey. Students performed at the 90th percentile. This is an improvement from previous
            6          Science including discrete              B+   years. However, the relatively small number of students that took the test makes it hard to draw conclusions that our program has improved student
                  mathematics, algorithm analysis, and              learning.
                              computability.




     Submitted Month/Day/Year                                                            eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                                            Outc Analysis
Program Improvement Recommendations Based on Assessment
      Program    BS/BA Computer Science
   Department    Mathematical Sciences
College/School   College of Arts & Sciences
Academic Year    2010
  Assessment
   Coordinator   Kenrick Mock


                                                                  TABLE 4
                                PROGRAM IMPROVEMENT RECOMMENDATIONS BASED ON ASSESSMENT RESULTS: CURRENT YEAR
                                 Recommendation                    Outcome(s)  Rationale for the Recommendation Resource Implications

We are seeking ABET accreditation and there will be program changes as a result of the                                                       Faculty time to develop
                                                                                                    ABET Accreditation.
                              accreditation process.                                                                                         curriculum and assessment tools



                                                                                                    Preliminary data suggests CS A101 may
                                                                                                    not help students succeed in CS A201
                                                                                                    (the first required course for CS majors)
                                                                                                    even though we recommend this to
       Examine role of CS A101 - Does it really prepare students for CS A201?             ALL       students that don't have sufficient math Faculty time to evaluate data.
                                                                                                    to take CS A201. Some preliminary
                                                                                                    analysis here:
                                                                                                    http://www.math.uaa.alaska.edu/~afkjm/t
                                                                                                    echteach/?q=node/87




                                                              TABLE 5
                       ASSESSMENT PROCESS IMPROVEMENT RECOMMENDATIONS BASED ON ASSESSMENT RESULTS: CURRENT YEAR
                             Recommendation                    Outcome(s)  Rationale for the Recommendation Resource Implications




                                                                                                                                             Faculty time to develop the
                                                                                                    This recommendation is from last year    rubrics. When implemented,
Design and implement a rubric for faculty to evaluate outcomes 1-4 based on the student
                                                                                          ALL       and remains in consideration for the     rubric scores may not be directly
                                  capstone projects.
                                                                                                    current year.                            comparable to previous year
                                                                                                                                             scores.




                                                                                                    Captive audience - students will
                                                                                                    complete the survey in person on paper.
                                                                                                                                            Slightly more faculty time to
                                                                                                    This semester I implemented the survey
      Implement the Exit Survey on paper when students give their presentation.           ALL                                               manually administer and
                                                                                                    using Qualtrics, which makes data
                                                                                                                                            aggregate data.
                                                                                                    aggregation easier, but only half the
                                                                                                    students completed the survey.




    Submitted Month/Day/Year                                         eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                              Current Yr Recommendations
Program Improvement Recommendations Based on Assessment
      Program
   Department
                 BS/BA Computer Science
                 Mathematical Sciences
                                                                                                This page shows recommendations made in previous years, along with action taken
College/School   College of Arts & Sciences                                                                       since that time and the effect of those actions.
Academic Year    2010
  Assessment
   Coordinator   Kenrick Mock


                                                                   TABLE 6
                                PROGRAM IMPROVEMENT RECOMMENDATIONS BASED ON ASSESSMENT RESULTS: PREVIOUS YEARS

    Year                                   Recommendation                                      Outcome(s)     Rationale for the Recommendation                 Resource Implications                    Actions Taken


                                                                                                                                                                                              Lab now an ITS supported lab
                                                                                                                                                                                              with regularly upgraded
                                                                                                            The facilities ultimately impact all                                              computers; block grants have
                                                                                                            outcomes. The student survey gave the                                             been used to add supplemental
                 Seek ways to upgrade the laboratory facilities with cutting-edge
    2003                                                                                           ALL      facilities the lowest ranking (2.4 out of a   Funding.                            technology (e.g. 3D monitors,
                 equipment
                                                                                                            scale of 1-5 where 5 is best) of the                                              robots), and equipment grants
                                                                                                            survey.                                                                           from NSF, HP have
                                                                                                                                                                                              supplemented technology
                                                                                                                                                                                              available to students.



                                                                                                            Many students enrolled in software
                                                                                                            engineering after completing the
                                                                                                            capstone course. Ideally, students                                                CS A401 was moved to Fall and
                 Align CS A470 with CS A401, software engineering, in terms of content
    2003                                                                                           ALL      should enroll in software engineering first None.                                 CS A470 to Spring to alleviate
                 and scheduling.
                                                                                                            to ensure knowledge of project                                                    this problem.
                                                                                                            management and software development
                                                                                                            techniques during the capstone project.


                                                                                                                                                                                              Changed course sequence to CS
                                                                                                                                                                                              A201, CS A202, then CS A330 in
                                                                                                            Enrollment in upper division courses is                                           Fall, 2005, providing a gentler
                                                                                                            down. We believe the high attrition in CS                                         introduction to data structures
                                                                                                            A203 is a significant contributor to the                                          and algorithms. An analysis in
    2004         Re-align content in CS A203, CS A303                                              ALL      drop in enrollment. Look at ways to re-   Faculty time.                           2009 showed that under the new
                                                                                                            design the curriculum to ease the                                                 reorganization the percentage of
                                                                                                            transition for the students from CS A201                                          students completing Data
                                                                                                            to CS A203.                                                                       Structures (CS A330) with a C or
                                                                                                                                                                                              better increased from 44% to
                                                                                                                                                                                              68%.

                                                                                                                                                                                               Some presentations at high
                 Seek ways to increase enrollment and retention (advertising, recruitment,                                                                Faculty time, cost for certain types school and fairs, but outreach
    2005                                                                                           ALL      Low enrollment and high attrition.
                 develop exciting recruitment presentation/activities, hold special events)                                                               of events.                           efforts still minimal. Tutoring
                                                                                                                                                                                               added (see below).
                                                                                                                                                                                              Lab fees increased in 2006 to pay
                                                                                                                                                                                              for student tutors in the CS lab.
                                                                                                                                                                                              The effect has not been
                                                                                                                                                                                              measured quantitatively but
    2006         Provide tutoring in the CS Lab.                                                   ALL      High attrition in introductory courses.       Lab fees to pay for tutors.
                                                                                                                                                                                              students have given positive
                                                                                                                                                                                              feedback about the tutors and
                                                                                                                                                                                              have requested additional
                                                                                                                                                                                              tutoring hours.
                                                                                                                                                                                              No new actions taken. From the
                                                                                                                                                                                              2009 assessment committee
                                                                                                                                                    Faculty/staff time to design and
                                                                                                            Our results are plagued by small sample                                           remarks, actions regarding this
                 Seek ways to increase enrollment and retention (advertising, recruitment,                                                          plan events and activities.
  2007-2008                                                                                        ALL      sizes because we have less than 10                                                recommendation have been
                 develop exciting recruitment presentation/activities, hold special events)                                                         Extramural funding likely to be
                                                                                                            graduates per year.                                                               removed from this document
                                                                                                                                                    necessary for some events.
                                                                                                                                                                                              since it does not directly involve
                                                                                                                                                                                              the CS curriculum.

                                                                                                                                                                                              We have seen a drop in
                                                                                                                                                                                              enrollment in several CS courses
                                                                                                                                                                                              due to changes in the CSE
                                                                                                                                                                                              program. From the 2009
                 Monitor changes in the CSE program which will result in fewer                                                                                                                assessment committee remarks,
    2008                                                                                          NONE      Impact on enrollment.                         Faculty/staff time.
                 engineering students taking computer science courses.                                                                                                                        actions regarding this
                                                                                                                                                                                              recommendation have been
                                                                                                                                                                                              removed from this document
                                                                                                                                                                                              since it does not directly involve
                                                                                                                                                                                              the CS curriculum.

                                                                                                                                                                                              The lost lab is now the SCICOMM
                                                                                                                                                                                              lab and can be used by CS
                                                                                                                                                                                              students on an as-available
                                                                                                                                                          Faculty/staff time, current plan
                                                                                                                                                                                              basis. Several CS students
                                                                                                            One CS lab lost ITS support due to low        requires sharing space with
                 Seek ways to maintain equipment in the CS lab, in particular                                                                                                                 lamented the loss of the lab, but it
    2008                                                                                           ALL      usage. Support will allow the machines        Science and Mathematics which
                 reinstatement as an ITS supported lab.                                                                                                                                       does not seem to have affected
                                                                                                            to be updated on a regular cycle.             will cut down on available space
                                                                                                                                                                                              academic performance. If
                                                                                                                                                          for CS students.
                                                                                                                                                                                              enrollment increases then the
                                                                                                                                                                                              single ITS lab may become
                                                                                                                                                                                              insufficient.




                                                                TABLE 7
                        ASSESSMENT PROCESS IMPROVEMENT RECOMMENDATIONS BASED ON ASSESSMENT RESULTS: PREVIOUS YEARS



      Submitted Month/Day/Year                                                                eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                   Previous Yr Recommendations
  Year      Recommendation                                                               Outcome(s)     Rationale for the Recommendation                     Resource Implications                      Actions Taken



                                                                                                      Faculty currently assign a grade of A-F for
                                                                                                      each outcome. However, from faculty
                                                                                                      comments the basis for this grade sometimes
                                                                                                      varies (e.g. for oral communication one faculty
                                                                                                      member might heavily weigh the design and
                                                                                                                                                        Faculty time to develop the rubrics.
                                                                                                      content of PowerPoint slides while another                                                 We have developed drafts of the
            Design and implement a rubric for faculty to evaluate outcomes 1-4                                                                          When implemented, rubric scores
  2010                                                                                       ALL      might focus on speaking skills). Implementing                                            rubric but nothing has been approved
            based on the student capstone projects.                                                                                                     may not be directly comparable to
                                                                                                      a rubric for each outcome will eliminate much                                                             yet.
                                                                                                                                                        previous year scores.
                                                                                                      of the ambiguity while providing finer
                                                                                                      granularity within each outcome for specific
                                                                                                      program improvements. Rubrics have
                                                                                                      recently received national attention as an
                                                                                                      effective way to perform assessment.


                                                                                                      Faculty evaluated outcome 4 close to a
                                                                                                      B+ while the ETS Field exam evaluated
                                                                                                      outcome 4 as a C+. Is the exam
                                                                                                                                                                                               The 2008 and 2009 results were
            Monitor assessment tools for Outcome 4 (Students exhibit abilities in                     measure the same thing as the faculty?
                                                                                                                                                                                               relatively close to each other, so
2006-2007   software development) to see if the tools are consistent in measuring the         4       Or do we not cover this topic adequately? Assessment coordinator time.
                                                                                                                                                                                               previous discrepancies appear to
            same thing.                                                                               (The ETS exam did change in 2006).
                                                                                                                                                                                               be outliers.
                                                                                                      This is difficult to determine because ETS
                                                                                                      does not tell us specifically what exam
                                                                                                      questions map to the outcome but we




                                                                                                      The return rate for the exit survey using
                                                                                                      mail has been low, approximately 20-
            Improve return rate for the exit survey by asking students to complete it                                                                                                          Done, return rate significantly
  2007                                                                                       ALL      25%. Handing out the survey and having Faculty, class time
            during a CS A470 class period.                                                                                                                                                     improved
                                                                                                      it completed in class should increase the
                                                                                                      return rate.




                                                                                                      Insufficient data makes conclusions
                                                                                                      unreliable; pooling across a larger
                                                                                                                                                                                               This has not been done, but the
            Consider measuring outcomes using a running average that includes prior                   sample size may generate more valid
  2007                                                                                       ALL                                              Assessment coordinator time.                     data exists if we wish to measure
                                          3 years                                                     conclusions if the assessment procedure
                                                                                                                                                                                               this way.
                                                                                                      remains the same during the sampling
                                                                                                      period.




   Submitted Month/Day/Year                                                             eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx                                                          Previous Yr Recommendations
Assessment Trends               ETS Exam and Capstone Evaluations

                                National Percentile
                                2003-2004 2004-2005 2005-2006 2006-2007 2007-2008 2008-2009 2009-2010
Programming Fundamentals                73          95     85        53        60        75        70
Computer Org/Arch/OS                    87          94     80        65        65        50        65
Algorithms/Theory/Math                  78          94     80        85        60        65        70

                                ETS EXAM


      100

       90

       80

       70

       60

       50

       40

       30

       20

       10

        0
             2003-2004   2004-2005   2005-2006   2006-2007   2007-2008   2008-2009   2009-2010   2010-2011


                                           470 Evaluations

                                2003-2004 2004-2005 2005-2006    2006-2007 2007-2008 2008-2009 2009-2010
Oral                             3.814583 3.636667       3.47     3.604167     3.406 3.116667 3.213636
Written                          3.702083 3.487667       3.12     3.533333      3.43 3.638889 3.027273
Critical Thinking                    3.875     3.54 3.408333      3.655556      3.24 3.174074 3.277273
Programming                      3.659375     3.564     3.355         3.425     3.22 3.192593 2.459091



      4.5

        4

      3.5

        3

      2.5
2.5

 2

1.5

 1

0.5

 0
      2003-2004   2004-2005   2005-2006   2006-2007   2007-2008   2008-2009   2009-2010   2010
2010-2011
       95
       90
       90




      Programming Fundamentals
      Computer Org/Arch/OS
      Algorithms/Theory/Math




2010-2011
   3.4375
       3.5
   3.1625
      3.25




                 Oral
            Oral
            Written
            Critical Thinking
            Programming




2010-2011
                           Values by Tool                Year Average
                            Grade Table                   Grade Table
                            0         F                   A         4
                           0.3        F+                 A-        3.7
                           0.7        D-                  B         3
                            1         D                  B-        2.7
                           1.3        D+                 B+        3.3
                           1.7        C-                  C         2
                            2         C                  C-        1.7
                           2.3        C+                 C+        2.3
                           2.7        B-                  D         1
                            3         B                  D-        0.7
                           3.3        B+                 D+        1.3
                           3.7        A-                  F         0
                            4         A                  F+        0.3
                           NG                            NG




Submitted Month/Day/Year    eaee0d17-5092-4fbf-8a9f-629c781f29a9.xlsx    Grade Table
Program          BS/BA Computer Science
Department       Mathematical Sciences
College/School   College of Arts & Sciences
Academic Year    2010


     Student                       Faculty Average   Overall Averages
        1        Outcome 1           4.00
                 Outcome 2           4.00            Outcome 1      3.44
                 Outcome 3           4.00            Outcome 2      3.50
                 Outcome 4           4.00            Outcome 3      3.16
                                                     Outcome 4      3.25
         2       Outcome 1            3.70
                 Outcome 2            3.70
                 Outcome 3            4.00
                 Outcome 4            3.70

         3       Outcome 1            3.70
                 Outcome 2            3.30
                 Outcome 3            4.00
                 Outcome 4            4.00

         4       Outcome 1            2.70
                 Outcome 2            3.00
                 Outcome 3            1.00
                 Outcome 4            2.00

         5       Outcome 1            3.70
                 Outcome 2            3.70
                 Outcome 3            3.00
                 Outcome 4            3.30

         6       Outcome 1            2.00
                 Outcome 2            3.00
                 Outcome 3            1.30
                 Outcome 4            1.00

         7       Outcome 1            4.00
                 Outcome 2            4.00
                 Outcome 3            4.00
                 Outcome 4            4.00

         8       Outcome 1            3.70
                 Outcome 2            3.30
                 Outcome 3            4.00
                 Outcome 4            4.00

         9       Outcome 1
                 Outcome 2
                 Outcome 3
                 Outcome 4

        10       Outcome 1
                 Outcome 2
     Outcome 3
     Outcome 4

11   Outcome 1
     Outcome 2
     Outcome 3
     Outcome 4

12   Outcome 1
     Outcome 2
     Outcome 3
     Outcome 4

13   Outcome 1
     Outcome 2
     Outcome 3
     Outcome 4

14   Outcome 1
     Outcome 2
     Outcome 3
     Outcome 4

15   Outcome 1
     Outcome 2
     Outcome 3
     Outcome 4

16   Outcome 1
     Outcome 2
     Outcome 3
     Outcome 4
Program          BS/BA Computer Science
Department       Mathematical Sciences
College/School   College of Arts & Sciences
Academic Year    2010



ETS Field Test Results - Computer Science

      Student                     Total Score (120-200)              ETS Assessment Indicators
          1                              170
          2                              164                         Programming Fundamentals
          3                              165                         Computer Org/Arch/OS
          4                              173                         Algorithms/Theory/Math
          5                              151
          6                              180
          7                              152
          8                              160
          9
         10


                 Average =            164.38              Nationwide Average=149.3
                 Median=               164.5              Nationwide Median = 148
                 StDev=           10.01338
                 Nationwide Percentile = 95%


                                  Other Data

                                  Gender       Male              8
                                               Female            0
                                               No Response       0

                                  Ethnicity    Am. Indian        0
                                               Mexican Am.       0
                                               Asian             0
                                               Black             0
                                               White             7
                                               Other             0
                                               No Response       1

                                  Education Junior               0
                                            Senior               7
                                            Other                1

                                  Transfer     Yes               2
                                  Student      No                5
                                               No Response       1

                                  Enrollment Fulltime            7
                                             Parttime            0
                                             No Response         1
English   Yes              7
Best Lang?No               0
          Equally Good     0
          No Response      1

GPA        2.0-2.49        0
           2.5-2.99        1
           3.0-.3.49       3
           3.5-4           3
           No response     1

Education Bachelor         4
Planned Master             0
          Doctoral         1
          Undecided        2
          No Response      1

CS GPA     2..0-2.49       0
           2.5-2.99        0
           3.0-3.49        1
           3.5-4           6
           No Response     1

Major
Distance
Learning
Courses    None            3
           Less than 40%   4
           40% to 90%      0
           More than 90%   0
           No Response     1
                        Range=0-100
                        National National
Program Outcome UAA Ave Ave      Percentile        Grade

       4                76       54.9         95           3.8
       5                56       42.3         90           3.6
       6                49       34.9         90           3.6




                Number of Institutions: 217
Program          BS/BA Computer Science
Department       Mathematical Sciences
College/School   College of Arts & Sciences
Academic Year    2010


Exit Survey Results                                                        1=Poor, 2=Fair, 3=Good, 4=Excellent, 5=Outstanding

Student          Outcome 1          Outcome 2          Outcome 3           Outcome 4        Outcome 5           Outcome 6
        1        3                  3                  4                   3                4                   3
        2        3                  3                  4                   4                5                   4
        3        3                  3                  4                   4                5                   4
        4        4                  4                  5                   5                4                   5
        5
        6
        7
        8
        9
       10
       11

      Ave        3.25               3.25               4.25                4.00             4.50                4.00
   ScaledAve     2.60               2.60               3.40                3.20             3.60                3.20



Misc comments    Even with the inadequate resources provided by UAA (IMHO), they still manage to provide quality instruction to those w

                 I did not use the computer lab, and I don't really feel like the physical facilities are related to the quality of the computer
lent, 5=Outstanding

              Advising     InstructionFacilities Labs                          Other Data
              3            4          1          1
              4            4          3          2                             Has Job:     Yes                     1
              5            4                                                                No                      4
              5            4
                                                                               Recommend UAA to others

                                                                                            Yes                     3
                                                                                            No                      0
                                                                                            Maybe                   1



              4.25         4.00          2.00        1.5
              3.40         3.20          1.60        1.20



uality instruction to those willing to work hard.

the quality of the computer science program. It should be the overall responsibility of the university to maintain classrooms and facilities.
Program          BS/BA Computer Science
Department       Mathematical Sciences
College/School   College of Arts & Sciences
Academic Year    2010


  Student Exit
    Survey
   Comments

                 UAA Strengths




                 Dr. Mock is an amazing teacher and adviser; I hope you truly appreciate the
                 outstanding contributions he makes to the department and the school in
                 general.




                 1. Very small class size. 2. A majority of quality instructors. 3. Extremely
                 challenging course materials, likely at the same level as many top tier
                 computer science programs in the nation.
Areas of Improvement / Suggestions


1. The CS department needs more staff. Not only do they need more professors to provide
the graduate programs advertised in the UAA catalog, but they also need more professors
to teach core CS classes (which students must currently wait semesters to take). 2. The
CS department needs more funding and resources, both to provide better lab equipment,
and to pay tutors/graders (which takes pressure off of professors). I think this is a vital
step, if UAA wants to retain and graduate more CS students.



1. Lack of options for classes, especially electives. Related to having too few instructors in
the department. 2. Several courses which I feel were very important were taught by a poor
quality instructor (also related to too few instructors). 3. I felt it would have been beneficial
to offer more courses geared towards teaching specific programming languages or
paradigms. Even though these technologies can become quickly outdated, it seems useful
to learn applied information along with the theoretical material currently offered. Instead,
many courses focused mainly on Java, and I was left for the most part to learn other
languages on my own (PHP, C++, Python, etc.). Also, more web development material
would have been interesting since it is such a large area in the industry today.



Additional coursework focusing on project and team methodologies beyond CS401. Add
an additional course or provide more instruction addressing it elsewhere in the curriculum.



First, please refer to my answer to the previous question, because I included my
suggestions in it. Computers will play an even *more* critical role in the future than they do
today. So, plan ahead, and give the CS department the tools it needs to become strong.



My guess is that the computer science department is underfunded at UAA. Perhaps some
of the massive amounts of funding could be diverted from the Conoco Philips Science
Building or the new Allied Health Sciences buildings and provided to the computer science
department. Overall a very solid program, and I believe a majority of the professors did the
best they possibly could to prepare us for the job market and higher level education. I have
a previous degree in Economics from the UAA College of Business and Public Policy, and
I did not feel nearly as prepared for the job market when I graduated as I do now.

								
To top