by The Citadel School of Education

The Citadel School of Education Unit and Program Assessment Report Summaries

The Citadel Annual CAEP Reports

The Citadel Annual CAEP Reports

Review and Summary of Unit Assessment Data

Professional Education Unit - Dean's Summaries

2011-2012 (Academic Year) Annual Professional Education Unit Assessment Report

Annual Professional Education Unit Assessment Report

 2011-2012

Submitted by Dean Tony W. Johnson

(Approved Dec. 6, 2012)


I.  Continuing development and Implementation of Assessment System for The Citadel’s Professional Education Unit.

During the 2011-2012, the unit continued to use, monitor, and refine our assessment system.  Initiatives for the 2011-2012 academic years included:

·Continued using LiveText to compile, aggregate, and disaggregate program and unit data on key common assessments for initial and advanced professional education programs;

·Focused on expanding and revising candidate portfolios as a significant curriculum component for all programs and as an authentic assessment for both program and unit evaluation; 

·Earned national recognition from IRA, NCSS, NCTM, ELCC, and NASP. Submitted response to conditions to NCTE, NCSS, and NASPE on March 15, 2012;   

·Program Coordinators met regularly with field placement and NCATE coordinator to address curricular issues such as meeting the needs of special needs students (including English Language Learners), creating non-threatening school and classroom environments;  ensuring that candidates engage in reflective behavior to promote rapport with students, parents, and the community; and ensuring that candidate field placements are of the highest possible quality; 

·Planned and conducted fall and spring semester faculty retreats and special meetings to foster continuous improvement of specific programs and the unit as a whole;

·Aligned revised conceptual framework reducing ambiguity, achieving greater clarity, and embracing a more positive approach to appropriate assessment instruments;

·Implemented Assessment Committee (1 of 7 constituent committees) recommendations for achieving target level compliance with NCATE standard;  

·Developed and submitted Institutional Report and electronic exhibit center for review by external Board of Examiners;

·Approved revisions of School of Education Faculty Handbook, Professional Education Unit’s Assessment Handbook, Field Experience Handbook, and program handbooks for Educational Leadership,  Counselor Education, Teacher Education, and Literacy; and

·Initiated preparation for CACREP visit and review in the spring of 2013.

II.   Analysis of Data

A.    Citadel PRAXIS II Data from 09/01/11 to 8/31/12

The following data is based on the scores received from the Citadel Graduate College by the Office of Internships and Field Experiences and indicates candidates enrolled in our programs.  It is not Title II graduation rate data.  The unofficial data for the initial and advanced programs from 09/01/11 to 08/31/12 is as follows:

Test

MAT Number
Passed/Total
Number Taken

MAT
% Pass Rate

CADET/ Undergraduate
Number Passed/  Total Number Taken

CADET
% Pass Rate

MATHEMATICS

 

 

 

 

0061

4/4

100%

0/0

NA

0063

4/4

100%

0/0

NA

BIOLOGY

       

0030

4/4

100%

2/2

100%

ENGLISH

       

0044 (NEW)

5/6

83.3%

0/0

NA

PHYSICAL EDUCATION

       

0095  (NEW)

7/8

100%

2/2

100%

SOCIAL STUDIES

       

0086 (NEW)

9/12

75%

2/4

50%

0081

3/4

75%

4/6

66.6%

0083

1/1

100%

6 /7

85.7%

         

 

Test

Passed/

Taken

% Pass Rate

Initial Teacher Preparation Programs

 

 

0522/0622  PLT K-6

 

 

MAT

5/5

100%

Undergraduate

0

NA

0523/0623  PLT 5-9

   

MAT

1/1

100%

Undergraduate

1/1

100%

0524/0624  PLT 7-12

   

MAT

14/14

100%

Undergraduate

6/8

75%

Advanced Programs

Passed/

Taken

% Pass Rate

0411 - Educational Leadership: Administration and Supervision

23/25

92%

0420 - School Guidance and Counseling

19/20

95%

0200 - Introduction to the Teaching of Reading

4/4

100%

 

The Citadel requires that all cadet teacher education candidates pass all relevant PRAXIS exams to graduate and that all MAT candidates pass the content exams prior to the internship and the PLT exam prior to graduation.  All exams required for South Carolina educator certification must be passed prior to posting of degree, resulting in a 100% pass rate for Title II reporting.  *Please note that several candidates take the exam more than once in order to receive a passing score.

B.   Unit Assessment Data from LiveText

1.    Discussion of Portfolio Assessment.  The unit portfolio assessment measures content knowledge and student learning and is administered in both initial and advanced programs during fall, spring, and summer semesters.  The response rates for the fall included 39 candidates and 102 faculty; 25 candidates and 154 faculty for spring 2012; and 29 candidates and 42 faculty for the summer 2012.  Using a three point scale (3=target; 2=acceptable; and 1=unacceptable), candidates were assessed on two content knowledge competencies and four student learning competencies.  Mean scores for each of the six competencies ranged from 2.7 to 2.84.

 

In developing a plan for meeting the target level for NCATE Standard 2, the Professional Education Unit concluded that the portfolio assessment completed during the capstone internship experience was an excellent and comprehensive means to measure candidate competency and program effectiveness.  Based on an increased emphasis of the portfolio, the unit decided to continue using the current assessment during 2011-2012 while substantially revising and expanding the rubric to be used beginning in the fall of 2012.

 

2.    Discussion of Professional Dispositions Assessment.  All initial and advanced professional education programs administered professional disposition assessments during fall 2011 and spring and summer 2012.  Candidates, faculty, and site supervisors/cooperating teachers rated candidates on six items using a three point scale.  This assessment yielded 151 midterm responses (59 candidate, 60 SOE faculty, and 32 cooperating teachers) and 972 end of term responses (384 candidate, 513 SOE faculty, and 75 cooperating teachers).  Mean scores for the six items for fall semester ranged at midterm from 2.75 to 2.90 and end of term from 2.82 to 2.96; for the spring semester scores ranged from 2.73 to 2.88 at midterm and from 2.83 to 2.93 at end of term.  End of term scores for the summer ranged from 2.75 to 2.84.
 

As a unit assessment, data indicates that candidates in our programs consistently meet or exceed expectations of their field on each dispositional item.  On an individual level, the disposition assessment is used to assist in determining if a candidate needs remediation in the relevant dispositional area or if he/she should be removed from the program.

 

To enhance the reliability of the disposition assessment as a unit assessment, dispositions are rated by college faculty, site supervisors/cooperating teachers, and by the candidate at several transition points.  The opportunity for use of this assessment by different evaluators increases the reliability of the instrument.

 

3.    Discussion of Research Competencies Assessment.   The assessment was administered in EDUC 512 and 549 research courses during the fall 2011 semester and during the 2012, spring and summer semesters.  A total of 94 candidates were evaluated by the faculty member teaching the research courses required of all graduate candidates in the Professional Education Unit (both School of Education candidates and physical education teacher education candidates).  All candidates were assessed using a three point scale on five different competencies.  Mean scores for the fall 2011 semester ranged from a low of 2.51 on the “candidate interprets descriptive and inferential data” assessment to a high of 2.84 on the remaining four items.  The Professional
Education Unit continued to use the LiveText rubric implemented in the spring of 2011 in the Research Competency Assessment.  Mean scores were better in the 2012 spring and summer administration ranging from a low of 2.75 “candidate demonstrates use of statistical procedures” to 3.00 on the  four remaining items assessed.  

In response to weaknesses identified in previous assessments, faculty created an assessment to measure candidates’ knowledge of basic mathematical and statistical concepts prior to enrolling in EDUC 512.  Based on this data, faculty modified instruction in the course to better meet the needs of candidates. In spite of this effort, mean scores on items requiring basic mathematical and statistical knowledge continue to be low.  Recognizing that EDUC 512 is a crucial course for assisting candidates in developing these essential skills, the assessment committee recommends that candidates must take EDUC 512  within the first 18 hours of graduate work and that candidates’ research skills must be assessed again in a content area course.
 

4.    Discussion of Program Exit Survey.  A total of 22 initial and 84 advanced candidates completed the survey during fall 2011 and spring and summer 2012.  The survey focused on the knowledge, skills, and dispositions program completes acquired during their Citadel experience and their satisfaction with other aspects of their programs.  Completers responded to question using a five-point scale that ranges from very dissatisfied (1) to very satisfied (5).

Scores on the 19 questions addressing the satisfaction of candidates in initial programs satisfaction with the knowledge, skills, and dispositions acquired through their program ranged from 4.43 to 4.86 during fall semester 2011 and 4.07 to 4.87 in the spring 2012 semester.  Scores on other aspects of the program – advising, internship placements and quality of instruction – ranged from 4.17 to 4.86 during the fall 2011 semester and from 4.13 to 4.79 during spring semester 2012.  The mean score for the overall quality of initial programs was 4.57 for fall semester 2011 and 4.62 for spring semester 2012.

Responses to the questions for the advanced programs indicate a similar positive experience.  Mean scores ranged from fall semester – 4.60 to 5.00 and spring/summer semester – 4.61 to 4.93.  The overall quality rating for advanced programs was fall – 4.70 and spring/summer – 4.55/4.83.

Initial and advanced program completers are clearly satisfied with the professional knowledge and skills that they acquired during their experience. Program completers of both initial and advanced programs responded to the statement:  “development of a principled educational leader who is knowledgeable, reflective, and ethical” with a consistently high rating. Since preparing principled leaders is the cornerstone of a Citadel education, this is a significant indicator of the quality of the Unit’s mission of preparing principled educational leaders.

 

Completer expressed dissatisfaction with the “availability of technology” in our programs.  The School of Education has upgraded our instructional technology capability in the past few years, but—as school districts continue to employ new technologies such as individual iPads for students—it is a constantly moving target.  We are continually exploring ways and means of enhancing our candidates’ abilities to effectively use instructional technology.

 

5.    Discussion of Graduate and Employer Follow-Up Surveys.  The employer survey for 2010-2011 program completers was administered using LiveText.  The first fifteen items are common for all programs and focus on the knowledge, skills, and dispositions acquired during the candidates’ Citadel experience.  Additional sections contain program specific items.

Twenty-nine of the 71 employer surveys were completed resulting in a return rate of 40.8%.  Forty-nine of 103 first year graduate surveys were completed resulting in a return rate of 47.6%.  Using a 5.0 point scale, the mean scores for the 15 common items ranged from 4.42 to 4.72.  Mean scores for  first year graduates ranged from 4.31 to 4.78.  Employer mean scores ranged from 4.45 to 4.83.
 

Significantly more surveys were completed and returned than in 2010-201, due perhaps to the personalized reminders sent to employers and candidates encouraging them to complete the survey.  Since this is the third year of disseminating the survey in an electronic format, responders’ familiarity with this process may have contributed to a higher return rate.

 

During the 2010-2011 academic year, the unit implemented program changes designed to provide candidates with more preparation in the area associated with the “Adapts Instruction” assessment item.  2012 results suggest that these changes have been effective since the current data no longer identifies this item as needing improvement.

 

Throughout program coursework, the Unit requires candidates to use technology effectively in instruction.  The lower ranking on these surveys of such items as “integrate the use of technology in his/her work” and “availability of technology resources” suggests a need for dialogue with the schools to determine what areas candidates may need additional work.

III.   Assessment for Change

Based on a continuous review of the data obtained through the assessment process, faculty in each program area implement curricular and policy changes aimed at continuously improving professional education programs at The Citadel.  Included among these changes are an expansion and refinement of our field placements and internships to ensure that Citadel professional education candidates experience both “best practices” and the diversity that characterizes the current educational environment.  In addition, the counseling program has added an ethics course to better meet the needs of counseling students.  In a similar fashion, The Citadel masters program in literacy has changed and updated textbooks and related course materials  and incorporated more research based assignments in literacy in response to candidates’ criticisms of the program.  As these and the additional changes indicate, The Citadel Professional Education Unit uses data to continuously improve our programs.  As part of our efforts at continuous improvement program faculty are considering the following responses to areas of concern derived from the assessment data.

Teacher Education:  A major focus for all programs during the 2011/2012 academic year was the revision of candidate teacher work samples, including changes to lesson plans and program completer portfolios.  Program faculty are responding to assessment results summarized below:

·         Data from fall 2011 assessments suggest that the modifications to the exceptionalities courses—including requiring an accommodations notebook—is producing higher candidate ratings;

·         Assessment data from the spring 2012 assessments suggest that methods courses and the internship need to focus more on enabling candidates to use instructional time more efficiently;

·         Data from the spring 2012 semester did not reflect the additional strategies employed to address classroom management concerns. Program faculty realize the need to continue monitoring candidates’ skill in this area;

·         Based on the data from the spring 2012 semester, additional training is needed to ensure that candidates are able to use instructional technology effectively and that they are capable of engaging parents to enhancing student learning; and

·         Assessment data from 2011/2012 suggest the need for more cultural awareness activities.

Literacy Education:  As noted in the 2010/2011 annual report, substantial changes were implemented to the literacy program in response to criticisms obtained through the assessment process.  These changes—updating the text material and incorporating more research into the program—have been effectively implemented.  As a result, the data from the past three years (2009-2010, 2010-2011, and 2011-2012) is largely positive.  Curricular changes have been made in recent years, but these are in response to state mandates and are not the result of data obtained through the assessment system.  Both tenure track and adjunct faculty indicate that reviewing the data is very valuable for it enables them to make subtle but significant changes in the courses and program to address the concerns of candidates and to ensure program improvement.

Educational Leadership:  Data obtained through the assessment process indicates that program candidates are developing the knowledge, skills, and professional dispositions of a principled educational leader, but—in response to weaknesses revealed through the assessment process--program faculty are taking action to correct weaknesses:

·         Since candidates experience difficulty in defining the problems and selecting a strategy for correcting the problem, program faculty are working with the instructor of the course in educational research to ensure that candidates have additional opportunities to interpret and synthesize data and to better define the problem; and

·         In a similar fashion, program area faculty are modifying instructional strategies to better assist candidates in using data to identify curricular or instructional problems and in taking the next steps for addressing the problem to enhance student learning.

Counselor Education:  in response to lower scores on assessment data, program faculty are considering:

·         Developing requirements for interns to identify list of community resources and to participate in a community service project;

·         Developing a consultation course rather than assuming that consultation skills are being infused in existing courses;

·         Developing a course in “Classroom and Behavior Management; and

·         Requiring a graduate level leadership project building upon The Citadel Undergraduate Leadership Program and the new counselor education  cadet course “Developing Leadership Skills Through Peer Counseling.”

IV.   General Recommendation

Our assessment is, in many ways, exemplary.  While refinement of this process and system is an on-going enterprise, the development of a viable and sustainable system is a significant accomplishment.  In reviewing our programs and policy while implementing an action plan for the upcoming NCATE review and visit, The Citadel Professional Education Unit selected Standard 2—the assessment system—as our choice for achieving compliance at the “target” level. During this past year and as part of a larger review process, an institutional wide assessment committee reviewed the unit’s assessment system and developed recommendations to ensure compliance at the target level for NCATE Standard 2.  The unit approved these recommendation in the spring of 2012 and implementation is currently underway.  The following recommendations are in various stages of implementation:

·         A visual model of the Assessment System illustrating the Unit’s relationship with program areas, the College, and the P-20 community has been created;

·         The Unit has established multiple assessment points of candidates’ content knowledge, including a baseline  assessment point of candidates’ content knowledge and additional assessment point(s) to measure improvements in candidates’ content knowledge;

·         To ensure inter-rater reliability, multiple raters assess the content knowledge, student learning, and portfolio unit assessments of candidates in both initial and advanced programs;

·         Assessment of content knowledge has been integrated into each candidate’s portfolio;

·         LiveText rubrics are now used for Unit and program assessments facilitating greater disaggregation by gender and ethnicity;

·         Dispositions Assessments are now aligned with the Unit’s Conceptual Framework, and with SPA and NCATE standards;

·         Candidates are now required to take EDUC 512 or EDUC 549 within the first 18 credit hours;

·         Multiple instructors are now required to rate the level of each candidate’s  level of Research Competency; and

·         LiveText rubrics have been developed to assess candidates’ level of Research Competency.

While these changes continue to be implemented, the Unit has submitted the Institutional Report and electronic exhibits to NCATE, and the specialty area reports to the appropriate SPAs. These reports and exhibits document compliance with NCATE and state standards and are part of the formal NCATE review process culminating in a visit scheduled for February 24-26, 2013.

*While the program in School Psychology does not participate in the unit’s assessment process, data from their NASP approved program is shared with the unit and is aligned with unit processes and standards.

Attachments

Annual Professional Education Unit Assessment Report - 2010-2011 Academic Year

Annual Professional Education Unit Assessment Report

 2010-2011

Submitted by Dean Tony W. Johnson

(Approved Dec. 8, 2011)


I.  Continuing development and Implementation of Assessment System for The Citadel’s Professional Education Unit.

During the 2010-2011, the unit continued to use, monitor, and refine our assessment system.  Initiatives for the 2010-2011 academic years include:

  • Continuing the use of LiveText to compile, aggregate, and disaggregate program and unit data on key common assessments for initial and advanced professional education programs;
  • Focusing on the expansion and revision of candidate portfolios a significant curriculum component for all programs and as an authentic assessment for both program and unit evaluation; 
  • Preparing and submitting SPA reports on March 15, 2011.  Reports to IRA, NCSS, NCTM, NCTE, NSTA, ELCC, and NASPE were submitted on March 15th.  NASP report submitted on September 15th;
  • Regularly scheduled meetings of the Program Coordinators  continued to focus on curricular issues such as better meeting the needs of special needs students (including English Language Learners), fostering a non-threatening school and classroom environment;  ensuring that candidates engage in reflective behavior to promote rapport with student, parents, and the community; and ensuring that candidate field placements are of the highest possible quality; 
  • Conducting faculty retreats and special meetings designed to foster continuous improvement of specific programs and the unit as a whole;
  • Establishing NCATE Steering Committee to oversee preparation of upcoming NCATE review and visit (spring 2013);
  • Establishing constituent sub-committees of NCATE Steering Committee focused on the Six NCATE Standards and the unit’s conceptual framework to determine and document compliance with relevant standards;
  • Modifying extant conceptual framework to reduce ambiguity and achieve greater clarity as well as embracing a more positive approach;  and
  • Charging Assessment Committee (1 of 7 constituent committees) to review our extant assessment system and to recommend changes to achieve target level of compliance with NCATE standard. 

II.   Analysis of Data

A.    Citadel PRAXIS II Data from 09/01/10 to 8/31/11

The following data is based on the scores received from the Citadel Graduate College by the Office of Internships and Field Experiences and indicates candidates enrolled in our programs.  It is not Title II graduation rate data.  The unofficial data for the initial and advanced programs from 09/01/10 to 08/31/11 is as follows:

 

 

MAT Number
Passed/Total
Number Taken

MAT
% Pass Rate

CADET/ Undergraduate
Number Passed/Total
Number Taken

CADET
% Pass Rate

MATHEMATICS

 

 

 

 

0061

0/0

NA

0/0

NA

0063

0/0

NA

0/0

NA

BIOLOGY

 

 

 

 

0030

2 / 2

100%

0/0

NA

ENGLISH

 

 

 

 

0041

4 / 4

100%

0/0

NA

0042

2 / 4

100%

0/0

NA

PHYSICAL EDUCATION

 

 

 

 

 091

8 / 2

100%

2 / 2

100%

 093

7 / 8

87.5%

2 / 2

100%

SOCIAL STUDIES

 

 

 

 

 0081

11 / 16

68.7%

4 / 6

66.6% *

 0083

12 / 13

92.3%

6  / 7

85.7%

         

 

Test

Passed/Taken

% Pass Rate

Initial Teacher Preparation Programs

 

 

0522  PLT K-6

 

 

MAT

4 / 4

100%

Undergraduate

0 / 0

NA

0523  PLT 5-9

 

 

MAT

0 / 0

NA

Undergraduate

0 / 0

NA

0524  PLT 7-12

 

 

MAT

25 / 25

100%

Undergraduate

8 / 8

100%

Advanced Programs

Passed/Taken

% Pass Rate

0410 - Educational Leadership: Administration and Supervision

39 / 39

100%

0420 - School Guidance and Counseling

19 / 19

100%

0200 - Introduction to the Teaching of Reading

24 / 24

100%

The Citadel requires all cadet teacher education candidates pass the South Carolina required Praxis exams prior to graduation and all MAT candidates pass the content exams prior to the internship and the PLT exam prior to graduation.  These requirements result in a 100% pass rate.  *Please note that several candidates take the exam more than once in order to receive a passing score and may not pass during the year identified above.

B.     Unit Assessment Data from LiveText

  1. Discussion of Content Knowledge, Student Learning, and Portfolio Assessment.  The Content Knowledge and Student Learning  Assessment is part of the unit portfolio and is administered in both initial and advanced programs during fall, spring and summer semesters.  The response rates for the assessments were fall 2010 – 30 candidates and 91 faculty, spring 2011 – 32 candidates and 104 faculty and summer 2011 – 36 candidates and 44 faculty.  Using a three point scale (3=target; 2=acceptable; and 1=unacceptable), candidates were assessed on two Content Knowledge competencies and four Student Learning competencies. 

    Mean scores for each of the six competencies ranged from 2.63 to 2.86.  Ratings were lower during all 2010-2011 semesters than previous semesters.  The lower ratings may be attributable to all programs employing the assessments for these six competencies into the portfolio assessment as a unit assessment.  The assessment items “positive environments for student learning” and “understanding diversity of students, families, and communities” continue to receive the highest ratings.
     
  2. Discussion of Professional Dispositions Assessment. All initial and advanced professional education programs* administered professional disposition assessments in 2010-2011 fall, spring, and summer.  Candidates, faculty, and cooperating teachers rated candidates on six items using a three point scale.  This assessment yielded 139 midterm responses (47 candidate, 77 SOE faculty, and15 cooperating teachers) and 1,072 end of term responses (410 candidate, 547 SOE faculty, and 92 cooperating teachers).

    Indicating candidate growth and development over the course of the semester, the mean score for each of the six dispositions are significantly higher on the end of term administration of the assessment than on the midterm administration of the same assessment.  Cooperating teachers and candidate evaluations are similar for each of the six items on the end of term assessment while the end of term rating of School of Education and Physical Education faculty is significantly more critical.  Consistent with previous administration of this assessment, SOE faculty generally rated candidates’ dispositions lower than the cooperating teacher and the candidate.  As is the case in previous years, overall ratings are higher in the spring semester.  It is interesting to note that candidates tend to rate themselves lower on the end of term assessments than on the beginning of term assessment, suggesting a growing awareness that they are—at best—novice professionals.
     
  3. Discussion of Research Competencies Assessment.   The assessment was administered in EDUC 512 and 549 research courses during fall 2010, spring and summer semesters 2011.  A total of 153 candidates were evaluated by the faculty member teaching the research courses required of all graduate candidates in SOE and MAT physical education candidates.  Candidates were assessed using a three point scale on five different competencies.  Fall semester mean scores ranged from 2.32 (candidate interprets descriptive and inferential data) to a high of 2.44 (cites reference using correct APA format).  Overall, mean scores were on par with spring 2010.

The Research Competency Assessment was converted from a form format to a rubric in spring semester.  Mean scores were better in the spring and summer administration of this instrument ranging from 2.22 (candidate interprets descriptive and inferential data) to 2.95 for three of five items assessed.   Ratings were substantially high in four of the five items of the spring and summer assessments.

Even though faculty created an assessment to measure candidates’ knowledge of basic mathematical and statistical concepts prior to enrolling in EDUC 512 and modified by the instruction in the course to better meet the needs of candidates, the mean scores on these items continue to be low.  In recognition of the candidates performance on such items the assessment committee recommends that—since EDUC 512 is a baseline course for such basic mathematical and research skills—candidates should take the course within the first 18 hours of graduate work and that candidates research skills should be assessed again in a content area course.

  1.  Discussion of Program Exit Survey.   A total of 41 initial and 32 advanced candidates completed the survey in fall, spring, and summer semester 2010-2011.  The survey focused on completers’ satisfaction with knowledge, skills, and dispositions acquired during their Citadel experience and their satisfaction with other aspects of their programs.  Using a five-point scale, completers responded that they were either satisfied or very satisfied with all aspects of their Citadel experience.

    The ratings on the 19 questions addressing the satisfaction of candidates in initial programs with the knowledge, skills, and dispositions acquired through their program ranged from fall semester – 4.47 to 4.95 and spring semester – 4.48 to 4.86.  Other aspects of the program (advising, internship placements, and quality of instruction) ranged from fall semester – 4.22 to 4.89 and spring semester – 4.41 to 4.85.  The mean score for the overall quality of initial programs was fall – 4.47 and spring – 4.64.

    Responses to the questions for the advanced programs indicate a similar positive experience.  Mean scores ranged from fall semester – 3.25 to 5.00 and spring/summer semester – 4.19 to 4.93.  The overall quality rating for advanced programs was fall – 4.25 and spring/summer – 4.78.

    Candidates at both the initial and advanced rated the overall quality of their program very high (4.64 and 4.78 on a 5.0 point scale).  Support by college supervisor and cooperating teacher during the internship received the highest ratings for the initial programs.  Both initial and advanced program completers rated the internship experience very high.  “Availability of courses” earned the lowest ratings for both initial and advanced programs.
     
  2. Discussion of Graduate and Employer Follow-Up Surveys.  The 2011 survey for 2009-2010 graduates was again administered using LiveText.  The first fifteen items were common for all programs and focused on the knowledge, skills, and dispositions acquired through the experience.  Subsequent sections contained program specific items.

    Twenty-two of 68 emailed employer surveys were completed resulting in a return rate of 32.4%.  Thirty-one of 119 emailed first year graduates surveys were returned resulting in a return rate of 26.0%.  Using a 5.0 point scale, the mean scores for the 15 common items ranged from 4.20 to 4.68.  Mean scores from first year graduates ranged from 4.0 to 4.68 and employer mean scores ranged from 4.32 to 4.77.

    Lowest responses from graduates included the integration of technology in their work, their ability to adapt instruction and assessment to meet the needs of all students—including those with special needs—and their ability to appropriately apply knowledge of students’ developmental levels.  While employer responses tended to be more positive, both graduates and employers rated “Adapts instruction, assessment, or services for all students, including students with exceptionalities” lower than other items.  This lower rating is similar to the first year administration of this survey.  Clearly, unit faculty need to examine course syllabi and other program materials in response to these lower ratings.

III.           Assessment for Change

Based on a continuous review of the data obtained through the assessment process described above, faculty in each program areas implement curricular and policy changes aimed at continuously improving professional education programs at The Citadel.  Included among these changes are an expansion and refinement of our field placements and internships to ensure that Citadel professional education candidates experience both “best practices” and the diversity that characterizes the current educational environment.  In addition to changes in our internship placements for both teacher education and educational leadership programs based on assessment data, the educational counseling program is adding an ethics course to its curriculum to better meet the needs of counseling students.  In a similar fashion, The Citadel masters program in literacy has changed and updated textbooks and related course materials  and incorporated more research based assignments in literacy in response to candidates’ criticisms of the program.  As these and the additional changes identified below indicate, The Citadel Professional Education Unit uses data to continuously improve our programs.  Changes anticipated or underway for the 2010-2011 academic year include:

Teacher Education:

  • Focus on improving candidates classroom management skills by requiring “Shouting Won’t Grow Dendrites” in EDUC 202;  requiring Classroom Management Reflection Project in EDUC 536; and instituting classroom management plans as an assignment for EDUC 401/501;
  • Promoting candidates’ professional dispositions and skills by developing a consistent process for evaluating late assignment;
  • Fostering candidates research skills and understanding by requiring students to complete The Citadel Institutional Review Board Process;
  • Enhancing the instructional technology skills of candidates by exposing them to school based technology experts and by ensuring that all Citadel faculty have access to SmartBoard websites; and
  • Modifying the exceptionality courses (EDUC 312 and 514) to include a notebook assignment requiring candidates to interact directly with exceptional students and develop and implement appropriate instructional strategies. 

 Literacy Education:

As noted earlier, substantial changes were made in previous years in response to criticisms obtained through the assessment process.  These changes—updating the text material and incorporating more research into the program—has been effectively implemented.  As a result, the data from the past two years (2009-2010 and 2010-2011) is largely positive.  While changes in the curriculum are underway, these are in response to state mandates and are not the result of data obtained through the assessment system.  Both tenure track and adjunct faculty indicate that reviewing the data is very valuable for it enables them to make changes in the courses and program to address the concerns of candidates and to ensure program improvement.

Educational Leadership:

Data obtained through the assessment process indicates that program candidates are developing the knowledge, skills, and professional dispositions of a principled educational leader, but—reviewing the assessment data—program faculty have identified problem areas and are committed to modifying course syllabi aimed at improving the following areas:

  • Empowering candidates to develop positive school climates;
  • Developing candidates’ ability to apply reflective practices;
  • Enhancing candidates’ skill in creating a vision for learning;
  • Providing candidates with authentic P-12  clinical experiences;
  • Enabling candidates to foster rapport with students, families, and colleagues; and
  • Improving candidates’ ability to base decisions on quantitative and qualitative data. 

Counselor Education:

Data obtained through the assessment process indicates that both program completers and employers view The Citadel’s Counselor Education program as “excellent” or “very good.”  In terms of continuous improvement, data also suggest the need for site supervisors to involve practicum/internship students in parent/teacher conferences.  In response to concerns raised in both completer and employer surveys, a new course in Legal and Ethical Leadership Issue in Education has been added to the curriculum and other additional coursework in “Classroom and Behavior Management” and “Consultation” is being considered. 

IV.           General Recommendation

Our assessment is, in many ways, exemplary.  While refinement of this process and system is an on-going enterprise, the development of a viable and sustainable system is a significant accomplishment.  In reviewing our programs and policy while developing an action plan for the upcoming NCATE review and visit, The Citadel Professional Education Unit selected Standard 2—the assessment system—as our choice from among the six NCATE standards for achieving compliance at the “target” level.  In pursuing this goal and as part of a larger review process, an institutional wide assessment committee conducted a comprehensive review of the unit assessment system during the spring 2011 semester.  Based on this review, the committee developed the following unit and program recommendations:

  • Create a visual model of the Assessment System that illustrates the Unit’s relationship with program areas, the College, and the P-20 community;
  • Identity multiple assessment points of candidates’ content knowledge by identifying an initial assessment point to establish a baseline level of candidates’ content knowledge and identifying an additional assessment point(s) to measure changes in candidates’ content knowledge;
  • To achieve inter-rater reliability, require multiple raters for the content knowledge, student learning, and portfolio unit assessments;
  • Integrate all assessments of candidates’ content knowledge into the portfolio;
  • Use LiveText rubrics for Unit and program assessments in order to achieve greater disaggregation by gender and ethnicity;
  • Align Dispositions Assessment Instrument with the Conceptual Framework, and the SPA and NCATE standards;
  • Require candidates to take EDUC 512 or EDUC 549 within the first 18 credit hours;
  • Require more than one instructor to rate candidates’ level of Research Competency;
  • Create a LiveText rubric to rate candidates’ level of Research Competency; and
  • Additional program assessments to ensure that programs are in compliance with the “target” rubric for NCATE Standard 2.

Implementation of these recommendations began during the fall 2011 semester and is ongoing. 

*While the program in School Psychology does not participate in the unit’s assessment process, data from their NASP approved program is shared with the unit and is aligned with unit processes and standards.

Attachments

Annual Professional Education Unit Assessment Report - 2009-2010 Academic Year

Annual Professional Education Unit Assessment Report

 2009-2010

Submitted by Dean Tony W. Johnson

(November 10, 2010)


I.   Continuing development and Implementation of Assessment System for The Citadel’s Professional Education Unit.

During the 2009-2010, the unit continued to use, monitor, and refine our assessment system.  Initiatives for the 2009-2010 academic year include:

  • Continuing the use of LiveText to compile, aggregate, and disaggregate program and unit data on key common assessments for initial and advanced professional education programs;
  • Focusing on the expansion and revision of candidate portfolios s a significant curriculum component for all programs and as an authentic assessment for both program and unit evaluation; 
  • Developing a timeline and process for preparing SPA reports for submission in the spring of 2011;
  • Conducting regularly scheduled meetings of the Program Coordinators  focused on such curricular issues as better meeting the needs of special needs students (including English Language Learners), fostering a non-threatening school and classroom environment;  ensuring that candidates engage in reflective behavior to promote rapport with student, parents, and the community; and ensuring that candidate field placements are of the highest possible quality;  and
  • Conducting faculty retreats and special meetings designed to foster continuous improvement of specific programs and the unit as a whole. 

II.    Analysis of Data

A.    Citadel Praxis II Data from 09/01/09 to 8/31/10. 

The following data is based on the scores received from the Citadel Graduate College by the Office of Internships and Field Experiences and reflects all candidates enrolled in our programs.  This is not Title II graduation rate data.  The unofficial data for teacher education programs from 09/01/09 through 08/31/10 are as follows:

 

A.    Citadel PRAXIS II Data from 09/01/09 to 8/31/10

The following data is based on the scores received from the Citadel Graduate College by the Office of Internships and Field Experiences and reflects any candidate enrolled in our program not Title II graduation rate data.  The unofficial data for the teacher education programs from 09/01/09 to 08/31/10 are as follows:

 

MAT
Number
Passed/Total
Number Taken

MAT
% Pass Rate

CADET/ Undergraduate
Number
Passed/Total
Number Taken

CADET
% Pass Rate

MATHEMATICS

 

 

 

 

0061

3 / 4

75%

None

 

0063

4 / 4

100%

None

 

BIOLOGY

 

 

 

 

0030

2 / 2

100%

None

 

ENGLISH

 

 

 

 

0041

 8 / 8

100%

None

 

0042

8 / 8

100%

1 / 1

 100%

PHYSICAL EDUCATION

 

 

 

 

 091

2 / 2

100%

4 / 4

100%

 093

2 / 2

100%

4 / 4

100%

SOCIAL STUDIES

 

 

 

 

 0081

15 / 20

75%

9 / 9

100%

 0083

13 / 17

76.4%

9  / 9

100%

 

 

 

 

 

 

 

 

 

 

Test

Passed/Taken

% Pass Rate

Initial Teacher Preparation Programs

 

 

0522  PLT K-6

 

 

MAT

3 / 3

100%

Undergraduate

0 / 1

0%

0523  PLT 5-9

 

 

MAT

2 / 2

100%

Undergraduate

1 / 1

100%

0524  PLT 7-12

 

 

MAT

29 / 30

96.6%

Undergraduate

8 / 8

100%

 

 

 

Advanced Programs

Passed/Taken

% Pass Rate

0410 - Educational Leadership: Administration and Supervision

19 / 19

100%

0420 - School Guidance and Counseling

24 / 24

100%

0200 - Introduction to the Teaching of Reading

11 / 12

91.6%

 

The Citadel requirement that all cadet teacher education candidates pass these exams as a graduation requirement and that all MAT students pass these exams prior to the internships result in a 100% pass rate.  Please note that several candidates take the exam more than once in order to receive a passing score.        

B.     Unit Assessment Data from LiveText

  1. Discussion of Content Knowledge, Student Learning, and Portfolio Assessment.  The Content Knowledge and Student Learning  Assessment is part of the unit portfolio and is administered in both initial and advanced programs during fall, spring and summer semesters.   A total of 23 candidate and 75 faculty responses were assessed during the fall 2009 semester with an additional 233 responses (35 candidate and 198) assessed during the 2010 spring and summer sessions.  Using a three point scale (3=target; 2=acceptable; and 1=unacceptable), candidates were assessed on two Content Knowledge competencies and four Student Learning competencies. 

    Mean scores for each of the six competencies ranged from a low of 2.71 to a high of 3.0.  Both content knowledge and student learning competencies were similar to prior semesters with the exception of the midterm data for the spring of 2007.  Overall candidates scored highest on understanding diversity of students, families, and communities which is consistent with previous semesters and with favorable diversity ratings on the completer’s survey.
     
  2. Discussion of Professional Dispositions Assessment. All initial and advanced professional education programs* administeredprofessional disposition assessments in the fall of 2009 and spring of 2010.  Candidates, School of Education faculty, and cooperating teachers rated candidates on six items using a three point scale.  This assessment yielded 113 midterm responses (35 candidate, 73 SOE faculty, and 5 cooperating teachers) and 1,047 end of term responses (414 candidate, 573 SOE faculty, and 60 cooperating teachers).

    As expected, the means for each of the six dispositions are significantly higher on the end of term administration of the assessment than on the midterm administration of the assessment.  Cooperating teachers and candidate evaluations are similar for each of the six items on the end of term assessment while the end of term rating of School of Education faculty is significantly more critical.  Consistent with previous administration of this assessment, SOE faculty generally rated candidates’ disposition lower than the cooperating teacher and the candidate.  As is the case in previous years, overall ratings are higher in the spring semester.
     
  3. Discussion of Research Competencies Assessment.   The assessment was administered in all research courses during the fall semester (2009) and spring and summer semesters (2010).  A total of 126 candidates were evaluated by the faculty member teaching the research courses required of all graduate candidates in the School of Education and MAT physical education students.  Candidates were assessed using a three point scale on five different competencies.  During the fall 2009 semester, mean scores ranged from a low of 2.06 (statistical procedures) to a high of 2.29 (use of technology).  Overall, the scores were the lowest since the fall of 2006 when this assessment was first administered.

    Mean scores were better in the spring and summer administration of this instrument ranging from a low of 2.3 to a high of 2.75.   Ratings for the summer were higher on four of the five competencies measured.  Students in the spring seemed less skilled in the use of inferential statistics.  In both terms candidates expressed difficulty in synthesizing and evaluating inferential statistical techniques.  While all ratings fall within the acceptable range, the mean scores are lower on this assessment than on any other unit assessment.

    To address the problems identified by this assessment, faculty are creating an assessment to measure candidates’ knowledge of basic mathematical and statistical concepts prior to enrolling in EDUC 512.  Based on a review of this data, faculty will modify the instruction in the course to better meet the needs of students.  Other changes include requiring candidates in EDUC 512 to complete the National Institute of Health’s (NIH) “Protecting Human Subjects” online course.  In addition, candidates will be required to complete the application for The Citadel’s Institutional Review Board for Human Subject Research as a course assignment in both EDUC 512 and 549.
     
  4.  Discussion of Program Completers Survey.   A total of 104 initial and advanced candidates completed the survey during 2009 and 2010 (fall, spring, and summer).  The survey focused on completers’ satisfaction with knowledge, skills, and dispositions acquired during their Citadel experience and their satisfaction with other aspects of their programs.  Using a five-point scale, completers responded that they were either satisfied or very satisfied with all aspects of their Citadel experience.                  

    The ratings on the 20 questions addressing the satisfaction of candidates in initial programs with the knowledge, skills, and dispositions acquired through their program ranged from a low of 4.21 to a high of 4.84.  Other aspects of the program (advising, internship placements, and quality of instruction) ranged from a low of 4.18 to a high 4.76.   The mean score for the overall quality of initial programs was between 4.63 and 4.71.

    Responses to the questions for the advanced programs indicate a similar positive experience.  Mean scores ranged from a low of 4.43 to high of 4.94.  The overall quality rating for advanced programs is a 4.64.

    Support by college supervisor and cooperating teacher during the internship received the highest ratings for the initial programs.  Both initial and advanced program completers rated the internship experiences very high.  “Availability of courses” earned the lowest ratings for both initial and advanced programs.
     
  5. Discussion of Graduate and Employer Follow-Up Surveys.  The graduate and employer assessment survey was revised this year and was administered using LiveText.  The first fifteen items were common for all programs and focused on the knowledge, skills, and dispositions acquired through the experience.  Subsequent sections contained program specific items.

    Twenty-two employer surveys were completed of the 68 emailed for a return rate of 32.4%.  Thirty-one surveys of first year graduates were returned of the 119 sent by email for a return rate of 26.0%.  On a 5.0 scale, mean scores for the 15 common items ranged from a low of 4.20 to a high of 4.68.   Mean scores from first year graduates ranged from 4.0 to 4.68 and employer mean scores ranged from 4.32 to 4.77.

    Lowest responses from graduates included the integration of technology in their work, their ability to adapt instruction and assessment to meet the needs of all students—including those with special needs, and their ability to appropriately apply knowledge of students’ developmental levels.   While employer responses tended to be more positive, both graduates and employers rated “Adapts instruction, assessment, or services for all students, including students with exceptionalities” lower than other items. This lower rating is similar to the first year administration of this survey.  Clearly, unit faculty need to examine course syllabi and other program material in response to these lower ratings.

III.           Assessment for Change

Based on a continuous review of the data obtained through the assessment process described above, faculty in each program areas implement curricular and policy changes aimed at continuously improving professional education programs at The Citadel.  Included among these changes are an expansion and refinement of our field placements and internships to ensure that Citadel professional education candidates experience both “best practices” and the diversity that characterizes the current educational environment.  In addition to changes in our internship placements for both teacher education and educational leadership programs based on assessment data, the educational counseling program is adding an ethics course to its curriculum to better meet the needs of counseling students.  In a similar fashion, The Citadel masters program in literacy has changed and updated textbooks and related course materials  and incorporated more research based assignments in literacy in response to candidate criticisms of the program.  As these and the additional changes identified below indicate, The Citadel Professional Education Unit uses data to continuously improve our programs.  Changes anticipated or underway for the 2010-2011 academic year include:

Teacher Education:

  • Focus on improving candidates classroom management skills by requiring “Shouting Won’t Grow Dendrites” in EDUC 202;  requiring Classroom Management Reflection Project in EDUC 536; and instituting classroom management plans as an assignment for EDUC 401/501;
  • Promoting candidates’ professional dispositions and skills by developing a consistent process for evaluating late assignment;
  • Fostering candidates research skills and understanding by requiring students to complete The Citadel Institutional Review Board Process; and
  • Enhancing the instructional technology skills of candidates by exposing them to school based technology experts and by ensuring that all Citadel faculty have access to SMART board websites. 

 Literacy Education:

As noted earlier, substantial changes were made in previous years in response to criticisms obtained through the assessment process.  These changes—updating the text material and incorporating more research into the program—has been effectively implemented.  As a result, the data from this year (2009-2010) is largely positive.  While changes in the curriculum are underway, these are in response to state mandates and are not the result of data obtained through the assessment system. 

Educational Leadership:

Data obtained through the assessment process indicates that program candidates are developing the knowledge, skills, and professional dispositions of a principled educational leader, but program faculty are committed to improving the following areas:

  • Empowering candidates to develop positive school climates;
  • Developing candidates’ ability to apply reflective practices;
  • Enhancing candidates’ skill in creating a vision for learning;
  • Providing candidates with authentic P-12 experiences;
  • Enabling candidates to foster rapport with students, families, and colleagues; and
  • Improving candidates’ ability to base decisions on quantitative and qualitative data. 

Counselor Education:

Data obtained through the assessment process indicates that both program completers and employers view The Citadel’s Educational Counseling program as “excellent” or “very good.”  In terms of continuous improvement, data also suggest the need for site supervisors to involve practicum/internship students in parent/teacher conferences.  In response to concerns raised in both completer and employer surveys, a new course in Legal and Ethical Leadership Issue in Education has been added to the curriculum and other additional coursework in “Classroom and Behavior Management” and “Consultation” is being considered. 

IV.           General Recommendation

Our assessment is, in many ways, exemplary.  While refinement of this process and system is an on-going enterprise, the development of a viable and sustainable system is a significant accomplishment.  In reviewing our programs and policy while developing an action plan for the upcoming NCATE review and visit, it became apparent to me that focusing on our assessment system (standard 2) is our best chance for meeting the new NCATE “target” designation for one or more of the six NCATE standards.     

*While the program in School Psychology does not participate in the unit’s assessment process, data from their NASP approved program is shared with the unit and is aligned with unit processes and standards.

Annual Professional Education Unit Assessment Report - 2008-2009 Academic Year

Annual Professional Education Unit Assessment Report

 2008-2009

Submitted by Dean Tony W. Johnson

(November 10, 2009)


I.   Continuing Development and Implementation of Assessment System for The Citadel’s Professional Education Unit.

            During 2008-2009 the unit continued to use, monitor and refine our assessment system.  Initiatives for the 2008-2009 academic year included:  

  • Continuing the use of LiveText to compile, aggregate, and disaggregate program and unit data on key common assessments for initial and advanced professional education programs;
  • Update and revision of the Professional Education Assessment Handbook comprised of the assessment procedures approved by the Professional Education Board during the previous academic year (2006/2007).  Included in this update is a streamlining of the reporting process focused on a beginning of the year retreat enabling all professional education faculty the opportunity to review trend data over several years and to make changes based on this data;
  • Continuing to keep the Exhibit room current with information, data and reports;
  • Monitoring the functioning of appeals committee for both initial and advanced programs to ensure that processes for considering waivers are fair and efficient;
  • Revising and expanding the program completer, graduate and employer surveys to ensure that data collected through this process is relevant and meaningful;
  • Preparation of assessment reports by each division and/or program area based on an analysis of the data derived from the key unit and program assessments; and
  • Conducting a retreat for all professional education faculty to explore EEDA’s impact upon professional education programs at The Citadel and to incorporate EEDA mandates into our assessment processes.

II.  Analysis of Data

     A. Citadel PRAXIS II Data from 09/01/08 to 8/31/09

           The following data is based on the scores received from the Citadel Graduate College by the Office of Internships and Field Experiences and reflects any candidate enrolled in our program.  This is not Title II graduation rate data.  The NCATE and SPA reports use the Title II data for passing rates.  The School of Education requires all candidates to pass required exams prior to the posting of their degree.  Based on Title II data, SOE reporting data pass rate is 100% for all graduates.   Overall our candidates are doing well on the exams, however, it should be noted that several candidates must take the test more than once in order to receive a passing score. 


The unofficial ETS data for the teacher education programs from 09/01/08 to 08/31/09 are as follows:

 

MAT
Number
Passed/Total
Number Taken

MAT
% Pass
Rate

CADET/ Undergraduate
Number
Passed/Total
Number Taken

CADET
% Pass
Rate

MATHEMATICS

 

 

 

 

0061

1 /1

100%

None

 

0063

1 /1

100%

None

 

BIOLOGY

 

 

 

 

0030

2 / 2

100%

None

 

ENGLISH

 

 

 

 

0041

 8/8

100%

1/1

100%

0042

 8/10

80.0%

None

 

PHYSICAL EDUCATION

 

 

 

 

 091

6/6

100%

4/4

100%

 093

6/6

100%

8/8

100%

SOCIAL STUDIES

 

 

 

 

 0081

12 /12

100%

9 /9

100%

 0083

12 /13

92.3%

9 /9

100%

 

Test

Passed/Taken

% Pass Rate

Initial Teacher Preparation Programs

 

 

0522  PLT K-6

 

 

MAT

2/3

66.6%

Undergraduate

4/4

100%

0523  PLT 5-9

 

 

MAT

5/6

83.3%

Undergraduate

0/1

0%

0524  PLT 7-12

 

 

MAT

26/ 28

92.3%

Undergraduate

18/19

94.7%

Advanced Programs

Passed/Taken

% Pass Rate

0410 - Educational Leadership: Administration and Supervision

38/40

95%

0420 - School Guidance and Counseling

16/17

94.1%

 

 

 

0200 - Introduction to the Teaching of Reading

39 /39

100%

     
B.  Unit Assessment Data from LiveText

1.  Discussion of Content Knowledge and Student Learning Assessment.   The Content Knowledge assessment is part of the unit portfolio and used in advanced level courses in fall 2008 and summer 2009 was administered with a total of 41 responders.  Overall, responders rated candidates with lower scores based on a three-point scale than in spring 2008 or summer 2008 but consistent with scores for fall 2007.  These results seem to reflect that faculty and candidates are applying the assessment in a more critical and honest mode. 

 

The Student learning assessment is assessed as part of the unit portfolio and in advanced level courses during fall 2008 and summer 2009.  39 candidates were assessed with the ratings remaining fairly consistent since fall 2007.  

 

        2.  Discussion of Professional Portfolio Assessment.  All undergraduate and graduate professional education programs * administered a portfolio assessment in fall 2008, spring 2009 and summer 2009 with a total of 321 responders.  A three-point scale was used for six items related to content knowledge and student learning.  Of the six items, the candidates were ranked highest in “The candidate creates positive environments for student learning” with scores ranging from 2.84 – 2.97.  Candidates received the lowest scores in item six, “The candidate understands policy contexts in work setting” ranging from 2.72-2.76.   Overall, ratings continue to remain clearly in the upper acceptable to target range.

 

        3.  Discussion of Professional Dispositions Assessment.   All undergraduate and graduate professional education programs * administered a professional dispositions assessment in fall 2008 and spring 2009.  Candidates, faculty members, and cooperating teachers rated candidates on six items using a three-point scale.  This assessment yielded 1131 end-of-term responses by candidates, faculty members and cooperating teachers.  The mean score for the total responses ranged from 2.78 to 2.95.  The candidates consistently rated high (above 2.90 in a 3.0 point scale) in “The candidate values diversity and exhibits sensitivity to and respect for cultures.”  The lowest ratings were in “The candidate applies reflective practices.” 

        4.   Discussion of Research Competencies Assessment.    The assessment was administered in all research courses during the 2008-2009 academic year.  A total of 117 candidates were evaluated by the faculty member teaching the research courses required of all graduate candidates in the School of Education and MAT physical education. Using a three-point scale, candidates were evaluated on five research competency items. 

Overall summer semester candidates scored higher than fall and spring semesters.  The competency item, “The candidate interprets descriptive and inferential data” scored the lowest during all three semesters and appears lower during the majority of the assessments data since fall 2006.

5.   Discussion of Program Completers Survey.   A total of 100 initial and advanced candidates completed the survey in fall 2008 and spring 2009.  33 initial teacher education candidates and 67 advanced program candidates completed the survey.  The survey focused on completers’ satisfaction with the knowledge, skills, and dispositions acquired during their Citadel experience and their satisfaction with other aspects of their programs such as the quality of advising.  Using a five-point scale, completers responded that they were either satisfied or very satisfied with virtually all aspects of their Citadel experience. 

Initial program completers rated the “ability to use technology in my teaching” as the lowest or second lowest in both fall and spring surveys.  Advanced program completers rated the “advisement provided by program advisor” and “Quality of instruction” among the four lowest ratings for both fall and spring surveys.  Overall ratings ranged from a low of 4.14 to a high of 4.90.

6.  Discussion of Graduate and Employer Follow-Up Surveys 

 

Graduate Follow-Up Survey:   The annual First-Year Graduate Follow-Up Survey was administered    

  electronically during spring 2009 via LiveText for the first time.  The survey went to 2007-2008 initial and

  advanced program graduates. The survey consisted of the same 15 questions aligned with the Conceptual

  Framework for all programs.  Additional questions specific to each program area were included.  The overall

  return rate was 40.9%, a significant increase over the previous year. While the rating responses do not

  demonstrate any discernible trends, overall the graduates rating were lower than ratings of the employers.

 

Employer Follow-Up Survey:   The annual Employer Follow-Up Survey was administered electronically during spring 2009 via LiveText for the first time.  The survey went to employers of initial and advanced program 2007-2008 graduates. The survey consisted of the same 15 questions aligned with the Conceptual Framework for all programs.  Additional questions specific to each program area were included.  The overall return rate was 37%.  This rate is down from the previous year when surveys were mailed out to employers.  Overall responses were relatively high suggesting that employers who responded are very satisfied with the preparation candidates receive at The Citadel.

 

Mean scores on the fifteen statements in section A. Information across the Unit ranged from 4.29 to 4.63.  The graduate responses ranged from a mean score of 4.00 to 4.52 while the employer responses ranged from a mean score of 4.61 to 4.90. 

Range of Mean Scores by Program          Employer Survey                  Graduate Survey

Initial teacher preparation program               4.38 – 5.00                            3.67 – 4.33

Literacy Education                                        4.67 – 5.00                            3.50 – 4.67

Counselor Education                                    4.00 – 4.67                            4.00 – 5.00

Education Leadership                                   4.60 – 4.83                            4.38 – 4.75

 

The highest responses by graduates were:

4.52 – Develops and manages meaningful education experiences that address the needs of all learners with

           respect for their individual and cultural characteristics.

         4.48 – Models professionalism with students, families, colleagues, and communities.

         4.40 – Constructs, fosters, and maintains a learner-centered environment where all learners contribute and

                     are actively engaged.

 

The highest responses by employers were:

          4.90 – Demonstrates knowledge of professional, state, and institutional standards.

          4.90 – Demonstrates a commitment to a safe, supportive learning environment.

 

The lowest responses by graduates were:

          4.00 – Adapts instruction, assessment or services for all students, including students with exceptionalities.

                4.05 – Develops and implements educational programs/classes that are varied, creative and nurturing.

         

The lowest responses by employers were:

          4.61 – Uses current research to inform his/her work.

          4.65 – Adapts instruction, assessment or services for all students, including students with exceptionalities.

          4.67 – Integrates the use of technology in his/her work.

         

The data reveals that the first-year graduates rated themselves lower in all areas when compared to the employer ratings.  Both the graduates and the employers rated “Adapts instruction, assessment or services for all students, including students with exceptionalities” in the lowest response ratings. 

 

III.  Assessment for Change 

        During the retreat in August 2009, faculty in each of the four School of Education divisions reviewed data compiled in the past year and in some cases, the past three years.  Based on this analysis of trend data, faculty in each of the divisions made recommended changes in their respective programs and are in the process of implementing these changes.  Recommended changes include;

Literacy:

        While the overall assessment process appears to be functioning as anticipated for the Division of Literacy with candidates performing well and generally satisfied with the program, candidates did indicate concern over what they characterized as a lack of current research in the program.  Actions are underway to correct this problem including assigning a different faculty member to teach the research and assessment course, using a recently published text for the course, and modifying the content of the course to include recently published assessment data. 

Teacher Education:

                After reviewing trend data from fall 2006 through spring of 2009, faculty in Division of teacher Education are

                implementing the following changes:

·         developing a clearer  timeline for candidates to plan and submit  lesson plans;

·         adding classroom management activities to the content of the course in educational psychology;

·         revising and carefully sequencing the content in EDUC 402 and internship;

·         integrating new instructional technology content (smart boards, twitter, IPod, technology projects, etc.) into teacher education programs;

·         modifying field experiences to include candidate exposure to rural , suburban, and urban schools;

·         meeting with school personnel to strengthen partnership in support of a more focused and challenging clinical component of our teacher education programs;

·         strengthening candidates’ opportunity to work in school settings with exceptional children and youth; and

·         sharing PRAXIS data (including names of candidates) with content faculty to ensure timely advising and interventions when needed.

       Educational Leadership:

Based on trend data compiled over the past three years, faculty in the Division of Educational Leadership are responding to such suggestions as devoting more instructional time to :

·         personnel matters;

·         programs that encompass all grade levels; and

·         the use of data to guide instruction.

 Faculty in the Division of Educational Leadership are participating in a Learning Community process that  

       includes a comprehensive examination of their programs---strengths, weaknesses, gaps, and overlaps—

      designed to improve all courses and programs. 

      Counseling Education:

                Based on a review of the data compile over the past few years, the faculty in Division of Educational    

      counseling have determined that they need to focus on helping candidates:

·         attain self-understanding;

·         acquire research skills;

·         improve their awareness of and skill in dealing with issues of social and cultural diversity; and

·         improve their ability to apply knowledge of group counseling theories to clients in school settings. 

  IV.  General Recommendations

             Our assessment system is, in many ways, exemplary.   While refinement of this process and system is an

      on-going enterprise, the development of a viable and sustainable assessment system is a significant

      accomplishment.  Congratulations and a very special thank you to all of you for this remarkable accomplishment. 

      Please consider my recommendations in light of this tremendous achievement.

·        The recommendation to keep it simple is not new, but it is still relevant.  Especially as we expand our offerings and as we prepare for the next NCATE review, including SPA reports,  it is recommended that each program area focus on 6 to 8 key assessments that both reflect the uniqueness of specific programs and serve as indicators of the values and objectives of  both the program and unit as a whole;

·        A further recommendation is that faculty continue the progress toward developing a more critical perspective in evaluating our candidates and our programs.  As we develop this more critical perspective, the data we generate becomes more meaningful and thus more useful for initiating changes for continuous improvement; 

·        The data collected must be shared in order to guide program development and contribute to our goal of developing principled educational leaders.  As noted above, the retreat is a step in the right direction as yearly and trend data is reviewed by program area faculty.   It is very important that program faculty continue to scrutinize this data for strengths and weaknesses of their respective programs as the first step toward correcting the weaknesses and building upon the strengths;

·        In our desire to make sure that we covered all the bases, it appears that we have developed a more complicated system than is necessary.  With that in mind, each program area needs to focus on the 6 to 8 key assessments required by their respective SPAs and align that data with the state and national standards; and

·        Each program needs to revisit rubrics used in their unit and program assessments to make sure that these rubrics are aligned with SPA and NCATE standards and that the data gained using these rubrics is helpful in determining changes that are needed for continuous improvement. 


* While the program in School Psychology does not participate in the unit’s assessment process, data from their NASP approved program is shared with the unit and is aligned with unit processes and standards. 

Annual Professional Education Unit Assessment Report - 2007-2008 Academic Year

Annual Professional Education Unit Assessment Report

 2007-2008

Submitted by Dean Tony W. Johnson

(October 14, 2008)


I.          Continuing Development and Implementation of Assessment System for The Citadel’s Professional Education Unit.

                        During the fall of 2007, a site visit by NCATE Board of Examiners and representatives from both the South Carolina Department of Education and Commission on Higher Education evaluated our assessment system and its efficacy in monitoring candidate progress through our programs and in facilitating continuous improvement of our professional education programs.  During the spring and summer, our focus shifted toward the refinement and implementation of the unit assessment system for professional education at The Citadel. Initiatives for the 2007-2008 academic year included:  

  • Continuing the use of LiveText to compile, aggregate, and disaggregate program and unit data on key common assessments for initial and advanced professional education programs;
  • Acquainting both faculty and candidates with the “new” LiveText and developing a training strategy and plan for implementing this new version;
  • Establishing functional appeals committees at both the initial and advanced levels to consider student requests for waiver of specific program or unit academic requirements and to recommend action to the unit head;
  • Completion and distribution of a Professional Education Assessment Handbook comprised of the assessment procedures approved by the Professional Education Board during the previous academic year (2006/2007);
  • Organizing an Exhibit room in preparation for the NCATE site visit (09/30/07-10/02/07) highlighting our assessment system and documenting compliance with state and NCATE standards;
  • Hosting a very successful NCATE and State visit resulting in a report by the  Board of Examiners and State team recommending that all standards are now considered met and that all areas for improvement have been corrected;
  • Developing a timeline and plan for sustaining the assessment system now established and for identifying the assessments for collecting the necessary data required for each program’s SPA reports;
  • Monitoring the functioning of appeals committee for both initial and advanced programs to ensure that processes for considering waivers are fair and efficient;
  • Revising and expanding the program completer, graduate and employer surveys to ensure that data collected through this process is relevant and meaningful;
  • Conducting a spring NCATE faculty workshop focusing on strengths and weakness of our assessment system and emphasizing what needs to be done in preparation for submission of the next round of SPA reports;
  • Preparation of assessment reports by each division and/or program area based on an analysis of the data derived from the key unit and program assessments; and
  • Conducting a retreat for all professional education faculty to explore EEDA’s impact upon professional education programs at The Citadel and to incorporate EEDA mandates into our assessment processes.

II.        Analysis of Data

     A. Citadel PRAXIS II Data from 09/01/07 to 8/31/08

            The Citadel Graduate College is the source of the following data.   This data has been compiled by the School of Education’s by the office of internships and field experiences and identifies candidate enrolled in professional education programs at Citadel during this past year.  It is not Title II graduation rate data. Overall our candidates are doing well on these exams.  The new graduation requirement  f a passing grade on relevant PRAXIS II exams for both MAT and cadet  candidates has significantly increased the passing rate.   The only concern is the passing rate for the physical education video exam, but current candidates still have time to achieve a passing score on this exam.   Once the official Annual ETS Institutional Report for the PRAXIS exams for this year (09/01/07 to 08/31/08) is received, each program area will receive a summary sheet detailing how candidates in their area performed on the relevant PRAXIS II exams.  The unofficial data for the professional education programs from 09/01/07 to 08/31/08 are as follows:

 

MAT
Number
Passed/Total
Number Taken

MAT
% Pass
Rate

CADET/ Undergraduate
Number
Passed/Total
Number Taken

CADET
% Pass
Rate

MATHEMATICS

 

 

 

 

0061

1 / 1

100%

None

 

0063

1 / 1

100%

None

 

 

 

 

 

 

BIOLOGY

 

 

 

 

0030

1 / 1

100%

1/1

100%

 

 

 

 

 

ENGLISH

 

 

 

 

0041

 7 / 7

100%

None

 

0042

 6 / 7

85.7%

None

 

 

 

 

 

 

SOCIAL
STUDIES

 

 

 

 

 0081

11 / 13

84.6%

9  / 10

90.0%

 0083

11 / 13

84.6%

10  / 11

90.9%

 

 

 

 

 

PHYSICAL
EDUCATION

 

 

 

 

 091

4 / 4

100%

7 / 7

100%

 093

2 / 3

66.6%

3 / 6

50.0%

Again, based on this unofficial data provided by the office of internships and field experiences, initial teacher education candidates are performing reasonably well on the Principles of Learning and Teaching Test (PLT) required for certification as of 07/01/06.  The pass rates are indicated in the following table:

Test

Passed/Taken

% Pass Rate

Initial Teacher Preparation Programs

 

 

0522  PLT K-6

 

 

MAT

8 / 8

100%

 

 

 

 

 

 

0523  PLT 5-9

 

 

MAT

1  / 1

100%

Undergraduate

None

 

 

 

 

0524  PLT 7-12

 

 

MAT

33 / 33

100%

Undergraduate

12 / 13

92.3%

 

 

 

Advanced Programs

Passed/Taken

% Pass Rate

 

 

 

0410 - Educational Leadership: Administration and Supervision

29 / 29

100%

 

 

 

0420 - School Guidance and Counseling

30 / 30

100%

 

 

 

0400 - School Psychologist

13 / 13

100%

 

 

 

0200 - Introduction to the Teaching of Reading

19 / 19

100%


Candidates in the advanced programs scored very well on the PRAXIS II specialty area exams with a 100% passing rate for all advanced programs.

      B.  Unit Assessment Data from LiveText

                  1.         Discussion of Content Knowledge and Student Learning Assessment.   A rubric (consisting of 6 competencies) assessing the content knowledge of candidates and their ability to use that knowledge to foster student learning  was administered to candidates and their instructors in all programs (except school psychology) during the fall (07), spring (08) and summer (08) sessions. During this academic year, a total of 154 candidates evaluated themselves using this instrument and faculty conducted 281 evaluations of the candidates using this rubric. 

      Using a 3-point scale, candidates evaluated themselves as least competent in their ability to apply their knowledge of content to professional standards through inquiry, critical analysis and synthesis.  The data suggests that candidates are more comfortable with their ability to foster student learning than they are with their ability to understand and apply the underlying concepts of their field in an educational setting.   The data suggests that both candidates and their faculty are reasonably satisfied with the knowledge and skills possessed by candidates.  While the overall mean for both candidate and faculty responses are in the acceptable range, this is a lower overall score than previously.  One interpretation of these lower overall scores is that both faculty and candidates are becoming more comfortable with this process and instruments and are more willing to apply them critically.   

      Based on the Praxis scores discussed above and the data summarized here, it appears that candidates are reasonably knowledgeable of their content area and possess some kill in applying this knowledge to enhance student learning.  The data also suggests that improvement is needed in helping candidates understand the conceptual structures of their field and in applying this knowledge to foster a higher level of learning in students. 

                  2. Discussion of Professional Dispositions Assessment.   Using a rubric that is closely aligned with the ethical indicators emphasized in the unit’s conceptual framework, the teacher education (including HESS) and counseling programs administered this assessment at midterm and at the end of the semester in practicum or internship courses.  Literacy and educational administration programs administered this assessment at the end of the semester.  During the summer 2008 session, the instrument was administered in twelve graduate classes only as end of semester assessment.  

      During the fall and spring semesters, there were 119 mid term responses: 58 candidate responses; 52 faculty responses, and 9 cooperating teacher responses.    It is interesting to note that cadets rate themselves much lower (2.20) on establishing “rapport with student, families, colleagues, community’’ than on any other category.  In contrast, SOE faculty responses indicate that candidates’ weakest areas are:  applying reflective practices (2.17) and establishing rapport with students, parents, etc. (2.17).  On the fall 2007 midterm assessments, SOE faculty rated candidates relatively low, ranging from a low of 2.17 to a high of 2.33 while for the spring 2008 midterm assessments, SOE faculty rated candidates ranging from a low of 2.53 to a high of 2.66.               

      End of year responders on the professional dispositions assessment totaled 1,267 responses: 535 candidate responses; 651 SOE faculty responses; and 81 cooperating teacher responses during the fall (07) spring (08), and summer (08) sessions.  It is interesting to note that the means for all candidates’ responses on all 6 items of this rubric are substantially higher than the means of the responses of the School of Education faculty.  For example, the fall 2007 the School of Education faculty mean score for item 1(reflective practice) is 2.66 while the candidate mean for the same item is 2.93.   The faculty mean for item 5 (values diversity) is 2.69 while the candidate mean is 2.97.   While this difference may or may not be significant, it suggests that School of Education faculty are becoming more comfortable with this assessment instrument and are assessing candidates with a more critical eye.  It is also encouraging to note that the mean score for all 6 items was higher at the end of the semester than the scores at midterm. 

      While the overall scores remain high, it is encouraging to note that faculty appear to be more discriminating in their scoring of the rubrics.  It is heartening to see that end of term scores when compared to scores from the mid-term evaluations of the internships—suggest that growth is occurring during this culminating experience. 

                  3.         Discussion of Research Competencies Assessment.    The rubric designed to assess the research competencies of candidates in the Citadel’s professional education programs was administered to 170 candidates during the fall (07), spring (08), and summer (08) sessions.   On a 3-point scale, the mean scores on the five competencies assessed range from a low of 2.31 (candidate demonstrates use of statistical procedures) to a high of 2.65 (candidate paraphrase information from research articles.   The competencies of “demonstrating use of statistical procedures” and “interpreting descriptive and inferential data” remain the most problematic for students, but comparison of the data from the past indicates that the fall 2007 students were clearly more competent in these areas than students in previous or subsequent semesters.  It appears as if there has been improvement in our students’ research competencies perhaps as the result of the pre-test developed by the instructor and administered early in the semester to identify and target for assistance those students in need of help.

                  4.         Discussion of Program Completers Survey.   During fall (07), spring (08), and summer (08) semesters, a survey –designed to assess the level of satisfaction of candidates completing their program—was administered to a total of 43 teacher education completers (cadet and MAT candidates) and to 105 candidates completing advanced programs in educational administration, literacy, and school counseling. 

All initial (BS and MAT )  candidates completed a survey of 32 common questions focused on their comfort level with the professional knowledge, skills and dispositions provided them by their program, their satisfaction with the operational aspects of the program (advising, course availability, etc.,) plus additional questions specific to their individual programs.

                  Using a 5-point scale ranging from very satisfied (5) to no opinion (1), the responses of the 43 completers were generally very positive.  For completers at the initial level, ratings on questions focused on candidate satisfaction of the knowledge, skills and disposition provided by the program ranged from a low of 4.22 (different teaching and learning styles) to a high of 4.67 (knowledge of dispositions expected of professionals). Responses to questions focused on operational aspects of the program ranged from a low of 4.05 (availability of courses) to a high of 4.65 (library resources). 

                  Advanced program completers expressed even higher overall satisfaction with their Citadel experience.  Responses to questions focused on candidates’ satisfaction with the academic aspects of their program range from a low of 4.59 (ability to adapt instruction for special needs students) to a high of 4.86 (becoming a principled educational leader who is knowledgeable, reflective, and knowledgeable).  Responses to questions focusing on the operational aspects of the program range from a low of 4.38 (course availability) to a high of 4.86 (support provided by school mentor). 

Completers at both the initial and advanced levels expressed concern over courses being available when needed.  Advising by program advisors also appeared as an area of concern by completers in both initial and advance programs. Advanced program completers identified admission procedures into their program as an area of concern.  While the overall reaction to the professional education programs at The Citadel is largely very positive, program faculty are encouraged to reexamine their advising procedures and to carefully attend to the scheduling of courses to address the availability issues identified by both initial and advanced completers. 

5.                  Discussion of Employer Follow-Up Survey  

 During the spring of 2008, The Citadel Professional Education Unit distributed 115 questionnaires to area employers designed to determine their satisfaction with graduates of our professional education programs.   Sixty six surveys were returned for a very positive return rate of 57%.  We received 32 responses from employers of BS and MAT graduates; 13 responses from employers of our counselor education graduates students, 15 responses from employers of educational leadership graduates; and 6 responses from employers of our literacy graduates.  

      On the 19 items in the survey of initial or teacher education graduates, employers rated the following highest:

·        Ability to present content in clear and meaningful ways… (4.73 on a 5 point scale)

·        Ability to use technology …(4.72)

·        Ability to model professional dispositions …(4.71)

·        Understanding the importance of diversity in teaching and learning (4.71)

      Employers rated our teacher education graduates the lowest on the following:

·        Ability to develop and teach lessons that incorporate diversity (4.46)

·        Awareness of different teaching and learning styles… (4.5)

·        Ability to use assessment to develop meaningful learning experiences… (4.53)

·        Ability to use appropriate instructional strategies… (4.55)

   While the overall ratings are relatively high, teacher education faculty are encouraged to examine the less positive ratings and consider changes for improvements based on this input.

      The highest ratings by employers of our school counseling graduates were:

·        Knowledge of field  (4.77)

·        Ability to know students , families, and communities ( 4.69)

·         Ability to understand and use the development level of students…(4.69)

·        Understanding the needs of diverse students…(4.69)

      Lowest ratings by employers of our school counseling graduates were

·        Awareness of different teaching and learning styles… (4.08)

·        Development as a principled educational leader … (4.23)

·        Ability to use technology… (4.23)

      Again, while the overall employer ratings are relatively high, school counseling faculty are encouraged to examine the less positive ratings and consider improvements based on this input.

      Highest ratings by employers of our educational leadership graduates were:

·        Ability to create positive environments for student learning… (4.73)

·        Understanding the needs of diverse students… (4.73)

·        Professional knowledge expected…(4.67)

·        Ability to know students… (4.67)

·        Ability to use current research… (4.67)

·        Ability to understand and build upon the developmental level of students…(4.67)

·        Understanding the policy contexts… (4.67)

·        Understanding the importance of diversity in teaching… (4.67)

      Lowest rating by employers of our educational leadership graduates were:

·        Ability to develop a school climate that values diversity… (4.07)

·        Awareness of different teaching and learning styles… (4.13)

·        Ability to adapt instruction or services for all students… (4.20)

·        Demonstration of dispositions that value fairness and learning…(4.20)

      While these overall employer ratings are relatively high, educational leadership faculty are encourage to examine the less positive ratings and consider improvements based on this input.

      Highest ratings by employers of our literacy education graduates were:

·        Knowledge of content… (4.83)

·        Knowledge of professional, state, and institutional standards ( 4.83)

·        Professional knowledge expected… (4.83)

·         Understanding the policy contexts… (4.83)

·        Under standing the importance of diversity in teaching… (4.83)

      Lowest ratings by employers our literacy education graduates were:

·        Ability to know students …( 4.00)

·        Ability to adapt instruction or services for all students… (4.17)

·        Knowledge of dispositions expected… (4.17)

·        Ability to understand and build upon the developmental  levels of students…(4.17)

·        Ability to model professional dispositions…. (4.17)

·        Demonstration of dispositions that value fairness and learning…. (4.17)

      Literacy faculty are encouraged to examine the less positive ratings and consider improvements based on this input. 

III.   General Recommendations.

                  Our assessment is, in many ways, exemplary.   While refinement of this process and system is an on-going enterprise, the development of a viable and sustainable assessment system is a significant accomplishment.  Congratulations and a very special thank you to all of you for this remarkable accomplishment.  Please consider my recommendations in light of this tremendous achievement.

·        The recommendation to keep it simple is not new, but it is still relevant.  Especially as we expand our offerings and as we prepare for the next round of SPA reviews,  it is recommended that each program area focus on 6 to 8 key assessments that both reflect the uniqueness of specific programs and serve as indicators of the values and objectives of  both the program and unit as a whole;

·        A further recommendation is that faculty continue the progress toward developing a more critical perspective in evaluating our candidates and our programs.  As we develop this more critical perspective, the data we generate becomes more meaningful and thus more useful for initiating changes for continuous improvement; 

·        The data collected must be shared in order to guide program development and contribute to our goal of developing principled educational leaders.  For this reason, it is recommended that program coordinators develop a process for sharing with program faculty the data collected on a yearly, if not semester, basis.  It is recommended that program faculty scrutinize this data for strengths and weaknesses of their respective programs as the first step toward correcting the weaknesses and building upon the strengths;

·        In our desire to make sure that we covered all the bases, it appears that we have developed a more complicated system than is necessary.  With that in mind, and as Dr. Feldmann suggested in our recent visit (August 21, 2008), each program area needs to focus on the 6 to 8 key assessments required by their respective SPAs and align that data with the state and national standards; and

·        Since the theme for this academic year is strategic initiatives and related curricular reforms, program faculty are encouraged to take a hard look at the data compiled in the past year and half and use that data in making program and curricular changes that both improve our programs and simplify our assessment system.

Summer 2008 Professional Education Unit Assessment Report

Professional Education Unit Assessment Report

Summer 2008

 Submitted by Dean Tony Johnson

October 14, 2008


I.               Continuing Development and Implementation of Assessment System for The Citadel’s Professional Education Unit.  During the summer of 2008, the refinement and implementation of the unit assessment system for professional education continued.  Initiatives for the summer  2008 semester include:

·        Continued implementation of the New Generation LiveText System and expansion in the training of faculty, intern supervisors, cooperating teachers, and candidates;

·        Preparation of assessment reports by each division and/or program area based on an analysis of the data derived from the key unit and program assessments;   

·        Monitored the functioning of appeals committee for both initial and advanced programs to ensure that processes for considering waivers are fair and efficient; and

·        Conducting a retreat for all professional education faculty to explore EEDA’s impact upon professional education programs at The Citadel and to further incorporate the EEDA mandates into our assessment processes. 

II.               Analysis of Data

                  A. Citadel PRAXIS II Data.   No additional PRAXIS II data has been collected during the summer. 

          B.  Unit Assessment Data from LiveText

1.         Discussion of Content Knowledge and Student Learning Assessment.      

      A rubric assessing the content knowledge of candidates and their ability to assist student learning was completed by a total of 70 responders, including 36 candidate self- evaluations and 34 faculty evaluations of these candidates.  An additional 6 educational leadership candidates responded to the content knowledge assessment in a separate course.  Overall the candidates’ ratings were at or near the target range for both content knowledge and student learning competencies.  On a three point scale, ratings ranged from a low of 2.67 on the content knowledge question assessing candidates’ ability to apply structures of the field to professional, state, and institutional standards to a high of 2.94 for the candidates’ ability to create positive environments for student learning.   These scores are higher than comparable spring 2008 scores but consistent with the scores of summer 2007.  It appears that both faculty and candidates are satisfied with candidates’ knowledge of their content field and of their ability to enhance student learning.    

2.                  Discussion of Professional Disposition Assessments.

      The rubric used here measures six competencies that are closely aligned with the ethical indicators emphasized in the revised conceptual framework for The Citadel’s professional education unit. During the summer of 2008, data from 276 responders from 12 graduate classes has been compiled.  Responders included: 136 candidates, 133 SOE faculty, and 7 faculty supervisors.  For all responders using a three point scale, scores ranged from a high of 2.96 to a low of 2.86.  Minor but insignificant differences appeared in the ratings by SOE faculty and supervisors and the candidates themselves.   These scores are consistent with those of summer 2007 and the scores from both summer 2007 and 2008 appear to be higher than fall and spring scores, with the differences relatively small.   Overall scores placed the candidates near or at the target level in each of the six competences identified in the professional disposition assessment.   

3.                  Discussion of Research Competencies Assessment

      The rubric designed to assess the research competencies in The Citadel’s professional education programs was administered to 41 graduate students in the summer of 2008.  Mean scores for each of the 5 competencies assessed for all responders was 2.49 on a three point scale.  Mean scores on each of the five competencies were higher than the spring 2008 scores and lower than the fall 2007 scores.  The summer 2008 scores are higher than the summer 2007 scores on the statistical procedures and descriptive and inferential data competencies but lower than the summer 2007 scores on the research and technology competencies.

      Candidates appear to be improving in the areas where weaknesses appeared in the past with the mean scores on item 2 (the candidate interprets descriptive and inferential data) falling between acceptable and target levels.    

4.                  Discussion of Program Completer Survey

      During the summer of 2008, The Citadel Professional Education Unit received 14 responses to the program completer survey administered to candidates in the advanced programs of educational leadership and literacy.    Completers of advanced programs responded to 30 questions concerned with their satisfaction with their program.   Nineteen questions focused on whether their specific program provided the completer with the appropriate knowledge, skills, and dispositions expected of the professional field.  Eleven questions focused more on operational concerns, such as quality of advisement, field/internship placement, quality of instruction, etc.  Additional questions were included addressing issues specific to each field of study. 

Responses to the 19 questions focused on content and professional knowledge suggest that advance programs completers are pleased with their Citadel experience with mean ratings ranging from a high of 4.86 to a low of 4.71 on a five point scale.    Responses to the 11 questions about program satisfaction produced mean scores ranging from a low of 4.14 to a high of 4.86.  Admission procedures to the program received the highest rating followed by accessibility of program advisor and availability of library resources.  The lowest ratings were given to the support provided by internship supervisors (4.14), the quality of instruction (4.21, and course availability (4.29).   While a rating of higher than 4.00 on a 5 point scale is relatively high, the lower ratings in program satisfaction need to be taken seriously.     

   

      III.       General Recommendations. 

      Our assessment system is, in many ways, exemplary.  While we can and should take pride in what we have accomplished, we must continue to refine our system and processes to ensure its sustainability. In our desire to make sure we covered all the bases, we may have developed a more complicated system than is necessary.

      Since it appears that NCATE is now seeking to streamline the process, I suggest we follow their lead and find ways to simplify our system.   As Dr. Feldmann suggested in her recent visit (August 21, 2008), each program area needs to focus on the 6 to 8 key assessments required by their respective SPAs and align that data so as to demonstrate that the unit is meeting state and national standards.   Program coordinators are encouraged to work with Ms. Habhegger this academic year in aligning SPA and program standards and assessments with unit goals and objectives.     

      As suggested in the welcome back document distributed to you in August, a major focus for this academic year will be on strategic planning and related curriculum reform.   With that in mind, it seems reasonable to suggest that program faculty take a hard look at the data compiled in the past year and a half and use that data in making program and curricular changes that both improve our programs and simplify our assessment system. 

      The final recommendation will sound familiar.  Faculty are encouraged to continue the progress toward developing a more critical perspective in evaluating our candidates and programs.  As we become more discriminating in our evaluation of students and programs, the data we generate becomes more meaningful and more useful in fostering continuous improvement of our candidates and programs.    

Spring 2008 Professional Education Unit Assessment Report

Professional Education Unit Assessment Report
Spring 2008

Submitted by Dean Tony Johnson
September 9, 2008


I. Continuing Development and Implementation of the Assessment System for The Citadel's Professional Education Unit. 

During the spring of 2008, the refinement and implementation of the unit assessment system for professional education continues.  Initiatives for the spring 2008 semester include:

  • Implementation of the New Generation LiveText System and expanding the training of faculty, intern supervisors, cooperating teachers, and candidates;
  • Preparation of assessment reports by each division and/or program area based on an analysis of the data derived from the key unit and program assessments;
  • Monitoring the functioning of appeals committee for both initial and advance programs to ensure that processes for considering waivers are fair and efficient;
  • Revising and expanding the program completer, graduate, and employer surveys to ensure that data collected through this process is relevant and meaningful; and
  • Conducting a spring 2008 NCATE faculty workshop focusing on strengths and weaknesses of our assessment system and emphasizing what needs to be done in preparation for submission of the next round of SPA reports.

II. Analysis of Data

A. Citadel PRAXIS II from 09/01/07 to 06/14/08

Based on data compiled by the office of internship and field placements, pass rates for the PRAXIS II and Principles of Learning and Teaching exams are improving. For the relevant PRAXIS II exams, Citadel candidates achieved a 100% pass on the following PRAXIS II tests: Biology and General Science; Mathematics; School Counseling; Educational Leadership; Literacy; and School Psychology.  In addition, a 100% pass rate was achieved by candidates on one of the PRAXIS II tests required for English and Physical Education. Pass rates on the two parts of the social studies PRAXIS II tests were 83.3% and 90% respectively. 100% pass rates were achieved for the grades K-6 and Grades 5-9 Principles of Learning and Teaching Exams with Citadel candidates earning a 97.4% pass rate on the grades 7-12 PLT exam.

B. Unit Assessment Data from LiveText

1.  Discussion of Content Knowledge and Student Learning Assessment

A rubric assessing the content knowledge of candidates and their ability to assist student learning was completed by a total of 190 responders, including 65 candidate self evaluations and 125 faculty evaluations of these candidates. On all six items, the candidates' evaluations were higher than the faculty evaluations, but the difference does not appear to be significant. It appears that both faculty and candidates are satisfied with candidates' knowledge of their content field and of their ability to enhance student learning.

2.  Discussion of Professional Disposition Assessments

The rubric used here measures six competencies that are closely aligned with the ethical indicators emphasized in the revised conceptual framework for The Citadel's professional education unit. The teacher education division continues to administer the disposition assessment instrument both at mid-term and at the end of the semester, but all other programs administer it only at the end of the semester. During the spring of 2008, there were 75 responses at mid-term: 36 from candidates; 34 from faculty, and 5 from cooperating teachers. End of semester responses totaled 505: 214 from candidates; 246 from faculty; and 45 from cooperating teachers.

On a three point scale, 2.53 was the lowest mean score rating. Faculty ratings tended to be lower than either the candidates or the cooperating teachers but the difference is not significant. The competency that had the lowest rating at midterm and end of semester was "establishes rapport with students, families, colleagues, and community." The competency receiving the highest rating among all assessors was "Demonstrates high values and a caring, fair, honest, responsible, and respectful attitude." The only clear trend that emerges from this data is that there appears to be improvement on all competencies from the mid-term to the end of semester.

3.  Discussion of Research Competencies Assessment

The rubric designed to assess the research competencies in The Citadel's professional education programs was administered to 52 graduate students in the spring of 2008. Mean scores for the 5 competencies assessed arranged from a low of 2.31 to a high of 2.44 on a three point scale.  The ranking of these competencies is lower for this semester than in previous semesters (spring and fall of 2007). Scores for educational leadership candidates were much higher than for candidates in other divisions.

Three candidates received unacceptable ratings on item 2: "the candidate interprets descriptive and inferential data." Candidates scored highest on use of technology and ability to cite references using correct APA format. As noted above, candidates' ability to interpret data was the weakest area.

4.  Discussion of Program Completer Survey

During the spring of 2008, The Citadel Professional Education Unit received 24 responses to the program completer survey administered to candidates in our initial (teacher education) programs. All initial BS and MAT completers responded to 32 common questions assessing the candidates' satisfaction with their program. Twenty questions focused on how well the program provided appropriate content knowledge, skills, and dispositions and twelve questions focused on operational issues such as advisement, field experiences, quality of instruction, etc.

Completers of initial programs rated the following areas the highest:

  • Development of a principled educational leader...
  • Knowledge of professional dispositions...
  • Ability to model professional dispositions...
  • Ability to use technology in teaching.

Initial completers were most critical of:

  • Availability of courses you need in the program
  • Ability to assess student learning
  • Quality of your pre-student teaching field experiences
  • Ability to use assessments to develop meaningful learning experiences
  • Ability to use technology in my teaching.

While the overall rating of initial completers was 4.71 on a five point scale, the teacher education division is encouraged to take seriously the more critical scores and to make improvements in these areas.

Completers of advanced programs responding to the survey included: 17 in counselor education; 36 in educational leadership; 7 in literacy; and 3 identified as undeclared majors. Completers of advanced programs rated the following areas the highest:

  • Development of a principled educational leader...
  • Quality of your internship placement
  • Demonstration of dispositions that value fairness...
  • Ability to create positive learning environments...
  • Support given by your mentor.

Advanced completers were most critical of:

  • Advisement provided by program advisor
  • Accessibility of your program advisor
  • Availability of courses
  • Availability of library resources
  • Understanding of the policy contexts in the school setting.

While the overall rating of advanced completers is a very positive 4.57 on a five point scale, advanced program coordinators and faculty are encouraged to take these criticisms seriously and to make improvements in the specified areas.

5.  Discussion of Employer Follow-Up Survey

During the spring of 2008, The Citadel Professional Education Unit distributed 115 questionnaires to area employers designed to determine their satisfaction with graduates of our professional education programs. Sixty six surveys were returned for a very positive return rate of 57%. We received 32 responses from employers of BS and MAT graduates; 13 responses from employers of our counselor education graduates, 14 responses from employers of educational leadership graduates; and 6 responses from employers of our literacy graduates.

On the 19 items in the survey of initial or teacher education graduates, employers rated the following highest:

  • Ability to present content in clear and meaningful ways...(4.73 on a 5 point scale)
  • Ability to use technology...(4.72)
  • Ability to model professional dispositions...(4.71)
  • Understanding the importance of diversity in teaching and learning (4.71)

Employers rated our teacher education graduates the lowest on the following:

  • Ability to develop and teach lessons that incorporate diversity (4.46)
  • Awareness of different teaching and learning styles...(4.5)
  • Ability to use assessment to develop meaningful learning experiences...(4.53)
  • Ability to use appropriate instructional strategies...(4.55)

While the overall ratings are relatively high, teacher education faculty are encouraged to examine the less positive ratings and consider changes for improvements based on this input.

The highest rating by employers of our school counseling graduates were:

  • Knowledge of the field (4.77)
  • Ability to know students, families, and communities (4.69)
  • Ability to understand and use the development level of students...(4.69)
  • Understanding the needs of diverse students...(4.69)

Lowest ratings by employers of our school counseling graduates were:

  • Awareness of different teaching and learning styles...(4.08)
  • Development as a principled educational leader...(4.23)
  • Ability to use technology...(4.23)

Again, while the overall employer ratings are relatively high, school counseling faculty are encouraged to examine the less positive ratings and consider improvements based on this input.

Highest ratings by employers of our educational leadership graduates were:

  • Ability to create positive environments for student learning...(4.73)
  • Understanding the needs of diverse students...(4.73)
  • Professional knowledge expected...(4.67)
  • Ability to know students...(4.67)
  • Ability to use current research...(4.67)
  • Ability to understand and build upon the developmental level of students...(4.67)
  • Understanding the policy contexts...(4.67)
  • Understanding the importance of diversity in teaching...(4.67)

Lowest rating by employers of our educational leadership graduates were:

  • Ability to develop a school climate that values diversity...(4.07)
  • Awareness of different teaching and learning styles...(4.13)
  • Ability to adapt instruction or services for all students...(4.20)
  • Demonstration of dispositions that value fairness and learning...(4.20)

While these overall employer ratings are relatively high, educational leadership faculty are encouraged to examine the less positive ratings and consider improvements based on this input.

Highest ratings by employers of literacy education graduates were:

  • Knowledge of content...(4.83)
  • Knowledge of professional, state, and institutional standards (4.83)
  • Professional knowledge expected...(4.83)
  • Understanding the policy contexts...(4.83)
  • Understanding the importance of diversity in teaching (4.83)

Lowest ratings by employers of our literacy education graduates were:

  • Ability to know students...(4.00)
  • Ability to adapt instruction or services for all students...(4.17)
  • Knowledge of dispositions expected...(4.17)
  • Ability to understand and build upon the developmental levels of students...(4.17)
  • Ability to model professional dispositions...(4.17)
  • Demonstration of dispositions that value fairness and learning...(4.17)

Literacy faculty are encouraged to examine the less positive ratings and consider improvements based on this input.

III.  General Recommendations

Our assessment system is, in many ways, exemplary. While we can and should take pride in what we have accomplished, we must continue to refine our system and processes to ensure its sustainability. In our desire to make sure we covered all bases, we may have developed a more complicated system than is necessary.

Since it appears that NCATE is now seeking to streamline the process, I suggest we follow their lead and find ways to simplify our system. As Dr. Feldmann suggested in her recent visit (August 21, 2008), each program area needs to focus on the 6 to 8 key assessments required by their respective SPAs and align that data so as to demonstrate that the unit is meeting state and national standards. Program coordinators are encouraged to work with Ms. Habhegger this academic year in aligning SPA and program standards and assessments with unit goals and objectives.

As suggested in the welcome back document distributed to you in August, a major focus for this academic year will be on strategic planning and related curriculum reform. With that in mind, it seems reasonable to suggest that program faculty take a hard look at the data compiled in the past year and a half and use that data in making program and curricular changes that both improve our programs and simplify our assessment system.

The final recommendation will sound familiar. Faculty are encouraged to continue the progress toward developing a more critical perspective in evaluating our candidates and programs. As we become more discriminating in our evaluation of students and programs, the data we generate becomes more meaningful and more useful in fostering continuous improvement of our candidates and programs.

Annual Professional Education Unit Assessment Report - 2006-2007 Academic Year

ANNUAL PROFESSIONAL EDUCATION UNIT ASSESSMENT REPORT

2006-2007 Academic Year

Submitted by Dean Tony Johnson

I. Development and Implementation of Assessment System for The Citadel's Professional Education Unit. The assessment system for the professional education unit was fully implemented in fall 2006 and refined throughout the academic year, 2006-2007. Implementation of the assessment system included the following primary initiatives:

  • Training of faculty and candidates in the use of LiveText;
  • Collection of data on all key common unit assessments and on individual course assessments;
  • Establishment of the Professional Education Board to serve as the assessment committee charged with reviewing program and unit assessment data each semester and making recommendations for changes;
  • Preparation of program assessment reports by program coordinators and faculty members in each division at the end of the fall, spring, and summer semesters;
  • Preparation of unit assessment reports by the dean at the end of the fall, spring, and summer semesters;
  • Preparation of a summary unit assessment report for 2006-2007 by the dean;
  • Submission of program and unit summary assessment reports to the School of Education faculty and the Professional Education Board at the end of the fall, spring, and summer sessions; and a summary unit assessment report for 2006-2007;
  • Development and dissemination of the "Professional Education Unit Assessment Handbook" that includes procedures for data collection, aggregation, disaggregation, analysis, dissemination, and use; and procedures for monitoring candidates' progress;
  • Development of a plan by each division faculty to demonstrate when and how the rubrics for all unit and program assessments are shared with faculty, candidates, and cooperating teachers/site supervisors who use the rubrics in the assessment process; and
  • Updating the School of Education website to include summary assessment reports for each semester for the individual programs and for the unit.

II. Analysis of Data.

A. Citadel PRAXIS II Data from 09/01/05 to Present.

Since the official Annual ETS Institutional Report for the PRAXIS exams, covering the year from 09/01/06 to 08/31/07 has not yet arrived, the data discussed here is unofficial and incomplete. Based on the information provided by the office of internships and field placements, our MAT candidates are performing well on these tests (see the Professional Education Unit Assessment Report for spring 2007). The high passing rates achieved by MAT candidates are not surprising since passing all the relevant PRAXIS II exams is a requirement for placement into the internship. As noted in the spring 2007 unit assessment report, Social Studies teacher education candidates did not fare as well. The unofficial data for the teacher education programs from 09/01/06 to 06/09/07 are as follows:

  MAT
Number
Passed/Total
Number Taken
MAT
% Pass Rate

CADET
Number
Passed/Total
Number Taken
CADET
% Pass Rate

MATHEMATICS        
0061 2/2 100% None  
0063 2/2 100% None  
         
BIOLOGY        
0030 7/7 100% None  
         
ENGLISH        
0041  6/7 85.7%
0/1
0%
0042  9/9 100%
 0/1  0%
         
SOCIAL
STUDIES
       
 0081 11/13
84.6%
10/13
76.1%
 0083 9/9
100%
7/11
63.6%
         
PHYSICAL
EDUCATION
       
 091 7/7
100%
6/6
100%
 093 6/6
100%
6/6
100%

Again, based on unofficial data provided by the office of internships and field placements, initial teacher education candidates are performing reasonable well on the Principles of Learning and Teaching Test (PLT) required for certification as of 07/01/06. The pass rate for candidate4s is 82% with 37 of the 45 candidates passing this test. Again, MAT candidates performed better than cadets with a pass rate of 91% (29 of 32) as compared to a pass rte of 62% (8 or 13) for undergraduates. The pass rates are indicated in the following table:
Test Passed/Taken % Pass Rate
Initial Teacher Preparation Programs    
0522  PLT 1-6    
MAT 3 / 4 75%
Undergraduate 2 / 3 66%
     
0523  PLT 5-9    
Undergraduate 1 / 1 100%
     
0524  PLT 7-12    
MAT 26 / 28 92.8%
Undergraduate 5 / 9 55%
     
Advanced Programs    
     
0410 - Educational Leadership: Administration and Supervision 10 / 10 100%
     
0420 - School Guidance and Counseling 8 / 9 88%
     
0400 - School Psychologist 9 / 10 90%
     
0200 - Introduction to the Teaching of Reading 9 / 9 100%

Candidates in the advanced programs tend to score well on the PRAXIS II specialty area exams with 36 out of 38 passing these tests for a pass rate of 95%.

Action. While the overall pass rate for the PRAXIS II exams is above the 80% pass rate threshold for initial programs and continues to improve, the performance of cadet Social Studies teacher candidates is still problematic. Candidates not passing one or more of the required PRAXIS II or PLT exams will in all likelihood achieve a passing score on the second or third try.  The fact that passing these tests is a graduation requirement should add an element of importance to these exams. This policy change along with a practice test developed by Dr. Barrett is expected to improve the pass rate of the cadet Social Studies teacher candidates.

B. Unit Assessment Data from LiveText.

1. Discussion of Content Knowledge Assessment. This assessment consists of two competencies. It has been administered in only two courses: EDUC 524: Techniques of School Supervision (summer) and EDUC 629: Practicum in School Counseling (fall and spring). These competencies are also assessed on the Portfolio Assessment.  The intent of the School Counselor Education program and the Educational Leadership program is to use this instrument to assess candidates during their program of study as the Portfolio Assessment is used primarily at the end of the program. The Literacy Education program has not used this assessment in any of their courses, although they do use the Portfolio Assessment at the end of the program.

The ratings on each of the competencies were very high in fall (range of 2.83 to 2.78) and spring (range of 3.00 to 3.00) when the assessment was used in a school counseling course, but much lower in summer (range of 2.29 to 2.29) when the assessment was used in an educational leadership course.

Action. For this to be a meaningful unit assessment for the advanced programs, it needs to be administered regularly and in more courses, including courses in the Literacy Education program. Additionally, the rubrics should be reviewed and revised to increase the reliability among the responders.

2. Discussion of Student Learning Assessment.  This assessment consists of four competencies. As with the Content Knowledge Assessment, it has been administered in only two courses: EDUC 524: Techniques of School Supervision (summer) and EDUC 629: Practicum in School Counseling (fall and spring). These competencies are also assessed on the Portfolio Assessment. The intent of the School Counselor Education program and the Educational Leadership program is to use this instrument to assess candidates during their program of study as the Portfolio Assessment is used primarily at the end of the program. The Literacy Education program has not used this assessment in any of their courses, although they do use the Portfolio Assessment at the end of the program.

The ratings on each of the competencies were very high in fall (range of 2.91 to 2.83) and spring (range of 3.00 to 3.00) when the assessment was used in a school counseling course, but much lower in summer (range of 2.44 to 1.94) when the assessment was used in an educational leadership course.

Of the four student learning competencies, "The candidate understands diversity of students, families, and communities," had the highest rating each of the three semesters.

Action. For this to be a meaningful unit assessment for the advanced programs, it needs to be administered regularly and in more courses, including courses in the Literacy Education program. Additionally, the rubrics should be reviewed and revised to increase the reliability among the responders.

3. Discussion of Portfolio Assessment.  This assessment consists of six competencies--the same two that are on the Content Knowledge Assessment and the same four that are on the Student Learning Assessment.

Overall, the faculty gave the candidates higher ratings than did the candidates in both the fall and summer semesters. Also overall, the ratings were higher in both fall and summer than in the spring semester.

Of the four student learning competencies, "the candidate understands diversity of students, families, and communities," had the highest or second highest rating each of the three semesters. The competency, "the candidate understands and builds upon developmental levels of students," had the lowest rating of the four competencies each of the three semesters.

Action. The portfolio rubrics were reviewed by the professional education unit faculty at the May faculty retreat and also at the August faculty retreat. Clarifications and refinements were made to the rubrics, which will be implemented in fall 2007.

Faculty members were asked to examine their programs to determine how they can better prepare candidates to understand and build upon the developmental levels of students.

4. Discussion of Research Competencies Assessment.  This assessment consists of five competencies.  It was administered in EDUC 512: Data Collection and Analysis and in EDUC 549: Applied Measurement Techniques each of the three semesters. It was also administered in HESS 540: Research Techniques and Methods of Analyzing Research in Health, Exercise, and Sport Science in the summer.

The rankings of the five competencies were the same each of the three semesters with the competency, "the candidate uses technology to present course projects," receiving the highest score each semester; and the competency, "the candidate interprets descriptive and inferential data," receiving the lowest score each semester. The scores ranged from 2.55 to 2.25 in fall, from 2.65 to 2.48 in spring, and from 2.71 to 2.21 in summer.

Action. The instructor of EDUC 512 and EDUC 549 developed a pretest and a posttest that was piloted in these two courses this summer to identify specific areas where candidates have difficulty. The data are then used to develop interventions to assist candidates in developing their research competency skills. The pretest and posttest will continue to be used by the instructor in these courses.

5. Discussion of Professional Dispositions Assessment.  This assessment consists of six competencies. It is administered in a variety of education courses throughout the candidates' programs. It was administered at the beginning, midterm, and end of the student teaching internship in the fall; however, beginning in the spring it was administered only at midterm and at the end of student teaching to simplify the data collection process. It was also administered at midterm and at the end of the practica/internships in th advanced programs.

The scores increased on most of the competencies from midterm to the end of the semester.

The scores for all competencies were higher at the end of the semester in summer than in spring, and they were higher at the end of the semester in spring than in fall. In other words, the scores increased each semester--from fall to spring to summer.

The competency that was rated highest most often was "demonstrates high values and a caring, fair, honest, responsible, and respectful attitude."  This is no surprise given The Citadel's honor code and core values. The two competencies that usually had the lowest scores were "Establishes rapport with students, families, colleagues, community" and applies reflective practices."

There was no trend in terms of which of the three groups (candidates, faculty, and cooperating teachers/site supervisors) typically gave the highest of the lowest ratings.

Action.  The professional dispositions rubrics were reviewed by the professional education unit faculty at the May faculty retreat and also at the August faculty retreat.  Clarifications and refinements were made to the rubrics, which will be implemented in fall 2007.  Also, all faculty members should emphasize the importance of reflection in class assignments, teaching lessons, assessments, etc., especially since we are preparing principled education leaders to be knowledgeable, reflective, and ethical professionals.

6. Discussion of Program Completer Survey.  This survey consists of 32 questions for the initial program completers--20 questions that address knowledge, skills, and dispositions, and 12 questions that address other aspects of their program, such as advising, field experience/internship placements, quality of instruction, etc. The survey was administered to only MAT program completers in fall; however, it was administered to both the BS and the MAT program completers in the spring. There were no initial program completers in summer. A second survey consists of 30 similar questions for the advanced program completers--19 questions that address knowledge, skills, and dispositions; and 11 questions that address other aspects of their programs. The survey was administered to all advanced program completers at the end of fall, spring, and summer semesters. Candidates were asked to assess their program on a five-point scale from "very satisfied" to "very dissatisfied."

The mean responses for the initial candidates ranged from 4.95 to 3.73 in the fall and from 4.81 to 4.52 in the spring. The mean responses for the advanced candidates ranged from 4.85 to 3.92 in the fall, from 4.85 to 4.42 in the spring, and from 4.83 to 4.32 in the summer.

Overall on both surveys each semester, higher scores were typically given in the ares that addressed knowledge, skills, and dispositions; and lower scores were typically given in the areas that addressed other aspects of the program, such as availability of courses, advising, and quality of field placements.

Action. The new director of internships and field experiences has taken steps to improve the placements of candidates in the field and to enhance the monitoring of candidates and the placements. For MAT candidates, the establishment of a dual advising system involving both education and content area faculty is helping to address the advising problem.  However, faculty members in each division should continue to identify other ways to improve the other aspects of their programs.

7. Discussion of Graduate and Employer Follow-Up Surveys. The survey for graduates one year ago from the initial programs and the survey for their employers consist of the same 20 questions that address knowledge, skills, and dispositions that were on the Program Completer Survey.  Likewise, the survey for graduates one year ago from the advanced programs and the survey for their employers consist of the same 19 questions that address knowledge, skills, and dispositions that were on the Program Completer Survey.  The survey was administered early this summer. The same five-point scale ("very satisfied" to "very dissatisfied") was used as for the Program Completer Survey.

The surveys were sent to 228 graduates of one year ago and to 17 of their employers who could be identified. A total of 17 graduates from the initial programs and 2 of their employers completed the survey; and a total 38 graduates from the advanced programs and 4 of their employers completed the survey.  The responses from the initial program graduates ranged from 4.47 to 3.94. The highest scores were in the areas of "knowledge of the dispositions expected of professionals" (4.47) and "ability to model professional dispositions in working with students, families, and communities" (4.44). The lowest scores were in the areas of "ability to present the content in my field in clear and meaningful ways to help all students learn" (3.94) and "ability to use technology in my teaching" (3.94). The responses from the two employers ranged from 5.00 to 4.00. The responses from the advanced program graduates ranged from 4.83 to 4.55. The highest scores were in the areas of "ability to create positive environments for student learning in my classroom or building: (4.83), "ability to model professional dispositions in working with students, families, and communities" (4.76), "ability to understand and build upon the developmental levels of students with whom I work" (4.76), and "understanding of the importance of diversity in teaching and learning" (4.76.). The lowest scores were in the areas of "ability to use current research to inform my work" (4.55) and "awareness of different teaching and learning styles that are shaped by cultural influences" (4.59). The responses from the four employers ranged from 5.00 to 4.50.

Action. Identify ways to increase the return rate for both surveys. Also, faculty members in each program should compare and monitor the data from the Graduate and Employer Follow-Up Surveys with the data from the Program Completer Survey.

8. Discussion of Assessments of Teacher Education Candidates. Parts of or all of the ADEPT instrument are used throughout the semester to assess initial (BS and MAT candidates) on the Long-Range and Short-Range Plan Evaluation (ADEPT 1-3); Field Experience Observation Evaluation (ADEPT 4-9); Content Indicator Evaluation (ADEPT 1-10); Final Portfolio Evaluation (ADEPT 1-10); and ADEPT Summary Consensus Candidate Evaluation (ADEPT 1-10). Also, the Candidate Evaluation of Field Experience, Candidate Evaluation of College Supervisor, and Candidate Evaluation of Cooperating Teacher are used at the end of the semester.

Overall, the data suggests that weaknesses are identified during the methods courses that result in improvements as the candidates progress through the program.

Action. The data for each of the two BS programs and the five MAT programs need to be more systematically shared each semester with the appropriate faculty members in Biology, English, Mathematics, Physical Education, and Social Studies by the faculty members in the Teacher Education division in the School of Education.

III. General Recommendations.  Commendations are in order for the phenomenal progress made during the past 15 months in implementing LiveText and in establishing a viable assessment system for The Citadel's Professional Education Unit.  As previously stated, together we have achieved much but much remains to be done. In addition to earlier recommendations (see the Professional Education Unit Assessment Reports for fall 2005 and spring 2007) regarding sharing and using the data generated to guide our programs and the need to simplify our process, a third recommendation is in order.

In reviewing the summary tables of the LiveText data compiled from the fall (06), spring (07), and summer (07), the most striking characteristic is the excessively high mean scores on almost all of the assessments. Using primarily a three-point scale, both candidates and faculty tend to assess candidates in the high 2s or even 3s on most unit assessment items. These excessively high ratings make it difficult, if not impossible, to identify strengths and weaknesses of candidates and provides little useful information for changing or improving our programs. For these reasons, the third recommendation is that we develop and apply a more critical perspective in evaluating candidates using the instruments that have been developed.

Progress toward this goal of employing a more critical perspective is already evident in the administration of the student learning and content knowledge assessments for candidates in Educational Leadership this past summer. In addition, considerable time and energy has been devoted to revising and refining the portfolio and dispositions rubrics discussed in our spring (07) retreat and fall (07) advance.  These are steps in the right direction, but it is imperative for The Citadel's Professional Education Unit to continually revise and refine these assessments in order to obtain more useful information that leads to program improvement and candidate progress. As a unit, we need to more critically assess our candidates' progress in becoming principled educational leaders and we need to more critically examine our programs to ensure that we are enabling candidates to achieve this goal.

Fall 2007 Professional Education Unit Assessment Report

 

 Professional Education Unit Assessment Report

Fall 2007

Submitted by Dean Tony W. Johnson

(March 11, 2008)
 

I.          Continuing Development and Implementation of Assessment System for The     Citadel’s Professional Education Unit.

                        During the fall of 2007, a site visit by NCATE Board of Examiners and representatives from both the South Carolina Department of Education and Commission on Higher Education evaluated our assessment system and its efficacy in monitoring candidate progress through our programs and in facilitating continuous improvement of our professional education programs.  In preparation for this visit and beyond, initiatives for the fall 2007 semester included:

  • Continuing the use of Livetext to compile, aggregate, and disaggregate program and unit data on key common assessments for initial and advanced professional education programs;
  • Acquainting both faculty and candidates with the “new” Livetext and developing a training strategy and plan for implementing this new version;
  • Establishing functional appeals committees at both the initial and advanced levels to consider student requests for waiver of specific program or unit academic requirements and to recommend action to the unit head;
  • Completion and distribution of  a Professional Education Assessment Handbook comprised of the assessment procedures approved by the Professional Education Board during the previous academic year (2006/2007);
  • Organizing an Exhibit room in preparation for the NCATE site visit (09/30/07-10/02/07) highlighting our assessment system and documenting compliance with state and NCATE standards;
  • Hosting a very successful NCATE and State visit resulting in a report by the  Board of Examiners and State team recommending that all standards are now considered met and that all areas for improvement have been corrected; and
  • Developing a timeline and plan for sustaining the assessment system now established and for identifying the assessments for collecting the necessary data required for each program’s SPA reports.

II.        Analysis of Data

      A.  Citadel PRAXIS II data from 9/01/2006 through 08/31/07

                  The official ETS Institutional Summary Report for 2006/2007 has been received by The Citadel Graduate College.  Based on this data each program area will receive a summary sheet detailing how candidates in their area performed on the relevant PRAXIS II exams.   Based on this data, it appears that candidates in all areas are doing reasonably well on these exams.  Overall, our candidates meet the 80% pass rate mandated by the state, but social studies remains an area of concern.  Of the 41 candidates from The Citadel taking both of the social studies exams, 32 candidates passed both exams.  Since these numbers include both cadet and MAT students, the assumption is that cadets are still experiencing difficulty in achieving passing scores.   For cadet teacher candidates seeking to graduate this spring (2008), the stakes are higher since a passing grade on the relevant PRAXIS II exams is required for graduation.   This requirement-along with the availability of a practice exam developed by General Barrett-should produce improvement in this passing rate in subsequent years.

      In addition to these specialty area exams, teacher candidates must now take and pass the appropriate level principles of learning and teaching exam.  During 2006/2007, 74 of 88 Citadel candidates earned passing scores on the relevant PLT exam for a pass rate of 84%.  Since a passing score on this exam is now a graduation requirement for cadets completing their programs this spring (2008), a higher pass rate is expected in subsequent years.     

      B.  Unit Assessment Data from LiveText

                  1. Discussion of Content Knowledge and Student Learning               Assessment.   A rubric (consisting of 6 competencies) assessing the content knowledge of candidates and their ability to use that knowledge to foster student learning  was administered to candidates and their instructors in all programs (except school psychology) during the fall 2007 semester.   A total of 53 candidates evaluated themselves using this instrument and faculty conducted 122 evaluations of the candidates using this rubric. 

      Using a 3-point scale, candidates evaluated themselves as least competent in their ability to apply their knowledge of content to professional standards through inquiry, critical analysis and synthesis.  The data suggests that candidates are more comfortable with their ability to foster student learning than they are with their ability to understand and apply the underlying concepts of their field in an educational setting.   The data suggests that both candidates and their faculty are reasonably satisfied with the knowledge and skills possessed by candidates.  While the overall mean for both candidate and student responses is in the 2.7 range,  this is a lower overall score than previous semesters.  One interpretation of these lower overall scores is that both faculty and candidates are becoming more comfortable with this process and instruments and are more willing to apply them critically.   

      Based on the Praxis scores discussed above and the data summarized here, it appears that candidates are reasonably knowledgeable of their content area and possess some skill in applying this knowledge to enhance student learning.  The data also suggests that improvement is needed in helping candidates understand the conceptual structures of their field and in applying this knowledge to foster a higher level of learning in students. 

                  2. Discussion of Professional Dispositions Assessment.   Using a rubric that is closely aligned with the ethical indicators emphasized in the unit’s conceptual framework, the teacher education (including HESS) and counseling programs administered this assessment at midterm and at the end of the semester in practicum or internship courses.  Literacy and educational administration programs administered this assessment at the end of the semester. 

      There were a total of 44 responses at midterm: 22 candidate responses; 18 faculty responses, and 4 cooperating teacher responses.    It is interesting to note that cadets rate themselves much lower (2.20) on establishing “rapport with student, families, colleagues, community’’ than on any other category.  In contrast, SOE faculty responses indicate that candidates’ weakest areas are:  applying reflective practices (2.17) and establishing rapport with students, parents, etc. (2.17).  On these midterm assessments, SOE faculty rated candidates relatively low, ranging from a low of 2.17 to a high of 2.33.     

      End of year responders on the professional dispositions assessment totaled 486 responses: 185 candidate responses; 272 SOE faculty responses; and 29 cooperating teacher responses.  It is interesting to note that the means for all candidates’ responses on all 6 items of this rubric are substantially higher than the means of the responses of the School of Education faculty.  For example the mean SOE faculty score for item 1 (reflective practice) is 2.66 while the candidate mean for the same item is 2.93.   The faculty mean for item 5 (values diversity) is 2.69 while the candidate mean is 2.97.  While this difference may or may not be significant, it suggests that School of Education faculty are becoming more comfortable with this assessment instrument and are assessing candidates with a more critical eye.  It is also encouraging to note that the mean score for all 6 items was higher at the end of the semester than the scores at midterm.  

      While the overall scores remain high, it is encouraging to note that faculty appear to be more discriminating in their scoring of the rubrics.  It is heartening to see that end of term scores when compared to scores from the mid-term evaluations of the internships—suggest that growth is occurring during this culminating experience. 

                  3.  Discussion of Research Competencies Assessment.    The rubric designed to assess the research competencies of candidates in the Citadel’s professional education programs was administered to 77 candidates during the fall 2007 semester.  On a 3-point scale, the mean scores on the five competencies assessed range from a low of 2.55 (candidate demonstrates use of statistical procedures) to a high of 2.65 (candidate paraphrase information from research articles.   The competencies of “demonstrating use of statistical procedures” and “interpreting descriptive and inferential data” remain the most problematic for students, but comparison of the data from the past three semesters indicates that the fall 2007 students are clearly more competent in these areas than students in previous semesters.  Such improvement can be attributed - at least in part - to the pre-test developed by the instructor and administered early in the semester to identify and target for assistance those students in need of help.

                  4. Discussion of Program Completers Survey.   Late in the semester, a survey-designed to assess the level of satisfaction of candidates completing their program-was administered to 19 teacher education completers (cadet and MAT candidates) and to 28 candidates completing advanced programs in educational administration, literacy, and school counseling. 

All initial (BS and MAT ) advanced candidates completed a survey of 32 common questions focused on their comfort level with the professional knowledge, skills and dispositions provided them by their program, their satisfaction with the operational aspects of the program (advising, course availability, etc.,) plus additional questions specific to their individual programs.

                  Using a 5-point scale ranging from very satisfied (5) to no opinion (1), the responses of the 47 completers were generally very positive.  For completers at the initial level, ratings on questions focused on candidate satisfaction of the knowledge, skills and disposition provided by the program ranged from a low of 4.22 (different teaching and learning styles) to a high of 4.67 (knowledge of dispositions expected of professionals). Responses to questions focused on operational aspects of the program ranged from a low of 4.05 (availability of courses) to a high of 4.65 (library resources). 

                  Advanced program completers expressed even higher overall satisfaction with their Citadel experience.  Responses to questions focused on candidates’ satisfaction with the academic aspects of their program range from a low of 4.59 (ability to adapt instruction for special needs students) to a high of 4.86 (becoming a principled educational leader who is knowledgeable, reflective, and knowledgeable).  Responses to questions focusing on the operational aspects of the program range from a low of 4.38 (course availability) to a high of 4.86 (support provided by school mentor). 

Completers at both the initial and advanced levels expressed concern over courses being available when needed.  This suggests a need for a comprehensive review of our scheduling procedures to ensure that the first priority in scheduling courses is in responding to the program needs of students. 

 Advising by program advisors also appeared as an area of concern by completers in both initial and advance programs. Advanced program completers identified admission procedures into their program as an area of concern.  While the overall reaction to the professional education programs at The Citadel is largely very positive, program faculty are encouraged to reexamine their advising procedures and to carefully attend to the scheduling of courses to address the availability issues identified by both initial and advanced completers. 

                  5.  School of Education Program Area Summary Reports.   Each semester each division of The Citadel’s School of Education develops a report summarizing the data compiled using LiveText.  These reports are shared with program area faculty and distributed to faculty in the School of Education and with members of the Professional Education Board.  In reviewing the data provided in these reports, program faculty and coordinators are encouraged to identify strengths and weaknesses identified through this process and to use the information provided to make program changes needed for improvement.  Included below are a few of the insights gleaned from the fall 2007 semester data along with suggested changes for improvement. 

                  Based on this fall 2007 semester data, the Teacher Education Division recognizes the need for additional and earlier support for candidates experiencing difficulty in their student teaching internships.  Among the recommendations resulting from this semester’s data is that faculty review the professional dispositions with candidates prior to their internships to focus on areas of weaknesses indicated by the data.  In addition to focusing on the dispositions and on candidates struggling in their internships, weaknesses identified by the data suggest a need for a greater program emphasis on the impact of cultural influences on teaching and learning.  Finally, data from this semester suggest the need for working with the content areas in developing a more reasonable sequence of course offerings in all content areas. 

                  The School of Education Literacy Division recommendations derived from this semester’s data focus on the assessment process and how to improve it.  Specifically, responses suggest that the Literacy Field Project Assessment is very demanding.  Based on this feedback, the course instructor is reexamining this assessment instrument to determine if the assignments and requirements can be reduced.  

                  While faculty from the Division of Educational Leadership are largely pleased that this semester’s data indicating target or acceptable performance by all candidates in educational leadership programs, they identified low candidate responses on the dispositional assessments as a problem.  Faculty have agreed to address this problem by allotting class time for students to complete this assessment instrument. 

                  Data from this semester’s assessment process suggests that School Counseling programs need more emphasis on assessment and appraisal skills.  Based on this data, the Division of School Counseling is considering adding content in classroom and behavioral management and in ethical and legal issues.  Again, based on this semester’s data, program faculty are pursuing ways of strengthening assessment and research content in their programs. 

III.   General Recommendations.

                  It appears that the assessment system recently established and implemented is working.  While refinement of this process and system is an on-going enterprise, the development of a viable and sustainable assessment system is a significant accomplishment.  Congratulations and a very special thank you to all of you for this remarkable accomplishment.  Please consider my recommendations in light of this tremendous achievement.

·        The recommendation to keep it simple is not new, but it is still relevant.  Especially as we expand our offerings and as we prepare for the next round of SPA reviews,  it is recommended that each program area focus on 6 to 8 key assessments that both reflect the uniqueness of specific programs and serve as indicators of the values and objectives of the both the program and unit as a whole;

·        It is also recommended that faculty continue the progress—so clearly evident in this semester’s data—toward developing a more critical perspective in evaluating our candidates and our programs.  As we develop this more critical perspective, the data we generate becomes more meaningful and thus more useful for initiating changes for continuous improvement; and

·        Finally, the data collected this semester and in previous and subsequent semesters must be shared in order to guide program development and contribute to our goal of developing principled educational leaders.  For this reason, it is recommended that program coordinators develop a process for sharing this semester’s—along with previous and subsequent semesters’—data with students, program faculty, advisory board, and site –supervisors.  .  It is recommended that program faculty take the lead in scrutinizing this data for strengths and weaknesses as the first step toward correcting the weaknesses and building upon the strengths. 

 

Summer 2007 Professional Education Unit Assessment Report

PROFESSIONAL EDUCATION UNIT ASSESSMENT REPORT

Summer 2007

Submitted by Dean Tony Johnson

I. Development and Implementation of Assessment System for The Citadel's Professional Education Unit.  During the summer of 2007, the implementation and refinement of the assessment system for the professional education unit continued. Initiatives for the summer 2007 included:

  • Finalizing the revisions of the Professional Dispositions and Portfolio assessment rubrics at the "ADVANCE" (retreat), which was attended by the professional education unit faculty in August;
  • Finalizing and distributing the "Professional Education Unit Assessment Handbook" to the professional education faculty at the "ADVANCE" in August;
  • Finalizing the program handbooks for each division and making them available to candidates;
  • Development of a pretest and a posttest that was piloted this summer in the two research courses to identify specific areas where candidates have difficulty; and
  • Updating the School of Education website to include summary assessment reports for each semester for the individual programs and for the unit.
II.  Analysis of Data.

A. Citadel PRAXIS II Data.  No additional PRAXIS II data has been collected during the summer. The ETS summary data for the 2006/2007 academic year should be available soon. This summary data was to be generated on August 31, 2007, but--as of today (09/10/07)--the data is not yet available.

B. Unit Assessment Data from LiveText.

1. Discussion of Content Knowledge Assessment. The assessment was completed at the end of the semester on 16 educational leadership candidates by the instructor of EDUC 524. The mean for each of the two competencies was 2.29 on a three-point scale.  The mean for each of the competencies was much lower this semester than it was for spring or for fall semesters.

Action. The question here is why are the mean scores so much lower in the summer than in the spring or fall. It may be that the lower scores in the summer are a more realistic evaluation of our candidates' knowledge and abilities. This data needs to be discussed by all faculty in order to more effectively calibrate the evaluations.

2. Discussion of Student Learning Assessment.  The assessment was completed at the end of the semester by 16 educational leadership candidates and by the course instructor for a total of 32 responses.  The scores on each of the four competencies given by the candidates ranged from 2.88 to 2.50 on a three-point scale; whereas, the scores on each of the four competencies given by the faculty member ranged from 2.00 to 1.38. The highest rating given by both the candidates and the faculty member was on "the candidate understands diversity of students, families, and communities." The lowest rating given by both the candidates and the faculty member was on "the candidate understands policy contexts in work settings."

Action. Here again, the question becomes why are the scores for the summer substantially lower than the scores for the fall and spring. Perhaps the summer scores reflect a more critical perspective in recognition of the high scores recorded in the fall and spring by both candidates and faculty. This could be a positive indicator of progress toward refining and increasing the efficacy of the assessment instrument. This needs to be carefully monitored to determine if the summer scores are an aberration or the beginning of a trend.

3.  Discussion of Portfolio Assessment.  The assessment was completed at the end of the semester by 39 candidates and 44 faculty members for a total of 83 responses.  Faculty members rated the candidates higher or as high as the candidates rated themselves on five of the six competencies, although the range for both the candidates and the faculty was 2.95 to 2.84 on a three-point scale. All 13 candidates and the 14 responses from faculty members in the Teacher Education program gave perfect 3.00 ratings on each of the six competencies.

Action. The excessively high scores across all program areas are somewhat problematic. These scores are comparable to scores from the fall semester but are higher than the scores for the spring semester. These high scores raise questions of the efficacy of the instrument or in our ability or willingness as faculty and candidates to discriminate regarding the knowledge and skills of candidates. Since completers of our programs are--at best--novice professionals, it is questionable that virtually all of them are performing at the target level. Faculty in all program areas are encouraged to review these rubrics with a more critical and/or discriminating eye.

4. Discussion of Research Competencies Assessment. The assessment was completed at the end of the semester by the course instructor in EDUC 512, EDUC 549, and HESS 540 on a total of 38 candidates. The means ranged from 2.71 to 2.21 on a three-point scale on the five competencies. Several candidates received an "unacceptable" rating on at least one competency. Candidates' use of technology was the strongest area. Candidates' ability to interpret data was the weakest area.

Action. The data here is consistent with the previous scores from the fall and spring semesters.  No other action other than the implementation of the pretest already developed and piloted is necessary at this time.

5. Discussion of Professional Dispositions Assessment. The assessment was completed at the end of the semester by 114 candidates, 128 School of Education faculty members, and 12 cooperating teachers/supervisors for a total of 254 responses. The cooperating teachers gave the highest possible rating (3.00) on five of the six competencies. The lowest rating given by any of the three groups on any of the competencies was given by the candidates on "applies reflective practices."  The lowest rating given by each of the groups and by all three groups combined was also on "applies reflective practices."

Action. All faculty members should emphasize the importance of reflection in class assignments, teaching lessons, assessments, etc., especially since we are preparing principled education leaders to be knowledgeable, reflective, and ethical professionals.

Action. The mean scores on each of the six items are comparable to the scores from the fall and spring semesters. As is the case for most of our assessments, the mean scores are excessively high across the board making it difficult to determine strengths and weaknesses. This is an issue that all faculty need to address as we continue to refine and improve our assessments.

6. Discussion of Program Completer Survey.  Candidates responded to 30 questions asking them to rate their satisfaction of their program in providing them with appropriate knowledge, skills, and dispositions (19 questions); and their satisfaction with other aspects of their program, such as advising, field experience/internship placements, quality of instruction, etc. (11 questions). The assessment was completed at the end of the semester by 29 candidates--27 educational leadership candidates and 2 literacy education candidates. There were no initial program completers this semester. A five-point scale was used ranging from "very satisfied" to "very dissatisfied." Scores ranged from 4.83 ("ability to use technology in my work") to 4.32 ("availability of courses that you needed in the program"). The five highest scores were given in areas that addressed the program providing appropriate knowledge, skills, and dispositions; whereas, the four lowest scores were given in areas that addressed other aspects of the program, including advisement and quality of the internship placement.

Action. Faculty members should identify ways to improve the other aspects of their programs, such as availability of courses, advisement, and quality of internship placements.

7. Discussion of Graduate and Employer Follow-Up Surveys. A survey was sent to 228 graduates of one year ago and to 17 or their employers who could be identified.  Graduates from the initial programs and their employers were asked to respond to the same 20 questions related to knowledge, skills, and dispositions that the candidates who completed their program this semester were asked to respond to on the Program Completer Survey. Graduates from the advanced programs and their employers were asked to respond to the same 19 questions related to knowledge, skills, and dispositions that the candidates who completed their program this semester were asked to respond to on the Program Completer Survey.

A total of 17 graduates from the initial programs and 2 of their employers completed the survey. The survey was sent to 86 graduates and to 5 employers who could be identified. The responses from the graduates ranged from 4.47 to 3.94. The highest scores were in the areas of "knowledge of the dispositions expected of professionals" (4.47) and "ability to model professional dispositions in working with students, families, and communities" (4.44). The lowest scores were in the areas of "ability to present the content in my field in clear and meaningful ways to help all students learn" (3.94) and "ability to use technology in my teaching" (3.94). The responses from the two employers ranged from 5.00 to 4.00.

A total 38 graduates from the advanced programs and 4 of their employers completed the survey.  The survey was sent to 142 graduates and 12 employers who could be identified. The responses from the graduates ranged from 4.83 to 4.55. The highest scores were in the areas of "ability to create positive environments for student learning in my classroom or building" (4.83), "ability to model professional dispositions in working with students, families, and communities" (4.76), "Ability to understand and build upon the developmental levels of students with whom I work" (4.76), and "understanding of the importance of diversity in teaching and learning" (4.76). The lowest scores were in the areas of "ability to use current research to inform my work" (4.55) and "awareness of different teaching and learning styles that are shaped by cultural influences" (4.59). The responses from the four employers ranged from 5.00 to 4.50.

Action. Identify ways to increase the return rate for both surveys.

Action. Compare and monitor the data from the Graduate and Employer Follow-Up Surveys with the data from the Program Completer Survey.

Spring 2007 Professional Education Unit Assessment Report

Professional Education Unit Assessment Report

Spring 2007

Submitted by Dean Tony Johnson


 

I.                   Development and Implementation of Assessment System for The Citadel’s Professional Education Unit. 

                During the spring of 2007, the implementation and refinement of the assessment system for the professional education unit continued.  Initiatives for the spring 2007 semester include:

·        Continuation and expansion of training of faculty and candidates in the use of LiveText;

·        Employing LiveText as an instrument for collecting and aggregating data on key common assessments and for initial and advanced professional education  programs;

·        Preparation of assessment reports by each division and/or program area based on an analysis of the data derived from the key unit and program assessments;

·        Submission of assessment reports to School of Education faculty and to the Professional Education Board for review and action;

·        Establishing more rigorous admission and completion standards for both initial and advanced programs and instituting processes for ensuring that these more rigorous standards are administered in a consistent and fair manner;

·        Compiling of assessment procedures approved by the Professional Education Board into a Professional Education Assessment Handbook for distribution in the fall of 2007; and

·        Development of an NCATE and state Institutional Report for submission in July 2007 in preparation for the focus visit scheduled for September 30 to October 2, 2007.

II.                Analysis of Data.

A.     Citadel PRAXIS II from 01/01/07 to present

        The ETS Institutional Summary reports are not yet available for 2006/2007, but the office of internship and field placements has provided us with unofficial PRAXIS II data for cadet and MAT teacher education candidates. For the 2006/2007 academic year, all MAT candidates in Mathematics, Biology and Physical Education passed all the relevant PRAXIS II exams. Six of seven (85.7%) English MAT candidates passed the 0041 English PRAXIS II exam while all nine English MAT candidates passed the 0042 exam. All nine of the Social Studies MAT candidates passed the 0083 PRAXIS II exam while eleven of thirteen (84.6%) Social Studies MAT candidates passed the 0081 PRAXIS II exam. These very high passing rates are to be expected since passing all the relevant PRAXIS II exams is a requirement for placement into the internship.

       Cadet teacher education candidates did not fare as well. Ten of thirteen (76.1%) cadet Social Studies teacher candidates passed the 0081 PRAXIS II exam and seven of eleven (63.6%) of the cadet Social Studies teacher candidates passed the 0083 PRAXIS II exam. All cadet Physical Education teacher candidates passed both the 091 and 093 PRAXIS II exams.

       While the overall pass rate on PRAXIS II exams for initial programs is well above the 80% pass rate threshold and continues to improve, the performance of cadet Social Studies teacher candidates is still problematic. Candidates not passing one or more of the required PRAXIS II exams will in all likelihood achieve a passing score on the second or third try. The fact that the test is now a graduation requirement should add an element of importance to these exams. This policy change as well as a practice test developed by Dr. Barrett is expected to improve the pass rate of cadet Social Studies teacher candidates.

       Congratulations to the Physical Education teacher track division and their teacher candidates for their significant improvement in their pass rate on all relevant PRAXIS II exams.

B.     Unit Assessment Data from LiveText

1.         Discussion of Content Knowledge and Student Learning Assessment.    A rubric assessing the content knowledge and ability to assist student learning  (consisting of 6 competencies)  was administered to candidates in all programs (excluding school psychology) during the spring 2007 semester.   A total of 59 candidates evaluated  themselves using this rubric and faculty evaluated 143 candidates using this rubric.  Faculty and candidate evaluations using this rubric were very similar. The mean scores for faculty responses on the six competencies ranged from a low of 2.72 to a high of 2.8  on a 3 point scale. Candidate responses ranged from a low mean score of 2.9 to a high of 2.97 on a 3 point scale. It appears that both faculty and students are satisfied with candidates' knowledge of their content field and of their ability to enhance student learning.     While there are differences in the mean scores among the various programs, the differences do not appear to be significant.

The mean scores for both faculty and candidates remain high, but it merits noting that the scores of a comparable number of faculty responses last fall (2006) were considerably higher.  The faculty mean in fall 2006 for 6 competencies was 2.94. 

2.                          Discussion of Professional Dispositions Assessment.  This rubric is closely aligned with ethical indicators emphasized in the revised conceptual framework for The Citadel’s professional education unit.   While the teacher education division continues to administer this rubric to candidates and instructors in methods courses and at mid-term and the end of the internship, the counseling program administered it at mid-term and at the end of the practicum or internship.  The literacy and educational administration programs chose to administer this disposition instrument at the end of the practicum or internship. 

        There were a total of 121 responses to the mid-term administration of this disposition rubric.  On a 3 point scale, the mean scores on each of the 6 dispositional items ranged from a low of 2.53 to a high of 2.82.   For the 46 candidates responding to this mid-term evaluation, the mean scores range from 2.72 to 2.89.  For the 75 faculty (including cooperating teachers) responses to this mid-term evaluation of candidates, the mean scores ranged from 2.4 to 2.77.

        There were a total of 397 responses to the end of term dispositions evaluation.  Mean scores ranged from 2.87 to 2.95.  For the 129 candidate responses on this end of term assessment, the mean scores ranged from 2.89 to 2.99.  For the 268 faculty (including cooperating teachers), evaluations using this disposition assessment, the mean scores ranged from 2.86 to 2.93. 

        It is not surprising –given The Citadel’s emphasis on honor and core values—that “demonstrate high values and a caring, fair, honest, responsible, and respectful attitude” received the highest rating.  It is also not surprising that the end of term ratings are higher than the mid-term mean scores.

        While the overall scores remain high, it is encouraging to note that faculty appear to be more discriminating in their scoring of the rubrics.  It is heartening to see that the end of term scores—when compared to the scores from the methods courses and from the mid-term evaluation of the internship—suggest that growth is occurring during this culminating experience.              

3.      Discussion of Research Competencies Assessment.  The rubric to assess the research competencies of candidates in The Citadel’s professional education programs was administered to 50 graduate students in the spring of 2007.  Mean scores for the 5 competencies assessed ranged from a low of 2.48 to 2.65 on a three point scale.   Item number 2 (candidate interprets descriptive and inferential data) and item number 1 (candidate demonstrates use of statistical procedures) were the most problematic for candidates.  For item number 1, the instructor assessed 10% of the candidates as unacceptable on this competency.  The instructor assessed 8% of the candidates as unacceptable on item number 2.  Candidates in almost all areas scored lower on item number 2 with MAT students in mathematics, social studies and physical education scoring the lowest. 

        The faculty member teaching these courses and assessing these competencies indicates that candidates often demonstrate a lack of understanding of items 1 and 2 at the beginning of the semester but are able to achieve proficiency by the end of the semester.  Since this assessment is administered at the end of term, the instructor has developed a pre-test to identify the weaknesses of candidates.   The instructor piloted the pre-test this summer and will use it as a diagnostic tool to determine early on which candidates need additional help.  Coordinators need to meet with the instructor of record to determine how to improve candidate performance.   

4.      Discussion of Program Completers Survey.  During the final week of the semester, a survey--designed to assess their level of satisfaction with their programs at The Citadel—was administered to candidates scheduled to complete an advanced  program at the end of the spring 2007 semester.   Using a 5-point scale, 70 completers responded  that they were satisfied or very satisfied with virtually all aspects of their Citadel experience.  Mean scores on the 19 items relating to the knowledge, skills, and dispositions acquired or developed during their tenure at The Citadel ranged form a low of 4.64 to a high of 4.84.  Scores on the 11  questions related to the operational aspects of the program were equally positive ranging from a low of 4.39 to a high of 4.83.  Availability of courses  that you needed in the program and  Advisement provided by your program advisor received the lowest scores.   The OVERALL QUALITY OF YOUR PROGRAM  received a 4.70 rating.

        As an addendum to this general survey, program areas added specific questions related to their program of completers. Based on completers responses, the counseling program is considering adding an additional course focused on ethical and legal issues.

5.      Discussion of Assessments of Teacher Education Candidates.  While data from teacher candidates is included in the unit assessment data discussed above, the significant emphasis given to ADEPT in the assessment of cadet and MAT students merits additional discussion here. 

        Using a 3 point scale, the ADEPT mean sores for cadets on the mid-term internship evaluation ranged from a low of 1.94 (Monitoring, Assessing, and Enhancing Learning) to a high of 2.61.  Mean scores for cadets on end of semester evaluations of internship on 10 ADEPT items range from a low of 2.83 to a high of 3.0.  This data suggests that candidates improved significantly on virtually all items through the course of the semester. 

        A similar, if less dramatic, improvement is evident among MAT students based on the spring 2007 ADEPT data.  Mean scores for MAT candidates on the mid-term evaluation of the internship ranged from a low of 2.44 to a high of 3.0.  This data suggests that candidates improved significantly on virtually all items through the course of the semester.

        A similar pattern emerges for cadet and MAT candidates in the methods courses.  Cadets scores on the ADEPT assessment ranged from a low of 2.05 (Monitoring, Assessing, and Enhancing Learning) to a high of 3.0.  MAT student scores on the methods assessment of the ADEPT competencies ranged from a low of 2.10 to a high of 2.71. 

        Again, this suggests that weaknesses identified during the methods courses result in improvements as both cadet and MAT candidates progress through the program.  It should be noted that both cadet and MAT candidates scored lowest the on the ADEPT items focused on using assessments to enhance student learning (AP3 and AP7).

        In comparing the mean scores across program areas, it appears that the scores for cadet teacher candidates in physical education are lower than candidates in other programs.  This needs to be carefully examined to determine why this is occurring. Are the physical education students being evaluated more critically; is there a programmatic deficiency that needs to be corrected; or are there other reasons for these lower scores?

        While not part of the of the ADEPT evaluation,  the field experience assessment administered to both cadets and MAT students suggest that candidates are not comfortable working student  with special needs and that cadets may need additional assistance in understanding how to assess student learning.

III. General Recommendations.

        Substantial progress has been made during this academic (2006/2007) year in developing and implementing a viable and sustainable assessment system. Congratulations to all of you for this significant accomplishment and thank you for your time, energy, patience, and perseverance. We have achieved much, but we must sustain as well as refine the assessment system we have created. With that in mind, I offer the following recommendations:

  • First of all, the data compiled during the past year must be shared if it is to guide program changes and to ensure that we truly are developing principled educational leaders. For this reason, I am directing each program coordinator to schedule special meetings with division faculty to carefully review all the data compiled during this past year looking for strengths and weaknesses. Several trends are noted in this document, but it focuses primarily on unit data and barely scratches the surface of the information to be found in the unit and program data.
  • The second recommendation is that we keep it simple. I think it is fair to suggest that—in a conscious effort to make sure that every base was covered—we may have developed a more complicated system than is necessary. As we review the data compiled over the past year and reflect upon its usefulness, I suggest that program area faculty concentrate on developing 6 to 8 key assessments that both reflect the uniqueness of specific programs and serve as indicators of the values and objectives of the unit as a whole. Basically, I am suggesting that each program develop 6 to 8 key assessments with some of them serving double duty as unit and program assessments.

Fall 2006 Professional Education Unit Assessment Report

PROFESSIONAL EDUCATION UNIT ASSESSMENT REPORT

FALL 2006

Submitted by Dean Tony Johnson

I.  Development and Implementation of Assessment System for The Citadel's Professional Education Unit

The assessment system for the professional education unit at The Citadel was fully implemented beginning in fall 2006. Implementation included the following initiatives:

1)  Training of all faculty members in the unit and many of the initial and advanced candidates in the unit on the use of LiveText;

2)  Collection of data on all key common assessments and on individual course assessments identified for the a) initial undergraduate; b)  initial graduate MAT; and c)  advanced education programs;

3) Preparation of assessment reports by coordinators and faculty members in each division based on the data obtained and analyzed from the common key assessments and from individual course assessments;

4) Establishment of the Professional Education Board to serve as the assessment committee charged with reviewing program and unit assessment data each semester and making recommendations for changes;

5) Beginning the development of a "Professional Education Assessment Handbook" that will be completed in spring 2007.


II.  Analysis of Data.

A. Citadel PRAXIS II Data from 09/01/05 through 08/31/06.

In reviewing the ETS Institutional Summary Reports for 2005/2006, candidates in The Citadel's advanced professional education programs appear to be performing reasonably well. For example, 25 of 27 (92.6%) Ed leadership candidates passed the PRAXIS II exam (0410). The pass rate is comparable for School of Education counseling candidates with 18 of 19 (94.7%) passing the School Guidance and Counseling exam (0420). LIkewise, 13 of 14 (92.9%) candidates in the Department of Psychology's School Psychologist program passed the appropriate PRAXIS II exam (0400). In reviewing the test category performance for each of these areas, it appears that Citadel candidates scored near the state and national averages on all areas tested. 

Institutional Summary Reports were not available for Literacy presumably since there were fewer than 10 test takers during this period.

Citadel candidates taking the PRAXIS II exams in social studies (0081) did not fare well.  Of the 36 candidates taking the exam during this period, 24 (66.7%) passed. Since, MAT candidates must pass this exam in order to receive an internship placement; the assumption is that the majority of candidates not passing this exam are cadets. Performance data on the separate test categories was not offered in the Institutional Summary report available at the time of this report.

In addition to the Social Studies exam, the Institutional Summary Reports for the content and essay PRAXIS II exams in English (0041 and 0042) indicate that all 7 (100%) Citadel candidates passed the content exam (0041) and 6 of 7 (85.7%) candidates passed the essay exam (0042). Performance data on the 4 test categories suggest that candidates from The Citadel scored above the national and state standards in 2 categories and slightly below the national and state averages on the remaining 2 categories.

In compiling this report, Institutional Summary Reports were not available in Physical Education, Biology, or Mathematics presumable since there were fewer than 10 test takers during this period.

Actions. Ms. Habhegger is charged with meeting with each of the program coordinators (Kathy Brown, Kathy Jones, Josey Templeton, George Williams, Jennifer Altieri, and Mike Politano) to review the data available from the Institutional Summary Reports for both the relevant PRAXIS II exams and Principles of Learning and Teaching (PLT) exams. After reviewing this data, coordinators are to include actions taken or proposed addressing problematic areas in their spring 2007 program area assessment reports.

A major area of concern continues to be cadet performance on the content PRAXIS II exam for social studies (0081). Steps underway to address these concerns--specifically, Dr. Barrett's development of a practice test for cadets and MAT candidates--needs to be accelerated. The importance for such a review test is magnified by the reality that cadets completing their internship graduating in the spring of 2008 must achieve a passing score on the relevant PRAXIS II and PLT exams to graduate.

B. Unit Assessment Data from LiveText

1. Discussion of Content Knowledge Assessment. Assessment of content knowledge competencies was administered in education leadership and counseling courses during the fall semester to both faculty and candidates. Though the N was small, both candidates and faculty gave the candidates high scores for both "demonstrating knowledge of the central concepts and tools of inquiry of the field" (2.83 on a 3.0 scale) and for "demonstrating and applying structures of the field in professional, state and institutional standards." While this is reassuring, the small N suggests that the data may or may not indicate content knowledge competency.

Actions. For this to be a meaningful unit assessment for The Citadel's advanced programs, the rubric needs to be administered in a literacy course, and all faculty and candidates in the specified course need to complete the rubric if the data is to offer a meaningful indicator of candidates' content knowledge competency.

2. Discussion of Portfolio Assessment. All professional education programs (excluding School Psychology) at The Citadel administered a portfolio assessment rubric to 109 candidates and to multiple professors teaching these 109 candidates. A total of 258 candidates and their professors rated each candidate using a 3-point scale on six items related to content knowledge and student learning. The mean score for all responders for the two content knowledge items is 2.91 and 2.88 respectively. The mean scores for all responders for the 4 student learning items are 2.95; 2.94; 2.91; and 2.90. While the significant number of responses makes this data interesting, the extraordinarily high scores raise questions about the utility of the data. It is somewhat ironic that although the candidates' self-evaluation is high-ranging from 2.80 to 2.93 on the six items--faculty evaluation of the candidates was even higher--an average score of 2.94 or higher on all six items ranging from 2.94 to 2.97.

Actions. This could be very important data. The rubric appears to be a reasonable one and the information could be very useful. Still the extraordinary high ratings are problematic. Since 181 of the 258 responders are from teacher education, the division's philosophy of mastery learning may account for these high scores. If so, a justification of the mastery learning philosophy needs to be provided, along with an explanation of how candidates are required to redo their work until mastery has been demonstrated. Given this concern, each coordinator should carefully review the portfolio assessment data from his/her division and report in the spring 2007 assessment report what is being done to address this concern.

3. Discussion of Professional Dispositions Assessment. This rubric is closely aligned with the ethical indicators highlighted in the revised conceptual framework for The Citadel's professional education unit. Using LiveText, this rubric was administered in three classes at the beginning of the fall (2006) semester, to 12 classes at mid-semester, and to 23 classes at the end of the semester. Since only 24 responses were generated from the administration of the rubric at the beginning of the semester, the responses at mid-term and at the end of the semester are likely to yield the most relevant data. The mid-term administration generated 280 responses combined from the 99 candidates, the 149 responses from multiple faculty to these 99 candidates, and from 32 cooperating teachers. The mean score for these mid-term responses ranged from a low of 2.68 (on a 3-point scale) to a high of 2.83.

The end of semester administration of this dispositions rubric generated 465 responses combined from the 169 candidates, the 255 faculty members, and 41 cooperating teachers. The mean score for these end of term responses ranged from a low of 2.68 to a high of 2.92.

It is not surprising--given The Citadel's emphasis on honor and core values--that "Demonstrate high values and a caring, fair, honest, responsible, and respectful attitude" received the highest rating. Though a substantial amount of data has been generated, there is no discernible trend among the three groups (candidates, faculty, and  cooperating teachers) that suggests specific strengths or weaknesses.  Though there are some variations in the range of scores from one program to another, the common characteristic appears to be the extraordinary high scores across all groups: candidates, faculty, and cooperating teachers.

Actions. Professional education faculty, including content area faculty (Barret, Darden, Stepp, etc.) should carefully review program specific responses to this disposition rubric. Program coordinators (Jones, Brown, Williams, Altieri, and Templeton) are expected to share and discuss this data with their colleagues in professional education--including content area faculty--and reach consensus on what merits an acceptable ranking, what merits a target ranking, and what justifies an unacceptable ranking.

4. Discussion of Research Competencies Assessment. The rubric developed to assess the research competencies of candidates in The Citadel's advanced programs was administered to two classes of candidates during the fall (2006) semester. A total of 48 candidates were evaluated by faculty teaching research courses required of all graduate candidates in the School of Education and MAT candidates in physical education. Using a 3.0 scale, candidates were evaluated on 5 research competency items. Mean scores ranged from 2.25 to 2.55. Unlike the high scores on other assessments, more candidates were rated as acceptable, than target.  "Candidates' use of technology" received the highest rating with "candidates' use of statistical procedures and the ability to interpret data" as weakest areas. The faculty teaching these courses indicates that candidates often demonstrate a lack of understanding of statistical procedures, but most are able to achieve proficiency by the end of the semester.

Actions. Coordinators are instructed to share this data with professional education faculty (including content area faculty) and to determine what, if anything, needs to be done to ensure candidate improvement.

5. Discussion of Student Learning Assessment. In addition to being assessed as part of the unit portfolio, a student learning competencies rubric was administered in two sections of a graduate class in counseling during the fall (2006) semester. Though the number participating in this initial administration of this rubric is small, it is a well designed instrument that should provide useful information in the future. As is the case in several of the assessments discussed above, the rankings here are unusually high ranging from 2.83 to 2.91.

Actions. For this to become an effective unit assessment instrument, it must be administered in an expanded list of classes to ensure that it is assessing the level of student learning competencies possessed by all candidates in our graduate professional education programs.

6. Discussion of Assessments of Teacher Education Candidates. In the data analysis provided for the teacher education program, comparisons were made between the performance of the undergraduate cadet candidates and the graduate MAT candidates. The dispositions ratings of candidates assessed in the methods courses were higher for the MAT candidates than for the cadets for five of the six dispositions. The exception was "Exhibits prompt regular attendance, wears professional attire, and communicates in standard English." The rating on each disposition competency was higher at the end of the internship than at mid-term for the undergraduate and the MAT candidates except on one disposition for the cadets--"values diversity and exhibits sensitivity to and respect for cultures." Overall, the ratings were somewhat higher at mid-term and at the end of the internship on some of the dispositions for the cadet candidates than for the MAT candidates. On each of the 11 portfolio assessment competencies at the end of the internship, each of the four cadet candidates received at target rating (3.00). Of the 22 MAT candidates completing the internship, all received an acceptable rating on each competency , except one candidate received an unacceptable rating for "understands policy contexts in the work setting." Most candidates received a target rating on each competency.

Actions. The data and analysis provided on the candidates in the teacher education program are incomplete and inconsistent. The program coordinator must work with Dr. Feldmann to further review the data and complete the analysis.

7. Discussion of Survey Data of Program Completers of MAT and Advanced Programs. Though intended for all candidates completing their graduate programs in professional education at The Citadel, the survey was administered to 13 School Counseling graduates and 22 MAT graduates in December 2006. Designed to assess the level of satisfaction that program completers experienced in their Citadel professional education program, the survey focused on completer's satisfaction with the knowledge, skills, and dispositions acquired during their Citadel experience and their satisfaction with other aspects of their programs such as the quality of advising. Using a 5-point scale, completers responded that they were either satisfied or very satisfied with virtually all aspects of their Citadel experience. It is worth noting that completers ranked the "quality of advising" and the "availability of advising" the lowest of the 32 items surveyed. For MAT candidates, the establishment of a dual advising system involving both education and content area faculty may have already addressed this problem.

Actions. The coordinators for the MAT and counseling programs need to examine advising procedures in these programs since both programs received low ratings on this survey. The information provided in this survey is useful for considering changes for program improvement. For this reason, it is very important to expand the survey to include program completers in all professional education programs. The same survey that is administered to the MAT completers will also be administered to the undergraduate program completers in spring 2007 and comparisons in the data will be made.

III.  General Recommendations

Substantial progress has been made this academic year in both implementing LiveText and in establishing a viable assessment system for The Citadel's professional education unit.  Based on this progress, it is now time to reflect back on this journey and determine the next steps in our preparation for the NCATE visit next fall, but more importantly, to further refine our processes for the never-ending and continuous quest for program and unit improvement. With this in mind, the following suggestions and recommendations are offered:

  • First of all, we need to focus on the transition points for both initial and advanced programs. Our goal must be to establish and institutionalize procedures for effectively monitoring our professional education candidates as they progress from one transition point to the next.

    Though important for all candidates and programs, this is crucially important for our teacher education programs at the cadet or undergraduate level. To make the monitoring of candidate progress more systematic, I suggest that we reestablish and reconstitute the School of Education's Admission and Retention Committee. Under the auspices of this newly reconstituted committee--expanded to be inclusive of all initial teacher education programs--policies and procedures need to be developed formalizing a viable formal admission to teacher education process. We need more formal procedures to ensure that all applicants for formal admission to teacher education, for admission to the internship, and for program completion and recommendation for certification meet or exceed all institutional, state, and national requirements.

    As part of these procedures for monitoring of candidate progress, we could develop a checklist of each applicant and candidate and check off the requirements and accomplishments as they are met and achieved. Upon reviewing the checklist for each candidate, the committee will forward to the Dean of Education or designated Certification officer a list of candidates successfully meeting the requirements necessary for moving on toward the next transition point. Developing procedures such as these will clarify the process for progressing through our programs for both candidates and faculty.

  • A second recommendation--again based on initial experience with LiveText--is that we keep it simple.  As we further refine both program and unit assessments, I suggest that program area faculty concentrate on developing 6 to 8 key assessments that represent the major objectives of your respective programs. It is better to have a few well conceived assessments that provide information useful to you for continuously improving your program than a plethora of meaningless data;
    and

  • My final recommendation is for program coordinators and program area faculty to share and discuss the LiveText data compiled from last semester's implementation. As part of this process, ask what does the data tell us? Does it or will it help us improve our programs or enhance candidate progress? Is it--in a word--meaningful data?


Counselor Education Division Summaries

Fall 2012 Counselor Education Division Program Report

No text or image added.

Spring/Summer 2012 Counselor Education Division Program Report

Fall 2011 Counselor Education Division Program Report

Spring/Summer 2011 Counselor Education Division Program Report

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT SPRING/SUMMER 2011 REPORT SUMMARY

Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for

 EDUC 650/652, 651/653 [School Coun]; EDUC 655 & 656 [Student Affairs &

 College Coun] ); Middle and End of Semester

  2) Counselor Education Student Evaluation Form (completed by course instructors

            for EDUC 514, 515, & 551 [School Coun]; EDUC 514, 551, & 622

            Student Affairs & College Coun] ); End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A

            (completed by candidates for EDUC 629, 650/652, 651/653 [School Coun]; 634,

            655, & 656 [Student Affairs & College Coun] ); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (Faculty Evaluation) (completed

            by candidates for EDUC 629, 650/652, 651/653 [School Coun]; EDUC 634, 655,

            & 656 [Student Affairs & College Coun] ); End of Semester

 

* UNIT Level: (5 instruments)

  5) Research Competencies (completed by core curriculum instructor for EDUC 512 &

            549); End of Semester

  6) Student Learning Competencies (completed by counselor education instructors and

            candidates for EDUC 629 [School Coun]; EDUC 634 [Student Affairs & College

            Coun]); End of Semester

  7) Portfolio Competencies (completed by counselor education instructors for EDUC

            652 & 653 [School Coun]; EDUC 656 [Student Affairs & College Coun] ); End of

            Semester

  8) Professional Dispositions (completed by counselor education instructors and

            candidates for EDUC 514, 515, 551, 629, 650/652, 651/653 [School Coun];

            EDUC 634, 655, & 656 [Student Affairs & College Coun] ); Middle and End of

            Semester

  9) Program Exit Survey [formerly known asProgram Completer Survey: Counseling

            Addition] (completed by counselor education candidates for EDUC 651 & 656);

            End of Semester

 

                                               Results and Conclusions

(Corresponding to numbered items above)

1) An emphasis on assessment and appraisal skills needs to be incorporated in the program.  Due to the number of “NA” responses, additional effort should be made by program faculty to ensure site supervisors fully understand the evaluation form.  Due to lower scores for consultation skills with parents, and knowledge of and/or skill in use of community resources for information and referral purposes, we need to consider adding coursework in these areas.  Examining the scores from a global perspective, practicing school counselors believe The Citadel’s Counselor Education programs (School Counseling and Student Affairs and College Counseling) is sending competent, professional, and prepared students into the marketplace.

2) In the 3 major areas of competency assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance for courses EDUC 514, EDUC 515, and EDUC 551, results rated personal performance the highest, followed next by professional performance and academic performance rated the lowest among the competencies.  Ratings were 100% for “target” or “acceptable” for academic performance and personal performance, while at least 91% were rated either “target” or “acceptable” for professional performance.

3) Given the number of “NA” responses in “General Characteristics of Supervision,” including items “kept sufficiently informed of cases,” “provided direct observation with clients (live/audio/video),”  “used effective aids in supervision (role-playing/recordings, etc.)” and in “encouraged appropriate independence” and responses in “Development of Clinical Skills (treatment, evaluation, and consultation skills),” including items “assisted student in coherent conceptualization of cases,” “was effective in providing suggestions for specific techniques,” and “was effective in helping to develop both short and long-range goals for clients” program faculty need to determine if candidates did not understand the competency and/or why competency is believed not applicable.  Site supervisors need to be encouraged to involve practicum/internship candidates in parent conferences.  It would be helpful to separate data for candidates in practicum vs. internship.

4) Candidates were satisfied with supervision provided by Citadel supervisors.  No patterns of concern were reported.

5) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) at least 80% of candidates were rated “above average” or “average.”   Faculty need to consider strengthening candidates’ research skills.

6) On Student Learning Competencies, 100% of candidates rated themselves “acceptable” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same 4 major areas, 100% of counselor education faculty rated candidates “target” or “acceptable.”  Based on the data, there are no possible changes to be made with the program at this time.

7) On Portfolio Competencies, for the 6 competencies assessed (2 for Content Knowledge, 4 for Student Learning), 100% of candidates were rated by themselves and their instructor as “target” or “acceptable.”  Program faculty are exploring possibility of requiring candidates to begin developing professional portfolio at beginning of the program and adding artifacts for each course.

8) On Professional Disposition Competencies, candidates and course instructors, 100% rated candidates “Consistently Evident” or “Somewhat Evident” for all 6 competencies at midterm and at least 96% were rated “Consistently Evident” or “Somewhat Evident” at end of semester.  Mean scores for all 6 competencies were higher at end of semester than at midterm. Future changes include clarifying competencies that were “Not Rated.”

9) On the Program Exit Survey: Counseling Addition, 100% rated program faculty and overall quality of their counselor education preparation experiences as “excellent,”  “very good,” or “adequate.”  It is worth noting the small sample size = 6 responses and the candidates were all Secondary School Counseling Degree Seeking.  A new course, EDUC-562: Legal & Ethical Leadership Issues in Education” has been added to the curriculum as a required course for the M.Ed. concentration in Student Affairs and College Counseling program (implemented fall 2009) and offered as elective course for School Counseling program.  The possibility of adding coursework in “Classroom and Behavior Management,” and “Consultation Skills” needs to be considered.  The suggestion for developing a School Counselor resource library full of curricula and materials for candidates to check out to use in one-on-one and group counseling sessions is being explored.  Assessment and research courses need to be strengthened for better preparing program completers.

Fall 2010 Counselor Education Division Program Report

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT FALL 2010 REPORT SUMMARY

Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for

            EDUC 650/652, 651/653 [School Coun]; EDUC 655 & 656 [Student Affairs &

College Coun] ); Middle and End of Semester

  2) Counselor Education Student Evaluation Form (completed by course instructors for

            EDUC 514 & 515 [School Coun]; EDUC 514 [Student Affairs & College Coun]

 ); End of Semester     

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A

            (completed by candidates for EDUC 629, 650/652, 651/653 [School Coun];

            EDUC 655 & 656 [Student Affairs & College Coun] ); Middle and End of

            Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC

            629); End of Semester

 

* UNIT Level: (5 instruments)

  5) Research Competencies (completed by core curriculum instructor for EDUC 512 &

            549); End of Semester

  6) Student Learning Competencies (completed by counselor education instructors and

            candidates for EDUC 629); End of Semester

  7) Portfolio Competencies (completed by counselor education instructors for EDUC 651

            & 653); End of Semester

  8) Professional Dispositions (completed by counselor education instructors and

            candidates for EDUC 629, 650/652, & 651/653 [School Coun]; EDUC 655 & 656

            [Student Affairs & College Coun] ); Middle and End of Semester

  9) Fall 2010 Survey of All Advanced Program Completers: Counseling Addition

            (completed by counselor education candidates for EDUC 650 & 651); End of

            Semester

 

                                                      Results and Conclusions

(corresponding to numbered items above)

1) There continue to be too many “NA” responses. Additional effort should be made by program faculty to ensure site supervisors fully understand the evaluation form and for program faculty to consider streamlining the form.  It’s interesting to note that only 11 of the 16 site supervisors completing the form responded having discussed the completed final evaluation with the supervisee.  Examining the scores from a global perspective, practicing school counselors believe The Citadel’s Counselor Education program is sending competent, professional, and prepared students into the marketplace.

2) In the 3 major areas of competency assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance for courses EDUC 514 and EDUC 515, it is not surprising to have results rating professional performance and personal performance lower than competency academic performance for Elementary, Secondary, and Licensure Only Counseling majors.  Many of the candidates are juggling numerous roles in addition to being a graduate student.  Also, many of the candidates experience exposure to professional associations for the first time.  100% of candidates were rated “target” or “acceptable” for all three competencies including academic performance, personal performance, and professional performance.

3) Given the number of “NA” responses in “General Characteristics of Supervision,”  “provided direct observation with clients (live/audio/video),” and “The supervisor used effective aids in supervision (role-playing/recordings, etc.) program faculty need to determine if candidates did not understand the competency and/or why competency is believed not applicable.  It might be helpful to separate data for candidates in practicum vs. internship.

4) Candidates were satisfied with supervision provided by Citadel supervisors.  Some valuable suggestions included for the supervisor to provide more resources for counseling strategies.  No patterns of concern were reported.

5) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use).  The majority of ratings are in the acceptable range.  Overall the scores for all 5 competencies were slightly the lowest since fall 2006.  One plausible explanation is candidates’ enrollment in EDUC 512 during first year in program and the adjustment to the rigors of graduate school.  It is recommended candidates enroll in EDUC 512 during the first 18 hours of graduate work to be used as a baseline course for assessing research competency and then assess them again when enrolled in a content area course.

6) On Student Learning Competencies, 100% of candidates rated themselves “target” or “acceptable” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same 4 major areas, 100% of counselor education faculty rated candidates “target” or “acceptable.”  Since fall 2009 candidates rated highest in understanding diversity of students, families, and communities. Since spring 2010 candidates’ understanding and building upon developmental levels of students has ranked the lowest. Based on the data, there are no changes to be made with the program at this time.

7) On Portfolio Competencies, for the 6 competencies assessed (2 for Content Knowledge, 4 for Student Learning), 100% of candidates were rated by themselves and their instructor as “target” or “acceptable.”  Program faculty are exploring possibility of requiring candidates to begin developing professional portfolio at beginning of the program and adding artifacts for each course.

8) On Professional Disposition Competencies, candidates and course instructors, at least 95% rated candidates “target” or “acceptable” for all 6 competencies at midterm and at least 98% were rated “Consistently Evident” or “Somewhat Evident” at end of semester.  Mean scores for all 6 competencies were higher at end of semester than at midterm. Future changes include clarifying competencies that were “Not Rated.”

9) On the Survey of All Program Completers: Counseling Addition, 100% rated program faculty and overall quality of their counselor education preparation experiences as “excellent.”  However, it is important to note the small sample size of only one candidate.  Assessment and research courses need to be strengthened for better preparing program completers.  Technology resources need to be updated.

Spring/Summer 2010 Counselor Education Division Program Report

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT SPRING/SUMMER 2010 REPORT SUMMARY


Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for EDUC 650, 651, 652, & 653; Middle and End of Semester

  2) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 514; End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A (completed by candidates for EDUC 629, 650, 651, 652, & 653); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC 629, 650, 651, 652, & 653); End of Semester

 

* UNIT Level: (5 instruments)

  5) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  6) Student Learning Competencies (completed by counselor education instructors and candidates for EDUC 629); End of Semester

  7) Portfolio Competencies (completed by counselor education instructors for EDUC 652 & 653); End of Semester

  8) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 629, 650, 651, 652, & 653); Middle and End of Semester

  9) Spring 2010 Survey of All Advanced Program Completers: Counseling Addition (completed by counselor education candidates for EDUC 650, 651, 652, & 653); End of Semester


Results and Conclusions

(Corresponding to numbered items above)

1) An emphasis on assessment and appraisal skills needs to be incorporated in the program.  Due to the number of “NA” responses, additional effort should be made by program faculty to ensure site supervisors fully understand the evaluation form.  Due to lower scores for consultation skills with parents, and knowledge of and/or skill in use of community resources for information and referral purposes, we need to consider adding coursework in these areas.  Examining the scores from a global perspective, practicing school counselors believe The Citadel’s School Counseling program is sending competent, professional, and prepared students into the marketplace.

2) In the 3 major areas of competency assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance for courses EDUC 514, EDUC 515, and EDUC 551, it is not surprising to have results rating professional performance and personal performance lower than competency academic performance.

3) Given the number of “NA” responses in “General Characteristics of Supervision,”  “used effective aids in supervision (role-playing/recordings, etc.)” and in “was effective in facilitating student in other relationships with other professionals in the agency or site” program faculty need to determine if candidates did not understand the competency and/or why competency is believed not applicable.  Site supervisors need to be encouraged to involve practicum/internship candidates in parent conferences.  It would be helpful to separate data for candidates in practicum vs. internship.

4) Candidates were satisfied with supervision provided by Citadel supervisors.  No patterns of concern were reported.

5) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) and at least 94% of candidates were rated “target” or “acceptable.”   Faculty need to consider strengthening candidates’ research skills.

6) On Student Learning Competencies, 100% of candidates rated themselves “acceptable” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same 4 major areas, 100% of counselor education faculty rated candidates “target” or “acceptable.”  Based on the data, there are no possible changes to be made with the program at this time.

7) On Portfolio Competencies, for the 6 competencies assessed (2 for Content Knowledge, 4 for Student Learning), 100% of candidates were rated by themselves and their instructor as “target” or “acceptable.”  Program faculty are exploring possibility of requiring candidates to begin developing professional portfolio at beginning of the program and adding artifacts for each course.

8) On Professional Disposition Competencies, candidates and course instructors, at least 95% rated candidates “Consistently Evident” or “Somewhat Evident” for all 6 competencies at midterm and at least 90% were rated “Consistently Evident” or “Somewhat Evident” at end of semester.  Mean scores for all 6 competencies were higher at end of semester than at midterm. Future changes include clarifying competencies that were “Not Rated.”

9) On the Survey of All Program Completers: Counseling Addition, at least 70% rated program faculty and overall quality of their counselor education preparation experiences as “excellent” or “very good.”  A new course, EDUC-562: Legal & Ethical Leadership Issues in Education” has been added to the curriculum as a required course for the M.Ed. concentration in Student Affairs and College Counseling program (implemented fall 2009) and offered as elective course for School Counseling program.  The possibility of adding coursework in “Classroom and Behavior Management,” and “Consultation Skills” needs to be considered.  Assessment and research courses need to be strengthened for better preparing program completers.

Fall 2009 Counselor Education Division Program Report

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT FALL 2009 REPORT SUMMARY


Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for EDUC 650 & 651;

            Middle and End of Semester

  2) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 514;

            End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A (completed by candidates for EDUC

            629, 650, & 651); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC 629, 650, & 651);

            End of Semester

 

* UNIT Level: (5 instruments)

  5) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  6) Student Learning Competencies (completed by counselor education instructors and candidates for EDUC 629);

            End of Semester

  7) Portfolio Competencies (completed by counselor education instructors for EDUC 650 & 651); End of Semester

  8) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 629, 650, &

            651); Middle and End of Semester

  9) Fall 2008 Survey of All Advanced Program Completers: Counseling Addition (completed by counselor

            education candidates for EDUC 650 & 651); End of Semester


Results and Conclusions

(corresponding to numbered items above)

1) An improvement on assessment and appraisal skills in comparison to former semesters is noted. There still exist too many “NA” responses. Additional effort should be made by program faculty to ensure site supervisors fully understand the evaluation form.  Examining the scores from a global perspective, practicing school counselors believe The Citadel’s School Counseling program is sending competent, professional, and prepared students into the marketplace.

2) In the 3 major areas of competency assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance for courses EDUC 514, EDUC 515, and EDUC 551, it is not surprising to have results rating professional performance and personal performance lower than competency academic performance for Elementary, Secondary, and Licensure Only Counseling majors.  Many of the candidates are juggling numerous roles in addition to being a graduate student.  Also, many of the candidates experience exposure to professional associations for the first time.

3) Given the number of “NA” responses in “General Characteristics of Supervision,”  “The supervisor used effective aids in supervision (role-playing/recordings, etc.) program faculty need to determine if candidates did not understand the competency and/or why competency is believed not applicable.  Site supervisors need to be encouraged to involve practicum/internship candidates in parent/teacher conferences, a common thread from former semester.  It might be helpful to separate data for candidates in practicum vs. internship.

4) Candidates were satisfied with supervision provided by Citadel supervisors.  Some valuable suggestions included better preparing interns prior to semester via email with deadline dates, more preparation at the beginning of semester for portfolio expectations, more preparation for CPCE.  Since this was the first semester administering the CPCE, more time can be spent during practicum to prepare interns.  No patterns of concern were reported.

5) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) and at least 84% of candidates were rated “target” or “acceptable.”   Faculty need to consider strengthening candidates’ research skills.

6) On Student Learning Competencies, 100% of candidates rated themselves “target” or “acceptable” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same 4 major areas, 100% of counselor education faculty rated candidates “target” or “acceptable.”  Based on the data, there are no possible changes to be made with the program at this time.

7) On Portfolio Competencies, for the 6 competencies assessed (2 for Content Knowledge, 4 for Student Learning), 100% of candidates were rated by themselves and their instructor as “target” or “acceptable.”  Program faculty are exploring possibility of requiring candidates to begin developing professional portfolio at beginning of the program and adding artifacts for each course.

8) On Professional Disposition Competencies, candidates and course instructors, at least 90% rated candidates “target” or “acceptable” for all 6 competencies at midterm and at least 90% were rated “Consistently Evident” or “Somewhat Evident” at end of semester.  Mean scores for all 6 competencies were higher at end of semester than at midterm. Future changes include clarifying competencies that were “Not Rated.”

9) On the Survey of All Program Completers: Counseling Addition, at least 85% rated program faculty and overall quality of their counselor education preparation experiences as “excellent” or “very good.”  The possibility of adding coursework in “Classroom and Behavior Management,”  “Ethical and Legal Issues,” and “Consultation Skills” needs to be considered.  Assessment and research courses need to be strengthened for better preparing program completers.

Fall 2008 Counselor Education Division Program Report

 

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT FALL 2008 REPORT SUMMARY

 

Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for EDUC 650 & 651;

                Middle and End of Semester

  2) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 514;

                End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A (completed by candidates for EDUC

                629, 650, & 651); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC 629, 650, & 651);

                End of Semester

 

* UNIT Level: (5 instruments)

  5) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  6) Student Learning Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  7) Portfolio Competencies (completed by counselor education instructors for EDUC 650 & 651); End of Semester

  8) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 629, 650, &

                651); Middle and End of Semester

  9) Fall 2008 Survey of All Advanced Program Completers: Counseling Addition (completed by counselor

                education candidates for EDUC 650 & 651); End of Semester

 

                                                      Results and Conclusions (corresponding to numbered items above)

1) An emphasis on assessment and appraisal skills needs to be incorporated in the program.  Due to the number of “NA” responses, additional effort should be made by program faculty to ensure site supervisors fully understand the evaluation form.  Due to lower scores for consultation skills with parents and teachers and classroom management skills, need to consider adding coursework in these areas.  Examining the scores from a global perspective, practicing school counselors believe The Citadel’s School Counseling program is sending competent, professional, and prepared students into the marketplace.

2) In the 3 major areas of competency assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance for courses EDUC 514, EDUC 515, and EDUC 551, it is not surprising to have results rating professional performance and personal performance lower than competency academic performance for Secondary Counseling majors.  Surprisingly, Elementary Counseling majors were rated highest for competency professional performance and rated equally lower for academic performance and personal performance.

3) Given the number of “NA” responses in “General Characteristics of Supervision,”  “Development of Clinical Skills,” and in “Practicum/Internship Site,” program faculty need to determine if candidates did not understand the competency and/or why competency is believed not applicable.  Site supervisors need to be encouraged to involve practicum/internship candidates in parent/teacher conferences.  It would be helpful to separate data for candidates in practicum vs. internship.

4) Candidates were satisfied with supervision provided by Citadel supervisors.  No patterns of concern were reported.

5) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) and at least 93% of candidates were rated “target” or “acceptable.”   Faculty need to consider strengthening candidates’ research skills.

6) On Student Learning Competencies, 100% of candidates rated themselves “target” or “acceptable” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same 4 major areas, 100% of counselor education faculty rated candidates “target” or “acceptable.”  Based on the data, there are no possible changes to be made with the program at this time.

7) On Portfolio Competencies, for the 6 competencies assessed (2 for Content Knowledge, 4 for Student Learning), 100% of candidates were rated by themselves and their instructor as “target” or “acceptable.”  Program faculty are exploring possibility of requiring candidates to begin developing professional portfolio at beginning of the program and adding artifacts for each course.

8) On Professional Disposition Competencies, candidates and course instructors, at least 95% rated candidates “target” or “acceptable” for all 6 competencies at midterm and at least 95% were rated “Consistently Evident” or “Somewhat Evident” at end of semester.  Mean scores for 5 of 6 competencies were higher at end of semester than at midterm. Future changes include clarifying 3 of the 6 competencies that were “Not Rated.”

9) On the Survey of All Program Completers: Counseling Addition, at least 70% rated program faculty and overall quality of their counselor education preparation experiences as “excellent” or “very good.”  The possibility of adding coursework in “Classroom and Behavior Management,”  “Ethical and Legal Issues,” and “Consultation Skills” needs to be considered.  Assessment and research courses need to be strengthened for better preparing program completers.

 

Summer 2008 Counselor Education Division Program Report

 

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT SUMMER I and II 2008 REPORT SUMMARY

 

Assessment Instruments Administered

 

* PROGRAM Level: (1 instrument)

  1) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 514 & 515;   End of Semester

* UNIT Level: (2 instruments)

  2) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  3) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 514, 515,

                & 551 – Summer I and II Combined); End of Semester

 

                                                      Results and Conclusions (corresponding to numbered items above)

2) On the Student Counseling Student Evaluation Form, three major areas of competency are to be assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance.  Evaluation forms are completed by instructors for courses EDUC 514, EDUC 515, and EDUC 551.  EDUC 551 was not offered summer 2008.  Among all responses for the 17 candidates evaluated, at least 90% were rated “target” or “acceptable” for academic performance and personal performance and 100% were rated “target” or “acceptable” for academic performance. Since both EDUC 514 and EDUC 515 are typically taken by candidates early in the program, it is not surprising to have results rating professional performance and personal performance lower than the competency academic performance.

3) On Research Competencies, a total of 41 School of Education candidates, including 17 (41.46%) Counseling responses in 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) 100% of candidates were rated “target” or “acceptable.”   

4) On Professional Disposition Competencies, candidates and course instructors, 100% rated candidates “Consistently Evident” or “Somewhat Evident” for all 6 competencies.  Future changes include separating the data for Counselor Education candidates and School Psychology candidates.

 

Spring 2008 Counselor Education Division Program Report

 

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT SPRING 2008 REPORT SUMMARY

 

Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for EDUC 650 & 651;

                Middle and End of Semester

  2) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 514;

                End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A (completed by candidates for EDUC

                629, 650, & 651); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC 629, 650, & 651);

                End of Semester

* UNIT Level: (6 instruments)

  5) Content Knowledge Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  6) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  7) Student Learning Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  8) Portfolio Competencies (completed by counselor education instructors for EDUC 650 & 651); End of Semester

  9) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 629, 650, &

                651); Middle and End of Semester

10) Spring 2007 Survey of All Advanced Program Completers: Counseling Addition (completed by counselor

                education candidates for EDUC 650 & 651); End of Semester

 

                                                      Results and Conclusions (corresponding to numbered items above)

1) An emphasis on assessment and appraisal skills needs to be incorporated in the program.  Due to the number of “NA” responses, additional effort should be made by program faculty to ensure site supervisors fully understand the evaluation form.  Examining the scores from a global perspective, practicing school counselors believe The Citadel’s School Counseling program is sending competent, professional, and prepared students into the marketplace.

2) Three major areas of competency are to be assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance.  Evaluation forms are completed by instructors for courses EDUC 514, EDUC 515, and EDUC 551.  EDUC 515 and EDUC 551 were not offered spring 2008.

3) Given the number of “NA” responses in “General Characteristics of Supervision,”  “Development of Clinical Skills,” and in “Practicum/Internship Site,” program faculty need to determine if candidates did not understand the competency and/or why competency is believed not applicable.  The possibility of adding additional coursework such as “Consultation Skills” and “Counselor Supervision” needs attention.  It would be helpful to separate data for candidates in practicum vs. internship.

4) Candidates were satisfied with supervision provided by Citadel supervisors.  No patterns of concern were reported.

5) On Content Knowledge Competencies the major areas of “Concepts and Tools” and “Professional Standards,” 100% of candidates were rated “target” or “acceptable.”

6) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) and at least 90% of candidates were rated “target” or “acceptable.”   Faculty need to consider strengthening candidates’ research skills.

7) On Student Learning Competencies, 100% of candidates rated themselves “target” or “acceptable” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same 4 major areas, 100% of counselor education faculty rated candidates “target” or “acceptable.”  Based on the data, there are no possible changes to be made with the program at this time.

8) On Portfolio Competencies, for the 6 competencies assessed (2 for Content Knowledge, 4 for Student Learning), 100% of candidates were rated “target” or “acceptable.”  Program faculty are exploring possibility of requiring candidates to begin developing professional portfolio at beginning of the program and adding artifacts for each course.

9) On Professional Disposition Competencies, candidates and course instructors, at lest 95% rated candidates “target” or “acceptable” for 5 of the 6 competencies.  Future changes include clarifying 4 of the 6 competencies that were  “Not Rated.”

10) On the Survey of All Program Completers: Counseling Addition, at least 75% rated program faculty and overall quality of their counselor education preparation experiences as “excellent” or “very good.”  The possibility of adding coursework in “Classroom and Behavior Management” needs to be considered.  Program faculty need to help candidates attain self-understanding and acquire research skills, need to interpret plausible explanations for lower ratings pertaining to issues of social and cultural diversity and knowledge and application of group counseling theories with clients in the school setting.  There appears to be greater satisfaction among degree seeking students than certification only students.

 

Fall 2007 Counselor Education Division Program Report

 

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT FALL 2007 REPORT SUMMARY

 

Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 514 & 515);

                End of Semester

  2) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for EDUC 650 & 651);

                Middle and End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A (completed by candidates for EDUC

                629, 650, & 651); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC 551, 650, & 651);

                End of Semester

* UNIT Level: (6 instruments)

  5) Content Knowledge Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  6) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 629, 650, &

                651); Middle and End of Semester

  7) Portfolio Competencies (completed by counselor education instructors for EDUC 650 & 651); End of Semester

  8) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  9) Student Learning Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

10) Fall 2007 Survey of All Advanced Program Completers: Counseling Addition (completed by counselor education

                candidates for EDUC 650 & 651); End of Semester

 

                                                      Results and Conclusions

(corresponding to numbered items above)

1) In the 3 major areas of competency assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance, over 95% of candidates were rated “target” or “acceptable,” and 4.08% was unacceptable for personal performance and professional performance.  Since both EDUC 514 and 515 are taken by candidates early in program, it is not surprising to have results rating personal and professional performance lower. 

2) An emphasis on assessment and appraisal skills needs to be incorporated in the program.  Given the number of “NA” responses in “B) Consulting and Coordinating Skills” and “C) Assessment and Appraisal Skills,” program faculty need to determine if site supervisors did not understand the competency and/or why competency is believed not applicable.

3) Given the number of “NA” responses in “Development of Clinical Skills” and “Practicum/Internship Site,” program faculty need to determine if site supervisors did not understand the competency and/or why competency is believed not applicable.  The possibility of adding courses in “Consultation Skills” and “Counselor Supervision” needs attention.

4) Overall, candidates were satisfied with supervision provided by Citadel supervisors.  One area of concern was confusion regarding course expectations for several of the candidates that may relate to hiring new faculty supervisors.

5) On Content Knowledge Competencies the major areas of “Concepts and Tools” and “Professional Standards,” 100% of candidates were rated “target” or “acceptable.”

6) On Professional Disposition Competencies, candidates and course instructors, 100% rated candidates “target” or “acceptable” for all six competencies.  Mean scores increased from middle to end of semester for all 6 competencies.

7) On Portfolio Competencies, for the 6 competencies assessed, 90% of candidates were rated “target” or “acceptable” for Content Knowledge and 90% were rated “target” or “acceptable” for Student Learning.

8) On Research Competencies, of the 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, & technology use) candidates were rated highest in technology use, published research analysis, and the American Psychological Association format.  Assessment of candidates’ research competencies skills prior to taking research course indicated difficulty in statistical procedures and descriptive/inferential data interpretation and improved to acceptable and target levels at end of course.

9) On Student Learning Competencies, 100% of counselor education candidates rated themselves “target” or “acceptable” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly, 100% of faculty rated candidates “target” or “acceptable” for all 4 areas.

10) On the Survey of All Program Completers: Counseling Addition, 100% of school counseling program completers rated program faculty and overall quality of their counselor education preparation as “excellent” or “very good” on all 6 items.  On 15 of the 16 items where program completers rated the program curriculum, 100% rated their experiences as “strong” or “good.”  The one rating of “adequate” was given to “classroom and behavior management.”   The possibility of adding coursework in “Classroom and Behavior Management” and “Ethical and Legal Issues” needs to be considered.  Also, the assessment and research courses need to be strengthened for better preparing school counseling candidates completing the program.

Summer 2007 Counselor Education Division Program Report

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT SUMMER 2007 REPORT SUMMARY

Assessment Instruments Administered

* PROGRAM Level: (1 instrument)

1) School Counseling Student Evaluation Form (completed by core curriculum instructor for EDUC 514 and

              counselor education instructor for EDUC 515); End of Semester

* UNIT Level: (5 instruments)

  2) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  3) Professional Disposition Competencies (completed by course instructor and candidates for EDUC 514 & 551);

                End of Semester

  4) Summer 2007 Survey of All Counselor Education (Elementary) Graduates (completed by counselor

                education graduates -1 year post program completion)

  5) Summer 2007 Survey of All Counselor Education (Secondary) Graduates (completed by counselor education

                graduates -1 year post program completion)

  6) Summer 2007 Survey of All Program Completers, All Counselor Education Employers (completed by counselor

                employers)

Results and Conclusions

(corresponding to numbered items above)

1) Three major areas of candidate competency assessed by course instructor include (a) academic performance, (b) personal performance, and (c) professional performance.  Evaluation forms were completed by course instructors of EDUC 514 for 13 candidates and EDUC 515 for 21 candidates, totaling 34 candidates evaluated.  The overall mean scores of all 34 responses for the 3 competencies include:  (a) academic performance (2.3), (b) personal performance (2.24), and (c) professional performance.  Among 16 of 34 candidates registered as Counseling Majors, the overall mean scores for the 3 competencies include:  (a) academic performance (2.13), (b) personal performance (2.13), and (c) professional performance (2.00).  Among the 18 candidates registered as Other Majors, the overall mean scores for the 3 competencies include:  (a) academic performance (2.50), (b) personal performance (2.33), and (c) professional performance (2.00).  Among all responses for 34 candidates, 94.12% were rated “target” or “acceptable” for academic performance, 97.06% were rated “target” or “acceptable” for personal performance, and 61.76% were rated “target” or “acceptable” for professional performance.  Among 16 Counseling Major candidates, 93.75% were rated “target” or “acceptable” for academic performance, 93.75% were rated “target” or “acceptable” for personal performance, and 81.25% were rated “target” or “acceptable” for professional performance.  Among 18 Other Major candidates, 94.45% were rated “target” or “acceptable” for academic performance, 100% were rated “target” or “acceptable” for personal performance, and 44.44% were rated “target” or “acceptable” for professional performance.

2) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) 100% of candidates were rated “target” or “acceptable.”   A Research Competency Pretest and Posttest was created and will be implemented in fall 007.  Results indicate one semester may not be sufficient for candidates to acquire research methodology and statistical knowledge, interpretation, and application.

3) On Professional Disposition Competencies, candidates and course instructors, 100% rated candidates “target” or “acceptable” for all six competencies.  Among 34 responses, 15 were completed by Counseling candidates and 19 were completed by Counseling faculty.  Also, among the total 34 responses, 21 were rated pertaining to Secondary Counselor Education candidates and 13 were rated pertaining to Elementary Counselor Education candidates. Whereas 94.74% Counseling faculty rated candidates “target,” 80% of all Counseling candidates rated themselves “target.”

4) On the Summer 2007 Survey of All Program Completers, All Counselor Education (Elementary) Graduates -1 Year Post Program Completion, 100% (5 total responses) of the graduates rated the 19 competencies as “very satisfied” or “satisfied.”

5)  On the Summer 2007 Survey of All Program Completers, All Counselor Education (Secondary) Graduates -1 Year Post Program Completion, 100% (7 total responses) of the graduates rated 17 of the 19 competencies as “very satisfied” or “satisfied” and 14.29% of the graduates rated items # 6 (Ability to use technology in my work) and #12 (Understanding of the policy contexts in the school setting) “dissatisfied.”

6) On the Summer 2007 Survey of All Program Completers, All Counselor Educator Employers, 100% (total of one response) of the 19 competencies were rated “very satisfied” by the employer who completed the survey.  Future plans include focusing on ways to receive a higher response rate for completed surveys.

 

Spring 2007 Counselor Education Division Program Report Summary

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT SPRING 2007 REPORT SUMMARY

Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for EDUC 650 & 651;

                Middle and End of Semester

  2) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 514, 515, &

                551; End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A (completed by candidates for EDUC

                629, 650, & 651); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC 629, 650, & 651);

                End of Semester

* UNIT Level: (6 instruments)

  5) Content Knowledge Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  6) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  7) Student Learning Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  8) Portfolio Competencies (completed by counselor education instructors for EDUC 650 & 651); End of Semester

  9) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 629, 650, &

                651); Middle and End of Semester

10) Spring 2007 Survey of All Advanced Program Completers: Counseling Addition (completed by counselor

                education candidates for EDUC 650 & 651); End of Semester

Results and Conclusions (corresponding to numbered items above)

1) An emphasis on assessment and appraisal skills needs to be incorporated in the program.  Due to the number of “NA” responses, additional effort should be made by program faculty to ensure site supervisors fully understand the evaluation form.  Examining the scores from a global perspective, practicing school counselors believe The Citadel’s School Counseling program is sending competent, professional, and prepared students into the marketplace.

2) Three major areas of competency are to be assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance.  Evaluation forms were not completed by EDUC 514 instructor, EDUC 515 was not offered Spring 2007, and EDUC 551 was not included on “Who, What, When” form for Spring 2007.

3) Given the number of “NA” responses in “General Characteristics of Supervision,”  “Development of Clinical Skills,” and in “Practicum/Internship Site,” program faculty need to determine if candidates did not understand the competency and/or why competency is believed not applicable.  The possibility of adding additional coursework such as “Consultation Skills” needs attention.  It would be helpful to separate data for candidates in practicum vs. internship.

4) Candidates were satisfied with supervision provided by Citadel supervisors.  No areas of concern were reported.

5) On Content Knowledge Competencies the major areas of “Concepts and Tools” and “Professional Standards,” 100% of candidates were rated “target.”

6) On Research Competencies, 5 major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, APA format, & technology use) 100% of candidates were rated “target” or “acceptable.”   Faculty need to consider assessing candidates’ research skills prior to taking research courses.

7) On Student Learning Competencies, 100% of candidates rated themselves “target” for the 4 major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same 4 major areas, 100% of counselor education faculty rated candidates “target.”  Based on the data, there are no possible changes to be made with the program at this time.

8) On Portfolio Competencies, for the 6 competencies assessed (2 for Content Knowledge, 4 for Student Learning), 92% of candidates were rated “target” or “acceptable.”  Beginning spring semester 2007, all candidates were required to submit their professional portfolios electronically.  Program faculty are exploring possibility of requiring candidates to begin developing professional portfolio at beginning of the program and adding artifacts for each course.

9) On Professional Disposition Competencies, candidates and course instructors, 100% rated candidates “target” or “acceptable” for all six competencies.  Future changes include compiling data completed by counselor education  instructors separately from data completed by candidates to compare instructor ratings with candidate ratings.

10) On the Survey of All Program Completers: Counseling Addition, 100% rated program faculty and overall quality of their counselor education preparation experiences as “excellent” or “very good.”  The possibility of adding coursework in “Ethical and Legal Issues” needs to be considered.  Program faculty need to help candidates attain self-understanding and become involved in professional organizations, need to interpret plausible explanations for lower ratings pertaining to the scheduling of available courses (timing and location), admission procedures, and program advising and how to strengthen these processes.

 

Fall 2006 Counselor Education Division Program Report Summary

 

Professional Education Unit, Division of Counselor Education

PROGRAM ASSESSMENT FALL 2006 REPORT SUMMARY


Assessment Instruments Administered

* PROGRAM Level: (4 instruments)

  1) Final Evaluation of Counselor Trainee—Form A (completed by site supervisors for EDUC 650 & 651;

                Middle and End of Semester

  2) School Counseling Student Evaluation Form (completed by counselor education instructors for EDUC 515 & 551);

                End of Semester

  3) Candidate’s Evaluation of Field Experience and Site Supervisor—Form A (completed by candidates for EDUC

                629, 650, & 651); Middle and End of Semester

  4) Candidate’s Evaluation of Faculty Supervisor (completed by candidates for EDUC 515, 551, 650, & 651);

                End of Semester

* UNIT Level: (7 instruments)

  5) Content Knowledge Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  6) Research Competencies (completed by core curriculum instructor for EDUC 512 & 549); End of Semester

  7) Student Learning Competencies (completed by counselor education instructors and candidates for EDUC 629);

                End of Semester

  8) Portfolio Competencies (completed by counselor education instructors for EDUC 650 & 651); End of Semester

  9) Professional Dispositions (completed by counselor education instructors and candidates for EDUC 629, 650, &

                651); Middle and End of Semester

10) Fall 2006 Survey of All Advanced Program Completers (completed by counselor education candidates for EDUC

                650 & 651); End of Semester

11) Fall 2006 Survey of All Advanced Program Completers: Counseling Addition (completed by counselor education

                candidates for EDUC 650 & 651); End of Semester

Results and Conclusions (corresponding to numbered items above)

1) An emphasis on assessment and appraisal skills needs to be incorporated in the program.  Given the number of “NA” responses in “C) Assessment and Appraisal Skills” and “B) Consulting and Coordinating Skills,” program faculty need to determine if site supervisors did not understand the competency and/or why competency is believed not applicable.

2) In the three major areas of competency assessed by instructor, including (a) academic performance, (b) personal performance, and (c) professional performance, 100% of candidates were rated “target” or “acceptable.”

3) Given the number of “NA” responses in “General Characteristics of Supervision” and “Development of Clinical Skills,” program faculty need to determine if site supervisors did not understand the competency and/or why competency is believed not applicable.  The possibility of adding additional coursework such as “Consultation Skills” needs attention.

4) Overall, candidates were satisfied with supervision provided by Citadel supervisors.  One area of concern was amount of feedback provided supervisor regarding taped sessions and transcripts submitted by candidate.

5) On Content Knowledge Competencies the major areas of “Concepts and Tools” and “Professional Standards,” 100% of candidates were rated “target” or “acceptable.”

6) On Research Competencies, five major areas on which candidates were assessed (statistical procedures, descriptive and inferential data interpretation, published research analysis, & technology use) 100% of candidates were rated “target” or “acceptable,” while the fifth major area, American Psychological Association format, only 30% of candidates were rated “target” or “acceptable.”  Faculty need to consider assessing candidates’ research skills prior to taking research courses and course instructor needs to consider incorporating APA format requirements for both EDUC 512 and 549 courses.

7) On Student Learning Competencies, 100% of candidates rated themselves “target” or “acceptable” for the four major areas including (a) environment for student learning, (b) development level of students, (c) diversity, and (d) policy.  Similarly on the same four major areas, 100% of counselor education faculty rated candidates “target” or “acceptable.”

8) On Portfolio Competencies, for the six competencies assessed, 100% of candidates were rated “target” or “acceptable” for Content Knowledge and 96% were rated “target” or “acceptable” for Student Learning. The response “NA” was given to one candidate on one of the competencies.  Program faculty have agreed to require candidates to submit their professional portfolios electronically beginning spring semester 2007.

9) On Professional Disposition Competencies, candidates and course instructors, 100% rated candidates “target” or “acceptable” for all six competencies.

10) On Survey of All Advanced Program Completers, 100% of school counseling program completers rated “#30 Overall Quality of Your Program” as either “Very Satisfied” (61.54%) or “Satisfied” (38.46%). 

11) On the Survey of All Program Completers: Counseling Addition, at least 84% rated program faculty and overall quality of their counselor education preparation experiences as “excellent” or “very good.”  The possibility of adding coursework in “Classroom and Behavior Management” and “Ethical and Legal Issues” needs to be considered.

Educational Leadership Division Summaries

Fall 2012 Educational Leadership Division Summaries

Spring / Summer 2012 Educational Leadership Division Summaries

Spring 2011 and Summer 2011 Educational Leadership Division Program Report

Spring/Summer 2011

THE CITADEL

PROFESSIONAL EDUCATION UNIT

Professional Dispositions Report—Educational Leadership


Name of Assessment Instrument:  Professional Dispositions (completed by candidate, on-site supervisor, college supervisor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Spring/Summer 2011

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

The Professional Dispositions Instrument collects professional ethics data on six areas that influence candidate behaviors:  (a) reflective practices, (b) commitment to a safe, supportive learning environment, (c) high values and a caring, fair, honest, responsible, respectful attitude, (d) rapport with students, families, colleagues, community, (e) values diversity and exhibits sensitivity to and respect for cultures, and (f) exhibits prompt regular attendance, wears professional attire, and communicates in standard English. Responses range from acceptable to target.

The spring/summer 2011 end-of-semester Professional Dispositions results for the Educational Leadership Division showed a high response rate—83 responses that represent 23.7% for spring and 130 responses for summer that represent 43.9% of all School of Education Responses. For spring, means for the six dispositions range from a low of 2.82, where 99% of responses were somewhat or consistently evident for Disposition 6, Exhibits prompt, regular attendance, wears professional attire and communicates in standard English to a high of 2.92, where 100% of responses were somewhat or consistently evident for Disposition 1. Disposition 2, Demonstrates commitment to a safe, supportive learning environment  merited a mean of 2.91, which represents 91.5% somewhat or consistently evident;Disposition 3, Demonstrates high values and a caring, fair, honest, responsible, and respectful attitude, showed a mean score of 2.89, which represents 100% somewhat or consistently evident; Disposition 5, demonstrates high values and a caring, fair, honest, responsible, and respectful attitude, indicated a mean score of 2.86, somewhat or consistently evidentwhich represents 100% somewhat or consistently evident; and Disposition 4, establishes rapport with students, families, colleagues, and community, generated a mean of 2.85, somewhat or consistently evident which represents 95.1%.

The summer 2011 data indicated a range of 2.78 for Disposition 1 to a high of 2.87 for Disposition 3.  Mean scores of 2.85, for Disposition 5, 2.86 for Dispositions 2 and 4 and 2.87 for Disposition 3.  The percentages significantly indicated a low for Disposition 2 at 89.1% to a high of 98.4% for Disposition 1.  All other percentages fell within the range of 89% to 97%.   

The spring/summer results reflect comparative data for monitoring future trends. The results reflected in the table below shows that the performance of DoEL candidates parallel the performance of SOE candidates is significant ± .5.

SOE

2.81

2.91

2.92

2.87

2.90

2.87

Educational Leadership

2.85

2.89

2.88

2.86

2.86

2.82

  In summary, of the six major areas on which candidates were assessed, DoEL candidates were consistently rated highest in “2) Commitment to save and supportive learning environment “3) High Values.” and “5) “Values Diversity”. The weakest areas were “1) Reflective Practices and “6) Exhibits prompt attendance, wears professional attire, and communicates in standard English.

Conclusions and Possible Changes to be Made based on the Data.

Mean results show that candidates possess the professional dispositions of principled leaders; however, the DoEL will continue to make revisions to the program to address the weak areas— exhibits prompt and regular attendance, wears professional attire and communicates in standard English and applies reflective practices. 


UNIT ASSESSMENT REPORT

SPRING 2011

THE CITADEL

PROFESSIONAL EDUCATION UNIT


Name of Assessment Instrument:  Needs Assessment

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Spring 2011

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  Educ 524 – Techniques of School Supervision

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used): 

The Needs Assessment Instrument measures candidate procedural knowledge of collecting and analyzing quantitative and qualitative data to improve teaching and learning needs of the diverse student population.  Candidates are required to analyze information data, interpret findings and select a problem, and define the problem.

Number of Respondents:  8

Number of Competencies Assessed

  1. Analyze Information Data
  2. Interpret Findings and Select a Problem
  3. Define the Problem

Scale Used: Target = 3 points; Acceptable – 2 points; Unacceptable = 1 point or NA = (not applicable).”  NOTES:  NA responses were omitted from the mean calculations.  Non-responders were omitted from all calculations.

Analysis of Data: 

A. Analyze Information Data
On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable.” The overall mean score was 2.88.

B.  Interpret Findings and Select a Problem

On the competency assessed, at least 100% of the candidates were rated “target”.

The overall mean score was 3.00.

C.    Define the Problem

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable”. The overall mean score was 2.12.

Conclusions and Possible Changes to be Made Based on the Data:

Course instructors will collaborate with the instructor for Educational Research to ensure candidates have an additional opportunity to interpret and synthesize data to identify and define the central problem.


UNIT ASSESSMENT REPORT

SPRING 2011

THE CITADEL

PROFESSIONAL EDUCATION UNIT

 

Name of Assessment Instrument:  Action Research Proposal

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Spring 2011

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  Educ 524 – Techniques of School Supervision

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used): 

The Action Research Proposal measures candidate procedural knowledge of collecting and analyzing quantitative and qualitative data to improve teaching and learning needs of the diverse student population.  Candidates are required to analyze information data, interpret findings and select a problem, and define the problem in a Needs Assessment, investigate research in a literature review, and prepare an action research proposal for improving teaching and/or learning.

Number of Respondents:  11

Number of Competencies Assessed

  1. Academic Task
  2. Action Plan Matrix
  3. Reflective Narrative

Scale Used: Target = 3 points; Acceptable – 2 points; Unacceptable = 1 point or NA = (not applicable).”  NOTES:  NA responses were omitted from the mean calculations.  Non-responders were omitted from all calculations.

Analysis of Data: 

A. Academic Task
On the competency assessed, at least 100% of the candidates were rated “target”.” The overall mean score was 3.00.

B.  Action Plan Matrix

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable”.

The overall mean score was 2.73.

C.    Reflective Narrative

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable”. The overall mean score was 2.36.

Conclusions and Possible Changes to be Made Based on the Data:

Course instructors will collaborate with the instructor for Educational Research to ensure candidates have an additional opportunity to synthesize, in a reflective narrative, how data results can be used for making curricular/pedagogical decisions.

Fall 2010 Educational Leadership Division Program Report

Fall 2010

THE CITADEL

PROFESSIONAL EDUCATION UNIT

Professional Dispositions Report—Educational Leadership


Name of Assessment Instrument:  Professional Dispositions (completed by candidate, on-site supervisor, college supervisor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Fall 2010

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

The Professional Dispositions Instrument collects professional ethics data on six areas that influence candidate behaviors:  (a) reflective practices, (b) commitment to a safe, supportive learning environment, (c) high values and a caring, fair, honest, responsible, respectful attitude, (d) rapport with students, families, colleagues, community, (e) values diversity and exhibits sensitivity to and respect for cultures, and (f) exhibits prompt regular attendance, wears professional attire, and communicates in standard English. Responses range from acceptable to target.

The fall 2010 end-of-semester Professional Dispositions results for the Educational Leadership Division showed an increase in previous terms with a response rate—107 responses that represent 26.62% of all School of Education Responses.  Means for the six dispositions range from a low of 2.76, where 96.3% of responses were somewhat or consistently evident for Disposition 4, Builds rapport with students, families, colleagues, and community to a high of 2.91, where 95.3% of responses indicated somewhat or consistently evident for Disposition 2, Demonstrates commitment to a safe, supportive learning environment.  Disposition 3, Demonstrates high values and a caring, fair, honest, responsible, and respectful attitude merited a mean score of 2.90 or 98.1% where the responses fell within somewhat or consistently evident.  Disposition 5, Values diversity and exhibits sensitivity to and respect for cultures showed a mean of 2.85 or 98.1% of the responses indicated somewhat or consistent and Disposition 6, Exhibits prompt regular attendance, wears professional attire, and communicates in standard English showed a mean score of 2.78 or 96.9% of the responders indicated somewhat or consistent.  Disposition 1, applies reflective practices improved from previous terms with a mean of 2.80 or 98.1% of the responders indicated somewhat or consistent.

The fall 2010 results reflect comparative data for monitoring future trends. The results reflected in the table below shows that the performance of DoEL candidates parallel the performance of SOE candidates is significant ± .5.

SOE

2.83

2.93

2.94

2.85

2.92

2.82

Educational Leadership

2.80

2.91

2.90

2.76

2.85

2.78

  In summary, of the six major areas on which candidates were assessed, DoEL candidates were consistently rated highest in 2) safe and supportive learning environment and 3) high values.  The weakest areas were 4) rapport with students, families, colleagues, and community and 6) exhibits regular attendance, wears professional dress.

Conclusions and Possible Changes to be Made based on the Data.

Mean results show that candidates possess the professional dispositions of principled leaders; however, the DoEL will continue to make revisions to the program to address the weak areas as indicated. 


PROGRAM ASSESSMENT REPORT

FALL 2010

THE CITADEL

PROFESSIONAL EDUCATION UNIT


Name of Assessment Instrument:  Needs Assessment

Unit or Program Assessment:  Program Assessment

Semester and Year Administered:  Fall 2010

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  Educ 531 – Principles of Elementary Curriculum Development

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used): 

The Needs Assessment Instrument measures candidate procedural knowledge of collecting and analyzing quantitative and qualitative data to determine if the curriculum meets the diverse needs of the student population.  Candidates are required to analyze information data, interpret findings and select a problem, and define the problem.

Number of Respondents:  6

Number of Competencies Assessed:  

  1. Analyze Information Data
  2. Interpret Findings and Select a Problem
  3. Define the Problem

Scale Used: Target = 3 points; Acceptable – 2 points; Unacceptable = 1 point or NA = (not applicable).”  NOTES:  NA responses were omitted from the mean calculations.  Non-responders were omitted from all calculations.

Analysis of Data: 

A. Analyze Information Data
On the competency assessed, at least 100% of the candidates were rated “target”.

The overall mean score was 3.00.

B.  Interpret Findings and Select a Problem

On the competency assessed, at least 100% of the candidates were rated “target”.

The overall mean score was 3.00.

C.    Define the Problem

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable”. The overall mean score was 2.33.

Conclusions and Possible Changes to be Made Based on the Data:

Course instructors will collaborate with the instructor for Educational Research to ensure candidates have an additional opportunity to interpret and synthesize data to identify and define the central problem.


PROGRAM ASSESSMENT REPORT

FALL 2010

THE CITADEL

PROFESSIONAL EDUCATION UNIT


Name of Assessment Instrument:  Needs Assessment

Unit or Program Assessment:  Program Assessment

Semester and Year Administered:  Fall 2010

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  Educ 532 – Principles of Secondary Curriculum Development

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used): 

The Needs Assessment Instrument measures candidate procedural knowledge of collecting and analyzing quantitative and qualitative data to determine if the curriculum meets the diverse needs of the student population.  Candidates are required to analyze information data, interpret findings and select a problem, and define the problem.

Number of Respondents:  5

Number of Competencies Assessed

  1. Analyze Information Data
  2. Interpret Findings and Select a Problem
  3. Define the Problem

Scale Used: Target = 3 points; Acceptable – 2 points; Unacceptable = 1 point or NA = (not applicable).”  NOTES:  NA responses were omitted from the mean calculations.  Non-responders were omitted from all calculations.

Analysis of Data: 

A. Analyze Information Data
On the competency assessed, at least 100% of the candidates were rated “target”.

The overall mean score was 3.00.

B.  Interpret Findings and Select a Problem

On the competency assessed, at least 100% of the candidates were rated “target”.

The overall mean score was 3.00.

C.    Define the Problem

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable”. The overall mean score was 2.00.

Conclusions and Possible Changes to be Made Based on the Data:

Course instructors will collaborate with the instructor for Educational Research to ensure candidates have an additional opportunity to interpret and synthesize data to identify and define the central problem.

Spring 2010 and Summer 2010 Educational Leadership Division Program Report

Spring/Summer 2010

THE CITADEL

PROFESSIONAL EDUCATION UNIT

Professional Dispositions Report—Educational Leadership


Name of Assessment Instrument:  Professional Dispositions (completed by candidate, on-site supervisor, college supervisor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Spring/Sumner 2010

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

The Professional Dispositions Instrument collects professional ethics data on six areas that influence candidate behaviors:  (a) reflective practices, (b) commitment to a safe, supportive learning environment, (c) high values and a caring, fair, honest, responsible, respectful attitude, (d) rapport with students, families, colleagues, community, (e) values diversity and exhibits sensitivity to and respect for cultures, and (f) exhibits prompt regular attendance, wears professional attire, and communicates in standard English. Responses range from acceptable to target.

The spring/summer end-of-semester Professional Dispositions results for the Educational Leadership Division showed a low response rate—68 responses that represent 16% for spring and 25 responses for summer that represent 12% of all School of Education Responses.  For spring, means for the six dispositions range from a low of 2.84, where 98% of responses were somewhat or consistently evident for Disposition 1, Applies Reflexive Practices, to a high of 2.94, where 99% of responses were somewhat or consistently evident for Disposition 1, 2.87, which represents 97% somewhat or consistently evident, for Exhibits prompt regular attendance, wears professional attire, and communicates in standard English, 290, which represents 98% somewhat or consistently evidentfor Demonstrates commitment to a safe, supportive learning environment,2.92, somewhat or consistently evidentwhich represents 88%, or Builds rapport with students, families, colleagues, and community, and for both Demonstrates high values and a caring, fair, honest, responsible, and respectful attitude,  and Values diversity and exhibits sensitivity to and respect for cultures which generated a mean of 2.94, somewhat or consistently evident which represents 99%. 

The spring/summer results reflect comparative data for monitoring future trends. The results reflected in the table below shows that the performance of DoEL candidates parallel the performance of SOE candidates is significant ± .5.

SOE

2.85

2.94

2.93

2.90

2.93

2.89

Educational Leadership

2.84

2.90

2.94

2.92

2.94

2.87

  In summary, of the six major areas on which candidates were assessed, DoEL candidates were consistently rated highest in “3) High Values.” and “5) “Values Diversity”. The weakest areas were “1) Reflective Practices and “6) Exhibits prompt attendance, wears professional attire, and communicates in standard English.

Conclusions and Possible Changes to be Made based on the Data.

Mean results show that candidates possess the professional dispositions of principled leaders; however, the DoEL will continue to make revisions to the program to address the weak areas—applies reflective practices. 

Fall 2009 Educational Leadership Division Program Report

Fall 2009

THE CITADEL

PROFESSIONAL EDUCATION UNIT

Fall 2009 Portfolio Report—Educational Leadership

Name of Assessment Instrument:  Portfolio Instrument (completed by candidate, college supervisor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Fall 2009

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

The Portfolio Report Instrument collects data on content knowledge, student learning, and candidate performance on the six Educational Leadership Constituent Council (ELCC) Standards and the Internship Standard 7.0: (a) Vision of learning, (b) Positive school climate, (c) Manage the organization, operations, and resources, (d) Collaborate with families and other community members, (e) Act with integrity, fairly, and in an ethical manner, and (f) understand, respond to, and influence the larger political, social, economic, legal, and cultural context. Candidate ratings ranged from acceptable to target.

The fall 2009 end-of-semester Portfolio Report results for the Educational Leadership Division showed a low response rate—26 responses that represent 26.53% of all School of Education Responses.  Means for the six ELCC Standards range from a low of 2.71, where 88.46% of responses were somewhat or consistently evident for ELCC Standards 1 and 2, Vision of learning and Positive School Climate, to a high of 2.92, where 88.46% of responses were consistently evident for ELCC Standard 5, Acts with integrity, fairly, and in an ethical manner. Standards 4, 3, and 6, Collaborate with families and other community members, Manage the organization, operations, and resources, and understand, respond to, and influence the larger political, social, economic, legal, and cultural context generated mean scores of 2.86, 2.79, and 2.87 respectively.

The fall 2009 results reflect high levels of candidate preparation for administration and supervision as well as school district leadership. The results in the table below show that DoEL candidates’ performance is significant ± .5; however Standard 3, manage the organization, operations, and resources and Standard 5, Acts with integrity, fairly, and in an ethical manner exceeds the performance of All SOE candidates.

ELCC Standards

1

2

3

4

5

6

School of Education

2.74

2.76

2.76

2.71

2.82

2.78

Educational Leadership

2.71

2.71

2.86

2.79

2.92

2.87

  In summary, of the six ELCC Standards on which candidates were assessed, DoEL candidates consistently rated highest in “5) Acts with integrity, fairly, and in an ethical manner.”  The weakest areas were “1) Vision for learning and  “2) Positive School Climate/Instructional leadership.”

Conclusions and Possible Changes to be Made based on the Data.

Mean results show that candidates are principled educational leaders who are ethical; however, the DoEL faculty must continue to make revisions to the program to address the weak areas—vision for learning and positive school climate/instructional leadership. 

 


 

 

Fall 2009

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report

 

Name of Program Assessment Instruments:  

  • CASE STUDIES REPORT
    (completed by instructor)

Program Assessment:  Program Assessment

Semester and Year Administered:  Fall 2009

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

NEEDS ASSESSMENT - ACTION RESEARCH PROPOSAL

  • EDUC 528 – School Administration

Summary:

The fall 2009 report: Case Studies Report, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in identifying critical issues in educational leadership, demonstrating content knowledge associated with ELCC Knowledge, Performance, and Dispositions, and taking leadership actions to generate anticipated results.

For EDUC 528 – School Administration, the sample consisted of 14 of 14 candidates; Case Studies Report responses were rated in five areas:

  • Critical Issues are Identified (N=14, Mean = 3.00)
  • ELCC Knowledge/Performance/Disposition Standard (N=14, Mean = 3.00)
  • Leadership Action (N =13, Mean = 2.93)
  • Anticipated Results (N =14, Mean = 3.00)

Rubric: Case Study Performance Assessment

 

Target 3 Points

(3 pts)

Acceptable
2 Points

(2 pts)

Unacceptable
 0 Points

(0 pts)

Mean

Mode

Stdev

I. Critical Issues are Identified

14

0

0

3.00

3

0.00

II. ELCC Knowledge/ Performance/ Disposition Standard

14

0

0

3.00

3

0.00

III. Leadership Action

13

1

0

2.93

3

0.26

VI. Anticipated Results

14

0

0

3.00

3

0.00

V. Standard Written English

12

0

0

3.00

3

0.00

Results show performance ratings of acceptable or target in all areas.  The sample size is sufficient to generate valid results, all candidates submitted assignments.  DoEL faculty will implement a more aggressive approach to ensure candidate results are provided additional opportunities to select a problem based on the findings generated by data results.

Strengths/Weaknesses

 Clearly candidates have performed above minimally acceptable level. 

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate time to provide candidates with authentic experiences in the P-12 context.


Fall 2009

THE CITADEL

PROFESSIONAL EDUCATION UNIT

Professional Dispositions Report—Educational Leadership

Name of Assessment Instrument:  Professional Dispositions (completed by candidate, on-site supervisor, college supervisor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Fall 2009

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

The Professional Dispositions Instrument collects professional ethics data on six areas that influence candidate behaviors:  (a) reflective practices, (b) commitment to a safe, supportive learning environment, (c) high values and a caring, fair, honest, responsible, respectful attitude, (d) rapport with students, families, colleagues, community, (e) values diversity and exhibits sensitivity to and respect for cultures, and (f) exhibits prompt regular attendance, wears professional attire, and communicates in standard English. Responses range from acceptable to target.

The fall 2009 end-of-semester Professional Dispositions results for the Educational Leadership Division showed a low response rate—38 responses that represent 9.36% of all School of Education Responses.  Means for the six dispositions range from a low of 2.76, where 95% of responses were somewhat or consistently evident for Disposition 1, Applies Reflexive Practices, to a high of 2.86, where 95% of responses were somewhat or consistently evident for Dispositions 2, 3, 5, and 6: Demonstrates commitment to a safe, supportive learning environment, Demonstrates high values and a caring, fair, honest, responsible, and respectful attitude, Values diversity and exhibits sensitivity to and respect for cultures and, Exhibits prompt regular attendance, wears professional attire, and communicates in standard English respectively. Disposition 4, Builds rapport with students, families, colleagues, and community generated a mean of 2.81. 

The fall 2009 results reflect comparative data for monitoring future trends. The results reflected in the table below shows that the performance of DoEL candidates parallel the performance of SOE candidates is significant ± .5.

SOE

2.81

2.89

2.90

2.84

2.90

2.87

Educational Leadership

2.76

2.86

2.86

2.81

2.86

2.86

  In summary, of the six major areas on which candidates were assessed, DoEL candidates were consistently rated highest in “3) High Values.”  The weakest areas were “1) Reflective Practices and  “4) Rapport with students, families, colleagues, and community.”

Conclusions and Possible Changes to be Made based on the Data.

Mean results show that candidates possess the professional dispositions of principled leaders; however, the DoEL will continue to make revisions to the program to address the weak areas—rapport with students, families, colleagues, and community and reflective practice.


Fall 2009

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report

Name of Program Assessment Instruments:  

  • NEEDS ASSESSMENT –ACTION RESEARCH PROPOSAL
    (completed by instructor)

Program Assessment:  Program Assessment

Semester and Year Administered:  Fall 2009

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

NEEDS ASSESSMENT - ACTION RESEARCH PROPOSAL

§  EDUC 524 – Techniques of School Supervision

Summary:

The fall 2009 reports: Needs Assessment and Action Research Proposal, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments and planning and developing appropriate next steps for implementing action research.

For EDUC 524 – Techniques of School Supervision, the sample consisted of 12 of 12 candidates; Needs Assessment responses were rated in three areas:

§  Analyze Data/Information (N=12, Mean =3.00)

§  Interpret Findings and Select the Problem (N=12, Mean=2.75)

§  Define the Problem (N =12, Mean =2.00)

 Rubric: Needs Assessment Rubric

 

Target

(3 pts)

Acceptable

(2 pts)

Unacceptable

(1 pts)

Mean

Mode

Stdev

Analyze Information Data

12

0

0

3.00

3

0.00

Interpret Findings and Select the Problem

9

3

0

2.75

3

0.43

Define the Problem

0

12

0

2.00

2

0.00

 

Analyze Information Data

12 (100%)

Interpret Findings and Select the Problem

9 (75%)

Define the Problem

12 (100%)

 

Target

 

Acceptable

 

Unacceptable

Results show performance ratings of acceptable or target in all areas.  The sample size is sufficient to generate valid results, all candidates submitted assignments.  DoEL faculty will implement a more aggressive approach to ensure candidate results are provided additional opportunities to select a problem based on the findings generated by data results.

Strengths/Weaknesses

 Clearly candidates are able to analyze information; interpretation of that data and determining a focus is a problem.  We will work with the instructor for EDUC 512 - Data Collection and Analysis to resolve this issue.

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to ensure candidates make data-driven decisions based on quantitative and qualitative findings.

Spring 2009 and Summer 2009 Educational Leadership Division Program Report

Educational Leadership Division

Program Summary Report

October 13, 2009


The Spring 2009 Survey of Advanced Program Completers for the Educational Leadership Division had a total of 26 responses to 20 questions. The results are as follows:

N=26

Top 5 Indicators - All Strongly Agree

  1. Develop, maintain, and enhance a school environment that promotes effective learning. 76.92% /mean 4.73
  2. Administrative internship experience. 73.08% /mean 4.65
  3. School Law 69.23% /mean 4.62
  4. Model principled leader practices. 69.23% /mean 4.65
  5. Apply/interpret educational research. 65.38% /mean 4.58

Bottom 5 Indicators - All Strongly Agree

  1. Manage financial resources efficiently. 42.31% /mean 4.23
  2. Staff Personnel Administration. 42.31% / mean 4.31
  3. Develop skills ofr human/public relations 46.15% /mean 4.35
  4. Techniques of supervision. 50% /mean 4.42
  5. School/community relations and the political processes 50% /mean 4.48

The Spring 2009 Survey of 1-Year Post-Graduates for the Educational Leadership Division had a total of 8 responses for 21 questions; 4 for Elementary; 2 for Secondary; 1 for Superintendent and 1 for School Psychology. The results are as follows:

N=8

Top 5 Indicators - All Very Satisfied

  1. Seeking opportunities for professional growth 87.50% /mean 4.75
  2. Modeling professionalism with all stakeholders 75% / mean 4.75
  3. Showing a caring, fair, honest and respectful attitude 75% / mean 4.75
  4. Using current research to inform my work 75% /mean 4.75
  5. Developing and managing meaningful educational experiences that address the needs of all learners with respect for their individual cultural characteristics. 75% /mean 4.75

Bottom 5 Indicators - All Very Satisfied

  1. Manage financial resources 25% /mean 4.0
  2. Develop essential skills needed to be an effective school administrator. 25% /mean 4.25
  3. Maintain focus on the special needs of individual learners 37.5% /mean 4.38
  4. Knowledge of content in my field 37.5 /mean 4.38
  5. Knowledge of content in my field 37.5% /mean 4.38

The Spring 2009 Employer Follow-up Survey for the Educational Leadership Division had a total of 6 responses to 25 questions. Of the 25 questions, all areas fell in the Very Satisfied or Satisfied sections with 83.33% or 66.67% in 21 questions. There were 4 questions which fell into the 50% Very Satisfied and are as follows:

  1. Maintain focus on the special needs of individual learners. Mean 4.50
  2. Model principal leader practices. Mean 4.50
  3. Integrates the use of technology in his/her work. Mean 4.60
  4. Manage financial resources efficiently. Mean 4.60

When asked to list the area(s) of Educational Leadership in which the graduate was best prepared, the results are as follows:

Special education, seeing the big picture, future administration position, involving stakeholders, preparing a budget and technology integration; and curriculum and instruction.

When asked what changes they would recommend for the Educational Leadership program, the results are as follows:

Dealing with personnel matters, spend more time looking at programs that are being implemented at ALL levels; ability for students to go to schools and talk with practicing administrators; spend less time on mundane tasks; and focus on data to guide instruction and improve student learning. Two indicated that no changes needed to be made.

Faculty Learning Communities - FLCs

We are in the beginning stages of developing FLCs where we have focused, intentional conversations about our courses, assessments, strengths, weaknesses, gaps and overlaps. We hope by looking at our work in this way, we will improve our Educational Leadership courses for all of our programs.

Fall 2008 Educational Leadership Division Program Report

THE CITADEL
SCHOOL OF EDUCATION
DIVISION OF EDUCATIONAL LEADERSHIP

FALL 2008 Program Assessment Summary Report
End of Semester Administration

Assessment Instrument Course Administered Summary Strengths / Weaknesses Conclusions / Changes
Needs Assessment EDUC524 -
Techniques of School Supervision

The sample, 14 of 14 candidate responses, was rated in three areas:

  • Analyze Data/Information (N=14, Mean=2.64)
  • Interpret Findings and Select the Problem (N=14, Mean-2.86)
  • Define the Problem (N=14, Mean=2.00)
Results show performance ratings of acceptable or target in all areas. However, means show that 35% (N=5) and 14% (N=2) candidates have more difficulty analyzing and interpreting data than defining the problem. Reduced the number of program assessments and correlate to Specialized Program Association assessments.
Action Research Proposal EDUC524 -
Techniques of School Supervision

The sample, 14 candidates, was rated in three areas:

  • Academic Task (N=14, Mean=3.00)
  • Action Plan Matrix (N=14, Mean=2.79)
  • Reflective Narrative (N=14, Mean=2.07)
 Although a mean of 2.00 shows candidates have the foundational skills for developing and implementing action plans. However, the ability to reflect on the soundness of actions taken and preparing a report that communicates the salience of actions taken is still lacking. Provide more opportunities for candidates to engage in reflection and prepare reflective narrative documents.
Case Study EDUC528 - School Administration

Number of Respondents - 20
Number of Competencies Assessed - 7
Overall there was a 90% target performance.

Disaggregated data reveal:

  • 100% Target performance - Case Study Synopsis and ISLLC/ELCC Knowledge/ Performance/ Disposition Standard
  • 85-95% Target performance - District/State/Federal Policy or Practice, Outside Reference, Text Reference, and Considered Opinion
  • 50% Target performance - Standard Written English
  • Strength - 100% Target - Performance Areas I and III
  • Weakness - 45% Unacceptable - Performance Area VII. Standard Written English
Given the 45% unacceptable performance in the areas of III faculty should look at earlier intervention. In looking at individual papers, particular attention should be given to proper APA citation and using research to support considered opinions.
Reflective Journal EDUC529 - Micro-Computers

Number of Respondents - 19
Number of Competencies Assessed - 5
Overall there was a 100% target performance.

Disaggregated data reveal:

  • 100% Target performance - Speakers/Interviews/Field Experience Site Visits
  • Overall there was a 21% unacceptable performance - V. Presentation/ Professional Look.
  • Strength - 94-100% Target - Performance Areas I, II, and V.
  • Weakness - 68% Target - Performance Areas III and IV.
Given the 21% unacceptable performance in IV. Research Reviews, faculty should assess submitted work periodically to ensure opportunities for correction. Particularly look at APA citation and relevance to technology plan.
Technology Plan EDUC529 - Micro-Computers Number of Respondents - 42 (Note: This represents 2 submissions per student.)
Number of Competencies Assessed - 22
  • Overall there was a 80-100% target performance in 20 to 22 performance areas.
  • Overall there were 2 areas (9 and 15) with 12% and 4% unacceptable performance.
Given the one area, 9. Funding, of 12% unacceptable performance, faculty should consider expanded discussion about funding opportunities and the process for grant applications.
Curriculum Project EDUC532 - Principles of Secondary Curriculum Development      

Summer 2008 Educational Leadership Division Program Report

Summer I 2008

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report

Name of Program Assessment Instruments:  

  • NEEDS ASSESSMENT –ACTION RESEARCH PROPOSAL
    (completed by instructor)

Program Assessment:  Program Assessment

Semester and Year Administered:  Summer I 2008

Beginning or Middle or End of Semester Administered:  End of Semester

 
Course(s) Administered: 

NEEDS ASSESSMENT - ACTION RESEARCH PROPOSAL

§         EDUC 524 – Techniques of School Supervision

Summary:

The summer I 2008 reports: Needs Assessment and Action Research Proposal, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments and planning and developing appropriate next steps for implementing action research.

The small samples:  Educ 524, 3 of 3 candidate responses, rated candidates in three areas reflected on the Needs Assessment:

§         Analyze Data/Information (N=3, Mean =3.00)

§         Interpret Findings and Select the Problem (N=3, Mean=2.00)

§         Define the Problem (N =3, Mean =3.00)

Similar results were reported for the Action Research Proposal.  Candidates were rated in three areas on the Action Research Proposal:

§         Academic Task (N=3, Mean=3.00)

§         Action Plan Matrix (N=3, Mean=2.50)

§         Reflective Narrative (N=3, Mean=3.00)

Results show performance ratings of acceptable or target in all areas.  However, the sample size is insufficient to generate valid results. Although, the summer I 2008 class was unusually small, all candidates submitted assignments.  DoEL faculty will implement a more aggressive approach to insure candidate results are posted and analyzed.

Strengths/Weaknesses

The sample size was insufficient to generate valid strengths or weaknesses

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to insure candidates complete program assessments. 
The number of program assessments will be reduced and correlated to Specialized Program Association assessments.

Summer I 2008

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report

Name of Program Assessment Instruments:  

  • NEEDS ASSESSMENT –ACTION RESEARCH PROPOSAL
    (completed by instructor)

Program Assessment:  Program Assessment

Semester and Year Administered:  Summer I 2008

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

NEEDS ASSESSMENT - ACTION RESEARCH PROPOSAL

§         EDUC 524 – Techniques of School Supervision

Summary:

The summer I 2008 reports: Needs Assessment and Action Research Proposal, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments and planning and developing appropriate next steps for implementing action research.

The small samples:  Educ 524, 3 of 3 candidate responses, rated candidates in three areas reflected on the Needs Assessment:

§         Analyze Data/Information (N=3, Mean =3.00)

§         Interpret Findings and Select the Problem (N=3, Mean=2.00)

§         Define the Problem (N =3, Mean =3.00)

Similar results were reported for the Action Research Proposal.  Candidates were rated in three areas on the Action Research Proposal:

§         Academic Task (N=3, Mean=3.00)

§         Action Plan Matrix (N=3, Mean=2.50)

§         Reflective Narrative (N=3, Mean=3.00)

ExhibitCenter Reports - Assessment

Report Title: Sum I 2008 EDUC524 Action Research Proposal

Date:

06/01/2008 - 06/27/2008

Milestone:

All Scoring: All

Rubric: Action Research Proposal Rubric

 

Target
(3 pts)

Acceptable
(2 pts)

Unacceptable
(1 pts)

Mean

Mode

Stdev

ACADEMIC TASK

2

0

0

3.00

3

0.00

ACTION PLAN MATRIX

1

1

0

2.50

2

0.50

REFLECTIVE NARRATIVE

2

0

0

3.00

3

0.00


ACADEMIC TASK

https://c1.livetext.com/assets/clear.gif2 (100%)

ACTION PLAN MATRIX

https://c1.livetext.com/assets/clear.gif1 (50%)

https://c1.livetext.com/assets/clear.gif1 (50%)

REFLECTIVE NARRATIVE

https://c1.livetext.com/assets/clear.gif2 (100%)

 

 

Target

 

Acceptable

 

Unacceptable

Results show performance ratings of acceptable or target in all areas.  However, the sample size is insufficient to generate valid results. Although, the summer I 2008 class was unusually small, all candidates submitted assignments.  DoEL faculty will implement a more aggressive approach to insure candidate results are posted and analyzed.

 Strengths/Weaknesses

 The sample size was insufficient to generate valid strengths or weaknesses

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to insure candidates complete program assessments. 
The number of program assessments will be reduced and correlated to Specialized Program Association assessments.

Summer I 2008

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Repor


Name of Program Assessment Instruments:
 

  • NEEDS ASSESSMENT –ACTION RESEARCH PROPOSAL
    (completed by instructor)

Program Assessment:  Program Assessment

Semester and Year Administered:  Summer I 2008

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

NEEDS ASSESSMENT - ACTION RESEARCH PROPOSAL

§         EDUC 524 – Techniques of School Supervision

Summary:

The summer I 2008 reports: Needs Assessment and Action Research Proposal, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments and planning and developing appropriate next steps for implementing action research.

The small samples:  Educ 524, 3 of 3 candidate responses, rated candidates in three areas reflected on the Needs Assessment:

§         Analyze Data/Information (N=3, Mean =3.00)

§         Interpret Findings and Select the Problem (N=3, Mean=2.00)

§         Define the Problem (N =3, Mean =3.00)

Report Title: Sum I 2008 EDUC524 Needs Assessment Report

Date:

06/01/2008 - 06/27/2008

Milestone:

All Scoring: All

Rubric: Needs Assessment Rubric

 

Target
(3 pts)

Acceptable
(2 pts)

Unacceptable
(1 pts)

Mean

Mode

Stdev

Analyze Information Data

3

0

0

3.00

3

0.00

Interpret Findings and Select the Problem

0

3

0

2.00

2

0.00

Define the Problem

3

0

0

3.00

3

0.00


Analyze Information Data

3 (100%)

Interpret Findings and Select the Problem

3 (100%)

Define the Problem

3 (100%)

 

 

Target

 

Acceptable

 

Unacceptable

Results show performance ratings of acceptable or target in all areas.  However, the sample size is insufficient to generate valid results. Although, the summer I 2008 class was unusually small, all candidates submitted assignments.  DoEL faculty will implement a more aggressive approach to insure candidate results are posted and analyzed.

Strengths/Weaknesses

 The sample size was insufficient to generate valid strengths or weaknesses

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to insure candidates complete program assessments. 
The number of program assessments will be reduced and correlated to Specialized Program Association assessments.

Summer II 2008

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report


Name of Program Assessment Instruments:
 

  • NEEDS ASSESSMENT –ACTION RESEARCH PROPOSAL
    (completed by instructor)

 Program Assessment:  Program Assessment

 Semester and Year Administered:  Summer II 2008

 Beginning or Middle or End of Semester Administered:  End of Semester

 Course(s) Administered: 

NEEDS ASSESSMENT – CURRICULUM ACTION RESEARCH PROPOSAL

         EDUC 532 – Principles of Secondary Curriculum Development

(Completed by Instructor)

Summary:

The summer II 2008 report: Needs Assessment and Curriculum Action Research Proposal, for Educ 532, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments and planning and developing appropriate curriculum initiatives for implementing needs identified through action research.

The small sample size reported for Educ 532, 7 of 17 candidate responses, were rated in three areas:

Needs Assessment:

§         Analyze Data/Information (N=7, Mean =3.00)

§         Interpret Findings and Select the Problem (N=7, Mean=3.00)

§         Define the Problem (N =7, Mean =2.43)

Curriculum Action Research Proposal

§         There were no candidate responses, 0 of 17,  recorded for the Curriculum Action Research Proposal

Results show performance ratings of acceptable or target in all areas.  However, the sample size is insufficient to generate valid results. DoEL faculty will implement a more aggressive approach to insure candidate results are posted and analyzed.

 

Report Title: Summer 2008 EDUC532 Needs Assessment Report

Date:

06/01/2008 - 10/02/2008

Milestone:

All Scoring: All

Rubric: Rubric

 

Target
(3 pts)

Acceptable
(2 pts)

Unacceptable
(1 pts)

Mean

Mode

Stdev

ANALYZE INFORMATION DATA

7

0

0

3.00

3

0.00

INTERPRET FINDINGS AND SELECT THE PROBLEM

7

0

0

3.00

3

0.00

DEFINE THE PROBLEM

3

4

0

2.43

2

0.49


ANALYZE INFORMATION DATA

https://c1.livetext.com/assets/clear.gif7 (100%)

INTERPRET FINDINGS AND SELECT THE PROBLEM

https://c1.livetext.com/assets/clear.gif7 (100%)

DEFINE THE PROBLEM

https://c1.livetext.com/assets/clear.gif3 (42%)

https://c1.livetext.com/assets/clear.gif4 (57%)

 

 

Target

 

Acceptable

 

Unacceptable

Inter-Rater Summary

 

2BROWNK

Mean

Stdev

ANALYZE INFORMATION DATA

3.00

3.00

0.00

INTERPRET FINDINGS AND SELECT THE PROBLEM

3.00

3.00

0.00

DEFINE THE PROBLEM

2.43

2.43

0.0


Strengths/Weaknesses

 The sample size was insufficient to generate valid strengths or weaknesses

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to insure candidates complete program assessments. 
The number of program assessments will be reduced and correlated to Specialized Program Association assessments.

 

Summer II 2008

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report


Name of Program Assessment Instruments:
 

  • NEEDS ASSESSMENT –ACTION RESEARCH PROPOSAL
    (completed by instructor)

 Program Assessment:  Program Assessment

Semester and Year Administered:  Summer II 2008

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

NEEDS ASSESSMENT – CURRICULUM ACTION RESEARCH PROPOSAL

§         EDUC 531 – Principles of Elementary Curriculum Development

(Completed by Instructor)

DUC 532 – Principles of Secondary Curriculum Development

(Completed by Instructor)

Summary:

The summer II 2008 reports: Needs Assessment and Curriculum Action Research Proposal, for Educ 531 and Educ 532, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments and planning and developing appropriate curriculum initiatives for implementing needs identified through action research.

The small samples:  Educ 531, 8 of 19 candidate responses, rated candidates in three areas:

Needs Assessment:

§         Analyze Data/Information (N=8, Mean =3.00)

§         Interpret Findings and Select the Problem (N=8, Mean=2.70)

§         Define the Problem (N =8, Mean =2.50)

Curriculum Action Research Proposal

§         There were no candidate responses, 0 of 19,  recorded for the Curriculum Action Research Proposal

Similarly, the small sample size reported for Educ 532, 7 of 17 candidate responses, were rated in three areas:

Needs Assessment:

§         Analyze Data/Information (N=7, Mean =3.00)

§         Interpret Findings and Select the Problem (N=7, Mean=3.00)

§         Define the Problem (N =7, Mean =2.43)

Curriculum Action Research Proposal

§         There were no candidate responses, 0 of 17,  recorded for the Curriculum Action Research Proposal

Results show performance ratings of acceptable or target in all areas.  However, the sample size is insufficient to generate valid results. DoEL faculty will implement a more aggressive approach to insure candidate results are posted and analyzed.

Strengths/Weaknesses

 The sample size was insufficient to generate valid strengths or weaknesses

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to insure candidates complete program assessments. 
The number of program assessments will be reduced and correlated to Specialized Program Association assessments.

Summer II 2008

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report


Name of Program Assessment Instruments:
 

  • NEEDS ASSESSMENT –ACTION RESEARCH PROPOSAL
    (completed by instructor)

Program Assessment:  Program Assessment

Semester and Year Administered:  Summer II 2008

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

NEEDS ASSESSMENT – CURRICULUM ACTION RESEARCH PROPOSAL

§         EDUC 531 – Principles of Elementary Curriculum Development

(Completed by Instructor)

Summary:

The summer II 2008 report: Needs Assessment and Curriculum Action Research Proposal, for Educ 531 provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments and planning and developing appropriate curriculum initiatives for implementing needs identified through action research.

The small samples:  Educ 531, 8 of 19 candidate responses, rated candidates in three areas:

Needs Assessment:

§         Analyze Data/Information (N=8, Mean =3.00)

§         Interpret Findings and Select the Problem (N=8, Mean=2.70)

§         Define the Problem (N =8, Mean =2.50)

Curriculum Action Research Proposal

§         There were no candidate responses, 0 of 19,  recorded for the Curriculum Action Research Proposal

Results show performance ratings of acceptable or target in all areas.  However, the sample size is insufficient to generate valid results. DoEL faculty will implement a more aggressive approach to insure candidate results are posted and analyzed.

Report Title: Summer 2008 EDUC531 Needs Assessment Report

Date:

06/01/2008 - 10/02/2008

Milestone:

All Scoring: All

Rubric: Rubric

 

Target
(3 pts)

Acceptable
(2 pts)

Unacceptable
(1 pts)

Mean

Mode

Stdev

ANALYZE INFORMATION DATA

8

0

0

3.00

3

0.00

INTERPRET FINDINGS AND SELECT THE PROBLEM

6

2

0

2.75

3

0.43

DEFINE THE PROBLEM

4

4

0

2.50

2

0.50


ANALYZE INFORMATION DATA

https://c1.livetext.com/assets/clear.gif8 (100%)

INTERPRET FINDINGS AND SELECT THE PROBLEM

https://c1.livetext.com/assets/clear.gif6 (75%)

https://c1.livetext.com/assets/clear.gif2 (25%)

DEFINE THE PROBLEM

https://c1.livetext.com/assets/clear.gif4 (50%)

https://c1.livetext.com/assets/clear.gif4 (50%)

 

 

Target

 

Acceptable

 

Unacceptable

Inter-Rater Summary

 

2BROWNK

Mean

Stdev

ANALYZE INFORMATION DATA

3.00

3.00

0.00

INTERPRET FINDINGS AND SELECT THE PROBLEM

2.75

2.75

0.00

DEFINE THE PROBLEM

2.50

2.50

0.0


Strengths/Weaknesses

 The sample size was insufficient to generate valid strengths or weaknesses

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to insure candidates complete program assessments. 
The number of program assessments will be reduced and correlated to Specialized Program Association assessments.

Spring 2008 Educational Leadership Division Program Report

Spring 2008

THE CITADEL

PROFESSIONAL EDUCATION UNIT

DoEL Program Summary Report

Name of Program Assessment Instruments:

  • CASE STUDY ASSESSMENT -  WORKING PORTFOLIO ASSESSMENT
    (completed by instructor)
  • TECHNOLOGY REFLECTIVE JOURNAL - TECHNOLOGY PLAN
    (completed by instructor)
  • NEEDS ASSESSMENT – CURRICULUM ACTION RESEARCH PROPOSAL
    (completed by instructor)

Program Assessment:  Program Assessment

Semester and Year Administered:  Spring 2008

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

CASE STUDY ASSESSMENT - WORKING PORTFOLIO ASSESSMENT      

§         EDUC 528 – School Administration

CASE STUDY ASSESSMENT – REFLECTIVE JOURNAL - POWERPOINT

§         EDUC 610 – Seminar in School Improvement

TECHNOLOGY REFLECTIVE JOURNAL - TECHNOLOGY PLAN

§         EDUC 529 – Microcomputers and School Management

NEEDS ASSESSMENT - CURRICULUM ACTION RESEARCH PROPOSAL

§         EDUC 531  - Principles of Elementary Curriculum Development

Summary:

The spring 2008 reports: Case Study, Working Portfolio, Technology Reflective Journal, Technology Plan and Curriculum Action Research Proposal, provided comparative program data on Division of Educational Leadership (DoEL) candidates’ performance in conducting needs assessments, developing technology plans, analyzing case studies and planning appropriate next steps, and preparing professional portfolios,

The small samples:  Educ 528, 2 of 5 candidate responses;  Educ 529, 2 of 5 candidate responses;  Educ 610, 11 of 24 candidate responses; and  Educ 531, 0 candidate responses, reported performance ratings of acceptable or target in all areas.  However, the size is insufficient to generate valid results. The DoEL will implement a more aggressive approach to insure candidate results are posted and analyzed.

Strengths/Weaknesses

 The sample size was insufficient to generate valid strengths or weaknesses

Conclusions and Possible Changes to be Made Based on Data:

DoEL Faculty will designate class time to insure candidates complete program assessments.  The number of program assessments will be reduced and correlated to Specialized Program Association assessments.

 

Fall 2007 Educational Leadership Division Program Report

Fall 2007

THE CITADEL 

PROFESSIONAL EDUCATION UNIT

Unit Summary Report

Name of Assessment Instruments:  

  • PROFESSIONAL DISPOSITIONS
    (completed by candidate, on-site supervisor, college supervisor)
  • PORTFOLIO REPORT
    (completed by candidate and college supervisor)
  • RESEARCH COMPETENCIES
    (completed by core curriculum instructor)
  • COURSE ASSESSMENT

(completed by instructor and candidate)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Fall 2007

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

DISPOSITION & PORTFOLIO

  • EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

RESEARCH COMPETENCIES

  • EDUC 512 – Data Collection and Analysis

Summary:

The summer fall 2007 reports, Dispositions, Portfolio, Research Competencies, and course assessment provided comparative unit data on DoEL candidates. Results suggest that DoEL candidates demonstrate comparable strengths in professional dispositions, portfolio report, and course assessments, but need improvement in research competencies.

PROFESSIONAL DISPOSITIONS

  • Professional ethics data was collected using the Professional Dispositions Instrument. In six areas candidates somewhat or consistently demonstrated evidence that they influence candidate behaviors toward students, families, colleagues, and communities related to student learning, motivation, and professional growth.   Mean scores were calculated to establish comparative candidate status.  Responses displayed candidates’ application of reflective practice, commitment to a safe and supportive learning environment, demonstration of high values and caring attitude, establishment of rapport with stakeholders, exhibition of respect and sensitivity for cultures and diversity, and exhibition of professional decorum. Results reflect a total of 67 DoEL Responses. 

Disposition Measured

F 07

N= 67 Mean

S 07

N=91

Mean

Sp07

N=87

Mean

1.        The candidate applies reflective practices.

2.88

2.92

2.95

2.         The candidate demonstrates commitment to a safe, supportive learning environment.  

2.91

2.98

2.97

3.         The candidate demonstrates high values and a caring, fair, honest, responsible, and respectful attitude.

2.94

3.00

2.98

4.         The candidate establishes rapport with students, families, colleagues, community.

2.89

2.97

2.95

5.         The candidate values diversity and exhibits sensitivity to and respect for cultures.

2.93

3.00

2.99

6.         The candidate exhibits prompt regular attendance, wears professional attire, and communicates in standard English.

2.93

2.98

2.94

Findings:  The results show that 100% of DoEL candidates, faculty, and mentors observed that candidate practice was somewhat or consistently evident in the six dispositions with overall mean scores ranging from 2.88 – 2.94 out a possible 3.00.  These results indicate that candidates possess dispositions consistent with administrative practitioners.  Data sets show that less than 100% of candidates and mentors are completing the professional dispositions instrument.  Inquiries about low completion rates by mentors show that district Internet filters incoming mail thereby restricting the transmission of the instrument. Low candidate responses will be addressed by instructors allotting time during the last class session to complete the instrument.

PORTFOLIO RUBRIC

  • Content Knowledge

Findings:  In reviewing data for the portfolio rubric, 33 responses rated candidates’ performance as somewhat or consistently evident with content knowledge of practitioners.  In area one of content knowledge, candidates demonstrate knowledge of the central concepts and tools of inquiry of the field, candidates achieved an overall mean score of 2.72.  In area two of content knowledge, candidates demonstrate and apply structures of the field delineated in professional, state, and institutional standards through inquiry, critical analysis and synthesis, candidates and faculty rated the overall mean score 2.65.  Results show that both candidates and faculty rate candidates as demonstrating content knowledge.

 

F07

N = 33

Mean

2.72

2.65

2.88

2.84

2.88

2.75

S07

N = 40

Mean

2.95

2.85

2.95

2.90

2.98

2.92

Sp07

N =

Mean

2.93

2.89

2.95

2.90

2.94

2.87

 

 

 

Student Learning:

Findings indicate that at least 96% of faculty and candidates assessed candidate performance on the professional portfolio as somewhat or consistently evident with facilitating student learning. Overall mean scores were 2.88, 2.84, 2.88, and 2.75 respectively.  The results show that candidates demonstrate the ability to create a positive learning environment that impacts student learning.

RESEARCH COMPETENCIES:

Findings:  

A core curriculum instructor rated six DoEL candidates in the five major areas of Research Competencies:

1.      Statistical Procedures

2.     Descriptive and Inferential Data Interpretation

3.     Published Research Analysis

4.     American Psychological Association Format

5.     Technology Use

 

F07

N = 28

Mean

2.50

2.50

2.54

2.52

2.71

S07

N =  4

Mean

2.25

2.25

2.75

2.75

3.00

Sp07

N =  8

Mean

2.57

2.29

2.71

2.71

2.57

 

 

 

 

 

 Candidates were assessed on a scale of “Consistently evident (3),” “somewhat evident (2),” or “rarely evident (1),” or “NR (not rated).” 

 

In summary, of the five major areas on which candidates were assessed, overall the candidates consistently rated highest (2.71) in “5., Using Technology to Present Course Projects.” The weakest areas were “1., Statistical Procedures,” and “2., Descriptive and Inferential Data Interpretation” (2.50 and 2.50 for each area).

 

COURSE ASSESSMENT – EDUC 524 – Techniques of School Supervision

 

Assessment Results to be Added

  • Needs Assessment
  • Action Research Proposal

Summer 2007 Educational Leadership Division Program Report

Summer 2007

THE CITADEL

PROFESSIONAL EDUCATION UNIT

Unit Summary Report 

Name of Assessment Instruments:  

  • PROFESSIONAL DISPOSITIONS
    (completed by candidate, on-site supervisor, college supervisor)
  • PORTFOLIO REPORT
    (completed by candidate and college supervisor)
  • RESEARCH COMPETENCIES
    (completed by core curriculum instructor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Summer 2007

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered: 

DISPOSITION & PORTFOLIO

  • EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

RESEARCH COMPETENCIES

  • EDUC 512 – Data Collection and Analysis

Summary:

The summer 2007 reports, Dispositions, Portfolio, and Research Competencies, provided comparative unit data on DoEL candidates. Results suggest that DoEL candidates demonstrate comparable strengths in professional dispositions and portfolio report but need improvement in research competencies.

PROFESSIONAL DISPOSITIONS

  • Professional ethics data was collected using the Professional Dispositions Instrument. In six areas that influence candidate behaviors toward students, families, colleagues, and communities related to student learning, motivation, and professional growth, candidates were rated as meeting Target (3), Acceptable (2), or Unacceptable (1).   Mean scores were calculated to establish baseline candidate status.  Responses displayed candidates’ application of reflective practice, commitment to a safe and supportive learning environment, demonstration of high values and caring attitude, establishment of rapport with stakeholders, exhibition of respect and sensitivity for cultures and diversity, and exhibition of professional decorum. Results indicated 26 candidate responses, 44 faculty responses, and seven mentor responses for a total of 91 DoEL Responses.

Disposition Measured

Mean

1.        The candidate applies reflective practices.

2.95

2.        The candidate demonstrates commitment to a safe, supportive learning environment.   

2.97

3.        The candidate demonstrates high values and a caring, fair, honest, responsible, and respectful attitude.

2.98

4.        The candidate establishes rapport with students, families, colleagues, community.

2.95

5.        The candidate values diversity and exhibits sensitivity to and respect for cultures.

2.99

6.        The candidate exhibits prompt regular attendance, wears professional attire, and communicates in standard English.

2.94

Findings:  The results show that at least 97% of DoEL candidates, faculty, and mentors rated candidates as acceptable or target in the six dispositions with overall mean scores ranging from 2.94 – 2.99 out a possible 3.00.  These results reflect an overall increase in candidate, faculty, and mentor responses over the 20065 data.  Data sets show that less than 100% of candidates and mentors are completing the professional dispositions instrument.  Inquiries about low completion rates by mentors show that district Internet filters incoming mail thereby restricting the transmission of the instrument.. For candidates, instructors will allot time during the last class session to complete the instrument.

PORTFOLIO RUBRIC

  • Content Knowledge

Findings:  In reviewing data for the portfolio rubric, 13 Education Specialist (EDSA) responses,  13 Elementary Administration (EADM) responses,  and 14 Secondary Administration responses revealed that both candidates and faculty rated candidates at the Target (3) and Acceptable (2) levels.  In area one of content knowledge, candidates demonstrate knowledge of the central concepts and tools of inquiry of the field, candidates achieved an overall mean score of 2.95.  In area two of content knowledge, candidates demonstrate and apply structures of the field delineated in professional, state, and institutional standards through inquiry, critical analysis and synthesis, candidates and faculty rated the overall mean score 2.85.  Results show that both candidates and faculty rate candidates as demonstrating content knowledge.

Mean

2.95

2.85

2.95

2.90

2.98

2.92

 

Student Learning:

Findings indicate that 97% of faculty and candidates assessed candidate performance on the professional portfolio as acceptable and target at overall mean scores of 2.95, 2.90, 2.98, and 2.92 respectively.  The results show that candidates demonstrate the ability to create a positive learning environment that impacts student learning.

RESEARCH COMPETENCIES:

Findings:  

A core curriculum instructor rated six DoEL candidates in the five major areas of Research Competencies:

1.      Statistical Procedures

2.     Descriptive and Inferential Data Interpretation

3.     Published Research Analysis

4.     American Psychological Association Format

5.     Technology Use

 Candidates were assessed on a scale of “target (3),” “acceptable (2),” or “unacceptable (1),” or “NA (not applicable).” 

In summary, of the five major areas on which candidates were assessed, overall the candidates were consistently rated highest (3.00) in “5., Technology Use.” The weakest areas were “1., Statistical Procedures” and “2., Descriptive and Inferential Data Interpretation” (2.25 for each area).

 

Spring 2007 Educational Leadership Division Program Report Summary


PROGRAM ASSESSMENT REPORT

Spring 2007

Kay D. Woelfel, Ed.D.

 Report Title: Spring 2007 Ed Ldr EDUC528 Professional Portfolio

 

  • Name of Assessment instrument: Final Evaluation of Portfolio (completed by instructor)

Unit or Program Assessment: Program Assessment

Beginning or Middle or End of Semester Administered: End of Semester

Courses(s) Administered: EDUC 528

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

  • Number of Respondents—18
  • Number of Competencies Assessed—5

I  Presentation/ Professional Look

II. Format

III. Work Samples (3 points for each of six ISLLC/ELCC standards) 18 points maximum

IV. Resume or Vita

V. Leadership Philosophy or Professional Identity Statement

  • Scale Used—Target (3), Acceptable (2), Unacceptable (1)

Analysis of Data:

            Overall there was 70-88% target performance.  Disaggregated data reveal:

           Target performance

Presentation/ Professional Look

Format

Work Samples

Leadership Philosophy or Personal Intention Statement

70-88% Target performance in areas I - V

Overall there was a 44% unacceptable performance in all sections.  This is a small sample size and this represents eight students.

Strengths and Weaknesses

            Strength—86-88% Target Performance— Presentation/ Professional Look, Format, Work Samples, and Leadership Philosophy or Personal Intention Statement

            Weakness—26% Unacceptable Performance—Resume or Vita and Leadership Philosophy

Conclusions and Possible Changes to be Made Based on Data:

            Given the 42% unacceptable performance in the areas I – IV, , faculty should look at earlier intervention.  Using the “practice” option available on LiveText may assist candidates in their performance analysis and provide opportunities for improvement prior to the “official” submission. 

 

Report Title: EDUC528-Portfolio-Spring 2007-1

Description: Spring 2007 EDUC 528 Professional Portfolio - Assessment 1 - Spring 2007

Date: 01/01/2007 - 05/11/2007

Milestone: All Scoring: All


Rubric: Portfolio Rubric  

 

Target
(3 pts)

Acceptable
(2 pts)

Unacceptable
(1 pts)

Mean

Mode

Stdev

ISLLC 7

15

3

0

2.83

3

0.37


ISLLC 7
 
ISLLC.7.2, ISLLC.7.3, ISLLC.7.3.1

15 (83%)

3 (16%)

 

 

Target

 

Acceptable

 

Unacceptable


Rubric: Portfolio Performance Assessment  

 

Target 6 Points
(0 pts)

Acceptable 3 Points
(0 pts)

Unacceptable 0 Points
(0 pts)

Mean

Mode

Stdev

I. Presentation/ Professional Look

15

1

1

0.00

0

0.00

II. Format

15

1

1

0.00

0

0.00

III. Work Samples (3 points for each of six ISLLC/ELCC standards) 18 points maximum

13

1

1

0.00

0

0.00

IV. Resume or Vita

13

1

3

0.00

0

0.00

V. Leadership Philosophy or Professional Identity Statement

12

3

2

0.00

0

0.00

Total=30 Possible Points

3

1

0

0.00

0

0.00


[i]

I. Presentation/ Professional Look

15 (88%)

1 (5%)

1 (5%)

II. Format

15 (88%)

1 (5%)

1 (5%)

III. Work Samples (3 points for each of six ISLLC/ELCC standards) 18 points maximum

13 (86%)

1 (6%)

1 (6%)

IV. Resume or Vita

13 (76%)

1 (5%)

3 (17%)

V. Leadership Philosophy or Professional Identity Statement

12 (70%)

3 (17%)

2 (11%)

Total=30 Possible Points

3 (75%)

1 (25%)

 

 

Target 6 Points

 

Acceptable 3 Points

 

Unacceptable 0 Points

Total Documents Assessed: 18  

 


Spring 2007

THE CITADEL  PROFESSIONAL EDUCATION UNIT 

Professional Dispositions Report—Educational Leadership

Name of Assessment Instrument:  Professional Dispositions (completed by candidate, on-site supervisor, college supervisor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Spring 2007

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

The Professional Dispositions Instrument collects professional ethics data on six areas that influence candidate behaviors toward students, families, colleagues, and the communities related to student learning, motivation, and professional growth.  Responses range from candidates’ application of reflective practice, commitment to a safe and supportive learning environment, demonstration of high values and caring attitude, establishment of rapport with stakeholders, exhibition of respect and sensitivity for cultures and diversity, and exhibition of professional decorum.

The spring 2007 end-of-semester Professional Dispositions results for the Educational Leadership Division showed a response rate of 87 responses, 31 EADM, 38 SADM, 16 EDSA candidates that represent 20.91% of all School of Education Responses. Means for the six dispositions range from a low of 2.94, where 100% or responses were rated as Target and Acceptable for Disposition 6, The candidate exhibits prompt regular attendance, wears professional attire, and communicates in standard English, to a high of 2.90, where 97.70% or responses were rated as Target, and Acceptable for Disposition 5, The candidate values diversity and exhibits sensitivity to and respect for cultures. The remaining four dispositions fell in the range of 2.95 – 2.98. 

The spring 2007 results show an overall increase in the number of respondents, the mean for each of the dispositions, and a significant increase in the 2007 mean for Dispositions 2, 4, and 5 over the 2006 mean.  In summary, candidates have shown an increase in Dispositions 2, 4, and 5.  Therefore, for the six major areas all candidates were assessed,  at the target and acceptable levels. “

Conclusions and Possible Changes to be Made based on the Data.

 

Mean results show that candidates possess the professional dispositions of principled leaders; however, the DoEL will continue to make revisions to the program to address the weak areas 1, 4, and 5—applies reflective practice, rapport with students, families, colleagues, and community and values diversity and exhibits sensitivity to and respect for cultures.

 

Fall 2006 Educational Leadership Division Program Report Summary

Fall 2006

THE CITADEL

 PROFESSIONAL EDUCATION UNIT

Professional Dispositions Report—Educational Leadership

Name of Assessment Instrument:  Professional Dispositions (completed by candidate, on-site supervisor, college supervisor)

Unit or Program Assessment:  Unit Assessment 

Semester and Year Administered:  Fall 2006

Beginning or Middle or End of Semester Administered:  End of Semester 

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

The Professional Dispositions Instrument collects professional ethics data on six areas that influence candidate behaviors toward students, families, colleagues, and the communities related to student learning, motivation, and professional growth.  Responses range from candidates’ application of reflective practice, commitment to a safe and supportive learning environment, demonstration of high values and caring attitude, establishment of rapport with stakeholders, exhibition of respect and sensitivity for cultures and diversity, and exhibition of professional decorum. 

The fall 2006 end-of-semester Professional Dispositions results for the Educational Leadership Division showed a low response rate—48 responses that represent 10% of  all School of Education Responses, 13 candidate responses which represent 3% of all SOE responses, and 35 faculty responses that represent 8% of all SOE responses.  Means for the six dispositions range from a low of 2.29, where 71.16% or responses were rated as Target and Acceptable for Disposition 4, The candidate establishes rapport with students, families, colleagues to a high of 2.96, where 100% or responses were rated as Target, and Acceptable for Disposition 3, The candidate demonstrates high values and a caring, fair, honest, responsible, and respectful attitude. The remaining four dispositions fell in the range of 2.50 – 2.88. 

The fall 2006 results reflect baseline data for future comparisons. In summary, of the six major areas on which candidates were assessed,  DoEL candidates were consistently rated highest in “5) Values Diversity.”  The weakest areas were “4) Rapport with students, families, colleagues, and community,” and “2) Commitment to a safe and supportive learning environment.” 

Conclusions and Possible Changes to be Made based on the Data.  

Mean results show that candidates possess the professional dispositions of principled leaders; however, the DoEL will make a plan to make revisions to the program to address the weak areas—rapport with students, families, colleagues, and community and reflective practice.


Fall 2006

THE CITADEL

 PROFESSIONAL EDUCATION UNIT

Portfolio Report—Educational Leadership

Name of Assessment Instrument:  Portfolio Report  (completed by candidate and college supervisor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Fall 2006

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 661, EDUC 662, EDUC 663, EDUC 664, EDUC 632, and EDUC 633 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

A total of 27 candidates were assessed in two major areas of competency by the instructor and the candidate:

1.                  Candidate Knowledge

a.       Demonstrates knowledge of central concepts

                        On the competency assessed, at least 100% of the candidates were rated                           “target” or “acceptable.” 

                        The overall mean score was 2.85

b.      Applies structures delineated through standards

     On the competency assessed, at least 97% of the candidates were rated       “target” or “acceptable.” 

                       The overall mean score was 2.88.

2.                  Student Learning

a.       Creates positive learning environments

      On the competency assessed, at least 97% of the candidates were rated       “target” or “acceptable.” 

                        The overall mean score was 2.93.

 b.      Builds upon developmental levels

      On the competency assessed, at least 97% of the candidates were rated       “target” or “acceptable.” 

     The overall mean score was 2.96

 c.       Understands diversity

       On the competency assessed, at least 97% of the candidates were rated       “target” or “acceptable.” 

      The overall mean score was 3.00.

 d.      Understands policy contexts

      On the competency assessed, at least 97% of the candidates were rated       “target” or “acceptable.” 

                        The overall mean score was 2.96.

The Portfolio Instrument is used to collect data on candidates’ content knowledge of the central concepts and tools of inquiry of the field and candidates’ application of structures delineated in professional, state, and national standards. Additionally, the portfolio instrument collects data on the impact candidates make on student learning through their ability to adhere to policy and create positive learning environments for diverse student needs.  Means, for the 27 responses, range from a low of 2.93, with 100% of respondents rated at the target and acceptable levels to a high of 3.00, with 100% or respondents rated at the target levels.

The mean results for fall 2006 end-of-semester reflect baseline data   All mean ratings were ≥ 2.85. Respondents used a rubric to rate two competences on a 3-point scale of Target, Acceptable, and Unacceptable.

In summary, of the two major areas on which candidates were assessed, overall the candidates were consistently rated highest in “5) Understanding diversity of students, families, and communities.  The weakest areas were content knowledge “1) Central Concepts,” and “followed closely by “2) Professional, state, and institutional standards.”

Conclusions and Possible Changes to be Made based on the Data.

Mean results show that candidates are meeting requirements; however, the DoEL will make a plan to address the weak areas identified—central concepts, developmental levels, and standards by focusing more closely on the inquiry tools and standards.


 Fall 2006

THE  CITADEL

PROFESSIONAL  EDUCATION  UNIT

Research Competencies Report—Educational Leadership  

Name of Assessment Instrument:  Research Competencies (completed by core curriculum instructor)

Unit or Program Assessment:  Unit Assessment

Semester and Year Administered:  Fall 2006

Beginning or Middle or End of Semester Administered:  End of Semester

Course(s) Administered:  EDUC 512 and EDUC 549 (All Sections)

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

A total of 6 candidates were assessed in five major areas of competency by the instructor:

A)     Statistical Procedures

B)     Descriptive and Inferential Data Interpretation

C)     Published Research Analysis

D)     American Psychological Association Format

E)      Technology Use

Candidates were assessed on a scale of “target (3),” “acceptable (2),” or “unacceptable (1),” or “NA (not applicable).”  NOTES:  NA responses were omitted from the mean calculations.  Non-responders were omitted from all calculations.

Analysis of Data:

The analysis of data is presented according to the five major areas on which the candidates were assessed.

A)  Statistical Procedures:

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable.” 

The overall mean score was 2.33.

B)  Descriptive and Inferential Data Interpretation:

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable.” 

Only 1 candidate received an “unacceptable” on the competency “statistical procedures.”

The overall mean score was 2.33.

C)  Published Research Analysis:

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable” 

The overall mean score was 2.50.

D)  American Psychological Association Format:

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable” 

The overall mean score was 2.33.

E)  Technology Use:

On the competency assessed, at least 100% of the candidates were rated “target” or “acceptable”  

The overall mean score was 2.33.

In summary, of the five major areas on which candidates were assessed, overall the candidates were consistently rated highest in “C) Published Research Analysis.”  The weakest areas were “A) Statistical Procedures,”  “B) Descriptive and Inferential Data Interpretation,” and “followed closely by “D) American Psychological Association Format,” and “E) Technology Use.”

Conclusions and Possible Changes to be Made based on the Data: 

Consider assessing candidates’ research skills prior to taking research courses.


 

PROGRAM ASSESSMENT REPORT

FALL 2006

Kay D. Woelfel, Ed.D.

 Report Title: Fall 2006 Ed Ldr EDUC661 Portfolio

 

Name of Assessment instrument: Final Evaluation of Portfolio (completed by instructor)

Unit or Program Assessment: Program Assessment

Beginning or Middle or End of Semester Administered: End of Semester

Courses(s) Administered: EDUC 661

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

  • Number of Respondents—4
  • Number of Competencies Assessed—5

Presentation/ Professional Look

Format

Work Samples (3 points for each of six ISLLC (or ELCC)standards) 18 points maximum

Resume or Vita

Leadership Philosophy or Personal Intention Statement

  • Scale Used—Target (3), Acceptable (2), Unacceptable (1)

Analysis of Data:

            Overall there was a 100 target performance.  Disaggregated data reveal:

            100% Target performance

Presentation/ Professional Look

Format

Work Samples

Leadership Philosophy or Personal Intention Statement

75% Target performance

            Resume or Vita

Overall there was a 25% unacceptable performance in the resume/vita section.  This is a small sample size and this represents one student.

Strengths and Weaknesses

            Strength—100% Target Performance— Presentation/ Professional Look, Format, Work Samples, and Leadership Philosophy or Personal Intention Statement

            Weakness—25% Unacceptable Performance—Resume or Vita

Conclusions and Possible Changes to be Made Based on Data:

            Given the 25% unacceptable performance in the area of resume or vita, faculty should look at earlier intervention.  Using the “practice” option available on LiveText may assist candidates in their performance analysis and provide opportunities for improvement prior to the “official” submission. 

 


 

PROGRAM ASSESSMENT REPORT

FALL 2006

Kay D. Woelfel, Ed.D.

Report Title: Fall 2006 Ed Ldr EDUC610 Reflective Journal

 

Name of Assessment instrument: Final Evaluation of Reflective Journal (completed by instructor)

Unit or Program Assessment: Program Assessment

Beginning or Middle or End of Semester Administered: End of Semester

Courses(s) Administered: EDUC 610

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

  • Number of Respondents—16
  • Number of Competencies Assessed—3

I. Text and Journal Article Reviews

II. Speakers/Site Visits

III. Presentation/ Professional Look

  • Scale Used—Target (3), Acceptable (2), Unacceptable (1)

Analysis of Data:

            Overall there was a 100% target performance.  Disaggregated data reveal:

            71-92% Target performance

I. Text and Journal Article Reviews

II. Speakers/Site Visits

III. Presentation/ Professional Look

Overall there was a 7% unacceptable performance in each of the three areas.  Because that represents 1 person it is hard to determine if that is the same person represented in each of the three categories.  If so, there is a performance issue that is student specific.

Strengths and Weaknesses

            Strength—100% Target Performance—All areas overall.

            Weakness—Not significant.

Conclusions and Possible Changes to be Made Based on Data:

            Given the 7% unacceptable performance in all areas, and because the possibility exits that 1 person may have low performance in each of the three categories, the instructor will want to identify specific (rather than collective) student performance.


 

PROGRAM ASSESSMENT REPORT

FALL 2006

Kay D. Woelfel, Ed.D.

 Report Title: Fall 2006 Ed Ldr EDUC610 Case Study


Name of Assessment instrument: Final Evaluation of Case Study (completed by instructor)

Unit or Program Assessment: Program Assessment

Beginning or Middle or End of Semester Administered: End of Semester

Courses(s) Administered: EDUC 610

Brief Information on Assessment Instrument (e.g., number of respondents, number of competencies assessed, scale used):

  • Number of Respondents—16
  • Number of Competencies Assessed—7

I. Case Study Synopsis

II. District/State/ Federal Policy or Practice

III. ISLLC/ELCC Knowledge/ Performance/ Disposition Standard

IV. Outside Reference

V. Text Reference

VI. Considered Opinion

VII. Standard Written English

  • Scale Used—Target (3), Acceptable (2), Unacceptable (1)

Analysis of Data:

            Overall there was a 100% target performance.  Disaggregated data reveal:

100% Target performance

I. Case Study Synopsis

II. District/State/ Federal Policy or Practice

III. ISLLC/ELCC Knowledge/ Performance/ Disposition Standard

IV. Outside Reference

V. Text Reference

60-80% Target performance

VI. Considered Opinion

VII. Standard Written English

 

Overall there was a 6% and 13% unacceptable performance in areas V. Text Reference and VI. Considered Opinion.
 

Strengths and Weaknesses

            Strength—100% Target Performance—Areas I -IV

            Weakness—13% Unacceptable Performance—Areas VI

Conclusions and Possible Changes to be Made Based on Data:

            Given the 13% unacceptable performance in the area VI, faculty should look at earlier intervention.  Using the “practice” option available on LiveText may assist candidates in their performance analysis and provide opportunities for improvement prior to the “official” submission.  In looking at individual papers, students need more practice using research to support considered opinions. Case Studies are also used in EDUC 528 and Section VI was the performance area weakness there.  Given that both courses show that weakness, particular attention needs to placed on instructional practice.

Literacy Education Division Summaries

Fall 2012 Literacy Education Division Program Report

No text or image added.
Attachments

Spring and Summer 2012 Literacy Education Division Program Report

No text or image added.
Attachments

Fall 2011 Literacy Education Division Program Report

No text or image added.
Attachments

Spring and Summer 2011 Literacy Education Division Program Report

PROGRAM ASSESSMENT REPORT SUMMARY

DIVISION OF LITERACY EDUCATION

SPRING 2011 and SUMMER 2011

 

Assessments Administered at End of the Semester

Unit Level

            Professional Dispositions (EDUC 590, 592 & 594)

            Program Completers’ Survey/Exit Survey (EDUC 594)

            Research Assessment (EDUC 512 & 549)

            Survey of All 1 Year Post Graduates (EDUC 594)

            Employer Follow-Up Survey

           

Program Level

            Introduction to the Literacy Field (EDUC 588)

            Internship Project (EDUC 594)

            Field Experience Evaluation Form Summary (EDUC 592)

            ADEPT Instrument (EDUC 592 & EDUC 594)

            Material Selection Activity (EDUC 608)

            Program Completion Portfolio (EDUC 594)

           

 

Results and Conclusions

Professional Dispositions: All candidates were able to earn either acceptable or target on the assessment.  The means ranged from 2.44 to 3.00.

 

Changes in Assessment- No changes are necessary at this time.

 

Program Completers’ Survey: For the seven spring graduates, the means were extremely high for all of the competencies.  Means from the seven Spring candidates ranged from 4.71 to 5.00.  Some semesters the question related to “awareness of significant research” was not scored high, yet it was a 5.00 this semester.

 

Changes for Assessment-No changes are necessary at this time.

 

Research Assessment: Only nine candidates were assessed.  The results are similar to previous semesters.  All of the candidates assessed on the five competencies achieved either target or acceptable.    

 

Changes in Assessment – No changes are necessary at this time.

 

Survey of All 1-Year Post Graduates:  For the second consecutive year, the results were very positive with this survey.  In fact, we once again saw no competencies rated dissatisfied or very dissatisfied.  While the means were similar to last year for many of the five competencies related to literacy, it is interesting that the two lowest rated competencies last year (knowledge of how to initiate and evaluate professional development programs and the use of informal and formal assessments in meeting the needs of diverse students) were the highest rated competencies.  Similar to last year, the large number of negative comments and the number of low rated competencies are no longer present. 

 

Changes in Assessment- No changes are necessary at this time.

 

Employer Follow-Up Survey:  Only three people completed this survey.  On all five competencies analyzed, one person selected dissatisfied or very dissatisfied.  The other two people selected either satisfied or very satisfied.  While two comments were very positive, a third employer clearly did not feel the teacher had the content knowledge or instructional strategies.  While the means are not high, it is difficult to generalize with such a limited number of respondents.  Further data will be closely monitored.

 

Changes in Assessment- No changes are necessary at this time.

 

Introduction to the Literacy Field: In recent semesters, there has been a strong correlation between those candidates who do not successfully pass the competencies and those who do not successfully complete the class.  Last Fall saw two candidates earn unacceptable on the first requirement which pertained to IRA 1.1.  Both candidates dropped the course.  This semester only one candidate did not earn acceptable on any of the competencies.  That candidate did not successfully complete the course, and has been removed from the program.  This course was offered during a summer term to the current cohort, and this assessment demands a great deal of work.  The fact that the vast majority of candidates did well on such a demanding assessment is amazing. 

 

Changes in Assessment- No changes are necessary at this time

 

Internship project: This is the second consecutive year where all candidates achieved acceptable or target on all competencies.  Instead of the 39 candidates which completed it last year, only 7 candidates completed the assessment.  On most of the competencies l00% of the candidates met the target level. 

 

Changes in Assessment – No changes are necessary at this time.    

 

Field Experience Evaluation Form Summary:  While the results have been more favorable in recent semesters, it is difficult to make any type of conclusions based on two responses.  Once again quite a few competencies (3) had l00% select “I agree.”  Similar to the last time the assessment was completed, no literacy candidates chose “I disagree” for any of the statements.

 

Changes in Assessment – The results will be discussed with the instructor to see if any changes are recommended.

 

ADEPT Instrument:  Overall, the percentage of responses with a mean of competent is very high and no students rated unsatisfactory this semester.  In Fall 2009, there was an issue with high numbers of students receiving NA on various competencies.  It was hypothesized that it was due to the large number of adjuncts hired to teach during that semester.  The percentage rated NA is again very low except for the competencies related to classroom management and the entire school.  That is due to the fact that the literacy candidates do not work with an entire classroom in the course, so the instructor cannot assess their ability to establish classroom management.  There does not appear to be a problem related to the inability to rate students on competencies. 

 

Changes in Assessment –No changes are necessary at this time.

 

Material Selection Activity:  While the means were slightly lower than last year, none of the students scored unacceptable on any of the competencies.  It is also important to note that halfway through the semester there was a change in professors, so the one who graded the assessment did not begin teaching the course until approximately a month before it was due. 

 

Changes in Assessment –No changes are necessary at this time. 

 

Program Completion Portfolio:  This semester, as opposed to last year, the assessment did not contain a lot of data because there was not a CCSD cohort completing the program.  Also, for the first time (and only time) there were only two reviewers reading the portfolios.  This was due to one faculty member needing to take FMLA during the semester.  Unlike previous semesters, none of the candidates failed an artifact.  In fact, only two artifacts had a candidate or candidates receive only an acceptable rating.  This was a unique semester because no candidates had to revise artifacts.  The data will be closely monitored next spring when the assessment is completed again because there will be a number of students completing the assessment due to the graduation of the cohort.

 

Changes in Assessment– No changes are necessary at this time.

Fall 2010 Literacy Education Division Program Report

PROGRAM ASSESSMENT REPORT SUMMARY

DIVISION OF LITERACY EDUCATION


Fall 2010  Assessments Administered at End of the Semester


Unit Level

            Professional Dispositions (EDUC 591 & 592)

Program Level

            Introduction to the Literacy Field (EDUC 588)

            EEDA Assessment (EDUC 570)

            Practicum Portfolio (EDUC 591)

            Field Experience Evaluation Form Summary (EDUC 592)

            ADEPT Instrument (EDUC 591 & EDUC 592)

Results and Conclusions

Professional Dispositions: While most semesters, one candidate does earn the “rarely evident” rating on one competency, this semester no candidates earned a “rarely evident” rating.  The means continue to be extremely high.  At this time no changes are necessary based on the data.  The means ranged from2.88 to 3.00.

Changes in Assessment- No changes are necessary at this time.

Introduction to the Literacy Field: This semester saw two candidates earn unacceptable on the first requirement which pertained to IRA 1.1.  Both candidates dropped the course.  It is evident that students who were unable to meet the IRA Standards at an acceptable level will not be continuing on in the program until they are able to do so.  This was not the case in the past, and that was one goal that the division set after prior semesters’ analysis of data. 

The means were quite high this semester.  The “professional organization activity” mean was high but lower than past semesters.  That is due to the professor teaching the course. In order to achieve target, candidates must join a literacy professional organization.  At the current time, with teachers experiencing furloughs and cut backs, money is “tight” for everyone in education.   However, candidates did learn about a variety of organizations, and more of them will hopefully join one in the future.  Since it is important to encourage their membership, I believe target should require that students join one organization.

Changes in Assessment – No changes are necessary at this time.    

EEDA Assessment:  This was the second time this rubric was used.  While last semester four earned unacceptable, three were given incompletes and had to successfully complete the assignment prior to earning credit for the class.  The fourth student failed the course.  This semester all students passed the course, and l00% achieved target on each area of this assessment. 

Changes in Assessment – No changes are necessary at this time.   

Practicum Portfolio: No candidates performed at the unacceptable level on any standards.  On all six of the standards the vast majority of students performed at the target level.  Target levels ranged from 72% to 100%.  While no candidates earned an unacceptable rating on Reading and Writing as Life-Long Activities (IRA Standard 4.3) this semester, that competency still appears to have a lower mean than the other standards.  In the future, more emphasis should be placed on explaining the requirements for that competency. 

Changes in Assessment –No changes are necessary at this time.

Field Experience Evaluation Form Summary: The results from this semester are extremely positive.  This assessment has seen mixed results in past semesters over the years, but the results have been more favorable in recent semesters. 

In semesters from previous years, there have at times been low percentages for both competency 6 and competency 8.  While last semester the mean increased to a 2.5 for competency 6, it maintained that rating this semester.  The mean for competency 8 increased from 2.00 to 3.00 this semester.  The data also showed that more students felt they had an opportunity to work with students with exceptionalities, and the mean for that competency increased from a 2.00 to 2.83.  The data this semester is extremely positive for this semester compared to recent semesters.  Collecting the data is very valuable because it allows the instructors to see where weaknesses exist and to make changes in the course even when the assessment does not need modified.

Changes in Assessment –No changes are necessary at this time.

ADEPT Instrument:  Overall, the percentage of responses with a mean of competent is very high and no students rated unsatisfactory this semester.  Similar to last semester, the percentage of students rated NA is again very low, so there does not appear to be a problem related to the inability to rate students on competencies.  Prior to last semester, that area was an issue. Since 1.D (The teacher develops appropriate processes for evaluating and recording students’ progress and achievement.) and 2.B (The student develops instructional plans that include content, strategies, materials, and resources that are appropriate for the particular students.) have slightly lower means, instructors will be asked to target those areas so that improvement is seen in future semesters. 

Changes in Assessment –No changes are necessary at this time.

Spring and Summer 2010 Literacy Education Division Program Report Summaries

PROGRAM ASSESSMENT REPORT SUMMARY

DIVISION OF LITERACY EDUCATION


SPRING 2010 and SUMMER 2010

Assessments Administered at End of the Semester

Unit Level

            Professional Dispositions (EDUC 590, 592 & 594)

            Program Completers’ Survey/Exit Survey (EDUC 594)

            Research Assessment (EDUC 512 & 549)

            Survey of All 1 Year Post Graduates  (EDUC 594*)

            *In addition, two other candidates were assessed in the summer.

Program Level

            Internship Project (EDUC 594)

            Field Experience Evaluation Form Summary (EDUC 592)

            ADEPT Instrument (EDUC 592 & EDUC 594)

            Material Selection Activity (EDUC 608)

            Program Completion Portfolio (EDUC 594)

            Literacy Leader Field Experience Folder (EDUC 599)

            Journal of Reflections on Literacy Research (EDUC 596)

Results and Conclusions

Professional Dispositions: In only one instance, did a candidate earn a “rarely evident” rating.  The means ranged from 2.88 to 3.00.

Changes in Assessment- No changes are necessary at this time.

Program Completers’ Survey: For the thirty-seven spring graduates, 96.79%-100% of the candidates were either satisfied or very satisfied with each of the seven competencies.  The means ranged from 4.59 to 4.95 for those candidates.  The two candidates from Summer I ranked all competencies as either satisfied or very satisfied.  A great deal of positive feedback was received from numerous candidates related to the level of knowledge gained, the overall program, and advising and helpfulness of professors.  Negative comments were only received from two candidates.  One set of those comments pertained to a faculty member no longer employed in the School of Education.

Changes for Assessment-No changes are necessary at this time.

Research Assessment: Only ten candidates were assessed.  Overall candidates scored very well on each of the five competencies.  All of the candidates assessed on the five competencies achieved either target or acceptable.    

Changes in Assessment – No changes are necessary at this time.

Survey of All 1-Year Post Graduates:  Last year the same number of graduates completed the survey, and yet the differences are amazing.  Last year 6 of the l5 general competencies related to graduate work received dissatisfied and 3 of the 5 literacy specific competencies saw a dissatisfied rating.  The means also saw a dramatic improvement and some of the competencies saw an increase of an entire point in the mean!  All means improved.  Last year there were several candidates dissatisfied with various competencies and comments related to the lack of current research taught, the assessments that graduates feel are outdated, and the fact that an out of print textbook is used for the literacy assessment course.  Changes were implemented to correct those issues, and data is monitored.    None of these concerns were mentioned in this year’s data.  This year the comments were all positive.  One graduate stated that “I would highly recommend this program to my colleagues and others seeking a Master’s degree.”  Another student “loved the Literacy program at The Citadel.” and one stated that the program provided “greater motivation to teach students how to read through a variety of teaching strategies.  This assessment saw some of the largest positive changes in means.

Changes in Assessment- No changes are necessary at this time.

Internship project: Candidates were assessed on fifteen areas on this assessment. Last year two thirds of the data was missing because one faculty member (who no longer teaches courses) did not complete the required rubric on students.  This year all data was available for analysis.  The data confirms that candidates are meeting the standards.

On another note, instead of 5 candidates completing the project, 39 completed the project.  The results reveal that the means overall are much higher than last year.  Last year 20% of candidates achieved unacceptable on one competency.  This year all candidates achieved either target or acceptable on that component.  There was only one component where one candidate earned a score of unacceptable, and it appears that the student may have misunderstood the directions.

Changes in Assessment – No changes are necessary at this time.    

Field Experience Evaluation Form Summary:  Once again quite a few competencies (4) had l00% selected “I agree.”  However, it is also noteworthy is that no literacy candidates chose “I disagree” for any of the statements.

In semesters from previous years there have at times been low percentages for both competency 6 and competency 8.  This time 50% chose “I neither agree or disagree” with the fact that the field experience helped them gain a better understanding of classroom management.  While Competency 8 was reworded in recent years to include “teaching career” instead of just “student teaching internship”, 50% still chose NA in relation to the field experience preparing them for their student teaching internship or teaching career.  The vast majority of literacy students are already teachers, so that may influence the value they see in gaining classroom management skills with the field experience.  However, the field experience should better prepare candidates for their career even though half of the candidates felt the statement was not applicable. 

Changes in Assessment – The results will be discussed with the instructor to see if any changes are recommended.

ADEPT Instrument:  Overall, the percentage of responses with a mean of competent is very high and no students rated unsatisfactory this semester.  In Fall 2009, there was an issue with high numbers of students receiving NA on various competencies.  It was hypothesized that it was due to the large number of adjuncts hired to teach during that semester.  The percentage rated NA is again very low except for the one competency related to classroom management in the Summer, so there does not appear to be a problem related to the inability to rate students on competencies. 

Changes in Assessment –No changes are necessary at this time.

Material Selection Activity:  While the means were lower on some components of the assessment compared to last year, they were higher on others.  This project is one of the most demanding in the program, and the fact that all candidates are able to achieve either target or acceptable is a positive sign that they can meet the standards.  The last two years the means have been lower than previous years, but it may be due to the fact that the components are more accurately analyzed and there is less inflation of scores.  The data will be closely analyzed next spring since this is a once a year course.  

Changes in Assessment –No changes are necessary at this time. 

Program Completion Portfolio:  This assessment is working very well.  Once again all artifacts were reviewed by three professors.  Numerous candidates did not pass multiple artifacts.  They revised them until they could demonstrate the necessary competency and were able to earn an acceptable rating from two professors.  This was the second year that candidates had to revise artifacts, and it appears overall the faculty members are discriminating between the artifacts that adequately meet the standards and those that do not.  Candidates successfully demonstrate knowledge of the standards even if it requires artifact revisions. 

There is one artifact which had l0% of the ratings at the not acceptable level.  The unacceptable ratings are primarily from one faculty member.  Since two of the three reviewers passed those artifacts, the candidates were not required to revise them.  The artifact pertains to modeling reading and writing for real purposes. 

Changes in Assessment– The three reviewers will discuss the one artifact requirement prior to reviewing program completion portfolios in Spring 2011 so that there is more consistency among ratings.  No other changes are needed.

Literacy Leader Field Experience Folder- Last summer there appeared to have been a misunderstanding regarding expectations for citations within the reflection component.  As stated on last year’s summary report, theexpectations were clarified this year, and the means were higher than last summer.  The number of students completing this assessment was much smaller this summer.  There were 8 candidates rather than 39 completing the assessment.  The data will continue to be monitored.  No candidates achieved an unacceptable rating on any of the assessment components.  Overall, candidates did well on the assessment. 

Changes in Assessment – No changes are necessary at this time. 

Journal of Reflections on Literacy Research-This year the assessment results reveal that means are higher on all three standards.   Unlike last year, there were not large numbers completing the course.  However, none of the candidates earned an unacceptable rating on the standards.   This course requires an extensive amount of reading and writing even though it is a summer class.  Students constantly comment about the requirements of the class.  Sometimes these are noted on course evaluations, but often they come in up oral comments when fall classes begin.

Changes in Assessment – No changes are necessary at this time.