Outcomes Assessment: Making Student Learning Visible

Print Friendly

Summary

Stella and Charles Guttman Community College (CUNY) identified ePortfolio and its central relationship to outcomes assessment in its initial concept paper and throughout the development and planning processes for the college.  Assessment is central to our learning at the college.  Faculty and staff at the college are engaged in a continual process of “assessment for learning and improvement” through regularly scheduled Assessment Days and professional development seminars series devoted to holistic assessment of our student’s curricular and co-curricular growth as demonstrated through their ePortfolios. With a widely established implementation of ePortfolio for Outcomes Assessment, our practices are transformative, providing opportunities for learning at the course, program, and institution level.

Authors

Laura M. Gambino
Chet Jordan
Nate Mickelson

For a printable, PDF version of Guttman’s Scaling Up Story, Click Here.

Description

Part I: Setting the Stage

Guttman’s approach to assessment for learning is based on four principles:

  • Assessment is an ongoing, recursive process.
  • Assessment provides an engaging space for learning.
  • Assessment focuses on improvement and learning for students, faculty, staff, and the institution.
  • Assessment is a shared endeavor.

Through regularly scheduled Assessment Days, faculty, administration, staff, and students are actively involved in using data for learning and improvement. In addition to the formal work of Assessment Days, faculty and staff meet weekly in instructional teams to reflect on student progress, assess curricular successes and challenges, and adapt and refine pedagogies and assignments.

Guttman’s Assessment Team is led by Laura Gambino, Professor and Faculty Scholar for Teaching, Learning and Assessment; Elisa Hertz, the Director of the Center on College Effectiveness; and Stuart Cochran, Dean for Strategic Planning, and supported by a Standing Committee on Assessment and Professional Development elected by the College Council. The college is accredited by the New York State Board of Regents and has begun a multi-year evaluation with the Middle States Commission on Higher Education.

Part II: Developmental Story

ePortfolio serves as the primary vehicle for assessment at Guttman, using authentic student work to create a culture of learning and continuous improvement.  Stella and Charles Guttman Community College (CUNY) identified ePortfolio and its central relationship to outcomes assessment in its initial concept paper and development and planning of the college.  In addition to emphasizing the importance of assessment and the role ePortfolio would plan in that process, those initial planning phases identified the creation of a Center for College Effectiveness, which brings together institutional research, assessment and professional development and an Assessment and Professional Development Committee, a College Council committee consisting of elected faculty and staff.  The initial planning documents also identified the need to hire a “faculty with assessment expertise” to oversee the use of ePortfolio and the assessment of student learning outcomes for the college.   That position was viewed as a hybrid position – someone with teaching responsibility who would also have an administrative role in the Center for College Effectiveness.  Lastly, the planning committees developed the concept of Assessment Days – non-teaching days built into the academic calendar where faculty and staff focus on assessing student learning.

In 2011, the college sent a team of faculty and administrators to participate in AAC&Us General Education and Assessment Summer Institute.  That team, along with a larger group of faculty and staff, developed the college’s Institutional Student Learning Outcomes and rubrics.  These outcomes “encourage students to aim high and provide them with a framework for their entire educational experience, connecting school, college, work and life. These outcomes build on Lumina Foundation’s Degree Qualifications Profile and are informed by AAC&U’s LEAP Essential Learning Outcomes. They are an inclusive framework for a contemporary liberal education, defined not as a selected set of disciplines, but as a set of knowledge and skills for all aspects of life:  school, work, citizenship, and social responsibility. They are reflective of the NCC’s mission and values.” In 2013, when our college was renamed, we “re-branded” our Institutional Student Learning Outcomes to Guttman Learning Outcomes.

In August 2012, the college hired Laura Gambino to fill the “faculty with assessment expertise” position.  The following month, Elisa Hertz joined the college as the Director of the Center for College Effectiveness.  They work together to guide the college’s assessment activities.

In a forthcoming article, Gambino described the development of Guttman’s Outcomes Assessment story:

Guttman has a strong commitment to on-going, comprehensive assessment which examines student learning at both the individual and aggregate levels. This commitment is centered on the concept of using assessment for learning (Barrett, 2004) across the various layers of the institution— students, faculty, and programs— with the ultimate goal of improving student learning, persistence, and success.  Since our assessment structures were constructed with a strong emphasis on examining student learning via ePortfolios, we were able to move quickly into practice – examining work, reflecting on findings, and identifying and implementing changes.

Our assessment of student ePortfolios began last year when we examined portfolios from the Summer Bridge Program.  Based on that evaluation, we found the majority of students were not comfortable using ePortfolio and that the SBP experience was fragmented. We revised the curriculum and created a more cohesive experience, integrating ePortfolio into each component of the program.

Ongoing assessment is essential to our work; coordinated by our Center for College Effectiveness, dedicated assessment days are scheduled into the mid- and end-points of each twelve week fall and spring session.  While a variety of assessment and professional development activities take place on these days, the primary goal is for faculty and staff to work together assessing first year student work and reflections from their ePortfolios.  Instructional teams, which include faculty and student success advocates, conduct an assessment of each student.  Teams then identify any individual interventions and action plans needed to help students succeed academically.

A team of faculty and administrators attended AAC&U’s General Education and Assessment Summer Institute in June, 2013 and, informed by  the work of our colleagues at LaGuardia Community College (Arcario, Eynon, Klages & Polnariev, 2013), developed a comprehensive GLO Assessment Plan. Teams of faculty, staff, and students at the college will engage in a 3-year inquiry, reflection, and integration cycle, examining snapshots of student ePortfolios collected at various milestones: the conclusion of Summer Bridge, the end of the first year experience, and graduation.  These snapshots will allow us to look longitudinally at how students are learning and growing in relation to the GLOs, identify any needed curricular improvements, and implement changes. (in press)

In December 2013, we launched our first two GLO teams.  These teams, which consist of faculty and staff from across the college, are beginning the inquiry phase of their work.  ePortfolios will be at the center of the assessment process.  Students will incorporate signature assignments and reflections in their learning ePortfolios.  Students will submit their ePortfolios to Digication’s assessment management system at the following milestones:

  • Conclusion of Summer Bridge Program
  • End of First Year
  • End of Second Year

In addition to the work of the GLO teams, we continue to examine student work and ePortfolios in relation to course, program, and student learning outcomes during our Assessment Days, identifying areas of strength and excellence along with areas for improvement.  [For more about Assessment Days, click here.]

Gambino summarized our work in this way:

We envision our assessment work as an ongoing, action-research process— planning, evaluating, reflecting and implementing changes in a cyclical process over time.  Our work will mature and deepen as our students move through their programs.  Our goal is to become a learning organization, where faculty, staff, programs, and the institution learn and grow as our students learn and grow. (in press)

The chart below identifies the key stakeholders in Outcomes Assessment at Guttman and describes their role in our current practices.

Stakeholder’s Description Role in ePortfolio Outcomes Assessment
Students: Guttman enrolled entering classes of 300 students fall 2012 and fall 2013.  First-year students are grouped in four Houses (extended learning communities). Second-year students elect one of five majors: Business Administration, Human Services, Information Technology, Liberal Arts & Sciences and Urban Studies. Students are active partners in developing a culture of evidence and assessment. ePortfolio allows students to engage in active reflection with one another and their instructors across courses and disciplines and to develop identities that relate to academic and professional goals.  Students are aware of and understand course-level learning outcomes as they relate to individual assignments.  Guttman’s First Year Experience requires students to complete integrated assignments that emerge from current issues facing New York City and cities around the world.
Faculty: Each member of the faculty holds a unique pedagogical and research role in support of the model of the college.  All full-time faculty teach First Year Experience courses and are part of instructional teams (learning communities). The faculty drives both course-level and institutional outcomes assessment.  To support this work, the Assessment and Professional Development Committee and C2L team offer ongoing professional development centered on ePortfolio-based assessment strategies.  As part of the college’s Assessment Days, instructional teams use collaboratively-developed rubrics to assess student work in relation to GLOs. Faculty members use the results of these assessments to refine their curricula and courses.
Center on College Effectiveness (CCE): The CCE merges the traditional functions of institutional research and a Center for Teaching and Learning. The CCE embodies Guttman’s commitment to integrating professional development, research, and assessment for learning.  The Center facilitates and enables collaborative inquiry into student learning and institutional effectiveness, serving as a partner to faculty, staff, and administrators in advancing the college’s culture of learning and bringing resources and attention to the value of intentional reflection for institutional learning.
Office of Student Engagement (Student Success Advocates, Graduate Coordinators, and Peer Mentors): The Office of Student Engagement offers structured academic, personal, and professional support for students.  Student Success Advocates provide individualized support to students and track their progress as part of the instructional team.  Graduate Coordinators bring their specialized academic and professional experience to the classroom and are responsible for coordinating student Group Workspace.  Peer Mentors serve as undergraduate models for students to learn academic planning, become work-conscious, and also provide students with a peer to grow alongside. As part of Guttman’s innovative learning community approach to community college education, the Office of Student Engagement engages with ePortfolio outcomes assessment in order to drive student success, not only within the curriculum, but also in an effort to link the coursework with the co-curriculum and professional service activities such as internships.  Each tier of this specialized, whole-student approach to advising relies on the results of academic outcomes assessment as well as holistic development as represented in student ePortfolios to guide and inform advising practices.
Administration: The President, Provost, Vice President for Finance and Administration, the Registrar and other members of the administrative team are active participants in developing curriculum based on focused outcomes assessment.  They are also responsible for reporting the results and implemented changes to the larger university structure. ePortfolio provides a pivotal point of connection within the larger structure of our college.  Our administration evaluates faculty outcomes assessment, holistic assessment from the Office of Student Engagement, and analysis from the CCE and integrates these results into course schedules and instructional teams that best support student learning.  In addition to collaborative evaluation practices, the administration is commitment to providing students with access to the state-of-the-art technologies needed to support ePortfolio pedagogies.
Community Partners: Our community partners are an integral part of the success of the model of the college.  Guttman’s mission states that the college serves to extend outward into the community and also weaves its partners into the fabric of the college culture. ePortfolio gives our community partners an opportunity to engage in our culture of assessment, as well.  Students are actively developing and reflecting on their personal ePortfolios and are creating a professional montage of their work for public consumption.  This mode of expression allows our partners access to reflect on the effects student work might have in the broader community.

Part III: Conceptual Framework

Guttman’s two main outcomes assessment structures – Assessment Days and GLO Teams – incorporate the design principles of inquiry, reflection and integration. These principles allow us to focus on assessment for learning and improvement across the college, keeping student learning visible at each phase of our work.

Our Assessment Days provide the community ample opportunities to use ePortfolio-based outcomes assessment as a point of inquiry for asking the larger questions about how the integrated curriculum is shaping student learning.  Through an intentional focus on student learning, the college is developing a culture of evidence and assessment that drives individual, team, unit and institutional growth and change.  Standing committees within the college governance structure are also actively engaged in assessment and are using outcomes to inform their decisions.

In our inaugural year, reflection was largely centered on anticipated outcomes. This type of calculated anticipation led to several changes between the fall and spring semesters and in preparations for the incoming class of freshmen for fall 2013.  For example, faculty and staff transformed Group Workspace, a required structured support session that is part of the First Year Experience, into Studio, a student-centered space for skills practice, through focused assessment of the student experience.  The redesigned Studio offers an ideal setting for students to engage in reflective practices around their integrated assignments using ePortfolio and take ownership of their learning in each of their courses. As we celebrate our first graduating class in spring 2014, our reflections will naturally shift to questions related to the ways Guttman prepared students for success at four-year colleges and in careers.

The GLO Team Assessment Plan outlines a three-year plan for the assessment of each of our Guttman Learning Outcomes.  The outline for the three years is as follows:

Year 1 – Inquiry:

  • Identify and refine course assignments and corresponding reflections in relation to outcomes and rubrics
  • Examine
    • what students are asked to do
    • how we communicate expectations to students
    • how we scaffold learning opportunities across the curriculum
  • Identify the research question that the GLO team will investigate

Year 2- Reflection:

  • Collect direct and indirect data
  • Conduct norming of authentic student work examples
  • Conduct assessment of student work using the GLO rubric(s)
  • Analyze and evaluate findings throughout the year, making mid-point corrections, as needed

Year 3 – Integration:

  • Identify areas for improvement
  • Modify curriculum
  • Begin implementing changes

 Connections to Other Sectors of the Catalyst:

Scaling Up:  Because we use ePortfolio “at scale,” the connection to Outcomes Assessment helps deepen our use of ePortfolio across the institution. Our assessment work is guided by our Faculty Scholar for Teaching, Learning, and Assessment along with the Assessment and Professional Development Committee.  Our strategic plan does include the use of ePortfolio for assessment at the college and faculty were invited to participate in our GLO teams.   Membership on a GLO team will be recognized in the reappointment, promotion, and tenure process.

Pedagogy:  Putting ePortfolio at the center of our assessment processes makes student learning visible.  As faculty, our assessment conversations focus on curriculum, pedagogy, and learning.  Reflective and integrative pedagogy is central to student learning and, therefore, central to the assessment of our Guttman Learning Outcomes.

Professional Development: Professional development will continue to be a major focus for the foreseeable future as we seek to fully integrate outcomes assessment strategies and ePortfolio into the daily practices of the stakeholders across the college.  Our efforts will focus on how reflective practices enhance the work of each stakeholder group within the college.  Activities for staff, administration and faculty alike will be directed toward the goal of fully integrating assessment as a tool for learning and assessment.  Course-level professional development will allow faculty members to see how assessment integrates with the design of their classes and assignments as well as how reflective assessment strategies foster the growth of lifelong learning.  Through these efforts and others, ePortfolio has the potential to serve as a practical demonstration of student and institutional success that supplements the data-driven analysis of grades, credits, persistence, retention, and graduation rates.

Technology:  Our ePortfolio platform, Digication, is a critical component of our outcomes assessment work.  Digication’s Assessment Management System enables us to capture snapshots of student ePortfolios at various milestones as they progress through the college.  We have also uploaded our GLOs and rubrics to the assessment system.  We then use the AMS to put together samples of ePortfolios that can be used for assessment, score those assessments using our rubrics, and produce reports based on those efforts.

Conclusion

Since fall 2012, faculty, staff and administrators at Guttman have used outcomes used to assess the quality of student learning as demonstrated in integrated assignments. The college has held six sets of Assessment Days (October 2012, December 2012, February 2013, April 2013, October 2013 and December 2013) and numerous professional development workshops. Substantive changes have been made to two key elements of the academic program, the Summer Bridge and Studio, based on the results of formative assessment. As of fall 2013, college-wide assessment teams have begun three-year cycles of inquiry, reflection and integration in relation to Civic Learning Intellectual Skills GLOs. Instructional teams continue to meet on a weekly basis to discuss student learning and curricular effectiveness.

As we continue to build our assessment practices and deepen our commitment to a culture of evidence and improvement, we have identified three main areas for improvement: increasing awareness of GLOs and assessment among students; building understanding among faculty, staff, and administration of the benefits of assessment to individual and unit-level work; and enhancing the usefulness of our data-gathering instruments.

To increase awareness among students, the College must make an intentional effort to introduce the GLOs. Telling students about the outcomes will not be enough. Outcomes should be referenced on all assignments (or at least on major assignments) and students should be invited to reflect on the degree to which they have demonstrated mastery in their work. Students should be asked to explain the outcomes in their own words and to reflect on how mastery of the outcomes will benefit them in the future.

To increase understanding among faculty, staff and administration, the College must continue to reserve time for plenary and small group assessment activities. Members of the College will not view assessment of student learning as a priority until they see assessment results used in institutional decision-making. More importantly, faculty and staff should be supported in applying assessment results to future assignments, activities and teaching strategies.  We must connect and align our outcomes assessment work with professional development activities and this work must be recognized as a significant contribution to the scholarship of teaching and learning that we value in our culture.

To improve the usefulness of our instruments (the outcomes and rubrics), the work of the Curriculum and Committee and the Assessment & Professional Development Committee should build on the discussions held during the Assessment Days in order to align institutional, program and course outcomes and to identify rubric score definitions that would benefit from revision. The larger college community should be engaged in this work.

References

Arcario, P., Eynon, B., Klages, M. & Polnariev, B. (2013). Closing the loop: how we better serve our students through a comprehensive assessment process. Metropolitan Universities Journal, 24(2), pp. 21-37.

Barrett, H. (2004). Electronic portfolios as digital stories of deep learning: Emerging digital tools to support reflection in learner-centered portfolios. Retrieved from http://electronicportfolios.org/digistory/epstory.html

Gambino, L.M. (in press).  Putting ePortfolio at the center of our learning.  Peer Review.

Skip to toolbar