CIPP model for evaluation
In this blog post I would like to discuss the CIPP model for evaluation Stufflebeam & Shinkfield (2007), but more importantly how we are using it to build our OER courses here at the University of Southern Queensland. The CIPP model provides 4 evaluation types which are as follows:
Firstly, Context, for example assessing the user’s needs and identifying opportunities or problems relating to particular users or groups (a little more on this later).
The second evaluation type is Input, this relates to the design aspect of how knowledge is acquired by the user. Some courses lend themselves to a more social or open design where micro-blogging or social media play a big part in the learning experience, personal mentoring is another consideration here. Perhaps the most important factor to consider when looking at this evaluation type is the software platform or delivery mechanism for the course in question. If we are designing the course for engagement, we need to be sure that the technology is able to support this.
Process is the third evaluation type. This relates to the actual implementation of the course and its related activities. We, will be focussing on what learners actually do during the course.
Finally Product; Product in a nutshell relates to return on investment, comparing actual outcomes against efforts made.
Putting CIPP into practice
Just to elaborate further on the first context evaluation type and pull this all together, let’s say we perform a survey at the start of a given course where a given number of users identify their intention to gain full credit for the course. If we have a KPI relating to this metric, we can make this part of the CIPP model for evaluation. We can combine survey results with empirical data or automated analytics (data mined from a course) then use this data to derive information and make decisions which improve the KPIs and ultimately the success of future course offerings. Essentially we are creating an opportunity to find out what is working, or more importantly what is not working for particular users or groups. Once this information is derived, we can improve the course with a degree of certainty.
We are focusing on improving user grades and creating independent self-directed learners. We are working to improve user retention and we consider future enrolment and further engagement a sign of successful implementation.
I recently spoke with Professor Ken Udas , Professor Jim Taylor  and Dr Wayne Mackintosh  about delivery mechanisms for open education resources. This conversation took place shortly after I had completed the ‘open content licensing for educators’ course (an OER course delivered on the WikiEducator platform).
We discussed the Wiki environment as a platform and commented on its usefulness around version control, co-authoring and collaboration.
The WordPress platform was introduced as a potential delivery mechanism. WordPress offers many advantages in this arena such as responsive design templates and WordPress themes as well as plug-ins, all of which allow rapid development and reuse.
Jim Tittsler and I have unofficially discussed a WordPress plug-in called wiki-embed which lets you embed MediaWiki pages into your WordPress site. I am in the process of installing and testing this plug-in.
A colleague and PhD candidate, Neil Martin  and I are also working together. Neil is investigating how User Experience design techniques could potentially improve the learning experience within the OERu.
Be sure to keep a keen eye on the Google summer of code as I understand The OER Foundation has been accepted as a mentor organisation for 2014.
Certainly a lot happening in this space, please leave any questions or comments below and enjoy open education week.
 Senior Analyst and Demonstrator at the University of Southern Queensland
 Board of Directors of the OER Foundation and Emeritus Professor for the Australian Digital Futures Institute (ADFI) at the University of Southern Queensland
 Deputy Vice-Chancellor, Academic Services and Chief Information Officer (CIO) at the University of Southern Queensland
 Learning Technologist for the Australian Digital Futures Institute (ADFI) at the University of Southern Queensland
Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models, and applications (pp. 240-241). San Francisco, CA: Jossey-Bass.