One of the important things to do with your PSOL data is to look at the gaps – the differences between the importance ratings and the satisfaction ratings expressed by the students. You’re always concerned about the larger gaps which indicate where you have significant room for improvement, if it is an item where you have much control over (or impact upon) the level of student satisfaction.

At the same time, I like to look at the smaller gaps. You also need to have some positive reinforcement for those things that are going very well – for those things where you are essentially meeting the student expectations. The seven items listed below come from the 26 standard items on the PSOL and indicate the lowest performance gaps for Lake Superior College in the FY06 satisfaction survey. The item number is shown first, followed by the full text of the item, and then the size of the gap (importance score minus satisfaction score).

08. Student-to-student collaborations are valuable to me = (.42)
This one is always interesting (see previous post about this item) because the students are basically telling you something like this: “I really don’t care much about this, but you’re doing a pretty good job with it!” This item has a decent (not great) satisfaction rating, but it has an extremely low importance score.

01. This institution has a good reputation = .02
This item is interesting to me because it tells me that we are serving the proper audience. Our students know who we are and what we can do for them. For the national numbers, the importance score is much higher than ours, which is why the national gap was .39 while ours was only .02. The satisfaction scores were almost identical, but reputation is more important to more of the students in the national survey. Keep in mind that many of those students are graduate students and have a very different demographic than our students.

18. Registration for online courses is convenient = .14
We have a low gap here because our registration system is completely online and has been for several years. It is managed by the state system (MnSCU) and seems to meet students’ needs quite effectively. This is always one of the most important survey items for students, but one with consistently high satisfaction scores.

21. Adequate online library resources are provided = .33
For several years now we have had significant online library resources available to all students. Our biggest issue is in getting them to use the resources, not whether the resources are online or not.

24. Tutoring services are readily available for online courses = .33
We are currently in our fifth year of offering online tutoring services through SMARTHINKING. We have always scored highly in this category, but you also have to keep in mind that only 10-15% of the students use tutoring services (either online or on-ground) and so the importance score is one of the lowest on the survey.

23. Billing and payment procedures are convenient for me. = .35
Much like our online registration, the billing and payment function is managed centrally for all 32 institutions. This is an item with a high importance score, but also a high satisfaction score.

19. Online career services are available = .35
This last one also is not very important to many of our online students, since so many of our online students are transfer students working on an A.A. degree of the Minnesota Transfer Curriculum. Our gap is low because this is not a hot topic for most of our students.

For me, the takeaways here are as follows:

1. Small performance gaps are often the result on items where the student ratings indicate a low importance score. That is the case for items 1, 8, 19, and 24 above. All four of those items have an importance score below 6.0 which generally indicates that most students rate them somewhere between “slightly important” and “important.”
2. Small performance gaps on the other three items (18, 21, and 23) indicate areas where you are really meeting the needs of the students. These items have high importance scores (above 6.0) and high satisfaction scores. These are things to be proud of.
3. Gap analysis is not an exact science. It is only a starting point in looking for places where you may be able to improve the services that are being offered to students.

(CC Flickr photo by Annie Mole)

Question 55 on the PSOL is one of three “Summary” questions that are placed near the end of the survey. Question 55 is a difficult question to make sense out of. It reads as follows: “So far, how has your college experience met your expectations?”

An earlier version of the survey appears to have included a slightly different question: “So far, how has the online experience met your expectations?” I actually prefer this older version of the question. Since we have students take this survey even if they are primarily on-ground students, their “college experience” can be quite different from their “online experience.”

Here are the possible responses to this question:

1. Much worse than I expected
2. Quite a bit worse than I expected
3. Worse than I expected
4. About what I expected
5. Better than I expected
6. Quite a bit better than I expected
7. Much better than I expected

On the satisfaction and importance scales, a score of “4” equals a neutral rating. Here a score of 4 equals “about what I expected.” Those are not the same things, or at least we have no way of knowing whether they might be about the same or wildly different.

The problem with this measure is that we have no idea what the student expectations WERE coming into the online learning experience. Meeting their expectations might be a great thing if they had high expectation, but clearly it could be a bad thing if they very low expectations. What if their expectations were that “online learning is going to be stink out loud,” would we be happy with a score of 4 for meeting those expectations? I think it is dangerous (and extremely presumptuous) to think that students came into online learning with high expectations. Is there any data to support this?

Could the differences in these columns shown above be caused primarily due to different expectations? I think that’s entirely possible. Even still, every group reports a better experience (above 4.0) than they expected, whatever those expectations might be.

PSOL Strengths for Lake Superior College

Our 2006 Institutional report for the Noel-Levitz PSOL identified the following strengths for LSC Online (listed in rank order beginning with the strongest item).

I’ve had a few discussions with Noel-Levitz representatives about how I think the “strengths” need to be redefined. Right now the strengths are what I would call “internal strengths,” without considering whether they are strengths in the larger context of the e-learning environment, or at least of the schools that have used this instrument.

• 11. Student assignments are clearly defined in the syllabus.
• 18. Registration for online courses is convenient.
• 33. Logging-in (managing usernames and passwords) for various services across the campus is easy and consistent. (Campus item 7)
• 6. Tuition paid is a worthwhile investment.
• 25. Faculty are responsive to student needs.
• 7. Program requirements are clear and reasonable.
• 3. Instructional materials are appropriate for program content. **
• 23. Billing and payment procedures are convenient for me.
• 13. The frequency of student and instructor interactions is adequate.

** Note: I am not willing to consider this a strength since our satisfaction score is slightly below (-.03) the national average for this item. Strengths are defined as those items above the mid-point in importance and in the top quartile of satisfaction, for your institution only. In other words, it is possible to have something identified as a strength when it is not a very favorable score, such as falling below the national average. I have a hard time seeing that as a strength. To really consider something as a strength, I believe that it must be both an internal strength (as defined by Noel-Levitz) and an external strength which means that it exceeds the performance level of the national group or preferably it exceeds the performance of a peer group (as defined by me).

Technorati Tags: PSOL, Noel-Levitz, LSConline