On Friday, February 8th at 2:00pm Eastern Time, join Taking the PULSE for a live webinar update. Thomas P. Jack, Kathryn Miller and Kate Marley will be briefing you on progress and taking your questions, input and feedback.
Simply reply to this discussion below with your questions and input. We'll cover as much as possible during the webinar, and the discussion can continue here after that.
I was talking to Kate about this but she is in class now and I may forget to ask her so I am posting it. And it is worth sharing this discussion with the community.
Just another thought on this ranking system. We grade students on all their work. We don't stop and give them a rank of C when they can't answer question #75, even if they get all the other questions correct on the exam. Kate made the point that there are/will be a huge set of criteria to rank institutions with. So being blocked here does not mean that the institution could not get high marks elsewhere.
I agree with that. But if we graded our students this way our evaluation of them would at beat be accurate. That is, if a student was a 75% student in all their classes and efforts and they were uniformly bad at every thought/concept above 75% this ranking system would be spot on. We don't write our tests this way. But even if we tried to write our tests that way, some students are good memorization but bad at concept development. A student like that would get a C. Others might be great at developing concepts but can't remember 1492. That student would rank very low on a rubric scale of this sort.
In short, it seems that the ranking system at best will be accurate, but will tend to rank programs lower than they are and not recognize them for all that they do.
I love the Lead certification system as applied to this problem, but I am still not sure I see the whole picture.
Thanks again for all your work!
Great Job! Are you collecting ideas (like Brad referred to) for how to approach administration and not just leave it to "hoping for recognition"?
One of the ways I am planning to leverage V&C is in articulation with universities. If we score highly on the rubrics then our students should be well-prepared to transfer. Our administration should support that goal.
great idea, thanks for sharing that. we will be compiling ways in which we can help department use this process to leverage resources and gather support and get their own faculties and administration on board.
To the PULSE community: please do share any ideas that you have on how such a system can be used to get administrators and all other stakeholders on board with V&C.
Broad input is vital
Melanie, We are hoping that once the V & C recognition is established, that there might ultimately be some tie-in with the funding agencies. For example, when you write an NSF proposal, you have to comment on the "Intellectual Merit" and "Broader Impact". If approval or ranking in terms of the V & C report was tied to grant funding, the administrators would pay attention very quickly. Of course, this would not occur immediately and would depend on how robust and valid the rubrics and recognition process becomes....
A huge thanks to everyone who joined us live, and please continue the discussion. Your input is vital : )
I can also see how a ranking system like this would have a threshold where schools in position to quickly take advantage of the new ranking system would be able to build there program in leaps and bounds via great recruitment. Schools without the resources or where support has to be built up would fall behind and likely continue to fall behind.
Sending a student to a college that doesn't rank, could be buffered at some level by large research/state institutions. As I suspect their ranking will always be lower than the smaller colleges.
for example, I see a very quick three-tiered system
Tier 1 are all the institutions with the resources in place now to take advantage of the new ranking system.
Tier 2 would be large institutions that have the resources but have an upper ceiling on how high they can rank due to the nature of their institution.
Tier 3 would be institutions without those resources And would probably be small colleges/universities.
I'm not sure what the gap would be between tier 2 and tier 3 institutions. A ranking system like this would definitely make it extremely difficult for tier 3 institutions to ever garner enough resources to build themselves out of the hole.
I'm not sure there is a fix for this. It may be worth evaluating in your rubrics. There are a lot of inexpensive ways to get very close to the top in your ranking. But I suspect to be at the top of all of these rankings resources need to be available. And the schools with those resources available to start with are the ones most likely to get to the top and stay on the top.
Brad, we certainly have discussed the issue of institutions with less resources vs those with more; we are struggling to make sure that every aspect of recognition does not rely on resources. But as you point out it is a tricky situation. That is why piloting the rubric and iteratively developing the final scoring system is key. thanks for the comments and the reminder about this important issue.
Great job, Kate, Kathryn, and Tom!
Thank you, this was very informative.
I really like the direction you are taking. Great job.
Some good ideas. I like the LEED type scoring system, with a menu of possibilities. Looking forward to seeing the actual rubrics and criteria for each category. It will be useful if only for institutional self-assessment. For certification to work as a driver for change, we need trend-setting institutions to adopt in a splashy way - look how many are jumping on to the MOOC bandwagon, afraid to be left behind.