Piskurich_Chapter_11_Notes

Chapter 11 Notes

 * Primary purposes of evaluation:** (p. 154)
 * Determine if program or intervention is accomplishing objectives
 * Determine if performance gap was closed or narrowed
 * Determine if intervention met intended business goals
 * Determine benefit-cost ratio of HPI or HRD program
 * Provide data for decision making about expanding or discontinuing programs


 * Why Interventions fail:** (p. 154)
 * Solutions not linked specifically to strategies, challenges or org problems
 * Solutions focus on individuals and not operating units
 * Participants attend HPI programs for reasons other than personal or org need
 * PI programs not aligned with participants’ daily work environment
 * Immediate supervisors do not support or reinforce application of skills or knowledge

Builds on Kirkpatrick’s Four levels plus ROI level:
 * Evaluation: the Fifth Level (Phillips)** – p. 155, see table
 * Reaction and Planned Action
 * Learning
 * Application
 * Business Impact
 * ROI

Important to recognize that the HPI professional cannot expect to improve and measure performance of organization without first evaluating his or her own knowledge, capacity, and motives around performance measurement and calculation. Reasons to apply evaluation strategies to HPI work:
 * Professional and Personal Perspectives on Evaluation** (pp. 156-57)
 * Business understanding
 * Economic sense
 * Budget approval
 * Survival
 * Rising evaluation standards
 * Self-satisfaction
 * Professionalism


 * Evaluation Lingo** – pp. 158-159 (he might ask what those terms are, keep that page marked)


 * Planning your Evaluation** (pp. 159-163)
 * //Establish an evaluation framework//

> Should tie into HPI objectives identified during needs analysis process (pp. 160-161) > Elements of evaluation – purpose, levels, tools and timing need to be considered
 * //Develop data collection plan//

> Determine if program or intervention is accomplishing objectives > Find out if performance gap is closed or narrowed > Gauge extent of transfer to the job and identify barriers and enablers to transfer > Assess improvement areas in needs assessment and program or intervention > Calculate BCR or an HPI or HRD program > Provide data for decision making about expanding or discontinuing programs
 * //Establish Evaluation Purposes//


 * //Set Evaluation Levels or Targets//

> Surveys > Questionnaires > Interviews > Focus groups > Tests > Observations > Performance records > Evaluation Timing- depends on the type of evaluation done
 * //Develop Evaluation Instruments//

Case Study – pp. 163-170 – probably won’t be covered


 * Isolating Effects of Intervention** (pp. 170-171)
 * Control groups
 * Trend line analysis
 * Forecasting
 * Customer input
 * Participants’ and supervisors’ estimates of intervention’s effects
 * Expert estimation

Ask these questions:
 * Converting data to monetary value** (p. 171)
 * What is value of one additional unit of production or service?
 * What is value of a reduction of one unit of quality measurement (cycle time, waste, error, process failure, call escalation)?
 * What are direct cost savings?
 * What is the value of unit of time improvement?
 * Are cost records available?
 * Is there a credible, internal expert who can estimate the value?
 * Are supervisors capable of estimating the value?
 * Is senior management willing to estimate the value?

Include:
 * Reporting results** (p. 172)
 * Background statement
 * Description of evaluation strategy and data collected
 * Analysis of findings
 * Assumptions used in the analysis