Section 1
Preview this deck
Effectiveness
Front
| 5 | 0 | ||
| 4 | 0 | ||
| 3 | 0 | ||
| 2 | 0 | ||
| 1 | 0 |
Active users
0
All-time users
0
Favorites
0
Last updated
7 years ago
Date created
Mar 14, 2020
Cards (74)
Section 1
(50 cards)
Effectiveness
Realistic potential under real life conditions
Project Evaluation Purpose
1. Demo the effectiveness 2. Improve the program services or methods 3. Document the cost effectiveness 4. Support replicability 5. Institutionalize it by involving key stakeholders 6. Provide data to the funder 7. Provide evidence to future funders
Organizational Plan
Personnel, Organization resources, capcity
Efficiency
Amount of output achieved from the inputs; Cost per unit of output
Intervention delivery
Requires the most effort; made easier through planning and a good process theory; must follow protocols; require pilot testing
Minimize Over- and Underinclusion
1. Specify how recipients get into the program 2. Good marketing plan
Evaluation Questions
1. Gives structure to the evaluation 2. Serves as a basis for essential discussions about who is interested in the answers 3. Define the issues the eval will investigate 4. Developed by the evaluator or stakeholders 5. Stated in the terms so that they can be answered by the evaluator
Human Resources considerations
1. Licensure/certification 2. Training 3. Cultural diversity/sensitivity 4. Volunteers
Service Utilization Plan Inputs
1. Social marketing 2. Eligibility screening 3. Intervention delivery
Target population
The entire population in need of the program
Data Collection Methods*
1. Activity Log 2. Checklist 3. Organizational Records 4. Client records 5. Observation 6. Questionnaire 7. Interview 8. Case Study
Implementation documentation
Tallying the activities and processes of the program implementation
Units of Service
1. Commonly used in programs at eh direct or enabling services level 2. Predetermined Unit 3. Stated in process objective
Challenges in Measuring Satisfaction
1. Ceiling effect 2. Measure can be culturally sensitive 3. Discrepancies btw questionnaire and interview responses 4. Must capture expectations and experiences 5. Scope may be difficult to define
Efficacy
Maximum potential effect under ideal conditions
Building an Evaluation Plan*
1. Identify Relevant Stakeholders 2. Obtain Background Info 3. Clarify the Purpose for the Eval 4. Specify the eval questions 5. Focus the eval 6. Develop an eval plan 7. Develop and management plan 8. Develop a communication plan
Informational Resources
1. Professional networks 2. Street Smarts 3. Professional knowledge/experience 4. Used to maximize staff retention
Process Theory
Resources and action
Program Theory
1. Process Theory 2. Effect Theory
Threats to Fidelity
1. No program provided 2. Different program provided 3. Inconsistent program provided
Theory-in-use
What staff do affects the participants; on-the-job training; watch staff while providing the program; is the actual cause of the program outcomes
Four P's + 2 P's
1. Product 2. Price 3. Place 4. Promotion + 1. Partnership 2. Policy
Espoused Theory
What staff says about how program affects the participants; program manuals and minds of staff; identified by staff; becomes what staff and stakeholders believe and expect of program
Purpose of Process Evaluation
1. Data about program delivery provides context for the effect evaluation 2. Operational information for program replication 3. Satisfaction is a component
Physical Resources
1. Material resources 2. Facilities 3. Supplies 4. Equipment
Service Utilization Plan
How to reach the target audience and deliver the intervention
Pyramid levels
1. Individual/direct services 2. Aggregate/enabling services 3. Population based 4. Infrastructure
Infrastructure level (Process Monitoring and Eval)
Units of service may be # employees involved, # procedures updated
Organizational Plan Outputs
1. Timeline 2. Operations manual 3. Organizational chart 4. Information system 5. Budget
Managerial Resources
1. Organizational abilities 2. Communication skills 3. Team-building skills 4. Leadership qualities 5. Coping skills 6. Technical skills
Population based services level (Service Utilization Plan)
1. Skills match between providers and the intervention 2. Social marketing to use mass media
Implementation evaluation
Retrospective determination of whether the program was implemented as designed; Process evaluation
Service Utilization Plan Outputs
1. Units of service provided 2. Quantity of service completions 3. Materials developed for the health of the program 4. Work flow
Enabling services level (Service Utilization Plan)
1. Providers need social services background 2. Volunteers used 3. Social Marketing target aggregate
Social Marketing
The application of commercial marketing technologies designed to influence voluntary behavior of target audiences in order to improve their personal welfare. (health marketing)
Sensitivity screening tool
Identify true cases; reduces underinclusion
Partial-coverage programs
Direct care or enabling services levels of the pyramid
Direct services level (Process Monitoring and Eval)
Units of service
Implementation assessment
Timely data collection to make corrections to the process theory during implementation; Program or process monitoring
Direct services level (Service Utilization Plan)
1. Tailor human resources to the intervention 2. Social marketing target individuals
Full Coverage Program
Population services level of the pyramid
Intervention Fidelity
The degree to which the program was offered as designed
Infrastructure level (Service Utilization Plan)
1. Strong org and services utilization plans 2. Adequate infrastructure needed for program success
Organizational Plan Inputs
1. Human resources 2. Physical resources 3. Time 4. Money 5. Managerial resources 6. Informational resources 7. Transportation
Effect Theory
Interventions and outcomes; explain how program affect participants; found in program description and manuals; identified in program materials/literature; guides program and evaluation.
Specificity screening tool
Identify true non-cases; reduces overinclusion
Target audience
The segment of the population for whom the program is specifically intended
Population based services level (Process Monitoring and Eval)
Units of service at community level
Dose Monitoring
1. Dosage elements: freq, duration, strength, route 2. Service completion 3. Level of participation
Enabling services level (Process Monitoring and Eval)
Units of service modified to be program specific using different data sources
Section 2
(24 cards)