ETHICAL ISSUES : Tips for conducting program evaluation

Text-only Preview

Tips for conducting program evaluation
Before beginning your evaluation project, it is important to consider and address
ethical issues and professional standards.

Strategies to protect the rights and dignity of
Help or benefit to others – promoting
evaluation participants should be incorporated
others’ interests, by helping individuals,
into the way that you design and carry out
organizations, or society as a whole.
your project. It is also important to consider
Do no harm – bringing no harm, such as
safeguards that may be needed when your
physical injury and psychological harm
participants are children or other vulnerable
(such as damage to reputation, self-
populations, including some victims of crime.
esteem, or emotional well-being).

Act fairly – treating people fairly and
Many professional organizations provide
without regard to race, gender, socio-
ethical guidelines (e.g., the American
economic status, and other characteristics.
Psychological Association, the National
Respect others – respecting individuals’
Association of Social Workers, and the
rights to act freely and to make their
American Counseling Association). While
own choices, while protecting the rights
their details vary, most guidelines address
of those who may be unable to fully
four over-arching issues:
protect themselves.
Key ethical issues related to program evaluation
Consideration of risks and benefits
Disruptions to participants’ life (e.g.,
Many benefits can result from evaluations.
sacrificing time and energy to participate).
In some cases, there may be direct benefits
Emotional consequences (e.g., answering
to participants, such as receiving a gift
painful questions about their victimization
certificate or other incentive in exchange
or traumatizing events).
for being interviewed. Other benefits emerge
Safety concerns (e.g., allowing an abuser
as a result of changes made at the program
to learn about their involvement in
or agency level – for example, the evaluation
services, exposing them to potential
may guide strategies for improving a
future victimization).
program’s impact, leading to more positive
Social harm (e.g., violating confidentiality,
outcomes for current or future participants.
so that others learn about their victimization

However, there may also be risks. You

should carefully consider any harm that may
In designing an evaluation, work to maximize
result from an evaluation, and take steps to
benefits and minimize risks. While you may
reduce it. With evaluations of crime victim
not eliminate risk, you should reduce it to
services, potential risks include:
an acceptable level relative to the potential
Risks and benefits, page 2

Page 2
Risks and benefits, continued:
benefits. Other sections of this tip sheet
Describe the benefits of participation
address safety and confidentiality. In
and any foreseeable risks, including
addition, consider these suggestions:
possible discomfort.
Keep evaluation procedures as brief
Share this information using under-
and convenient as possible to minimize
standable language – avoid jargon and
disruptions in subjects’ lives.
translate if needed.
Do not ask emotionally troubling
Answer any questions they have about
questions, unless they are necessary
the evaluation.
to help you improve services.

Provide incentives, such as food,
Participants may not need to sign a consent
money, or gift certificates.
form if they are capable adults, have not

been coerced, and will not be put at risk.
The time and money spent on evaluation
For example, if you ask clients to fill out an
are maximized when the results have value.
anonymous survey about their satisfaction
Target your evaluation to key questions,
with a shelter, the fact that they complete
carefully review findings, and use your
the survey can be construed as providing
results. Upcoming tip sheets will provide
consent. Signed consent forms may be
strategies for using evaluation to improve
necessary, however, especially if you plan to:
services, demonstrate your program’s value,
Include children or others who cannot
and guide policy and advocacy efforts.
provide their own consent (in which

case you need consent from a legally
Informed consent
authorized person, such as a guardian).
Everyone who participates in the evaluation
Collect very sensitive information.
should do so willingly. In general, people
Use the results for purposes other than
participating in any research project, including
program improvement, such as publication,
an evaluation, have the right to:
training, or participation in a larger
Choose whether or not to participate
research project.
without penalties (e.g., participation
Gather information about participants
should not be a requirement for
from third parties, such as program staff,
receiving services).
case workers, or family members.
Withdraw from the project at any time,
Require significant time or effort, such
even if they previously gave consent.
as time-consuming interviews.
Refuse to complete any part of the project.

The word “informed” is important – people
It is not always possible to conduct
have the right to understand all implications
evaluations without identifying information,
of their decision whether or not to participate.
such as names. However, all evaluation
To ensure that potential participants can
information should be kept confidential
make an informed decision:
and not shared with others. To ensure
Provide them with information about
the evaluation (what you will ask them
Collect data in a private location where
to do, how you will you use the results,
surveys cannot be seen and interviews
and how long it will take).
cannot be overheard.

Page 3
Do not discuss information about
for an interview, consider whether it is
individual participants with other
appropriate to leave a message. In the
people, including other agency staff.
course of collecting information, you may
Keep completed surveys or interviews
learn that a client is in an abusive situation.
in a secure location where they cannot
While your ability to intervene may depend
be seen by other people.
upon the level of imminent risk, it might be
Securely dispose of completed material
appropriate for evaluation staff to refer
when it is no longer needed.
participants for assistance if desired.

You may face situations in which you feel
Other considerations
that it is important to disclose confidential
Health Insurance Portability and
information. This may be due to a legal
Accountability Act (HIPAA) – if you are a
requirement (e.g. a mandated reporter of
health care provider collecting information
child abuse). In other cases, you may learn
about physical or mental health, you may
through the evaluation that someone plans
be required to comply with HIPAA, a 1996
to harm themselves or others, or is at risk
federal law designed to protect the privacy
of harm from others. To the extent possible,
and security of health information. If you
consider in advance the types of disclosures
are unsure whether HIPAA applies to your
that may be needed and develop a plan
evaluation, research this issue in advance to
to handle these situations. When you
ensure that you comply with the guidelines.
obtain consent, provide information about

circumstances in which you might share
Institutional Review Boards (IRBs) – An
confidential information.
IRB is a federally-recognized committee

authorized to review research projects
Ensuring safety
and ensure that they comply with ethical
In conducting an evaluation, you may have
standards. Many colleges, government
concerns for participants’ safety, especially
offices, hospitals, and research agencies
when working with victims of crime. Be
have established IRBs. Usually, IRB approval
thoughtful about participants’ needs and
is not required for evaluations. In some
take care to protect them. For example, if
cases, it may be needed, especially with
participants are not at home when you call
some federal funding.
Key ethical issues related to evaluator’s roles
There are also ethnical guidelines you
understandable results, and include
need to follow as an evaluator. The Joint
meaningful recommendations.
Committee on Standards for Education
Feasibility – Evaluations should be realistic
Evaluation issued a series of standards in
and practical, so that they can be completed
1994 that have been widely adopted. Under
in a time- and cost-efficient manner.
this framework, evaluators’ work should
Propriety – Evaluations should be legal
reflect the following four standards:
and ethical.
Utility – Evaluations should address
Accuracy – Information should be
important questions, provide clear and
collected, analyzed, reported, and
interpreted accurately and impartially.

Page 4
Addressing ethical challenges
In some cases, you might face situations in
5. Which overarching ethical issues apply
which the ethical direction is not clear. Ask
(e.g., helping others, doing no harm,
yourself the following questions when
acting fairly and being respectful)?
faced with an ethical challenge:
Does a clear solution to the challenge

emerge when considering these
1. What does my intuition tell me? Am I
feeling stress or self-doubt about my
6. What are my personal values and beliefs?
chosen direction?
What guidance do they provide?
2. Is there an established way that my

colleagues would act in the same
If you are unable to decide the best course
of action, consult with others, including
3. Does my profession have a set of
colleagues, supervisors, your board of
ethical guidelines? If so, do they
directors, evaluators or researchers, or legal
suggest a course of action?
4. Are there existing laws that apply? If

so, what requirements do I need to follow?
Quick links to more information
For more information about HIPAA, go to

For more information about IRBs and research with human subjects, go to

For links to other sites providing guidance on conducting ethical program evaluations, go to

In future tip sheets
Analyzing and understanding data (1/08)
Communicating evaluation results (4/08)
Using evaluation for program improvement (7/08)

Find previous tip sheets on the web: or
October 2007
For more information or additional copies, contact:

Cecilia Miller
Author: Cheryl Holm-Hansen
Minnesota Office of Justice Programs
Wilder Research
[email protected]