Policy-Press-Logo

Research and evaluation for busy students and practitioners

Information for lecturers

Welcome


Please use the menu on the left to navigate your way through the lecturer resources for Research and evaluation for busy students and practitioners.

There is help for you if you are teaching research or evaluation methods, including discussion topics and essay questions for each chapter, and some more general essay questions. There are some scenarios for use in teaching research ethics. The book may also be useful in helping you with your own research or evaluation, and with your supervision of Masters' students’ dissertations or theses. And lastly there are links to online resources which you may find useful.

Discussion and essay topics


Here is a list of discussion topics related to each chapter of the book. These are designed to be useful for both group seminars and one-to-one tutorials. This section also includes some suggested essay questions.


CHAPTER 1

Discussion topics
How clear (or unclear) is your identity of ‘researcher’ or 'evaluator'?
How does your identity of ‘researcher’ or 'evaluator' interact with your other identities?
How do you think your current research could benefit other areas of your life?
To what extent is your research or evaluation ‘insider’ or ‘outsider’ research, and why?
Which parts of the research process appeal to you, and why?
Which parts of the research process do you find daunting, and why? What steps could you take to overcome this?

Essay questions
1. What are the pros and cons of ‘insider’ versus ‘outsider’ research?
2. What is the role of identity in research?
3. What is the purpose of research in the world?



CHAPTER 2

Discussion topics
What do you see as the pros and cons of quantitative and qualitative methods?
Under what circumstances do you think service users should be involved in research or evaluation?
Can there ever be research which is not in some way value-based?
What ethical dilemmas might you encounter during your research or evaluation?
How can you manage these dilemmas?
What do you find most stressful about your research?
How can you reduce that level of stress?

Essay questions
1. Can the gap between qualitative and quantitative research be bridged?
2. Discuss the extent to which research can be value-free.
3. Can research ever be truly ethical?



CHAPTER 3

Discussion topics
What type of methodology appeals to you most? Why? What are the implications of this for your research or evaluation work?
Should research support social justice, or should it be neutral? Why?
What are the pros and cons of using arts-based research methods?
What are the pros and cons of digitally mediated research?
How do you envisage linking theory and practice through your research or evaluation?

Essay questions
1. What helps, and what hinders, the process of linking theory and practice through research?
2. What is the role of methodology in research or evaluation?
3. In what ways is academic theory relevant to practical research or evaluation?



CHAPTER 4

Discussion topics
What are the most effective types of research question? Why?
What is the best way to decide how to collect data?
How are you going to sample your population?
Why do you think your plans will produce good evidence of an answer to your research question(s)?
Why is your research or evaluation worth doing?
Who might benefit from your conclusions, and how?

Essay questions
1. What is good practice in designing a piece of research or evaluation?
2. How do decisions about sampling affect the results of research or evaluation?
3. Why is the design of research or evaluation rarely if ever perfect?



CHAPTER 5

Discussion topics
What is the purpose and function of project plans?
How can you develop organisational skills?
Is time management an art or a science? Why?
What is the relationship between your research and your other work, or your plans for work? Is there more scope for one to enhance the other than you are currently using?
What is the relationship between self-discipline and negotiating skills?
How can you keep yourself safe and healthy throughout your research or evaluation?

Essay questions
1. Should research aim to be socially responsible, and why?
2. How can research and evaluation support practice? Give examples from at least two public services.
3. What are the key elements to consider when planning research?



CHAPTER 6

Discussion topics
What are the differences between a document review and a literature review?
Why is record-keeping important for background research?
How can you read more critically?
How can you read more strategically?
Why is it important to make notes as you read? How can you most usefully do this?
How do you know when you’ve read enough documents or literature?

Essay questions
1. Compare and contrast any two of the following: document review, literature review, systematic analysis, meta-analysis.
2. Why is context important for research?
3. Discuss the assertion that only formally published work should be included in a document or literature review (choose one).



CHAPTER 7

Discussion topics
What are the pros and cons of using secondary data?
How can you assess the integrity of secondary data?
What are the pros and cons of using 'big data'?
What problems might you face in trying to complete a research project using only secondary data?
What factors can make it difficult to compare data from different datasets?
Should all data be openly accessible? Why?

Essay questions
1. Discuss the assertion that secondary data is second-hand data, while primary data is the real thing.
2. Should all data be openly accessible, and why?
3. What are the ethical issues to consider when using secondary data?



CHAPTER 8

Discussion topics
What are the pros and cons of using primary data?
How do you know how much data to collect?
Is it better to collect a little data from a lot of people, or a lot of data from a few people? Why?
What are the relative merits of interviews, focus groups, and case studies?
Under what kinds of circumstances might it be particularly useful to collect visual data?
What are the ethical aspects of collecting primary data online?

Essay questions
1. Discuss the assertion that the only role for research participants is to provide data for researchers.
2. Discuss the relative merits of text-based and visual methods of data collection.
3. To what extent can you measure people’s attitudes using survey methodology?



CHAPTER 9

Discussion topics
What is data preparation and why is it important?
What is the point of coding data?
What ethical dilemmas might you encounter during data analysis?
Do the results of statistical calculations tell the truth?
What are the relative merits of content analysis, thematic analysis, discourse analysis and narrative analysis?
How might Geographic Information Systems (GIS) help your data analysis?

Essay questions
1. Are numbers neutral?
2. Compare the strengths and weaknesses of any two methods of analysing qualitative data.
3. What steps can researchers take to ensure they are acting ethically when analysing data?



CHAPTER 10

Discussion topics
What methods might help to overcome anxiety about writing?
Why is it almost impossible to write a good essay in one draft?
Who are you writing for? What do they want to read?
What is wrong with plagiarism?
Why is citation important?
How can you make your writing the best it can be?

Essay questions
1. Discuss the importance of accessibility in writing up research.
2. Discuss the assertion that writing is a research method.
3. Is there a role for creative writing in research?



CHAPTER 11

Discussion topics
What are the ethical aspects of disseminating research or evaluation?
When should you present your findings, and to whom?
How can you work out which method(s) of presentation and dissemination will be most effective for a particular piece of research?
What creative methods could you use to disseminate your research?
Why is dissemination a political act?
What is the role of the Internet in disseminating research and evaluation?

Essay questions
1. Paying particular attention to research ethics, discuss the political implications of disseminating research or evaluation.
2. Should research participants be involved in presentation and dissemination? Give reasons for your answer.
3. How might you draw on the performing arts and visual arts in disseminating research or evaluation?


And here are ten more general essay questions:

1. How does social research interact with social class?
2. Is it possible to conduct evaluation research using entirely quantitative methods?
3. Why is research a political act?
4. What effect does research or evaluation have on organisational and government policy?
5. Evaluate the claim that the social survey is the most useful method for carrying out research in society.
6. Critically assess and compare two pieces of openly accessible research or evaluation.
7. Why is it that research and evaluation methods can be controversial?
8. What are the roles of ‘facts’ and ‘truth’ in social research?
9. Where does social research sit on the spectrum between science and art?
10. How is knowledge generated by research or evaluation different from other forms of knowledge?

Scenarios for teaching research ethics


A good way to teach research ethics is to ask your students to discuss scenarios and try to work out the most ethical course of action. Here are some scenarios you can use.

SCENARIO A

You are doing an evaluation of a horticultural therapy project for people with mental health problems. You have received formal ethical approval for your study design. You are collecting data through observation and interviews, and you plan to maintain the anonymity and confidentiality of your participants. In the middle of your data collection, your participants tell you they don't want to be anonymous in your research. They think their participation is important and they want their contributions to be acknowledged. The staff at the project support them in this, telling you they think it would be good for the participants' self-esteem because they could show other people they took part in worthwhile research. What do you do?


SCENARIO B

You are researching people's experiences of medical treatment in a local hospital accident and emergency department for your Masters' degree dissertation or thesis. There is a triage system in place which is intended to prioritise patients for treatment solely on the basis of the severity of their symptoms. The hospital has a well-written equal opportunities policy which you have quoted in your literature review. However, you are surprised when your data analysis reveals that in this department white patients are treated, on average, 10% faster than patients of colour. Also, white men are treated 18% faster than women of colour. What are the ethical implications of these findings? How would you address the ethical issues raised?


SCENARIO C

You and a colleague are working on an evaluation of a community centre. The evaluation has been commissioned by the local council. The community centre is not well used and is losing money. The council has to make up the shortfall and, while they haven't said so, you suspect they want evidence to support the centre's closure. You can see that the centre has considerable potential as a local resource. However, the evidence you have gathered from questionnaires and interviews shows that most local people are not interested in using the centre. During data analysis, you realise that your colleague is massaging the findings to make them look more positive. You question this, and your colleague breaks down in tears, saying that a family member has a job at the centre and they can't manage without that income. What do you do?


SCENARIO D

You have been doing research in a school for children aged 5-11, studying how best to help pupils cope with the transition from one class teacher to another. You have collected data from children, teachers, and family members. Your findings show that a number of changes are necessary, particularly for children with extra support needs, who have the most difficulty coping with these transitions. You have also found that one teacher is deeply unpopular with children and families. The head teacher regularly expresses confidence in your work, but she doesn't know much about research, and is very busy and difficult to contact. She has asked you to present your findings to her and the school community at a special assembly for all the children, teachers, and governors. Family members have also been invited. What ethical dilemmas does this raise? How would you address them?


SCENARIO E

The Government pays you to do an evaluation of a pilot project to help people who finish custodial sentences. At the initial meeting with the project manager, you establish that the evaluation report will be sent to all prisons in the country, and disseminated online for prisons in other countries, to help them if they want to set up a similar project. The evidence you assemble shows that the project reduces reoffending and is cost-effective. However, your research also shows that there is still room for improvement. You write twelve carefully worded recommendations which are firmly rooted in the data. Nine are unequivocally positive and three offer specific and implementable suggestions for improvement, designed to reduce reoffending further and increase cost-effectiveness. The project manager is very upset, taking the suggestions for improvement as criticism of his work. He says if you do not remove those three recommendations, he will not disseminate the report. What do you do?


SCENARIO F

You live in the housing estate where you were born, among a close-knit community. Research is unpopular there: people don't see the point of answering the questions of strangers when nothing ever seems to change or improve for the community as a result. However, in the course of your work, you have had to do some research, so you have developed an understanding of what the point might be. Two new researchers come to your community and tell people they want to do participatory research with community members to investigate ways in which research could be made more user-friendly for communities like yours. A meeting is set up for community members to discuss this with the researchers and make a decision about whether or not to go ahead. Because you know a bit about research, you are asked to join the meeting. What ethical issues does this raise? How would you address them?

Why this book might be useful for you, too


Shall we admit that some lecturers, even some professors, can be unsure about aspects of the research process, or struggle with organisation and time management?

Part of the background research for this book included interviews with 20 practitioners, eight of whom also worked in academia. They were candid about their difficulties in advising students about research methods, and in managing their own research and juggling it with all their other professional and personal commitments. Several told me, off the record, that they thought my book would be helpful for academics as well as students, and they wished they could read it already.

The good news is, you can!

The book may be particularly helpful for people supervising Masters' students doing dissertations or theses. When you are supervising the research training of any student, you need to be knowledgeable about research methods and competent in the theory, if not the practice, of time management and organisational skills. This book can help in several ways:

It gives a good grounding in research methods, so is a useful resource for anyone who may be shaky in one or more aspects of the research process.

It offers advice on managing the process through planning, organisation and time management, so can be helpful for students who have difficulty with one or more of these skills.

It is full of practical tips, making it immediately applicable.

It contains numerous quotes from people who have been through the process, which can help students to find and keep a sense of proportion about the problems they are facing.

Resources


There are lots of resources for teaching research methods on the UK's National Centre for Research Methods website.

The National Education Association of the US has a collection of resources for teaching education research, many of which are applicable to other types of research and evaluation.

The US Institute of Museum and Library Services has a collection of resources on evaluation, many of which are applicable beyond museums and libraries.  

Similarly, the US Centers for Disease Control and Prevention has a collection of resources on evaluation, many of which are applicable beyond the health industry. 

The Higher Education pages of the Guardian website are a great resource.  These pages hold a vast quantity of useful information for researchers, and more is produced weekly. It's often worth reading the comments below articles as well as the articles themselves.

Blogs can be useful, too. Some blogs are institution-based, such as the excellent blog from the London School of Economics about the impact of social sciences.

Others are group blogs, such as the very useful Research Whisperer and Pat Thomson's blog is invaluable for anyone doing academic writing.

The Research Ethics Guidebook is an excellent resource on research ethics.  

If there are any essential resources we’ve left out, please let the publisher or the author know.