Category: Feedback

This is reposted from the TAAOnline blog.

Defining a good research question is crucial to developing a successful research project, and it is no easy task. For some, defining a good question comes easily, but for many, especially doctoral candidates who may have never developed their own research project before, it is a great hurdle. And, as I suggested in the previous post, if the research purpose and question aren’t defined, then there’s no point in your looking at other stuff: if your student hasn’t defined the research purpose clearly, they’ll have trouble making progress.

A good definition of research question or purpose is not only crucial, it’s usually really easy for a reader to find in skimming through a paper. Most drafts have several sentences that say things like “the purpose of this research is…” and “the research question is…”. These sentences need to be clear, and they need to agree with each other (multiple conflicting statements of purpose can often be found in early drafts of research). One of the first things I look for when I read a draft is a clear statement of purpose. If a draft looks problematic—poorly edited or shows other obvious signs of weakness—I’ll almost immediately limit my review to skimming in search of any statements of purpose.

Often, there will be several statements of purpose in any given draft of research writing: the abstract states the purpose of the work, as does the beginning of the introduction. Many introductions include a statement of the problem and a statement of the significance, which are a pair of related statements about purpose. And then there are research questions, which are defined by how they respond to the intended purpose. If these several statements aren’t coherent, or don’t suggest a coherent and practical research project, feedback needs to start there, with the foundational stuff. Spend your time explaining the problems and offering suggestions on refining and focusing the research project into one single task that is enough for a research project. If you’re doubting the quality of the work, skim for such statements, and if they don’t all line up, focus your effort on getting a clear statement of purpose.

Students who have not done their own research before often try to stick many related but distinct questions into one study. It’s pretty easy to slide between distinct but related facets of one project, especially if one has limited experience setting up a research project.When faced with some problem or phenomenon, they shift between asking about causes, asking about impacts, and asking about responses. All three of these things are reasonable areas for study, of course, but they are also disparate areas of research. To study causes of poor educational outcomes, for example, different data and different analyses are required than to study the effects of poor educational outcomes or ways to improve poor educational outcomes.

For students who are aiming at professional careers—educators, clinicians, etc.,—such ideas naturally work together to address the practical concern for responding to some real-world phenomenon. An educator quite naturally might think about the causes, effects, and possible responses to poor educational outcomes as part of the same problem because from the perspective of a professional practice, they are part of the same problem. But from a researcher’s perspective, they are quite distinct, and many students benefit from having those distinctions made clear. Viewing the different dimensions as part of an integral whole is a great view for a professional practice, which can create a strong emotional attachment to addressing all the issues. But it’s a lousy practical approach to research, especially when the researcher running the project is a student who has never before completed an independent research project, so it’s good to force focus. To get over that kind of resistance, it might be effective to tell them to try to focus on one aspect as a first step to defining the larger array of issues. Once students start to see the complexity involved in researching the single dimension, they start to appreciate the need to leave out the other issues.

Checking for consistent and useful statements of purpose can be an effective tool to review documents quickly. Those that have conflicting statements of purpose can get feedback on narrowing their focus without having to wade through details of a project. Keep in mind that if they redefine the purpose, a good deal of the rest of the material will also need to be rewritten, which means that effort spent giving feedback on the body of a work with poorly defined purpose may be wasted.

Often, I see feedback that is grossly inefficient, both in terms of helping the student learn and in terms of saving the professor time. Most often this is feedback that focuses on grammar instead of examining and critiquing the focus and purpose of the work. An example of this that I recently saw was a student’s methods chapter draft for which the professor’s feedback had been that it needed better sentence structure and paragraph structure before higher-level feedback could be given. The professor had read the whole draft and commented in many places on a 25-page draft. It must have taken at least an hour, if not twice that, to do all that reading and to make all those comments. But in the first two pages there were obvious problems with the hypotheses. As soon as I saw that, it gave me a clue of what else to look for and I quickly found large chunks of the chapter really belonged in the literature review. Bad definition of the hypotheses (i.e., the research questions) led to bad choice of content. These problems were much more pressing than the grammatical problems, which were no more than one would expect in any early draft (it was far from perfect, but it was not hard to read). In this case, by focusing on the problems with the hypotheses (which are an expression of the researcher’s purpose), big problems with the study’s foundations were identified.”

Until a sense of purpose is clear and research questions well defined, there is little need to attend to other aspects of a project, so if you’re looking to save time, you benefit from starting with the statement(s) of purpose. Skipping the details of a project with a poorly defined purpose isn’t a failure to give students the attention they deserve, it’s giving attention to the top priorities. And, from the perspective of the student, although it might be difficult to hear that the project is not well defined, focusing attention on defining the research purpose and questions can help avoid many pitfalls that graduate students fall into (especially the trap of trying to read everything ever published). Save time by focusing on getting the purpose and question right before looking at other stuff.

This was originally posted on the TAAOnline blog.

Dear dissertation advisors, as a dissertation coach, I don’t actually want you to do your jobs better, because that might cut into my business. But if you’re interested in saving yourself effort and hassles in working with your thesis and dissertation candidates, I have a few pieces of advice for you.

As a dissertation coach, most people who contact me are struggling with their work, and often those struggles are exacerbated by poor feedback or support from professors. This biases my view of the general quality of research feedback, but the general patterns of what makes good vs. bad feedback are still useful to keep in mind. Good feedback helps the student effectively, reducing demands on the teacher; bad feedback will hinder progress, and may ultimately increase teacher workload. It’s good when students finish their projects, for both student and professor!

Not only does effort spent giving good feedback pay off in satisfied and successful students, the good feedback practices that I recommend are less time consuming than what I discourage. In this series of posts, I make some suggestions for giving feedback that I think would help both students and professors work more efficiently toward better outcomes.

The best feedback is limited in scope and suited to the context, focusing on the important issues, and avoiding tangential concerns. The best feedback probably won’t touch on all the issues that need to be addressed; it will address a focused set of issues that should be addressed as a next step. Contextual issues are maybe obvious: if a student is going to file her dissertation next week, you probably ought to give her different feedback than if she’s still struggling to formulate her research proposal. By scope, I want to focus on giving a good amount of work—focusing feedback on a limited set of issues to keep students from getting overwhelmed, and to encourage their more frequent contact with you (more frequent, while potentially less intense, as things move more smoothly and evenly).

It’s no great insight to say “focus on what’s important and skip the tangential,” but it’s a principle that can easily be taken for granted.  What is important, and what is tangential? Your mileage may vary, but one way I would approach this is to say that what is most important is the same knowledge and insight that makes you—their professor—special, in other words, that which is at the heart of your research expertise. What is tangential is stuff that other people could tell them.  Approaching this point from another angle, we can say that the theoretical and intellectual content are important, and many or most practicalities and formalities are secondary. And from yet another angle, we could ask what a dissertation or thesis is for? Is it to teach them to do research in your field (where your expertise is crucial) or to teach them to write (an area where your expertise is less rare)? Spend your feedback time on things that require your expertise: the subject matter of your field, research methods in your field, and their research in your field.

Although most dissertation advisors would agree, I think, that dissertations and theses are meant to teach students how to do research, and that therefore feedback should be focused on research issues, I see a lot of feedback that skips by fundamental research issues like poor question definition or mismatch of method to question. (One concrete suggestion that could save dissertation advisors a lot of time is to look at the research question first: is it defined well? Only once you can answer that question in the affirmative is there any reason to look at anything else.) Of course, there are a lot of surface issues that can distract, especially the obvious errors in presentation, like grammar and style.

The paradigm for poor feedback, in my experience, is to focus on grammar and other formal elements (like citation style) when basic conceptual and content issues are obviously flawed. Grammar and formal elements should not be a professor’s main concern. I’m not discounting the importance of being a good writer and of producing works that are grammatically sound. Writing is an invaluable skill, and inability to write cripples an academic career. But writing is also something that a graduate student can learn from many people, while research is something that far fewer can teach. Additionally, I imagine that you would rather teach your expertise (research) than a general skill (writing). Here’s an argument to convince you to leave aside grammar: according to standard academic ethics, a dissertation writer can hire an editor to fix grammar and citation style, but cannot have someone else design or carry out the research. Help your students to do the things that they cannot ethically have someone else do.

Teach your subject and put the burden for producing a good document on the students—whether they learn to fix their writing, or they work with an editor is mostly immaterial. Tell them “Learn to write, it will help your career,” and “this will be inadequate for a final draft,” and move on to research problems. If a draft is so messy that it cannot be read, say “This is so messy it cannot be read. Edit it and return it.” Don’t spend your time fixing stuff that they should fix themselves. It’s worthwhile effort to look past grammatical errors and to focus on their intentions and ideas. Pragmatically speaking, while it may be really easy and quick to identify a single grammatical or typographic error, and it may take more time to find a conceptual error, on the whole, it’s much quicker to look for conceptual errors than it is to fix a string of minor errors. If grammatical errors are common, there will be a lot of them, which can suck away your time while adding little of value to the student. But, to give good feedback, you only really need to find one significant conceptual error–if for example, the purpose is stated poorly, or there is a problem with the method, then feedback can focus on that issue, leaving other concerns for a later draft (and if there is a conceptual problem, it doesn’t really matter much whether the grammar is correct).

Spend your time on the research issues, where your expertise is rare and necessary. Is the research question defined? Is the scope of the research reasonable? Is there a match between method and question? Are there important voices in the discourse that absolutely need to be considered?

Save time by focusing on fundamentals: If the research question needs to be defined, then there’s little need to give feedback unrelated to the research question.