Learning and Life: What is Meant by Rigor of PhD Research

The rigor with which your hypothesis are tested and the methodology that is pursued differentiates a PhD from an MS/MPhil [1]. A PhD is held to higher standards of meticulousness and scholarship which becomes evident from the rigor of the analysis of conditions, constraints, variables that influence the experiment, and may have an impact observations, and use of established theories for justifying their inclusion or exclusion.

Ruling out all the Alternatives

Rigor refers to the identification of all possible steps or stages of your research, and doing a thorough job at each stage. By thorough it means that all possible alternatives of performing that step have been identified. These alternatives are not only identified but also compared as to their suitability to solving the problem at hand. Then systematically all excepts the chosen one are ruled out. All alternatives may not be all equally important. If there are a few alternatives then each can be analysed, compared and then ruled out except the one that is found to be most suitable. However, if the number of alternatives at a given stage is large then they need to be categorized and then some categories are ruled out and two or three more relevant categories can then be explored and evaluated in detail. You can see the rigor of a particular work if all questions coming in the mind of the readers are found to be dealt with systematically. There needs to be reason and justification for each decision. Not only that but there should be reason for ruling out all the alternatives.

See also: Why PhD is Difficult to Complete and Why there are so many ABDs (All But Dissertation) and PhD dropouts.

Rigor relates to the consideration of all relevant methodologies, all relevant alternatives, and all relevant approaches in the analysis to arrive at the selected methodology and use of convincing reasons for ruling out the alternatives, approaches and methodologies that have not been selected. Furthermore, rigor is indicated by the meticulousness and care with which the data was handled, organized, and analysed to come up with results. You are supposed to defend every decision that you took. Not only you need to defend why you took that decision, but also what were the other alternatives that were there but you did not take, and why you did not take. This is known as rigor. Decisions include but are not limited to:

Why you have selected certain theories and not others. Why you ruled out others. How the theories make an ontological or epistemological contribution.
Why you have selected a certain definition for a term and not others. Why you ruled out others. You need to do this for each LOADED term in your problem statement/thesis statement. Using terms loosely or interchangeably indicate that you are not sure about what you are saying. You can not understand the depth of the above statements if you have not read my posts: (i) What is a Problem Statement and its role in MS-PhD Research, (ii) What is a Thesis Statement and its Role in PhD-MS Research. Your ease of going through your PhD is determined by how well you have internalized the above two posts.
Why you have selected certain relationships and not others. Why you ruled out others. Typically some relationships are considered, included in the model and others are ignored. Simply ignoring is not enough. The relationships that were considered but left out need to have a proper justification.
Why you have selected certain variables/factors and not others. Why you ruled out others. Typically I have seen researchers just cherry picking certain variables. Adding another variable or variables or other relationships for the sake of showing originality. Originality does not come by simply adding a few variables or relationships to a conceptual framework or model. Originality comes through a new way of looking at the relationships. This new way of looking should have a justification in reality or results and should also make a theoretical contribution.
How well your model represents the reality. Have you identified all the relevant variables. Have you controlled for any extraneous effects.
Why you have selected a certain methodology and not others. Why you ruled out others. A methodology that you use should be contrasted to the state of art in that methodology. At times your contribution could also be methodological. Just using an existing methodology is not enough. How that methodology supports a better understanding or resolution of the problem is important.
Why you have selected a certain population and not others. Why you ruled out others. Here I say people saying that we doing this work for Pakistan or some specific country. This needs to be defended on the ontological basis. What is the necessity of looking at the problem from the perspective of a particular country, community or a population. The selection of your population should be determined by the problem and not convenience.
Why you have selected a certain sampling strategy and not others. Why you ruled out others.
Why you have selected a certain method and not others. Why you ruled out others.
Why you have applied certain tests and not others. Why you ruled out others.
Why you have used a particular software and not others.
How do you validate your claims? To what extent? Are you just providing usage examples that make your claims plausible or did you perform a thorough experiment where you were able to control for most variables and assess how the issue of interest really behaved?
etc. etc. etc.

Every Strong Statement is Defended

Rigor is indicated when every strong statement in the dissertation is corroborated by some existing research or is the outcome of this PhD research. A good thesis does not mince words or avoid taking stands. A good research also does not involve itself in irrelevant debates. Relevance is held together by the thesis statement and problem statement. A good dissertation avoids using words such as "may", "typically", "usually", or other such non-committal words.

Rigor of research methodology is a subjective assessment of whether the students is eligible for a PhD or not. A supervisor typically comes to the realization that the student has now sufficiently completed the work and has become eligible for PhD when (i) he sees that the student is now critically analyzing his work the way the supervisor analyzes the work, and (ii) when he sees that the student can critique the work the way the community of researchers working in the selected area of research would analyze the work. The latter gets determined by the publication of paper in the journal of the chosen area of research.