Select your language

Suggested languages for you:
Log In Start studying!
StudySmarter - The all-in-one study app.
4.8 • +11k Ratings
More than 3 Million Downloads
Free
|
|

All-in-one learning app

  • Flashcards
  • NotesNotes
  • ExplanationsExplanations
  • Study Planner
  • Textbook solutions
Start studying

Quality Criteria

Save Save
Print Print
Edit Edit
Sign up to use all features for free. Sign up now
Quality Criteria

Research quality criteria are requirements for research that psychologists have recommended and agreed upon.

Data and reports must meet these requirements to be considered quality scientific research. There are several types of quality criteria for qualitative and quantitative research.

Research quality criteria for qualitative research

The purpose of using research quality criteria for qualitative research is to determine if it is credible and trustworthy based on the following criteria:

  • Credibility whether the research findings contain credible information based on data collected from participants and whether the interpretations reflect that data. The results should accurately reflect the experiences of the participants. It is similar to internal validity, a requirement of quantitative research.

  • Transferability whether the results are transferable to other situations, environments, and participants.

  • Dependability whether the results are consistent and repeatable.

  • Confirmability whether other researchers can confirm the results.

Quality Criteria Research quality StudySmarterResearch quality, Pixabay

Summary of research quality criteria for qualitative research

The following table summarises the methods researchers can use to meet the requirements of the research quality criterion for qualitative research:

Qualitative quality research criterion

How research can meet this criterion

Credibility

  • Triangulation use of multiple data collection methods.

  • Identifying key aspects of the research question and focusing on them.

  • Allowing sufficient time, e.g., to build relationships with participants, analyse data, become familiar with the setting and context to avoid misinformation/misinterpretation that could affect results.

Transferability

  • Taking detailed notes on what was observed during the research.

  • When describing participant behaviours and experiences, also make notes on context so that other researchers can understand and interpret the researcher’s perspective.

Dependability

  • Audit trail: the researcher could write down how they collected, analysed, and interpreted the data, and other researchers could then follow this to see if they come to the same conclusions.

Confirmability

  • The same method used for reliability is used to ensure that the research is confirmable.

Research quality criteria for quantitative research

The purpose of using quality criteria for quantitative research is to determine if it is credible and trustworthy. The following criteria should apply to the research:

  • Internal validity how much of the observed effects are due to the independent variable, not other factors.

  • External validity whether the sample results can be generalised to the broader population.

  • Reliability whether similar results would be obtained if the study were repeated.

  • Objectivity whether potential biases (researchers and experimental) that could influence the results are excluded.

Assessing reliability

Researchers can assess the reliability of their study using test-retest reliability and inter-observer reliability.

Test-retest reliability tests whether the results of a study are consistent over time.

Test-retest reliability is when the same test/experiment is administered to the same participants at two different time points. If the correlation between the two results is high, this is a good reliability indicator.

Let us say you administer a personality test to a group of participants. A month later, you give the same group of participants the personality test again. If the personality scores this time are drastically different from those of the first test, the test does not have good test-retest reliability.

Researchers can improve the test-retest reliability by:

  • Redesigning the test, or perhaps improving or removing some questions.

  • Controlling external factors as much as possible, e.g., by ensuring that participants take the test under the same conditions (e.g., in the same room).

Inter-observer reliability refers to the extent to which different researchers (observers) agree and give the same ratings for a phenomenon.

In Bandura’s Bobo doll study, researchers measured the inter-observer reliability by determining whether observers agreed with how many acts of aggression the children exhibited.

In a study, if one observer gives many ratings but another observer gives few, then inter-observer reliability is low.

Researchers can improve the inter-observer reliability by:

  • Giving all observers the same training in observation techniques.

  • Clearly defining the variables and how they will measure.

Assessing validity

Validity can be assessed in several ways: face validity, concurrent validity, ecological validity, and temporal validity.

Face validity means assessing whether a test measures what it claims at first glance.

For a test measuring depression, you would expect it to ask questions about low mood and motivation. If you review a test to measure depression and it contains these types of questions, then it appears to have good face validity.

Face validity is the weakest type of validity criterion because it is based on people’s assumptions about their behaviour.

Concurrent validity means that you compare the results of one test to the results of an existing test to see if they give similar results. Participants must take the tests at approximately the same time to reflect their current state.

A well-known measurement of aggression is Buss-Perry’s aggression questionnaire (1992). Suppose you have developed a new questionnaire on aggression and test its concurrent validity. You could ask participants to complete both questionnaires in one sitting and then compare your participants’ results with your questionnaire with the results they obtained with the Buss-Perry aggression questionnaire. If the results are similar, there is concordant validity.

Ecological validity is the extent to which the study results can be applied to real-life situations.

A study may work well in a laboratory, but the results are not as good when transferred to the outside world. We can improve ecological validity by conducting studies in natural settings.

Temporal validity measures whether the study results are generalisable or applicable over time.

Asch’s (1951) study on conformity does not have good temporal validity because it has been criticised for reflecting the American conformist culture of the 1950s.

Summary of research quality criteria for quantitative research

The following table summarises the methods that researchers can use to meet the requirements of the quality criteria for quantitative research:

Quantitative quality research criterion

How research can meet this criterion

Internal validity

  • Standardising variables studied (define the variables and how the study measured them).

  • Providing sufficient details about the context of the research and the interventions used (can you identify extraneous/ confounding variables).

  • Using control groups.

  • Using standardised instructions for all participants, so they all receive the same information.

  • Counterbalancing in repeated measures designs, participants do not all go through the study conditions in the same order to account for practice or order effects.

  • Controlling for demand characteristics in participants and experimenter effects in researchers. Perhaps by not informing participants of the true goals of the study and using a research assistant who also does not know.

External validity

  • Use random sampling methods.

  • Replicating the study in other settings to assess ecological validity.

  • Examining scales used in the study with other similar scales to determine if they measure the same thing. This is used to assess construct validity. Similar results mean that the scale has high construct validity (it measures what it is supposed to measure).

Reliability

  • Reviewing internal reliability/consistency of scales. For example, all questions in a scale that measures depression should also measure all depression scores.

  • Assessing the generalisability theory determining the consistency of instruments used in research or whether the results obtained by participants are due to specific conditions.

Objectivity

  • Using blind methods when collecting and coding data, i.e., a trained professional who is not part of the research team should do so to prevent bias from influencing the data.

  • Data should follow the empirical method (stages research should follow to produce scientific, reliable, and valid research)

  • Using only data that were generated as part of the research.

  • Retaining original data used for research for accountability purposes.

Empirical research

Empirical research is research based on direct observations rather than subjective opinions, data, and analysis techniques.

Empirical data should allow valid, reliable, and objective conclusions to be drawn. This research method can provide qualitative or quantitative data. There is an ongoing debate among psychologists about whether empirical research is the right approach to conducting research.

The main characteristics of empirical research are:

  • This framework follows the stages of the scientific method and provides step-by-step guidance on how scientific research should be conducted.

  • Data should be observable.

  • Data should be verifiable.

The following table describes the main features of empirical research and their relationship to quantitative quality criteria:

Quantitative quality criteria
Characteristics of empirical research
How is it achieved?
Validity

Observable

Observable data reduce the likelihood that subjective perspectives and experiences will influence data and analysis.
Reliability Verifiable Suppose we repeat the research in the same way/ in a different context/ in a different setting and similar results are obtained. In that case, the researcher can verify that the results and the conclusions drawn are reliable.
Objective
Follow the scientific method to research
The scientific method provides researchers with an empirical technique to use, limiting the effects of bias and thus increasing validity. Therefore, statistical inferences are deduced from data-driven, empirical evidence.

Quality Criteria - Key takeaways

  • Research quality criteria are requirements for research that psychologists have recommended and agreed upon.
  • Data and reports must meet the requirements to be considered quality scientific research.
  • The quality criteria for qualitative data are credibility, transferability, dependability, and confirmability.
  • The quality criteria for quantitative data are internal validity, external validity, reliability, and objectivity.
  • The main characteristics of empirical research are that it follows the scientific method's phases, data should be observable and verifiable.

Frequently Asked Questions about Quality Criteria

Research quality criteria are requirements for research that psychologists have recommended and agreed upon. 

The research quality criteria for qualitative research are credibility, transferability, dependability, and confirmability.

The research quality criteria for quantitative research is internal validity, external validity, objectivity, and reliability.

Objective research is defined as research that is ‘scientific’ and measured without the influence of subjectivity, such as the researcher's personal opinions.

Empirical research is defined as research that is based on direct observations, rather than subjective opinions, data and analysis techniques. For data to be classified as empirical it should be tested following the scientific method stages and be observable and verifiable.

Final Quality Criteria Quiz

Question

What is the definition of research quality criteria?

Show answer

Answer

Research quality criteria are requirements of research psychologists recommended and agreed upon.

Show question

Question

Why does research in the psychology field need to meet the requirements of the research quality criteria?

Show answer

Answer

Data and reports need to meet the requirements to be considered as quality, credible, trustworthy, empirical, valid and reliable research.

Show question

Question

What are the research quality criteria for quantitative data?

Show answer

Answer

  • Validity.
  • Reliability. 
  • Objectivity. 
  • Empirical research.

Show question

Question

What is the definition of standardising variables and how is it used to measure internal validity?

Show answer

Answer

Standardised variables refer to providing a definition of the variable and providing details of how the research is measuring the variable. This allows the researcher and the readers to identify and provide instructions on what the variables are, how they are measured and analysed. They can use statistical analyses to identify if variables are assessing what they are intended to measure (internal validity).

Show question

Question

How could researchers standardise depression (variable), in accordance with the following research scenario: ‘Research investigating changes in depression before and after cognitive behavioural therapy (CBT)’?

Show answer

Answer

Depression scores before and after CBT treatment were measured using Beck’s Depression Inventory. This inventory uses a likert scale to measure depression scores and the tests were scored using the instructions provided by Beck’s Depression Inventory. 

Show question

Question

How can researchers check the reliability of quantitative data? 


Show answer

Answer

Checking the internal reliability/ consistency of scales - for instance all of the questions in a scale that is measuring depression should be measuring depression levels

Assessing the generalisability theory - identifying the consistency of instruments used in research or if results obtained from participants are due to specific conditions   


Show question

Question

What are the research quality criteria for qualitative data? 

Show answer

Answer

  • Credibility.
  • Transferability.
  • Dependability.
  • Confirmability.

Show question

Question

How can researchers ensure that qualitative research produced is transferable?

Show answer

Answer

  • Taking detailed notes on what was observed during the research.
  • When describing participant behaviours and experiences, also make notes on context so that other researchers can understand and interpret the researcher’s perspective.

Show question

Question

What is the definition of triangulation and how does it ensure that research is credible?

Show answer

Answer

Triangulation refers to researchers using multiple data collection methods to confirm hypotheses. Triangulation ensures research is credible as using various data collection methods will provide data from various settings and contexts. This allows a generalised, observable overview of the phenomenon the researcher is interested in.

Show question

Question

What is credibility and what is its quantitative research quality criterion equivalent?

Show answer

Answer

Credibility refers to whether research findings are believable based on data collected from participants. Interpretations drawn should be reflective of this data.

Show question

Question

Which two research quality criteria use audit trails to ensure the criterion assumption is met? 

Show answer

Answer

Dependability and confirmability.

Show question

Question

What is the definition of empirical research?

Show answer

Answer

Empirical research is based on direct observations rather than subjective opinions, data and analysis techniques.

Show question

Question

What type of data can be considered empirical?

Show answer

Answer

Both qualitative and quantitative data can be empirical.

Show question

Question

What are the key characteristics of empirical research?

Show answer

Answer

  • This framework follows the stages of the scientific method and provides step-by-step guidance on how scientific research should be conducted.
  • Data should be observable.
  • Data should be verifiable.


Show question

Question

How does the scientific method ensure objective data is drawn?

Show answer

Answer

The scientific method provides an empirical technique for researchers to follow, limiting biases affecting data thus, increasing validity. Therefore, statistical inferences are deduced from data-driven, empirical evidence.

Show question

60%

of the users don't pass the Quality Criteria quiz! Will you pass the quiz?

Start Quiz

Discover the right content for your subjects

No need to cheat if you have everything you need to succeed! Packed into one app!

Study Plan

Be perfectly prepared on time with an individual plan.

Quizzes

Test your knowledge with gamified quizzes.

Flashcards

Create and find flashcards in record time.

Notes

Create beautiful notes faster than ever before.

Study Sets

Have all your study materials in one place.

Documents

Upload unlimited documents and save them online.

Study Analytics

Identify your study strength and weaknesses.

Weekly Goals

Set individual study goals and earn points reaching them.

Smart Reminders

Stop procrastinating with our study reminders.

Rewards

Earn points, unlock badges and level up while studying.

Magic Marker

Create flashcards in notes completely automatically.

Smart Formatting

Create the most beautiful study materials using our templates.

Sign up to highlight and take notes. It’s 100% free.