English:Research Methods in Psychology
Research Methods in Psychology
Introduction
Research Methods in Psychology is the toolkit psychologists use to turn questions about mind and behavior into evidence that can be evaluated, challenged, replicated, and refined. In this university-level aiMOOC, you learn how to move from a broad idea to a researchable question, choose an appropriate design, define and measure variables, recruit or identify a sample, collect and analyze data, interpret uncertainty, meet ethical responsibilities, and communicate findings transparently.
Psychological research is methodologically diverse. Depending on the question, researchers may use experiments, correlational designs, surveys, naturalistic observation, case studies, psychophysiological measures, archival data, longitudinal studies, qualitative interviews, or mixed methods. The central challenge is not to find one universally "best" method, but to create the strongest possible match between the research question, the evidence collected, and the claims made.
The historical laboratory image above illustrates how strongly psychology has been shaped by systematic observation and measurement. Modern research extends far beyond laboratories, but it retains the same core commitment: claims should be supported by procedures that others can scrutinize.
The video introduces several foundational psychological research approaches. As you work through this course, move beyond memorizing labels: ask what each design can establish, what threats to validity it faces, and what alternative explanations remain possible.
Learning Goals
By the end of this aiMOOC, you should be able to:
- Research question: Transform a psychological topic into a precise, answerable research question and distinguish exploratory from confirmatory aims.
- Operational definition: Translate abstract constructs into observable or measurable variables while evaluating construct validity.
- Research design: Select and justify experimental, correlational, descriptive, longitudinal, qualitative, or mixed-methods approaches.
- Sampling: Distinguish sampling from assignment, evaluate representativeness, and identify sources of selection bias.
- Research ethics: Apply informed consent, risk-benefit reasoning, privacy protection, debriefing, and institutional requirements to research scenarios.
- Statistical inference: Interpret effect sizes, uncertainty, confidence intervals, statistical tests, and power without overclaiming.
- Validity: Analyze construct, internal, external, and statistical conclusion validity.
- Open science: Use transparency practices such as preregistration, clear reporting, data documentation, and reproducible workflows when appropriate.
- Scientific communication: Critically read research reports and communicate methods, limitations, and conclusions accurately.
The Research Process
Psychological research is often represented as a sequence, but real projects are iterative. A literature review may change your question; a pilot may expose a measurement problem; an ethics review may require a safer procedure; unexpected results may motivate a new study. Good research therefore combines planning with documented, justified revision.
A typical research cycle connects several activities: identify a problem, review previous evidence, formulate a question or hypothesis, define constructs, choose a design, plan sampling and analysis, obtain required ethical approval, collect data, analyze evidence, interpret results, report limitations, and make materials or records sufficiently transparent for evaluation.
From Topic to Research Question
A topic such as "stress and memory" is too broad to test directly. A useful research question identifies the population or context, the constructs of interest, and the relationship or process you want to investigate. For example, a question might ask whether experimentally induced time pressure changes immediate recall performance in university students, or whether self-reported academic stress predicts recall scores during an examination period.
The wording of the question should match the design. Questions about association can be addressed with correlational evidence. Questions about causal effects require stronger design features, usually manipulation, comparison, control of plausible alternatives, and an assignment strategy that supports causal inference. Questions about lived experience or meaning may be better suited to qualitative inquiry.
Theory, Hypotheses, and Predictions
A theory is a structured explanatory framework, not a guess. A hypothesis derives a testable expectation from theory or prior evidence. A prediction specifies what pattern you expect to observe under defined conditions.
Good hypotheses are clear enough to be evaluated and potentially contradicted by evidence. Confirmatory hypotheses should ideally be specified before examining the outcome data. Exploratory analyses are valuable too, but they should be identified as exploratory rather than presented after the fact as if they had been planned in advance.
Operationalization
Psychological constructs such as anxiety, attention, prejudice, motivation, or working memory are not observed directly. Researchers create an operational definition: a rule linking the construct to a measurable procedure or indicator. Working memory might be operationalized as performance on a specific span task; anxiety might be represented by a validated questionnaire score, physiological response, behavioral measure, interview coding, or a combination.
Operationalization always involves a theoretical choice. Two measures that carry the same label can capture different aspects of a construct. You should therefore ask whether the operationalization adequately represents the intended construct and whether competing explanations for the measure are plausible.
The Stroop task is a classic example of operationalizing aspects of selective attention and interference through performance under conflicting stimulus conditions. A task becomes scientifically informative only when its procedures, scoring, reliability, and validity are understood.
Quantitative Research Designs
Quantitative research represents observations in numerical form and uses mathematical or statistical methods to summarize patterns, estimate parameters, compare conditions, test models, or quantify uncertainty. Quantitative evidence can be generated by experiments, surveys, observational studies, longitudinal designs, psychometric assessments, and many other methods.
Descriptive Research
Descriptive methods characterize what is occurring without necessarily testing causal relationships. Examples include structured observation, prevalence surveys, archival analyses, and case descriptions. High-quality description depends on systematic measurement, appropriate sampling, transparent coding, and explicit limits on generalization.
Descriptive evidence can be scientifically important. It may reveal the frequency, distribution, context, or sequence of psychological phenomena and can generate hypotheses for later research.
Correlational Research
A correlation summarizes the direction and strength of association between variables. A positive association means higher values on one variable tend to occur with higher values on another; a negative association means higher values on one tend to occur with lower values on the other.
A scatterplot should be inspected because a single correlation coefficient can hide nonlinear patterns, outliers, clusters, restricted ranges, or other features. Most importantly, correlation alone does not establish causation. An observed association may reflect a causal effect in either direction, a third variable influencing both, selection processes, measurement artifacts, or a mixture of mechanisms.
When interpreting a correlational result, separate the observed statistical relationship from the causal explanation you might be tempted to attach to it.
Experimental Research
In an experiment, the researcher manipulates an independent variable and measures its effect on a dependent variable. A strong experiment includes a meaningful comparison condition and controls alternative explanations.
Random assignment distributes participants among experimental conditions by chance. When implemented well with an adequate sample, it helps balance both known and unknown participant characteristics across conditions on average. Random assignment supports internal validity; it is not the same as random sampling from a population.
Experimental controls can include standardized instructions, blinding when feasible, placebo or active control conditions when appropriate, counterbalancing, manipulation checks, and consistent measurement procedures. The best controls depend on the causal question.
Between-Subjects, Within-Subjects, and Factorial Designs
In a between-subjects design, different participants contribute data to different conditions. In a within-subjects design, the same participants experience multiple conditions. Within-subjects designs can increase efficiency by controlling stable individual differences, but they may introduce order, practice, fatigue, or carryover effects. Counterbalancing can reduce some of these problems.
A factorial design manipulates or classifies more than one factor. It can estimate main effects and interactions. An interaction occurs when the relationship between one factor and the outcome differs across levels of another factor. Interactions are often theoretically important because psychological processes are frequently context dependent.
Quasi-Experimental and Natural Experiments
Sometimes random assignment is impossible, unethical, or impractical. A quasi-experimental design compares conditions without full random assignment. Natural experiments use naturally occurring events or policy changes that create informative contrasts.
These designs can provide stronger causal evidence than a simple cross-sectional correlation when they incorporate credible comparison groups, baseline measures, time trends, matching, discontinuities, or other design features. However, causal claims depend on the assumptions of the specific design and must be argued carefully.
Sampling and Generalization
A population is the broader set of cases to which a study seeks to speak. A sample is the set of cases actually observed. A sampling frame is the operational list or process from which cases can be recruited or selected.
Probability sampling gives population members a known chance of selection and can support population inference when the sampling process and response patterns are appropriate. Common probability approaches include simple random, stratified, cluster, and systematic sampling. Psychological research also frequently uses convenience, volunteer, purposive, quota, snowball, and other nonprobability samples.
A large sample does not automatically become representative. If selection systematically excludes important groups, increasing sample size may reduce random sampling error while leaving selection bias intact.
Random Sampling Versus Random Assignment
These concepts solve different problems. Random sampling concerns how people or cases enter a study and mainly affects generalizability to a population. Random assignment concerns how enrolled participants enter conditions and mainly supports causal inference within an experiment.
A study can have one without the other. For example, an experiment may use a convenience sample of students but randomly assign them to conditions. Such a design may have strong internal validity for the sample while still requiring caution about generalizing to other populations or settings.
Sample Size and Statistical Power
Sample-size planning should be connected to the research design, expected effect magnitude, measurement precision, statistical model, desired error rates, and practical constraints. Statistical power is the long-run probability that a statistical procedure will detect a specified effect when that effect exists under the model assumptions.
Power analysis can help justify sample size, but it is only as informative as its assumptions. Researchers should avoid treating a single conventional threshold as universally correct. Precision-based planning, simulation, feasibility constraints, sequential designs, or prior evidence may also inform sample-size decisions.
Measurement and Psychometrics
Measurement quality can determine the quality of a psychological conclusion. A technically sophisticated analysis cannot rescue a measure that poorly captures the construct of interest.
Levels and Forms of Measurement
Psychological data may include categories, rankings, counts, durations, reaction times, scale scores, physiological signals, text, audio, images, or behavioral traces. The data structure and measurement process should guide the analysis.
Single questionnaire items are often ordinal response categories, while multi-item scale scores are frequently analyzed using methods that treat the composite as approximately continuous. Such choices should be justified rather than assumed automatically.
Reliability
Reliability concerns consistency or precision of measurement. Relevant forms include test-retest reliability across occasions, interrater reliability across coders, and internal consistency among items intended to measure related content.
Reliability is necessary for many forms of valid measurement but does not guarantee validity. A scale can produce highly consistent scores while systematically measuring the wrong construct.
Validity of Measurement
Construct validity concerns whether evidence supports the interpretation of a measure as representing the intended psychological construct. Researchers examine patterns such as convergence with theoretically related measures, distinction from different constructs, expected group or experimental differences, internal structure, response processes, and consequences of use.
Validity is not a permanent property of an instrument detached from context. It concerns the interpretation and use of scores in particular populations, languages, settings, and purposes.
Qualitative and Mixed-Methods Research
Qualitative research is especially useful when the aim is to understand meaning, experience, interpretation, identity, social process, or context in depth. Data can come from interviews, focus groups, diaries, documents, observations, images, online interactions, or other records.
Common analytic traditions include thematic analysis, grounded theory, interpretative phenomenological analysis, discourse analysis, narrative analysis, and qualitative content analysis. These approaches rest on different assumptions and should not be treated as interchangeable labels.
Quality in Qualitative Research
Quality involves coherence among the research question, theoretical perspective, sampling strategy, data generation, analysis, reflexivity, and claims. Researchers should document how themes or interpretations were developed, how alternative readings were considered, what role the researcher played in the process, and how context limits transfer.
Qualitative sampling is often purposive rather than statistically representative. The goal may be depth, variation, theoretically informative cases, or saturation-related reasoning rather than population estimation.
Sharing qualitative data can support transparency, but privacy and contextual integrity may limit what can responsibly be shared. Ethical data stewardship can therefore require controlled access, careful de-identification, rich documentation, or a justified decision not to release sensitive material.
Mixed Methods
Mixed methods intentionally integrates quantitative and qualitative evidence to answer a research question more fully than either component could alone. A study might first identify a statistical pattern and then use interviews to investigate how participants interpret the process, or it might use qualitative work to develop constructs and quantitative measures for later testing.
The value of mixed methods comes from integration, not merely from placing two methods side by side. Researchers should explain how the components inform one another and how disagreements between them are interpreted.
Validity, Bias, and Alternative Explanations
A useful framework distinguishes several forms of validity.
Internal validity asks whether the evidence supports the proposed causal relationship rather than a competing explanation. Threats include confounding, differential attrition, history, maturation, instrumentation changes, expectancy effects, and systematic differences between conditions.
External validity concerns how findings may generalize or transfer across people, settings, tasks, measures, and time periods. Generalization is an empirical and theoretical question, not an automatic consequence of statistical significance.
Construct validity concerns the quality of the link between theoretical concepts and their operationalizations.
Statistical conclusion validity concerns whether the statistical analysis and uncertainty statements appropriately support the claimed relationship, given assumptions, data quality, model specification, and error rates.
Common Sources of Bias
Bias is systematic error, not merely random noise. Important examples include selection bias, nonresponse bias, measurement bias, observer expectancy, demand characteristics, recall bias, attrition bias, publication bias, and selective reporting.
Researchers reduce bias through design, not through confidence. Strategies include randomization, masking where possible, standardized procedures, validated measures, explicit exclusion rules, preregistration, blinded coding, adequate follow-up, transparent reporting, and sensitivity analyses.
Ethics in Psychological Research
Ethical research protects participants while preserving the social value and integrity of science. Requirements differ across countries and institutions, so researchers must follow applicable law, professional standards, institutional policies, and review procedures.
Core considerations include informed and voluntary participation, understandable information about procedures and foreseeable risks, the right to withdraw when applicable, fair recruitment, privacy and confidentiality, appropriate handling of sensitive data, proportionate incentives, protection from coercion or undue influence, and special safeguards when participants may be vulnerable.
Researchers should obtain required institutional approval before beginning data collection. Deception requires strong justification, must not conceal information that would make participation unreasonably risky, and should be followed by appropriate debriefing when used. Recording voices or images raises additional consent and privacy issues.
Ethics continues after data collection. Researchers must protect stored data, avoid fabricating or falsifying evidence, credit contributions appropriately, correct serious errors, and consider how findings may affect participants and communities.
Ethical Scenario Analysis
When evaluating a proposed study, ask:
- Scientific value: Is the question important enough to justify the burdens placed on participants?
- Risk: What physical, psychological, social, legal, reputational, or privacy harms are reasonably foreseeable?
- Consent: Can participants understand what they are agreeing to, and can they decline without inappropriate pressure?
- Privacy: What information is collected, who can access it, and how will it be protected?
- Justice: Are the burdens and potential benefits of research distributed fairly?
- Debriefing: If full information cannot be provided initially, what explanation and support will be provided afterward?
- Governance: What ethics review, institutional approval, or data-protection requirements apply?
Data Analysis and Statistical Reasoning
Analysis should follow the structure of the design and the measurement process. Begin with data quality: check coding, missingness, impossible values, distributions, outliers, repeated observations, and whether assumptions of the planned model are plausible.
Descriptive statistics summarize the observed data. Inferential procedures quantify uncertainty when moving from observed data to broader claims under explicit assumptions. Statistical models are tools for reasoning, not machines that convert data into truth.
The normal distribution is central to many statistical models, but not every psychological variable is normally distributed and not every analysis requires normal raw scores. You should examine the assumptions of the specific model rather than apply a generic normality rule.
Effect Sizes and Confidence Intervals
An effect size describes the magnitude of a relationship, difference, or model parameter in a meaningful metric. Its interpretation depends on the scale, context, design, uncertainty, and practical consequences.
A confidence interval is a procedure for constructing intervals that, under repeated sampling and the method's assumptions, achieves a stated long-run coverage rate. A single 95% confidence interval should not be interpreted as assigning a 95% probability to a fixed population parameter in ordinary frequentist analysis.
Confidence intervals help you think about precision. Wide intervals often indicate substantial uncertainty; narrow intervals indicate greater precision, provided the model and data are appropriate.
Null-Hypothesis Significance Testing
A p-value is computed under a statistical model and a null hypothesis. In simplified terms, it measures how incompatible the observed data, or more extreme data according to the chosen statistic, are with that null model. It is not the probability that the null hypothesis is true, and it is not a direct measure of effect importance.
Statistical significance should therefore be interpreted alongside effect sizes, uncertainty intervals, design quality, prior evidence, multiplicity of analyses, and the substantive meaning of the result. A small p-value cannot repair biased sampling, weak measurement, confounding, or selective reporting.
Type I Error, Type II Error, and Multiplicity
A Type I error occurs when a testing procedure rejects a true null hypothesis. A Type II error occurs when it fails to reject a false null hypothesis. Their probabilities depend on the testing framework, effect size, sample size, variability, and decision threshold.
Running many analyses and selectively reporting only favorable results increases the chance of misleading evidence. Researchers can address multiplicity through planned analysis strategies, adjusted error control where appropriate, transparent distinction between confirmatory and exploratory analyses, and replication.
Open Science, Reproducibility, and Reporting
Open science aims to make the research process easier to understand, evaluate, reuse, and build upon. Practices may include preregistration, registered reports, open materials, documented analysis code, data sharing when ethically and legally appropriate, persistent identifiers, transparent reporting, and replication.
Preregistration creates a time-stamped research plan before outcome data are analyzed. It can distinguish planned confirmatory tests from later exploration and make deviations visible. Preregistration does not forbid change: justified changes can be documented, and exploratory analysis remains valuable when labeled accurately.
Reproducibility and replicability are related but distinct ideas. Computational reproducibility asks whether the same data and analysis procedures produce the same results. Replication asks whether new data collected to test the same or closely related claim produce evidence consistent with the original finding.
Transparent reporting is essential even when materials or data cannot be openly shared. Researchers should explain procedures, exclusions, sample-size decisions, measures, transformations, analytic choices, deviations from plans, and limitations clearly enough for informed evaluation.
Reporting Standards
The American Psychological Association's Journal Article Reporting Standards provide structured guidance for quantitative, qualitative, and mixed-methods manuscripts. Reporting standards help readers assess rigor and understand exactly how evidence was generated.
Useful authoritative resources include:
- APA Style Journal Article Reporting Standards: Guidance for transparent reporting of quantitative, qualitative, and mixed-methods research.
- APA Ethical Principles of Psychologists and Code of Conduct: Professional ethical principles and standards relevant to psychological research and publication.
- Center for Open Science - Lifecycle Open Science: Guidance on transparent planning, connected research records, preregistration, and open research practices.
Reading Research Critically
A strong critical reader reconstructs the chain of inference from question to conclusion. Do not ask only whether a study "found something significant." Ask whether the design and evidence justify the exact claim.
Use the following sequence when reading a paper:
- Research question: What question is being asked, and is it descriptive, associational, causal, interpretive, predictive, or evaluative?
- Population: Who or what is the intended target of inference, and how were cases sampled?
- Operational definition: How were key constructs manipulated or measured?
- Research design: What comparison or observational structure makes the inference possible?
- Bias: What systematic errors or alternative explanations could generate the same pattern?
- Statistical model: What assumptions connect the observed data to the reported estimate or test?
- Effect size: How large is the observed relationship, and how precise is the estimate?
- Robustness: Do alternative reasonable analyses or specifications change the conclusion?
- Transparency: Were hypotheses, exclusions, measures, materials, and analyses reported clearly?
- Generalization: Which populations, settings, measures, and time periods are actually supported by the evidence?
Worked Example: Designing a Study
Suppose you want to study whether smartphone notifications reduce sustained attention during a reading task.
Question: Does receiving intermittent smartphone notifications during a 20-minute reading task reduce comprehension compared with a no-notification condition among university students?
Design: A randomized between-subjects experiment could assign consenting participants to a notification condition or a no-notification condition. The manipulation must be standardized, and the comparison should differ only in the intended notification exposure as far as practical.
Outcome: Reading comprehension could be measured with a prespecified set of questions. If the construct is broader sustained attention, comprehension alone may be insufficient, so a second validated attention measure might be justified.
Sampling: A convenience sample of one university can be appropriate for a course study but limits generalization. Random assignment can still support causal comparison within that sample if implementation is sound.
Analysis: The primary outcome, exclusion rules, statistical model, effect-size measure, and uncertainty interval can be planned in advance. Assumption checks and missing-data handling should also be defined.
Ethics: Data collection must follow applicable institutional approval and consent procedures. Avoid collecting unnecessary personal information, explain foreseeable burdens, and make withdrawal procedures clear.
Interpretation: Even if the notification group performs worse, conclusions should be restricted to the tested task, notification pattern, participant population, and study context. Broader claims about all smartphone use would require additional evidence.
Interactive Tasks
Quiz: Test Your Knowledge
Which feature most directly supports a causal comparison in a well-controlled experiment? (Random assignment to conditions) (!A large correlation coefficient) (!A convenience sample) (!A statistically significant result)
What is operationalization? (Defining how a construct will be measured or manipulated) (!Selecting only results that support a hypothesis) (!Repeating a study with a new sample) (!Removing all variation from a dataset)
Which statement about correlation is correct? (Correlation can describe association without proving causation) (!Correlation always identifies the direction of causation) (!Correlation removes all third variable explanations) (!Correlation is only used in experiments)
What is the main purpose of random sampling? (To support inference from a sample to a target population) (!To guarantee a causal effect) (!To eliminate measurement error) (!To make every study double blind)
Which statement best describes reliability? (It concerns the consistency or precision of measurement) (!It proves that a measure captures the intended construct) (!It guarantees representative sampling) (!It establishes that an effect is practically important)
What does a p value not directly provide? (The probability that the null hypothesis is true) (!A quantity calculated under a null model) (!A measure affected by sample information) (!Evidence that must be interpreted with design quality)
Which practice most clearly distinguishes planned confirmation from later exploration? (Preregistration of hypotheses and analyses) (!Increasing the number of outcome measures after analysis) (!Reporting only statistically significant results) (!Changing exclusions without documentation)
What is a central purpose of informed consent in research? (To support voluntary participation based on understandable information) (!To guarantee that participants know the study results in advance) (!To remove the need for ethics review) (!To make all research data publicly identifiable)
Which validity question asks whether the measure represents the intended psychological concept? (Construct validity) (!External validity) (!Statistical power) (!Random sampling)
What is a defining feature of mixed methods research? (Intentional integration of qualitative and quantitative evidence) (!Using two statistical tests on the same outcome) (!Collecting only interview data) (!Replacing research design with data visualization)
Memory Game
| Operationalization | Linking an abstract construct to a measurable or manipulable procedure |
| Randomization | Chance-based allocation used to reduce systematic differences between experimental conditions |
| Reliability | Consistency or precision of a measurement process |
| Confound | An alternative factor that varies with the proposed cause and can explain an outcome |
| Preregistration | A time-stamped research plan documented before outcome analysis |
| Triangulation | Examining a question through multiple sources, methods, investigators, or perspectives |
| Debriefing | Providing appropriate study information after participation, especially when full disclosure was initially limited |
Drag and Drop
| Match the correct terms. | Topic |
|---|---|
| Random sampling | Population generalization |
| Random assignment | Causal condition comparison |
| Test-retest reliability | Stability across occasions |
| Interrater reliability | Agreement among coders |
| Construct validity | Interpretation of a psychological measure |
...
Crossword Puzzle
| Randomization | What process allocates participants to conditions by chance? |
| Validity | What concept concerns whether an inference or interpretation is well supported? |
| Reliability | What term describes consistency or precision of measurement? |
| Sampling | What process selects cases for observation from a broader population? |
| Preregistration | What open science practice records a study plan before outcome analysis? |
| Debriefing | What process provides participants with appropriate information after a study? |
LearningApps
Cloze Text
Open-Ended Tasks
Before collecting any data from human participants, follow your institution's ethics and approval requirements. For course exercises, prefer simulations, open datasets, published materials, or nonresearch practice activities unless formal approval is already in place.
Easy
- Research Question Workshop: Choose a broad psychological topic and rewrite it as one descriptive, one correlational, and one causal research question; explain how the wording changes the evidence required.
- Variable Map: Create a one-page visual map linking one psychological construct to at least three possible operational definitions and annotate the strengths and limitations of each.
- Media Critique: Find a news report that makes a psychological claim, identify the implied research design, and write a short critique separating association from causation.
- Methods Glossary Video: Produce a three-minute explainer video that accurately distinguishes random sampling, random assignment, reliability, and validity using your own examples.
Standard
- Study Design Blueprint: Design an ethics-safe hypothetical experiment including a question, hypothesis, independent variable, dependent variable, comparison condition, assignment method, possible confounds, and planned interpretation.
- Open Data Analysis: Use a publicly available or instructor-provided psychology dataset to create descriptive summaries and one appropriate visualization, then write a paragraph explaining what the data can and cannot establish.
- Researcher Interview: With the interviewee's consent, interview a psychology researcher about one methodological decision they found difficult, then summarize how theory, feasibility, ethics, and evidence shaped the final design.
- Measurement Audit: Select a published psychological scale and evaluate the construct definition, item format, reported reliability, validity evidence, target population, and limitations of score interpretation.
Advanced
- Preregistration Prototype: Draft a preregistration for a hypothetical or already approved study, specifying hypotheses, sampling, exclusions, variables, primary outcomes, analysis, and how deviations would be documented.
- Replication Proposal: Choose a published psychological finding and design a direct or conceptual replication; justify what must remain constant, what may change, and which outcomes would meaningfully update confidence in the claim.
- Mixed Methods Project: Develop a proposal in which quantitative and qualitative components address complementary parts of one psychological question, and explain exactly where the two strands are integrated.
- Peer Review Simulation: Write a structured peer review of a published psychology article focusing on design, sampling, measurement, analysis, ethics, transparency, generalization, and at least two concrete improvements.
Learning Assessment
- Design Diagnosis: Given a claim that a campus mindfulness program causes higher grades because participants report both greater mindfulness and higher grades, identify the design problem, propose at least two alternative explanations, and redesign the study to strengthen causal inference.
- Measurement Reasoning: A new five-item scale has excellent internal consistency but weak correlations with established measures of the intended construct; explain why reliability alone is insufficient and propose a validation strategy.
- Sampling Transfer: Compare conclusions that could reasonably be drawn from a national probability sample, a volunteer social-media sample, and a randomized experiment using students from one university; distinguish population generalization from causal inference.
- Statistical Interpretation: Interpret a small estimated effect with a wide confidence interval and a nonsignificant p value; explain what can be concluded about effect magnitude, precision, and absence of evidence without equating nonsignificance with proof of no effect.
- Ethics Application: Evaluate a hypothetical deception study involving social rejection, identify participant risks and consent limitations, specify safeguards and debriefing requirements, and explain which issues require institutional ethics review.
- Open Science Transfer: A research team deviates from a preregistered exclusion rule after discovering an unexpected data-quality problem; explain how the team can respond transparently without treating preregistration as an inflexible contract.
- Integrated Critique: Select one published empirical psychology article and trace the complete inference chain from theory to question, operationalization, sampling, design, analysis, results, limitations, and generalization; identify the strongest and weakest links.
Evidence of Learning
Evidence of successful learning in this course includes knowledge of core designs, sampling logic, measurement theory, ethics, statistical reasoning, and open-science practices; skills in formulating questions, operationalizing constructs, diagnosing bias, selecting designs, interpreting uncertainty, and evaluating claims; products such as a study blueprint, measurement audit, data analysis, preregistration prototype, replication proposal, mixed-methods plan, or peer review; and transfer achievements in which you can apply methodological reasoning to unfamiliar studies, media claims, professional decisions, and new psychological questions.
Strong evidence of learning is not merely the ability to name a method. It is the ability to explain why a method fits a question, what assumptions support the inference, which limitations remain, what ethical duties apply, and what further evidence would change your conclusion.
OERs on the Topic
The following open or freely accessible resources can extend your study:
- Scientific method: Connect psychological research to general principles of hypothesis formation, testing, criticism, and revision.
- Experimental psychology: Explore experimental traditions and laboratory methods in psychology.
- Qualitative psychological research: Examine methods for studying meaning, experience, discourse, and context.
- Psychometrics: Develop deeper knowledge of psychological measurement, scale construction, reliability, and validity.
- Open science: Study transparency, reproducibility, data stewardship, preregistration, and replication.
- Meta-analysis: Learn how researchers statistically synthesize evidence across multiple studies.
Linked Learning Areas
Research methods connect psychology with statistics, philosophy of science, data science, ethics, cognitive science, sociology, education research, public health, and evidence-based professional practice. The same methodological habits—precise questions, valid measurement, appropriate comparison, transparent analysis, uncertainty awareness, and responsible inference—support rigorous work across disciplines.
