Research Methods -Psychology - class notes

Research Methods -for psychology - class notes

Class, Oct. 22, 2001

Scientific method: tries to avoid in distortions made by perception/judgment

-->reduces but doesn’t avoid mistakes of perception/judgment

-need to be able to answer the following:

1. is it an exact knowledge/science?
2. Can it be generalized, or is it specific to a specific population
3. Can it be reproduced independently?

-is there a specific/unique methodology to a social science? (i.e. observation/questionnaire)

-->not really! You can use many diff. kinds of methods

views:

1. general approach: in daily life, we rely on logic/intuition/what’s accessible to us

-->but is it universal?

-->i.e. more people on the street-->more likely to help us in an emergency

-->but it is not true -->the opposite is true

• needs to be statistically based:

-can’t be based on a single case: could be the exception

• needs to be detached from the ind. observer/experimenter: even a theory he believed in!!
• need set terminology
• when there is ambiguity (i.e. fear/anxiety), one needs to define each term we use.

Tools: need to be reliable and applicable

terms

Reliability: it always gives the same results -->the watch could be fast, but it is always fast by the same speed.

-->the ruler isn’t reliable since the temperature always changes its length a bit.

Hypothesis: a temporary assumption of explanation of a thing/relation b/w thing.

• need to be open for testing
• need to be phrased as a prediction/hypothesis -->that you can be able to test it

-in daily life, we tent to be ‘large’: general --> we like to generalize

-->let in science, we’re skeptical

Charles Spears: (philosopher)

4 sources of knowledge:

1. level of tenacity (intuition)
2. method of authority
3. logic/honest mind
4. scientific study

-scientific truth: probability… doesn’t have to happen: it is just probable it would happen.

-->Tylenol doesn’t always help

4 goal of science

1. Describing
2. To predict
3. To understand
4. Control

Tools of the scientific method

• scientific observation -->has to be methodolical
• sometimes you have to observe indirectly -->i.e. there is no actual ‘starvation’ -->you have to measure it indirectly -->i.e. body weight/food amount
• you can’t see anxiety, but you can see related b/h

• observation needs to be detached from personal theories/theories believed in

-->i.e. if I think that all balds are active, I’ll write ‘he’s tired’ instead of he’s inactive’ -->it infl. how I observe!!! (bad!)

• objective: if independent people will bring in the same result.

• measurement
• experiment
• good thinking
• repetition

Measures:

-in psychology, the measures aren’t as clear as other sciences

->measures don’t exactly measure what we mean

-->sometimes have to make own measures and test out if their measures are correct

-psychologists try to measure how accurate their terms are

-it doesn’t really matter how people use the term as long as they define it properly – so that at least everyone knows what they mean when they use the terms

Experimentation

-we try to experiment

-in order to run an experiment and prove something, we need to have a situation

• manipulate (tif’ul) the situation in a way that we measure only 1 thing – so that we won’t have various alternative explanations
• we also need a minimum of 2 cases that we compare against each other

• sometimes we make an experiment in a lab – to reduce interfering variables: might not be of representative of reality
• they do the same in physics
• this is the best way we have to seek causations

• need randomness

• Observable

--

-science: tries to look for general rules

Experimentation

Labs – help isolate individuality causes/effects (causality)

àmore possible factors - less certain what the real answer is

things that give experimentation their strength

1. we control which order
2. we control the size of items
2. we can repeat it
3. randomness

Good thinking

-hard to define

-has to have a logic

àwon’t be infl. by things like out biases/thoughts/etc…

Law of Parsimony

-Shortest/most inclusive rule

ài.e. you want to explain all the diff. outcomes of diff. populations w/I the experiment w/ 1 rule, and not many diff rules – 1 for each group

Open mindedness

-you take into consideration unexpected things.

-gov’t involvement – usually hinders the science

Replication

-that is why you need to write the process

-in psych – hard to replicate some things: like studying dreams – can’t necessarily repeat dreams

Ethics

Basic rule the person wont be hurt physically/emotionally in any way

àespecially when the damage is irreversible

• therefore: not allowed to activate self-image

Example:

• collecting brains for research w/o family permission
• romance w/ patient

APA’s standards: you’ve got to weigh the benefits vs. the ethical expense

àethical boards

• kids need parental permission
• needs to be told that they could end experiment whenever they want
• secrecy is promised
• need to be told beforehand if pain is involved
• informed consent

-ethics also apply to experiments on animals

Class, Dec. 24, 2001

theories/hypothesis

-the more cases/issues/variance that are explained by the theory – more inclusive – it is a better theory

Theory: a a system of principles/assumptions that explains/predicts b/h.

-explains relationships b/w things

-->an organizing scheme.-->organizational schemes

-->a point of origin for future research

-info are just building blocks. The theory arranges them into a structure

-internal vs. external locus of control:

-Rotter: you can divide people,e ito 1 of 2 groups:

• Internal locus (point) of control
• Mom yells at me, depending on what I do
• External locus of control
• Mom yells at me, regardless of what I do

-when 1 is promoted to a managerial job from the group – there is a feeling of inequity, yet if a manager is brought from the outside -->higher motivation to listen to him

Theory develops, and exchanges others

Norman triplet

If a person does something which another person does as well, I do it better/faster:

-->coaction: almost instinctual tendency to b/c aroused and do it faster

-->this instinctual energy is called dynamogenesis

Psychologists don’t like to speak of instincts:

1. there are things which we used to think are instinctual, which turned out to be a learned thing
2. anthropologists found things which are seemingly ‘instinctual’ yet diff b/w tribes

-->can’t necessarily be instinct

3. humans are beyond animals/instincts
4. I don’t explain it when I define it as an instinct -->it just labels the act

Floyd Olport

(cousin of the one who defines milgram’s experiment as the eichman experiment, -b/w they blamed everyone else)

-->changed name ‘coaction’ to social facilitation

-change of name = usually shows a change in direction

F. Olport: gets people to do simple vs. hard tasks (i.e. math/puzzles/etc.

-->social facilitation helps when simple task/distracts when task is hard

-->vs. triplett, who thought that coaction only delps

Zajonc:

-the mere presence of others – not only of they’re doing the same act

-->also happens to animals as simple as ants!

Zajanc (psychologist) notes:

-highly practical response/instinctive =facilitated in presence of coaction/audience

-newly learnt responds/actions =impaired by presence of coaction

-everything has a hierarchy of responses

-->simple tasks =the higher responses of the hierarchy are easily accessible (i.e. pressing the pedal of the car to a red light)

-->yet when there is 3 buttons, each having to be pressed to red/yellow/green light, until it is well learned, then social facilitation will make you make more responses, since pressing the button is not the highest on the hierarchy of response of seeing those color lights

2 stepped-study:

-people had to read sounds w/o meanings, each w/ various rated of repetition (of it returning)

-->the more it returned, the more dominant response

-then they were told that they were shown flashes of those nonsensical words (yet in reality, they were just show flashes of scribbles)

-->they answered that they were shown flashes of the repeating words more

-->but the curve was steeper w/ social facilitation

-Zajanc: only people who could appreciate our actions (as opposed to people who can’t, like sleeping people)

-->the uncertainty of if we’re ok, that dev. when someone is looking at us, leads to the social facilitation

Study:

Scenarios:

• Alone
• Experts
• average people
• video camera which will be shown to experts

Results: order of social facilitation, from highest to lowest:

• experts
• video
• experts
• alone

-Conclusion: the mere knowledge of being watched, w/o being watched almost has the same effect

Study

Duval/wickland

Objective self-awareness: it is an issue of self-awareness – not others’ evaluations

-->here, the ind. checks his ‘standard of correctness’ – how in-line he is w/ being correct. -->yet people find gap b/w what they want to be and what they are.

-->so they try to bridge the gap b/w reality and ideal, or escape the gap

[subjective self-awareness: how I look at someone else.]

Study

-waiting for results for a test:

• if a mirror is facing them: more likely to leave, especially if he did poorly.

-->evaluation of oneself

-->development of theory of how we change our actions

-->after all, triplet’s theory can explain cases like video or mirrors

Class, Jan 7^th, 2001

Operational definition

-each paper has to define its terms, since each person has diff. usage of the terms

àlike a recipe – has to tell what exactly the researcher meant

i.e. Anxiety is unclear unless you clearly state what is meant meaning of the word

àI might use anxiety diff, but it doesn’t matter, since I know what he means, since he told me

-i.e. ‘learning’ = too vague

àso I might define it as 3 times right in a row(?)

Validity

Interval vs. external validity

Internal validity

Internal validity- to what degree can we claim that the change is b/c of the indp. Variable and not b/c of other reasons

àin other words, the researcher had control over the results

-as opposed to having rival/alternate explanations

3 criteria for internal validity

1. Control group

àThe standard to which we compare the dep. variable to.

àmakes the baseline – w/o it – we don’t know how to relate the differences in the results to.

2. Extraneous variables: variables that effected our results w/o our intention
3. Confounding variables: variables that disturb the result: can’t know whether it or the indp. Variable brought the result.

àthe diff b/w this and extraneous variables is that here, the variables are mixed in from the beginning. Otherwise, they are the same.

-I need at least 2 levels in the indp. Variable to make an experiment

àone is given to the control group/condition

àthe other is given to the experiment group/condition

External validity

Can It be generalized to other cases?

Jan 14, 2001

The ideal is to test ALL the populations.

-->since this is impossible, you have a representative sample, not a small population

-->i.e. Bar-Ilan population is not representative of I. society

• one’s answering must not infl. the others’ individual answering

Table of random numbers: a chart made by a computer where all the # are random

-the external validity is hurt if you didn’t choose random

Important:

• if you did not divide the participants randomly into the various variables of your experiment -->decrease in internal validity

-->subject variable: w/o randomness of division of (randomly chosen) participants, there is possibly going to be one trait in one side and another side on another side

Example - Brady

Question

-Why do managers dev. more ulcers than others

Answer

-studies on monkeys – 1 is a leader monkey -->has to press a lever in order so shock qll monkeys

-->manager monkies had more ulcers than regular

-->learnt helplessness -->tells us the opposite: that more control = better health

-but then people started realizing than that Brady didn’t have randomness: faster learners were managers/slower learners were ‘other’ monkeys

-->NO RANDOMNESS

-->therefore, we have an alternative answers: the monkeys went through testing for sensitivity to electricity/etc.

->random repetition of the above experiment w/ rats

=> helpless rats = more ulcers

Size of sample

-the bigger the # - the more reliability the results are.

-the weaker the strength of diff. (how diff. the 2 variables are), harder the tell diff -->need more #

-never less than 10 participants

The experimental design

=the structure of the experiment

-there are many diff. kinds of structures

• Experimenting same thing could be using a diff structure of experiments
• Studying diff. things = same structure experiment

The differences lie in:

• The diff. indp/dep variables
• number of levels
• who participated in which part of the experiment
• between subjects design vs. within subject design vs. mixed design

Example

Good vs. bad news

Between subjects design

-diff participants in each of the categories

Within subject design

-the participants participate in some/all of the experimental situations

-->also called repeated measures design

Mixed design

-one variable is between subject design and another variable is within subject design

between subject design:

-simplest experimentation: 2-group-design

• Independent groups: control (baseline group) vs. experimentation group
• Matched groups: when there is a small # of participants, it is hard to say that the groups are matched ànot enough people of be representative, despite the randomness (b/c of the small #’s)

Matched groups

-only do hat’ama when the literature said that there is a involving varialbe

-making pairs of things that are similar àthen putting them on opposite groups (control/experimentation), based on say: a toss of coin

àcan do this b/f or after the setting up of the experimentation

• after: you take out the extreme ones who don’t have a pair in order to not have an alternative reason

1) Multiple group design

-some groups

àyou need more than 2 groups, since you also want to take into account amount of the variable

ài.e. if you want to measure effect of coffee on schooling, you also want to take amount into account: diff. amounts of coffee

F-test – to see if there is a diff. b/w individual diff and diff. b/w it and the average

àanalysis of variance: is the diff. b/c ind. differences w/I the group, or the diff. is actually b/w groups?

àlook up chart in stats class of last year

-if calculated F is:

• smaller than above # in the chart: not significant diff.
• b/w 2 # (or ) = w/I 5% significance
• equal/more than bottom # in chart – 1% significance

-deal 2/ charts of h.m. people are in the experiment

2)Factorial design

example

Report: F(1,3) = 15,45, p<0.05

--

factor = same thing as independent variables in factor analysis

Example 2x4 experiment

àsince there are 2 levels of the independent variable: 1 df

à3 levels of B = 3 df

Questions to do in an experiment

1. h.m. indp. variables are there
2. how many factors
3. h.m. participated in the experience
4. fill in the df
5. find the minimum F that the experiment needs in order to all the cells to be significant

Reporting formula:

(AxB,WI) = F P<sig. level

example – a researcher reports that in a 2-fsactorial experiment, there is a sig. Diff.

f(2,54) = 11.8 P<05)

answers

3x2

6 cells

3 factors

à10 in each group

100 ot 99 participated

2X5

or

3 X 3

# of groups:

9 or 10

facrtors 2 4 or 5

àimportant: there is no correct answer since there are alternative answers

--

from here on, unless noted: 2X2

example

-we decide that any /x bigger than 1.5 needs to be clarified

Main effect

• a is significant (in each column, the diff b/w average b/w a1 and a2 is biger than 1.5
• B is not significant, since the diff b/w average of the 2 bs is not above 1.5

Interaction

Subtract A1B1 from A1B2 and A2B1 from A2B2 and see if the diff. b/w the 2 #’s is more than 1.5 – if yes àinteraction. 8-8.5 =-0.5 and 1.5-1 = 0.5 = diff = 1

èno interaction

-the question is is AXB sig? Is /a sig diff from /B

àno – in each case (A1-A2 / B1-B2), the diff. b/w them is smaller than 1.5)

**-w/ graphs thynggy

To find on a line graph:

• Main effect (= significant) èfind the midpoint
• Interaction: – find if there is a diff. b/w A and b line in diff parts of the graph

Important: if 1 is significant – than it is all significant

In other words:

A =the diff b/w A and B àsig. If their diff. are sig.

B = the diff b/w averages of all of (interaction)

àif the diff of averages stay constant = no interaction

à find if there is a diff. b/w A and b line in diff parts of the graph

** all of the above

-to find the source of interaction: see if A rows is sig or B columns is sig

--

** Graphs/DF

March 11 2002

àwe know how to compare 2, but in cases of more than 2, we pair 2 at a time of the many variables (B1B2, B1B3, B2B3) as long as 2 of the many variables are significant, then we say it is sig.

--

within subjects design

-until now, we spoke of b/w subject design – how diff people react in diff situation

ànow, we’re speaking about the same person in 2 diff cases.

Benefits of w/I subject design:

1. you reduce diff. in groups b/w of individual diff., since it is the same people on both groups
2. you can adapt test to individuals (i.e. political people he likes)

1. less people needed in the experiment àresources
2. time saved teaching participant what to do in the experiment
3. sensitivity to W/i group error is lower since it is same people
4. we watch the same person over time and now diff. people at diff. times

Downsides of w/I subject design

1. a person has to be in a lab for a long time àperson mightget tired and react diff. to experiment
2. sometimes, longitude studies are too long-timed to be done (40 yrs. On the same person
3. sometimes you have alternative answers: i.e. arts who’s brain areas have been removed and are violent – perhaps[s it is b/c of the medicine that puts them to sleep) àso you put a group w/o brain removal to sleep and another group that also the brain removed
4. carry-over effect: when your reaction to something is infl. by something you did b/f. Therefore, you could be infl. by the previous part of the experiment!!!

**\\>

Small N design

-kind of w/I subject design.

ABBA design

Where you reverse the order of the manipulation to see the effects.

àcan also be don on a group as a unit

Example: behavior modification: teaching person new ways of b/h as alternative to maladaptive b/h. in those cases, you don’t want to return to the maladaptive situation àit is not ethical

questions

-table of random numbers -->what is the story here?

March 18, 2002

-external control, i.e. dealing w/ things like social factors.

• Internal validity: that the results are because of the things I did and not other factors
• Randomness
• Matched groups: control over important factors which might infl. result i.e. age
• Within individual experiment: eliminates some external influencing factors

Physical factors:

• Way participants are instructed = infl. their performance
• Surrounding: noisy/weather/pressure from friends to finish experiment

Resolution:

1. Try to negate the distraction
2. Balancing: run the experiment in 2 different environments [splitting between the 2 situations randomly] and see if there if a diff.

àthis doesn’t negate the infl. of the external factors, but it makes sure that it doesn’t infl. one side and not the other (control group/experimental group)

Within-subject experiment: has a unique problem: order effect

-Order effect: with 2 stimuli, perhaps the order in which you gave it makes a difference to the result. I.e. perhaps it effects participant’s moods/cognitive process/etc.

àyou need to counterbalance

-Progressive error: same thing as order effect- just with more than 2 stimuli

-Counterbalance: give all the stimuli more than once, and see what the infl is of giving the same stimuli. 0) S1 1) S2 2) S2 3) S1àmight over-do the stimuli. [0- hunger 1- hunger…..) (0+3, 1+2 = balances out)

Between subject counter-balancing: compare

1. Complete subject counter-balancing: ˝ do one order of stimuli. Others do the reverse order of stimuli. In more than 2 stimuli, N! Orders of stimuli
2. Partial subject counter-balancing: when you have many different orders of stimuli (i.e. 120), it is hard to do 120 different orders. So you do like 10% or something of the orders and report on them. Note: we didn’t negate the external stimuli, but we balanced it out across the variables of the experiment
3. Latin square design: we ensure that the balancing b/w the diff. experimental scenarios (the diff. stimuli) will 1) appear in at least once in every stage of the order, and 2) each variable will have at least once be b/f and after each other variable. Helps minimize order effect w/ least # people. I.e. (A will be b/f and after B, C, D at least once) ABA, BAB, CAC, DAD

àformula: (to choose orders to minimize # of orders needed to have both above rules) ABLCL-1DL-2E àensures the 2 aforementioned rules.

Explanation of formula

A B L C L-1 D L-2 E

Take A, B, Last, C, Last –1, D, Last – 2, E etc….

-then fill out downwards

example of formula

-in case of A, B, C, D, E, F

First:

A B F (last) C E (Last-2) D

ABFCED

BCADFE

CDBEAF

DECFBA

EFDACB

FAEBDC

-in Latin square, you really need double as many as letters, when the number of stimuli is uneven

example (of 5)

ABECD

BCADE

CDBEA

DECAB

EADBC

-now reverse the orders:

DCEBA

EDACB

AEBDC

BACAD

CBDAE

àthose are the 10 orders really necessary

control over miscellaneous variables:

i.e. personality/causal

-infl. internal and sometimes external validity

Personality variables

-there is a response style: answer consistent answers (i.e. ‘yes’) regardless of the content of the question

-->to narrow down extreme scores b/c of this, you can ask the question in reversed manner, therefore, narrowing the infl. of those who have an extreme response style

MMRI= tests personality things like OCD/depressions/mania/etc.

Response set: tendency to answer in a way which one ought to answer and not truthfully, in order to give a certain impression

Crown/Harlow: social desirability scale:

Social desirability scale: if the person lies on the tests

Example of questions:

-do I get sometimes get angry

-I know my voted party’s statement of mission

-->high score =high set-response

solution

-you can phrase question in a way which the people don’t know how to answer.

-->i.e. ‘pets are fun to raise but some people don’t enjoy it’

situational/social factors

Demand characteristics: just like in the response set: people wonder what the experiment is really about and therefore try to realize the experimenter’s hypothesis (they don’t want to screw it since they want to look normal and not crazy)

-->no external validity, since they don’t act in a spontaneous way

-->studied by Orne

way to deal w/ demand characteristics:

Single blind experiment

-experiments, especially in medicine, where all the info is given, except he doesn’t know what situation he is in.

-->you’re told that you’re in a situation i.e. headache medicine, where one group is getting the medicine and one group is getting a placebo. The participant doesn’t know which he is getting

cover story:

-we distract the person from real issues of the experiment

accident

-make them think that what is happening is an accident, but it is really the experiment’s issue

example

-darlie/natane:

• 3 participants by themselves
• 1 participant/2 experiment helpers

-see how long it takes them to leave (experiment helpers were supposed to stay)/report

Accomplice/confederate

-accomplice doing the important thing, while the experimenter just doing the measuring. The participant just doesn’t know that the experiment is really done mostly by the person who is really an accomplice.

Example:

-while waiting for experiment to start, the accomplice nods at a certain word. Then, when the participant enters the experiment, he is interviewed, where the measurement is really how many times he said that word after being conditioned earlier (by the accomplice)

Detachment between the experiment and the measurement

-i.e. to check cognitive dissonance, see if the attitude change is there the day after when participant doesn’t know that he’s really asked about the thing he took place earlier (i.e. day b/f)

Experimental bias: the resulting bias made by the experimenter: even the instructions have an effect on the participant, even though what he said never happens

April 22, 2002

Experimenter bias: diff. terms for the same thing: way the experimenter b/h in order to effect the results

Also called:

• Experimenter bias
• Experimenter effect
• Rosenthal effect
• Pygmalion effect
• Self-fulfilling prophecy

àeven on animals!

Examples

• how people treat the experiment, given what they assume the experimenter told them that they’re handling
• (i.e. smart rats vs. dumb rats)
• teachers told that specific students will be ‘intelligence bloomers’ àthey really increased in intelligence (teachers asked them more/gave them more chances to correct themselves/hint to them more the right answers)

àThey even interpreted ambiguous stimuli diff.

Double blind effect: both the participant and the experimenter don’t know which of the experimentation settings the participant is in

Quasi-experimental design

-researches similar to experiments

ài.e. we run them in labs, where we can control variables/their intensities/order of events/right control groups

àin field-research, we are limited in our ability to control.

ài.e. in comparing effects on divorced people’s kids: you can’t make people divorce for the sake of the experiment

-natural setting experiments are used to:

1. isolate external validity to the lab experiment
2. improve the conditions/to get better results

àkinds of ways to help released criminals

3. things that you can’t do in a lab

Differences b/w lab and natural settings:

• you can’t manipulate the independent variable
• experiments in natural settings has external validity
• in what they try to achieve
• lab: academic purposes: to find out things that bother the specific researchers (called: basic research)
• natural settings: in order apply it (called applied research)

hardships in field research

• hard to randomize/split participants
• When it is, it is usually not ethical (i.e. in dividing the groups in schoolings/in medical issues)
• can’t control the manipulation of the independent variables
• no available control groups appropriate to the experiment

Threats: things that might weaken the validity of the experiment

Possible threats:

1. History: influences other than the ones that we thought/wanted to measure, that could possibly infl. the dependant variables.

à I.e. after announcing a new economic plan, people asked how their satisfied from gov’t economic plan. Their answer is problematic, b/c it could be that the $ is low and the mortgage rates went down and not only had to do with their recent announcements

2. Maturation: changes w/i our participants

ài.e. you measure something in the beginning of grade 7 and end of grade 7 is problematic since they mature a lot in that age, independently of your experimental manipulation

àalso: person might be more tired/more relaxed through the experiment

àhistory w/I the participant

3. testing: weakness when we measure the thing twice and the 1^st testing might infl the 2^nd testing:
• the guy knows the questions
• the guy already know what you are trying to achieve
• the guy already asked others the answers
4. instrumentation: the tools/machines got worse

ài.e. the interviewer got tired/the exercise machine got old

àquestion: is it me that got better, or the tool which got worse

difference:

-testing is change in the person. Instrumentation is change in the tools (interviewer could also be a tool)

5. statistical regression (regression = mean): tendency of experimental trial results to move towards the average: tall/short parents: kids =closer to average

ànow, I don’t know whether the change is b/c of statistical regression or the experimental manipulation

scatter diagram

Pretest scores↓	7	8	9	10	11	12	13	Avearge
13				1	1	1	1	11.5
12			1	1	2	1	1	11
11		1	2	3	3	2	1	10.5
10	1	1	3	4	3	1	1	10
9	1	2	3	3	3	1		9.5
8	1	1	2	1	1			9
7	1	1	1					8.5
Mean pretest	8.5	9	9.5	10	10.5	11	11.5

-regression line (line that goes through the most predictive scores)

• goes b/w the pretest 8/9 to 11/12
• goes b/w posttest 8-9 to 11/12

àshows the people moving to the center (average)

6. selection: perhaps the results are b/c of the participants chosen (lack of randomness) àThey were prone to those results from b/f the experiment! àmanipulation didn’t work that the groups are diff. from the beginning (chose a specific course/class àspecific groups would go to that course)
7. mortality (attrition) –people leave the experiment
8. interaction: interaction b/w the diff. threats. For example: the manipulation only worked on a certain group àonly worked by it was a special group.

àgrade 7 kids chosen, the manipulation worked, though if it was grade 5 kids, the manipulation didn’t work b/c they are diff. populations

àinteraction b/w manipulation and maturity

3 kinds of experiments

1. experimental
2. quasi-experimental

Bad types of experimental setup

1 shot case study

XO (x = manipulate/O =measurement

àwe measure 1 group -after manipulation, we measure its effect

problems:

• History: History: other factors, other than the experiment infl.
• Testing/instrumentation irrelevant, since there is only 1 testing
• Regression: irrelevant since you don’t have an extreme score to deal w/

1 group pretest-posttest design

O₁XO2

Problems

1. History: other factors, other than the experiment infl.
2. Testing
3. Instrumentations -->whenever there is a problem w/ testing, there usually is a problem w/ testing
4. Regression: you don’t now whether it is extreme
5. Maturation:
6. Interaction: still can infl. only this group w/b they are unique.

**

Static group comparisons

-I have 2 groups:

-XO and O

-->1 group w/ manipulation and 1 w/o

-->not random (i.e. 1 underwent the course and 1 didn’t)

Problems:

• History: other factors, other than the experiment infl.
• Testing: no 2 measurements -->can’t compare the two
• Instrumentation: same
• Regression- can’t know it they’re extreme
• Selection: maybe they were diff. beforehand
• Maturation: perhaps people left the course (manipulation) b/f
• Interaction: perhaps this group is unique

29.4.02

îç÷řéí ăîĺéé đéńĺé, çĺěůä ůě äđéńĺé - äîůę

5) Statistical regression – řâřńéä ěîîĺöň, đčééú ä÷öĺĺú ěäâéň ěîîĺöň.

îîĺöň âĺáä äéěăéí ěäĺřéí âáĺäéí îŕĺă éäéĺ đîĺëéí îîîĺöň ääĺřéí, ĺîîĺöň âĺáä äéěăéí ěäĺřéí îŕĺă đîĺëéí éäéĺ âáĺäéí îîîĺöň ääĺřéí. ěîä ćĺ çĺěůä ? äŕí äůéôĺř äĺŕ úĺöŕä ůě ŕéîĺď çĺćř ŕĺ ň÷á řâřńéä ěîîĺöň, ÷řé äđčééä ůě áňěé öéĺď đîĺę ěůôř áđéńéĺď đĺńó ŕú öéĺđéäí.

äđçä ůě řňéĺď äřâřńéä, äéŕ ůéů ěëě îáçď ůâéŕä, ëůá÷öĺĺú äůâéŕĺú äď éĺúř âăĺěĺú. áŕîöň ääúôěâĺú äůâéŕä éëĺěä ěäéĺú âí ěëéĺĺď äâáĺä ĺâí ěëéĺĺď äđîĺę ĺěëď đůŕřéí ôçĺú ŕĺ éĺúř áŕîöň. ěđîĺëéí éů ůâéŕĺú ůđĺčĺú ěîňěä ĺěâáĺäéí äôĺę.

čáěä ăĺ ëđéńúéú Scatter gram – ôéćĺř äöéĺď ěôé îăéăä řŕůĺđä ĺůđééä

Mean posttest	Posttest score								Pretest score
	13	12	11	10	9	8	7
11.5	1	1	1	1					13
11	1	1	2	1	1				12
10.5	1	2	3	3	2	1			11
10	1	1	3	4	3	1	1		10
9.5		1	2	3	3	2	1		9
9			1	1	2	1	1		8
8.5				1	1	1	1		7
	11.5	11	10.5	10	9.5	9	8.5		Mean pretest

ä÷ĺ äĺŕ ÷ĺ äřâřńéä, äĺŕ ä÷ĺ ŕĺúĺ ŕđé îđáŕ. ŕĺúĺ ÷ĺ ůîńáéáĺ ńëĺí äůâéŕĺú äĺŕ ä÷čď áéĺúř.

posttest	pretest	posttest	pretest
13			13
12			12
11			11
10			10
9			9
8			8
7			7

řâřńéä ěîîĺöň – äúĺôňä ůě äđčééä ěäúëđń ěîîĺöň.

ä÷ĺ ůě ä÷ĺřěöéä äĺŕ äîîĺöň ůě ůđé ÷ĺĺé äřâřńéä.

äîůę äŕéĺîéí / çĺěůĺú äîç÷ř :

6. áçéřä Selection

äúĺöŕĺú ůŕđĺ î÷áěéí ŕéđí áâěě äúôňĺě ŕěŕ ň÷á ůäŕđůéí ěôđé äúôňĺě ëář äéĺ ëŕěä.

7. đůéřä Mortality (attrition)

đůéřä ůě đáă÷éí. ńĺâ îńĺéí ůě đáă÷éí đůř ĺáâěě ćä ÷áěúé äáăě áúĺöŕĺú ĺěŕ ÷ůĺř áîä ůçůáúé.

8. ŕéđčřŕ÷öéä Interaction

ŕéđčřŕ÷öéä áéď äŕéĺîéí äůĺđéí. éëĺě ěäéĺú ůéů ńě÷öéä ůě đáă÷éí, äúôňĺě ôňě ŕę ř÷ ňě ÷áĺöä ńě÷čéáéú, ňě ŕçřú ćä ěŕ äéä ôĺňě. úëđéú ěéîĺăéí ěëúä ä ĺ-ć', äůôňä ůĺđä ňě ëě ëéúä- éëĺě ěäéĺú ůćä áâěě ůéů áâřĺú âăĺěä éĺúř áëéúä ć, ëę ůäčéôĺě îůôéň ŕçřú ňě ä÷áĺöĺú âéě äůĺđĺú.

îňřëé îç÷ř ářîĺú ůĺđĺú

1. äřîä äřŕůĺđä - Preexperimental design

ŕěĺ äď řîĺú đîĺëĺú ůě îç÷ř ëîĺ :

ŕ. One shot case study (XO). XO – X ä÷ĺřń, O ńĺěí äňîăĺú ůňůéúé ěŕçř îëď. îç÷ř áĺ ÷áĺöä ŕçú ůŕçřé äúôňĺě áĺă÷éí äůôňä ŕçú ůŕđĺ çĺůáéí ůäúôňĺě âří ěä . ěăĺâîŕ- äŕí ä÷ĺřń áéäăĺú äáéŕ ěŕçĺć âáĺä ůě çéěĺđééí áňěé ăňĺú çéĺáéĺú ňě ăúééí ? ëëě ůěîç÷ř éĺúř çĺěůĺú ëę äĺŕ éĺúř âřĺň. ěŕ îúâář ňě îĺůâ ääéńčĺřéä (îůäĺ ŕçř îäúôňĺě âří ěúĺöŕĺú), ěŕ îúâář ňě äáůěĺú, ěŕ ňě äîăéăä, Instrumentation ćä áňéä ůě ůúé îăéăĺú ĺëŕď ŕéď ěé ůúé îăéăĺú, řâřńéä ěŕ řěĺĺđčéú, ńě÷öéä ŕĺîřú ëé îěëúçéěä ěńčĺăđčéí äéĺ ăňĺú çéĺáéĺú ĺěëď äĺŕ ěŕ îúâář ňě ńě÷öéä, đůéřä- ëě îé ůěŕ ŕäá ăúééí ňćá ŕú ä÷ĺřń ĺěëď ŕéđđĺ îúâář, ŕéđčřŕ÷öéä äéŕ äřňéĺď, ůéëĺě ěäéĺú ůä÷ĺřń äůôéň ę ř÷ ňě îé ůáçř áĺ ĺěŕ ňě ëě äŕđůéí áöĺřä ăĺîä.

á. One group pretest post test design (O1XO2) äĺńôúé îăéăä ěôđé ä÷ĺřń. ěŕ îúâář ňě äéńčĺřéä, áůěĺú, îăéăä éů ěé 2 îăéăĺú ĺěëď ééúëď ůäáéđĺ ŕú äůŕěĺď, äëěé – äîřŕééď îëéř ŕú äńčĺăđč ĺěëď îăář ŕěéäí ŕçřú ĺäí éůúôřĺ. řâřńéä- ŕé ŕôůř ěăňú ŕí äňîăĺú ÷éöĺđéĺú. ńě÷öéä- îúâář ňě ćä îëéĺĺď ůŕđé éĺăň ŕéę äđáă÷ äéä ěôđé ćä. đůéřä Mortality- îúâář ňě ćä, ëé ŕôůř ăřę ůúé îăéăĺú ěăňú ŕí đůřĺ ŕĺ ěŕ ĺěŕ ëîĺ áîăéăä ŕçú ůěŕ éĺăňéí ŕć ŕí đůřĺ ŕĺ ěŕ. ěŕ îúâář ňě ŕéđčřŕ÷öéä- ëé âí ôä äúôňĺě éëĺě ěäůôéň ňě ä÷áĺöä äćĺ ůîěëúçéěä äâéňä ě÷ĺřń áâěě ňîăĺú ŕĺ đčéĺú îńĺéîĺú.

â. Static group comparison (XO äçĺ÷ř ěŕ ńéĺĺâ ŕú äđáă÷éí

( O

ě÷áĺöĺú äůĺđĺú ŕěŕ ÷éáě ŕĺúí ëîĺ ůäí. ŕđé îůĺĺä ůúé ÷áĺöĺú, çéěĺđééí ůě÷çĺ ŕú ä÷ĺřń áéäăĺú ĺ÷áĺöú çéěĺđééí ůěŕ ěîăĺ ŕú ä÷ĺřń. ÷áĺöä ŕçú ňí úôňĺě ĺäůđééä ěěŕ úôňĺě. ŕđé îá÷ů ŕú ăňú äçéěĺđééí ěâáé ăúééí. îúâář ňě äéńčĺřéä, îúâář ňě áůěĺú, ěŕ řěĺĺđčé äîăéăä (ŕéď ůúé îăéăĺú) ëę ěâáé äëěé, řâřńéä, ěŕ îúâář ňě ńě÷öéä (ëé ŕéď ěé ňĺă îăéăä ěôđé), đůéřä, ŕĺ ŕéđčřŕ÷öéä.

2 experimental design

Solomon four-group design

R = random

4 patterns of manipulation/observation

R O1 X O3

O2 O4

X O5

O6

àyou see if there is

Solomon four-group design is really comparing 2 experiments

Pretest-post test control group design

R O1 X O3

O2 O4

Randomized-two-group design

R X O5

O6

--

Quasi-experimental design

-(interrupted) times-series design

imp:->NO RANDOMNESS IN Quasi-experimental design

-as seen above - if you want to reduce weakness --> add observations or measurements

-quasi-experimental design: some measurements beforehand/manipulation/a few measurements afterwards:

i.e.-O1, O2, O3, O4, XO5, O6, O7, O8

example:

-astronauts in space-walks =nervous. Therefore, more swearing. I.e. O1-4 X=space-walk [O5 measured during at x (space-walk)] and then O6-8

• projected estimate/projected baseline: what (which line on the graph) you estimate that the guy will do

O1 O2 O3 O4 O5 O6 O7 O8

Variations of the times series response

àeffect is lasting (i.e. vaccination)

àdelay b/w manipulation and reaction

Multiple time series design

-no randomization/no groups

-i.e. after a certain thing happens in 1 city, compare it to another place

ànot a random place, i.e. after a thing was screened in 1 city, but not in another ity, seeing the change in one city, and compare it to what happens in a city where the movie wasn’t screened

Group 1

O1 O2 O3 O4 O5 O6 O7 O8

O1 O2 O3 O4 O5 O6 O7 O8

Group 2

Multiple time series design:

àunlike time series –multiple time series can deal w/

regression discontinuity

-continuation of static group comparison

• (40-49) àX (they get help in math class)

• (50-59) àX
• (60-69) àX
• (70-79)
• (80-89)
• (90-100)

Cross-sectional study: comparison diff. groups at same time

Longitudinal study: take a group over a long time

X1 X2 X3 O4 O5 O6

Pretest/posttest non-equivalent control group

O1 X O2

O1 O2

àdotted line = non-compared [b/c you didn’t choose them]àunlike the Salomon – where they a equivalent (controlled)

àfor example 1^st is frontal learning classroom. Other is jigsaw classroom

Measurements

-Giving #’s systematically to phenomenon

-# have no meaning w/o definition of what each # on the scale means

-you have a set of # -Set A (say participants) A1, a2, a3, a4, a5

-you have a set B – gender (b1, b2) you classify A into set b = that is a kind of measurement

measurement = turning results into # that rank

àyou have to have isomorphism: connection/logic in the arrangement of the numbers:

àneed to have connection b/w the numbers and what you’re measuring

àthough there might mistakes, i.e. in exceptions: not all males are masculine to the same extent even though all males are classified 1 on a gender question

àif no consistency = lower isomorphism

for example: if you have test results, and you have same marks for several participants, you add up all the ranking # and divide by # of participants w/ than score

Real mark: the accurate ranking

Used mark: taking same results into account (as seen above)

--

90à1.5

90à1.5

70à6

80à3.5

80à3.5

70à6

70à6

70à6

70à6

60à10

Correlation: to check isomorphism

p =1- (6åD²/N³- N)

D² = diff. b/w original mark and measurement-given #

àthe sum of all if diff. b/w original results and given

N = # of pairs

àthe answer of the formula: how good the correlation is

ranking/real ranking D²

1 1 0

2 2 0

4 3 1

5 4 1

6 5 1

8 6 4

1-7/216-6

=1-(1/30)

=1-0.0333333…

r=0.96666666…

Measurement Rules

1. Isoid (equivalence) rule: A=B and A≠B are mutually exclusive

àais diff than b

2. If A=B and B=C then A=C i.e. the results of 1^st place to 2^nd place is not the same thing as the results from 2^nd place to 3^rd place.
3. if A>B and B>C then A>C transitivity

ànot all things in life are transitive= but then it is not a tool

scales/levels of measurements

1. nominal: only rule #1 exists = a is diff. than b a=b OR a≠b
• # doesn’t have any quantitative value!!! Only qualitative diff.!
• # is arbitrary àdon’t have anything to them

2. Ordinal: not only quantity, but also an order. The number has no value, but the order does.

ài.e. the Richter scale

àrule #2

3. Interval: I have no info regarding the relations: I don’t have the 0

àrule #3

4. Ratio: I have all the aforementioned info, including the absolute 0, therefore, I have the exact relation b/w all the numbers used

May 27, 2002

-last class, we saw that there is a diff. b/w real measurement and wanted measurement

àeach term is measured/expressed/defined in diff. ways

examples:

-intelligence: speed of answering/amount of knowledge/how much math you know

-gender: what people are biologically/what they define themselves socially

-now that we decided how to measure a variable, we need to measure:

1. Reliability: is the tool ok for measurement of what we want?
2. Validity: is this a ok way to measure the specific variability

-reliability is a necessary but not sufficient for measurement.

àif it is not reliable, then it is not valid.

Reliability

Reliability: accuracy/consistency of the term/variable across situations

3 approaches to measure reliability

1. if we measure the set of objects the second time (given they weren’t changed) àwill we get the same results
2. is it a ‘true’ measurement?
3. What is the sized of error of measurement.
1. 2 kinds of errors.
1. Systematic error: constantly making the same mistake: the watch is 1 hour lagging, each 24 hours àreliably, but no validity.
1. X_t =Xµ + e

àXµ = true score X1+X2……/N

2. Random error: inconsistent mistakes: lagging by various times in diff. cases. ànot reliable and validity

--

***

àget help for when/how to use formula

To know for exam:

Reliability formula:

R=Vt-Ve

Vt

Or

Rtt = 1-ve/vt

Error = also called residual – given on the test

So plug into formula

=30.17-7.14/30.17

Example

Note: in d, everyone is more reserved about answering: systematic error

Variance: ind’s diff. from average squared

s

²=

å

(Xi-/X)²

n

--

C=total squared –n

**get help

Subject

-I Can compare diff. parts of testing items to get variance

à

i.e. 1^st half w/ 2^nd half

à

split-half reliability: compare the odds w/ evens (might get tires, so no comparing 1^st half w/ 2^nd half

à

comparing the

å

o and

å

e gets reliability

P=1-(6

å

d²/n³-n)

N=# of pairs

--

P=1-(6

å

d²/n³-n)

N=# of pairs

1-6(18.5)/120

0.75

coefficient varient??**

r² = the common part of both tests

-there is a rule in reliability: the more items, the more chance of being similar

à

but in splithalf reliability, we reduced the # of items

à

so there is a way to fix it: spearman-brown formula

spearman-brown formula:

Rn=NR/1+(n-1)r

R=wanted correlations (how much there really is)

N = wanted # of items (the # that really exists)/existing items in out split half test

Example: 10 items of half-split

Rn=20/10 x 0.6

1+(20/10-1) x 0.6

1.2/1.6

=6/8

=0.75

--

Question: why not compare all of them to each other?

N = n(n-1)/2

4 = 4(3)/2 = 12

chronbach alpha/ internal consistency: ranking each one againstg each other: a vs. b’s ranking a

à

b, c, d,, B vs. c, d.

->when using correlation as a score = correlation of a pair = 1 score. 2 pairs = 2 scores

-question on the test: where is there most reliability

chronbach alpha:

µ

=N

Ø

/ 1+(n-1)

Ø

n=# of TOTAL items

Ø

= the average of correlations of the N [= n(n-1)/2]

test-retest reliability: measured against all of the items

à

very similar in reliability to cronbach,

à

since it tests twice, and each item is considered, since Xt = X + e[rror] **

alternate forms reliability/parallel forms: when the same question is rephrased when retested