A. Identify and Describe What is to Be Studied, and Justify the Selection

1. Develop a clear and concise statement of what generally is to be studied
and what general questions you are interested in answering. These might include
the following.

a. Processes. Processes might be understood as changes in events and in
configurations of events through time. For example, (1) how students form cooperative-learning groups; (2) how teachers burn out; (3) how students drop out vs
complete school; (4) how reform efforts are conducted; (5) how changes in ideas foster
(or fail to foster?) changes in organizational structure or practices.

Notice that studying processes already suggests longitudinal research. We might be
interested in discovering: 1) what factors "initialize" a process (start it and set its course,
or trajectory); 2) which factors sustain a course and which factors alter a course; and
3) apparent phases or stages.

b. Problems. For example, (1) teacher burn-out (which is also a process); (2) low student
achievement; (3) the rate and severity of children's aggressive behavior; (4) counterproductive child-caregiver exchanges

Here, we might be intersted in discovering the origins and development of problems (seen
as processes), and the factors that might lessen or prevent problems. (This would be basic
research.) And (later), we might be interested in pilot testing, then replicating (further
testing), then demonstrating, then evaluating, and then disseminating programs of prevention
or remediation. (These would be applied research.)

c. Relationships. There are two general sorts of relationships. One is causal
relationships. These imply that there are "necessary" (determined) relationships among
variables. For example, if the bowling ball hits the pins with a certain force and at a
certain angle (independent variables), then certain pins will (must) fall in a certain direction (dependent variables).

A second is functional relationships. These do not necessarily imply determinism,
but merely co-occurrence (e.g., job satisfaction tends to go along with working hard) or
concomitant change (as one variable changes, the other variable changes). In either type of relationship, antecedent (independent) variables might be seen as predictors of the existence
or change in dependent variables.

Examples include the following.

1. What are the causes, or what merely are the predictors (possibly implying mere
correlation, or concomitant variation), of student achievement vs. failure? This suggests
survey research of schools differing with respect to achievement.

2. Which instructional design fosters the most participation and achievement in students?
This suggests experimental research in which one design is pitted against another.

3. Which school reform strategies and tactics produce more beneficial change, faster
change, and fewer problems, with the least investments and costs? This would be evaluation research, perhaps using case studies of contrasting reforms.

2. Conduct a review of relevant (obviously relevant and potentially relevant)
theory and research, and your own experience.

First, concentrate your knowledge of theory and research, and your own experiences,
on the targets identified in 1 above.

Second, expand the search; that is, look into related literature, talk to more people,
and deepen your knowledge by reading and critically examining more on the same subjects.
For example, if the target is teacher burn-out, study relevant literature
on psychological stress, social stress, illness, and job turnover in industry.

Also, what gaps (things that need explaining) poor reasoning, and weaknesses in research
(e.g., definitions, samples, instruments) can you find--for your research possibly to remedy?

Finally, summarize "the current state of knowledge" or "our current understanding of this
problem" in the form of a table, a model, or both. A table might cite important pieces of
literature and briefly identify concepts and propositions in them that contribute to the "state
of knowledge." From this, you then argue how gaps in knowledge need to be filled or how obvious next steps can be taken to extend what we know and/or can do. In a model, you
draw a flow diagram that depicts interconnections among variables--possibly through time.
(See pages 12, 13, 20, and 22 in "Studying social systems as objective....," and pages 5-7 of "Applied educational research..." in the course packet.) Then translate what the model says,
and indicate gaps to fill, next steps to expand the model, and/or the need to test the model.
This sets you up for describing your proposed research.

Following is an example of a model that conceptualizes a complex process.

3. On the basis of 1 and 2, list more specific questions or phenomena to
be studied. These can be framed as follows.

a. Hypotheses to test. For example,

"Based on our review of relevant theory and prior research, we expect to find that students
with diagnosed learning disabilities will learn to solve mathematics problems faster, with
fewer errors, and with greater satisfaction (dependent variables) using the methods of
back-chaining or shorter total cycle instruction than they will with forward chaining or
whole task instruction (independent variables)."

Notice that this suggests either: a) experimental research pitting one method against another
(which would provide fairly clear support for the hypothesis, or would demonstrate the falsity
of the hypothesis), or b) some kind of survey research; e.g., examining classrooms where the
different methods of task presentation have been tried [retrospective] or are being tried [predictive]--which might enable you to infer (but not exactly "see") which method works better.

b. Specific possible relationships or processes to examine. For example,

(1) "Specific relationships that (will be investigated, we will seek to discover) through the
proposed research include the following: a) a relationship between teachers' ratings of
children's achievement and potential in elementary school, and children's subsequent
academic achievement; b) patterns of early parent-child interaction and children's subsequent prosocial vs antisocial behavior; and c) a relationship between children's prosocial and
antisocial behavior and teachers' ratings of school achievement and potential." [Note the
triangle.]

(2) "We are interested in describing in a more comprehensive way the process by which
students introduced to cooperative learning form or fail to form productive and cohesive learning groups. We will focus in particular on students' sense (shared, not shared) of their objectives and expectations; emerging divisions of labor; emerging differentiations of power, authority, and prestige; patterns of instructional interaction; mechanisms of social control; and students' participation (e.g., attention, satisfaction, effort) and achievement (e.g., scores on standardized tests and rates of correct answers)."

4. Show that the specific questions or phenomena identified in 3 above are able
to be addressed in a study.
For instance, can you obtain relevant information? Can you:

a. find and recruit participating organizations, groups, and individuals?

b. obtain relevant information (data)? For instance, can you: (1) define relevant concepts/variables (2) create instruments that measure the concepts/variables (3) create procedures or settings for collecting the information; e.g., experimental protocols,
interview procedures, access to opportunities for direct observation, sampling methods to
use in a survey, collection of organizational documents)? c. analyze the data? For example,
do you have knowledge of how to perform, and persons available to perform, qualititative
and/or quantitative analyses?

5. Justify the study.
For example, show that it is:

a. Timely. "We must begin to focus up-stream, on the origins of fetal alcohol syndrome,
and not spend millions of dollars and children's lives trying to remedy it after it is too late."

b. Relevant to a practical problem. "Research shows that education for children with disabilities is wasted if efforts are focused on skill acquisition and ignore generalization and maintenance."

c. Relevant to a theoretical problem. "What variables help to explain the maintenance
of change, as, for example, in school reform efforts and teacher training?"

d. Likely to fill gaps in practical (how to) knowledge or theory. "Which is the
better school reform strategy--quick and radical or slow and reformist."

e. Likely to permit of generalization to other persons, settings, and situations. "Instructional techniques developed in this pilot project with autistic children may be useful
with children having less severe impairments and/or impairments of a different sort."

f. Likely to sharpen definitions or suggest limits of empirical generalizations.
"It may be that early child-caregiver interaction is not as strong a predictor of elementary
school prosocial and antisocial behavior as was previous thought. An antisocial trajectory
begun in the family, for example, may be re-set by experiences such as..." [You would
complete the sentence with variables that you identified in the literature and displayed on
your model.]

g. Likely to suggest the generalizability or the limits of practices.
"It is reasonable to suggest that back-chaining techniques used to teach certain problem
solving skills to children with learning disabilities work best on tasks that..."

h. Likely to create or improve research instruments and methods.
"With modification, the 'School Assessment Guide' developed for this project could be
used also to assess family structure and function."

6. Who might be the audiences for the findings or methods produced by your research?

a. Will the main audience be custodians of theory? If so, you might think of your research as "basic research."

b. Will the main audience be practitioners of some kind--teachers, administrators,
therapists? If so, your research might be labeled "applied research." If it is applied research,
is it "intervention research"; that is, are you stepping into and trying to change a situation?
If it is intervention research, is it for a specific or local setting; e.g., a class or a school
("action research"), or is it for a larger population (e.g., classes for children with learning disabilities)--in which case, you may be doing research that is, or that leads to, "dissemination research"?

c. Will the audience be policy makers and/or administrators, who might use the findings or methods to make policy or to reform programs? If so, we may be talking about "evaluation research." For example, your general research question might be, "How
effective was the intervention in contrast to other interventions?"

7. In view of what you said in 1-6, identify and discuss what sort of research is planned.
For example,

a. Is it to be a first contribution or effort ("pilot project")? If so, you might
be able to use a sample that: (1) is small; (2) is not representative of the population for
which the research might be relevant; (3) will not provide information regarding all of the variables in the theories and past research that informed the research; and (4) does not
examine or test the whole conceptual model that summarized the state of knowledge.

For example, initial applied research to develop programs for children with autism often
used small samples of highly impaired children. The rationale was: "If the intervention
works with these children, it surely ought to work with less-impaired children." Or, "If the
pilot test (or pilot project) yields significant findings (fosters significant improvement), we
will replicate the intervention with a larger and more representative sample. For example,..."

b. Is it "replication research"? If so, is the research designed to: (1) examine the
reliability, or repeatability, of a phenomenon ("We really do have something here."), or
(2) examine the generalizability of empirical generalizations and/or practical methods to
different samples, settings, and/or populations? Moreover, will it be: (1) an exact replication
(i.e., the new sample and setting have the same characteristics as the earlier sample and setting), or (2) a replication with some alteration? For example, we might wonder whether a training program for families of high-functioning children with autism works as well with a heterogeneous group of children.

c. Is it to be a demonstration of what you or other persons have come to believe or
have already tried in a pilot test ("demonstration project")? For example, after revising and replicating different sorts of lessons, instructional methods, and levels of classes for autistic children for some years, you now might examine the entire program. This might require a longitudinal case study--following children as they move through the program, and identifying weaknesses and strengths in the program. Or, you might frame the research as an experiment, and compare your program with a different sort of program.

d. Is the research a "critical experiment," in which: (1) one theory is pitted against another; e.g., predictions from Piaget vs predictions from behavioral principles (2) one hypothesis (frustration ---> aggression) is pitted against another hypothesis (frustration -->
[only if reinforced for prior aggression]--> more aggression) (3) one practice (e.g.,
intervention, instructional technique, school reform strategy) is pitted against another.

B. Discuss Background Assumptions That Will Guide the Research

In this section, present and justify biases, beliefs, values and philosophical positions
that may guide and/or limit your thinking, research methods, and interpretation of findings. Following are things to consider.

1. Etic (observer's) vs emic (members') perspective
Having reviewed the literature and conceptualized the phenomenon of interest (number
2 above), do you believe that research must have an etic perspective; i.e., focus on what is objectively real from the point of view of observers, but which may be invisible or may be understood differently by "members" or participants--e.g., patterns of interaction, the
structure of activities, relations of power, details of teaching methods, features of the
physical environment? If so, data collection might be more quantitative, and involve direct observation--event recording, maps, narrative recording, rating scales, task analysis, tests.

However, do you believe that research must have an emic perspective; i.e., focus on what is subjectively real (for individuals) and intersubjectively real (shared by members) from the point of view of members or participants--members' typifications of time, space, objects, persons, activities; rules; norms; commonsense reasoning (explanations, descriptions); justifications, legitimations, valorizations? If so, data might be more qualitative, and data collection might involve interviews (especially open-ended and in-depth interviews); questionnaires; diaries and journals; home videos (what do people record? how do they talk about it?); and direct observations (using narrative recording and field notes).

From these one tries to extract evidence of how members make sense of what is going on.
Or do you believe that you must have both perspectives to better understand a phenomenon.
For example, you might want to obtain: (1) an observer's typifications and explanations of child-carergiver interaction ("There is a high rate of punishment exchanges in this family. Punishment usually occurs when caregivers have many tasks at once."), and (2) caregivers' typifications and explanations ("We hardly ever punish; when we do, its when the kids need it").

2. Nomothetic vs idiographic
Are you interested in finding what is or what might be the case generally; i.e., are you
looking for "laws" or "law-like relationships"? If so, your aim is nomothetic. The following questions reveal a nomothetic aim. They also reveal: a) a metaphysical assumption that the
world is organized in a law-like way, and b) an epistemic assumption that such law-like relationships can be discovered and communicated (e.g., via equations and "If X, then Y" propositions).

"What are the mathematics of learning? Is the phase of acquisition best described by linear functions or by power (exponential) or logarithmic functions?"

[You might collect data on many students learning many tasks; graph the data (e.g., new tasks learned per week); and derive the input-output equations.]

"What is the nature of interaction between the physical environment, ideas and social
structure?"

[Here, you might do case studies of new organizations or of organizations that have initiated reform. You would look for pathways by which earlier changes in some variables appear to effect change in other variables. For example, what happens after a new mission statement is published?]

"Does social organization tend towards increasing petrification or towards chaos?"

[Here, you might do a longitudinal cohort or panel study of classes/teachers and schools at different times in their development; e.g., first year teachers or first year of school
operation, second year teachers or second year of school operation, etc. You could use contrasting schools to see if features of schools-- size, achievement of students, outside
political processes--affect petrification vs chaos.]

If one's aims and assumptions are nomothetic, then one is likely to conduct experimental research or survey research that is deductive in nature. That is, one is likely to: 1) assert hypotheses (hypothesized relationships among independent and dependent variables);
2) conceptually define each variable; 3) operationally define each variable; 4) develop instruments for measuring the operationalized variables, and develop procedures and
protocols for using the instruments to collect information (probably quantitative) to test
the hypotheses; 5) compare what the data say with what the hypotheses asserted; 6) draw conclusions regarding the validity of the hypotheses.

On the other hand, are you interested in how things work, or in how certain things are accomplished, in a more local environment? Do you, for instance, expect to find that each school's efforts at reform are unique? Or do you simply not care about general laws? If so,
your aims and assumptions are idiographic. "What fosters aggression in this school?" In this
case, your research may be ethnographic research involving lengthy and extensive observation, and in-depth (story-telling) interviews. Or it may be survey research.

Either way, your research would be inductive in nature. That is, you would not begin
with hypotheses, specific questions, and pre-selected variables and relationships to examine. Rather, you would: 1) begin with an interest and some rather broad questions ("How is aggression accomplished?"); 2) probably collect more qualitative data (interviews, field notes);
3) examine the data to identify classes of events (e.g., kinds of aggression and kinds of
settings), relationships among classes of events (e.g., certain kinds of aggression occur
in certain kinds of settings), sequences (e.g., how fights are started, progress, and end),
and how members make sense of what is going on; and 4) end with empirical generalizations
that summarize what you learned. This sort of research may not enable you to generalize
outside of your sample.

3. What level of organization are you studying?
Individual? Interpersonal? Group? Organization? Organization-in-environment? Or,
within individuals, are you studying actions, tasks, activities, roles? Note that relevant
variables and definitions of variables depend on the level of organization. For example,
the operational definition of aggression in children would differ from the definition of aggression in adults. Likewise, the variables that describe change in an individual's
behavior would have to be supplemented by many more variables if we were examining
change in an organization. For example, in an organization, we would have to consider
relations of power and divisions of labor. See "Levels of analysis" in the course packet.

4. How complex and large is the population to which you may want to generalize or apply your findings or methods?
If the population is large and complex, and if you want to be able to obtain generalizable
findings or methods, then you may need a large and representative sample (unless you are
doing a pilot test/project or original research).

C. Identify and Justify the Proposed Research Methodology

1. Will you conduct case studies, ethnographic research, phenomenological research, and/or historical research--generally idiographic, descriptive, inductive
and qualitative?

2. Will you conduct nonexperimental surveys?
A survey can be either descriptive or explanatory research.

a. If your aim is descriptive, you are interested in what is going on, not why it is going
on. "How many students of different kinds drop out of school?"

b. If your aim is explanatory (hypothesis-testing), then you are interested in: 1) discovering predictors/antecedents/causes of other (dependent) variables, or 2) testing whether
hypothesized relationships ("Poor students have a higher probability of dropping out"--big news!) are supported or are false. Explanatory research would probably be deductive, nomothetic, and quantitative.

3. Will you conduct experimental research--which is generally nomothetic, deductive,
and quantitative?

In view of everything that you have written in response to the above, why is your selection a good one?

D. Describe and Justify Methods of Data Collection

1. Are you planning to do research that is cross-sectional--a slice of a
population's characteristics or one or more organizations' doings at one point in time?
If so, you cannot really speak about processes. Or do you plan to do research that is
longitudinal. If longitudinal, you might do a cohort study (e.g., study schools in different phases of reform) or you might do a panel study (repeated data collection--interviews, direct observations, tests, etc,--on the same group--a panel).

2. What methods of data collection will you use?

a. Will you use direct observation: 1) narrative recording (running records and specimen description; 2) field notes; 3) rating scales; 4) tests; 5) event recording (continuous and intermittent, or time sample); 6) task analysis?

Will observers collect all of these data, or will participants collect some of their own data?
If participants will collect some of the data, will they collect data on the same "things"
using the same instruments as outside observers, or will participants collect data on the
same "things" using different instruments (e.g., an observer uses event recording to measure children's disruptions and the teacher uses a rating scale), or will participants and observers collect data on different things?

b. Will you use interviews? If so, will they: 1) be formal or informal (e.g., more casual);
2) be in-depth or not in depth; 3) use pre-selected questions (e.g., standarized interview
schedule) or be more creative (explore issues); and 4) have closed-ended or open-ended questions?

c. Will you use questionnaires?

d. Will you use participants' logs, diaries, videos?

e. Will you use official statistics and organizational documents? Note: In
selecting instruments and methods, you have to decide how precise you want the
information to be? If you want greater precision (exactness), you may have to collect
data on a ratio level (real numbers)--by event recording, tests, or documents of some
kind. If information does not have to be precise, you might use rating scales (e.g., ordinal
level: "often; sometimes; rarely") and narrative recording (describing via a running
record).

3. If your research is deductive (involves pre-selected variables and relationships to examine), you must identify, describe, and justify these variables and relationships in light
of the problem or phenomenon and your conceptualization of it. Then give conceptual
definitions and operational definitions for each variable. Then describe your instruments and methods of data collection.

Discuss: a) how different instruments will measure different variables (e.g., direct
observation and rating scales might be used to measure teachers' teaching competence,
and task analysis might be used to measure children's competence); and b) how different instruments (or portions of instruments) might measure the same variables (e.g., a
questionnaire might measure parents' perceptions of their competence at teaching,
while direct observation and rating scales might be used to measure how parents actually teach--"triangulation"). If your research is more qualitative, describe methods of data
collection; e.g., how you will make field notes and conduct informal interviews.

Indicate how the range of variables and instruments is sufficient to address the selected
questions.

4. Describe protocols (operating procedures). For example,

a. How will you distribute and collect questionnaires? What will you do when people do
not send them back? Will you call them? Will you obtain new participants?

b. How will an experiment be conducted? That is, what is the design; how will participants
be solicited, selected, prepared, and protected; what will happen in each experimental period; how will data be collected; how will sources of internal and external invalidity be handled?

c. How will interviews be conducted? That is, how will participants be solicited, selected, prepared, and protected; how will interviews be conducted (where, when, how long);
how will interview data be collected (taped and transcribed; notes; tape plus notes)?

d. Where will documents and official statistics be obtained? In what order?

E. What is Your Sampling Plan?

1. What is the population for which the research is relevant: individuals, schools, families, communities, etc. with certain characteristics?

2. What is the population, pool, or frame from which your sample will be
drawn? Note that this is not necessarily the same (although it ought to be) as the
population for which the research is relevant. For example, if you were doing research on parents' satisfaction with their children's schooling in a particular county, would the
population, frame, or pool from which you draw your sample be: a) all families of children
in school in the county; b) all families of children in school in the county listed in the phone book; c) all families of children in school in the county who are members of the PTA;
d) families from a list of names supplied by teachers?

Notice the trade-off. Choice "a" is the same as the population for which the research
is relevant, but it is going to be more expensive to sample from this large pool. The pool
in choice "b" will be easier to reach, but how well does it represent the population for which
the research is relevant? (What about families with no telephone or with unlisted numbers?)

3. How will the sample be drawn from the pool? For example,

a. Will you use simple random sampling? This means that families are selected at
random from a list ("subject pool" or "sampling frame") of, for example, all families of
children in school in the county.

b. Will you use stratified random sampling? For example, families, perhaps from
a list of all families of children in school in the county, are divided into three smaller lists
(strata) based on some important variable, such as income--"comfortable," "marginal," "dependent." Families are then randomly selected from each of the strata lists until a large enough sample is obtained.

c. Will you use proportional sampling? For instance, families, perhaps from a list of
all families of children in school in the county, are (similar to stratified sampling) divided
into subgroups based upon some characteristic (e.g., religion). If 10 percent of the families
are Buddhist, then 10 percent of the families on the Buddhist list are randomly selected. If
30 percent of the families are Catholic, then 30 percent of the families on the Catholic list
are randomly selected.

d. Will you use cluster sampling? Here, instead of sampling from a list ("sampling frame"
or pool) of individual families, you sample from increasingly smaller clusters. For example,
first you might randomly select from clusters (lists) of pre-schools, elementary schools,
middle schools, high schools, and special schools in the county. Then, within these school clusters, you randomly select grades. Within these clusters of grades, you randomly select
classes. And within these clusters of classes, you randomly select children and their families.

e. Will you use snowball sampling? Here, your sample consists of a growing number
of participants who recommend additional participants. "Who else might have a valuable
opinion on this matter?" This is not random sampling.

f. Will you use purposive sampling? Here again, the sampling is not random; each
element of the sampling pool does not have an equal chance of being selected. However,
you still want representativeness. And so, you purposefully sample from important subgroups--wealthy, middle income, and poor families; "movers and shakers" and "the
masses"; families of the brightest and families of the most struggling students; families of children of different ethic groups and religions.

Make sure to justify your choice with respect to a need for a representative sample, the availability of participants, and resources.

4. If comparison groups will be involved (as in an experiment or survey), how
will participants be assigned to groups?

For example, will you use: a) random allocation (e.g., 50 participants are allocated to a
25 member experimental group vs a 25 member control group based on a coin toss);
b) matching (e.g., you make sure that each group has equal proportions of males and
females, religions, and other variables that you think are important); c) self-selection (e.g.,
a principal says, "We can't receive your training this year. We'll serve as your
delayed-training control group."); d) existing groups (e.g., one math class is the
experimental group and the other is the control group).

Justify and suggest limitations of this choice with respect to ethical principles and the need for sample equivalence.

5. How will participants' rights be protected?

F. How Will Data Be Analyzed and Presented?

1. Will you use statistical techniques? If so, identify and justify them with respect
to the precision of findings and the power to find relationships.

2. Will you use qualitative data analysis? If so, how will you "read" the data;
identify "things" of interest; "discover," test the reliability, and assess the generalizability
of patterns?

3. Will you use graphs, flow charts, case-building, and/or statistics to present findings?

G. Conclusions and Interpretations

What sorts of things will you look for? How would certain findings affect your
conceptualization of the phenomenon? How might certain findings affect theorizing, future research, practice, and social policy?