We have been taught through our blogs and classroom lectures about the distinguishing features of quantitative and qualitative research.  In addition, we have debated and discussed an important topic in educational research, that is, “are you a pirate and ninja?”  Similarly, the ongoing battle within the community of educational researchers is deciding what type of research design is the best, the traditional quantitative design or modern qualitative design?   I have continuously compared and contrasted these two designs through blogs and our sorts/fill in charts in class.

As I have learned, quantitative and qualitative methods have different assumptions and ways of evaluating data.  For instance, I compared the two different types of research designs to how a researcher would select a toy in my blog 7.  A qualitative researcher would select a kaleidoscope to understand art.  The quantitative would see shapes, colors, and sizes of objects (i.e. multiple perceptive) when collecting data and then come up with a conclusion, model, or framework about the topic of art.  On the other hand, a quantitative researcher would choose a microscope at a specific lens resolution (35 mm).  The researcher would be interested in the objective based on single theory; perhaps, it would be to measure the angles used in a particular art piece.  Then, quantitative research would collect data to support that theory.  In mixed methods, you would be examining a piece of art using both a kaleidoscope and a microscope.

I we have also learned in the class to select the design that best suits the parameters of a particular question of interest.  I realized how difficult it can be to choose research problem/hypotheses and formulate just the right title.  In the 3^rd blog, I asked the following quantitative “phased” question:  What are the effects of two different Algebra II curriculum structures, Virginia Standards of Learning (SOL) verses independent self created curriculum, on the SAT math scores of high school juniors?  My research hypotheses might have been, “Students that are taught under a self created teacher curriculums are associated with having higher math SAT scores than student taught under the SOL based curriculum.”  Later, I discovered that this question does not make sense; it would be like comparing a private school (that have self-created curriculum) to a public school (that are mandated to create curriculum based of the SOLs).  In addition, there were way too many confounding variables (e.g. teachers experience, IQ, previous knowledge in Algebra II by students, SES, etc) and threats to internal validity (e.g. history, maturation) to do this question.  Thus, I changed my research question.

Even more in depth than quantitative and qualitative research design is experimental, quasi-experiment and non-experimental research designs.  We learned in our 5th blog that correlational studies are a type of non-experimental design.  I learned that a correlational relationship tells us possible causes of an education outcomes but not the actual cause itself.  I also wrote about threats to internal validity in experimental designs, quasi-experimental and qualitative designs.  I used my experimental research question:  What are the effects of public non-tangible extrinsic rewards/encouragement (independent variable) on engagement levels of sixth gradient students (dependant variable)?  Then, I discussed the threats to validity concerning the question.  For instance, the threats of maturation on internal validity are increased for sixth grades as they are rapidly growing emotional, physical, and cognitive.  A maturation threat to validity might be the participant’s mood.  Developmentally, sixth graders exhibit changes in mood and energy levels because of their hormones.

In blogs 2 and 9, we spend quite a bit of time discussing the “fruit salad” (i.e. literature review) of our lunch tray.  I think that college students and professors affiliated with a college or university are at a distinct advantage when using conducting an educational literature review.  We can access texts, journals, and search within a computer library databases with just one click of a button.

I have a preference in graduate school to use the main library web site (http://www.library.vcu.edu/) and search for topic within specific peer reviewed journals.  For instance, I submitted a recent research paper on multiculturalism issues and the professor asked to use journals related to counseling that are published within the last 10 years.  So I went to journal tab on the main library web site and search Professional School Counseling and a new window popped up where I could search for a specific article or search within the journal.  I clicked on EBSCOhost Academic Search Complete, then once a new window opened up which provided the title, publisher information, title history, availability of full text (from 1990 to present).  I checked on the “search within the publication” and when the new window appeared I typed key vocabulary.   If the query could not find the topic it would search in related journals such as The School Counselor, Guidance and Counselling, and Journal of Multicultural Counseling & Development for my topic!  The words I used were multicultural education/issues, multiculturalism, student culturally competency, social justice in our schools, and diversity in middles schools.  I actually like search specific journals rather than cross searching because I get more relevant information about school counseling use this method.  Once I compile my most relevant sources, I look in the reference section and saturate my research by look at the most commonly cited research.

In addition to learning the components of our lunch try, as a class we learned about the ethical guidelines of educational research.   There are several ethical principles that educational researchers should abide by when using human participants and conducted research in educational settings.  The reoccurring themes in ethical research which was created and coded by our classroom sample were as follows:  participants rights (i.e. do no physical or mental harm, student assent, parental consent, inform consent, debriefing), qualities of the researcher (e.g. competent and capable, honest, trustworthy), abiding by professional ethical standards (e.g. IRB, AERA, APA, and ACA), and refraining from self interest or gain (e.g. keeping dignity of the study, refraining from manipulating the data).

As mentioned before, one example of ethical practice in educational research is informing the participants of their rights.  It is essential in studies with children and adolescents that they understand fully the conditions, such as the disadvantages and advantages, of a study through informed consent.  There should be no ambiguity and the participant’s right to discontinue in the study at any time should be clear even though in some studies (especially in case studies) this would completely put a halt the plans of the researcher.  In addition to assent, parental consent and debriefing is necessary when working with people under the age of 18.  All of these procedures can prevent harm to participants and save researchers from getting into legal trouble.

• Research doesn’t really talk about his expectations of the study.  His word choice indicates he unbiased in his observations (i.e. uses the word: it seems, appears to, etc.)

• In the abstract (and title), it indicates an interpretive analytic framework

• Because this is a case study, the author gives a detailed description of the teacher but does not include how and why the teacher was selected.  It merely states this teacher is experienced in his profession (14 years) and “midcareer”.

• Author doesn’t really discuss how his presence in the classroom effected Bob’s teaching.
• However, it seems that the Schempp was very involved because observed and interviewed in many capacities (i.e. interviewing Bob and Kathy, observation)

• There is field notes and observations about the five distinct knowledge categories of teaching, they are as follows: class organization and operation, teaching behavior, subject matter, pedagogical content knowledge, and external conditions
• Schempp provides a excellent detail about the climate of the school setting (i.e. small, rural community, education is held in high-esteem, population of student body)
• Merely states that field notes were recorded during and after observations and summarized after each day of data collection

• Does not state his personal training, but describes in detail data analysis procedures based on the work of Glaser & Strauss (1967).  Schempp uses themes and categories as criteria for acquiring pedagogical knowledge.  Then, the use of tentative scheme, and the constant comparison method of analysis.

• Case study methodology
• Discusses external conditions were influences outside of the classroom specifically laws and school board policies
• Describes that a key informant, Kathy, helped enhance credibility of Bob’s stories by cross-checking the stories
• Schempp thoroughly describes the creditability of his study by states that he checked the accuracy of the data (reliability) and validate the findings of the report by reading Bob’s comments and reactions.

• Data analysis and finding selection omits descriptions of what was observed.  There is a separate section of the paper discusses each category of teacher knowledge.  Also,

• Used multiple ethnographic techniques for collecting data including nonparticipant observations, artifact and document analysis, stimulated recall from videotapes, and formal and informal interviews
• Schempp discusses triangulation in the data analysis section: “specific strategies employed to insure data trustworthiness included triangulation of methods, member checks, disconfirming case analysis, and cultivating reactions from the case-study teacher to the themes, categories, and events to be included in the final report”

• Schempp states he was present in the school every day for the first month and made field trips twice a week on average afterwards.  That is about 1 year worth time spend on understanding Bob’s knowledge of teaching.

1.)  Researchers using a qualitative approach believe that there are multiple realities represented in participant point of views, and that context is vital in providing an understanding of the phenomenon being examined.  On the contrary, a quantitative study assumes that there is a single, measurable reality.  I would compare the two different types of research designs to select of child toys.  A qualitative researcher would use clydascope lens to understand art.  So, he would see multiple perceptive when collecting data and then come up with a conclusion, model, or framework about the topic of art.  On the other hand, a quantitative researcher would choose a microscope at a specific lens (35 mm).  The researcher would be interested in the objective based on single theory; perhaps, it would be to measure the angles used in a particular art piece.  Then, quantitative research would collect data to support that theory.

2.)  Qualitative approaches are characterized by the assumption that the researcher’s biases and perspectives must be understood and used in interpreting findings, on the other hand, in quantitative studies a researcher’s bias is a threat to internal validity.  Qualitative researchers are like school psychologist, they believe they must understand how their biases and use it in interpreting their findings in child study evaluation.  Quantitative researchers are like school counselors, they understand that their biases can be a threat a student’s treatment.

3.)  Qualitative researchers do not create hypotheses and gather data to prove or disprove them (deduction).  Instead, they collect the data first and then analyze it inductively to generate generalizations, models, or frameworks.  Qualitative researchers want to be open to new ways of understanding.  They believe that predetermined hypotheses limit what data will be collected and may cause bias.  For example, in the study Causal Beliefs of Public School Students about Success and Failure in Music, Legette used a nonexperimental quantitative study design.  Legette used deduction by using the Attribution theory to form his hypothesis and using data to support the concept that causal attribution influence success or failure in music.  If Legette used a qualitative design, perhaps, he would first interview middle grade students about belief about success and failure to collect data.  Then, Legette would formulate a model or framework based on his coding or themes in his results.

4 technical issues related to credibility in qualitative research design: (pg. 296-300)

Creditability has to do with the trustworthiness and believability of the data, data analysis and conclusions of a study.  Creditability is like a good friend; the person is honest, trustworthy, and listens carefully to you.  You believe in your friend creditability because he/she would never lie to you and he/she is a good judge of character.  But there are technical issues that can come up in your friendship with creditability.

Lack of triangulation (which means using different methods of gathering data) in qualitative research design can decrease the creditability of a study.  For instance, a research assistant forgot to consider the different times of the day when testing engagement of kindergarten school students in her research design.  Thus, the research missed the possibility that students were more engaged in the afternoon because they had lunch in comparison to the morning when they are groggy and miss their mommies.  Another technical that might arise in qualitative studies is the inconsistency in what has been recorded and what actually occurred, in other words, lack of reliability.  For example, if the field notes to do correctly depict the message the participant was trying to get across than the creditability of the study would be compromised.

Because qualitative researchers are often the instruments by which data is gathered and interpreted, it is important for the sake of creditability that the qualitative researcher be subjectivities and aware of potential bias that may affect the results.  Any opinions, biases, or expectation that the researcher has about the outcome can weaken the creditability of the study.  For instance, a researcher might have a pre-conceived notion about a specific racial group and he is unaware of those notions.  He might unintentionally let his notions influence how he codes an interview or how the researcher perceives the actions of his participants.

Lastly, the presence of an observer may change the natural behavior of the participants in a study; this change is called observer effects.  Good observers would blend into the background of the natural setting and draw as little attention as possible to themselves. An example of bad an observer that chronic interrupts an individual counseling session to clarify a message, the observer might affect the rapport and trust that the participant build with the counselor.  Another instance, an observer in group counseling session might make ruckus setting up a videotape and affect their ease to talk during the session.

Threats of maturation on internal validity are increased for sixth grades as they are rapidly growing emotional, physical, and cognitive. A maturation threat to validity might be the participant’s mood. Developmentally, sixth graders exhibit changes in mood and energy levels because of their hormones. For instance, one day a participant’s engagement may increase because their endorphins are up from getting physical exercise in gym class, not because they enjoy encouragement. On another day a participant’s engagement may decrease because their partner dumped them (or they on their menstrual cycle) so no encouragement seems to help the student become engaged. Additionally, sixth graders experience changing peer relations. I wonder how conscious a student might feel if he received constant encouragement from his history teacher, would he feel like he is the “teacher’s pet” in the eyes of his peers.

A correlation can predict the value of one variable from the value of another variable.  Yet again, we cannot say which variable, if any, causes an outcome.  Legette’s found “80% of reasons cited had to do with effort and ability.  In addition, students tended to change their causal attribution with grade level.  As the music student advanced in grade level, there was a gradual shift from effort attributions to ability.”  Thus, Legette can conclude that effort and ability are more likely to be possible causes of success/failure than the variables luck and task difficulty.  However, Legette cannot conclude that student’s beliefs about effort and ability cause success or failure.  Rather Legette states, “The results of this study indicate that student tend to place more importance on ability and effort as causal attributions for success or failure in music.”  Therefore, Legette is not claiming cause; rather, he is telling us that there is a tendency toward one attribution over another.

When considering correlation, there is the chance of confounding variables can interfering with the data and results.  Legette mentions that participants gradually shift their attribution beliefs depending on grade level.  Thus, grade level can be a confound variable in the study.  Similarly, variables such as gender, race, and class environment can all be confounding variables.  Thus, it is nearly impossible pinpoint one cause of success or failure in music.  We can only say there is positive or negative relationship between variables.

Katie’s Personal “Ah-hah” Moment

As a side note, I remember my research methods professor, Dr. Shivy, asked us an interesting question to prove her point about correlation is not causation.  She asked us why there is increased mosquito population during the summer.  As students we assumed it was because more people were outdoors more during the summer.  It seemed like common sense.  But there was a third variable in the mix….beer.  Mosquitoes love the sweet smell of beer.  The mosquitoes were a confounding variable, thus, there is no way to prove that the increased mosquito population in the summer was caused by people enjoying the fresh air.

Probability sampling is a method of sampling in which the participants are selected randomly in such a way that the researcher knows the probability of selecting each member of the population.

Non-probability sampling is one in which the probability of including population elements is unknown.

It was to no surprise that after careful consideration, I thought proportional stratified sample would best represent Virginia’s schools.  I knew right off the back that non-probability samplings were not as accurate as probability samplings because they do not randomize, thus, leave room for bias and error.  Proportional stratified sample seemed like the most representative sample possible that had randomization.  Additionally, the demographics of urban schools are very different from rural and suburban areas because there are more minorities that live in city.  Perhaps, that is an assumption but it seems like a fair one when considering the entire population of Richmond City (both student and adults).  If I was to use a proportional stratified sample, I would pick at least 3 urban school districts and collect data about gender and race of middle school students to make the sample representative.  I do think that race is relevant to motivation because some cultures emphasize the significance of education over others.  I’m not too sure how much gender has an influence on academic achievement, though.  Also SES and parental involvement are also important to proportionally stratify.