Final Grades!

Grades have been posted. While I no longer hold office hours, I am on campus on and off throughout the summer. You're always welcome to stop by my office to speak to me about the course or grad school or other future academic plans (I just don't have a set schedule). I'm also available more regularly in the fall.

It was a great semester, enjoy your summer, and I hope to see you in future courses.

Chris:-)

Final exam review

Here are some review questions to help you study.  The questions have multiple parts and I’ve indicated point values to help organize a response.  Remember I have extra office hours on Monday and Wednesday.  Good luck studying!

===========

Ch 6—Levels of Measurement

You begin with the concept of “education” and must now operationalize it.  Create questions that measure education at nominal, ordinal, and ratio levels.  Provide response categories. (6 points)  (Hint:  Here’s how to approach this question.  You’re asked to write both questions and responses for each level of measurement.  There are three levels of measurement.)

Ch 7—Sampling

I am interested in the voting behavior of young adults (ages 18-24) in Canada.  I have limited resources, so I’m going to confine my study to a survey of York undergrads in this age group.  However, this still amounts to over 40,000 people!  I can’t survey that many, so I’ve decided to randomly sample 400 students.  What are the population and target population?   What are the sampling ratio and sampling elements?  What is the sampling frame and how might I obtain it?  (6 points)

Ch 8—Survey Research

In tutorial we saw a news article.  It reported that “68% of Torontonians were opposed to forcibly removing the homeless from public places.”  Presumably, what was the question asked?  What’s wrong with how it was phrased?  How would you fix it? (3 points)

Ch 9—Experiments

Ontario has the highest rate of workplace injury.  A group of researchers has developed a training module specifically designed to teach people how to avoid repetitive strain from activities like typing and how to lift heavy items without straining the back.  The module must be tested via experimental design before being adopted by the province.  Two workplace offices have volunteered to participate in a study.  Describe the key elements of your experiment and explain how you can conduct a double-blind study (Hint: The elements include variables, pre and post-tests, etc.)  (8 points)

Ch 10—Content Analysis

Think back to the in-class content analysis of film sequences from Requiem for a Dream and Trainspotting.  What was the unit of observation?  What was the unit of analysis?  Explain how the analysis was “structured” (frequency, space, intensity, or direction).

Ch 10—Content Analysis

Imagine you and I are asked to work on a content analysis of newspaper comics.  The research question is “How is violence portrayed in weekly comic strips?”

1. What is the unit of analysis?  What is the unit of observation? (2 points)

2. Provide an example of a manifest code.  Explain exactly what has to be seen for the comic image to qualify as manifest content. (2 points)

3. Provide an example of a latent code.  Explain exactly what has to be seen for the comic image to qualify as latent content. (2 points)

4. You and I rate the manifest and latent content of a bunch of comics.  Our percent agreement is 90%.  That’s really high!  Or, is it?  Are there any potential problems with the apparent success of our ratings? (2 points)

Key Points: Chapter 15 Analyzing Qualitative Data

Analyzing qualitative data—i.e, coding—has many of you frustrated.  It shouldn’t because it’s really no different than what you’ve already in essays for other courses—namely, present a reasonable argument based on evidence.

By way of quick overview, we have three basic stages of coding: open, axial, and selective.  Keep in mind my “funnel” metaphor from lecture:  each stage of coding is meant to distill or refine the bulk data into a more concise understanding of the phenomenon under study.

Open codes are the first pass through qualitative data where we “boil down” or summarize the information.  Remember we will have LOTS of open codes.  And remember that we open code with reference to your specific research question.

Axial codes are where we begin to make linkages between open codes, either by grouping together like codes or by organizing codes that are related in some pattern.  For example, in the lecture slides we noticed that “loneliness” and “desire for company” were like codes, while “conflict” and “moving out of parents house” were patterned because one led to the other.

Selective coding is where we try to identify a core category around which the bulk of the information will fit.  The goal is to locate a concept or theme that explains a good chunk of what’s going on in the axial codes.  There could be more than one selective code since transcripts cover a lot of material.

Once we’re done coding, we turn to analytic memo writing, which basically means we develop a theoretical explanation.  Now we’re stepping back to reflect upon and interpret what we see in our data thanks to these successive coding stages.  Ideally, we produce some kind of general causal explanation, however rough it may be.  The important point is that the explanation is supported by our coding; it must also make sense in the context of the larger interview data or field notes.

It’s the last part that has many of you especially concerned.  This is why I started this entry by saying that this coding process is much like any essay you’ve ever had to write:  make a logical argument based on evidence that fits the topic.  You’ll be evaluated mostly on your understanding of the process and how well you demonstrate systematically coding qualitative data.  How well you’re able to reflect upon that process and present an analysis is a much smaller part of the evaluation.

Five things to know about Pearson r

The final topic for Chapter 11 is the Pearson r coefficient. You won’t be asked to calculate this statistic, but you will need basic conceptual understanding of it. You’ll need to supplement this blog entry with the textbook because I can’t include diagrams here.

1. The general idea

The Pearson coefficient (often referred to as “r”) is a measure of bivariate correlation. This means it measures the strength of a relationship between two variables. It does NOT measure causation (remember there are three criteria for causation and correlation is only one).

For instance, it seems intuitive enough that there is a positive relationship between the variables annual income and years of education (the more money you make, the more education you likely to have). Therefore we’d expect a Pearson coefficient to indicate a strong relationship between these two variables. By contrast, it’s hard to imagine that there’s a relationship between the variables eye color and income. It doesn’t make sense that the two have anything to do with one another. In this case, we’d expect our Pearson coefficient to indicate either a weak or nonexistent relationship.

Now let’s talk about specifics.

2. The coefficient

The Pearson coefficient ranges from -1 to +1. The closer the value is to -1 or to +1, the stronger is the relationship between variables. Negative and positive values that are close to “0” indicate a weak relationship between variables. You’ll recall that we have two kinds of relationships between variables, negative and positive. Those relationships are reflected in the Pearson coefficient, which is why both -1 and +1 indicate a “strong” relationship.

3. Type of variable

Pearson r can only be used to measure variables at the ratio level of measurement. Nominal and ordinal variables are null and void. The short and sweet of it is that the Pearson coefficient relies on a calculation of the mean. And as you already know, the mean can only be calculated for ratio level variables. So, a red flag should go up if you’re asked to interpret a Pearson coefficient for the variables age and gender. This would be an invalid use of Pearson r because gender is a nominal variable.

4. Type of Relationship

Pearson r can only be used to measure linear relationships. Curvilinear relationships make the Pearson statistic null and void (the curvilinear relationship between the variable may be real, it’s just that the Pearson statistic cannot be used to measure or evaluate it). In lecture, I used “income” as an example of a curvilinear relationship. I said that income over a lifetime is not a straight line; most people make no money as a child, lots of money in their prime, and then minimal income after retirement. If you can imagine plotting that relationship on an x/y graph, you’d have a curve. Another curvilinear relationship is between health and age; the health of children and the elderly tends to be poorer than the health of young and middle-aged adults. Again, you’d have a curved plot on a graph. You may be asking: How do I tell if a variable is linear or curvilinear? The short answer is that you’d actually have to plot it and look for a visible pattern. But don’t worry about that, for our purposes you simply need to know that Pearson r is only appropriate for linear relationships.

5. Interpreting examples

Interpreting a Pearson coefficient is simple as pie. A relationship between variables can be: a) weak, b) moderate, or c) strong. You’ll have to double-check this in the book (I don’t have mine handy) but the guideline is something like 0-.3 is weak, .31-.69 is moderate, and .7-1 is strong (same for negative numbers). So if age and years of education have a Pearson coefficient of .4, you’d conclude that a moderate positive relationship exists (the older you are, the more education you have). If amount of smoking and life expectancy in years have a Pearson coefficient of -.8, you’d conclude a strong negative relationship exists (the more you smoke, the fewer years you live). If income and IQ have a Pearson coefficient of “.1”, you’d say a weak relationship exists (so weak, in fact, you’d probably conclude that no relationship exists).

That’s it. Those are the five key points to know about Pearson r. To close, here’s a quiz to test your understanding. Bring questions on Monday.

1. Interpret a Pearson’s coefficient of .75 for the variables age and number of children.

2. How would you draw a Pearson’s coefficient of “0” on a scatter plot?

3. True or False: A coefficient of .5 is a stronger indicator that your hypothesis is correct than a -.5 coefficient.

4. True or False: A coefficient of -.75 for age and religion indicates a strong negative relationship between age and religion?

5. True or False: For the ratio level variables age and income, a coefficient of .8 means there is a strong causal relationship between age and income.

More on Standard Deviation

The idea of SD is that you want to know how much variation there is in your data around the mean. The smaller the SD, the more closely the data are concentrated in the distribution; the larger the SD, the more widely or loosely distributed are the data in the distribution. In lecture, I referred to the SD as the “average freak.” Freaks are abnormal and by definition are not average and do not conform to the mean. So the standard deviation tells us how far from the mean value the “average freak” is.

Here’s an example. One distribution has a SD of 6 and a different distribution has a SD of 20. A SD of 6 is much smaller than an SD of 20. That means that the data in the first distribution are much closely grouped than the data in the second distribution.

Remember that the mean is simply the average score. For example, the average of 20 and 0 is 10 (twenty plus zero divided by two equals ten). The average of 11 and 9 is also 10 (eleven plus nine equals twenty divided by two equals 10).  But in the second case our data points range from 9 to 11 and are much more closely concentrated around the average (10) then in the first case where our range from 0 to 20. Therefore, the SD in the second case would be a much smaller number because the values are more closely concentrated around the mean.

In-class example:

Today we looked at test scores. I told you that both of my classes have the same average of 70%, but that they have different SDs. Tutorial 1 had a SD of 15% and Tutorial 2 had a SD of 10%. This means that Tutorial 1 scores ranged between 55-85% (or 15% plus and minus 70%). Tutorial 2 scores only ranged between 60-80%. From this we would conclude that test scores in Tutorial 2 are more closely grouped around the mean; therefore, the mean score of 70% is actually more representative of students in Tutorial 2.

Let me help you study...

1.  A research decides she wants to study new immigrants to Canada and their experiences in the Canadian labour force.  From this general topic, there are two ‘concepts’ she has to defined.  What are they?

2. She decides to do a quantitative study of the above topic. After she has decided on her concepts, WHAT does she need to do to measure them?

3. The researcher is going to do a survey on the above topic. She needs to construct good survey items. What are some items she could ask people to determine their immigrant status? Include all categories of the variable.

4. The researcher made a variable for immigrant status and found that other researchers thought that it looked like a pretty good way of measuring the concept. What kind of validity does that demonstrate?

5. She also found her variable was pretty close to how another famous researcher was measuring immigrant status. What kind of validity does that demonstrate?

6. The measure also seems to fit pretty close with its conceptual definition. What kind of validity does that demonstrate?

7. The researcher walks around York university and asks random people her survey questions.  What kind of sampling is that?  What is a disadvantage of that kind of sampling?

8. If the researcher did a simple random sample of York university students, she would need a list of all elements of her population, which is known as a _______________.  Where could she get such a list?

9. After getting some surveys completed, the researcher has some data to analyze. She wants to get some measures of central tendency. What measures of central tendency can she obtain for a variable that assesses whether someone is an immigrant or not?

10. She also asked respondents how many hours per week they worked and obtained data on the number of hours they spent in paid employment. What measures of central tendency could she examine there?

11. She found that immigrants had an average of 30 hours per week in paid employment, while non-immigrants has 32. The standard deviation for immigrants was 15, but for non-immigrants it was 2. What does this mean roughly?

Test your understanding: Measures of Central Tendency and Variation

Understanding measures of central tendency and variation (or dispersion) are key to analyzing quantitative data because they enable us to summarize large quantities of data.

Imagine you are asked to describe this data:  0, 1, 2, 3, 4, 5, 6, 7, 8, 9

What can you say about this data?  How it be summarized?  By the end of this post, you’ll be able to answer these questions in several ways.

Measures of Central Tendency: measures of tendency tell us what our values tend to be on average.  There are three: 1) mode, 2) median, and 3) mean.

The mode is the most frequently occurring case.  It is the most general and least precise measure of tendency; it can be used with any level of measurement.  A sample of smokers reveals that 10 smoke Camels, 15 smoke du Maurier, and smoke Dunhill.  The mode is Dunhill (not 20) because it occurs most frequently.  If 10 students scored 60 on the midterm, 15 scored 70, and 5 scored 80, then the mode is 70.

The median is the midpoint in a data set.  Once you have rank ordered the cases, the middle value is the median.  It can only be determined for ordinal and interval/ratio levels (nominal data cannot be rank ordered).  For example, 5 is the median of these data: 3,3,4,4,5,6,6,7,7 (if there were an even number of values, then the median would be the average the middle two).

The mean is the average value.  It is determined by adding up the individual cases and dividing by the total.  5 is also the mean for the above data.  It can only be calculated for interval/ratio data (nominal and ordinal data lack equal distance between categories).

The advantage of the mean is that it’s technically the most accurate estimation of data because it takes into account every value, unlike the median that only represents a single value.  The disadvantage is that the mean is sensitive to outliers (extreme high or low values) and can be misleading, whereas the median is unaffected by outliers.  As a rule of thumb, it is better to rely on the median if you know or suspect outliers in the data.

Measures of tendency are easy enough to figure out, but they’re of limited use because most values vary to some degree from the mode, median, and mean.  For example, the mean of 9 and 11 is 10, but so is the mean of 0 and 20.  Here’s a better visual example:

Data set 1:  1, 1, 2, 5, 8, 9, 9

Data set 2:  2, 4, 4, 5, 6, 6, 7

5 is the mean value of both data sets.  But as you can see, there’s a lot more variation in the first set than in the second and so the mean actually better reflects the second data set.  These data sets are small and easy to grasp visually, but imagine if you were dealing with thousands of values!  Then it wouldn’t be so easy.  Fortunately we have measures of variation to tell us how spread out are the data.

Measures of Variation:

There are several measures of variation but we’re focusing on three: the range, interquartile range (IQR), and the standard deviation (SD).

The range is easy enough—it’s the span of values listed lowest to highest.  The range for the first data set is 1-9 and the second is 2-7.  It’s a vague measure, but nonetheless gives us some sense that the values in the first data set are more spread out than in the second.

The interquartile range (IQR) tells us how closely the data are dispersed around the median.  There’s a formula for calculating IQR, but the short and sweet of it is this:  First, order the data least to highest, next identify the median, then divide the data into four sections (or quartiles), and finally drop the first and four quartiles (the highest and lowest values).  What remains are the second and third quartiles separated by the median.  The IQR tells us how closely dispersed half our data are around the median.

Consider data set 3:  1, 2, 3, 4, 5, 6, 7, 8

The data are ordered, the median is 4.5.  We drop the first quartile (1, 2) and the fourth quartile (7, 8), and we’re left with an IQR of 3-6.  Now we know our median value is 4.5 and that the middle half of the data are between 3-6.

The standard deviation (SD) tells us how closely the data are dispersed around the mean.  It is the most accurate measure of variation.  There’s a formula for calculating SD that is relatively straightforward but involves several steps.  I’m only going to talk conceptually about SD, so you’ll need to refer to the text for learning how to calculate it.

As with the mean, the benefit of SD is that it incorporates all values into the calculation and therefore is more representative of the entire data set; the disadvantage is that, like the mean, SD is also sensitive to outliers.

Let’s say that our mean value is 20 and that we know a single value in our data, 17.  How close is this value to the mean compared to the rest of the values in the data set?  17 seems close to 20, but if most of our data points are 18 and 19, then 17 is actually far from the mean compared the majority.  Fortunately, SD comes to our rescue!

If SD=5, then we know that on average the values in our data set are within 5 points of the mean.  In this case, we would conclude that a value of 17 is close to the mean of 20.  By contrast, if SD=1 then we know that on average values are within 1 point of the mean.  In this case, we’d conclude that 17 is far from the mean and therefore not representative of most values. 

On a final note, SD is related to the idea of the “normal” or bell curve, where the mean is at the center of the distribution and the SD represents sections of data to either side.  Data are usually located within three SDs above and below the mean.  For example, if SD=5 with a mean of 20, then we know that the first SD is between 15-25, the second SD is between 10-30, and the third is between 5-35.  In this case, a value of 17 would be within 1 SD of the mean whereas a value of 7 would only be within the third SD.  And “outlier” is any value beyond three SDs of the mean.

Chapter 11 Summary

As you know, Wednesday’s lecture was interrupted by a fire drill. Ugh, how inconvenient. We must, however, move forward so below I’ve provide a conceptual synopsis of Chapter 11.

============

Up to now we’ve looked at methods of collecting quantitative (numeric) data.  Surveys, experiments, and content analysis are all methods that generate this data.  And we looked at different ways to create measures for this type of data, namely nominal, ordinal, and ratio levels.

Now we’ve come to Chapter 11.  This chapter is about making sense of the numeric data that we collect.  It teaches us how to describe data generally instead of on an individual basis.  We do this all the time in our lives.  For example, suppose you’re asked to describe how good of a student you are.  You wouldn’t list off every single grade you’ve ever received in school.  That would be ridiculous.  Instead, you’d give some kind of summary statement, like describing yourself as an “A” or “B” student, or by giving your overall GPA.  This idea of descriptive summary of data is the focus of Chapter 11.

First, think of data as a group of information about something.  If I wanted to estimate the age of 2030 students, then I’d ask a bunch of people, not just one.  And that bunch of people that I asked would provide me with a pool of information to work with, or a “dataset.”  Once we have data, then we have to make sense of it.

Second, groups of data can be analyzed individually or they can be compared to other groups.  Univariate analysis examines a single group of data that are represented as one variable.  For instance, age is a variable that we can analyze all by itself (e.g., the average age of students in Soci 2030).  Or, we can use bivariate analysis to compare age with another variable, like gender (e.g., the average age of women in 2030 compared to the average of men).  In this post, I’m going to deal only with univariate analysis.

Third, we talk about data two ways:  1) measures of central tendency and 2) measures of variation (or dispersion).  Central tendency is simply what data points “tend” to be, or their average.  For example, the average age of students in 2030 is 21.  However, that’s just a tendency; in fact, many students are not 21.  Measures of dispersion tell us how closely most of the cases in our data are to the average.  For example, most students in 2030 are within 1 year plus or minus of 21.  This gives us two related pieces of information: we know that on average 2030 students are 21 years old and we know that most of those who are not 21 are still between 20-22.  In this example, the average and the dispersion are pretty close.  But this is not always the case.

Here’s a different example.  Consider the age difference between students registered at York and students at Atkinson.  York is set up on the traditional university model and therefore students tend to be quite young.  York students are usually fresh out of high school, live at home or in residence, receive partial or full financial support from parents, and have limited professional work experience.  Not surprisingly, York students tend to be about 20 years old on average, give or take two years.  There’s not a lot of variation in age.

Now contrast York students with Atkinson students.  Atkinson was originally set up to accommodate mature students.  Mature students are older on average for a variety of reasons.  Maybe someone worked for a few years after high school and then decided to go to university.  Or, maybe they worked for 20 years, realized that they couldn’t get promoted further without more education, and are now attending night school to earn extra credits.  Or, maybe they worked their whole lives, retired, and decided to go to school and earn that degree they always dreamed of.  In short, Atkinson students tend to be older and are much more diverse in age.  The average age is 35, with most between 25-45 years.  Compared to York students, Atkinson students have a higher average age, plus there is also more variation from that average (plus or minus 10 years).

In short, central tendency gives us a general idea about what our data tend to be, while measures of dispersion help us understand how much data vary from that tendency.

Fourth, now that we have the basics, we need to identify the different measures of central tendency and variation.  Measures of central tendency are the mode, median, and mean.  Measures of variation are the range, percentiles, and standard deviation.  The level of measurement of a variable (nominal, ordinal, ratio) determines which measure of central tendency or variation can be used.

In tutorial we collected data on the kind of drugs people used over the weekend.  This was a nominal level of measurement and therefore we could only identify a mode, or most commonly occurring value. Next, we looked at data on 15 quiz scores.  This was a ratio level measurement and therefore we were able to identify a mode, median and mean.

Measures of variation require an ordinal or ratio level measure because the data must be ranked before we can describe how it varies (remember nominal data cannot be ranked).  We’re only focusing on the range and standard deviation.  The range is a listing of data from smallest to largest value.  For example, the range in age of 2030 students is 18-28 years old.  This doesn’t tell us much.  We know the age of the youngest and oldest students but we don’t know the variation of ages.  For example, there could be nineteen 18-year olds and one 28-year old or there could be 2 students who are 18, 19, 20…28.  Either way the range would be the same.

The standard deviation (SD) is more precise and tells us how far from the mean that our data vary.  Because the mean can only be calculated using ratio level data, this implies that SD also requires ratio data.  Remember that you will not need to calculate SD.  It is sufficient to simply know that it tells us how varied are our data—the larger the SD, the more variation in the data.  One SD refers to about 2/3 or 66% of our data, two SDs refers to about 95%, and three SDs refers to about 99% of the data.  For example, the mean annual income in a dataset is $50,000/year with an SD of $5,000.  One SD is plus or minus $5,000 of the average value.  This means that approximately 2/3 (or 66%) of people earn between $45-$55,000/year.  Two SDs means that approximately 95% earn between $40-$60K/year, and so forth.

Stay tuned for more!

Key Points: Chapters 9 & 10

First, some house keeping.

1) Monday is review day. The format will be the same as before where you ask questions of one another. I distributed a review sheet, but also bring your own questions (b/c what’s on the review sheet won’t be on the test!).

2) I will hold office hours after Monday’s lecture. If lots of people show then be prepared to share me or limit your questions. If nobody else shows, then I’m all yours.

Now a recap. The recipe for this week was experiments and content analysis with a dab of validity and a pinch of official stats. Delicious!

Regarding experiments. Knowing the components of experimental design is only the beginning. You also need to understanding how experimental research can be compromised by different threats to internal validity. I also sent additional practice questions for internal validity through Moodle (some are tricky, so pay attention). The difference between laboratory and natural experiments was also emphasized in lecture and tutorial. Ask yourself: How do internal and external validity compare between natural and field experiments? Why does this matter to methodologists? In lecture, I also touched on statistical regression (remember the “smart and dumb babies” example?).

Regarding secondary analysis. Here we focused on content analysis, a non-reactive method for systematically analyzing texts. There are different techniques for coding text (frequency, space, intensity) as well as types of codes (manifest, latent). You had two exercises: one on drug use in film and the other on gender in magazines. There’s no point memorizing specific answers to those exercises because the test will have different examples. Instead, study by applying the process to new examples. For instance, what if you had to conduct a content analysis of “anger” in film? Would you “count” anger using frequency, space, or intensity? What are some examples of manifest vs. latent representations of anger?

Finally, in lecture I talked about the use of existing statistics in secondary analysis. I emphasized the importance of social context in accurately interpreting rates and trends. I distributed a similar review exercise on youth homelessness. It asks you to interpret rates and trends and to speculate about social context. Rates refer to instances at a given moment (e.g., rates of homelessness in 1990) while trends refer to changes over time (e.g., trends in homelessness for different groups over many years). It might help to think of them like cross-sectional and longitudinal time dimensions.

Good luck studying. See you Monday.

Test your understanding: Survey Questions

Creating survey questions is trickier than you might think. Here are more examples on survey design. You can also google "survey questions" or "bad surveys" or something along those lines and examples abound! Take some time to work through the examples—practice makes perfect!

http://www.statcan.gc.ca/edu/power-pouvoir/ch2/exer/5214910-eng.htm

http://www.statcan.gc.ca/edu/power-pouvoir/ch2/questionnaires/5214775-eng.htm#a15

Test your understanding: Measurement and Sampling

As promised, here are the quiz questions you worked on in groups. For those absent on Monday, the questions are drawn from Ch 6 & 7 material, and are similar to the kind of test questions that I typically ask (a stronger hint, I cannot think of). I hope they are helpful when studying. Come see me if you still have questions—good luck!

1. Two sociological constructs are income and higher education. Working from a quantitative perspective (as we’ve been doing in class), I want you to first conceptualize the constructs and then operationalize them (4 pts). Specify which construct will be your independent variable and which will be the dependent (1 pt). Now present a conceptual hypothesis linking your constructs (1 pt). Finally, develop a testable empirical hypothesis for the variables (2 pts). See pp. 108-10

2. I want to compare levels of homophobia between people who live in downtown Toronto and those who live in GTA suburbs. I’ve talked with my peers about this research project. They agree with me that homophobia is an irrational fear of gay and lesbian people. The existing literature is even more detailed in specifying that homophobia is an intense disliking of gays and lesbians as opposed to “fear” per se; the literature also discusses acts of overt discrimination toward gays and lesbians. I’ve decided to include all three as indicators in my measure of homophobia. This will be my dependent variable. One existing study successfully used “church attendance” during childhood as a predictor of homophobia; it found a correlation between high church attendance in youth and increased likelihood of eventually disliking gays and lesbians. I’ve decided to use this same measure as one of my independent variables. I’ve also created a related measure, “religiosity,” or how religious people are; I believe it is similar because people who are more religious are likely attend church more often. I predict both will correlate with the dependent variable. I decide to pilot test my measures in a survey of two local political groups. Interestingly, the pilot results showed that political conservatives showed higher levels of homophobia than the political liberals. Which of the four types of measurement validity have been met and which have not (4 pts each)? Explain why (4 pts each). See pp. 114-5

3. I am interested in the voting behavior of adults aged 18-22 in Canada. Because I have limited resources, I’m going to confine my study to York undergrads. However, this still amounts to 10,000 people! I can’t survey that many, so I’ve decided to draw a sample (n = 100). What are the population and target population? What are the sampling ratio and sampling elements? What is the sampling frame and how might I conceivably obtain it? (6 pts) See pp. 141-3

4. Similar to the scenario as above. I am interested in the voting behavior of young adults aged 18-22 in Canada. I have two research questions: 1) what percentage of young adults voted in the last federal election and for whom did they vote, and 2) how did deciding to vote or not to vote emotionally affect them? The first research question is quantitative in nature while the second is qualitative. Due to limited resources, I will confine my study to York undergrads. I want to conduct one probability and one nonprobability sample. For each research question above, explain why it would be better to use a probability or nonprobability sample (2 pts). Now choose an appropriate type of probability sample and describe how it would be applied to one of the two research questions (3 pts). Do the same with a nonprobability sample for the other research question (3 pts). You have many options but NO convenience samples! See pp. 137-55

Test your understanding: Margin of Error

Today in lecture I touched on margin of error. I want to make sure this was clear because it will be on your next exam. Below are two examples of interpreting margin of error.

===============

--Example 1--

According to a recent Globe and Mail poll, 70% of Torontonians would vote Liberal if a federal election were help today.  The poll surveyed 500 people living in Toronto on February 3 with a margin of error of 5 percentage points.  It’s considered accurate 19 times out of 20.

1. Interpret the margin of error.

If given again, the same poll would have results within plus or minus 5 points of 70 percent (or between 65-75%) 19 times out of 20.  Only 1/20 polls would have results outside the 65-75% margin of error.

===============

--Example 2--

OTTAWA—The Conservatives, hit by bad publicity in the wake of Stephen Harper’s second effort to suspend Parliament, have slipped against the Liberals in a new poll released Monday but still maintain a 6 percentage point lead.

The CBC poll taken February 8 of 1095 registered voters showed the Conservatives would maintain their minority government if an election were held now.  Public support for the Conservatives is 43%, with the Liberals 37%, NDP 9%, the Bloc trailing at 7%, and 4% of voters undecided.  The poll has a margin of error of plus or minus four percentage points 95/100.

A similar poll on January 11 showed the Conservatives with 46% support, Liberals 34%, NDP 14% and Bloc 6%.

1.  What is the margin of error?

Plus or minus 4 percentage points 95/100 times.  For every 100 polls of the same question, results for 95 will be within 4 points.  Put another way, only 5/100 (or 1/20) polls will be completely off-the-wall different!

2.  Is there reasonable statistical basis for claiming that favorability for the Conservatives has declined?

No, statistically there is no change.  If you gave this poll 100 times, 95 of those times the percentage of people giving a particular answer would be within 4 percentage points of this poll.  The percentage of people in the latest poll who say they support the Conservatives is within 4 percentage points of those who said they supported them in the previous poll (43-46%) and the same goes for the Liberals.  Therefore the Liberals have made up no measurable ground on the Conservatives.

3.  What conclusions can be drawn?

There results are more or less consistent.  If anything, support for the NDP has measurably dropped.

Key Points: Chapters 7 & 8

As we enter into reading week, I want to provide an overview of what we’ve discussed so far. In particular, I want to make sure it’s clear how this material builds on what we learned about measurement in Chapter 6.

Chapter 7 is about probability and non-probability sampling. Remember that probability sampling techniques are associated with quantitative research, while non-prob techniques are associated with qualitative. In lecture, I also discussed margin of error, which has to do with how accurate we believe our probability sample to be. I distributed a handout with sampling exercises on one side and margin of error exercises on the other. Finally, we discussed recent research on oral sex as an example of the “science of the sophomore” (those who were absent really missed out!).

Chapter 8 is about surveys. We looked at a poll on attitudes toward the homeless and identified problems with question wording. Lots more examples can be found online if you want additional practice. While we focused on identifying and fixing “bad questions,” there is additional Ch 8 material that you’re responsible for knowing (e.g., types of surveys, pros and cons).

We’ll talk about experiments when we reconvene. For now, enjoy reading week.

More on measurement!

In lecture, I stressed that some variables can be measured with more precision than others—hence there are “levels” of measurement.  And the level(s) at which we can measure depends on the nature of the variable—i.e., how it has been operationalized (this is important later!)

There are two types of variables (discrete & continuous) and four levels of measurement (nominal, ordinal, interval & ratio).  Discrete variables are fixed and descriptive while continuous variables have infinite attributes.  Each type of variable contains two levels of measurement.  Discrete variables are nominal or ordinal while continuous variables are interval or ratio.

Nominal measures contain attributes or “qualities” that are either present or they are not.  There is no inherent hierarchy for the attributes.  For example, sex/gender is a nominal measure with two attributes: male and female.  The attributes are not ranked or privileged in any way; they’re simply qualities that you either are or are not.

Ordinal measures are ranked according to some characteristic.  For example, “happiness” has attributes that can be ranked from most to least happy: ‘very happy’, ‘sort of happy’, ‘not too happy’, and ‘the world sucks’.  It’s not a very precise ranking, but it is nonetheless a ranking.  She who is ‘sort of happy’ is clearly having a better day than he who thinks ‘the world sucks’.

The shortcoming with ordinal measures is that we don’t know the exact distance between the levels.  For example, the difference between ‘very happy’ and ‘sort of happy’ the same as the distance between ‘not too happy’ and ‘the world sucks’?  This brings us to continuous variables as the distance or difference between attributes is fixed and can be precisely measured.

Interval measures are ranked but also tell us how far apart the attributes are on some scale.  Temperature and IQ scores are common examples.  The distance between an IQ score of 100 and 101 is the same as the distance between a score of 115 and 116.  Likewise, the distance between a temperature of 50 and 60 Fahrenheit is the same the distance between 90-100F.  The key point is that interval measures have no meaningful “zero”—an IQ score of 0 doesn’t mean you have zero intelligence nor does 0 degrees F mean you have no warmth.  Interval measures are uncommon in sociology though one example you might come across is time measured in years.  For example, the distance between the years 1980-85 is the same as the distance between 1995-2000.

Ratio measures can be ranked, the difference between attributes is fixed, and they have a meaningful zero.  You can be 0 years old and the difference between 10-12 years is the same as the distance between 20-22 years.  Income is another popular ratio measure.

Finally, I has stressed the difference between measurement precision and measurement detail.  Nominal level measures are considered the least detailed while ratio measures are considered the most detailed.  However, that doesn’t mean that ratio measures are necessarily more precise.  For example, it doesn’t make sense to say that knowing someone is male is more precise than knowing that his annual salary is $15K.  These are different variables.  Precision and detail become more meaningful when a single variable is measured at different levels.  For example, an ordinal measure of annual income is “a little” while a ratio measure is $15k.  The latter is certainly more precise than the former.

Consider another example—height.  Ordinal measures of height are: very tall, tall, medium, short, and very short.  Ratio level measures are 84 inches, 83 inches, 82 inches, etc.  The second measure is much more precise.  Ideally, we’d always like to have such precision but it in reality this is not always feasible.  More important is that we understand different levels of measurement so that we can better work with the data we are presented.  And that’s the moral of this posting!

Key Points: Chapter 6 Measurement

I’m going to talk about two key facets of measurement: 1) conceptualization and operationalization, and 2) validity and reliability.  I cannot stress how important it is that these concepts are clear in your heads.

One question had to do with the difference between conceptualization and a conceptual definition. Conceptualization is the process of developing or “fleshing out” a theoretical construct by giving it a working definition. The book uses the example of “prejudice” (108-9). The discussion is thorough so I won’t repeat it here, but it may help to think of conceptualization as a process that culminates with a conceptual definition. This “process,” incidentally, doesn’t mean a haphazard personal definition, but rather doing one’s homework by consulting multiple sources to produce an informed conceptual definition of a construct that other people can clearly understand.

In turn, the conceptual definition informs the next step of the process, operationalization. When operationalizing, we determine an appropriate method to use in measuring our original construct. Depending on how specific our conceptual definition and what we want to know, a researcher could use a survey or field observation or personal interviews or any number of methods to measure a construct.

Another question had to do with the difference between conceptual and empirical hypotheses. A conceptual hypothesis is when a researcher surmises that a relationship exists between variables, whereas an empirical hypothesis is a definite claim about how variables are related or influence one another. In the conceptual stage we think through options of what variables mean and how they are related, while in the empirical stage we assert a definite claim about how variables interact.

Think of it like betting on horses at the racetrack (something I’m sure all of you do regularly). First, you would conceptually review the options: “3-Legged Nag” doesn’t sound very promising but “Thunderbolt” just screams of a big-money-winner. Next you observe a few races and indeed find that Nag looses every time while Thunderbolt consistently places in the top three. Eventually you put money down on the horse you think will win based on what you’ve studied and reflected on. You’ve gone from a conceptual process to an empirical venture.

I recommend rereading page 111, paragraph 1 as it walks you through the conceptual-empirical process. Reread that paragraph and then actually map out the stages on a piece of paper. Yes, I’m serious—drawing diagrams is a great way to learn this stuff!

One last question was about why internal consistency or reliability matters. I guess the most basic answer is that it only matters if you value reliability. Given the complexity of social phenomena, we want our measures to be as reliable or dependable as possible. The text notes that we improve reliability by clearly defining constructs, using precise levels of measurement, using multiple indicators, and by pilot testing.

Specifically, multiple indicators enable us to measure a construct in different ways. Returning to the text’s example, prejudice does not exist in people’s attitudes and actions in a single way. Rather, it is manifest in different feelings and behaviors. It is therefore much more informative if we can measure multiple facets of prejudice such as attitude, popular belief, ideology, and behavior. In developing multiple indicators, we increase reliability by measuring more content. It also helps us root out weaker measures. For example, say that 3 of 4 measures are highly correlated while 1 is not. It is likely, then, that the lone measure is either a bad indicator of the construct or that it is somehow erroneous (like maybe it’s ambiguously worded which leads people to respond erratically instead of reliably).

Test 1 Review

Here are some review question to help focus your studying.  Bring questions to class.  Good luck.

1.  From lecture, what distinguishes a research topic from a research question? A research question from a testable hypothesis?  (Hint: Consider the function of variables)

2. What is meant by linear and nonlinear paths of research?  What this means for qualitative and quantitative research questions?

3. From the text, why are variables important in quantitative research? Why might they problematic for qualitative research? From lecture, what is the difference between the independent and dependent variables? From tutorial, what’s a trick for distinguishing between them?

4. In you own words, why doesn’t qualitative research typically begin with a hypothesis?

5. According to lecture, most errors in reason are a result of what?

6. Understand the breakdown of a hypothesis statement. For example: Smoking one or more packs of cigarettes per week increases risk of lung cancer by 20%.  Answer the following questions.

• What are the independent and dependent variables?
• What kind of relationship is implied?
• What would be the null hypothesis?
• What is the research question?
• From the text and lecture, why are hypotheses never proven?
• In your own words, what is the "process of a hypothesis" over time? How might this process apply to the smoking example?

7. What is the difference between the ecological fallacy and the error of reductionism?  Consider the difference between the follow statements.

• I’ve seen an awful lot of people in California driving Hybrid cars. As a result, the California state government is going to give tax breaks to people who purchase Hybrids.
• Hybrids are now the top selling car in California.  Chris is from California, therefore he’s probably going to purchase one too.

8. What’s the difference between an ontological position and an epistemological one?  Why does it matter for methodology?

9. What are the different characteristics associated with qualitative and quantitative types of research?  Although these dichotomies are more flexible in actual research, why are they a useful way to learn about research types?

Key Points: Chapters 3 Ethics

This week we had an ethics marathon.  We discussed many examples including Milgram, Zimbardo, Humphreys, Tuskegee, Olivieri, and Chandra.

For the exam, you’ll need to know about each case discussed in the text and during lecture.  While memorizing every detail is unnecessary, you should be able to recognize the details of a specific case even if you’re not given a name like Milgram or Zimbardo.  For each case you should be able to identify the key ethics issue and explain why there was a violation.  And if you want to go hog wild, try locating the different cases in the typology of legal and moral action (p.46).

Remember that there are different kinds of ethics violations.  On the one hand is scientific misconduct—i.e., types of cheating like fraud and plagiarism.  On the other are issues involving research participants that occur when people are harmed.

Also remember that there are different types of harm that human subjects can experience in research.  The harm experienced by subjects in the Milgram study is different than that incurred by subjects in the Tuskegee case.  You will need to be specific on the test.

And finally, realize that we don’t show the films for entertainment value alone.  We expect you to have viewed and to understand them.  The films demonstrate course themes.  You should be able to identify which theme(s) and then explain how it is illustrated in the film.

More Practice: Spuriousness

During lecture we discussed spuriousness. Here are more examples. Take some time to think through them...they would make excellent test questions (hint, hint)

Based on the criteria we discussed, are the following examples cases of correlation or causation? If correlation, can you identify spurious influences?

Higher beer prices 'cut gonorrhoea rates' http://news.bbc.co.uk/2/hi/health/729298.stm

Video games 'increase aggression' http://news.bbc.co.uk/2/hi/health/720707.stm

Or, ponder this: Dr. John Harvey Kellogg (co-inventor of the breakfast cereal Corn Flakes in the late 1800s) warned against the dangers of self-abuse including smoking, drinking, non-procreative sex, and (*gasp*) masturbation! In fact, he reasoned that masturbation caused acne. (Some sound reasoning, eh?) Might this be a spurious relationship? Can you think of another variable that might explain the apparent relationship between pimples and self-gratification?

Refresher: Quantitative Research

On Wednesday, I reviewed the relationship between theory and research. I explained that there are two basic types of social research: quantitative and qualitative. Each is associated with its own ontology, epistemology and direction of theorizing. Several students came to office hours with questions that I want to share for everyone's benefit. Here's the short and sweet of it:

* Quantitative research is associated with realism, positivism, and deduction.

* Qualitative research is associated with the constructivism, interpretivism, and induction.

* We worked through an example of quantitative research.  We tested our theory of political ideology by surveying 1000 YU undergrads.  Everyone we surveyed was asked the same question and given the same response options. Each tutorial established a different “threshold” to confirm or reject the hypothesis. If the threshold was met and the hypothesis was confirmed, then we concluded that our theory was supported; however, if the threshold was not met and we rejected the hypothesis, then we concluded that the theory was not supported.

* In sum, we began with a theory about neoconservatism and tried to test it with a hypothesis about voting behavior (deductive). We assumed that our definition of neoconservatism was shared by our subjects and would be consistent with their voting behavior (realism). All steps of the research were predetermined, followed the scientific method, and the process was objective (positivism). We surveyed 1000 people and collected numeric data (quantitative).

* This example is meant to illustrate how quantitative research reflects the particular concepts associated with it. Similarly, you should understand how qualitative research is reflected in its associated concepts.

Key Points: Quantitative vs. Qualitative research

Last week I reviewed Ch 2 material on the relationship between theory and research.  I began by explaining that there were two types of social research: quantitative and qualitative.  Each is associated with its own ontology, epistemology and direction of theorizing.

Quantitative research is associated with realism, positivism, and deduction.

Qualitative research is associated with the constructivism, interpretivism, and induction.

We worked through an example of each.  In the first example, we tested our theory of political ideology by surveying 1000 YU undergrads.  Everyone was asked the same question and given the same response options.  In both tutorials, we had to reject our hypothesis because the survey results were below the minimum threshold.  Our data did not support the theory we were testing.  All steps were predetermined and the research was objective and could be easily replicated with similar results. 

In the second example, I was running for office and needed to develop and idea of what issues were important to you as voters.  I personally interviewed each of you.  We chatted for a bit a what issue was important to you, why it was important, and what kinds of things I could do if elected to help address your concerns.  My assistant took interview notes and then as a class we looked at all the data and tried to develop a broader understanding of what most voters wanted.  Every interview was different and raised different issues, explanations, and solutions.  I learned a lot more about what was important to all of you, but keep in mind that another group of students might have voiced entirely different issues and solutions.

Hopefully these examples helped clarify the relationship between theory and research.  And hopefully it’s clear why the different examples represented specific ontological and epistemological assumptions and directions of theorizing.  Come see me in office hours is you still have questions.

On a separate note, thank you for humoring me through the examples.  I was elated with the level of participation by everyone last week!

See you Monday.

Key Points: Chapter 2 Approaches to Research

Here’s more discussion of chapter 2.  Normally I don’t go into such detail but time was short on Wednesday, so I’m spoiling you this week!

First we said that ontology concerns the nature of reality. Is there a concrete social reality that we all encounter or do we each experience life as a series of unique, subjective events? We also focused on two extreme positions: 1) objectivism or realism, which maintains that we do share a common experience of reality, and 2) contructivism, which contends that our experiences and understandings of the world are completely unique to us and only us.

Next we said that epistemology is the study of knowledge. How can we best learn about something social? We focused on two extreme epistemological stances: 1) positivism, which advocates that we can study and learn through the objective application of scientific method, and 2) interpretivism, which asserts that social life is too complex to study methodically; instead we must immerse ourselves in research and do research on a case-by-case basis.

Okay, that’s the theory. Now let’s apply these general concepts to the specific experience of illness (or racism, homophobia, etc). Some 70,000 people die of cancer every year in Canada. Length of survival varies, as do types of treatment and care. There are many kinds of cancer with different causes. Is the experience of this illness the same or different for everyone?

When the text says that the ontological and epistemological stances of the researcher matter, this means that what we believe influences how we research issues.

1. An objectivist would argue: Although there is no single type of cancer or treatment, and although the disease attacks people’s bodies differently, as human beings we still share common experiences of pain, pleasure, sadness, and happiness. Therefore, people with cancer likely share many common experiences.

2. A constructivist would argue: Not only do types and treatments of cancer vary, but so do the ways that people ail and cope with the disease. Therefore the experience of this illness depends solely on the individual.

3. A positivist would argue: We can accurately study and learn about collective experience of having cancer is by asking many people systematic questions. If lots of people tell us the same (or similar) account, then that is likely a pretty accurate knowledge of the experience of this illness.

4. An interpretivist would argue: The way we accurately study and learn about the experience of this illness is to become deeply immersed in people’s lives. General questions only tell us so much; in-depth study of representative cases also yield valuable knowledge.  Only this will enable us to truly understand how people similarly or differently experience this disease.

You’re all entitled to your own epistemological and ontological positions on these. We care simply that you understand the different positions and how they influence the way we methodologically conduct research.

Key Points: Chapter 2 Critical Research Approach

Monday lecture touched on “critical” research. I want to make sure it’s clear.

According to lecture, the critical approach is situated closer the interpretivist perspective than the positivist. And if you think back to Chapter 1, the critical approach is reflective of applied research (in particular action research, pp. 12-13).

Technically speaking the critical approach blends multiple epistemological and ontological perspectives. This really isn’t important since we don’t want you bogged down in lay-philosophical debates. So, the take-away point is that the critical approach views knowledge as power, both as a form of oppression and of empowerment.

Knowledge can be oppressive when it used to negatively impact people’s lives, yet it can be empowering if mobilized to improve social conditions. For example, for a long time many people assumed that homeless people were on the streets because they were lazy, subject to vice, and made poor decisions. This was common wisdom and, as a result, the public had little interest in helping the downtrodden. But once scholars collaborated with social workers to identify and the many structural factors contributing to homelessness, we began to see a greater public willingness to allocate resources to mitigate factors influencing homelessness.

In this example, knowledge was mobilized as a form of activism for the express purpose of improving public welfare. Knowledge was sought not for knowledge sake, but rather in the interests of social reform. This is what the text refers to as praxis. However, implicit in this approach is the assumption of an objective reality—that is, that harmful structural conditions truly exist but can nonetheless be curtailed. There is an implicit assumption of shared experience—i.e., homelessness—that can be remedied to some degree. At the same time, few would argue that the experiences of all homeless people are the same or that people on the streets are all there for the same exact reasons. This means that the critical approach sometimes straddles that ontological border between realism and constructivism.

In short, the critical approach is about producing knowledge for the sake of mobilization and reform. Whether we employ a positivist or interpretivist approach toward knowledge is irrelevant; more important is an understanding that power and use of knowledge to fix social problems.

More Practice: Name that Time Dimension!

For the following scenarios, name if the time dimension is cross-sectional or longitudinal. If it is longitudinal, is it a time series, cohort, panel, or case study?

1. Last week a market research company phones up 2,500 people in the GTA to out if they like Rob Ford.

2. A group of small children all affected by autism are being studied from onset to their teens.

3. Each year, York gives the same student satisfaction survey to a random sample of York students. It has been going on for 10 years.

4. I’m going to do a study of 300 households in TO and administer surveys to the same household every year for the next 15 years.

Come see me in office hours or after class for the answers!

Key Points: Chapter 1

1. Key Course themes

One goal of this course is that students become informed consumers of information because there is a lot of "bad research" out there. Learning research methods is an excellent way to develop a skill set that will help us evaluate information we encounter on a daily basis.

As social researchers, we must be especially skeptical of information that is based solely on claims of authority, tradition, common sense, etc. However, we also need to be skeptical of information that claims to be science-based as even scientific research can be poorly conceived, incorrectly done, or presented in a misleading fashion. Don’t confuse skepticism with cynicism; cynicism implies distrust and negativity while skepticism refers to evaluating evidence instead of blindly accepting claims.

According to the lecture, the scientific method may not be perfect but it is the most accurate and most consistent technique we have for evaluating the physical world. “Positivism” is the position that principles of the scientific method—originally developed for the physical sciences—can be effectively applied to social research.

2. Key Lecture Points

Know the steps of the research process (which we will continue to go over), such as selecting a topic, developing a research question, designing a study, etc.

Understand the significance of the peer review process. Who qualifies as “peers?”

Understand the differences between basic and applied research. How are they differently applied? What are pros and cons for each?

Know the three types of research: exploratory, descriptive, and explanatory. Why did the lecture emphasize explanatory research?

Understand differences between cross-sectional and longitudinal dimensions of time.

Don’t over-think examples of cohort studies.  If you look close enough, I’m sure that you could find some commonality between the research subjects.  But this isn’t the point.  The key to a cohort study is that the common quality or characteristic is directly relevant to the research question.

Finally, a case study is an in-depth analysis of a small group, organization, event, etc.  Don’t confuse it with a panel study.  As the name implies, a case study analyzes a few select cases that serve as exemplars of a larger social phenomenon.  For example, you might study the activities of a political group over time. The members may come and go over the course of the research, but this doesn’t wreck your study because you’re interested in the group as a whole and not the specific members.