POL 850 Spring 2021 Homework 1
This homework is due by 5 PM on Friday, February 19. Please use this R Markdown template to report your
code, ouput, and written answers in a single document. Turn in your homework by uploading the pdf file to
NYU Classes. Make sure to comment your code (using the # key). Report results in the correct units of
measurement. Do not report more than two digits to the right of the decimal point.
Name:
TA:
Exercise 1: Bias in Self-Reported Turnout
Researchers frequently use surveys to measure political behavior such as voter turnout, but some are concerned
about the accuracy of self-reports. In particular, we might worry about possible social desirability bias: people
might tell an interviewer what they think the interviewer wants to hear. In post-election surveys, respondents
who did not vote might lie about not having voted because they may feel that they should have voted. We
will check whether such a bias is present in the American National Election Studies (ANES).
The ANES is a nation-wide survey that has been conducted for every election since 1948. The ANES conducts
face-to-face interviews with a nationally representative sample of adults. The table below displays the names
and descriptions of variables in the turnout.csv data file.
Name Description
year Election year
VEP Voting Eligible Population (in thousands)
VAP Voting Age Population (in thousands)
total Total ballots cast for highest office (in thousands)
felons Total ineligible felons (in thousands)
noncit Total non-citizens (in thousands)
ANES Percentage of ANES respondents who report having voted
Question 1.1
Load the data into R and check the dimensions of the data. How many observations are there? Using the
function head() and the argument n, list all observations. What years are included in the dataframe? What
type is each variable in the dataframe?
Answer 1.1
insert code here
Insert written answer here.
Question 1.2
There are different possible measures of turnout. We will construct two alternative measures of turnout.
First, construct a new variable in the turnout dataframe that is defined as the total number of ballots cast
divided by the voting age population, times 100. Next, construct a new variable in the turnout dataframe
that is defined as the total number of ballots cast divided by the voting eligible population, times 100. Use
the function View() to browse the newly created variables. What difference do you observe across the two
measures of turnout, and why do you think that difference exists?
1
Answer 1.2
insert code here
Insert written answer here.
Question 1.3
Construct a new variable in the turnout dataframe defined as the difference between the voting age population
measure of turnout that you created for Question 2, and the ANES measure of turnout. What is the
average difference between these two turnout measures? Conduct the same comparison for the voting eligible
population measure of turnout that you created for Question 2, and the ANES measure of turnout. Briefly
comment on the results.
Answer 1.3
insert code here
Insert written answer here.
Question 1.4
Presidential elections and midterm elections occur every four years, staggered by two years with respect to
each other. Let’s investigate differences in midterm and presidential elections. First, create a new dataframe
that is a subset of the turnout dataframe, containing only the years in which presidential elections were held
(1980, 1984, 1988, 1992, 1996, 2000, 2004). Then create a new dataframe that contains only the years in
which midterm elections were held (1982, 1986, 1990, 1994, 1998, 2002). Compare the mean VEP turnout
rate (defined as in Question 2) with the ANES turnout rate, separately for presidential elections and midterm
elections. Does the difference between the VEP and ANES turnout rates vary across election types?
Answer 1.4
insert code here
Insert written answer here.
Question 1.5
Now create 3 separate dataframes: one containing turnout data for only the 1980s, one containing turnout
data for only the 1990s and one containing only the data from the 2000s. We can now separately examine the
three decades of the year range of the original turnout.csv dataset. Calculate the mean difference between
the VEP turnout rate (defined as in Question 2) and the ANES turnout rate within each period. Has the
bias of the ANES increased over time?
Answer 1.5
insert code here
Insert written answer here.
Question 1.6
Let’s calculate an adjustment to the VAP turnout rate. First, construct a new variable for adjusted VAP
in the turnout dataframe that is defined as the voting age population, minus the total number of ineligible
felons and non-citizens. Next, construct a new variable in the turnout dataframe defined as total ballots cast,
2
divided by the adjusted VAP, times 100. Finally, construct a new variable in the turnout dataframe defined
as the difference between the adjusted VAP turnout rate and the ANES turnout rate. Compare the mean
differences between the adjusted VAP turnout rate and the ANES turnout rate, the unadjusted VAP turnout
rate and the ANES turnout rate, and the VEP turnout rate and the ANES turnout rate. Briefly discuss the
results.
Answer 1.6
insert code here
Insert written answer here.
Exercise 2: Causality
Question 2.1.A
Do stay-at-home orders help curb the spread of Covid-19? You are interested in estimating the causal effect
of implementing stay-at-home orders on state i’s Covid infection numbers.
What is the treatment variable (Xi)? What is the outcome variable (Yi)?
Answer 2.1.A
Insert written answer here.
Question 2.1.B
According to data collected by the New York Times, 9 U.S. states have currently some form of stay-at-home
orders in place; the other 41 states place no restrictions on people’s movements. The New York Times also
collects data on Covid-19 cases per 1 million people. Figure 1 summarizes the findings.
Taken at face value, these results suggest that implementing a stay-at-home order leads to a higher number
of cases per 1 million residents. What is the problem with this conclusion? Provide an answer by comparing
the likely pre-treatment characteristics of those who are treated vs. those who are not treated.
Answer 2.1.B
Insert written answer here.
Question 2.1.C
To make an causal argument, it is important to think about the factual (Yi) and counterfactual outcomes
(Yi(1) and Yi(0)).
What do Yi(1) and Yi(0) mean in this context? For a state that implemented a stay-at-home order, among
two potential outcomes (Yi(1) and Yi(0)), which one do you actually observe? For a state that did not
implement a stay-at-home order, among two potential outcomes (Yi(1) and Yi(0)), which one do you actually
observe?
3
Answer 2.1.C
Insert written answer here.
Question 2.2.A
Do small classes improve student performance? Many studies with observational data suggest that there is
little or no link between class size and student learning. However, the observed lack of relationship between
class size and student performance should not be taken at face value.
Why? Could there be systematic differences between students in smaller classes and students in regular
classes that could be obscuring a true causal effect?
Answer 2.2.A
Insert written answer here.
Question 2.2.B
You are determined to know the causal relationship between class size and student performance, so you design
an experiment. At NYU, you randomly assign a group of Politics majors into a 10-person (small) POL 850
class and assign the rest of the students into a 100-person POL 850 class (regular). You will measure their
score in the final exam at the end of the semester.
What is the treatment variable (Xi)? What is the outcome variable (Yi)? What do potential outcomes, Yi(1)
and Yi(0), mean in this context?
Answer 2.2.B
Insert written answer here.
Question 2.2.C
For those who are assigned to a 10-person class (small), which potential outcome do you observe? For those
who are assigned to a 100-person class (regular), which potential outcome do you observe?
Answer 2.2.C
Insert written answer here.
Exercise 3: The Mark of a Criminal Record
In this exercise, we analyze the causal effects of a criminal record on the job prospects of white and Black job
applicants. This exercise is based on:
Pager, Devah. (2003). “The Mark of a Criminal Record.” American Journal of Sociology 108(5):937-975.
You are also welcome to watch Professor Pager discuss the design and result here.
To isolate the causal effect of a criminal record for Black and white applicants, Pager ran an audit experiment.
In this type of experiment, researchers present two similar people that differ only according to one trait
thought to be the source of discrimination.
To examine the role of a criminal record, Pager hired a pair of white men and a pair of Black men and
instructed them to apply for existing entry-level jobs in the city of Milwaukee. The men in each pair were
matched on a number of dimensions, including physical appearance and self-presentation. As much as
possible, the only difference between the two was that Pager randomly varied which individual in the pair
would indicate to potential employers that he had a criminal record. Further, each week, the pair alternated
which applicant would present himself as an ex-felon. To determine how criminal records and race influence
4
employment chances, she compared callback rates among applicants with and without a criminal background
and calculated how those callback rates varied by race.
In the data you will use (criminalrecord.csv) nearly all these cases are present, but 4 cases have been
redacted. As a result, your findings may differ slightly from those in the paper. The names and descriptions
of variables are shown below. You may not need to use all of these variables for this activity. We’ve kept these
unnecessary variables in the dataset because it is common to receive a dataset with much more information
than you need.
Name Description
jobid Job ID number
callback 1 if tester received a callback, 0 if the tester did not receive a callback.
black 1 if the tester is Black, 0 if the tester is white.
crimrec 1 if the tester has a criminal record, 0 if the tester does not.
interact 1 if tester interacted with employer during the job application, 0 if
tester does not interact with employer.
city 1 is job is located in the city center, 0 if job is located in the suburbs.
distance Job’s average distance to downtown.
custserv 1 if job is in the costumer service sector, 0 if it is not.
manualskill 1 if job requires manual skills, 0 if it does not.
Question 3.1
Begin by loading the data into R. How many observations are there in the data? In what percentage of
observations is the tester Black? Don’t forget to comment your code.
Answer 3.1
insert code here
Insert written answer here.
Question 3.2
Now we examine the central question of the study. First, calculate the proportions of callbacks for people
without a criminal record who are white and who are Black; then calculate the proportion of callbacks for
felons who are white and felons who are Black. Hint: Consider using the function subset() to create
separate dataframes for those with and without criminal records, and the symbol [] to select observations.
Briefly discuss the results.
Answer 3.2
insert code here
Insert written answer here.
Question 3.3
What is the difference in average callback rates between white and Black testers, among the felons and among
the non-felons? What is the ratio of average callback rates for white testers relative to Black testers, within
each group (felons and non-felons)? What do these results tell us?
5
Answer 3.3
insert code here
Insert written answer here.
Question 3.4
Compare the callback rates of white individuals with a criminal record versus Black individuals without a
criminal record. What do we learn from this comparison?
Answer 3.4
Insert written answer here.
Question 3.5
When carrying out this experiment, Pager made many decisions. For example, she opted to conduct the
research in Milwaukee; she could have done the same experiment in Dallas or Topeka or Princeton. She
ran the study at a specific time: between June and December of 2001. But, she could have also run it at a
different time, say 5 years earlier or 5 years later. Pager decided to hire 23-year-old male college students as
her testers; she could have done the same experiment with 23-year-old female college students or 40-year-old
male high school drop-outs. Further, the criminal record she randomly assigned to her testers was a felony
conviction related to drugs (possession with intent to distribute, cocaine). But, she could have assigned her
testers a felony conviction for assault or tax evasion. Pager was very aware of each of these decisions, and she
discusses them in her paper. Now you should pick one of these decisions described above or another decision
of your choosing. Speculate about how the results of the study might (or might not) change if you were to
conduct the same study but alter this specific decision. This is part of thinking about the external validity of
the study.
Answer 3.5
Insert written answer here.