AD654: Marketing Analytics
Boston University
Assignment III: Classification: Will this Traveler Be Satisfied?
For this assignment, you will use the file euro_hotels.csv, which can be found on our course Blackboard
page.
For Parts I and II of this assignment, you will upload two files into Blackboard: The .ipynb file that you
create in Jupyter Notebook, and an .html file that was generated from your .ipynb file. If you run into
any trouble with submitting the .html file to Blackboard, you can submit it as a PDF instead.
For Part III of this assignment, you will submit either your .twb file (if you used Tableau Desktop) or a
link to your dashboard on the Tableau server (if you used Tableau Public), along a one-paragraph
description of your dashboard.
For any question that asks you to perform some particular task, you just need to show your input and
output in Jupyter Notebook. Tasks will always be written in regular, non-italicized font.
For any question that asks you to include interpretation, write your answer in a Markdown cell in
Jupyter Notebook. Any homework question that needs interpretation will be written in italicized font.
Do not simply write your answer in a code cell as a comment, but use a Markdown cell instead.
Remember to be resourceful! There are many helpful resources available to you, including the video
library, the lecture notes on Blackboard, the Friday facilitator-led sessions, the Zoom office hours
sessions, and the web.
Some content posted here in this assignment will be covered in our next class session.
Part I: Logistic Regression Model:
A. Bring the dataset into your environment, and use the head() function to explore
the variables.
B. Which of the variables here are categorical? Which are numerical?
C. Use the value_counts() function from pandas to learn more about the outcome
variable, satisfaction. Describe your findings -- what are the different outcome
classes here, and how common are each of them in the dataset?
D. The outcome variable in this model will be satisfaction. If satisfaction is not
currently in numeric format, use scikit-learn’s Label Encoder to make this
change. Then, call the head() function again on your dataframe.
a. Comparing the first five rows now, vs. the way they looked when you
originally called the head() function, what changed?
E. For your categorical input variables, do you need to take any steps to convert
them into dummies, in order to build a logistic regression model? Why or why
not?
a. If you answered “yes” to the previous question, dummify your categorical
inputs now, being sure to drop one level as you do.
F. Create a data partition. For your random_state value, use a number based on
either your work, home, or school address, or just a number that you like (For
example, I live at 200 Market St, I work at 1010 Commonwealth Avenue, and my
lucky number is 80, so I could use either 200, 1010, or 80). Assign 40% of your
rows to your test set, and 60% to your training set.
a. How did you pick your seed value?
G. Build a logistic regression model using Python, with the outcome variable
satisfaction. Use the rest of the variables from the dataset as inputs (except ID).
Remember to use only your training data to build this model.
H. Show your model’s coefficients.
a. Which of your numeric variables appear to influence the outcome
variable the most? Which ones have the least impact? Choose any
three variables that seem to matter for this model, and write a sentence
or two for each with your thoughts about their importance. You can
think about your own experience as a traveler as you write this.
b. Now look at the categorical variables and their coefficients. Write a
paragraph with your opinion about the ‘type of travel’ and ‘purpose of
travel’ coefficients shown here.
I. Assess the performance of your model against the test set. Build a confusion
matrix, and answer the following questions about your model. You can use
Python functions to answer any of these questions or you can use your
confusion matrix to determine the answers in a slightly more manual way. The
‘positive’ class in this model is represented by the “1” outcome.
a. What is your model’s accuracy rate?
b. What is your model’s sensitivity rate?
c. What is your model’s specificity rate?
d. What is your model’s precision?
e. What is your model’s balanced accuracy?
J. Compare your model’s accuracy against the training set vs. accuracy against the
test set (just use accuracy only for this).
a. What is the purpose of comparing those two values?
b. In this case, what does the comparison of those values suggest about the
model that you have built?
K. Make up a traveler. Assign this person a value for each predictor variable in this
model, and store the results in a new dataframe. Now, put your person through
this model.
a. What did your model predict -- will this person be satisfied?
b. According to your model, what is the probability that the person will be
satisfied?
L. When using a logistic regression model to make predictions, why is it important
to only use values within the range of the dataset used to build the model?
a. Make a new dataframe, but this time, for the numeric predictor variables,
select some numbers that are outside the range of the dataset. Use your
model to make a prediction for this new dataframe. What do you notice
about the result? (To answer this, don’t simply state the predicted
outcome, but also write 1-2 sentences of explanation for what you see).
Part II: Random Forest Model
M. Read the dataset back into Python. Again, use the Label Encoder to convert the
outcome variable into 0 or 1 format. Dummify the categorical inputs again, but
this time, don’t drop any levels.
N. Re-partition the data, using the same seed value that you used in the previous
part of this assignment.
O. Build a random forest model in Python with your training set. Use the same
input variables, and same output variable, as you used in the logistic regression
model (the only difference here is that the categories should not have any levels
dropped). Use GridSearch CV to help you determine the best
hyperparameter settings for your model.
P. How did your random forest model rank the variables in order of importance,
from highest to lowest? For a random forest model, how can you interpret
feature importance?
Q. Assess the performance of your model against the test set. Build a confusion
matrix to do this. You can use Python functions to answer any of these
questions or you can use your confusion matrix to determine the answers in a
slightly more manual way. The ‘positive’ class in this model is represented by the
“1” outcome.
a. What is your model’s accuracy rate?
b. What is your model’s sensitivity rate?
c. What is your model’s specificity rate?
d. What is your model’s precision?
e. What is your model’s balanced accuracy?
R. Compare your model’s accuracy against the training set vs. your model’s
accuracy against the test set. How different were these results?
S. Use the predict() function with your model to classify the person who you
invented in Step I. Does the model think this person will be satisfied?
T. For this question, no Python code is required -- just use a Markdown cell to
answer. Assume that Lobster Land is thinking about opening an on-site hotel on
its theme park property. Write a 3-5 sentence paragraph that speculates about
why Lobster Land might care about being able to use this model. There is not a
single “correct” answer to this question. Be thoughtful and be creative, and
consider the impact of being able to predict a particular customer’s likely
satisfaction. You can mention a marketing angle, an operations angle, or anything
else that comes to mind.
Part III: Using Tableau to Build a Dashboard:
A. Bring the lobsterland_2020.csv into your Tableau Public environment.
B. Using a tiled layout, build a dashboard that includes any four unique visualizations
of your choice. By “unique” this just means that you should not build two of the
same type of plot. Give a title to each of your four plots.
C. Write a one-paragraph description of your dashboard. Write about the plots
that you made and describe your process. You can do this in any file format, and
upload it with your Assignment 3 submission.
D. Paste a link to your file in the same document that you used to write the
description.
Note: This section is intentionally very open-ended. Each submission will be unique. The
goal here is not to arrive at a single “correct” answer but to have everyone gain some hands-on
experience with building a dashboard in Tableau. The dashboards will not be scored by some
‘beauty contest’ measure -- the key here is to (1) make a good-faith effort to build a dashboard
with four separate types of visualizations, and (2) include a thoughtful narrative paragraph.
Every answer that does those things will receive full credit for this section.
