In her Marketing Analytics class for NYU Shanghai and NYU Stern’s Joint MS in Data Analytics & Business Computing (MSDABC) program, Professor Grace Haaf introduces students to a wide array of models used in business by marketing analytics professionals, from CMOs to data scientists.
Topics such as data acquisition and analysis environments, stakeholder scoping and communication, and product integration feature prominently in Haaf’s course, along with a dedication to skills development. Students learn how to analyze raw data sets for concrete business insights, and then translate them into risks and rewards that decision makers can understand. As she explains, “a lot of advanced data science programs are very focused on the machine learning aspect of modeling, but not focused on how to communicate the findings to an executive management team or board of directors.”
Haaf’s emphasis on honing students’ communication and presentation skills is inspired by her own academic and industry background. Following the completion of her Ph.D. in Mechanical Engineering from Carnegie Mellon University, where she focused on the intersection of statistical modeling, market-driven product design, and business, Haaf worked for over a decade in analytics program strategy and implementation at startups and in consulting. At McKinsey, for example, Haaf used advanced mathematical techniques to build an optimization engine for a major US railroad to improve efficiency of asset scheduling, netting a $120 million cost reduction. Regardless of the business context and whether she was looking at transportation or cybersecurity or digital marketing, she viewed her role as a blend of “high-end math with on-the-ground implementation.”
Marketing Analytics reflects this junction between math and the practicalities of business. Group projects recur throughout, each built around a different analytical model, such as A/B testing, churn, customer channel attribution, customer segmentation, lift analysis, and platform advertising spend allocation.
For the course’s final project, each student puts these models into practice in a simulated business scenario. Much like they might experience in a future job, the scenario imagines students meeting an accelerated timeline with realistic stages: from midpoint “working meetings'' to conduct due diligence with their client, to follow-up questioning and evaluation, to a final presentation by each student at a mock “board meeting” where business insights and recommendations are unveiled.
Student sharing their course projects in class.
“Some classes focus only on how to build a more efficient algorithm, but don’t challenge students to think about how they can quantify that value to the rest of the business,” Haaf states. “My goal is to give them the opportunity to have those types of conversations with management teams and role play.”
During the Fall 2020 semester, students were asked to determine a more efficient or lean marketing strategy to increase revenue and reduce churn rate among subscribers to a fictitious over-the-top (OTT) media service named Mevod (Middle East Video on Demand). While they had all been given the same prompt and data set, students tackled the assignment differently based on their chosen analytical models, their understanding of the “problem,” and their own creativity.
Song Qinyan, MSDABC ‘21, utilized integrated cluster and churn models and marginal customer acquisition cost (CAC) calculations to increase joint accuracy of all the behavior models' predictions and to suggest actions to reduce churn and increase revenue. Through her data experimentation, Song explored techniques to handle missing data and discovered Mevod can generate the highest lift rate if they focus on reducing the churn rate of the users who are in the free trial period.
Reflecting on how Haaf’s course has benefited her, Song says she has improved her ability to work with large quantities of data. She remarks, “this course emphasizes a lot on our insights about business problems instead of just coding skills. I think we can learn a lot from each other regarding different perspectives and data process logics.” Following graduation, Song will join Meituan in a full-time position, where she is sure that the ability to analyze complex data and communicate with clients will prove indispensable.
Yang Yiwei, MSDABC ‘21, ran A/B tests to understand what aspects of the company's pricing strategy had a statistically significant impact on customer behavior and conducted customer segmentation to better understand customer motivations. From these models, he discovered that the OTT market in the Middle East is a female-dominated market, with exclusive and comedy videos providing the best opportunities to attract a greater viewerbase. Yang says, “the useful and practical applications I learned will help me handle massive information and find some great insight,” skills that he’ll undoubtedly use again during his Spring capstone project and later on in his career after graduation.
Haaf was impressed with her students’ ability to meet the challenge of the final assignment head-on. “They did a really good job of adapting to the ambiguity that is present in real contexts,” she explains of one of the assignment’s central features. “Even when given, by design, an imperfect data set that I knew would not lead them to an obvious answer, students fully grasped the problem, being creative and coming up with insights that would help the business.”
Song notes that she was surprised at first by the assignment’s flawed dataset, but once she realized it was intentional on the professor’s part in order to simulate a realistic dynamic between analyst and client, she learned that “it’s okay for us to negotiate with the clients rather than dealing with it all on our own.”
Looking beyond the six weeks of her course, Haaf expresses confidence that her students are set up well for success as future business analysts and leaders, capable of joining any industry. “One day, they’re going to make decisions that will affect large amounts of money and human lives,” Haaf concludes. “I want them to be able to connect the output of their modeling, the math they’re doing, with the impact of those decisions, so they can make more informed decisions.”