More Intelligent Tomorrow: a DataRobot Podcast

Why “Know Thy Data” Is a Rallying Cry in Retail - Danielle Crop

March 11, 2022 DataRobot
More Intelligent Tomorrow: a DataRobot Podcast
Why “Know Thy Data” Is a Rallying Cry in Retail - Danielle Crop
Show Notes Transcript Chapter Markers

Danielle Crop is Chief Data Officer (CDO) at Albertsons Companies, a recent transition from her nine-year term as CDO for American Express. Throughout her career, she has orchestrated big data projects that have exponentially increased customer conversion rates and helped millions of customers make smarter purchasing decisions. 

In this episode of More Intelligent Tomorrow, Ben Taylor talks to Danielle about the role of creative design thinking in data, how ethics can help us avoid unconscious bias, and how data science mitigates retail shrinkage. Danielle shares astute observations about what data scientists and business leaders need to learn from one another, why “Know Thy Data” remains the first law of statistical inference, and why you can never replace human relationships in data science. 

Understanding Business Problems 

85% of AI projects fail today, arguably due to a lack of understanding how an AI project should be driving toward business value. Yet often data science teams are far-removed from business teams. Danielle works to enable her data scientist teams to fully understand how their company makes money and what they can do to help their company make money.

“If you don't understand the problem, you can't solve it. You have to enable that understanding for data scientists. It's about creating an opportunity to learn about the business. I make sure that my teams get into the stores and actually work there. ”

Retail is very complex, both in the way that it makes money and in the way that it needs to be managed. There are a lot of people involved and there is a lot of room for error. Knowing how data is created and used, where it comes from, and how it can be used to optimize value can make or break the success of any AI project. 

“Know Thy Data is the first law of statistical inference. You have to know where your data comes from.” 

Shrink occurs throughout the retail supply chain and illustrates this complexity. Perishables expire, purchases are returned, or items arrive damaged from a vendor: this “entropy of retail” is extremely difficult to forecast and control. Having an intimate understanding of data sources and data gathering techniques has been shown to help control this endemic problem. 

In the end, retail is about people. Data science can be used in retail to help customers make informed, contextual choices at the point of a purchasing decision. For example, there is a lot of data available on the link between what you eat and how long you live. Making that data actionable could change peoples’ lives. 

Listen to this episode of More Intelligent Tomorrow to learn: 

  • How business value thinking can improve AI project outcomes
  • The important distinction between the discovery aspect and the modeling aspect of data 
  • How ethics can tackle the problem of unconscious bias in data science  
  • How data science is used to mitigate the “entropy of retail”
  • How AI can help people make better buying decisions 



Danielle Crop (00:00):
So how do you make sure that your bananas are in the right state for different types of customers? Because not everybody likes the same types of bananas, right? I eat green bananas. I will only eat green bananas. Whereas other people are like, "That's gross. I want to eat only the lightly black spotted bananas because that's when they're sweet." And that's the fascinating thing about it is everybody's tastes are different, and what is fresh for one person is not fresh for another. And so you have to actually look at the types of customers, the types of trips that they do, and what is most important to them in their trips?

AI Voice (00:39):
Welcome to More Intelligent Tomorrow, a podcast about our emerging AI driven world, critical conversations about tomorrow's technology today. On today's episode, Ben Taylor sits down with Danielle Crop, Chief Data Officer at Albertsons Companies.

Ben Taylor (01:03):
Danielle Crop, I am very excited for this interview.

Danielle Crop (01:06):
Thanks, Ben. It's good to be here.

Ben Taylor (01:08):
And I know today you are the Chief Data Officer for Albertsons, but I want to go back at the beginning of your career to understand what brought you to this point.

Danielle Crop (01:17):
So well, it's a long meandering journey, but it starts off with an undergraduate degree in forestry, of all things, quantitative terrestrial ecology.

Ben Taylor (01:27):
And I think for a field that specific, you had to be inspired to go into it. So what was the inspiration that drew you to that?

Danielle Crop (01:36):
So I have always been a big proponent in the environment. My grandfather was actually a forester as well, and so he inspired me and I wanted to go protect the environment and enjoy the environment as part of my career. So that's where I started out in forestry.

Ben Taylor (01:53):
And did you work at all during the summer in the forestry?

Danielle Crop (01:55):
I actually worked mostly in the summer as graphic designer and as office help, etc. I actually didn't do a whole lot of work in the outdoors in the summer. But I did a lot of camping, fishing, hiking, things like that.

Ben Taylor (02:08):
And so what prompted you to continue to move on or move into another... What was the next step after that?

Danielle Crop (02:16):
So after that, at the end of it, I was like, "I want to make better decisions about the environment." So how do you make better decisions? And that led me to data. That led me to statistics. And so with those statistics programs, I decided, "Okay, I can do that. I want to go to grad school." And so spent some time getting some higher in math because obviously you don't get that with a forestry degree, and then went back to school and got a master's degree in statistics so I could be an environmental statistician.

Ben Taylor (02:49):
And during that journey, did you run into certain problems that drew your passion? I think when I think of the types of problems you might have been exposed to, there's the classical predator, prey modeling. Was there anything like that that you ran into where you thought combining data in this field is fascinating?

Danielle Crop (03:06):
Yeah. I really wanted to do studies about endangered species. So that was something that I really wanted to do and sampling and modeling of data from the environment to make better decisions about what areas of the forest to protect, and that was what really drew me to the space.

Ben Taylor (03:27):
And then what happened after that?

Danielle Crop (03:30):
So after getting a master's degree in statistics, my husband was still finishing his degree program in computer science, and so I had to stay put where I was for a little bit. And so I had to find a job, and I couldn't find a job in surveying birds in the forest. So I found a job at a small subprime credit card company in Portland, Oregon, and began my career there. And as I was getting into that data, I found it really fascinating. It was just a lot of data and a lot of things that you could do with it in a lot of ways to optimize. And also, I just had that moment of, "My husband is a computer scientist. I'm probably not going to live in the woods." And that was fine. And so by just having interesting problems to solve, I learned [inaudible 00:04:16]

Ben Taylor (04:19):
So you have a trait that we don't often find in data science, but I celebrate it when I do find this. Take us back to the creative designer. So what were some of the things you were doing? What were some of the focuses you had when you worked as a designer?

Danielle Crop (04:35):
I got into graphic design because I was my college's undergrad yearbook editor. So I did writing and designing and... And then I feel like throughout my entire career, the thought process of design, whether it was in designing actual physical things in graphic design or whether it was designing response for modeling or if it was designing websites and experiences in my previous roles, that design mindset has been very essential, actually, to my entire career path. So I think about when I am designing even data management and data governance, I'm thinking about it from the perspective of, "How does this process work together visually? How are people going to work together?" So I think that it's been part of my career growth all the way along, that creative design type of mindset.

Ben Taylor (05:38):
Yeah. And I love coupling those because I feel like in data and thinking more rational and objectively, there's always is a right and wrong answer. It's left or right. But in the creative space, there's thousands of options. Does it really need to be this or this? And so I love the chaos that comes as you play in the creative space.

Danielle Crop (06:01):
Well, I think that's maybe why I went into statistics, because statistics is not black and white. Statistics is that decision science where depending on what inputs you put in, you get different answers. And so there's creativity in that as well.

Ben Taylor (06:15):
Yeah. Well, you're reminding me of, I was talking to, blanking on the name, but it was an individual who's the head analyst for the EEOC, and they dealt with discovery. So when they would discover a potential class action lawsuit, she was complaining because when the other side would engage the ability for them to do gerrymandering and sub sampling with the data, it seemed like they had very sophisticated ways to tell a different story that they had a very hard time defending against. And so I thought that was pretty surprising to find out that you can lie even more with data if there's millions of dollars at stake and you need to.

Danielle Crop (06:56):
Yeah. It's all about how you design it. It's all about simplistically, what is your outcome that you're trying to model to, and what is the data you put into it? Because if you subset your data down, you can come up with any outcome you want. So this is where ethics comes in. And fortunately, I find that most statisticians, data scientists, analytics people are quite ethical. They really care a lot about coming to the right answers with the data that they have. So that's a really good thing, but there is a lot of ways to lie with statistics. That's a fact.

Ben Taylor (07:35):
And sometimes that can happen by accident, right? Because humans have bias. We have different behaviors that we're maybe unaware of. In HR, I dealt with this where we have unconscious bias. And if you don't know you have it, it's hard to proactively defend against it.

Danielle Crop (07:49):
Yeah, and that's why I love in statistics in grad school, and as you go through the discipline, one of the things they teach you is the difference between that discovery aspect of data and the modeling aspect of data. So you're actually told not to data snoop if you're doing modeling. So if you're doing strategic analytics and if you're generating a hypothesis, don't snoop. Once you've generated the hypothesis, you are not allowed to snoop anymore. You must then test. And that is just a basic fact of statistics. And we are very keen on what's the inference that we can make from this data? And depending on how you design it, once again design, you can make different inferences with your data. But you have to separate these two things, or you're not following the science of data. You're just creating confirmation bias with data.

Ben Taylor (08:45):
Yeah. And I'm laughing listening to you. I love the snoop mention because you know every junior data scientist has that magical epiphany when they have 99% accuracy on some impossible problem, and they're so excited to come tell you because it's a big deal. And you and I, once you've been around the block for a while, you know that if it's too good to be true, we can bet a lot of money that it's not.

Danielle Crop (09:12):
Yes.

Ben Taylor (09:13):
So with model and balance and the different things you run into, everyone's been exposed to that where they're looking at a 99% AUC score. And the first pass doesn't seem obvious that it's wrong.

Danielle Crop (09:28):
I know that if that's the case, you've either manipulated your population or your variables, one of the two.

Ben Taylor (09:36):
So continue on and take us to your time at American Express.

Danielle Crop (09:41):
So at American Express, I started off building models for, and this'll date me as far as my age, for countries that are no longer capitalist countries like Venezuela. So I built credit card approval models, so decisioning models, for some of our countries that we actually partnered with. So it was a consulting job actually, inside of American Express. And I learned a lot within that of building models for countries like Croatia and Switzerland and Belgium, Venezuela. And that was a lot of fun.

Danielle Crop (10:11):
And then after a couple years, moved into fraud and spent seven years of my career in building fraud models, building capabilities. So this is where the path the CDO, I think, started for me, was that I was working in a job that was both about the data and about the capabilities that needed to deliver the outcomes for that data. And then from then on, I had goals that were always about those two things. It was always about the data being together with the capability that could drive the business outcome, and that's been my career ever since. But making decisions about fraud in real time in 2004. So that was some interesting capability builds, and I've been doing that ever since.

Ben Taylor (10:56):
And you've seen firsthand the capability of evolution then going from more traditional models, logistic regression, linear regression, things that are very defensible where regulators like them, we can have a conversation about that, where the industry's had to adopt to newer methods that are more exotic. And I know there was pressure to not adopt things like radiant boosting and other more exotic tree based method. So maybe explain that transition, because it wasn't something that happened overnight. So how did that come about where compliance was able to accept some of these more advanced techniques?

Danielle Crop (11:30):
Well, I think that when I'd started on all this, we were back in the beginning of it, there was still neural net. And we would look at that in comparison to logistic regression, which was more explainable. And what we found was that at least back then, now with big data, it's a different world. So I think that the real transition wasn't necessarily from the type of modeling technique necessarily, it was more of what was possible with big data. And you had to bridge that gap between the explainability and then the scalability. And in financial services at American Express, we were able to make it so that you could use the boosting and still make it explainable to the regulators. And that was key in that, and then we needed to be able to make these models that were more sustainable across time, and actually did machine learning. But they were still explainable to the regulators. But it took time to get there from the traditional models. But it's certainly possible. And I think that with ethical AI coming into other areas beyond financial services, you're going to have the same transition.

Ben Taylor (12:41):
Yeah. Well, I feel like there's a lot of industries that are highly regulated that have to go through these growing pains. Healthcare and a lot of different industries deal with this, especially when you're dealing with people, whether it's hiring or approving loans.

Danielle Crop (12:55):
Yes, definitely. Whenever you're taking any sort of adverse action against the individual, you have to explain it.

Ben Taylor (13:01):
Yeah. I'd love to lean into fraud a little bit because I'm sure we have a lot of listeners that have been personally impacted, where they're suddenly on a vacation in Tijuana and they're swiping a credit card and they're doing a purchase that even they know looks really bad because the size of the transaction, the frequency. And so how do you find that magical false positive, false-

Danielle Crop (13:26):
In fraud, you spend a lot of time looking at cases. So this is where fraud is a little bit of the snooping variety of statistics. So you have to look at what's happening in the fraud and then you build your decision against that so that you reduce the false positives to as much as possible. So in that, let's just take that specific case. you're in Tijuana and you're doing a transaction and it'd be unlikely that this is something you would do. This is something that's out of pattern for you. Then we compare it to all the other transactions that are happening in Tijuana and we say, "Oh, this one's not as risky will gain as these other ones." And that's how you decrease the false positives is you look at other factors beyond the fact that you are in Tijuana. And then there's also, of course, a lot of aspects of trying to get more data from customers, trying to get customers to tell us when they're going on vacation, which can then reduce the positives.

Ben Taylor (14:23):
Yeah. And I was thinking about that as well. What other information can you pull in, whether it's the merchant or the transaction data, what are they actually buying? Do they have an app on their phone? Can you confirm that they are there? And what time of day is it? You probably have a lot of other data items that continue to grow to give you more confidence. So I imagine you probably saw that firsthand, your fraud prediction models improved over time, and it wasn't just the modeling capabilities, it was your data quality as well.

Danielle Crop (14:52):
It was the data. Yeah, I would say that there's probably no area of data science I've ever worked in that is more data hungry than fraud.

Ben Taylor (15:02):
Yeah. I imagine with a big organization like that, when you're analyzing fraud, if you look at the top 1% or the one in 100,000 fraud use case, they have to be fascinating, they have to be crazy. Did you run into some of these anomalies where you guys were reviewing them and you're wondering, "Well, why didn't our algorithms catch this?" There has to be a lot of epiphanies, right? Where you look at something and that inspires a new feature?

Danielle Crop (15:31):
Yep. That was one of the funnest parts. Because we do case reviews regularly, once, twice, three times a week, depending on what was going on. And we'd go through the cases and we'd look at everything that the fraudster did, and from all of the different data sources, we'd look at it and we'd see, "What was it that they were doing that the average person would just not do?" And then you'd find that and you'd model around it. And it was always a lot of fun to see the crazy behavior, and how smart they are. They are incredibly smart. It's like having an adversary. There was no other area I've ever worked in where it was so much like there was an adversary on the other side.

Ben Taylor (16:17):
That's got to add some excitement to it.

Danielle Crop (16:20):
Yeah, it was definitely exciting. It was a lot of fun at times too, but it was also very challenging because it's fast, very fast moving. So you always have to be on top of it.

Ben Taylor (16:36):
So Danielle, I have a terrible fraud story to tell you that happened to a friend of mine at a startup in Chicago. And I'd love for you to say that, "Oh yeah, we would've completely flagged that." So his credit card was stolen, and this individual goes into a Walmart, a Best Buy, and a Target, and at each stop buys 10 Xboxes. And I'm almost more mad at the humans at those stores than the algorithms. Because if you're checking out with 10 Xboxes, I'm going to call the cops. But this individual was able to get away with this unbelievable buy, and it happened within 24 hours before they froze the credit cards. And I couldn't believe that humans... And this reminds me of this augmented intelligence concept or collaborative AI. How do we work together? And so what's your reaction to a fraud use case like that? It happened three years ago. Is that even possible for something that moronic to hit the system? And algorithms must be furious with that.

Danielle Crop (17:41):
It's more than possible.

Ben Taylor (17:42):
Oh yeah?

Danielle Crop (17:42):
I'm kind of shocked that it wasn't flagged, quite honestly. So out pattern, for one credit card to be doing that across multiple merchants in a short period of time, that should have been immediately flagged. Not just flagged, but probably the second or third store, declined. So I'm confused as to why that did happen for him. But fortunately, he was not responsible for any of the charges. But sadly, we have a bunch more money in the hands of not so great people.

Ben Taylor (18:17):
And you mentioned earlier how much fun it was when you're dealing with criminals on the other side where it's almost like you're in a movie. You've got these criminals, you're trying to outsmart them, and criminals are really smart. When you gain ground, I'm sure they gain ground in ways that surprise you.

Danielle Crop (18:34):
It's very organic is the way I think I would describe it. Well, it's a little bit like the vaccination situation. It learns to get around you and then you have to evolve and adapt and do something new. So it was just fun to be able to say, at the end of the day, coming home, that something you did actually prevented money getting the hands of possible terrorists. There's something really gratifying about that.

Ben Taylor (19:09):
So as your career evolved where you became the Chief Data Officer at American Express, what were some of the lessons that you learned or the top concerns that you had?

Danielle Crop (19:19):
Well, I think from fraud to CDO, it's like there was a lot of things I learned. Big data came in shortly after I left fraud. Some of them were Spring Data instances at American Express. And so building that capability up was pretty exciting. You're going from that in our network business and having responsibility in marketing or the credit card application experiences and growing test and learn program in American Express from something very small to something that saves the company billions of dollars and [inaudible 00:19:55] increased conversion. So that was really fun experiences, and then I got the CDO role, so after all of that.

Danielle Crop (20:01):
And it was the experiences that I had in product and data, so it was about data and analytics and data science and all that, but also the fact that I understood how the capabilities in the company worked and I understood how the data flowed from X to Y. And I built up teams of all the brand new skillsets that the company hadn't had before. So all of that is how I got the nod to be the Chief Data Officer at American Express, was to build those capabilities out. And so I had all those experiences. And as I go into the Chief Data Officer role, I have a risk background. So I understand how to work with regulations. I have a capabilities background and a big data background, so I understand how to manage data lakes and what it takes to do that. So I understand what data scientists need. So all those things is how I ended up being the Chief Data Officer at American Express, but it was a winding journey.

Ben Taylor (21:04):
For people that are more experienced in the industry, it's funny because when you talk about being close to the business and getting to value, it feels like common sense. But it's not. 85% of AI projects fail. A lot of data science silos are so far away from the business and they can't communicate. And I feel like you are naturally marching closer and closer to the business to understanding value and understanding attribution. And so maybe describe that journey or describe that perspective, because I don't want our audience to miss the significance of that.

Danielle Crop (21:34):
I think maybe because I do have that product management background as well, which is about driving value for the business through capabilities and data to me. But I expect my teams to understand how the company makes money, and to then understand from that what they can do to help the company make money, save money, do things better for our customers.

Ben Taylor (22:01):
Did you ever have data scientists where you felt like they would struggle to understand that, whether their emotional intelligence wasn't sufficient, or did you hope that most of the analytical talent that worked for you would try to make that a priority?

Danielle Crop (22:16):
I think you have to enable that for them. I think that they're all capable of learning it. They're very smart people. They wouldn't have gotten through the degree programs they did if they weren't. So I think it's about having the expectation of them that they're going to understand this and that they can do it, and then also enabling training and opportunities to learn about the business from colleagues in the business. And then also one of the paths I'm on is, COVID has made this a little more challenging, but we're still going to do it, is making sure that my teams get into the stores and actually work in the stores so that they understand how that works and they can see the pain points of people that work in the company.

Danielle Crop (23:00):
I think that makes all the difference, goes back to my experience on case reviews and fraud. If you don't understand the problem, you can't solve it. It's the difference between the kid thinking that the milk in the fridge comes from the store versus the cow. I want my data scientists to know it comes from the cow. So you have to go to the farm and understand it. And so I'm very passionate about where your data comes from. The first law of statistical inference, know thy data.

Ben Taylor (23:29):
What prompted you to jump out of this industry and go join Albertsons? That seems like a massive jump.

Danielle Crop (23:37):
So I was at American Express almost 20 years, did a lot of really great and wonderful things there and made a lot of great friends and still miss it. But it was about learning about a new vertical. It was intentional. So when I'd been approached by recruiters for other CDO roles, when they brought me other financial services roles, I was like, "Nope," intentionally. I wanted to do something in a different vertical. And when this Albertsons opportunity came along with all of the data that they have and the first party relationship they have with their customers and the depth of that relationship that they have, the opportunity to evolve the customer experience in this omnichannel way was really too much to pass up. It's just super fun to be able to be in this space of transforming the way that people get their food, and everybody needs food.

Ben Taylor (24:36):
And I think that's so inspiring because you're intentionally deciding you want to be uncomfortable. So you're intentionally deciding that, "I want to go back to school, in a way." And to lean into that, what elements of Albertsons felt familiar? Where like, "Oh, I've been here before, transaction data." And then what elements felt very foreign where you had to quickly ramp on it?

Danielle Crop (24:58):
So I think that what is challenging about retail is it's very complicated. At least the business model I was from before was very straightforward. You have a few ways you make money. Retail is more complicated, both in the way that it makes money and in the way that it has to be managed. There's a lot more people involved. American Express had 60,000 people, Albertsons has 300 and something thousand people. It's a totally different order of magnitude in how people work together and then how data is created and how it's used. And that's really interesting to try to solve those problems. So I think that all of the aspects of data are the same. So what was common was the fact that data is data is data, and how the cloud, all these things are very common.

Danielle Crop (25:52):
But the aspects that are different are in the way the data comes in and how you need to use it and how you need to optimize it and how do you use it to optimize value? And so I really had to learn about the complications of retail. If you think about supply chain for retail, it's incredibly complicated. If you think about shrink for retail, how we lose money, you can actually have your own customers lose you money, either through theft or through just pick something out the shelf and they put it somewhere else and it gets lost. And just what I call the entropy of retail. There's a lot more places where error can happen.

Ben Taylor (26:36):
I want the audience to appreciate how impossible some of these problems are. Because I think we have a large analytical audience in data science. And I think a lot of people when they hear Albertsons or a retail setting, they say, "Oh, so you're putting the beer by the diapers." And we were talking about this earlier, where that's a nightmare. So if you disorganize the store and you try to optimize based on likely basket, maybe talk for a minute why that won't work? Because I think maybe the naive data scientist who's focusing on the data and forgetting the customer experience, they might be very excited about putting beer by the diapers or some other department jump that they think makes a lot of sense.

Danielle Crop (27:17):
So I think that there's two aspects of this. So I thought about this a little bit more since you and I talked about it. There's the customer side of it, how you organize the store has to be logical to the shopper because if it's illogical to the shoppe,. Then they are not going to come back and shop with you again, and that's a big deal. So if you make it difficult for them to shop in your store, it's a classic design usability problem. So even though you could optimize the sales for one particular assortment, you will then end up decreasing basket size because they can't find what they need in the right amount of time. So you can't optimize in a small narrow window, and then the other side of this is how do our associates stock the shelves? Because there's an experience for them as well.

Danielle Crop (28:06):
How do they know where the stock is in the back room? How do they know where the stock is that they need to put on the shelf? And all these data issues are really very challenging to solve because people are the ones who are putting the stock where it is, and based on dynamics that change every single day. So let's just say that they were expecting to get in a shipment of strawberries, but something happened to the farm and they didn't come. So now they have to figure out how are they going to fill the area of the store for produce? Now, what are we going to substitute in, because we don't want it to look empty? So it's just the entropy, once again, it's just an order of magnitude different, and how do you solve those problems with data? I know we can, but it's going to take a lot to do it.

Ben Taylor (28:55):
I love this topic you're bringing up, because I imagine a brilliant store manager that's very pissy and saying, "If strawberries don't come, it's a disaster." And on the data side, you have all of this data to say, "Actually, they're right." So I imagine there's certain products and different things that if they're not available, if bacon is not available, people will complain. You have other points of escalation beyond a smaller basket size or customer not coming back if you have complaints in different things. So I love this concept of human experience and expertise existing in the data experience.

Danielle Crop (29:32):
Right. I enjoy the human aspect of it a lot because I think that data comes from humans or it comes from systems that are created by humans, one of the two. It's all about people at the end of the day.

Ben Taylor (29:45):
I was also chuckling about the human experience because I was imagining if you look at anomalies in your stores and you find stores that perform unbelievably well, they're outliers, you need to go study them, you might actually find that there's a human touch, that Norma the cashier has a line of people every day that will drive out of their way to talk to her. And you say, "Shoot, I can't replicate that. I can't expand that." Do you run into things like that where you see there's a human touch with an outlier store? You just kind of shrug your shoulders and say, "Well, maybe we can learn, but it's not a data problem, it's more of a human experiential problem."

Danielle Crop (30:23):
Absolutely. There's only so much of customer experience you can optimize in a store environment that isn't about people. In data, you can't replicate human relationships.

Ben Taylor (30:35):
At least not yet. Maybe another 50 years or so. But I love that part. I love that element of humanity, how special that is, the soul to soul interactions, the love and the friendliness and how that can make someone's day. And AI is so far away from doing that. So how hard is it to make changes in these stores? So if the data is demanding some key changes when it comes to placement or the way the stores are outlined, is that something where you guys can go an AB test a store or do a subpopulation, or is there not that element of alignment between all the stores? Is it much more difficult to apply a sweeping change like that?

Danielle Crop (31:22):
Well, we usually start small. So we'll start with a store or a set of stores and test something. And we can do that, and we're on that journey to add more data into those processes and that's really fun. But I think that we're very focused on the digital aspect of the experience. So that would be, if you're anything like me, when I'm going through the store, I'm holding my phone. So my grocery list is on my phone and I'm looking at my phone. And so how do we do that online, offline experience in real time and use the data that we know about a store, about where things are in a store, and about the person's previous purchase history, and customize their in-store experience? The classic example that I use with my team, which is the stretch example thing I want to shoot for, is if you are standing in the produce aisle and you've got your phone in front of you, I want to be able to push to you what you bought the last time that you were in the produce aisle.

Ben Taylor (32:31):
Oh, yeah. Creatures of habit.

Danielle Crop (32:36):
Right? Because if we can do that with location information and all the data that we have, wouldn't that be a delightful experience for a customer? Before I took this job, I read a few books to get myself ready for it. And I don't know that I even really appreciated the miracle of the modern American grocery store before I read those books. And I was like, "You know what? It really is miraculous that we're able to get all this produce out of... Our grandparents, they never had this opportunity. They weren't getting berries in winter."

Ben Taylor (33:11):
And it's so complicated too. Because those growing seasons are other parts of the world, they have to ship them when they're not ripe and then they have a process for ripening them. And then once they're ripe, you have a very short window to move that produce, and there has to be a lot of waste. There has to be a lot that people can't buy because it's moldy and it's soft. I'm thinking of avocados. Avocados sound like a nightmare. But you probably have other items where you think, "No, they're actually really easy." I'd be curious, what are the items that are almost not worth it? So people that are so happy to have their raspberries or some other item? Are there items that you really do have to nearly take a loss just to make the customer happy overall?

Danielle Crop (33:56):
We're taking a different spin on it, Ben, which is basically what are the fresh items that our customers are most concerned about? And then looking into those and helping to optimize as much as possible, the freshness of those items that are most important to people. So how do you make sure that your bananas are in the right state for different types of customers? Because not everybody likes the same types of bananas. I like green bananas. I will only eat green banana. Whereas other people are like, "That's gross. I want to eat only the lightly black spotted bananas because that's when they're sweet." And that's the fascinating thing about it is everybody's tastes are different. What is fresh for one person is not fresh for another. And so you have to actually look at the types of customers, the types of trips that they do, and what is most important to them in their trips?

Ben Taylor (34:55):
So for the data science scientists that are listening, what are some of the big categories we've talked about? Shrink, we've talked about the transaction data, supply chain. Are there other categories of problems that are top of mind when you think about leveraging data for better business insights?

Danielle Crop (35:10):
There's a ton of different spaces where you can optimize. One of the most fascinating areas includes the supply chain, all the way through store operations, all the way to the customer, is out of stock. So you have to figure out how to balance when you order things all the way through to how do they get stocked on the shelf, all the way to the customer. It's a really fascinating end to end process and it's not just supply chain, it's the whole thing. And so that's one of the fun problems that we're tackling because obviously it's a big deal to our customers and it's a big deal to our top line and it's a big deal to our CPG partners. They want to make more sales.

Danielle Crop (35:50):
So when the shelves aren't stocked with something people want, we've just lost a sale, and that's a problem. So that's a really fascinating, interesting one. And there was lots of different places where data science can play in that experience, which is all the way from ordering at the CPGs, and when do we order and how much do we order, all the way through to how many people do we have working in the stores? All the way through to, "Okay, how do we know what's in the back room versus what's on the shelf?" Interesting, basically AI vision type of stuff, making sure that the customers and or the pickers for digital or e-commerce, they can get what they need and deliver it to the customer.

Ben Taylor (36:31):
I love thinking about terrible outlier events. So think of an Albertsons in the middle of Wyoming. It's too far from me anywhere. And they got a triple delivery of bananas all at the same time, where now the store manager is realizing, "So are bananas free? Are we discounting down to a dollar?" And I know there's got to be some stories like that where the supply chain oversupplied, but rerouting that is problematic because it's perishable and the store manager just has to react to it. So I hope today, those types of stories are less frequent, but I imagine there's has to be some legends during the last 10 years where you can find a particular store and say, "Oh my goodness, this particular store had the crazy sale on X."

Danielle Crop (37:13):
Well, I think, and this goes back to around time I was interviewing for Albertsons, was that during COVID, we kept on getting shutdowns. So there was a store, I believe it was in New Mexico somewhere, that had to shut down because COVID protocol said, "You have to shut down." There's all this fresh food in the store. So he's going, "Well, if we have to shut down, what are we going to do with all this food? Because it's just going to go bad." And this was before the store had DUG and delivery and all those kind of... Which is drive up and go is DUG. And so they worked with, I think, local food banks. They really scrambled to make sure that that food was actually going to be used by humans and not just shut down the store and discard it. So I think that was a story I heard, I believe it was during my interview process, that really made me go, "This is the right company to work for."

Ben Taylor (38:10):
And I wonder if the listeners don't appreciate the magnitude of the financial loss. So when you added all up, all of those potential baskets, all of that produce, it's a huge loss to the business.

Danielle Crop (38:23):
It's a massive loss.

Ben Taylor (38:24):
I was reminded of that lady that was sneezing on lettuce intentionally during COVID. I don't know if you saw that in the news. She went through the whole line intentionally just upset people, and I'm sure she was arrested for it because they had to throw all that produce away. Again, an outlier human story.

Danielle Crop (38:42):
I'm perplexed by that one.

Ben Taylor (38:44):
Yeah. Explain the experience that led to that decision. It's hard to explain. I think that's what makes people so fun and interesting. And honestly, your problem's very challenging because I'm imagining stores down in Louisiana versus stores in Vegas, they have very different buy behaviors, very different food interests. And so that's very challenging with data where you have to figure out, "Well, what can we generalize and do we have to do micro or area analysis as it's too different?" Are there any areas in the US or in the world you'd want to call out that they do stand out as an outlier experiences just because the cultures are so different with food?

Danielle Crop (39:28):
I think that the way I would explain it is it's a large number of small problems. You can still do it. It's like what I did in fraud. Fraud was also a large number of small problems. So each store, each location, each banner that we have has different types of customers that attracts. So what you want to build for assortment for those different banners and stores and different customers is different. You can still solve that problem with data. It just is a matter of we have to do it at a different scale. And it's still possible to use all of that data about the customer types in the store, the banner type, the location, to build into the algorithm. It just becomes another variable and then you make your decisions off of that. So there's a ton of differences in the banners that we have when you think about Haggens and [Andronico's 00:40:24] all the way to Vons and Pavilions in Safeway and Albertsons itself and Jewel-Osco. And it's a lot of different types of stores, and that's okay.

Ben Taylor (40:35):
And very different customers that shop at the stores?

Danielle Crop (40:38):
Very different customers.

Ben Taylor (40:38):
What do you imagine will be the types of problems that you're trying to work on and address five or 10 years from now? I know that's really far into the future, but what is a great outcome for society for some of the work that you're focused on?

Danielle Crop (40:55):
I think it really is, with the scale of change, hard to imagine five up to 10 years in the future. But I guess I'll give you my hopes. And my hope is that we'll be using data to help people, help individuals with making better decisions about their help, making better decisions about what they eat, giving them more information. I hope that we'll be attracting and supporting and having the backs of our customers based on the way that we give them data and the way that our digital assets, as well as our stores and continuing to optimize that based on what is good for our customers. I want to really have that customer lens and focus in what we do.

Ben Taylor (41:37):
I love the focus on customers because when we think of our customers, we think, "Well, we want to get them promoted. We want to make them look good." And when you're talking about your customers, you want them to be healthy and happy, which is wonderful. What a great thing that we should all strive for, that we should all be healthy and happy and spend more time with our families and less time at the grocery store, confused, running around, wondering why some data scientists put the beer next to the diapers, but it was helpful in the moment.

Ben Taylor (42:04):
And then other things we can't find. How often we can't find things at the grocery store without asking a human. And so I definitely think of saving time. I did want to ask you one question before we wrap up. How do you compete against Walmart? Actually, to give you some more context, just totally random, I went back country skiing today with the Chief Project Officer of overstock. And I was thinking, how on earth do you compete with Amazon? How does an e-commerce store compete with Amazon? So how do you compete with a monster like Walmart? Yeah, what's your response to that?

Danielle Crop (42:39):
Well, there's a lot of different aspects, but I think that one of the greatest assets that Albertsons has is the number of different types of banners it has and it's 2200 different locations around the country. We have some of the best real estate in the country, being just very close to our customers. And I think that makes all the difference. I think that Walmarts are farther apart and they do different things. We serve people through food and health. Walmart, they are a different model and that's okay and there's room for those different models. And I think that the company is showing that through our results [inaudible 00:43:18] really well. I think there's a lot of people who want a more convenient and focused healthy experience in food. We want to really make our customers' lives better based on the food that they eat and the way they experience our stores.

Ben Taylor (43:32):
I like that response, and I'm thinking it is definitely complicated because you can even get into the details of how good do you want that peach to taste, the out of season peach. That's actually very complicated.

Danielle Crop (43:47):
We actually have some scientists who can tell us that.

Ben Taylor (43:52):
That reminds me of Starbucks. They have all these scientific coffee tasters in their Seattle office where they're constantly testing and trying to apply the science. But I've gone into other grocery stores that are not Albertsons and I've grabbed a peach that looked delicious and I've bit into it and it tasted like it was dead. There was no soul to this peach.

Danielle Crop (44:12):
And you're speaking to my soul because peaches are my favorite.

Ben Taylor (44:15):
Oh, they are?

Danielle Crop (44:17):
And I agree, I'm always searching for the right peach in peach season.

Ben Taylor (44:22):
Yeah, I think this peach problem is so funny because you know what I'm talking about. You see these massive, gorgeous peaches. So visually, they have won you over. You look at this and you say, "This is gorgeous. I can't believe I'm so excited to eat this thing." And then you go buy it and when you break it open, it's like-

Danielle Crop (44:42):
It's mealy. Yeah.

Danielle Crop (44:42):
Yeah, it's mealy. And I don't want to say it's gray inside, but it's not sweet. And you begin to wonder, "I wonder what the supply chain journey was for this peach. And it's been optimized for looks, and maybe it was ripened 48 hours ago rapidly from a state that is impossible and I'm tasting the difference." And so I have noticed that for higher quality grocery stores, they really do care about the user experience.

Danielle Crop (45:09):
Yeah. And I think that there's also of a trend, back to, like I said, I think that there's been, I don't know, for the last 50 years, maybe even, it's a lot about looks of the produce. And I do think that there is a trend back to taste that is happening because people are more aware. I think people are more aware that their tomato is gassed on the way to the store.

Ben Taylor (45:32):
Yeah. With the ethane gas. Is it ethane that they use?

Danielle Crop (45:34):
Right. And they're asking questions about freshness that they didn't maybe ask in the past. And so I think that there's a real opportunity in this area to be a differentiator, and we already are in many ways. And so I'm looking forward to helping the company be even more differentiated in the space.

Ben Taylor (45:53):
I love coming full circle with the creative artist here, where we're talking about the creativity or the artistry behind the taste of a peach. And did you optimize on price versus pound, or did you optimize on the moments? And I think people who enjoy food, they realize, "I'll happily pay three times the price if the moment is right."

Danielle Crop (46:17):
And that's why measuring customer is much more important than measuring sales on individual items. Because you'll see the repeat experience and the increase in value for the company if you measure customer. If you measure just, "How many peaches did I sell?" Maybe you're not right because you're not necessarily looking at that repeat customer, you're looking at that one time purchase. And maybe that one time purchase was based on how it looked. But you know a customer's going to return to you if they get better flavor versus better looks. So I think that that's where the difference is between maybe the way groceries have been done traditionally, which is around item sales, and the way that we're moving the company forward at Albertsons, which is all around customer experience, which I love. Because coming from American Express, that's my heritage. It's all about the customer.

Ben Taylor (47:15):
I love that we have that common enemy. If you bait and switch peach me, I hate you. If you trick me into something that's not what I anticipated... Well, we are coming towards the end and I definitely want to... So for our listeners, what types of positions do you guys have open? You're hiring right now. What are the skill sets that you look for in people that might want to join the effort to delight the customer?

Danielle Crop (47:39):
So the data office is new. We're seven months in, and we have a lot of open positions. We have different areas that we are looking at. We have data management and data governance, we have data products, and we have data science. And all of those different areas we are hiring for today. So whether or not you want to be part of doing the data science work and models around what is going to be the assortment in fresh that we want to have as a company, to the point of, "Okay yeah, I want to be involved in the data management and data governance to make sure that our data is managed as an asset and is clean, is usable for the rest of the company and creates that data driven culture around a customer." Or if you're a products person and you want to build that decision engine for delivery personalization in real time, we have all of those roles available.

Ben Taylor (48:31):
And where would they go to apply? Is there an Albertsons careers where they would find many of these?

Danielle Crop (48:36):
They can go to the Albertsons site and look at the careers tab.

Ben Taylor (48:40):
Perfect. Danielle, it's been a pleasure. I've loved talking to you, and I'm sure our users will love it as well, and I love these topics we've hit on. I'm definitely thinking about food differently next time I go to the grocery store.

Danielle Crop (48:51):
Thanks, Ben. It's been fun.

Speaker 2 (48:55):
Thank you for joining us on this More Intelligent Tomorrow journey. Discover more and join the conversation at moreintelligent.ai. The future is closer than we think.


MIT - S2 REC - Danielle Crop
Marker