More Intelligent Tomorrow: a DataRobot Podcast

You Can't Be in Business and Not Take Risks - H.P. Bunaes

June 14, 2022 DataRobot Season 2 Episode 18
More Intelligent Tomorrow: a DataRobot Podcast
You Can't Be in Business and Not Take Risks - H.P. Bunaes
Show Notes Transcript

This episode of More Intelligent Tomorrow brings together H.P. Buanes, the Executive Director for AI. & Machine Learning at JPMorgan Chase and Diego Oppenheimer, the Executive Vice President of Machine Learning Ops at DataRobot for a conversation about artificial intelligence, machine learning, and their roles in the financial services industry. 

Data analytics and AI/ML wasn't a thing in 1983 when H.P. Buanes started his career. But since then, it’s become a core element for businesses who want to succeed in the new economy. 

"It's never been a better time to be in data and analytics than it is right now."

With that being the case, Diego wonders why banks, who have so much data, still struggle with implementing AI/ML.

H.P. believes it’s because executive management delegates the task to technical teams and hopes for the best. Successful implementation of AI/ML requires leadership from the very top.

It also requires selecting the right project to apply AI/ML. The temptation is to start with big, high visibility projects. But you have to walk before you can run. Starting with overly complex projects will lead to disappointment and wasted resources. Don't just pick what's shiny and new. Come at AI/ML from a business perspective and select a project where there’s potential for attainable and measurable success.

Another crucial decision is to have the right people working on your projects. There are few data scientists who understand banking, and you need someone who not only knows the data, but also your business needs. 

H.P. suggests imagining the press release for your project and working backward from there. Know the end goal and build your solution to reach it. Too often there’s a tendency to look at the data available and see what you can learn from it. Instead, figure out what you need and build your AI/ML solution to meet that need.

Turning specifically to the world of banking and financial services, Diego asks H.P. where he sees opportunity.

Credit. It's where analytics began in the world of financial services, and it remains a deep well of opportunity. From predicting default to more advanced areas like collateral valuation, severity loss, forecasting, and reserve analysis, there are a lot of possibilities. Using AI/ML to differentiate risk and go deeper into the credit spectrum is a terrific way to separate yourself from your competitors.

AI/ML should be used to manage risk rather than to minimize it. If you design your control infrastructure for the highest risk, you'll find you can't move. Make a control infrastructure that's sensitive to the data, and you'll be more agile.

"You can't be in business and not take risks."

Much of the financial services industry resists moving to modern AI/ML solutions. The cloud is still an unknown for a lot of executive leadership, and there’s an inertial bias toward proven legacy policies and procedures. Moving to the cloud means having to rethink those processes. 

For a company to survive, moving to the cloud is inevitable. With access to unlimited computing power, endless storage, and the latest machine learning models in the cloud, the legacy model of an on-site data center is becoming obsolete.

The companies who do AI/ML right will separate themselves from the rest of the pack and be more competitive. You must have the right talent, the right support, and the right goals, all implemented to work with as little friction as possible.

Get educated. Know what AI/ML can achieve. Be able to spot an opportunity and understand what your analytics teams are working on and why.

HP Bunaes (00:00):
And if you just leave it up to the data scientist to kind of pick what they're going to work on, they're going to pick the coolest, sexiest stuff every time, but you really have to bring a business perspective to the table, right? Where am I going to get value? Where am I going to move the needle in terms of a business metric that really counts? Where do I have the data that's actually going to make it feasible to pull it off without a massive data project that's going to take a year, cost a million dollars before it can ever do anything? So you have to manage data scientists carefully, and you've got to try to channel that creative energy into the areas that make the most sense for the business.

AI Voice Over (00:46):
Welcome to More Intelligent Tomorrow, a podcast about our emerging AI driven world, critical conversations about tomorrow's technology today. On today's episode, post Diego Oppenheimer sits down with HP Bunaes, executive director of AI and machine learning at JP Morgan Chase and company.

Diego Oppenheimer (01:11):
Hello, everybody, super excited to have you here on the more intelligent tomorrow podcast. My name is Diego Oppenheimer, and I'm an executive vice president for machine learning operations here at DataRobot. And I'm very excited about today because I've known HP for a while. I'm going to let him introduce himself, but he is a world expert in ML governance and in risk management in financial services. So HP, why don't you introduce yourself today?

HP Bunaes (01:37):
Diego,, thank you. Pleasure to be here and thank you for having me. Wow, what an intro. I hope I can live up to that billing. HP Bunaes. I've been in banking for 35 years, all of it and data and analytics, and I'm currently at JP Morgan Chase in AI and machine learning.

Diego Oppenheimer (01:55):
Fantastic. Great. So maybe we can kick this off. We got a lot to cover today. I think there's a lot of things that we're going to talk about, but how did you get into AI and machine learning? I mean, not exactly something that was implemented in financial services 30 years ago, when you got into the business. Not to date you, but maybe you can walk us a little bit through the journey, and maybe even, in particular, why do you think it's relevant today?

HP Bunaes (02:20):
Yeah, it wasn't a thing in 1983 when I got my first bank job, that's for sure. I'm not sure I got into AI and machine learning as much as AI and machine learning found me and pulled me in. I was always quantitative by nature. I liked physics and engineering, then I got into IT. And then of course IT led naturally into data and all things data, and then into risk management. First job in banking was in finance. My second job was in risk management. And this was during one of the first banking crises that I ever have lived through, and I've lived through many. And one of the things we discovered very quickly was that we needed a whole lot more data to figure out what the heck was going on, and we needed better ways to quantify our risk and to understand dynamics the portfolio.

HP Bunaes (03:11):
So it wasn't a really long trip from data to risk analytics, and I went from analytics for risk to analytics for business more broadly. I spent a lot of time leading data and analytics for the consumer bank at a top 10 bank. So in terms of how it's relevant, the importance of making sense of all your data and learning from your data, and making better business decisions based on your data to grow the top line, to get more efficient, to service clients better, to find new clients, to mitigate risk better. Those pressures have never been more intense. And companies are sort of a wash in data, but making sense of it all has always been the hard part, and learning from it. And AI machine learning, I think, is the very best way to uncover what's going on in the business and to drive the business and make better decisions based on data. So I think it's never been a better time to be in data and analytics than it is right now.

Diego Oppenheimer (04:17):
So, it's interesting. So I mean, I started my career very, very long time ago, working at investment banks straight out of my master's degree in data analytics. And I've always been fascinated. I always kind of look at why financial services... They have the amount of data, the precision, to a certain degree, the structure. So they have all the reasons, in my opinion, around, exactly what you said, right? How do you turn data into a first class citizen? And then how do you react on that first class citizen as a competitive advantage in your business? So, why do banks and financial services always struggle with AI? I mean, they're the prime ones to get into it. I think they've been doing it for probably the longest in a lot of cases, especially some of the top institutions. Machine learning is not new to them. Why do you think they struggle?

HP Bunaes (05:03):
Yeah, it's a good question. And it's not new. I mean, we've been doing linear and logistics regression since the world was flat. But I think the reason that banks struggle is that, in a lot of cases, executive management delegates the development of analytics to the clients and the data scientists and the technical people, and sort of sit back and hope good things happen. And you just can't do that. It's too transformative. It's too strategically important to delegate that responsibility. I think top executives have to really be involved. And it starts with picking the right applications of AI and machine learning. And especially at the beginning, when you're just starting out, I've seen some banks start out trying to do some really sophisticated things requiring enormous amounts of data and some of the best data scientists in the world. And then they're very disappointed that they don't get incredible results very quickly.

HP Bunaes (06:07):
You have to be really careful about where you apply this stuff, especially early on. And if you just leave it up to the data scientist to kind of pick what they're going to work on, they're going to pick the coolest, sexiest stuff every time, but you really have to bring a business perspective to the table, right? Where am I going to get value? Where am I going to move the needle in terms of a business metric that really counts? Where do I have the data that's actually going to make it feasible to pull it off without a massive data project that's going to take a year, cost a million dollars before it can ever do anything? So you have to manage data scientists carefully, and you've got to try to channel that creative energy into the areas that make the most sense for the business.

HP Bunaes (06:58):
There's some really, really great data scientists that know the business of banking and know the data science as well, but they're rare. That's a hard person to find, the person that can figure out what's the best application of the data science within a business context. It's more likely you're going to have data scientists that don't know the products and don't know the business lines and don't know where the opportunities are. So you really have to put the right talent together from both the business side and the data science side, and be very careful about what you pick and how you start out, and where you put your talent.

Diego Oppenheimer (07:34):
It's really interesting because one of the things that I've tried to really push forward, and I've borrowed a couple of frameworks from other folks, but it's like, start from the end. What's the end result? How are you going to deal with this in production? How are you going to be able to govern over this? That is actually the first question, because if you don't map it, if we can't map to the result, if we can't map to the outcome, then is there really a reason to be able to do this? And one framework that I followed in love with is, in traditional product development, you have this concept of the minimum viable product, and the MVP, and I kind of fell in love with this framework in data science and machine learning, which is called the minimum justifiable improvement tree. And it's really a set of questions around, okay... And exactly what you said, right?

Diego Oppenheimer (08:18):
If we're going to be picking an optimization, a use case that's an optimization or a next best decision, what is the minimal justifiable improvement to go do the investment? Because I know I have to go acquire the data, I know I have to go do the model feasibility, I know I would have to implement it, I know I would have to align executive buying power behind it, I know I'm going to have to have a certain investment level, and so what is the minimal justifiable improvement that will justify doing that? Because to your point, spending $2 million to make $2 million, well, I guess it depends on the timeframe, but it's probably not the wisest use of time. And so it's amazing how many organizations that I've seen that the first thing is like, "Well, do we have data?"" It's like, well, maybe you do, maybe you don't. It's an important step, but what are you trying to do with that? Right? What is the actual objective?

Diego Oppenheimer (09:05):
And I kind of like starting from top down, in terms of getting the executive buy, and be like, okay. At the end of the day, anything in the world of ML and AI is probably going to follow the category of optimization, a next best decision, or kind of like a use case in one of those categories. And so I should be able to verbalize at least what I expect the return to be that is worth the investment.

HP Bunaes (09:27):
Yeah, I totally agree. In fact, in one agile framework that I recently got a chance to learn about, they suggest you write your press release, and then work backwards. And I really like that. Figure out exactly what the really exciting payoff is that you're working towards, or that you'd love to be able to announce, and then work backwards from there.

Diego Oppenheimer (09:51):
That's actually Amazon's way of building products. It's called the PRFAQ. So that's actually implemented at every level. You write the press release exactly in FAQ and you answer these questions, but you never talk about implementation, you never... Literally, what's the result? What's the result of the customer? What is it actually doing? Then you can build backwards. So yeah, it's a pretty popular framework used in iterating very quickly, because it starts making you think about the end result. It's just amazing how many times people don't think about these machine learning, ai project. I think it's the nature of it. If you think about data science, it really started on the academic side of the house. In most cases, these folks who have traditional data science backgrounds, they think like academics. And that is not a bad thing, right?

Diego Oppenheimer (10:35):
It's the ultimate pursuit of truth, right? And so it's like, what do we get to increase that accuracy, or something like that. And it's amazing to me because is it worth 10 years of investment to get from 92% accuracy to 93% accuracy? Only in science, right? Usually, right? I mean obviously it depends on the number, but most businesses are like, "Hey, you know what? We would've stopped investing in this at this threshold because we're already seeing that advantage." But having to think through the business metrics and the KPIs that you're actually developing is such more core into it. So I love these work back. I recommend it all the time. I think if you're not thinking about the end result and how this gets implemented at scale, how are you going to govern it over it, how are you going to affect the business, and you have that metric in mind, it's a potential endless pit of money that you will never get to a result in.

HP Bunaes (11:22):
Yeah. And the other step I would say is critical is, if you come up with 10 great ideas, figure out the value versus the complexity. And it may sound simple, but start with the highest value, lowest complexity use cases. And the ones that you think are going to be the highest value or the most transformative may also be the ones that have the highest execution risk that your chances of actually delivering are pretty small based on all the miracles that have to happen between here and there. Maybe you're better off going after some of the low hanging fruit that may have some slightly lower value, but a much better chance of actually delivering.

Diego Oppenheimer (12:05):
So actually, talk about higher value and complexity and ability to deliver. Where are the use cases in financial services that you're really seeing advantages of? Where are the categories that, in your mind, that you've observed, that you were like, "Hey, you know what..." proven true use cases in here that you see?

HP Bunaes (12:24):
The first one obviously would be credit. And we could probably spend the next hour just talking about credit. That's where analytics really began, and it's a really deep well of opportunity, from the bread and butter stuff like predicting default to collateral valuation and loss severity and loss forecasting and reserve analysis and risk adjusted pricing. I mean, there's just lots and lots and lots and lots of water in that well. But I think some of the opportunities there that banks are figuring out is that with better credit models and with more data that go into those credit models, you can rely less on some of the traditional sources of information for credit decisions. Credit score doesn't necessarily tell you very much that every other bank doesn't already have, right? The trick is, can you find those opportunities that your competitor is not able to spot? If you're just using the same credit score as the guy next door, big deal, right?

HP Bunaes (13:29):
If you can find a way to differentiate risk within a particular credit score, or to go deeper into the credit spectrum, take more risk, but to do it safely because you've figured out some unique features that are informative that nobody else has figured out, or how to reach a different set of clientele that perhaps has been underserved with a particular banking product. There's some incredible opportunities there that I think are even still just at the very beginning of really being tapped into. But there's a lot in credit still. There's a lot of room to go. In other parts of risk, obviously fraud is a really, really big area for banks. Banks don't like to talk about it. Fraud is a really big deal. And the hard part about fraud is you can be really, really good at reducing fraud, but you're probably going to have some pretty serious client impact along the way.

HP Bunaes (14:26):
So the question, of course, is how do you get better at preventing fraud, but without the negative client impact. Better client service, figuring out exactly what somebody needs before they even know they need it, right? Or if somebody's trying to contact you, figure out exactly what they're likely to be contacting you about, right? So you can get them to the right place without having to go through five layers of call center or menus on the phone or whatever. Client service is a really big opportunity. And the trick, of course, is better client service at lower cost. And I think AI and machine learning is a way you can definitely do that. You can figure out what your clients are looking for, what they need, what their problem or issue is likely to be, and service them much more efficiently and give them a better experience. So I think there's a lot of opportunity there too.

Diego Oppenheimer (15:17):
Awesome. So you talked about risk, and this is one of the things that I think we've spent a lot of time chatting about this over months and months. And one of my favorite HP teachings from my perspective is the pendulum between strategic risk, operational risk, and brand risk. And I think this is so relevant, especially for financial services institutions. We see this all the time now. We'll dive a little bit deeper into the model risk management and the responsible AI, but let's hear it from HP himself. I love that definition and the pendulum and why, so maybe you can kind of talk to our listeners today about that balance between strategic risk, operational risk, and brand risk.

HP Bunaes (15:56):
Yeah, it's a concept that I'm often surprised at how few bank executives get this, but if you want me to manage your risk of, for example, making mistake because the model is bad, or risk of having an operational problem that impacts clients or the risk of something going wrong with a new product, I can slow everything down to a crawl by putting controls over everything. And will that lower my operational risks, execution risks, client service risks, regulatory risks? Absolutely, but at the cost of a massive strategic risk, because now I'm moving at a snails pace, if I'm moving it all. And my competitor who is racing ahead of me, because they're much smarter about how they manage risks, is implementing better pricing models and better credit models and reaching deeper into the credit spectrum and finding new borrowers before they even know what they need or when they need it.

HP Bunaes (17:02):
And there's always the temptation to try to manage risk to as low as you can go, or as a lot of people say, to minimize risk. And I always sort of admonish someone want to hear that, is no, no, no. Our job is not to minimize risk. And you can minimize risk by just shutting the doors. There, I've minimized risk. You're not trying to minimize risk. You're trying to manage risk. You're trying to mitigate risk. You're trying to balance risk and reward. You can't be in business and not take risks. And if you try to manage all your risks to zero, you're just going to find that you can't move, because you got so many controls in so many places to manage so many risks that you can't even get out of the bed in the morning.

Diego Oppenheimer (17:52):
So yeah, this brings up a super interesting point. So in the last six months, probably for the first time in my career at least, I've started hearing heads of financial services institutions, potentially behind closed doors, but talk about risk management and what's being done around model governance as an actual competitive advantage. So taking it out of the behind the scenes, compliance, nobody wants to do this to now actually having a voice, and Hey, this done well actively and quickly is a competitive advantage to the organization, to your point, right? That risk mitigation now, this will be used. This will actually allow these organizations to react quicker, take advantages of what I would call opportunities in the market, charge opportunities in the market, while providing the proper guardrails around the risk mitigation that's needed and compliance and stuff like that. And that's really interesting, because I feel like, for the most part, incorrectly, in my opinion, the folks in model risk management and compliance have been seen as a impediment to the business. And now they are a partner in the business and in moving quicker. Is that something that you've seen as well?

HP Bunaes (19:04):
Oh, absolutely. And the trick is... And it's not easy to do, but the trick is to put the right controls in the right place and have just as many controls as you need, but no more, and to make those controls as efficient as possible. And the ideal control is one that nobody even notices it's there, because it's embedded in the process. It's very efficiently handled. It's not the type of control we are waiting for somebody for months to get an approval on, or it's a process that's so complex it's impossible to navigate. The right control infrastructure is built into the workflow. It's highly automated, and it follows sort of the 80/20 rule, right? The 80% of the things that are very low risk just fly through, and the 20% of the things that are higher risk, those are the ones that get slowed down. Right?

HP Bunaes (20:03):
The worst thing you could do is a one size fits all risk management or control infrastructure that treats everything as the same level of risk. It's not, right? There's some things that some uses of data, some modeling techniques, some applications of AI machine learning that, in my view, should get minimal risk. Whereas there's some that absolutely deserve enormous risks and lots of controls and ought to be really checked every way there is before you implement them. But that's a small percentage of the global set of opportunities to apply AI machine learning. So if you design your control infrastructure for the riskiest possible thing that goes through, then you're going to find you just can't move. Whereas if you really make the control infrastructure sensitive to the nature of the data, the nature of the use case, the nature of the modeling technique, how it's going to be used, who's going to use it, you'll find that a lot of things can just fly through with a very light touch, right?

HP Bunaes (21:10):
And that's the trick. And if you can get the right controls in the right places built into your process, highly automated, risk sensitive, then you'll find that you can move really fast, and you can focus the risk mitigants where they need to be focused and not gum up the works with all sorts of stuff that really doesn't need that level of attention. So I absolutely agree with whoever it is you're talking to, that if you get it right, it absolutely is a competitive advantage. Why? Because you'll retain better data scientists. You'll get more solutions to market faster and at lower cost. You'll be able to innovate and try a lot of different things. And take that capacity that would've been eaten up by navigating the control infrastructure and put it into experimentation and finding cool ways to learn from your data and deploy better analytics. So yeah, totally agreed. I think the companies that get it right are just going to clean the clocks of the companies that get it wrong.

Diego Oppenheimer (22:15):
And I like what you said there, because I've been multiple times said, to me, the best on [inaudible 00:22:20] strategy is... It's the invisible one, fully automated, behind the scenes, guardrails. I've used the same quote every time, but to me, today, the ability to go from that kind of model visibility and creation process to affecting a business metric using machine learning, in most cases is like a dirt road, right? If you go too fast, you're going to skid out, right? You're going to skid out. No matter how much you want, and even if you have the best car ever, you're going to skid out. You have a speed impediment. What you want to build is a high speed highway, where the infrastructure allows for speed. It doesn't mean there's no controls. It doesn't mean that there's no police and toll booths and stuff like that, but what you've actually done is created a infrastructure and a scaffolding around your machine learning processes, that includes model risk management and governance, that allows you to be confident in pressing on that accelerator-

HP Bunaes (23:16):
Exactly.

Diego Oppenheimer (23:16):
... as far as you want. So how do you actually build out that confidence is really, really important. So do you think that this applies in terms of... We talked a little bit about kind of risk and risk mitigation and strategic. How much do you think that this has applied or slowed down the adoption of cloud technologies inside financial services institutions? Because I see a lot of the same concerns around like, hey, one size fits all in terms of risk management, legacy policies and procedures. Do you see a shift in that, in terms of how these technologies are being adapted?

HP Bunaes (23:52):
Yeah, it's a great question. And clearly I've seen a lot of resistance to change, just inertia based. I think a lot of people have spent their whole careers managing stuff that you could drop on your foot, that sits on prem and you can see it and you can manage it and you can touch it. And they've spent all their whole careers figuring out how to do that well. And moving into the cloud, that raises a lot of questions. And I think there's been a lot of resistance based on not understanding the payoff and the benefit of moving to the cloud. Even if you don't move your applications to the cloud, at least moving your data to the cloud. But I remember early on, and it wasn't that early on, just four or five years ago, I got a lot of pushback from the IT people who said they had real concerns about data security, which is a bit of a smoke stream, right?

HP Bunaes (24:52):
Because the data security concerns are largely unfounded. I mean, are there data security concerns? Yes. But it's largely in the transport, not in the storage, and there's pretty well established ways to mitigate that if you do it properly, to put that concern to rest. But I think in other big sorts of inertia has just been legacy policies and procedures, right? They were built for just the assumption that everything was going to be on prem. And going through and figuring out what kind of new policies and new procedures do we make to build to make the cloud usable, that takes some thought. And the companies that really put either their fintechs who started in the cloud and really never had to make the transition are the ones who understood really early that this was going to be a transformative moment in the industry and they needed to get there quickly and tackled all those legacy policies and procedures, I think it'd been able to move a lot more quickly.

HP Bunaes (25:51):
But I think it's going to change. It has to, because just the availability of unlimited compute, unlimited storage. Some of the models that are getting built now require enormous capacity and specialized GPU based training and enormous data sets for training. And building that kind of surge capacity on prem, it's a massive, massive capital investment, and it's going to take a long time. And why? Right? When you can pay for it when you need it, but only when you need it and avoid that massive expenditure of building capacity internal. So I think it's changing, and I think that changes accelerating.

Diego Oppenheimer (26:42):
So, we're switched subjects here a little bit, talk about everybody's favorite subject, which is regulation. So it's interesting, especially in financial services, we hear about our AI regulation, right? And Stanford's AI index just put out a ranking of all the AI regulations that have come out around the globe and a bunch of the countries. And it seems like one of the interesting things is you hear about it more, but my perception, and correct me if I'm wrong is, at least in financial services, after SR117, there hasn't been a ton of new stuff being created, but a lot more of that collaboration around working with the financial services institutions around compliance and actually getting people on board with these regulations. But from your opinion, where do you think the different regulators are going with this? Where do you think this is driving towards?

HP Bunaes (27:32):
I think everybody in banking and analytics is asking the same question. I think there's little question that the US is behind. Europe is further ahead in terms of their thought process around governing AI and machine learning. APAC, I think, is ahead. I think the US is behind, but I think they're moving, and they're definitely starting to focus some of the effort around governing AI machine learning in targeted ways. So for example, I think there's going to be a lot more consideration of fairness in the use of AI machine learning, particularly as it impacts individuals, not so much companies, or corporate entities, but certainly with individuals. And there's going to be more sensitivity towards societal impact and embedded bias using training data that you know is based on decades, perhaps, of bias decision making, unintentionally so, but unfortunately those things exist. And explainability is getting to be a really, really big deal.

HP Bunaes (28:44):
If you make a decision that adversely affects a client, let's say you decline to extend credit, or you decide not to renew somebody's credit card, or turn them down for a mortgage or what have you, you have to be able to explain why. And you have to be able to explain that both to the borrower and to regulators, and both globally and locally, right? You have to both be able to explain individual decisions, as well as to explain globally, how your models are working and whether you can truly defend your models as being as fair as it could possibly be. So I think there's going to be a lot more of that. I think the other thing that clearly is happening is a lot of the initial focus on governance and regulatory requirements were around validating models before they're deployed, making sure they're fully tested.

HP Bunaes (29:38):
You've used out of sample or out of time data to make sure that you have an overfit, there isn't generalization error, that you fully documented the models. Maybe you've stressed heads to them to see where they might start to break down. And that's all good and fine and wonderful, but I think the realization is that's just not sufficient anymore. As you go from dozens of models to hundreds of models to thousands of models deployed, it gets really, really hard to manage the complexity. And you have to not just know that what you deployed was good when you deployed it. You need to know that it's still good, which means you got to know exactly how well each of your models is working now. Nobody wants any unpleasant surprises, right? Something that ceased to work because some key piece of data disappeared from the data stream and nobody knew it, which I've seen happen.

HP Bunaes (30:37):
You have to be able to convince yourself that everything not only worked when you deployed it, but continues to work well today. And knowledge management is getting to be a big deal, right? Not just, what do you have in the pipeline, but do you know everything that's out there, and what data it's consuming, and what results it's producing, and where they're going, and who's using them, and what the impact of a big change in your data architecture would be downstream to all these different models and consumers. I mean, the knowledge that you have to have of everything that's been deployed and how it works and what the potential impacts are to a big change in the environment or in the data, that's getting to be a bigger and bigger deal, as models become more ubiquitous, right?

HP Bunaes (31:24):
And they're in every application, and every business line, and every function. Your operational risk inevitably goes up, right? And the risk of a break because of something you didn't anticipate is going to increase as the number of deployed models increases, and as the interconnectedness of all the data and all the models increases. So the requirements around knowledge management and making sure you really know what's out there, what data it's using it. Do you have up to date documentation? Do you know who's using them? And do you know what they're doing with the results? These are really, really important questions.

HP Bunaes (32:01):
Chief model risk officers I've talked to tell me the focus is now less on what happens before you deploy to what happens after you deploy. And a lot of data scientists will say, "Look, 95% of our work stuff that we've already built and deployed, and keeping those models up to date and keeping the data up to date and continuously tuning those models and looking for new features when the models start to drift and they're no longer working as well as they used to." As the world gets more complex and as your organization gets more sophisticated, that knowledge management and change management and managing all that operational risk get really, really important, and I think the regulators are beginning to pick up on that.

Diego Oppenheimer (32:49):
Yeah, I think that... You mentioned a couple things that I really relate to. And IT governance framework is not a new thing, right? We've created it governance frameworks in the past. We understand it. We kind of have to review it. A lot of it is really about aligning stakeholders in the organization and actually providing framework for observability in a common language around it. And what we're starting to see now is that evolution of those it governance frameworks into AI governance frameworks organization, which is starting to manage AI the same way we manage software, I think is one of those components. But I really like about the post deployment that you talk about, because one of the things is, at all times, you should be able to say who called, what, when, with what data, and why, right? I think that's so important, that observability across the organization of being able to just provide that while really focusing your framework on speed and scale and automation.

Diego Oppenheimer (33:44):
So one of the things that I'm curious, so we see these ML governance frameworks being developed, and we start seeing how organizations are snapping to a common language in their ML governance frameworks. What are some of the common mistakes that organizations are making today that you would specifically call out?

HP Bunaes (34:03):
Yeah, I think one of the mistakes is treating AI like any other software. It's code, therefore let's manage it like code. And we got lots of procedures built around that, so let's just manage it like code, but it's a completely different animal. When you're building software, it's a very disciplined, rigorous process where engineering and architecture are really important. Whereas in AI and machine learning, there's a hell of a lot of experimentation, and a lot of failed experiments, before you get to something that you want to put that kind of discipline around. And applying those disciplines too early in the process is really crazy. And applying SDLC to the early phase of AI machine learning is just nuts. The other thing is SDLC sort of assumes that once a piece of software is built and deployed, it's going to work forever, right? Well, not so with AI and machine learning, which is going to start to degrade practically immediately. As soon as you put it in, it's going to start to degrade.

HP Bunaes (35:11):
So you can't just put all the disciplines and all the rigor around the design and the build.. You have to put that level of rigor into making sure that when something begins to degrade, you know it, and you can take appropriate action. And then that leads into the third thing, which is, when you're making a change to a piece of software, that has to go through a pretty rigorous change management process, and for good reason, right? You don't want to throw a change and find that it breaks two other things. Or what worked really well in test, in isolation, all of a sudden, in combination with other changes, doesn't work. It happens all the time. You have to keep a model up to date. And you're retraining a model, let's say monthly, maybe weekly, maybe even daily, right?

HP Bunaes (35:58):
Putting a model change through that kind of rigor is insane. I had one data scientist say to me, "If it takes me three weeks to get a change through on a one week refresh, there's something wrong here. This is just not going to work for us." So it's really great that we've thought up all these wonderful development life cycle procedures, rigors, disciplines for software, but it's a horrible mistake to think that we can just apply those to AI machine learning. You can't. And if you do, you're going to find that, all of a sudden, there's a skunk works right out there because people have avoided dealing with you. They're building things on their own because you've just made it far too hard for them to navigate through that whole process.

Diego Oppenheimer (36:55):
And so I'm going to try to distill some of those problems and test it out on you live here at HP. So here, what I've been saying. So I think in the world of ML ops, we can borrow a lot from the software development life cycle. We can borrow a lot. And so I think one of the ways of looking at what the differences are, and what I've said, is one, we're dealing with probabilistic code versus the deterministic code, right? So look at the components and figure out what breaks, because that needs to be changed as one of those things. Second, from a software development life cycle perspective, data science is the fastest moving software that's ever existed, because of iteration speed because of the testing balance. And so again, we can go back to that analogy of the dirt road. If you're moving through this thing faster and everything else, what breaks? Right?

Diego Oppenheimer (37:42):
And then the other way, which is like, okay, but there's certain pieces that can be used. For example, using [inaudible 00:37:47] for code and source code management. So there's certain components around automation, DevOps principles, around speed and scale and security that could absolutely be borrowed, but when looking at the software development life cycle, to create your machine learning development life cycle, take into account those couple of things, deterministic versus probabilistic code, speed of iteration, and figure out, in those two things... And then ability to continuously monitor, because things will break without anything changing. And so once you have that mentality around that, you now suddenly have a framework that allows you to make an equivalence between the processes that you've had and how you're developing this and your software.

HP Bunaes (38:28):
Yeah, agreed. And the other thing I'd say is road test it, right? Take a couple of good examples of analytics and try them and see where your process doesn't work very well. And one of the things that I see at a lot of banks, and I'm happy to say that is not the case at my current employer, is the partnership between analytics and traditional IT is not very good, where they, quite honestly, they just don't like each other very much, right? The analytics people look at the IT people as being way too slow and way too expensive, and don't understand the power of analytics and the value of what they're doing. Whereas IT looks at the analytics people as sort of rogue, right? They've gone way off the reservation. They don't have any controls. They do things that we would never dream of doing. They take enormous operational risks and seem not to give a damn.

HP Bunaes (39:30):
And that's a real problem. If you're analytics people and your IT people are not working well together, you're leaving a massive opportunity on the table, because then by definition, analytics people are going to have to get into the IT business, right? If they can't rely on it to support them appropriately, then they're going to have to get into the IT business. And you do not want your analytics people getting into the IT business. That's not what they're good at.

Diego Oppenheimer (40:02):
I would say the reverse too, right? Because to that point of rogue, so it's not just one sided. I would say I find that the IT folks are being asked to take on the operational risk of the business in a lot of cases. That's what their job is, say, "Hey, these systems are going to run. They're going to be running all the time. They're secure." So when you're translating that risk down to who's being responsible for it, that first line, right? Because who runs the operation? If I have a banking system that's just being, we're doing kind of automated fraud detection, well, it's the SREs that's responsible for that thing going down, and all the time that goes down.

Diego Oppenheimer (40:37):
So it really is a partnership, which I think takes you back to where we started today, which is executive management and saying what the business goal is and determining it, and so aligning it and analytics under what is, for the business, that they need to do, right? Because at the end of the day, that's where the joint venture happens. I think too many times it's seen as one against the other, versus it's truly a partnership to achieve a business result, that is sponsored by an executive.

HP Bunaes (41:04):
Absolutely.

Diego Oppenheimer (41:09):
Okay. So to wrap up here, even though it's been a fascinating podcast, what does a more intelligent tomorrow in financial services look like?

HP Bunaes (41:20):
No doubt, I think the banks that get this right, and sort of not just banks, but fintechs, asset management, anybody in financial services. See, as you said at the beginning, they have enormous data, and they have data riches that in a lot of other industries, they can only dream of, right? And the companies that get this right, that get the right talent in, set them up to be successful with the right support from the right areas, guide them into the right kind opportunity set where they can have a great chance of success, put the right support around them, eliminate friction wherever they possibly can, they're going to separate from the rest of the pack, and they're going to innovate faster. They're going to get solutions implemented, better solutions more quickly implemented.

HP Bunaes (42:11):
The companies that get this right are going to just separate themselves from everybody else in terms of tangible business performance. And to me, that's the opportunity. And for the banks that are just sort of hanging back and taking a wait and see approach, or for the executives that aren't really taking this seriously and aren't really getting involved in thinking strategically about where the opportunities are, I think they're going to find themselves in a situation where they just can't catch up, right? And they're in pretty deep trouble. So I think business leaders, if they haven't already done it, they need to get educated. Do they need to know how stochastic grading and descent works? Absolutely not, but they better know what AI and machine learning is capable of, and better be able to spot a good use case, and better be involved in understanding what is it that our analytics people are working on and why, and really should be driving the priorities, and then making sure that as innovative things get developed, really make sure the business heads are taking it seriously in terms of rollout and adoption and training and communication.

HP Bunaes (43:31):
So you're getting the value from these really cool things. I mean, a lot of great stuff sometimes happens in the lab, and then it never gets rolled out because the changes that have to happen in the business, or the lack of a strong business champion, just prevent it from ever getting used, which I think is just a tragedy. So I think executive leadership has to take it really seriously, both in the development phase, making sure that you're working on the things that are going to be high value, but also in the deployment phase and making sure your business heads are really making sure that you get the value, and that everybody's always on the lookout for the best new opportunities. And I think that's got to be how companies evolve and how leadership evolves around it. So it can't be something that they delegate, like they delegated technology to their CIO and delegated data to their chief data officer. It's a different animal. It's too strategically important. It's too potentially transformative. This has to now be on the CEO's agenda. And if it's not, then I think that's a problem.

Diego Oppenheimer (44:37):
Fantastic. Well, thank you so much, HP, for today. It's been fascinating to conclude. I think it's been such a cool episode, everything from talking about how we're going to use management of risk as a competitive advantage, building out a framework for ML governance, potentially bringing teams together in the IT and analytics to kind of see under the executives how to move faster. At the end of the day, this is really about, we have a really unique opportunity in what is the most transformative technology that will probably see in our lifetime. And there's going to be clearly a set of adopters and people will get it right and figure this out. It's going to be a massive competitive advantage, and it is going to be a existential risk to the organizations that missed the boat on it. And so very exciting times, especially around thinking obviously this entire area of ML ops and how it's opening up to really going to be transformative in businesses. So HP, thank you so much for your time. Really appreciate it. Always fascinating to talk to you.

HP Bunaes (45:35):
Always fun talking to you, Diego. Thank you for having me.

Speaker 2 (45:41):
Thank you for joining us on this More Intelligent Tomorrow journey. Discover more and join the conversation at moreintelligent.ai. The future is closer than we think.