The Hidden Reason Why Companies Waste Millions on AI

Most companies are pouring millions into AI and have very little to show for it.

The problem isn't the technology. It isn't the budget. And it isn't the team.

Dr. Brindha Jeyaraman has seen it from every angle as a regulator, inside big tech, as Head of AI Governance at a major regional bank, and now as founder of an AI governance infrastructure company. The reason most AI transformations fail starts at the top.

In this episode, Brindha breaks down what most organizations are getting wrong about enterprise AI strategy and what the ones succeeding are doing differently.

We cover:

- Why AI projects fail to scale beyond the experiment stage

- The "braking system" case for embedding AI governance frameworks into your architecture from day one

- Why the bottom of your organization is moving faster than the top — and what that leadership gap is costing you

- The real reason hiring consultants to lead your AI implementation backfires

- How AI has cut cybersecurity exploitation speed from months to hours and what that means for enterprise risk management

- What the right AI team structure actually looks like today — and why most organizations have it upside down

- Why human in the loop is one of the most misunderstood concepts in LLM deployment

Whether you are a Chief AI Officer, Chief Data Officer, Head of AI, AI governance lead, or senior executive navigating AI transformation in banking, financial services, or regulated industries — this conversation gives you a practitioner's honest view of what actually works. If you are researching enterprise AI strategy, AI governance frameworks, LLM deployment best practices, agentic AI implementation, AI risk management, how to scale AI across your organization, or why so many AI projects fail to deliver ROI — this episode covers it all.

-
⏱️ CHAPTERS

00:00 — Why AI Replacements Still Need Human Reviewers
01:32 — Brinda's Journey: Regulator, Big Tech, Bank, Founder
01:55 — How a Family Health Crisis Shaped Her View of AI's Purpose
03:12 — What Senior Executives Get Wrong About Deploying LLMs at Scale
05:29 — Operationalizing AI at the Speed of Change
06:37 — Why Organizations Get Distracted by New Tech Instead of Readiness
08:22 — Experimentation vs ROI — The Tension No One Talks About
09:52 — Why Companies Overestimate How Far Along Their AI Journey They Are
10:38 — The Role of Data Strategy, AI Strategy and Board-Level Buy-In
12:26 — Why the Top of the Organization Is Falling Behind
13:01 — What Separates Companies That Accelerate From Those That Stall
14:31 — The Middle Management Problem: Who Actually Gets the AI Mandate
15:50 — How to Push Leadership Into the AI Journey From the Start
16:45 — The Case for an AI Council
17:12 — Why Siloed Ownership Kills AI Transformation
18:05 — Building AI Foundations: Data, Knowledge and Tech Readiness
20:03 — "AI Governance Is Just Added Cost" — Brinda's Response
22:05 — Why Hiring Consultants to Lead Your AI Strategy Backfires
23:27 — Where the True Risk of AI Implementation Actually Lies
25:21 — The Human in the Loop Problem: Speed, Accountability and Design
26:18 — How to Design Human Oversight Into an AI Architecture
28:15 — How Organizational Culture Shapes Your AI Governance Framework
30:27 — What Worked Across Regulator, Big Tech, Bank and Startup
31:00 — Why Data Teams Are the Hardest to Bring Along
32:49 — What Actually Made Brinda Succeed Across Very Different Environments
34:18 — Why What Works in Big Tech Won't Work in a Bank
36:08 — The Hardest Leadership Conversations on AI Risk
38:28 — Posting AI Success Stories on LinkedIn Without the ROI
38:51 — Building the Right AI Team Structure Today
41:37 — Andrej Karpathy Stopped Writing Code — What That Means for Teams
42:09 — Why Coding Skills Still Matter in an LLM World
43:51 — How AI Team Structures Will Evolve Over the Next Two Years
45:49 — Are Companies Justified in Cutting Headcount Because of AI
47:02 — If You Could Redesign AI Governance From Scratch
48:46 — Claude Mythos, Cybersecurity and the New Threat Landscape
51:21 — Exploitation Speed: From Months to Hours
52:02 — Governance as the Answer to AI-Enabled Cyber Threats
53:45 — Rapid Fire Round Begins
54:07 — ChatGPT, Gemini or Claude?
55:27 — Head of Data Governance or Head of AI Governance?
55:34 — Centralized vs Decentralized AI Teams
55:41 — Most Overrated AI Trend Right Now
55:59 — Most Underrated AI Trend
56:23 — One Global AI Regulation You'd Implement Tomorrow

🎙️ The AI Visionary Podcast is hosted by Joel Azariah and Milind candid conversations with the leaders shaping AI.


#AIGovernance #EnterpriseAI #LLMOps #AIStrategy #AILeadership #AITransformation #AIRisk #ResponsibleAI #ChiefAIOfficer #AIImplementation #TheAIVisionaryPodcast

The AI Visionary Podcast is a platform where AI leaders share bold, unique and tangible insights about pathways to realizing meaningful ROI from AI, pitfalls to avoid and their success stories.

These conversations go beyond the hype to uncover what it takes for organizations to be successful in their AI transformation journeys from a leadership, talent, strategy and operating model perspective. 

Co-hosted by Joel Azariah and Milind, new episodes release every month via You Tube, Spotify, Apple Podcasts and various other Podcasts Channels

Links

YouTube Channel

https://www.youtube.com/@TheAIVisionaryPodcast

Spotify Channel

https://open.spotify.com/show/3Bt27yjIzXP9Yr1JyqzpTv?si=feq6M3RTR1ywJS_694-Q7A&nd=1&dlsi=ce2a21ba58db44b6

LinkedIn Podcast Page
linkedin.com/company/the-ai-visionary-podcast
Joel Azariah LinkedIn Page
...

Transcript

SPEAKER_05 0:00

And when you don't create one, you kind of revisit the same again and again every three years with realizing very minimal ROI, sometimes even zero ROI. Which means you replace ten software engineers with 10 AI coding agents. Now you need to have two to three reviewers to ensure that it's uh it's coding the right way or it's coding exactly what you want.

SPEAKER_01 0:20

One of the key threats is that the speed of exploitation of these has gone down from months to hours.

SPEAKER_05 0:27

In the hands of the wrong, again, we are speaking about exploitation of the loopholes.

SPEAKER_00 0:33

Welcome everyone to this episode of the AI Visionary Podcast. We have with us uh a very unique guest today, Dr. Brinda Jay Raman. Um, I've known Brinda now for I think more than three years, and during this time, I think what's amazed me is her ability to take risk in a way that always returns with not just value for the organizations she works, but the community at large. So Brinda's journey is very unique. She's worked in you know, in the regulator, she's worked in big tech, she's been the head of AI governance for a large regional bank. And then she keeps not just surprising us, but also herself. And this next step she's taken as a founder of an AI governance infrastructure company called Atrix is even more amazing. So, Brinda, thank you so much, and welcome to this episode of the AI Visionary Podcast.

SPEAKER_05 1:31

Yeah, thanks, Jewel.

SPEAKER_00 1:32

Great. So just getting to it, Brinda, like I said earlier, your journey's been very unique. Um, and you know, you've spoken about to start off with a personal loss in your family that changed how you saw AI's role in healthcare. Can you share a bit about how this uh event shaped your view about AI?

SPEAKER_05 1:55

Yeah, so a few years ago, I did um have close encounters with health issues that my family members were going through. And that kind of made me realize that with the advancements of tech, if we are not creating a real impact to the mankind at large, although we are kind of creating indirect ecosystem impact everywhere. And at that point, I was working in the AI field quite deep with the machine learning and traditional AI. So, having said that, I kind of drifted towards ROI quite early. How can we create impact to the society through the little avenues that you get through your job or you know different hats that you wear? And uh so I learned very early that you have to think about impact and ROI, no matter what kind of magical technology that comes in. It could be Gen AI, Agentic AI, AGI, but you need to wear the lens of how will this create a value that will bring you returns, materialistic returns, or change people's life if it's healthcare, if it's finance, how would you uh let uh maybe the un the people that are not able to access certain financial privileges to maybe start something off, microfinance. So there are many avenues in every industry which you can touch or impact with powerful tech. So my focus shifted uh quite drastically back then.

SPEAKER_00 3:12

And then to build on that, you've written three books. One of them is LLM ops in finance, and that's also building on your knowledge as you know, you've done your PhD in uh engineering, focusing on temporal knowledge graphs. What do you feel now that you've had this experience in regulatory body, in big tech, in a bank, that you know, senior executives are getting wrong when it comes to deploying AI or you know, LLMs at scale?

SPEAKER_05 3:42

So some of the things that that I noticed quite early is with the arrival of LLMs, everybody wanted to adopt them. Great. We have different sizes of models, and you know, even a child gets access to it, which is again good. But when you think of enterprise AI, when you deploy these models within the enterprise, you have to look at the bigger picture, which is operationalization at scale, not just that one-time experiment that you did and you had high, you know, maybe uh results on your testing, high accuracy, which is again good. But how does that fit in into your operational life cycle of that? Uh, maybe it's a department of retail. How does it fit in into their uh process pipeline? So, my book also focuses on operationalizing LLMs. So that was quite early when LLMs had just arrived that I thought of writing something which is applying my software development lifecycle mind, uh, wherein we always think about uh production ready software. We don't talk about experiments, they are just university experiments, right? Production grade software is what we deploy. But with ML and machine learning and LLMs and Gen AI, we are often talking about, we are stuck in the uh you know, life cycle of experiments, of I want to show you how great my model is, how great my use case is, which is great. But how does it fit into your um the larger ecosystem that exists within the organization? That is the operationalization, you know, life cycle, uh, which again is so different from the software development uh operationalization, wherein you had enterprise architecture certifications to enable people to architect in a way that it works. But today we lack that sort of uh architecture mindset with LLM ops. Uh so that is the focus of the book as well, and uh, which is something which we need to think about.

SPEAKER_01 5:29

The field of AI is changing so fast. Yes. Umly uh the reasoning models have slightly less than one and a half years old. Um and the capability seems to be doubling every six uh six months, uh depending on who you uh believe, somewhere four to six or seven months. Which uh unlocks completely new kinds of deployment possibilities. Yeah. From um small LLM-based applications to uh atomic agents uh uh orchestrated through workflow platforms uh are there. And now fully agentic uh AI, which is like yeah, uh taking decisions and uh designing its own workflow on the fly. With this mind-bending speed of development, yeah, how do you uh think about creating that yeah, that uh uh uh unlike software uh uh software development where the pace of change is slow and you can develop those best practices, think about those architectures, train your people. How do we do this in the field of AI with this pace of change?

SPEAKER_05 6:37

Yeah, that's a really good question. So within software development, our options were limited. We were tuned to um drive architectural development in a certain direction. Uh, the maturity of that we haven't reached with the new technologies. But however, we shouldn't um get distracted by the noise of the different technology that is launched in the market. So, as an individual learning something at the university, performing research, it's good to be aware of every single new tech that comes up and try it out, which is great. But when you again go back to organizational impact of enterprises who wants to create a real ROI, you need to look at where you are now in your readiness or your maturity of your AI stack or your data stack, even. And how do you use this technology to complement to uplift your uh gains from where you are? That's the bare minimum step you've got to look at. Today, people look at the new tech to say Agen TKI, I want it today on the table. It's like available on the fly. It is, but does it really suit your, does it embed well or integrate well with your current decision-making processes? Because Agen TAI, we are talking about autonomous decision making, which is even possible traditionally with ML models or uh if you automate a traditional system, that's again automation. So we have another way to automate using um Gen AI or Agen TKI. That's another way to automate. But we have to go back to the table to think about does it really add value to my existing life cycle? Or do I am I embarking on it, being aware that it's an experimental venture? And if it succeeds, I'm going to put effort to build a team that will help me to integrate this into my existing life cycle. Unless you integrate it, it's going to be isolated development and then a failure down the line.

unknown 8:22

Yeah.

SPEAKER_00 8:22

But you mentioned just to follow up on that, Brinda, experimenting and generating value. A lot of times that these can be in conflict with each other. Um, and so how do organizations reconcile and say and balance these two conflicting uh areas of you know pathways to realizing ROI? Is it that you see from your experience um some companies doing better versus others and what what are the factors that influence that?

SPEAKER_05 8:50

Yeah, good question. So almost many companies have AI labs set up which focuses on experimentation. But what they are not aware of is from the moment you experiment to say out of three experiments, two have succeeded, from that moment to integrate to your life cycle of the system or the organizational pipeline, it's another humongous effort. It's not an overnight effort to say, now my experiment has succeeded. I can overnight or maybe two, three months just plug and play into my uh existing pipeline because that requires organizational change, which is in the form of cultural change, in the form of mindset change, maybe a skill gap that is required to monitor these pipelines that you're bringing in. So you need to uh perform an AI readiness. In my mind, it's called AI readiness, organizational readiness for the new tech. It could be AI, it could be blockchain, a readiness which um highlights where you are today and what you need to kind of bridge the gap, to make uh the new experiments, you know, come to life.

unknown 9:52

Yeah.

SPEAKER_00 9:52

But for that, uh, if I may add, an organization needs to be self-aware about their current capabilities.

SPEAKER_05 9:58

Exactly. Exactly.

SPEAKER_00 9:59

But we're seeing a lot of times on social media, for example, companies talking a lot about their success stories when it comes to AI. So they already feel that they're far along the journey. But what you're saying is the readiness aspect needs to be there, which in itself is a transformation. So what's what's missing in your view? Because you've been in those situations yourself. You've gone in, whether it's your clients or whether you worked with stakeholders when you were working in the bank as well. What's missing there? Is it a fact that you feel people or companies need more time to understand this technology? Or what there are some other factors involved there?

SPEAKER_05 10:38

Yeah, so in today's age uh with digital media, right, all the success stories come out quite quickly, which is which is good again. At the same time, uh, you need to measure the success of uh this transformation, not as a one-time success story or a use case that worked one time, more of over a period of time, how did this team or this technology transform their business processes over a period of time? And for that, you need a good data strategy, AIA strategy, a business strategy, which comes top down from the board. And again, today's uh you know, day and age governance strategy, which again, risk and governance has to come in early to kind of check if the technology is worth pursuing, or would it be facing a roadblock down the line? So uh there are multiple uh, you know, hats that you need to wear, and this decision making has to be you know coming down from the top. So the top leadership awareness or the readiness matters more. They need to go through some courses to upgrade themselves to understand the pros and cons of the tech so that they're able to make the right decisions, which then percolates down. What we observe today is the bottom layer has started rolling, right? Like the organization, the teams have started to implement the projects already.

SPEAKER_03 11:49

Yeah.

SPEAKER_05 11:49

And then the pyramid at the top is kind of catching up with the new technology. And this gap is stuck. So you realize that with maybe sometimes two to three years down the line, when you have to revamp your entire AI strategy and begin from the scratch, to now again look at your readiness, your data transformation. So, what's important is your foundational core layer, right? Your data layer, your AI layer, the knowledge layer. Previously, we had data and information. Today, the knowledge layer is super critical with Agen TI. So there's a lot more that they have to uh look beyond the external successes that they have.

SPEAKER_01 12:26

So you uh highlighted that uh the bottom layer of the organization is moving faster and the uh the top is uh trying to catch up. Considering uh your uh wealth of experience dealing with the top layer, what are the challenges and opportunities that you see on how to drive and motivate or uh uh the top layer to become more proactive about this, not just beyond saying that, telling their team that I want something, but uh themselves embarking on that learning and understanding journey.

SPEAKER_05 13:01

Yeah, I think in um the past two years, I have seen a lot of leadership courses being launched where people get to understand the tech a bit more. And if you notice the different organizations, those that succeed uh fast or accelerate forward faster, are where the leaders themselves are tech savvy and they understand the tech enough to make those right decisions. Because the decisions percolate down to various teams and any change takes time. Once the change is made, again to re-shift the direction, again, it takes a lot to steer the wheel in different directions. So those that have succeeded are those where they have hired the right kind of resources to make those decisions very early. And uh those cases where things have slowed down a bit are those where there's a bit of a gap and everybody is catching up, and tech is changing every day. That is like said and done. But the faster you catch up at the top, the leaders need to be more accountable, I would say, rather than uh um just relying on the teams to be accountable, which is correct. But accountability should go from the bottom to the top. Like, for example, any AI project, a use case that have succeeded to demonstrate value and ROI. The ownership has to exist right from the start until you sunset that project. Uh the moment it goes live doesn't mean now the accountability is on just the automated agent or the system. But ownership should always be held by a human. Could be business, could be tech, could be risk or governance. You need to figure out the right mix that works.

SPEAKER_01 14:32

One practical uh example on this one, yeah. Uh maybe uh your insights will be very helpful uh or experience. So I was uh having a conversation last uh week with somebody who's uh uh a young guy, fully motivated to drive AI across the organization. And he's been given that mandate also, uh, that yes, go ahead. Um and he's uh now working to get the people trained, uh uh uh uh become more aware of all the possibilities, etc. However, within that mandate, uh the message is that yeah, middle management and people who are actually doing the uh work, focus on those. Yes. The top management, yes, uh not so much, they are already very busy. If you need something, we can have a 15-minute conversation with them. Yeah, but other than that, this and this is uh something that uh is um uh from my uh experience and interactions is fairly common. True. This is one of those challenges how where how do we uh how does one uh reach out and say the people at the top also need to be part of this journey from the beginning, otherwise it is a it is going to be a challenge with the transformation.

SPEAKER_05 15:50

Yeah, that's an interesting question. So that problem is quite common. Uh but again, uh the solution cannot be provided by we can try as much. Like you, it's good to kind of have stakeholder engagement monthly or uh fortnightly or whatever works for that project, and the depending on the criticality of the project. That's an attempt the project um the team can do from their side.

unknown 16:12

Completely.

SPEAKER_05 16:12

And um, so with that, you kind of bring in stakeholders, um, feedback or inputs quite early. And uh if that doesn't solve the problem, it's a management decision to make that every project might probably run through every quarterly basis. You run through prioritize and you look at the status or the progress. Um so most likely big organizations do have those forums where the leadership steps in to look at the various projects that are uh you know ongoing and the progress and the impact that it brings in.

SPEAKER_01 16:45

That sounds interesting. Yeah, forming something like an AI council would be, yeah.

SPEAKER_05 16:49

Definitely. At least the proposition could be made by the project teams. And largely what I've observed is uh when the when they proactively reach out, most likely they would agree. Because they kind of also understand what the teams are going through. So we need to uh step up to kind of uh push the boundaries if needed to again bring the uh leadership and stakeholders right from the start.

SPEAKER_00 17:11

Yeah. That I mean, what you said suggested, I think it works really well, but a lot of it I think in c most companies folks are reluctant to seize like their control over certain areas that they are working on or that they're responsible for. And I think what happens, what you're saying in AI councils, it means that everybody is contributing to the project or whatever they want to achieve, but at the same time, people want to still hold responsibility for their specific area, which in a way also hinders the progress of those projects and maybe the transformation at large. So, what's your advice to maybe CEOs or folks who are searching for, like that you said, they're very keen on generating ROI. They've invested a lot of money, but there's something that's jeopardizing their possibility of generating that ROI.

SPEAKER_05 18:05

Yeah, so uh today people look at AI as a plug and play tool, even the CEOs, because we are now saying you can access it at your fingertips, everything is available, you know, right in uh in one hour, two hours. But the way to look at it is to build your core capability of AI, the foundational capability, AI foundation, you may call it, of the organization. So for that, you have to look at different pillars like your data pillar, the data readiness, your knowledge readiness, and then look at your tech readiness. Maybe you have data and knowledge, but you're not ready in terms of operationalizing AI, which requires a different mindset, not just monitoring, but also looking at mitigating factors. What are the kind of risks that the system uh will bring in? And how do I kill switch if I need to, or how do I mitigate the risk for the various types of risk that comes in today? Because most of the AI pipelines today are hybrid. They are not purely Gen AI, they are not purely traditional AI, they're a hybrid uh platform. So the risk that each part of the system brings in is different. That doesn't mean we need to make the entire system as high risk and over-govern it. So governing it right is also important. So they need to look at their organizational readiness, and ROI may not be the immediate uh target. Sometimes looking at just the short-term target doesn't help over a longer term. So you've got to bring in the initial ROI using maybe internal productivity tools of AI and to uplift the incremental revenue, at the same time, build your AI capability, the core foundational capability. It could be as simple as where will you host your LLM. Say now you've been given the smallest LLM that works perfectly with highest forms of accuracy. Do we have an environment or an on-prem uh you know landing zone where this LLM can be hosted and monitored and you know, guardrails implemented? So all that is the surrounding um effort that is required. Yeah.

SPEAKER_00 20:02

But Brinda, a lot of CEOs may say, oh, AI governance, I think it it's like another added cost, it's gonna reduce my possibility of generating ROI in many ways. What's your response? I'm sure now that you're this is like your core focus.

SPEAKER_03 20:17

Yeah, yeah.

SPEAKER_00 20:18

What's the feedback and what are you hearing from the folks that you're speaking to?

SPEAKER_05 20:23

Yeah, so I understand where they're coming from because traditionally governance came in as your less than a last mile effort, right? Where then they come in to check things. But what I feel is um I have a slightly different point of view here. So I feel this is a myth. Uh basically because um governance has to be embedded in your architectural or your design lifecycle.

SPEAKER_03 20:44

Yeah.

SPEAKER_05 20:44

So it's no longer an additional effort, but rather you're bringing things into your architecture so that you can go live faster, innovate faster. It's like you're you have a maybe a race car which can which can you can drive very, very fast with high speed. But say, for example, you know that it doesn't have brakes, would you drive it at a high speed? No. So these are like your braking system. As simple as that, governance can be seen as your braking system, wherein people can safely innovate, knowing that if things go wrong, I have a way to mitigate. So governance is not like a road blocker, like it was seen earlier, because traditional governance had policies and checklists and more document-heavy. But today it's changing. We are seeing the change. There are tech experts that are coming in into the governance space to guide through the technical architecture so that it's more secure and more governed. The kill switches are applied when needed. And those are the extreme cases. But the normal cases, the guardrails would suffice. And the different kinds of guardrails for different AI is different. So there are many types of Gen AI, a marketing gen AI or assistive AI. So every AI requires different treatment.

SPEAKER_03 21:51

Yeah.

SPEAKER_05 21:51

So this is an evolving space. Uh, but sooner or later, this mind shift will happen where people stop looking at governance as added responsibility. Or effort and look at it from embedding governance into their architectural life cycle.

SPEAKER_00 22:05

I think what I see from my lot of the first reactions of folks who are have the responsibility of making these decisions will be like, let's call a consultant and have their view and they'll design it for us. What's your view on that approach?

SPEAKER_05 22:20

I've seen that problem firsthand. And one of the things that I have issues with that is with this kind of large-scale AI that you're developing, you need ownership and accountability. As simple as that. Not even the technical skill set of consultant. That is great. They're bringing in some adding great value to you. But who's accountable for it? You need somebody within the organization with a core team that is accountable for the decisions that the governance team is making or your AI COE team is making in terms of prioritizing the AI use cases. Sure, the consultant can come in, add some color to it, add value in terms of helping you. So taking help from consultants is great, but again, ownership responsibility, unless uh it's taken by the organizational stakeholders, it's not going to scale. It's like a temporary fix that you're giving at that moment, and that hole is going to leak again. You will spend a larger amount to fix that. So it's always important to build a team with the right skill set right at the start. Yeah. Consultants could be helping you, but they couldn't could not be the core players of your game.

SPEAKER_01 23:27

One quick question. Uh, since we were talking about governments, yes, uh and risk, where does the true risk of AI implementation, AI transformation lie? We talked about consultants with uh yeah, coming in with technical expertise, yes. But uh uh what in your opinion and experience is uh is the best way to identify and uh somehow control and mitigate such AI-related risk.

SPEAKER_05 23:56

So AI risk falls under different buckets, right? There's the technological risk, there's the organizational risk and regulatory risk. So the technological risk can be solved with tech. That is one. You can build your tech capability, the teams to solve it. Again, the issue is um the decision making that happens at the leadership level. Uh so that is a gray area where things are just picking up because technology can only solve so much, but then it's all tied to your organizational decision-making checkpoints, be it a retail, be it a private banking customer use case, be it any department that you pick. There are critical decision-making checkpoints with which AI has to be embedded within. So those are the risky areas because uh you need to define your ownership and accountability there. Unless you provide that clarity to the individual, they are not going to be accountable or solving the problem for real. They would provide short-term fixes, which works for six months. Even if it works for six months, it's it's um hidden, you know, uh, you will incur a lot of cost to fix that six months issue. It's not worth it to even have a short-term fix for six months. You need to look at a fix that works, uh, that works for the organization depending on their culture, their risk appetite, their innovation appetite. Uh, what sort of technology to invest in so that they are covered, the insurance cover, like your insurance coverage, they are the governance risk coverage is there for two to three years at least.

SPEAKER_01 25:21

So you seem to be hinting at some kind of a process change uh at these critical decision-making points. Yes. Also, uh one other uh topic that uh often comes up is um this responsibility lies still with the human or the human in the loop. But one of the challenges that comes uh that becomes quite evident is that most of the time these AI can work at many times the speed that a human can uh can. Which essentially means that uh how do you design a system where the human is still able to control the immense amount of output throughput that uh uh an AI-enabled pipeline can uh can create. And how do we fix the responsibility to the human where the human is not just a scapegoat, yes, but actually has the time to understand, analyze, and take the right direct uh decision.

SPEAKER_05 26:18

Yeah, uh great question. So, which is why when when I talk about governance through design or architecture, even the smallest aspect of human in the loop has to be designed in your architecture. So you can use AI to accelerate, like for example, we use computers to uh compute faster. But that doesn't mean we outsource our decision making to the compute. We just outsource the compute to the you know computer. So similarly, here you can use AI to accelerate the areas to automate it faster, but you need to design the human to come in at the right checkpoints, which is very different for every use case is different. There's no one size fits all. And human in the loop is just one of the controls that is proposed. So every control that Gen AI uh has in terms of guardrails or uh maybe process control or technological controls has to be plugged and played in the architecture in the right form. And that uh walkthrough or review is important, architecture review, which has to happen with the governance team members as well, which today is lacking. There's an architecture committee to review the architecture, give uh like a green signal out, but there is no risk and governance personnel involved in there to verify if this caters to the risk appetite of the bank or is even human in the loop applied in the right manner rather than being used as otherwise, right? Which is typically the not the right way of uh using human in the loop. Okay. It's a very important position, by the way, human in the loop, staying accountable, it's a very important position. And as AI comes into like AI is not going to take away our jobs, right? There's going to be a shift in the job market. So AI reviewers, so human in the loop are typically the reviewers who are super critical. So they shouldn't be using AI to review. They're supposed to be reviewing. So you need a guardrail for them to verify whether it is really human that reviewed the output. So there's so much nuances to that role itself.

SPEAKER_00 28:15

And how much, in your view, Brinda, does the existing culture of a company influence the type of AI governance framework that they have in place or they want to implement and how they implement that framework?

SPEAKER_05 28:29

Yeah, so any transformational effort is tied closely with the culture of the company. The tech is the easy part to solve, right? So the the culture has to be kind of coached and transformed even before you bring in the right tech. So definitely it's the same, it's the same situation for risk or any sort of governance change. Um, so you need to make them understand through various sessions or explain it, uh, you know, have monthly kind of casual uh, you know, presentations or sessions to make them understand why you're bringing in governance through design, for example. That's one of the transformational changes that they would see, right, as a risk and compliance, stepping in right at the last to now like coming, you know, hand along with them in every step as they design, as they architect. So that's a huge shift for even as a developer or any kind of LLM, you know, developer that works on the project. So organizational culture is definitely important. And this is very deliberately should be steered if you want to adopt the latest and the greatest. The culture has to be more open and more resilient because things are going to change and you should be able to make those decisions on the fly while you're experimenting. Of course, once it moves to production, it's more stable and structured. While you're experimenting, you need to have the resilience and the change mentality to be able to make those decisions, to steer it in a direction that works for your organization. Because a simple chatbot could be applied differently by various organizations. An educational sector may say, I don't even want LLMs, like I want exact information to be shown to my customers. So I don't want LLM to hallucinate there and even rewrite my statement. Every statement has to be rendered the same. So you actually don't need LLMs there. It's more of a stable software automation. So that differentiation has to be done at the organizational level. Not every use case uh has to be solved using a Gen AI or LLM.

SPEAKER_00 30:27

That last point, actually, because most folks think that AI can solve all their problems. And then you've seen it from three different perspectives regulator, big tech, bank, and actually the fourth perspective is now your own company. Every obviously every environment culture has its own pros and cons. But what do you think if you can just elaborate worked well in those sites? I know it's quite a big question in each of those situations that you felt that oh, yeah, this sort of thinking and culture is productive to AI transformation.

SPEAKER_05 31:00

Good question. So if you're looking at AI transformation, uh you have to look at your uh data teams especially. Because usually data are your traditional, you know, traditional uh teams that have uh worked in a very structured manner. And any change like your ETL pipelines or anything, any change is a big change for them. It's a change request. So you have to look at end-to-end, you know, the teams that are involved and governance again. That's a very serious mode. You know, you're governing, you're taking accountability, real accountability. So you want to be very careful on what you approve and uh not approve. So together, they need to find a middle ground where they are agreeable to innovate. So for innovation, you have to be a little less, you know, um little less on governance, I would say. The the risk appetite should be a little higher. And you need to monitor. It doesn't mean you need to innovate and let it fly off and just sit and watch. You need to monitor. So that is where the operationalization, the LLM ops, uh, which includes monitoring and mitigating action for every kind of risk type that comes with monitoring. So this has to go hand in hand. So you have a tech team that has its responsibility and the data, and then you're looking at AI. So looking at your ecosystem of the teams that adjacent teams that you're going to work with, the culture has to kind of find a common footing. That is where things have worked. It's difficult because every team has a different uh responsibility and role in the organization. But they have to dedicate 30 or 20% of their time with a different wear a different hat for the purpose of innovation. And if it doesn't work, fine, go back to your previous hat and then do your day job, which is just purely data or purely governance. But they need to step up a bit and uh, you know, cross the boundary a bit with that 20% of change.

SPEAKER_00 32:49

And what what makes, because obviously you've straddled these different environments, what help you to achieve success? Because they're very, very different environments. And you can share some examples of what worked or what didn't work for you.

SPEAKER_05 33:02

Yeah, so what uh usually works is bringing stakeholders right at the start. So different organizations functions differently, but uh when you join in first, if you work in isolation, that's not going to work for sure. Yeah because you need to understand and form a rapport with your uh organizational leaders and the stakeholders. It could be tech, it could be your end customers, it could be retail business users. So bringing them on early gives them a flavor of what you're trying to do. Because the governance role that I took up is again a different mindset that I brought in as from the tech practitioner angle, which was quite different from how traditional governance worked. So engaging them early in conversation, prioritizing that engagement, although that is not building my core capability. 20% or 30% of your time has to go in to engage with the adjacent teams and the leaders to get to know what are their pain points. And you need to start off by solving some of their pain points. That is the motivation for them to give you the time so that, and then support to kind of let you keep solving one after another, you know, all their pain points. So you need to sense uh where each one is coming in and what are their priorities and pain points, and aim to solve some of their pain points at the same time, build your core capability. That would be well appreciated.

SPEAKER_00 34:18

But let's say it can work both ways. So coming from big tech, obviously they they think you have the knowledge and experience to help them accelerate the transformation, but it could be the other way, like what worked in big tech is not going to work in a bank.

SPEAKER_05 34:30

Yeah, uh every every company is different. The organization culture is different, the strategy you work with is very different. Like I worked for research organizations, startups. So I'm quite quick in shifting my mindset according to customize myself to the organization. So usually in big tech, you are more on innovating and pushing the boundaries, bringing in those edge use cases that people haven't done before. And you want to be the first player to at the same time realize the ROI. You don't want to invest too much of your time where ROI doesn't get generated. So again, it's like multiple uh criteria to meet and move forward with all the criteria met. And with banks and other organizations, they're more conservative.

SPEAKER_01 35:16

Yes.

SPEAKER_05 35:16

You need to uh uh slow down a bit, look at where each one is coming from, what are their priorities, and slowly tackle each of their priorities, contribute in a way that works because every team could be at a different cultural mode. Like a risk and governance is more of uh, you know, more serious on what do I allow? And mostly it's it's no-go. It's like I don't allow this because we want to safeguard and be more conservative. Uh, but it's kind of eye-opening because people are now opening up to conditional maybe allowance. Previously it was like a yes or no.

SPEAKER_03 35:50

Yeah.

SPEAKER_05 35:51

And if it's a no, it's a hard no. You come back again, it's again a no. But today there are ways to kind of build the architecture, reconstruct, and then maybe reappear to say, hey, I've solved wherever you mention there are you know risk, high risk, or gaps. I've kind of fixed it. So now do I get a chance to move forward?

SPEAKER_01 36:08

Is there some uh experience that you can share with us which is uh which has been one of your hardest conversations on the topic of AI or AI risk with leadership? Uh and uh what did you come away uh from that conversation with?

SPEAKER_05 36:25

What were the There are a few, right? Like one is people want ROI or speed, accelerate. So I have I have my Gen AI team experimenting team. I have resources um that are willing to work on AI projects. Now I want to accelerate and bring ROI in two to three months. But then uh I don't know that it's I mean you can bring in something in two to three months that is not going to sustain for a long period of time. So what is hard is to make them understand that yes, you can bring in the revenue, maybe 50% of the effort can go in to bring those initial low-hanging uh you know use cases, but invest 50% of your time, all the team's time to also build the core capability. It could be as simple as prompt engineering as a capability, like two years ago when LLMs came in. So we need to look at where we want to go in your AI strategy, which is why, again, the AI strategy is super important, and that is what you need to live by for the next two, three years. So you have to create your strategy in such a way it's relevant for the organization. So now I would say for a tech innovator, so your strategy has to move with the tech. So you could give an excuse that tech is moving so fast, so therefore my AI strategy will move. Okay, for uh maybe a tech company, that makes little sense. But I would say for other organizations, no, because your strategy is dependent on where you want to head in the next two to three years, and you should be aware where you are today. Tech could keep changing. There could be AGI, there could be Agent Tech, there could be LLM. But your direction or your North Star should not drift too much. You could replace a machine learning system with a Gen AI system.

SPEAKER_04 38:02

Yes.

SPEAKER_05 38:02

That is possible. That's architectural, technological change. But your strategy should pretty much remain the same for the two to three years. So creating the right strategy is where people struggle. And when you don't create one, you kind of revisit the same again and again every three years with realizing very minimal ROI, sometimes even zero ROI. Right. And then you call it the experimental AI lab. So now I'm embarking on my next, you know.

SPEAKER_00 38:28

Or they can post on LinkedIn all their success stories without worrying about ROI. That's the ROI, maybe generating media content from it.

SPEAKER_05 38:37

Yeah, that kind of gives the wrong impression sometimes.

SPEAKER_00 38:40

Internally and externally, also.

SPEAKER_05 38:42

Yeah, true, true. But it's good to be rooted to reality within your organization. I mean, it doesn't matter, the perception from the outside could be something else.

SPEAKER_01 38:51

We've spoken about the challenges on the strategy part. Yeah, um, uh we've uh spoken about challenges uh related to risk. But let's say uh once we have some clarity, yeah, uh the the leadership has given us clarity, this is what we want to do. As soon as we start getting into the execution part, yeah, what are the key challenges in, let's say, building a team and designing that team's interaction with the rest of the organization? How do you go about embedding these uh embedding AI across an organization and what are the key challenges that you have faced in life?

SPEAKER_05 39:29

Uh yeah, this very good question. So today, once the AI project is a success, right? A use case, they want to launch to production or maybe deploy on the cloud, as simple as a scenario where I want to adopt cloud. But then how are you doing so? I have one cloud architect or one engineer. But I would say building a team is very important because even if it's as simple as you want to be on the cloud, you need to have a cloud infra person, you need to have a cloud strategy person and somebody who's equipped with the certain cloud that you're interested in, knowledge about architecting over there, and a business strategy person to kind of strategize what you want to do with the use case, even as simple as deployment. So, having said that, now you're looking at an AI kind of a team. That's a different um proportion these days. So traditionally you had software engineers, data scientists, and they would run the show end-to-end. But today you need to have MLOps or LLM ops, operationalization folks, uh, 30%. And then you may need uh 20% of uh strategizing your uh creating your new use cases or tuning your ROI for the business, which is again from the business, you need a rep. And then you need one or two governance folks to sit there and uh go hand in hand in architecting, helping them to architect. And then you need data scientists. So, data scientists, traditionally, it was only the data scientists that took up all these roles.

SPEAKER_03 40:53

Yes.

SPEAKER_05 40:53

So now there's a pyramid structure that I look at it as. And they are expected to deliver more than what they they should because now they are seniors and then they're they constitute 50% or higher. Now the pressure is more. No, they should be maybe 20%. You need to have like junior architects, junior prompt engineers. And this entire pyramid has to scale in with proportion. You can't have the bottom layer, you know, scale down and the top layer uh scale up, then it that disproportion doesn't work.

SPEAKER_01 41:37

Even smart, uh smart and famous people like uh Andre Karpathi uh have started saying, especially this year, that they have stopped writing code. Yeah, and they primarily write uh prompts. Of course, the reality might be more nuanced in practical uh uh uh development teams, yes. But uh the key increase in capability, software engineering capability of AI, what kind of impact is it having on the structure and composition of the teams that we must build for uh yeah AI? Yeah, good question.

SPEAKER_05 42:10

So to that point of coding, right? It's a very interesting area. So traditionally, we wrote code and we learnt it through practical application. But today for the junior pass out of the universities, there is no such avenue. They can go to an LLM to kind of generate a code. Uh but again, I would say the the skill of being able to code is super critical because as LLMs generate more and more code, you need reviewers to review. The faster you review, the faster you roll out. Because you're never going to rely completely on LLMs to deploy the whole thing end-to-end. It doesn't make sense. Then who's accountable again? Right? There's a human that needs to be accountable. So that means you need more reviewers who have more expertise of coding, expertise in their lives, which will now diminish with uh LLMs, you know, Advent and uh, you know, people using more LLMs from universities. It's harder to get them to believe there are no LLMs and learn it the traditional way. So I think that's an important skill to hold, whichever way. And you can use LLMs to accelerate your work, but the faster. Like I'm a software developer 20 years ago. So when I use my LLMs, I can review and say this is correct, this is wrong, and then move things forward. But if I didn't have that decision-making ability, I would either blindly rely on the code to fail and then let me know that, oh, this didn't work. Now use another LLM to again do the failure analysis. So relying too much on LLM is like you're automating your own decision making ability, which is not good. So people fail and then come back to learning more of the foundational uh principles and foundational engineering foundation, basically. Your computer science degrees and yeah.

SPEAKER_00 43:51

And then from just to build on that, Brenda, because of this development, how do you see the structure, the teams evolving in the next six? To 12 months, or maybe like two to three years. Is it that they become less dependent, like you said, on software developers, or like you said, you still feel it's like a core skill that people should have to be able to build better uh AI products and solutions?

SPEAKER_05 44:16

Yeah. So see the teams that kind of replace their developers with uh Gen AI will realize sooner that they need to have strong reviewers if they are replacing. So uh there is no shortcut to this uh ecosystem in enterprise, right? If you are a personal developer blogger, or you could just replace things with something and you can be done with it. But at an enterprise level, you need to evolve and be responsible as you evolve, which means you replace 10 software engineers with 10 AI coding agents. Now you need to have two to three reviewers to ensure that it's uh it's coding the right way or it's coding exactly what you want. And things will mature. After one year, when you know that the software is doing exactly what you wanted it to do, so it behaves like a deterministic system. Now you can then automate it without any kind of review. So every area that you are replacing with AI, you definitely need one or two strong reviewers to review for one year. So this is an iterative method that companies need to follow. Uh, even the prompt engineers that they hire, the prompting strategy could change depending on the different models. And you could be writing the wrong strategy and maybe opening up to more risk. So the guardrails also have to change. So everything is an evolving uh you know unit at this stage. So you need to work in a way that you are not, there's no shortcut. As you're replacing, you need to bring in the expertise to monitor the replacement until it gets stable to a stage where it's like a software that is automated, and then you let go that portion.

SPEAKER_00 45:49

So do you think it's fair that companies are saying they are using AI and therefore they need less people to do the same job? Like the job cuts and everything. Do you think that's fair?

SPEAKER_05 45:59

If you go one level deeper, yes, you can use AI to accelerate.

SPEAKER_00 46:02

Yeah.

SPEAKER_05 46:03

But uh we cannot blindly assume that uh because who knows if the risk appears at some point, you will end up spending more money. Maybe you had less people, but then you uh spend more money to fix the risk. So you need to monitor for a while before um assuming or making it like a statement that uh you definitely can use AI and uh make your, you know, the redundancy, reduce the redundancy. It you have to look into what type of redundancy. If it's operational redundancy, definitely that'll work. But if it's human redundancy, like what basis have you decided this is a redundancy? So you need to observe for a while, a year or two to see how things stabilize, if that's really true, if they're trying to attempt it. There are many consultancies that try to do it and then shift it back to the old style.

SPEAKER_00 46:53

That's how they make money. Is they they try to solve a problem, but they create another problem so that they can be continuously employed in that company.

SPEAKER_01 47:02

Sounds like a money machine, yeah. Your perspective on this. If you could redesign AI governance from scratch, yeah, uh knowing what you know today with all of your experience, how would you go about it?

SPEAKER_05 47:16

Yeah, so in my uh new redefinition, right, it would be embedded within your development lifecycle. So traditionally you had SDLC, the software development lifecycle, then you had your ML lifecycle, and then your agent tech lifecycle with your context and knowledge layer and things like that. So define the life cycle, let the governance itself define the entire development lifecycle and embed itself wherever relevant, maybe right at the start before prioritization, to do an initial risk assessment to see is this project worth pursuing or is it really high risk? Uh, does it involve as simple as PIA information, which we are not quite comfortable to launch as a product? Because all this risk sits in different minds of the different people. You wouldn't know as such, but when you come together as your AI council or as bring the stakeholders together, they would help you highlight whether the risk of the system is low or a high. So that is one area of embedment of the governance lifecycle into your AI lifecycle. The second would be in your architectural decision making. While you clear off your architecture, um, you could have a risk person to revisit to see have I brought in the governance elements into it. Like your traditional security, for example. Like you have a checklist to say you need to ensure you know XYZ is checked before you launch a product. And the lastly, of course, governance comes, steps in to review everything, but that is your last, you know, it becomes a basic organic next step if you did the prerequisites.

SPEAKER_01 48:46

On the topic of security, yeah, these days uh anthropic mythos is, for example, uh a lot in news. And uh, of course, we still don't know the f true reality uh because very few people have access to it. But the news is that the software layer which forms the core uh infrastructure, the core architecture of of the internet, yeah, all of the um all of the for example browsers and uh uh uh cloud infrastructure, etc. All of them have had uh bugs or uh potential exploits that were discovered by this AI and actively uh exploited. So one of the big uh reasons why uh mythos is being so much talked about is because even though earlier models were able to identify some isolated ones, uh isolated potential exploits, they were not able to string together uh all the activities needed to be to uh uh to harm uh the system. But mythos apparently can. So even if it is yeah, only partially true, the trajectory of development is that our models will continue to keep getting better. In a scenario like this, yes, where uh the AI development poses such a stark and imminent threat to all of the software uh across the world, how do we go about thinking and redesigning our approach to cybersecurity in enterprises?

SPEAKER_05 50:31

So that's a really good example, right? Where AI has helped to identify, which we couldn't identify as humans for the longest period of time. So if you were to start it today to design something, we would definitely leverage something like that to bring in at an early stage, at your architectural stage, to identify those gaps and areas and loopholes, which is really helpful and kind of accelerating as a security team or a governance team. So that's a good way to embed and use that as an advantageous manner. But definitely we need to go back to look at all the systems that are there and re-look into it more as an exercise of learning, not as a you know, blame game of like it is possible to make errors while building softwares, right? So yeah.

SPEAKER_01 51:22

But uh one of the key threats is that the speed of exploitation of these has gone down from months to hours. Yeah. And this uh for cybersecurity teams, I worry for my for our cybersecurity teams across companies. Yeah. And I'm sure all of them are also thinking about and looking for advice on how they can become more adaptive, how they can uh so that they can respond to this completely new speed uh of attack. Detection and detection and fix without have uh without having to bring down the system saying that okay, we will fix it in two days, but for those two days, the system has to be down.

unknown 52:01

Yeah.

SPEAKER_05 52:02

Yeah. So this is another example to show how a system or a technology that is coming off use in a different, you know, wearing a different hat, in the hands of the wrong. Again, we are speaking about exploitation of the loopholes and the areas. So, which is why governance, when you look at governance, it doesn't stop with policies. When such a technology is brought to the table, for example, you need to look at what would happen for the different personas, characters involved, right? Not just the enterprises and customers and the end users, but also the maybe the hackers. How can they misuse it to identify it faster than others and then you know leverage that? So it has to be in the hands of the responsible initially. It shouldn't be launched in a way that everybody is able to use it as a public technology. That is number one. Like if there was a World Governance Council or something, they need to look at every technology from the lens of end-to-end. How can it impact the different uh categories of people and involved? Right. Had they thought through that, definitely the proposition would have been that maybe enterprise use it, having a clause or you know, contracts saying use it for the right purpose. And I think those are also available, right? We are used to hearing such cases where, okay, this is this must be used for the purpose that you have requested it for, not for any and every kind of purpose, which is the case today. So now we are reacting to it. But hopefully, looking at such incidents, the governance team or the councils come up with a structured way to monitor and uh launch as they launch the technological, you know, big big players. They go through a structure of governance there.

SPEAKER_00 53:42

Right.

SPEAKER_05 53:43

Who should be able to access and yeah.

SPEAKER_00 53:45

Definitely a key space to watch out for the developments are happening thick and fast. Excellent. So that brings us the end to the first part of our um uh session. We move on to the next session, which is called Rapid Fire. Which is um yes, pretty spontaneous, um, short answers. So let's get started. Okay, Brenda. Chat GPT, Gemini or Claude.

SPEAKER_05 54:14

Gemini.

SPEAKER_00 54:15

Gemini. That's interesting. Is that that must be like your previous affiliation? No, I feel a good guy.

SPEAKER_05 54:24

I feel you need to understand they are like personalities, right? Three different personalities. Once you understand what they are good at, you use them for that purpose. Like you don't want to over-rely on them. Maybe Claude for coding, of course. It's uh it cannot be placed on the same list. Like Claude is more of coding and things like that.

SPEAKER_01 54:41

But in fact, here I might uh uh share a very uh interesting personal experience I had with Gemini last week. Yeah, uh so Gemini had uh generated some output, and I was uh and it was fairly verbose. Yeah. So I asked Gemini to check with another model if this is uh alright or if there is something we can do uh about this. And the response for Gemini is no need to check with another model. Yes. I am telling you this is the right thing to do. I found this quite hilarious.

SPEAKER_05 55:16

Yeah, I think all these models have edge cases where you know they react in a way it's kind of interesting. But uh, yeah, depending on the purpose that you're using and yeah.

SPEAKER_00 55:27

Okay, next one. Head of data governance or head of AI governance?

SPEAKER_05 55:31

Head of AI governance for sure. Okay.

SPEAKER_01 55:34

Centralized AI teams or decentralized AI teams?

SPEAKER_05 55:37

Interesting. If you're going to scale, decentralized. If you're starting off, centralized.

SPEAKER_00 55:41

What's the most overrated AI trend right now?

SPEAKER_01 55:45

Um there are too many rapid fire, yeah. Just fire away.

SPEAKER_05 55:56

Maybe AI in education.

SPEAKER_01 55:57

Okay. Most underrated trend in AI.

SPEAKER_05 56:02

I think the productivity that AI can bring in, not many are aware of how you can stitch the agents together, for example, as simple as that, to even enhance your personal productivity to begin with. Once they get a hang of that, I'm sure enterprises will start steering more towards that to increase the personal productivity of every employee. Then compounding effect comes to play, right?

SPEAKER_00 56:23

If you could implement one AI regulation globally tomorrow, what would it be?

SPEAKER_05 56:28

I think before launching any of these technologies into the public hands, right? Like think through different, wear different hats and look at it from different lengths, you know, different um sort of regional uh exposure that the technology is going to have, the sensitivities that it brings in to the humans, the children, the elderly. Look at all these aspects before you launch. Because I often do come across like technology overtaking, um, as in you have a huge elderly population, for example, with a bank or et cetera. They are slow to understanding or even comprehending smallest change. And when you bring in some change in their app through AI or any tech, they take time to understand and it creates such a hindrance for the category that requires more assistance. We can learn and upskill, but they can't. But today things are being launched blindly and then reactively fixed over a period of time. But I would say a more thoughtful move should be helpful.

SPEAKER_00 57:26

Great. That's it. Thank you so much, Vinda. Thank you for joining us. Fascinating conversation. Great to have you, and thanks to our audience for joining us. Uh, please uh like and subscribe to us on YouTube, Spotify, and Apple Podcasts. Thank you.