Credo AI founder and CEO Navrina Singh will present two types of responsible AI: regulation-induced and value-induced. Regulations are changing rapidly—for financial services, there’s SR11-7, GDPR has AI implications, and regulation has become a top legislative agenda item. Some 60 countries are designing strategies and there are 1,000 responsible AI frameworks in development across OECD (the Organization for Economic Co-operation and Development) countries. Perhaps of most recent interest, Singh discusses New York City’s hiring bias law and what her customers are doing to comply by January 2023. Some regulations are still ambiguous, such as “disparate impact” referenced in the New York law. Singh explains how you can integrate appropriate bias monitoring and compliance tooling into your product to determine if you’re compliant and whether your latest model aligns with your business’s established values. Only then is it safe to deploy and serve your customers with a model. A technology leader with over 18 years of experience in enterprise SaaS, AI and mobile, Singh has held multiple product and business leadership roles at Microsoft and Qualcomm. Her company, Credo AI, specializes in trust, safety, and transparency for many industries—particularly for sensitive ones like financial services, human resources, and government.

👉 Check out more here: https://scl.ai/3SnFrMd

Next up, I'm excited to introduce Navrina Singh.

Navrina is the founder and CEO of Cradle AI, a governance platform empowering enterprises to deliver responsible AI.

A technology leader with over 18 years experience in enterprise SaaS, AI, and mobile, Navrina has held multiple product and business leadership roles at Microsoft and Qualcomm.

Department of Commerce National Artificial Intelligence advisory committee, which advises the president and the National AI Initiative Office.

Navrina is an executive board member of Mozilla, guiding their trustworthy AI charter.

Navrina is also a young global leader with the World Economic Forum and was on their future council for AI guiding policies and the regulations in responsible AI.

Naveena holds a master's in Electrical and Computer Engineering from University of Wisconsin, Madison, an MBA from University of Southern California, and a bachelor in Electronics and TANY Communication Engineering from India.

I am so excited to be here at the Scale AI Transform X again this year.

And today I'm gonna be talking about 6 myths about responsible AI governance.

By way of introduction, I'm Navreena Singh.

We are building industry's first SaaS platform that helps you figure out risks within your artificial intelligence systems and proactively manage those risks so that you can deliver responsible AI at scale.

I'm going to be taking you through this journey of how responsible AI has become 1 of the top areas, emerging areas for AI enterprises to be focusing on.

And at the end of the presentation, I'm hoping that you can walk away with steps that you can take today to start on your AI governance journey.

But before we get started, given that responsible AI governance is a brand new area, and there's a lot of conversations happening from the newsroom to the boardroom around how enterprises are going to call themselves accountable in this new age of artificial intelligence, let's ground ourselves in what responsible AI really means and how you can tackle it.

At Credo AI, we define responsible AI as artificial intelligence that is in service of humanity.

And governance is a mechanism to accomplish your responsible AI goals.

So responsible AI governance is the discipline of designing, developing, and deploying artificial intelligence, not only to manage the unintended consequences, but also to engender trust.

And if you're able to do that, you'll be able to scale artificial intelligence with confidence in service of your business as well as society.

You know, it's interesting, recently there has been an uptick in the conversations around responsible AI and how critical it is to enterprises.

Recently, a survey done by Appen published in 2022 called the State of AI Report indicates that 93% of the respondents believe that responsible AI and its governance is going to be the foundation of all the successful projects moving forward.

Hence, there's a sense of urgency in not only understanding how you can use and develop artificial intelligence responsibly, but the steps that you can take today to make that happen.

And today I'm here to really demystify Responsible AI for you, and also highlight some of the core myths that we are hearing in this ecosystem.

Now, the myth number 1 is responsible AI governance is the responsibility of a data scientist.

If you take a step back and think about the different stakeholders that are involved in the design, development, use, and the application, procurement of artificial intelligence, there are multiple stakeholders.

And 1 of the core things that we are finding is that the companies that are able to deliver on responsible AI at scale are able to bring in that multi-stakeholder perspective.

They're able to bring in that diverse perspective from the business stakeholders coming from compliance, audit, policy, risk, and they're able to bridge the gap, that oversight deficit that exists with the technical teams, by making sure that there is not only an understanding among these stakeholders, but there's also a shared accountability of the implications of artificial intelligence.

Later in the presentation, I'll be talking about the translation imperative.

When you have all these different stakeholders from your C-level all the way to your data scientists to machine learning engineers to your governance managers involved in a responsible AI initiative, it is critical to not only align on what does good look like, but also a shared understanding of what success would mean once those responsible AI objectives are met.

And 1 of the core things that we are finding in our Responsible AI journey is how critical it is to involve the actual consumers, the end users who are getting impacted by artificial intelligence technologies.

Now, this is critical because as you can imagine, everything from a loan application to insurance models, to facial recognition, to risk scoring algorithms.

In all these scenarios, there's a lot of unknowns till you actually deploy these systems out in production.

Hence feedback from individuals who are impacted by these AI systems is imperative.

Myth number 2, responsible AI governance is only about fairness.

You know, recently there has been a huge social movement around artificial intelligence, really focusing on how unintended consequences are showing up for demographics that might not be represented in the data.

But today, I would like to challenge you on that because there is aspects and dimensions of responsible AI that go beyond fairness.

So responsible AI in Credo AI's world is all about matching the intention, the purpose of utilization of artificial intelligence, and matching that up with accountability within an enterprise.

Now for that to happen, AI not only needs to be fair, which obviously there's a lot of social movement around, but it also needs to be compliant, it needs to be safe, and more importantly, it needs to be auditable.

And the only way you can make all this happen is when artificial intelligence values, the purpose and fit, are aligned with not only your enterprise values, but also centered around the end consumer capabilities and expectations.

So if you think about some of the use cases in object detection, if you think about use cases in misinformation management, if you think about recommender systems, in all these areas, aspects like transparency of the entire system, your end-to-end AI lifecycle is really critical, not just the explanation of models.

When you think about performance of these systems, how are these systems performing not only for certain demographics, but the thresholds that you've set for measuring success for your enterprise, are they getting met or not?

And then as you can imagine, with large scale systems and ML models, there's a big focus right now on the energy consumption associated with these models.

And I'm sure your enterprises have made commitments to carbon footprint and being carbon footprint neutral till certain years.

So how do you actually do a check against, are you meeting the goals that you've set for yourself in your sustainability metrics?

So responsible AI stepping back is not just about fairness.

It goes way beyond that and covers dimensions of performance, compliance, transparency.

Now this is a very interesting myth that we hear a lot from our prospects, customers, ecosystem partners, that AI regulatory risk is a key driving force for responsible AI governance adoption.

1, absolutely give you a sense of what is happening globally.

So globally, there's a lot of AI regulations that are not only coming, but they already exist.

And enterprises need a new capability set to lead the way in this new global AI world.

Now, there are existing regulations, many of which you might know, whether it is SR 11.7, applied to model risk management in financial services.

If you think about GDPR, if you look at the FTC's business guidance, there's already an extensive amount of focus on how we are going to ensure accountability when it comes to AI and machine learning.

But in the past 5 years, AI and ML has become 1 of the top legislature agendas around the world.

You know, over 60 countries now are developing national AI strategies.

If you go to the OECD website, you can see over a thousand responsible AI frameworks being tracked.

And then some of the pivotal moments in regulation, especially with respect to ML, are showing up.

In regards to that, I'm sure many of you are tracking Europe's Artificial Intelligence Act.

You've paid attention to Singapore's model governance, and you're also looking at aspects happening in China as well as Canada.

So yes, absolutely, there is a lot of global regulation at local, federal level that is showing up, but there's a different aspect and lens that we are finding within the leaders who are emerging as the AI first organizations and ethic forward companies within our set.

So If you step back and think about enterprises and what are their motivations for not only transparency requirements, but doing and developing, procuring artificial intelligence responsibly, what we find is, 1, Responsible AI really provides a lot of competitive differentiation.

Now, competitive differentiation is because through transparency, by being very proactive about how you're building and procuring these AI systems, you can actually showcase and build trust with your end consumers.

And that trust is resulting in not only increased sales, retention of customers, but also increase loyalty with the employees, with your boards and other stakeholders.

Now that competitive advantage is even more pivotal for enterprises to pay attention to rather than just the regulatory moments.

Now brands are really showing up not only in service of their business objectives, but making sure that the consumer trust is bolstered.

In our work, we are finding that enterprises who are able to build that consumer trust through governance practices are able to win deals, are able to shorten their procurement cycles, and more importantly, they're able to build that long lasting relationships with the customers.

So I think as stepping back, 1 of the key takeaways I would love for you to take from this slide is really around that responsible AI is not just around risk management.

It is not just about managing the regulatory risk, though that is a big component, it is really around how can you deliver and get the benefits and the ROI of artificial intelligence at scale.

And then lastly, you can imagine that there is a lot of oversight that is coming from investors who are looking at their portfolio from perspective of impact assessments as well.

And these investors are going beyond just looking at how an enterprise is managing risk and showing up in the marketplace.

A big component of what we are betting on is again, this trust capability through transparency reporting that a lot of these enterprises are going to be focused on.

Myth number 4, Compliance with regulations and AI policies will ensure responsible use and development of artificial intelligence.

I spent the past 5 minutes talking about regulatory movements, and we are seeing a lot of activity because of local and state regulations, where enterprises are trying to be proactive to tackle some of the unintended consequences of this technology head on.

However, many of the enterprises are also thinking about it as a rubber stamp activity.

And I think that is an important myth for us to sort of step back and evaluate.

We believe that enterprises who will lead in this new age of AI will do even more than what is just required by the law of land.

And here it is really important for an enterprise to build that differentiation.

Now, I want to give you an example of New York City bias law.

Last December, New York City passed a law which states clearly that by January

all the enterprises that are operating in New York City will need to provide a bias audit publicly on their website around all the automated employment decision-making tools that they are either building or they are buying.

But as you unpack that regulation, you realize that how many holes and unknown questions are still associated with that law.

For example, the law still has some misreadings around what does an independent audit really look like?

How are the end consumers who might not want these tools to be used, how can they provide feedback to the enterprises?

So even though New York City law is a great first step in ensuring accountability and transparency around the use of machine learning systems, 1 of the things that is very clear is a good policy, a good set of regulation will need more interaction between the private sector and the public sector.

It will need more conversations between the technologists and the policy makers so that the policies and regulations that get built are more comprehensive.

And in case of New York City law right now, as I mentioned, there are a lot of gaps.

So even though enterprises are trying to figure out how they can comply with that law, it is again becoming a checkbox activity rather than really thinking deeply about how can you go on this responsible AI journey.

Now this brings me to the next myth, which we have seen extensively, I would say 95% of our customers, prospects that we talk to, they really step back and say, hey, you know what, we already have a strong ML infrastructure stack, we have a lot of ML ops tools, and we have risk management tools.

We can deliver on responsible AI governance goals using those tools.

And that is a big myth that I would love to demystify for you.

So, when you look at artificial intelligence and how it is getting built right now within all the enterprises, at least the ones that we are working in, they have a siloed implementation.

There are many point solutions being used across ML Ops, across Model Ops, across model monitoring, because production model monitoring is becoming really critical need in this market.

In addition, many of these enterprises do have GRC tools, Governance Risk and Compliance tools, like ServiceNow and others.

Plus, they also have model risk management tools, especially if you are in the banking or the financial sector.

And the purpose of those business risk management tools is basically to look holistically across your enterprise and figure out the risks that might be emerging from your resource allocation, from your financial planning, from your regulatory aspects.

So if you look at these 2 areas, 1 of the biggest challenge with these 2 areas is that they don't specifically understand emerging AI risk, which is still a brand new category and there's a lot of work happening on fairness, on robustness, on performance, etc.

So our thesis is that enterprises, instead of shoehorning existing tools to solve these new risk in AI, need to have a new strategy for responsible AI governance.

And the need for this new strategy is primarily driven by 4 core factors.

The first is, as you can imagine, a big component of governing these AI system is based on context.

I'll give you an example which I use extensively in all my other conversations, is facial recognition systems used for unlocking a phone and the same set of models being used for access to a building.

Now, as you can imagine, access to your phone versus access to a public building or a private building with proprietary information require 2 very different levels of risk controls and understanding.

As a result of which, when it comes to responsible AI governance, it is really important for organizations to understand the context and to really figure out how do you manage risk around that context.

A lot of these model op tools as well as MLOps tools, they are unable to understand that context and as a result of which they are unable to provide you that business perspective around risk that that application could be providing.

Secondly, translation is really critical.

So even with the AI infrastructure tools, which can provide you a view into how your technical systems are doing, how your models are performing, how your data sets are they comprehensive or not, and then what are they doing in production, whether there is a drift associated with data or concept.

So certainly all the AI infrastructure tools have the capability to provide those technical measurements that can inform your governance practices.

But as you can imagine, governance is a more holistic process where you absolutely need to understand what is the risk threshold for your organization and how you can effectively tackle it.

As a result of which understanding the brand risk implications, the financial risk implications, the regulatory risk implications are super critical.

And this requires a new level of translation between these technical metrics and these business objectives.

And then this brings me to the third point around standardization.

Absolutely, we are seeing across our customer base that there are a lot of point solutions being used for machine learning operations.

However, there is no consistent system of governance that sits on top of this MLOps infrastructure to be able to look at your data pipeline, your AI pipeline, your context, and your risk aspects coming from your business to basically create a standardized mechanism to drive that operational efficiency, and also the cost of savings, which are derived from the standardization.

And then lastly, you know, we've spoken at length around regulations and not only at the global level but at local and federal level.

It is going to be important for businesses to pay attention to it and to keep up with these emerging regulations.

So regulatory relevance across your entire AI lifecycle so that you as an enterprise can manage this risk is really critical.

And this is why responsible AI governance is absolutely critical and bringing in new capabilities so that you can bridge the gap between your ML infrastructure, your risk infrastructure through this new layer is critical.

With all these myths, I'm sure right now, there is a sense that responsible AI governance is impossible to accomplish.

Because again, depending upon the pervasiveness of artificial intelligence within your organization, based on the maturity of implementation, based on the number of resources you have within your organization, you could be at different stages of your AI implementation journey.

What we are finding within our customers is that the journey that they're going on starts out with really building awareness of what responsible AI is, what it means to the organization, how can they encode their values and principles into governing not only the people and the processes, but also doing technical assessments of data sets and models.

So exploring and building awareness, aligning on how do you build that responsible AI framework that works for your organization, whether you're adopting it from a standard or whether you're creating your own is really important.

And then obviously then the organizations go on this journey of formalizing, building infrastructure, really bridging the gap between your ML infrastructure and your risk infrastructure so that you can do governance at scale.

And at Credo AI, we are really simplifying this for you.

We are offering targeted solutions for each and every stage of the journey.

So just at the onset, when you're exploring and trying to get visibility into all the machine learning-based application within your organization, Credo AI has a mechanism by which we can create your use case registry.

In addition, we can encode your company values.

We can encode the existing regulations or upcoming regulations.

Also, we can bring in standards, whether they're emerging from IEEE, from NIST, and others, into your enterprise boundaries.

So that as you go on this journey, you can start not only doing gap assessment and baselining, but also then go on this continuous risk and compliance management of these systems.

Now, we'll be happy to talk to you more about it, but I think really just stepping back and looking at this responsible AI journey, it is really important to understand that if you try and tackle from a holistic risk management perspective, especially if you don't have the people or the processes or the tools ready for it, you're not gonna be successful.

So I think really thinking about it methodically and thinking about the framework and the policies that are important to you, as well as navigating the different risk profiles around your use cases.

As an example, we work a lot with financial services and insurance.

And as you can imagine, within those sectors, there are some high-risk use cases, everything from credit underwriting, life insurance, risk scoring, as well as some of the very highly visible marketing models.

Now, in case of those high risk models, you will certainly need some sort of a human oversight and accountability along with audit trail, so that you can have this entire decision provenance of how the different decisions were made when the system was getting built and how it was getting deployed.

So what Credo AI provides you is that entire journey and a set of transparency reports, which not only shows you, are you compliant to your internal company values, but are you also in compliance with external regulations?

And this is where the power of Credo AI really comes to bear, is once you've understood holistically what your risk is, then you can go on this journey of mitigating those risks.

And as I mentioned, that there is this evolving ecosystem right now of regulators, policymakers, AI enterprises, AI vendors that are all coming together to really define sector-specific, use case-specific thresholds that are going to be required to prove compliance against.

And at Credo, we've already effectively done that for you through the policy packs.

With that, just to close it out, as you think about your responsible AI journey, which should not be a complicated 1, and it is certainly not an impossible 1, it's really critical to think about the value proposition of the capabilities you want to bring in.

And across our customer base, ranging from financial sector to government to HR technologies, all the way to high tech, standardizing governance, having the single layer or single view of governance across your entire AI life cycle, and sitting on top of your ML infrastructure so that you can basically not only make it more efficient, but you can also have complete visibility at all the levels is really critical.

Secondly, just to underscore, there's a lot of regulatory movement.

How do you keep up with those upcoming regulations and how are you proving compliance against those regulations is really critical.

Third, as you can imagine, risk appetite for each and every enterprise varies.

So active and proactive AI risk management, not only into the understood AI risk, but new kinds of risks that are emerging day in, day out, as you're deploying machine learning applications at scale, is another capability that you might want to think about.

And then lastly, if you are able to do all these things effectively, we are finding within our customers that there is an increased AI return on investment, because now you have higher confidence in deployment of artificial intelligence.

You have implemented that proactive risk and compliance management.

You have a view into your brand exposure risk, as well as your financial risk and your regulatory risk.

And all that is really grounded in understanding the context behind these applications.

So in the end, I would love to wrap up with, you know, responsible AI is still an emerging area and it is going to become a dominant prerequisite for artificial intelligence.

So you as an enterprise need to take proactive stance to not only build trust, but also manage risk and get the ROI on your artificial intelligence.

We are betting big on the space at Credo AI and happy to address any questions that you might have around how you can get started on this artificial intelligence and responsible AI governance journey.

I want to thank you for your time, and you can certainly reach out to us at Credo AI on Twitter, or at Navreena Singh at Twitter as well.

See all Scale AI transcripts on Youtube

Debunking the Core Myths of Responsible AI | Navrina Singh (Credo AI) | TransformX 2022