Does your AI count: Scoping ADMT before California’s Jan. 1, 2027 deadline

[00:00:00] Zach (moderator):

Hello and welcome everyone to the IAPP web conference, Does Your AI Count? Scoping ADMT before

California's January 1st deadline, sponsored today by DataGrail. We'll be getting started with the

presentation in just a moment, but before we do, a few program details. Participating in today's web

conference will automatically provide IAPP certified privacy professionals who are the named

registrants with one CPE credit.

 

[00:00:20] Zach:

A recording of this web conference and its slides will be available on the IAPP website within 48 hours. Access to the material can be found by navigating to the resources section of the site and then selecting videos. Throughout the web conference today, you may have questions, you probably will, and you can submit them using the Q&A feature in Zoom and we will carve out some time at the end to try to get to as many of them as we can.

 

[00:00:55] Zach:

And now I'm happy to hand things over to Daniel to kick off the program.

 

[00:01:00] Daniel Barber:

Thanks, Zach. Yeah, thrilled to be here and thrilled to have this group in particular to go through

this session.

 

[00:01:15] Daniel Barber:

I feel like in many of the conversations I have with privacy teams, people may be aware of ADMT. It might have come up in conversation, but from our observation, it feels like something that is, given we're in September, we should be looking at. And so I'm thrilled to have this group with us today.

 

[00:01:40] Daniel Barber:

And so I'll let folks go through their introductions, maybe David, just given you're on the far left, do you want to kick us off?

 

[00:01:45] David Khalily:

Sure. Yes. David Khalily. I'm a senior privacy and AI counsel at Harvey. I've held similar roles in-house at a leading US gaming company and a European bank. Also a founder of the Cyber Council Group, which is a nonprofit for in-house cybersecurity lawyers.

 

[00:02:00] Daniel Barber:

Lovely. Thanks for joining us. Omer, do you want to keep us going?

 

[00:02:05] Omer Tene:

Yes. Hi, good morning, folks. Omer Tene. I'm a partner at Goodwin focusing on privacy, of course, and also AI regulatory issues.

 

[00:02:15] Daniel Barber:

Lovely. Neelofer, bring us on.

 

[00:02:18] Neelofer Shaikh:

Hi, everyone. I'm Neelofer Shaikh. I'm an attorney with CalPrivacy. I've been at the agency for over four years now.

 

[00:02:30] Neelofer Shaikh:

And in that time, my primary role has been to advise our board on the development and finalization of our CCPA regulations, including the ADMT regulations that we're going to talk about later today. Back to you, Daniel.

 

[00:02:45] Daniel Barber:

Lovely. Thank you. And Daniel Barber, co-founder and CEO of DataGrail. DataGrail is a complete agentic data privacy platform. We work with brands like Life360, Dexcom, and Major League Soccer. And so we'll get right into it. 

 

[00:02:55] Daniel Barber:

And just in terms of what we're covering today, so Neelofer is going to provide us an overview of just what is ADMT? Why is it important? Why should we be thinking about it today? What does it mean in terms of the dates that we should consider? And we'll follow that forward with just a little bit more detail around what some recent litigation looks like in related fields, what the enforcement deadlines look like.

 

[00:03:20] Daniel Barber:

And then in terms of the points that really matter, how do we think about a significant decision in a practical context within the program? And then how we think about humans in the loop as they're operating different software across the business. Where is it showing up? I think AI is obviously in use in many different applications at this point, whether you're a software provider or even just using software like Zoom today. And then how do you think about the inventory of applications that you have as well as building risk assessments to address any known or unknown risks. And so Neelofer, I'll kick it over to you for the next few slides.

 

[00:03:50] Daniel Barber:

Feel free to guide me when you'd like to move us forward, but I imagine you're just going to give a quick definition first.

 

[00:03:58] Neelofer Shaikh:

Yes. Hi, everyone.

 

[00:04:00] Neelofer Shaikh:

So I'm going to provide a brief overview of the ADMT regulations. I'll note here, and you'll see in the disclaimer below, the full text of our regulations are available publicly at cppa.ca.gov regulations. And in the bottom right corner of all the slides I'm going to be covering, I included the reference to the relevant regulatory provisions so that if you want to follow along or look at it later, you are more than welcome to.

 

[00:04:30] Neelofer Shaikh:

Next slide, please, Daniel. All right. So starting with what I think is probably the question on most folks' minds, which is what is automated decision-making technology or ADMT? And the full text of the definition, again, bottom right corner is in our regulations.

 

[00:05:00] Neelofer Shaikh:

However, if you're thinking at a high level, what is ADMT? It's helpful to ask yourself three questions. Are you using a technology that's processing personal information? So is it collecting? Is that technology using computation? And third, is it replacing or substantially replacing human decision-making? And substantially replacing human decision-making means your business is using the technology's output to make a decision without human involvement. I'm going to go into that on the next slide.

 

[00:05:35] Neelofer Shaikh:

Before I do so, I'll just quickly note here, ADMT also includes profiling, which is a defined term in

the CCPA. It covers things like evaluating a Californian in different respects in a way that replaces or substantially replaces human decision-making. Next slide, please.

 

[00:06:00] Neelofer Shaikh:

So what does human involvement actually require a human reviewer to do? And there's three components, and you need all three. You cannot have only two of these or one of these. Without all three, you no longer have human involvement.

 

[00:06:20] Neelofer Shaikh:

So what are the three things? So first, the human reviewer needs to actually know how to interpret and use the technology's output to make the decision. So they need to actually understand what they're looking at. They also need to review and analyze the output and any other information relevant to making or changing that decision.

 

[00:06:45] Neelofer Shaikh:

So they need to actually be taking the time to review and analyze the technology's output and other relevant information. And lastly, they need to actually be able to make or change the decision based on their analysis. So you can see they need to know what they're doing, they need to take the time to

actually review and analyze, and they need to be able to actually make or change the decision.

 

[00:07:00] Neelofer Shaikh:

Without each of those things, you do not have meaningful human involvement. Next slide, please. One thing that I'll flag here is not all ADMT is subject to the CCPA's ADMT regulations.

 

[00:07:25] Neelofer Shaikh:

You need to be using ADMT for a significant decision. And a significant decision is one that results in the provision or denial of financial or lending services, housing, education enrollment opportunities, employment or independent contracting opportunities or compensation, and healthcare services. Now I think a few of these are pretty self-explanatory.

 

[00:07:50] Neelofer Shaikh:

One thing I will flag is that these are each defined in further detail within our regulations. And one thing I'll note, because I think the other panelists will be getting into this later in the

presentation, employment or independent contracting opportunities or compensation. It includes things that I think most folks would think about.

 

[00:08:05] Neelofer Shaikh:

So hiring, demotion, suspension, promotion, termination. It also importantly includes allocation or assignment of work to your employees, as well as allocation or assignment of compensation to your employees and independent contractors. So please keep that in mind when you're thinking about whether you're using ADMT for a significant decision.

 

[00:08:30] Neelofer Shaikh:

Next slide, please. So if you're using ADMT for a significant decision, what do you actually have to do? Now, besides complying with the CCPA's general requirements for consumers, you must also do each of the following. So first, you must conduct a risk assessment before using ADMT for a significant decision.

 

[00:09:00] Neelofer Shaikh:

And one of the things I really want to flag for everyone today is that a risk assessment is due before using ADMT, and that requirement went into effect on January 1 of this year. So you should be doing these risk assessments now. You should not be waiting until January 1, 2027 to start your risk assessments.

 

[00:09:30] Neelofer Shaikh:

And a risk assessment at a high level, it's helpful to think of as a tool for your business to identify benefits, privacy risks, and relevant safeguards of a given activity. So here, using ADMT for a significant decision about a consumer. Section 7152 of our regulations provides the detailed requirements on how you need to conduct and document your risk assessment.

 

[00:10:00] Neelofer Shaikh:

One thing I'll just flag here is that if you're using ADMT, your risk assessment must identify and

document the logic of the automated decision making technology, including any assumptions or

limitations of the logic, as well as the output of the ADMT and how your business will use the output to make a decision. So please keep that in mind. Once you've conducted a risk assessment, you also will need to provide a pre-use notice to consumers and the ability to opt out and access ADMT.

 

[00:10:30] Neelofer Shaikh:

And that's where that January 1, 2027 deadline is important. So our board gave businesses an additional year to come into compliance. So you have all of 2026 to bring your ADMT into compliance before using it for significant, when using it for significant decision.

 

[00:11:00] Neelofer Shaikh:

However, by January 1 of 2027, if you're using ADMT for a significant decision, it needs to comply with these three requirements. You need to provide a pre-use notice to consumers whose information you want to process using that ADMT. That pre-use notice at a high level covers the purpose of using the ADMT, how the ADMT would work, and the consumer's rights to opt out and access that ADMT.

 

[00:11:30] Neelofer Shaikh:

You also need to provide consumers with an easy way to opt out of the business's use of ADMT. There's two exceptions to that. I'm happy to address that in the Q&A if helpful.

 

[00:11:45] Neelofer Shaikh:

And then lastly, you must give consumers who did not opt out the ability to access information about how you actually used ADMT with respect to them. And what I want to emphasize with the access right is first, you have to give this to consumers. There's no exception to the access right.

 

[00:12:00] Neelofer Shaikh:

There's certain information you're not required to include, but you must always provide the access right if you are using ADMT to make a decision with respect to the consumer. And the second thing is, the access right is specific to the consumer, so it is not about how you use ADMT generally with respect to consumers more broadly. It's about how you used ADMT to make a decision about that consumer specifically.

 

[00:12:20] Neelofer Shaikh:

All right, I'll pass it back to Daniel.

 

[00:12:25] Daniel Barber:

Thank you, Neelofer. Yeah, I think as expected, we have some questions already popping up.

 

[00:12:35] Daniel Barber:

Thank you for submitting those. We have allocated time for Neelofer to answer as many questions as we can at the end, so we'll look to do that as we go through. So keep them coming.

 

[00:12:50] Daniel Barber:

We're just going to try to bulk answer them at the end so that we get through as many as possible.

 

[00:13:00] Omer Tene:

Daniel, can I just jump in for a second?

 

[00:13:02] Daniel Barber:

Please, please.

 

[00:13:05] Omer Tene:

I want to just mention something that's definitional and foundational here. There's actually a misnomer in the name of this session because I think you called it is your AI subject to- Does your AI count? Does your AI count? So I just wanted to say that if you were listening closely, Neelofer did not mention AI. Artificial intelligence is not mentioned once in these regulations. And in fact, the regulations are kind of agnostic to whether or not the ADMT is an AI.

 

[00:13:40] Omer Tene:

And certainly, generative AI, that's completely kind of besides the point. It talks about the technology that processes personal information and uses computation to substantially replace human decision-making. So the PII is an important component and computation replacing decision making.

 

[00:14:00] Omer Tene:

And the reason I'm saying this, a lot of clients tell us, well, we're not using AI. This is machine learning. We have a deterministic algorithm.

 

[00:14:15] Omer Tene:

It's not iterating. Is it covered by these regulations? Absolutely. This isn't about AI.

 

[00:14:20] David Khalily:

That's a great point, to be honest. And one, I think that probably, yeah, in the name doesn't actually cover the scope of what we're talking about here, but that's, yeah. Yeah, we can get into it.

 

[00:14:35] Daniel Barber:

But yeah, fantastic call out, Omer. All right. So I think just to rehash two particular dates that Neelofer brought up that, you know, many teams are looking at today.

 

[00:14:50] Daniel Barber:

So one, the risk assessments did go into effect January 1st. So if you're not doing those today, I would definitely encourage you to take a look at the requirements for those to get started on that work. To Omer's point, I think the, you know, it's not just AI specific.

 

[00:15:10] Daniel Barber:

So where there are automated decisions, decision-making happening in your program, you do want to start those risk assessments now. The specific ADMT requirements go into effect January 1st. So we are towards the back end of the year, which is why we felt this topic was very important to bring up.

 

[00:15:40] Daniel Barber:

Just want to kind of rehash those two dates before we go into kind of the details. But I think, Omer, we talked about this one. I just feel like given your scope at Goodwin, I'm sure you see all different examples of these kind of five areas.

 

[00:16:00] Daniel Barber:

Do you have any that come to mind that you want to share with folks? And David, feel free to chime in as well. I just, you know, I feel like you have an interesting purview, Omer, that folks would appreciate.

 

[00:16:10] Omer Tene:

Yeah. So, you know, there are dozens of use cases where I think it becomes interesting is where it kind of comes close to the line of is it or isn't it a significant decision? Is human decision-making actually replaced or substantially replaced? And let me give you one kind of use case. You have an AI that summarizes performance reviews in an HR process. Now, the performance reviews are written by humans.

 

[00:16:40] Omer Tene:

So, the AI just creates a summary and basically aligns the outcome against the framework. So, it says at the end of the day, these performance reviews indicate that, you know, some employees meeting the frameworks, not meeting the frameworks. Well, is that subject to this entire process? And consider that the person who's actually delivering the decision or the performance review will be a human too.

 

[00:17:00] Omer Tene:

It won't be the AI delivering it. So, is there a significant decision here? Well, you know, it might affect comp, allocation of assignment of work, promotion, demotion. Probably yes.

 

[00:17:20] Omer Tene:

Is the decision made by an AI? Well, the AI just summarized content, right? That's what we do with chat, CPT for every phone call now. Is it profiling? Like Nilo first said, that's an independent decision-making on a definition essentially. So, profiling means analyzing aspects concerning a natural person's performance at work.

 

[00:17:45] Omer Tene:

It does seem to be doing that. Does it substantially replace the decision maker? Well, will the person delivering the review just kind of click through or will they replace the review's outcome with their own sort of analysis and decision-making? Those are questions that come to mind.

 

[00:18:00] Omer Tene:

Yeah, and sort of just building on that and applying a lot of the same reasoning. I'm particularly interested in the profiling piece of this. Organizations are using a variety of productivity monitoring tools or sort of just productivity tools in general that assign workflow to people who have a lighter burden at any given time, or in some cases are systematically observing how they conduct tasks. And those are going to be feeding into decisions big and small every day with varying degrees of human decision-making.

 

[00:18:30] Omer Tene:

And I think that last piece is where the rubber really meets the road. And I think we're going to be talking about that a lot, but I think you should probably think broadly about this, especially in the employment context, which is, I think, the most relevant for most of our listeners. But I think there's also, I mean, there's also a lot of similar things across the board with all of the different areas that the rule is focused on.

 

[00:19:00] David Khalily:

Yeah, I think it's interesting you both picked up the employment example. I think that applies to all industries. And I think given it's not just employment decisions, it's also compensation decisions, then you actually have a lot of impact across many different areas of a program that folks may not be considering.

 

[00:19:30] David Khalily:

They might be looking at just the recruiting aspect, which is only one area of employment and compensation, obviously. So just calling out what's out there today. And Omer, thank you for our session on Friday, talking through this.

 

[00:20:00] David Khalily:

We added a couple more. While we were talking about California specifically today, there are other states with similar provisions in place. We would anticipate that similar provisions will be put forth in states that have comprehensive privacy frameworks in effect today.

 

[00:20:30] David Khalily:

So naturally, you know, we expect the number to grow beyond just the, you know, California, which we're talking about today, and then other states like Colorado and Connecticut. So, you know, something to follow just as, you know, regulations do pass across additional states. And hopefully, you know, they follow the same type of framework that we're talking through today.

 

[00:21:00] David Khalily:

So just a couple of examples. These are not ADMT related, but the CPPA has been very active from an enforcement perspective. You know, we do anticipate that will continue.

 

[00:21:30] David Khalily:

There's an enforcement agency obviously attached to the CPPA. So, you know, just kind of highlighting a couple of the few of the examples here. I think where we wanted to also spend time, and Omer, I appreciate your guidance on looking at the litigation here, because there also is litigation in this area, and a few different examples here worth sharing.

 

[00:22:00] David Khalily:

You know, several of these are related to employment. And, you know, that's not surprising given AI has lots of potential in that area, but also can make automated decisions that are worth considering. So just this is something obviously we'll share in the materials, but something worth following as well.

 

[00:22:30] Daniel Barber:

So let's move to significant decision. I think this is the area that, you know, is challenging to understand in a practical sense. I'm sure many folks are trying to understand that.

 

[00:23:00] Daniel Barber:

Some of the questions we've already got in the chat are definitely in this area. Maybe, Omer, do you want to just kick us off of like, how do you think about this? I know, Neil, a few provided a great definition to kind of get us going, but from a practical sense, what are you seeing teams consider as they look at this from a significant decision? I appreciate the example you gave earlier too, from a performance standpoint.

 

[00:23:20] Omer Tene:

Well, just to, you know, work with the example you have on this slide here, Daniel, so this is a model that scores job applicants.

 

[00:23:30] Omer Tene:

What if the recruiter only ever sees the top 5% that it surfaces? So again, you have to ask yourself, is this a significant decision? Yes, it's about hiring, firing. Is this an automated decision-making tool? Yes, because it substantially kind of reaches an outcome here that affects the decision. And then oftentimes the question will be, as David highlighted, what is the degree of human involvement? And, you know, if you're listening closely, when Ilofer kind of talked about the language of the regs, there are a lot of verbs there, like does the human reviewer, does he have the knowledge, he or she, sorry, have the knowledge? Do they interpret? Do they use additional information? Do they review and analyze additional information? Do they have the authority to make the decision and essentially overrule the machine? What I wanted to point out is that even if you check all of those boxes and you have a human reviewer here who could overrule the, you know, the software or the algorithm, I think there are very strong institutional incentives to not do it.

 

[00:24:20] Omer Tene:

Because think about it, if you are in this context, an HR person, and you need to justify a decision you've made in retrospect, what will be easier for you to justify a decision where you could point to this incredibly powerful, smart, and expensive tool that the employer kind of provided you with to make this decision? Or if you have to say, well, the tool recommended something, but I actually overruled it and thought something else, turns out your decision was terrible, and now you need to justify that. So, you know, even if the model lets the recruiter see more than just the 5% that it surfaced, let's imagine I just pushed them sort of to the top of the list. You have a thousand resumes, right? You have 50 to look at, which the algorithm basically fed you with.

 

[00:25:00] Daniel Barber:

What are the chances that you'll actually review or substantially review 950? Exactly.

 

[00:25:10] Omer Tene:

So, I think that, you know, in another context, college admission essays, right? So, a couple of months ago, they get hammered by like thousands and thousands of applications. They need to read all these essays.

 

[00:25:40] Omer Tene:

Well, it's much easier if an AI reads the essays and then just provides you with some quick summary. Are you going to actually go back to scratch and read the raw material? I think there's a big question mark. Even if technically it's 100% possible and you can do it, you know, back to my initial performance review example.

 

[00:26:00] Omer Tene:

Yes, it provides you a summary, but you can look at the actual performance review. But this saves you a lot of time, and this is an expensive, you know, tool that the employer trusted. Why would they trust you over it? So, I think institutional incentives might really weigh heavy against individual decision makers replacing the tools with their own judgment.

 

[00:26:30] David Khalily:

Yeah, and I think those are great examples. And then there's also, I think, just in sort of more quantitative areas, right? Employers or businesses will typically look at the performance of the tool. And, you know, my thinking is in those areas, those tools are going to be really accurate, especially now, and certainly in the future as they get better.

 

[00:27:00] Daniel Barber:

And so, then you kind of have a problem where the tool is so accurate and so good at what it does, it's as good or better than, in many cases, human performance. And so, then it's sort of like your overturn rate is going to be very low. And I think we're going to speak to that in the next slide.

 

[00:27:30] Daniel Barber:

But, you know, that is something to keep in mind. Like, you know, in those quantitative areas, it gets significantly harder to make these arguments, I think.

 

[00:27:40] Omer Tene:

Yeah, I agree.

 

[00:27:45] Daniel Barber:

All right. So, I think you had a good example, David. I don't know if you wanted to share.

 

[00:28:00] David Khalily:

We don't need to name the technology. But I think that just, you know, as an example for productivity, I don't know if that's relevant for this slide or how you're thinking about that. But I just thought that was a... Well, yeah, so I think we were talking, I think, about certain tools that are embedded in workflow and productivity management.

 

[00:28:20] David Khalily:

This is not like employee surveillance type tools. It's more like generally available for your employee base. I think it's really hard to argue that something that's automatically assigning people, like, a task, it's really hard to argue that, you know, those small decisions are not... Those everyday decisions are not, you know, being fully automated.

 

[00:28:45] David Khalily:

I suppose there's a chance a manager could overturn them. But what I really was thinking about in looking at this slide is two things. One is, what does your documentation look like around these tools? Whether it's in your risk assessment or your procedures and run books for humans in the loop.

 

[00:29:10] David Khalily:

Those docs can help you establish your position that the process does not substantially replace human review by telling people what to do with each proposal. That does kind of slip away the more you really want to rely on automation. But I still think it's helpful.

 

[00:29:30] David Khalily:

And then the setup really matters as well. The setup of the tool, how the tool works. So in the example of tools that perform automated scoring or propose a decision for a reviewer, does the thing surface the data points it weighted the most heavily in producing a score? Is the underlying data still readily available for the human to look at and investigate if something feels off or if it's otherwise appropriate to do so? You know, I don't know how much that's going to happen in a tool that's like an everyday workflow allocation tool that's sort of just making these kind of decisions.

 

[00:29:55] David Khalily:

I don't want to say in a black box, but I don't think it's going to necessarily surface why it made a certain decision on that basis.

 

[00:30:00] Omer Tene:

Just wanted to add another thought here. I see it emanating from a couple of the questions.

 

[00:30:10] Omer Tene:

We, you know, our discussion here kind of suggests or assumes that the human decision maker is some kind of all-knowing saintly figure who actually reviews, you know, gets down to the into the weeds and isn't biased or discriminating. Now, against that baseline, I think the automated decision making might, you know, pale and look not so great, but we know that that's not the real baseline. So, you know, if you think about it, the Fair Credit Reporting Act, which I kind of referred to as the first AI law, one of the reasons that it was passed in the early 1970s and sort of the emergence of the FICO, the credit score, was to actually prevent bias and discrimination by human decision makers who kind of eyeballed the candidate for a loan and said, you know, what the rate would be or if they will get or not get a loan.

 

[00:31:00] Omer Tene:

So they wanted the more kind of scientific or automated process and we got the FICRA and the FICO score, everything that goes into that. Now we are kind of, you know, pushing in the other direction and we have these amazing tools that are providing so much value, but we're saying, wait a minute, let's, you know, allow human decision makers to make the decision instead of them. Will it be a better decision? That we haven't even started debating, right? It could be a worse decision.

 

[00:31:40] Omer Tene:

It's human made, so, you know, maybe that makes us feel a little happy. Anyways, yeah, just wanted to raise that point. What is the baseline we are assessing this against?

 

[00:32:00] Daniel Barber:

Yeah, I think, you know, as we have insight into, you know, over 5,000 different systems and we'll talk about this as we go forward in a couple slides from now, but I would say it's very likely we see automated decision capabilities or areas of capability in almost every functional area of software.

 

[00:32:20] Daniel Barber:

When you think about agentic kind of workflows, you are effectively proposing that the user of the software can complete work in their discipline in which they are operating in. Now, by definition of completing work, obviously in the legal construct, there is review as you go to complete work, but in many other fields, there may not need to be legal review of that functional work, and so I would encourage folks to really look beyond just the tertiary sort of use cases that we're looking at today, because I do expect, you know, most software providers will be trying to advance their agentic capabilities, which agentic really in itself is, you know, completing work, in many cases making decisions for the user. So, you know, Omer, I completely agree this is where we're headed.

 

[00:33:00] Neelofer Shaikh:

So, you know, disclosure statements and the ability to provide visibility, David, to your point, to the actual data set, become even more important than they did before. I just kind of want to tie something that Omer said to the actual rule. So, I think if you're relying on the human appeal exception to the opt-out requirements for allocation and assignment of work and compensation decisions, I think you're obligated to, first of all, only use that ADMT solely for that assessment, and then ensure that it does not unlawfully discriminate on protected characteristics itself.

 

[00:33:20] Neelofer Shaikh:

So, there is some sort of bias testing obligation there if you're relying on the human appeal exception, which I think in many of these cases would seem quite attractive as opposed to an opt-out or just simply much more operational.

 

[00:33:35] Omer Tene:

Yeah, but, David, bias testing of the automated decision-making — what I was suggesting is that the human adjudicator might be way more biased than the machine.

 

[00:33:50] David Khalily:

I think it's interesting because it kind of implies, like, make sure that the machine is really working as intended.

 

[00:34:00] Daniel Barber:

Yeah. All right. Well, let's push this forward a little bit.

 

[00:34:10] Daniel Barber:

I can see we're going to have a lot of questions. We're going to do our best to get through as many as we can of those, and, Nilofer, you're probably going to be on rapid fire for a few of these. So, we talked about some of these.

 

[00:35:00] Daniel Barber (?):

You know, we've covered, obviously, the area in HR tech. I think, you know, decisions related to fraud, this is going to be very popular for organizations that need to do this kind of work. Underwriting is the other one where we see lots of folks considering this technology or, to Omer's point, you know, they might even have already implemented this type of technology.

 

[00:35:20] Daniel Barber (?):

So, the assessments certainly need to cover both the work that's been operating through 2026 as well as opt-out requirements going forward. I know, David, obviously, you've worked in regulated industries. Anything that you'd share here, folks, as you think about what folks should consider?

 

[00:35:40] David Khalily:

Yeah, I mean, I think if you want to think about what's most critical here, it's pretty hard to honor opt-outs and things like that with fraud scoring.

 

[00:35:55] David Khalily:

So, I think you might want to really think about carefully like how you implement that one. I haven't really done too much with underwriting.

 

[00:36:00] Daniel Barber (?):

Omer, any other comments you'd share on just like these examples or any others that folks might be not considering?

 

[00:36:05] Omer Tene:

Yeah, I agree with your earlier comment, Daniel, that the HR one kind of hits hardest here, partially because it applies to everyone, like every company at the very least has employees, but also because lending or insurance are largely exempt from CCPA.

 

[00:36:30] Omer Tene:

They benefit from the exemptions, or at least insofar as consumer lending goes, that's not in scope here. Fraud scoring, I think, benefits from some of the exemptions for security or for prevention of fraud. So, that's kind of a way to get out of it.

 

[00:37:00] Omer Tene:

Plus, in many contexts, it won't trigger the significant decision, sort of condition to all of this applying. But HR, I think, hits hard because there are no exceptions here, like it's not GLBA, it's not HIPAA, it's not even Fair Credit Reporting Act, except for background checking. The opt-out right is really awkward because if an employee opts out, they get the human decision making, and then it's kind of the perfect setting for a discrimination claim, or for an retaliation claim, at the very least.

 

[00:38:00] Omer Tene:

And then the access rights are also really difficult because if you provide kind of the logic of how the decisions are made, that too can feed into a Title VII claim. There's a lot of contention in relationships between employers and employees, and the fact that now employers kind of have to open the kimono and show the underlying logic of these decision-making technologies, I think will definitely fuel some litigation. And as you said earlier, we've already seen it happen.

 

[00:39:00] David Khalily:

Yeah, yep. So just as we think about this challenge, right, as Omer kindly shared, kicking us off, this is not just related to AI, it could be systems that you use for, that use machine learning as well. I think the first step is obviously understanding the inventory that you have, and so how do you do that? I think the establishment of a method for surfacing systems that are known obviously is important, but also classifying what type of decisions, if those are assisted in the inventory, this is something that folks might have been doing previously.

 

[00:40:00] Daniel Barber:

I would say this is an area where we see teams really challenged. They might be using a thousand different applications, and so just trying to figure out which ones would be doing ADMT-type work, and is it then considered a significant decision? I think prioritization is probably going to be very important here in relation to what Omer was mentioning there on the HR systems, probably worth looking at all of those, especially those involved in recruiting, those involved in compensation. If you're in an industry that has areas that we covered, right, of education or finance, obviously you have your internal systems and probably industry-specific systems that are worthwhile looking at in how those decisions are made, but I would say just for as a broad statement, everyone that's hiring, employing full-time employees or contractors, and the comp-related decisions, I would start there.

 

[00:40:45] Daniel Barber:

That's where you're going to see the most likely scenarios of ADMT in the field. DataGrail does surface this within the product today, so this is something that because of our inventory of applications looking at over 5,000 different systems, we're able to identify which ones are using ADMT or could provide that functionality today, so this is certainly something worth taking a look at. Is there anything else, David, I know from a practical sense you would guide folks just as they're thinking about building their inventory or thinking about classification of different systems?

 

[00:41:00] David Khalily:

When it comes to inventory, I think that you can maybe head some things off at the pass with really strong policies.

 

[00:41:15] David Khalily:

There's a lot of general-purpose tools or tools that are multi-purpose or have a chatbot or something like that as part of it that are pretty open-ended in how they can be used and might have a lot of access to a lot of different data sources, either through MCP or other connections, so you might want to consider putting some guardrails around how those tools can be used and for what purposes, and you do have to monitor and enforce for that, and then you can just focus from there your inventory work on scanning for SaaS tools or dedicated tools that do a specific thing or that you already are pretty aware that you're using in this more regulated space. And I think when it comes to prioritization, for sure, geographic footprint, like how much is happening within the in-scope jurisdictions, I think for employers, there's always this kind of decision when it comes to the privacy laws. California is, I think, the only—there is the New York rule, but California is the only rule that in its comprehensive privacy law brings employees into the consumer definition.

 

[00:42:30] David Khalily:

How much automation is occurring? I think we talked about that a bit. Risk bias are really likely to attract enforcement. For educators, if you're dealing with minors, I think there's always some sensitivity there in terms of enforcement patterns.

 

[00:43:00] David Khalily:

And then just kind of look across in the absence of enforcement until Nilofer and team start really moving, I think you can look globally at some of the things that are happening around the world for some color. I mean, this kind of helps sometimes. I think the Dutch DPA just fined a ride-sharing organization for violating GDPR on some of these ADMT decision-making things.

 

[00:43:30] David Khalily:

So in that case, I think you can kind of look to what the rest of the world is doing as a predictor for what's going to happen next here.

 

[00:44:00] Daniel Barber:

That makes sense. Yeah, I think we'll move us forward just to try to get to questions.

 

[00:44:10] Daniel Barber:

I think what we're seeing in terms of what this means for your program, as I mentioned earlier, assume your own AI tools, your own tools that use machine learning are in scope. And so not just vendor tools. Obviously, if you're a business at this point, I'm sure you have an engineering team that's probably considering using AI in their own application.

 

[00:44:40] Daniel Barber:

I would say starting with the back office tools, as I mentioned, particularly in HR, particularly in performance management. And then, you know, kicking off a real risk assessment and looking at which areas need a risk assessment. I think this is, you know, just will reduce your exposure as we think about January 1st deadline coming around.

 

[00:45:00] Daniel Barber:

Those were intended to kick off January 1st this year. So as Nilofer mentioned, you know, assessments should be taking place now. Enforcement kicks in January 1st going into next year.

 

[00:45:20] Daniel Barber:

And ADMT opt outs and the associated work needs to be done by that deadline. But assessments, risk assessments should be happening now as of January 1st, 2026. So just to sort of wrap us up here.

 

[00:45:40] Daniel Barber:

Any comments, Omer, as you think about just the discovery of applications, classifying those and assessing those? I'm happy to provide some color too, but I know we're going to get to questions here in just a minute.

 

[00:45:50] Omer Tene:

Yeah, I think this is kind of AI governance on and on, right? Which is a relative of what we've been doing in privacy for many years, data governance. It's kind of making sense of the different tools that you're using, the data flows and, you know, kind of labeling them appropriately and in some cases escalating to counsel or, you know, to whoever makes the policy decisions.

 

[00:46:15] Omer Tene:

But given that, as you said, Daniel, businesses today like deploy dozens or hundreds of tools and each one of them has many use cases, it can't be the case that every use cases gets kind of escalated to the general counsel. I kind of view it as a red, yellow, green, you know, type of paradigm where you need to let like 90% of the use cases go and really focus on the ones that can significantly impact individuals. And I'm not sure that the CCPA ADMT achieves all of that partially because of the pretty broad exemptions the CCPA itself has.

 

[00:47:00] Omer Tene:

And remember the CCPA has a privacy law, right? So all of this is housed under a privacy law compare it, for example, to Colorado's SB 189, which is coming into effect same day. And that is an ADMT specific law. So Colorado also has the privacy laws, or if you look at NYC 144, the ADT employment related automated decision making technology law that New York City passed in 2021, I believe it's been in effect for a couple of years.

 

[00:47:30] Omer Tene:

That too is not a privacy specific law. So I think it's really important to remember this law and these regulations are only triggered by processing personal information. If that's not the case, then none of this applies to you.

 

[00:47:50] Daniel Barber:

Yeah. I think that's actually a good call out. All right.

 

[00:48:00] Omer Tene:

So we've got a lot of questions. I'm going to try to get through. I can promise in the spirit of what we're talking about today, I have been reviewing all of them.

 

[00:48:15] Omer Tene:

So I'm going to try to not introduce my bias in the decision of which ones I select. So I, but I do think the first one here from Jewel Darlington. So where, and Nilofer, I'm just going to direct these to you.

 

[00:48:40] Omer Tene:

And then if David or Daniel, you'd like to add commentary, feel free. Where an opt-out must be honored. How should leadership validate that the alternative decision process can handle actual demand without creating delays or disadvantages for consumers? What evidence would you want before signing off on an operation on operational readiness? Thanks, Daniel.

 

[00:49:00] Neelofer Shaikh:

And thank you for the question, Jewel. So I think your question is just getting to, to me, at least what I think is part of the value of the risk assessment for your businesses, which is again, before you're actually using ADMT, you should be identifying what are the potential risks of doing so and what are the relevant safeguards. So I think that is an important opportunity for businesses to be thinking about how to safeguard their processes.

 

[00:49:30] Neelofer Shaikh:

And I think your question, as well as I, I'm going to answer actually several questions and probably try to like, yeah, I see. How about this? Instead of trying to go question by question, I'm going to try to see, Daniel, if I can scare you a little bit and maybe identify some common themes and some things to consider with that in mind. I think one theme that I'm seeing is a concern about scalability and feasibility, particularly with respect to opt-outs.

 

[00:50:00] Neelofer Shaikh:

And so one thing I'll flag there, of course, again, our regulations do require opt-outs. There are two exceptions that I had previewed before. And so you all have teed me up to raise these exceptions for you now.

 

[00:50:20] Neelofer Shaikh:

One exception to providing the opt-out is where you provide the opportunity for consumers to appeal to a human reviewer after the fact. And so this exception addresses essentially a human appeal mechanism. And honestly, in shorthand, it's easy to think of it as the human appeal exception, but that is not the official name of it.

 

[00:50:45] Neelofer Shaikh:

And it's in 7221B of the regulations for those, again, who are following along. And so that could be, again, one way to address the feasibility concern that folks are having. Another one is, I personally call the limited use exception.

 

[00:51:00] Neelofer Shaikh:

It's not actually called that in the regs, but again, in 7221B, for certain decisions. So again, there's only a limited subset of decisions that are subject to this exception. It's admission, acceptance, hiring, or those allocation or assignment decisions that I discussed earlier.

 

[00:51:20] Neelofer Shaikh:

A business does not have to provide the opt-out when two conditions are met. Those conditions are set forth in the regulations in more detail. At a high level, it's that your ADMT is being used solely for that limited purpose.

 

[00:51:40] Neelofer Shaikh:

So for instance, solely for the allocation or assignment of work and compensation, and that it actually works as intended for that purpose and does not unlawfully discriminate. And so for folks who are thinking through feasibility issues, again, look at the opt-out requirements. Use your risk assessment in a meaningful way to think through these issues ahead of time.

 

[00:52:00] Neelofer Shaikh:

And then also look at 7221B for the opt-out exceptions, to the extent that they might be relevant for your team. Another theme I'm seeing, just because I want to address some of these.

 

[00:52:15] Neelofer Shaikh:

I think I saw some questions about the agency itself and in terms of resources as well as teams that might be complying with other jurisdictions. I just want to address that as well. So for the risk assessment, we did specifically look at other frameworks.

 

[00:52:40] Neelofer Shaikh:

You know, GDPR, the Colorado Privacy Act, we're mindful that businesses are potentially complying with these other laws as well. And so if you look at our regulations, we do allow you to use that type of risk assessment for compliance with the CCPA's risk assessment requirements, provided that you include any outstanding information that's required by the CCPA. And so you're not required to do duplicative assessments, but you are, if you're trying to use a different jurisdiction's risk assessment, you would need to make sure that you actually are including all the information that's required under 7152B, 7152 of the regulations.

 

[00:53:00] Neelofer Shaikh:

So please keep that in mind. And then with respect to guidance more broadly. So again, regulations, like the regulations themselves provide both the requirements as well as guidance on how to comply.

 

[00:53:20] Neelofer Shaikh:

The agency is going to come out with additional resources for businesses, but your best resource continues to be the regulations themselves. I think I hopefully addressed a bunch of them in that response, but again, and of course, Omer and David, please jump in with any thoughts that you may have. Cool.

 

[00:53:40] Omer Tene:

Yeah, go ahead. I was just about to say that there's kind of a more philosophical question in the background here, and I'm not sure it's the right time to open it, but does this framework even make sense, or is it good, or is it optimized? Because it kind of rests on the assumption that a human decision maker is better placed for these tasks than the machine. So one, I guess, question kind of testing that is, you know, just workload, what if the human decision maker gets hammered with like thousands of decisions, which I think started this discussion.

 

[00:54:00] Omer Tene:

But another kind of goes back to my earlier comment against the baseline. Is it true that the human decision maker is better placed to make kind of more important decisions here? This rests on logic that kind of emerged from the GDPR article 22. That's kind of the genesis of this.

 

[00:54:20] Omer Tene:

So, you know, GDPR is from 2018, was negotiated before that. So it's really like 10 years before the emergence of AI. And in fact, I think it might have even been in the 1995 data protection directive.

 

[00:55:00] Omer Tene:

So does the thought process that went into those laws that predate AI by 30 years still hold today? Is that like where, as a policy decision, we want to kind of shift the focus from these tools to human decision makers? I'm not sure. Will human decision makers make better decisions at scale? Just throwing it out there.

 

[00:55:30] Daniel Barber:

Yeah, I mean, I think that's, there's actually a few questions related to that, Omer.

 

[00:55:40] Daniel Barber:

I think, you know, Jeff's question of if we review the regulation, we find our products, oh, sorry, Chandra's question of what would happen if there, when there is a variance in implementation of human involvement or role in the significant decision, e.g. one manager uses discretion and their own judgment in a compensation decision, whereas other managers go with whatever the ADMT recommended. I think that's actually what you're picking at, right? And I think probably what we're all trying to understand from a policy perspective, how would one think about that, Neelofer, just in terms of, you know, anything you'd guide in terms of practical implementation based on the policy or the regulations as they exist today?

 

[00:56:00] Neelofer Shaikh:

You know, I mean, our regulations specifically, again, address things like the risk assessment and how to address privacy risks to consumers, including things like discriminatory harms. And so, again, that is something to think through.

 

[00:56:20] Neelofer Shaikh:

And you, again, I think businesses should think about those risk assessments as important tools to help work through these issues. Honestly, though, I think it really comes down to me to just, like, this is me now speaking very much in, like, my own perspective as a consumer and not as a government employee. But I think it really is thinking through, like, how would, like, put yourself in the shoes of the person you're making the decision about or using ADMT to make a decision about and really think through, like, what would feel, you know, like, consumer protective of them? What feels fair to them? What feels just to them? I think that honestly should be a really, like, that should be a guiding principle here throughout your processes.

 

[00:57:00] Neelofer Shaikh:

And so that's all I can really say about that. But, again, I'm sure from a practitioner's perspective, the other panelists might have different ideas.

 

[00:57:20] Omer Tene:

And, you know, I think Niloufer is charged with enforcing the regulations. She didn't necessarily write them or kind of conceive the logic. But, again, my point is, you know, be careful what you wish for, because you might get the human decision making and might then want to opt out of that or kind of appeal to the automated decision making.

 

[00:58:00] Omer Tene:

So, you know, which kind of type of decision making is more anchored in kind of truth, if you will, or facts or logic, as opposed to, you know, bias, discrimination or randomness? I'm not sure what the answer is. I do, you know, sense that there is kind of a feeling that automated decision making is cold, right? It's like a machine decided something. And in certain contexts, it seems wrong.

 

[00:58:20] Omer Tene:

For example, in judicial, you know, sentencing, like, you don't want, I think, an algorithm kind of sending people to jail. But, like, does that cut across all of these very wide, you know, this wide swath of decisions? I think, policy question.

 

[00:58:40] Daniel Barber:

Yeah. Anything you want to add, David, as we wrap up?

 

[00:58:50] David Khalily:

I think, and I appreciate everyone asking all the questions. I'll try to do my best to answer those in a different forum, and I'm sure IAPP can support.

 

[00:59:00] David Khalily:

Any last? My last thought is sort of, you know, just always kind of reflecting on the pace of advancement and sort of, you know, Anthropic's open sourcing of MCP, sort of powering this agentic revolution. I think we're headed towards a future where systematic end-to-end automation of processes is going to be pretty normal. And so I think a lot of what Omer is raising is really relevant that in, you know, in the future, hopefully those tools will be built and operate.