Closing Fireside Chat: Can Data Privacy & Ethical AI Coexist

Daniel Barber (00:12):

Wonderful. Thank you Mary. So actually I think just a little bit of an expanded intro might be nice. Barb, do you want to just give us a little bit of your background and where you've come from and how we got to here?

Barbara Lawler (00:24):

Sure, thanks. Thanks to everybody for sticking out with this. Before we head up to cocktails, I understand that we are the last stop before that. So my name is Barb Lawler. I am the president of the IAF, which is the Information Accountability Foundation. We are a not-for-profit think tank focused on global information policy. We function as a convener for multiple stakeholders, regulators, academic civil society around information governance issues. We provide a leadership forum for privacy and data protection leaders to essentially have comfortable, confident, confidential conversations. We help organizations think about what responsible and accountable data use frameworks should look like.

(01:09):

I'm a Bay Area native, I was born and raised in San Jose. And prior to coming to the IAF, I've been a Chief Privacy Officer three times for Hewlett Packard when it was one great company, yay, for Intuit for several years, and then more recently, for Looker Data Sciences, which was acquired by Google Cloud. I spent the entire first year of the pandemic under the Google Cloud, so to speak, and then more recently joined the IAF. And the reason I do this work is that I've implemented a variety of complex multi-business unit privacy programs and data governance efforts, and now I get to turn it on the other side and help other organizations and my current and former peers do the same.

Daniel Barber (02:00):

Lovely.

Joshua Browder (02:03):

My name is Josh Browder. I'm the founder and CEO of DoNotPay, which is an AI legal assistant to give power to the people across all of consumer rights. I moved here eight years ago from the UK to study at Stanford, and the first use case was parking tickets because I was a terrible driver. And we've since expanded to hundreds of different areas within consumer rights.

(02:25):

It started out using templates, sending templates to the government. But what's really exciting now is, AI has leveled us up 10 times. So we're doing things like negotiating people's bills with Comcast where the AI is negotiating on people's behalf. And what's interesting is the big companies are using AI, DoNotPay is using AI, so the two AI are talking to each other to lower people's bills. And so I think it'll be an interesting discussion on how AIs can chat with each other and building useful products with AI.

Dazza Greenwood (02:57):

Hi, I'm Dazza Greenwood, a former in-House counsel, and more recently, going back to my technology roots through my company Civics.com, which is a legal tech consultancy. And I also run a research group at MIT called Law.mit.edu where we do what we like to call computational law. But basically, my area is in the intersection of law and technology. I should probably mention, since I see they say internet protocol consultant there, one of the main things I've been doing through Civics.com is something called the Data Rights Protocol, which is based on a consortium of companies developing a standard technical specification for sending and receiving CCPA and similar requests.

(03:51):

And DataGrail is a member of that consortium, consumer reports and other companies. But really, since December of last year, I've been really all about generative AI, which is, I've been able to achieve things I've been trying to do since the 90's in law practice when I try to script things, and certainly in the research lab and different consulting engagements and industry and government over decades now. This stuff is delivering, and I can't wait to talk more about the capabilities and also the risks and perils.

Daniel Barber (04:30):

Lovely, thanks Dazza. So I think it could be interesting just to kick us off, Barb, when we connected, you mentioned a reset on what data means under the context of AI. Perhaps you want to share a little bit about what that means to you and why that's important?

Barbara Lawler (04:49):

We were just talking about that right before coming on stage. And I think the first thing is, what I heard earlier today, is references to what I think of as older, more structured thinking about data. So I heard PII, I heard personal information, I heard a little bit personal data, more of the European Union style term, but I think when we're talking about AI, what we really need to do is think about data pertaining to or about people, which yes, it broadens it, but if you look at definitions we're seeing coming out of the US states in particular, it's really talking about data pertaining to people. And then when we look at proposed principles, whether it's coming out of the White House and the proposed principles, there are, gosh, hundreds of AI governance principles coming out of corporations and think tanks and academia around the world, quite honestly. But I think it's helpful into reset is, think about data pertaining to people.

(05:52):

Now I realize if one is in compliance or security that we think in terms of data classification, and that there are specific controls and requirements set for each classification of data. But when you think about these large sources of data that go into training and testing a model before the AI is even created, it is an amalgamation of all different types of data, from people, about people, slightly pertaining to people, and not at all pertaining to people. And so, how do you govern it in that sense? So on one hand, just calling it all data seems simplistic, but thinking about what we should be governing and thinking about, we need to think about data pertaining to people in a very broad sense.

Daniel Barber (06:39):

That's interesting. I mean I think it was related to your comment just now, Joshua. I am curious how you think about that. You've got access through DoNotPay to so much data. Some of it is PII. How do you think about that with respect to AI?

Joshua Browder (06:54):

So I'm worried about all of these AI laws coming into place where they say you can't use any AI on people's data because AI is only useful just like with humans, if it has some data to operate on. If only the evil people get to use AI to impersonate people's voice and scam people and stuff like that, then AI can't be used for productive use cases.

(07:18):

So to your point, we use AI to negotiate people's bills, and that requires the address of the person so that AI can say, I got five outages in the past year, and it takes the address and puts that through and works on it. And so, what I worry about is people's data coming up in another conversation maybe from a different company using the model. So we use open AI's model, and may say we will not train on the API data, but I don't think that's true for the regular ChatGPT. And so, we had competitors of OpenAI approach us and say, "Do you want to use our platform?" And we were less sure about whether they're using the data, because the last thing we want is someone typing on ChatGPT and our data is appearing there. So I think there has to be safeguards to make sure that if you give data to a model, it doesn't spit it out elsewhere.

Daniel Barber (08:19):

Curious as to what you've seen in the field, you're involved in so many different organizations, how do you think about the relationship there?

Dazza Greenwood (08:27):

The relationship between data and generative AI, it's profound. So I think superficially, the first thing people think of is what was in the training set, was there PII there, could it pop out later or fine-tuning list like Joshua was just mentioning through prompts for example. The second thing people think about is leaking. So am I leaking something confidential through my prompts that could be part of the training data, but just the normal data hygiene, it's how secure are all the endpoints, everything like that. But I think below the surface is it's almost existential. It really comes down to the question of what is the data when it's been transmuted into vectors after the training process and the other processes, it's basically become math. And so that raises questions. Arguably the most existential questions, what is it, what kind of property is it, and who owns the property?

(09:34):

These questions are up for grabs now and they're adjacent to privacy and personal data rights. But I think the other thing is, if you flip it, although there's a need for prudence and risks, to me, the big story with data and generative AI is how powerful data is when you have a lot of parameters and you make a big foundation model and how much more powerful it can be when you add your own data to it. So I've been doing a lot of stuff with companies and public sector entities using their knowledge bases a lot with law firms and in-house counsel, and people is going to be, I think, the next wave in who knows 2024, '25, '26.

(10:16):

And we can start to ingest and load all of our personal data into private on-premises data models that can then be our agents to do things like what Josh is doing as a service now and even gain insights into ourselves and to help us use the same technology to find deals and to do personalization. I think personal data could be the rocket fuel for individual sovereignty and autonomy in the future by safely using it to feed on premises, maybe on phone, models.

Daniel Barber (10:55):

That's interesting. I mean we talked about in our session prior, just how technology can be used for good and evil. I think Joshua, what you're working on is super cool and I think that there's many other ways that AI can be used in very positive ways. I'm curious if you've seen any particular ways that have stood out to you, and some of them can be do DoNotPay, if you choose, or perhaps others.

Joshua Browder (11:25):

I think the technology is, it feels like we're making decades of progress every few weeks in terms of the AI technology landscape. The thing that really excites me is the rise of AI agents. So AI agents is when you kind of string together a bunch of GPT4 or API requests, so you have an almost autonomous agent that can take actions.

(11:47):

So we were testing something at DoNotPay. I connected my bank account to chatGPT and I asked it, "How can you save me money?" Just a broad question. And it started by disputing transactions, canceling subscriptions. And AI is not useful unless you connect it to the real world, so you can't get anything done in America apparently, unless you connect it to the mail service. So we connected it to the mail with Lob API. So I think connecting it to the real world and having something that can actually do multiple tasks to achieve a goal is what's most exciting, versus just talking to a generative AI like ChatGPT.

Daniel Barber (12:28):

Curious, Barb, if you've seen anything cool that is similar from a “good” standpoint.

Barbara Lawler (12:35):

I think we get so wrapped up into all of the potential bad cases and juicy stories that it's easy to forget about the good stuff. One of the things that I've certainly learned from the companies that are members of the IAF, and a lot of those are household brand names that you would recognize, is at least for larger to mid-sized companies, the vast majority of AI usage has nothing to do with the bad scary use cases. It's all about organizational optimization, faster order to cash, faster order to inventory, inventory to cash, improving distribution channels. And I think where it starts migrating into the end user, the end customer spaces, then obviously you see this in a CRM system, whether it's HubSpot or other companies.

(13:24):

But here's some examples that we don't always think about. We hear AI and data as input to AI as providing benefits for society. So what do we mean by that? And talking about climate change. So how many people have been tracking all the smoke in the air around here in the last couple of days? All of that information that we get real time in those little maps that we can zoom in on our phone, that's really AI sitting behind that. Does anybody have the app on their phone yet that tells you when an earthquake's coming, which is still kind of a beta thing? That is incredibly AI based, not so much related to people, except for the location piece, depending upon when you get a notification.

(14:06):

One of our IAF members, it works for a very large pharmaceutical and medical device company. That's a name you would recognize. They make band-aids also. But what they talked about where they're going with AI are smart joint replacements with sensors tied to an app on your phone that also communicates with your physician that tells you when your knee replacement or your shoulder adjustment or your hip replacement, which no one in this room is probably old enough for either one of those yet, but maybe it's your parents or your grandparents.

(14:43):

But smart joints and devices that show basically, did the surgery take, is it working properly or are you doing something that perhaps disrupts that? So that's just a couple of examples. Some really interesting, I think, fascinating stories and usages about AI that often don't really get the coverage or attention they deserve.

Daniel Barber (15:06):

Dazza, I know when I go to you, you're going to have a list of about 15, so…

Dazza Greenwood (15:10):

I know. I'll try to control myself, I promise.

Daniel Barber (15:13):

Just curious what you've seen.

Dazza Greenwood (15:15):

I'll take your prompt to say what are some good things, be concise, a concise bullet list. So I am very bullish on generative AI. This has transformed my life, my professional life and some social things that I'm doing as well. It's transformed my client's work and I can't think of an area that it isn't already beginning to transform. And it's almost a hyperbolic, overused term in the technology space. But look at education. I don't know if you've tracked what Khan Academy is doing. For individuals to be able to have tutors, that's the dream, for personalized education. And then for law practice, being able to upskill people, being able to upskill virtually every kind of worker, especially knowledge workers. There's great evidence out of Sloan School and Bureau of Labor Statistics and others are already doing a lot of good studies in everything from call centers to highfalutin consultants.

(16:32):

This stuff is changing everything in law. One of the things I love and I just posted on LinkedIn about it is summarization. It's not even that mind-blowing, it's not a hip replacement, but it's just the number of hours I saved. Being able to quickly, accurately, very on point, basically zero hallucination, to summarize things and then to be able to use some of that time and give it back to thinking about it going back and going deeper has been a game changer. And also issue spotting in a legal context. So by putting in information about a case or about a client or a deal's disposition and trying to get what are the key issues, what are different options, pros and cons for structuring, it actually does reasoning pretty well. If you want the whole download, look me up on LinkedIn and I post at least once a day or several times a week on bullseye after bullseye of the very beneficial, very useful, very productive, very democratizing in some cases, capabilities that are being unleashed with regenerative AI specifically.

Daniel Barber (17:47):

Lovely. All right, let's switch gears a little bit. Josh, I'm going to go back to you. What do you see in terms of some of the maybe unintentional risks or consequences of AI? We just talked about some fun stuff, but there are some things that are a little scarier. Where do you see those going?

Joshua Browder (18:06):

A year ago when you saw a video or a photo on Twitter or the internet, you could trust that it was real, or maybe two years ago. Now you can't trust that. And I think that's going to lead to an exponential rise of scams. We're already seeing a lot of reports, like elderly people, they receive a call, supposedly from their grandson saying, "I need money, grandma, can you please send me the money?" And a lot of people don't understand what's going on with technology. And so when they hear their grandson's voice, they think that's who's asking them for money. So I worry about an exponential rise of scams, and I think that the big platforms like the telephone companies or social media companies should use their own AI to say, "This is a synthetic voice talking to you. This is not your grandson."

(18:57):

Earlier today, everyone was talking about dark patterns. I have two views on this. On the one hand, I think AI will end a lot of dark patterns because you have, the whole reason these big companies do dark patterns is because no one has the time to jump through these hoops. And that's a good job for AI. But at the same time, AI can also exacerbate dark patterns. So AI can calculate just when you're about to cancel your subscription, it throws you a popup because the AB testing is so sophisticated, they know that you'll give up at that specific popup. So I worry about dark patterns and scammers.

Daniel Barber (19:37):

That's interesting. Barb, I'm curious what you've seen. I mean you're working with a large number of organizations through the group.

Barbara Lawler (19:46):

One of the things that I'm most concerned about is basically a new wave of identity takeover and the faking of identities. Back when I was at Intuit, we dealt with some of the early waves of account takeover and automated account attacks where people were trying to create false identities to file tax returns. That was a big problem. It had some financial implications that also gave us the opportunity to have really nice close relationships with the FTC and the IRS and multiple state attorneys general. But when we think about, going back to what I was saying earlier and not contradicting myself, but when we think about the data that's out there about people and the type of synthetic profiles that can now be created because of AI that could, whether it's financial fraud, but really falsifying identity, it also creates the situation, Joshua, that you were describing.

(20:46):

My mother got one of those calls, but it was several years ago before AI, and it's only scarier now. So when I think about risk areas, I think some of the other risk areas, and hopefully we had a chance to talk about how are companies thinking about preventing some of the bad things, certainly in hiring, where we think about bias, in lending. And hiring and lending already have rules around them.

(21:10):

And then the last piece would be just around, what do I want to say? Does anyone know who Michael Reeves is? Just any YouTube followers in the room? Nope. Okay. All right. I thought I'd give it a shot. Robots doing bad stupid things because they do what we tell them to do, not what we need or want them to do. Look up Michael Reeves. He's hilarious.

Daniel Barber (21:40):

Noted. Dazza, curious [about] your perspective here.

Dazza Greenwood (21:44):

Okay, I'm a robot that will do what you tell me to do. Nice crisp answer. I agree with everything that was said and I might just add some color to some of it. So on the identity, it's not identity theft so much, it's more like, it could be like a very convincing criminal impersonation, like doppelganger, which is weirder than identity theft. I don't know how many who looked at ElevenLabs and any of the deep fake stuff, but it really is something to look at and something that we have to get ready for. I think that's part of Superset though, which is just, stuff is so good that it is very convincing at developing things that may seem like fact, and this shows up in a lot of ways. So synthesizing persona, is a problem for a person that you're impersonating, but it's a problem more generally in terms of just accelerating this corrosive trend of what is a fact that that we've been plagued with, most notably since the 2016 election.

(22:58):

But it was very manual to synthesize facts and to do all the data and analytics needed to manipulate people appropriately or effectively, I should say. It's very inappropriate and unethical. I think the cost structure in terms of the volume and philosophy of being able to flood the zone with very convincing indistinguishable from human content is going to be difficult. I run a publication through my MIT affiliation called MIT Computational Law report. We're already getting this huge uptick in stuff, our volunteer editorial team is crushed. And there's no question in our minds that a lot of this is due to the easy access to generative AI.

(23:44):

I think more broadly though, if you look over the horizon slightly, and this is the last quick point I'll make, in the law where I really focus mostly, there's a question that's arising right now with the new crop of associates that are entering, especially some of the smaller firms and smaller in-house council where they're adopting and embracing generative AI for certain tasks. I think there's actually potentially a problem of over-reliance, because when you're not doing these things manually, if you project out two, three, four years, and especially if we're reducing the number of associates and new hires, how are people going to learn how to do the basics of the practice of law or their other professions such that they could compose intelligent prompts or evaluate the outputs or just learn how to do things and be able to apply critical skills and judgment and experience. So I think there's any number of risks. Nonetheless, it's awesome. Go for it.

Daniel Barber (24:43):

I loved how you just spun that around. All right, so changing topic here. Obviously, we might've seen that there are tech leaders that met in DC talking about civil rights and advocates were part of the conversation as well. I'm really curious, Barb, actually, your perspective on where they landed and what you see happening here. Are we moving too fast? Are we moving too slow? What's your take?

Barbara Lawler (25:13):

Yes. It is a little bit of both, but when we think about what's happening and not happening in the regulatory space, I think there's a couple of layers. We need to think about it in the US environment, which is, there's a lot of wonderful political and policy talk and there will always be wonderful political policy talk in Washington DC and in the state houses. 20 years ago, I sat in front of both the house commerce and senate commerce committees when it was John McCain and John Kerry talking about privacy. Yes. You know, right?

Dazza Greenwood (25:49):

I remember.

Barbara Lawler (25:50):

Privacy legislation. Because it was right around the corner.

Daniel Barber (25:52):

How's that going?

Barbara Lawler (25:55):

Yeah, exactly. Well, in 2011 kind of did the same thing. Another version of McCain Kerry and privacy legislation. We had the Obama administration putting forth the privacy bill of rights for Americans. The FTC was starting to build out something that they would do. And then again, as you said, where are we now? It's just right around the corner. We talked about 80 PPA last year. It's right around the corner. I don't know if it's ever around the corner. It's like one of those games where it's just you keep going around the corner and it never happens. But here's where it is happening. It is happening at the states [level].

(26:29):

And we heard a lot of talk about California earlier today, but the question I have for you, particularly if you are a privacy officer or privacy consultant or general counsel in the room, how many have read the Colorado State Privacy Rule regulations? Thank you. Yay. A couple of people. Usually I get zero on that. We have looked at that because Colorado is already out there with very specific rules on what to do about profiling and automated decision making, which as we heard earlier today from Rick, also in California is the proxy for AI regulation. So Colorado's regulations already in place require you to look at, from a governance perspective, the impact on all stakeholders, not just us as individuals, not just the corporation, but to groups of people and to society. That's a fundamentally different change.

(27:26):

The other change is they can ask for that assessment to prove you've done that assessment and thinking on demand. How many of you are ready to just hand that over within the next couple of weeks if you get that request? Now, I may not care about Colorado, but California and Colorado, absolutely. I will tell you behind the scenes are talking the proposed California regulations look like Colorado plus. So it's coming. We see what's coming in Washington state with My Health My Data, which is an incredibly complex morass of well beyond health. And then we see things like in the city of New York where you have to prove that you've done an anti-bias assessment and statement for hiring. So the regulation is here, and I think if you talk to state AGs or listen to what the FTC and other agencies at the federal level are saying, there are already rules that apply to the world of AI, even if it's not AI specific rules.

Joshua Browder (28:24):

Yeah, I completely agree with that. A few days ago, the Consumer Financial Protection Bureau issued guidance and they said, "Whenever you deny someone credit, you have to say why." And all of these credit companies were using these AI models and they were just doing these standard forms saying, "We've denied you because your credit score is too low." But in reality, that wasn't true. What they were actually doing is using an AI model and they don't even know why they were denying people credit. And the CFPB said, that's not acceptable. Similarly, the FTC went after some people doing a Ponzi scheme with AI. So I think that everyone says we need all these new AI regulations, but there are so many great existing laws on the books to help people and that should be used too.

Daniel Barber (29:10):

So you really set that up for me, thanks Josh. So just in the absence of regulations, so we all agree there's some value there, we should continue. And I think, Barb, you were touching on it earlier, what can companies do now? I'm curious [about] your perspective, Josh, advancing AI, this is great, we want this, but how do we think about putting the right guard rails in place or seat belts as the example was described earlier?

Joshua Browder (29:41):

Just being a foreigner, I love America. I think it's the greatest country in the world, but I think it's deeply broken and there are lots of problems with how things work here. And one of the biggest problems I see is concentrated benefit, but spread out harm. So United Airlines can charge a million people a $10 fee, they make $10 million, but the people being charged $10 can't really fight back.

(30:06):

And so I think that to the extent there are some new rules, we need to stop AI being used to hurt ordinary people. But I don't think it's a good idea to put any limits on what consumers can do. I think the CCPA is a great example where it had these amazing intentions of people being able to delete their data and stop it being sold. But the implementation from a lot of shady companies, a lot of good companies implemented it very well. But a lot of shady companies didn't do anything or said you have to send in an email or send in a registered mail letter. And so I think that AI will help automate some of those things. But you shouldn't put guardrails on it for consumers, but you should put it on big corporations.

Joshua Browder (30:54):

Dazza, I'm sure you have commentary on that one.

Dazza Greenwood (30:57):

Yeah, I see it the same way broadly. One of the things, specifically because we've been pushing the boundaries of what generative AI can do for law and legal processes, it became very aware very early that there are risks and flaws and limits with this technology that can get you in big trouble if you overly rely on it or use it for the wrong thing. So one of the things we did to fill the gap between statute and regulation and responsible practice at MIT was convened a task force called the MIT task force on responsible use of generative AI for law.

(31:38):

And so it's a great group of members, and we've done public comment and have a whole bunch of people involved, but you can see the report of the principles and guidelines draft at law.mit.edu/ai. And I think that's a work in progress. But that's a good example of the need to look context by context, domain by domain, company by company, and develop guidelines and policies and procedures for the responsible use of the technology. I think that's the very first thing to do. At a higher level, I should just say I'm a big fan of the, I mean, welcome to America. I'm a big fan of the UK where you came from.

Joshua Browder (32:24):

Everything works better there.

Dazza Greenwood (32:26):

And if you all haven't checked out their generative AI policy, take a look at it. And just yesterday or two days ago, they announced another of the implementations where they're basically helping businesses to figure out how to use the technology. So it's very pro-innovation, but they've also got a bunch of different regulators that are available to talk about how existing regulations may apply to your novel use case. Very balanced, very forward-looking, but also applying existing bodies of law. That seems like a very sweet spot to go for a while until those demonstrated market failures or other harms that we could generate regulatory and statutory remedies for, not based on speculation, but based on demonstration of where the need is.

Daniel Barber (33:15):

Barb, why don't you close this out on this one? I feel like you've got [something to say]

Barbara Lawler (33:19):

I do, I do. I can't help myself. What can companies do now? So I think the first thing is, if you have developed or are in process of any kind of data mapping and data inventory work that is foundational to privacy, you can't even have an accurate privacy policy, in my opinion, if you don't know what data you have and where it is and why and where it's going. So that is foundational. We heard some talk about privacy by design earlier. I would say put a twist on that, and that's really assessments by design. In other words, maybe it's not a monolithic sort of PBD kind of activity upfront, or maybe for a lot of privacy teams, it's at the end of a process right before launch. It's what are the chunks that can happen at different states of the product development or IT solution or model development process.

(34:11):

I think if we look at what's come from the states, California, around the corner, Colorado today is, there are mechanisms that you can use today to look at what should be in place. The IAF actually developed a draft Colorado assessment framework. It's on our website, that's information accountability.org. You can also find out more there if you'd like to join our really small group of forward-looking companies who are also thinking about these issues. Then the last point I would make is, I've been at three major sessions in the last six weeks where big brand names that you can think about are talking about how wonderfully they're doing AI governance as if it had been around for years, and good for them. But the fact is, there are leading companies or companies who want to be leaders and responsible and accountable and answerable, who are developing frameworks based on existing mechanisms - whether it's GDPR, CCPA - and they're looking ahead to the EU IA act, which will get passed by the end of this calendar year and probably go into effect sometime in the next year.

Daniel Barber (35:22):

Well, lovely. Thank you, Barb. Thank you, Josh. Thank you, Dazza. And that wraps up this session.