This episode is audio from The Consumer Scientist panel from the Engelberg Center's Conspicuous Consumers Symposium. It was recorded on October 16, 2025.
Announcer 0:00
Welcome to engelberg center live a collection of audio from events held by the engelberg center on innovation Law and Policy at NYU Law.
Announcer 0:10
This episode is audio from the consumer scientist panel from the engelberg Center's conspicuous consumers symposium. It was recorded on October 16, 2025
Christopher Morten 0:23
great to see you all. My name is Chris Morton. I am an associate professor here at NYU Law also director of a clinic here at NYU Law School called the Science, Health and Information clinic, and a faculty affiliate of the engelberg center. And I'll echo others in expressing thanks to
Christopher Morten 0:45
the organizers to the engelberg center itself for today's exciting symposium. This panel is called the patient, excuse me, the consumer scientist, and I'm going to introduce our eminent panelists in just a moment. But in short, to give you a preview, this is a discussion about some of the ways that consumers of medical technologies, that is, patients can and do much more than just consume those technologies. And we've brought together some eminent patient advocates and professors to talk about how patients and their loved ones become experts and agents for change. They advocate for new laws, new policies and new research. Sometimes they themselves conduct research. They invent and test and popularize new technologies. I hope that this discussion will underscore the ways that innovation law and policy are not just for patients or about patients, not just for consumers or about consumers, but that consumers themselves actually drive a lot of the innovation, a lot of the law, a lot of the policy. Three quick points I hope you may see emerge from this discussion, one, that the technical information that our law and policy generate and disseminate. So think, for example, that FDA regulation that that information is used not just by self styled experts, but also used by everyday patients, consumers and their loved ones. Second, you may see that some important features of our law and policy, especially in health, law, have been invented and advocated by patients themselves and their advocates. And third, I think, a theme that may emerge is that law and policy should think carefully about how to generate and present patients, sorry, generate and present information to patients and their caregivers in ways that support all of the things that patients do, their decision making, their research, their advocacy. Okay, we have 75 minutes, and we're going to be a relatively informal discussion. I think I'm going to introduce the panelists, ask them a couple prepared questions, let them discuss amongst themselves, and then we will save some time for Q and A as well. Okay, so I'm joined by four people, and I'll introduce them sort of down the line. Fran visco is the president of the National Breast Cancer Coalition NBCC and a member of its board of directors. NBCC, if you don't know, is a grassroots patient focused advocacy organization that unites 10s of 1000s of people from across the country around the goal of ending breast cancer. Fran herself is a survivor of breast cancer. She is also a lawyer with a very distinguished litigation career before she dedicated herself to patient advocacy. Fran is also a member of the Board of Directors of translational research in oncology, an international nonprofit Clinical Trials Consortium, and a leader in both cancer research and in empowering patients to shape and use medical research. She's a recipient of the Presidential Citizens Medal, the second highest civilian award in the United States. Hilary Cook is a public school teacher and newspaper columnist. She's also the parent of a son living with type one diabetes because of her son's condition. She's become a noted advocate for people living with diabetes. Hillary has led the Maine insulin for all chapter. She's testified before the Maine state legislature about access to insulin legislation, and indeed has gotten not just one, but I think two, bills passed into law in Maine. She's also served on Maine's expert commission to study the possibility of nonprofit insulin manufacturing, as you'll likely hear more from Hillary's remarks. She's also an expert on patient driven innovation in the diabetes community, especially looping the patient invented artificial pancreas technology that links an insulin pump with a continuous glucose monitor so that insulin is adjusted automatically and continuously throughout the day. Next we have Steve woloshin, a general internist and professor of medicine at Dartmouth's. Eisele School of Medicine. He's also Director of the Center for medicine and media at the Dartmouth Institute for Health Policy and Clinical Practice. Steve's research addresses the excessive fear and hope created by exaggerations and selective reporting in medical journals, advertising and the health news. He's worked to improve communication of medical evidence to physicians, journalists, policymakers, and, of course, patients and the broad public. He's co author of two books, know your chances and overdiagnosed. He's founder of the Lisa Schwartz foundation for truth in medicine. He's received the American Medical Writers Association John P McGovern award for preeminent contribution in research and enhancing the communication of medical evidence. And last but not least, he has an op ed out in the Washington Post literally today about communication of drug information to patients. So check that out. And finally, we have Charles Dwan, Assistant Professor of Law at American University, Washington College of Law. His research focuses on how intellectual property and technology law interact with public policy and the public interest. He's written on patent law and drug pricing conflicts between regulation and intellectual property and much more. Charles was a prolific and prominent public interest lawyer before he joined the academy, in addition to his academic publications, He has authored over 200 amicus curiae briefs, policy papers, administrative comments and media articles which have been cited in the Supreme Court, Federal Courts of Appeal, and more particularly relevant to today's discussion, Charles studies non commercial innovation by patients and other end users, and he's written on How law and policy might be reformed to better empower patient innovators. Okay, enough from me. The first question I'm going to put to these distinguished panelists is, how do patients and their advocates participate in the development of medical science and technology beyond simply consuming medical products? Can you give us an example or two of how patients and patient advocates have used scientific and technical information and Fran I'll turn to you first.
Fran Visco 7:07
Okay, thank you. So I want to say to begin with that the National Breast Cancer Coalition is a coalition primarily of diverse organizations from around the country, and our mission is to change, is to end breast cancer. And the reason we are a coalition is because we're about system change, and when you change systems, you have to have a diverse perspective. I always say it cannot be what five middle class white women think should happen in breast cancer. So that is sort of who we are, and to achieve our mission to end breast cancer, we have three goals. One is research, to make certain that there is sufficient funding for meaningful Breast Cancer Research access, to make certain that everyone has access to the quality care they need, because we won't end breast cancer unless everyone has access to the results of science and research, and the third equal goal is influence, to make certain that advocates have a seat at every table and are part of All decision making at every level in breast cancer, everything that affects their lives. So some examples of how we do our work. The very first thing we did when we were formed in 92 was figure out how much money the federal government should appropriate for breast cancer research. We did our own research. We heard from scientists across the country of what research was promising but underfunded. We did independent look into what was funded and what we saw was lacking, and then we came up with a plan that the government could spend more than 300 million new dollars for breast cancer research, and that that money could be well spent. We had no interest in throwing money at the problem. We were successful, and as a result of our work, the Department of Defense peer reviewed Breast Cancer Research was launched. Now that program funded through the defense budget, and I'm not going to go into the details of why that happened. Gave us a seat at the table to help influence how those funding was going to be spent. So we had a seat at the table in the beginning, and we continue to have a seat at the table. And because of our influence, that program is required to include educated and inform patient advocates at every level, at the level where we set the agenda of what's going to be funded, we design the mechanisms of how those funding, that funding will be spent, and we also require that. Scientists who submit proposals to that program have to collaborate with science, with advocates, so in meaningful ways. And so we have never believed that patients are just consumers. We have always believed that patients are leaders and collaborators, but a very important piece of that is they have to know what they're doing, because you can't just be a patient advocate and sit at the table to bring your own story to the table, and you can't do it when you don't really know how to meaningfully influence and so we created education and training programs, the only one of which I'll speak about at this very moment is project lead, and that is a science training program for lay advocates. It is incredibly complex, difficult. We're educating advocates to understand the language, the concepts, the process of science mean, how to analyze clinical trials, how to interpret statistical information. What does the self cycle look like? Because if we're going to be leaders, and we believe we must be in setting an agenda, in determining what endpoints are, in developing public policy, we need to be informed and we also educate. We also have a definition of what we need by an advocate. I've been on many committees where I've heard scientists talk about how they work with advocates and they work with patients. You know my aunt who had breast cancer, that is not an independent voice in giving you advice in your research or setting an agenda. We define an advocate as an individual who is personally affected by the disease but also is part of a patient led advocacy organization that has a constituency again, because we believe that if you are going to participate, it can't just be about what you think. It has to be informed by the diversity that is breast cancer. And I have so many examples of different situations where we've been involved in setting agendas, in developing and designing research and science, but I'm out of time for this one, so
Christopher Morten 12:27
All right, thanks, Fran, we'll go next to Hillary.
Hilary Koch 12:31
So my story sort of falls on both sides, and I'm happy to follow you, because I think a lot of what you said, I can sort of echo as well. My my youngest son was actually first diagnosed with an incurable disease called hydrocephalus when he was six months old. He had his first neurosurgery when he was six months old, and I was told that he would have potentially 30 neurosurgeries throughout his lifetime. When he was two and a half, he was diagnosed with type one diabetes, and there's also no cure for that as well. And there's nothing more motivating though, in in your life than knowing that one mistake that you make with giving your child some medicine could kill them. You give too much insulin and your child could die. Give too little insulin and your child could die. And it's 24/7 so you go into the hospital and you get a four day crash training, and you're sent home with some insulin and some syringes, and you're told, good luck. And I am one of millions of people around the world who have to deal with this every single day, and I I gave up my career to be a caretaker for my child, and I was told by the endocrinologist that a cure was coming. I was told that there were pumps that would give insulin automatically, and I was told that there were continuous glucose monitors that would check their glucose. But what they didn't do is they didn't talk to each other. And the other thing about it is those continuous glucose monitors would send it to a receiver that had to be a certain distance to that person, and so suddenly I had put it next to a baby monitor, and then I would run downstairs, and I would hear an alarm go off, and I would go upstairs, and then I would hear the alarm, and I'd have to verify it with a finger stick, and then I would go through this on and off all night long, and and so these, these, these images that you see in commercials on television of a continuous glucose. Monitor, meaning you don't have to check glucose, is just not the reality. What I did have is I had a really great education, and I liked to read, so I read myself into medical advances, and I found an online community of people who lived with this type of thing every day and wanted to make life better, and they weren't interested in monetizing on it. They just wanted to live normal lives. So say someone happened to work for Microsoft, someone happened to have a scientific background, someone just wanted to make that alarm louder in the middle of the night, and that's sort of how it started, and they made an open source medical community so that this continuous glucose monitoring system could be viewed remotely, not just for parents or caregivers, but for friends. Lots of people live by themselves, and they want to share it with a loved one, a brother, a sister. So when my child was, I think, in second grade, I made a website for him, not that I knew how to do, but because someone gave the information, they made it open source, so that I could view his glucose remotely while he was at school. And I made a website, and I website, and I shared it with the nurse at school so that she could see it, so that he didn't have to miss important educational opportunities that are guaranteed to him by law, to walk down the hall to have a nurse check his blood sugar, but he could stay in the classroom. This was revolutionary and but the thing that was still missing was that that little insulin pump still wasn't talking to that monitor, and people were frustrated by it, and so they all put their heads together, and they figured out how to make them talk to each other. And one of the first systems that came out was by a husband and wife, Dana Lewis and Scott librand, and it was called Open APs. And another system that was made that was Swift code based, which was an apple based system called loop, and they put everything out there, and they made it open source, and people all joined forces, and they communicated with each other, and they said, here's how you do it. Here are the supplies you need, download the code, try it out, give us feedback, see what you think. And a couple years later, I did it for my son, and suddenly, for the first time in 10 years, I slept through the night, and his diabetes was managed better than it had ever been managed ever in his life, so much so that we didn't have to have quarterly appointments with his endocrinologist. We started going to appointments, and the doctor said, What are you doing? We want to learn from you. When he had to have a couple of neurosurgeries, we had to go to Boston for them, our doctors wrote us letters saying, Please let her manage his blood sugar. Boston gave us a hard time. I had to bring the letter. I had to bring data and charts. If you had tested his blood sugar, you never would have diagnosed him with type one diabetes. He would have said maybe he was pre diabetic, but you never would have said he was type one just based on his blood sugar and his a 1c but this was revolutionary, and for years, we used a DIY, open APS system that was non FDI, FDA approved device to manage his blood sugars. It was better than anything on the market, because everybody decided that this was nothing that they were going to wait for. They knew that if they put their heads together, they could make it better. They didn't hack a pump. They just hacked the code. And everybody knew that you had to make it yourself. If someone made it for you, then you know it would have been subject to laws and FDA compliance. But if you did it yourself, you knew you were taking a risk, and then if you had any questions, you'd go online and say, I'm having a question. And someone was available, 24/7, and for years we did this, and for years, it was incredible. My son now uses an FDA approved device. He is 19. There are devices that do talk to each other now, and part of the reason why they talk to each other now is because of the looping devices that were out there, because of the communities. And there are systems that are based on the algorithms that were in this community. There are still non FDA approved devices that are out there that are still years and light years ahead of what is available. But I would say that it is because of consumer scientists that there are better things out there, but we need patients at the table who are informed. Informed, making suggestions and being part of those conversations, because that's how their lives will be better and and I'll get into the second part. Then once I started getting sleep, then I started changing laws.
Christopher Morten 20:17
Thanks so much. Hillary, Steve,
Steve Woloshin 20:18
sure, so as we're hearing patients and advocates are often underrepresented in the development of medical science and technology, and that's at every phase, from the idea generation to the design testing and regulatory approval. But I don't want to use my time to talk about the products themselves, but I want to talk about the messages which explain what devices do, how well they work, and whether and how to use them. And this communication is really critical, but it's often an afterthought. But without it, products aren't going to be used properly. It's going to undermine the potential benefit and introduce harm. And unfortunately, public communication about tests and treatments is often persuasive. It's often commercial rather than informative, so trying to get people to buy stuff rather than informing decisions about whether or not to buy them. So think about direct to consumer drug ads. A lot of us have been writing about them for years. How they magnify benefit, they minimize harm, and they really fail to tell people what they need to know, which is how well drugs work, and that's despite the fact that these ads are regulated by FDA, but the oversight is spotty, enforcement is weak, and the result is a big missed opportunity for public health. And I'm going to say more in my next answer about ways to improve it, but right now, I just want to go through a quick example of how bad communication really undermined important product. I'm talking about covid home test kits. So early in the covid pandemic, the NIH identified testing as the key to getting back to normal. This was before there were vaccines, before there were treatments. But the problem with testing was that was expensive. You know, clinic based testing was expensive, and it was inconvenient, and it's also crazy, you know, to tell people, if you think you're sick you have covid, go to this public place, infect everybody while you're getting a test. So there was a lot of enthusiasm for the home test kits when they were developed, and early on, FDA gave emergency use authorization for some home test kits, and in order for to get that authorization, FDA required some device testing. You had to show that the device reliably identified covid Right. So that's basic stuff, and they also required some testing of the instructions so that people use them right? Because, you know, could you stick that thing up your nose? I mean, you all, everyone did a lot of that. Could you stick it up your nose and put it into the thing, and could you read the result positive or negative? So that's great, but the problem is that's just the beginning, right? Then what they didn't require testing to show was that people knew what to do with the results, right? And it's tricky, because interpreting test results is not so straightforward. You have to apply Bayes rule, that's the fancy name for it, which means you have to account for the accuracy of the test, but also the probability of in this case of an infection before you do the test, and by putting those things together, you can figure out what's your chance of being infected given a positive or negative test result, okay? And there's extensive literature showing that people have trouble doing this, and they can if you explain it, but it's easy to get get it wrong, and the authorized instructions didn't explain this stuff at all. So I was very worried about it. To me, they gave the impression that a positive test meant you had covid. Negative test meant you didn't, and that would be true if the tests were perfect, but they're not perfect. There's false positives, where the test says it's you have covid but you really don't, and they're false negatives, which says you don't have it when you do, and there are public there are important harms, both kinds of error, false positive, then people quarantine unnecessarily. Can be a real big problem from public health perspective. False negatives are probably more important because you go out there and you infect people. So we designed see whether people could understand this stuff. And what we did was we created scenarios, and we randomized people to, for example, a scenario like, where I live in Vermont, middle of nowhere. There are no people, no one had covid. If you felt well, you had no exposures, and your test was positive. It's very likely that's a false alarm, right. On the other hand, in the beginning of covid, like in New York City, if you're sick as a dog with typical symptoms, you have an exposure to a known person with a known covid infection, and your test. Comes back negative, you know, then you ignore that, right? It's crazy. That's that's what you have to do in order to interpret the test properly. So we did a trial, and we we found that we were, our concerns were really justified with some that authorized FDA instructions, 36% of people got within the high risk group, like the sixth dog, had an exposure negative test. Would go visit my mother, my poor elderly mother, which is terrible. And so we designed instructions which we thought were better, and they did work better. With our instructions, only 3% of people made that error, so it was a really big effect, and we didn't do the It wasn't hard to do. We used basic decision science principles and simple user testing. The whole study cost less than $1,000 this is very easy to do, curiously, because some people don't read instructions. So we had a third group in the Renmin, and we want to see what would happen if you didn't have any instructions. And it turned out they didn't do nearly as well as with our good instructions, but they did much better than with the FDA authorized instructions. So when we published the article in JAMA Internal Medicine, the original title was, nothing beats the FDA, but they didn't allow that. But the idea is that good instructions can make things good. Bad instructions can make things worse. It's a simple message. One of my mentors, Baruch fishhoff, is a cognitive psychologist at Carnegie Mellon University. He has a great quote. He said, an untested message can be as dangerous as an untested drug, and in this case, I think that's really true. So the point I'm trying to make is that effective communication of health information is really critical. Bad messaging can undermine even the best technology, and simple consumer testing is feasible and crucial. So we have to keep the target audience in the loop if we're going to make sure that the products that are developed careful rigorous testing, that we don't undermine it all by getting it out to market and not helping people understand how to use them.
Christopher Morten 27:16
Great. Charles,
Charles Duan 27:18
all right. Thank you. Thank you everyone. Thank you for inviting me to this conference. This has been really delightful to be among all of these friends and thinkers. I'm delighted to be at the end of this panel, because I get the benefit of all of the interesting things that they've been talking about. I'm like, the thing I really get out of this is, you know, the word consumer, right? Means somebody who just like, consumes things, but clearly we have people who are much more engaged in the process, in terms of advocacy, in terms of learning, in terms of actually doing things to figure out what the best treatment is. And it's that last part that I think is the one that interests me the most. There's a there's a somewhat cheeky phrase in the statistics literature called the N of one trial in which you're experimenting on yourself. I get that phrase from a wonderful article in the Harvard Data Science Review by a by two scientists, Kravitz and Duan. The Duan is not me. The Duan is a world famous statistician named naihua Duan, who happens to be my dad. And my dad is a very interesting story, because some years ago, he was diagnosed with sleep apnea. And being the statistician that he was, he wanted to know everything about it. He bought every device he could find. My poor mom had to deal with shelves of CPAP machines, and the real thing he wanted to do was he wanted to just figure out exactly what was going on with his sleep. And so he wanted to pull all the numbers and see what the patterns looked like. This was very hard, because the devices don't really let you do that. They're intended for you to just take the information over to the doctor and have the doctor do some sort of analysis on it, but what he was able to find was, and this is very similar looping story, an open source program called Oscar, which allows you to pull the SD card out of the CPAP machine, plug it in. It reads all the data. The data is encrypted, so some poor guy had to actually figure out all the codes and see every minute of what your sleep looked like. And this was just a wonderful thing for him. I think that these stories of patients who really kind of take their health care into their own hands, they're just so inspiring to me number one, but they're also interesting to me as an intellectual property lawyer, because, in a sense, they're the honey bee that shouldn't be able to fly. The standard theory of intellectual property is that innovation is costly. Innovation is difficult. In order to get people to innovate, we need to give them some sort of incentives. And the simplest incentive we can give them that actually accounts for the market value of their innovations is to give them some sort of. Of exclusivity, because that exclusivity ensures that other people can't just take advantage of what you've discovered and go make it that allows for the innovator to recoup their costs. But you look at the folks who are doing these sorts of things, who are releasing the software open source, and you have to ask yourself two questions. Number one, why are they doing it? Because they're investing a lot of time to develop this sort of stuff. The guy who developed the predecessor to to Oscar, it was a program called sleepyhead. His name was Mark Watkin. I think he spent hours, weeks, years of his time putting together the software, trying to figure out the codes on these machines so that he could actually download the data and put it together into these sorts of graphs that would be useful, right? He did that without any expectation of profit. And by the way, he shared, and that's the second question, why do people share these sorts of things without any expectation of returns. Now, I would actually be really interested in hearing what the rest of the panel thinks about that. Why do people develop and why do people share without that sort of standard IP incentive, and how do we account for that within the IP system? But I can tell you at least what motivated my dad. Why did he Why did he invest so much time in figuring out what was going on with his sleep apnea, because it helped him, because he slept better. And fixing your own health is a pretty important thing. Now, the last time I told people about this, the response was, okay, fine, so world famous statistician, of course, he's going to do that. But like, what about the rest of us? Well, let me ask you, have any of you, like, changed your diet to try to feel better. Probably a lot of you, well, you've just done an experiment on yourself, too. We all do this sort of thing. This is something that everybody wants to do, because everybody wants to feel good. Everybody wants their health to be in good condition. And so the motivation from just yourself is very important in literature. We call this user innovation, and I think this is sort of an understudied area of how the innovation economy works that really challenges that standard Intellectual Property Model. Why do we share because we believe in the sort of community. We believe in the sort of Commons. If everyone else is helping me out resolve my health issues, shouldn't I pay it forwards? And I think those sorts of things present this very different model of how we can innovate in this world, particularly in the healthcare space. So if I can be a bit of a counter moderator, I certainly would be interested in hearing whether or not that sort of tracks everybody else's thoughts.
Christopher Morten 32:38
Terrific, Charles, that is an excellent segue, I think, to our second canned, prepared question to be clear, which the panelists should know is coming, and it is, how well do our existing law and policy support patients in their many roles beyond mere consumers, as scientists, as political actors and so on, and are there any changes to law and policy that you would recommend to better empower patients and patient advocates? And again, I'll ask Fran to kick us off.
Fran Visco 33:08
So obviously, the existing laws and policy do not work well to advance that goal. But I think one you know, patients have been involved in helping change law and policy, and two, we found a way around them. So in changing law and policy, I think we, for example, worked very hard when the Affordable Care Act was being introduced to make certain there was language in there that required consumer participation in committees that came out of that law. And we also managed to get some language in there about defining what an educated consumer would be. Of course, really, it's not a priority for everybody else, so we have to be on top of it constantly to make certain that that's happening. You know, we also were very involved in the, as I said, the DoD program, and mandating consumer involvement there, but in a way to get around what exists. For example, we have a project called the Artemis project where, you know, we're seeing that in traditional NIH and other funding entities, really, the questions that are being asked are not the questions that we care that much about. So we identified two questions, how do you stop people from getting breast cancer in the first place? And second, if you do get breast cancer, how do we stop it from killing you? How do we intervene in and stop the process of metastasis? And we created our own project without very much money, because we don't fund people. And so we brought together what we saw as visionary scientists and advocates to develop plans on, how are we going to answer these two questions? And, you know, so the incentive there? There is just giving people the opportunity to be incredibly innovative and bold. As one of the scientists said to me, I went into science to be bold, but they don't allow me to do that you do through the Artemis project. So the very first question we asked was, can you have a preventive vaccine in breast cancer? And everybody rolled their eyes and thought we were crazy, but they helped us work through that question, and actually, in the spring, we're going to be entering into a phase one clinical trial for a preventative vaccine, with not a lot of money to get to that point. But we had some barriers along the way, one of which being intellectual property and the needs of different institutions out of which some of these researchers came, and we found a way. Took us about a year to work around that and work through it, and now we have a provisional patent and BCC on that vaccine, not because we want to make money. But because we want to be able to influence if this is effective, how can we make certain that everybody has access to it, rather than the market determining, you know, what's the largest profit we can make from this vaccine? So there are a lot of different ways that we have been involved in changing the law, example, the Affordable Care Act, changing the way research happens, and, example, the DoD program, and then going around existing law, like we have with the Artemis project, in order to make certain that we have some control over what happens. But I think what we're really about is, you know, we need to be there, leading and helping determine what the questions are, setting the agenda so that the agenda isn't set in order to determine and respond to the market or someone's career, but the agenda is set in order to say what is best for patients. So we are really about changing those systems so that the questions that are asked are really going to save lives.
Hilary Koch 37:11
Terrific. Hilary, yeah, I'll echo that as well. It really has to be about what is best for the patients and and a lot of times when we're talking about laws, that piece is lost. And having worked on passing some really, really important laws in Maine, I know that sometimes intentions can be really different than impact, right? You can have a really good intent on passing something that you think is important, but the impact is not the same thing. So when we talk about something like the price of insulin, for example, right? If you make insulin affordable for senior citizens, that's fantastic for senior citizens. What does that mean for people who aren't senior citizens? Right? If my son had to pay out of pocket for his insulin, he would be paying almost $18,000 a year. So it's something to think about when people come asking and pleading for a medicine, not that something like a pill to maybe lower cholesterol. It's not something about, you know, paying for groceries or paying for heat. It's something to keep someone alive. And so I can remember testifying for a law and them wanting to have a work day and them realizing that there wasn't a doctor around, and if my son's pediatric endocrinologist had been around, they would have said, Hillary can help you. Hillary can help you. But I don't have an MD behind my name, and so it didn't matter what I said to them. It's understanding that sometimes experts don't always have the initials behind their name that you think they need, but sometimes they have the information that you need, and sometimes you have to slow down and find out, what is it that I need to know that I don't know? So that's the one thing that's really important. The other thing is not not everything. When we're talking about like medicine, not everything is a matter of like, go to a doctor, get a medicine, and go home and take care of it. Diabetes is something so unusual, it's so peculiar, it's almost comical. You go to the hospital and they tell you it's more of an art than a science. If you're feeling sick, take. Some sugar or some candy, that's how we're going to cure it, right? Like it's it's bizarre, how much well, until you're feeling a little bit better, right? And everybody's different. Your diabetes may vary. Is what they say there. Every single person who manages it manages it on their own. They are the expert in their own body. And you can even go to a doctor and they'll say, How are you doing? What are you doing? What are your your ratios? Right now, every single patient makes those life decisions on a day to day basis, but the average person doesn't understand this. So when it comes to making customizations in product design, it's not necessarily that doctors know what patients need. Sometimes patients know what they need. Those customizations are really important, and sometimes customizations like differences in hormonal changes on a month to month basis that women might have, that men don't have. Those are things that women want in devices. They want a button to press when something happens each month, and they want their device to change for it. Those are things that are really, really, really important. So those are the kinds of things that I think can make a really, really big difference and and then the other thing is when, when you go into a hospital, the same kind of thing. There are laws that are really important, that are intended to keep people safe, but are really detrimental with people with diabetes. And we've talked about this before, for most people, when you go into a hospital, there are hospital policies that say if you have an insulin pump, you can't use an insulin pump. Most people who do use insulin pump, it automates everything, and suddenly they say, we're going to take care of your diabetes. People with diabetes are terrified to spend even more than just a couple of hours in the hospital because they're afraid they're going to die, and that's why, when my son is he's had five neurosurgeries. Now I have to, really, I have to fight tooth and nail to be able to take care of him. And a lot of hospitals fight it. And I understand why. But those are the kinds of things where, if patients take care of themselves on a day to day basis, trust them to take care of themselves when they're in the hospital, if they can, and and that's where the laws don't make sense, and those are the kinds of things that should be changed so that people can take care of themselves.
Christopher Morten 42:31
Thanks, Hillary, I'm going to go next to Steve and then to Charles, but I just want to flag. I'm going to give our panel a chance to react to others comments, and then I'm going to open the floor up for Q and A. So if you have questions, start jotting them down and getting them ready.
Steve Woloshin 42:45
Steve, okay, so I'm gonna come back to messages I talked a little bit about direct to consumer advertising before I just want to come back to them, because they're probably the most pervasive kind of health messaging out there. Companies spend, most recent data, over $6 billion on traditional, direct consumer advertising, which is a big underestimate, and how much there is because there's an account for social media, there's an estimate I've seen, that people see on average, nine ads a day. So it's a tremendous amount of advertising going on, and they make economic sense because the companies do get a substantial return on investment from advertising, but they may not make public health sense because a lot of the ads, and probably most of the ads, overstate benefit, understate harm, downplay uncertainties, and the result is people are overusing drugs of marginal benefits and may actually have harm. So many people have criticized the FDA for years for not paying enough attention to ads. Some people call for banning them. I mean, you guys are lawyers. You know more than I do that it's unlikely that there'll be a ban, but still, there's a lot FDA could do to make ads better, but they haven't done much. The system is reactive. It's slow, it's not very effective. But recently, the FDA announced, and this is a quote that they're no longer asleep at the wheel. That's a quote from the director and in an extraordinary and unusual burst of regulatory activity, they issued dozens and dozens of warning letters to companies about ads that they thought were misleading, largely because the visual messages implied that like a sick person with some bad chronic disease, will all of a sudden be out there dancing and singing and running through the mountains if they just take their drug. So the FDA said they don't want to see that anymore, so they're holding they're yelling at the companies. So that's a good start, for sure, but the miscellaneous images are just the tip of the iceberg. And if. If FDA is serious, they have to address the most basic problem, which is the ads fail to tell consumers what they need to know, how well the drug works. So the kind of information you need to know is, what is the drug for? Sometimes that's not clear. What outcomes does it treat? That's often very unclear. Does it lower your cholesterol? Does it reduce your chance of a heart attack? Does it reduce your chance of dying from a heart attack? That's critically important, and how well does it do it? That means quantifying the outcomes, right? You have to quantify it so people know what happens if I do take the drug or if I don't take the drug, and then the same thing for the harms, the most common and most serious harms. And it's only by putting all this together, having access to the benefits and harm information and weighing them, people can make decisions based on their values if the drug is worth it for them. Now, to do this would require changes in FDA regulations, and right now, the regulations are pretty not great. For example, benefit, right? You're required to talk in an ad. You're required to talk about benefits and harms, but benefit is defined as indication. So if I say the drug lowers cholesterol, I've satisfied the regulation. I haven't told you anything about how well it does, right? That's crazy, and there are no rules on how to present information. You have to be consistent with the approved labeling. There has to be this thing called fair balance. So if you say something about what the drug does, you have to say something about the harms. And then there's thing called adequate provision, which means you have to set let people know where they get a complete list of all the harms. But there's nothing about quantifying benefits or harms, how to do it, and there are no rules about pre testing or vetting the communicative accuracy of advertising, so that means seeing if consumers get the message right. Okay, so it's like the example I gave you before you can do simple testing to do that. So imagine there's a medicine that reduces heart attack risk. So here the relevant information is your chance of a heart attack with or without the drug. Let's say you have a 2% chance of a heart attack. If you sorry, if you 2% chance, if you take the drug, don't take the drug, 1% chance. If you do so, it reduces your chance 2% to 1% so that might seem like a big or small effect to you. I mean to a lot of doctors would say, anything that reduces your chance of your serious potential fatal thing by one percentage point, that's something to think about. But a lot of people would say, well, they're numerically small, and they almost round to the same number. I don't care. That's a value judgment, but you can't make that value judgment if you don't have the data. The other thing you need is not just benefit information you need harm. So let's say, if you take the drug and you get that benefit, what are the what's the cost in terms of harm? Does it cause bad indigestion in a lot of people? Does it cause rare but potentially fatal liver problems? Again, that's the sort of information that people need to make wise decisions about drugs, and you can't do that with the way it's done, like television ads, these rapid fire recitation of it's a joke that you see on the comedians make fun of because it's ridiculous. So FDA has an enormous opportunity to help and make things better, because unless consumers understand the risks and benefits, a drug ad hasn't informed them, and the only way to know if the ads work is by asking the people in the target audience if it works. And that's simple, basic consumer testing. So if FDA were serious about improving drug ads, you would require those kinds of tests before the ads can be released, and as I showed you in the covid example, the testing doesn't have to be expensive, and the companies are already doing tons of user testing to make sure that they spike interest in the drug it's very easy to add this kind of testing. So the test would be simple, show consumers the ad and ask them about the risks and the benefits and see if they're in the right ballpark. Yeah. So this could be really powerful tool for adding transparency to advertising into the to the market. Years ago, Lisa Schwartz and I developed a very simple method for doing this, a straightforward way to solve this problem called the drug fact box. So if you look at any consumer product, if you if you look at Franz syrup here on the back, like all consumer products, it has a little table which gives you the ingredients and number of calories, and calories broken down from fat or protein or or carbohydrate. So if you can do that for food products, why can't you do for drugs? Not give the ingredients, but these are the benefits. These are the harms, based on the best available data which FDA has, because that's the basis of drug approval. So the information is there, okay? And the idea. We just created this very simple display to present that we did tons of user testing, and we did trials showing that consumers want this information, they can use the information, and it actually improves decision making. So FDA Risk Advisory Committee voted unanimously that FDA should adopt this format, but they never did. It's a part of the Affordable Care Act also. But FDA, after one year, they were required to review all the data and decide what to do. They said they wanted three to five more years to review the data even better. And then, after three to five years, they issued a report saying that they weren't going to do it because it's too hard to do that, to let people know how well drug products work. So there's a lot of opportunities to change the rules here, but we need it's about a political will, really. So anyway, that's that's really all I want.
Christopher Morten 50:55
Great. We're developing a comprehensive reform agenda.
Charles Duan 51:00
That's great. You know, doing doing my summing up function as I am at the end of the table, I think there's this wonderful message of just everybody's different, right? Consumers are all individuals. We all have individual needs. And that explains why we need to have this sort of information so we can make decisions about ourselves that explains why we need to have why we need to have innovation that is directed in the ways that patients need, and why it's good for us to be able to adjust treatments in accordance with individual needs, in terms of things that I think About for legal reform, I'll lay out three buckets. The first of those is going to be intellectual property, and I'll come back to that in just a second. The other two I would think about would be the regulatory space and building community. So in the regulatory space, we have an FDA approval system. The FDA approval system is very much built around the idea that there are these large companies that develop drugs and that approval needs to go through them. How does that work with regard to an open source product like looping or like Oscar for sleep apnea, that's been a really difficult question. Who's liable for that sort of thing. So that's something that I think needs to be thought about more as patients become more engaged in the sort of process in terms of communities. It was an internet forum that created a lot of these sorts of things, right? I think Hillary we talked about. It was a Facebook group that brought a lot of these, these patients, together, that created these sorts of software products, maintaining those sorts of communities, building Commons of data so that people can do this sort of research. That seems like an important function as well, and some of that can be done privately, but the infrastructure that allows for that sort of thing to happen is pretty important. It is often a function of government. And, you know, I can point to any number of NIH databases of research that have allowed for that sort of activity to go forward on the intellectual property point Aaron. Aaron's no longer here, is he? So I can make fun of him? No, actually, I'm gonna say some really nice things about like what Aaron was talking about. Because Aaron was talking about the right to repair, and we've been talking about sort of this right to customize. I think those two things are very, very closely linked, because when a company develops a product, they're developing the product to maximize profits out of the largest population that they can find, and that might end up meaning cutting out some of the parts of the market that aren't necessarily as profitable. Now one solution to that is that we lobby the companies to be more inclusive. I think, Jessica, when you were talking about inclusivity, I think that that has been a very successful effort, particularly in terms of the disability community. There have been a lot of companies who have been very responsive to this idea that they need to serve larger communities. But a simpler way is, what if those communities could just change the devices to work for themselves. Why can't they? Well, the reason that they can't is because these devices end up being locked down for the same reasons that Aaron was talking about with regard to the right to repair, partly out of cost, because developing repairable or modifiable devices is more difficult, but partly out of these sorts of IP concerns, out of being able to make sure that they are able to extract as much profit out of the downstream revenues of the product as possible. And to the extent that we want to sort of swing the balance in favor of the sort of citizen science and turn in favor of these people who are doing these sorts of important work, making sure that we don't have devices that use encrypted data that nobody can read seems like a pretty important step. Now the counter argument to that, of course, is well, do you need intellectual property incentives to develop a lot of the sorts of devices that generate these sorts of data? Do you need intellectual property. Rights to incentivize the creation of the insulin pumps? And the answer is yes, and they already have them, and it's not clear why that extends to the data in the devices. Data is just numbers about a person. It's not clear why there's a copyright in those they're not really trade secrets. They're not commercially valuable to anybody other than the person who has them, they're not patentable. But because of the fact that we have some fairly robust intellectual property protections, particularly section 1201, of the DMCA that Aaron talked about, companies are often able to make it fairly difficult to get access to those sorts of things. And so a fairly simple reform, which is very much in sync with what Aaron was talking about in terms of the right to repair, to repair. Would be making that sort of thing more accessible, to enable this sort of customization on behalf of patients, to enable that sort of citizen science, so that people can really customize their treatments to work the best for them, and in doing so, advance the technology for themselves in the community. I think that that would be a pretty
Fran Visco 56:00
useful thing to do. Thanks so much. Charles, yeah, friends. You know, I want to say a couple things. One, the risk benefit discussion. It's very important that people understand and people are told risk benefit, but more important that they actually know what they're hearing. Right? How do you analyze risk? I have heard so many times if this just saves one person's life, that is, you know, we should spend a million dollars on it and and not really care if there is a lot of evidence that this is the thing that saved that one person's life. And I think we really need to, we need to have, we have to figure out how to better educate the public, public so that they understand exactly what it is they're reading, what they're hearing, what they're being told. I also think that the idea of, you know, very high level evidence is something that the public needs to understand. I know it is incredible, and it's wonderful when you have consumer scientists invent something, but you know, is that thing really something that is going to help a lot of people? What is the risk benefit balance there. What is the evidence behind it? You know, my organization is not that supportive of off label drugs and access to off label drugs. You know, are they doing more harm than good? Is saving five people? Is it killing 10 that we don't know about? Like, what is the evidence that got us to that point, how does public policy and law help the public get to the right place they need to be? And the other thing I just want to say is, because I hope we we might not have time to talk about it, is, what's the benefit of having consumers and advocates at the table? You know, I look at when I was diagnosed with breast cancer 38 years ago, there were certain treatments around there aren't that many different treatments around now. But in March of 19 of 2024 I was diagnosed with metastatic breast cancer, and there is no cure for that, and I am really fed up about seeing these metastatic patients running through the fields of lilies and how wonderful they're feeling, but I am responding well to a treatment that came out of the DoD program, not NIH, and the person who was funded through DOD, you know, couldn't get funded through NIH, and that is because you have patients at The table helping us focus on, what's the end game? What should we be really looking at? What's the innovation that we really care about that isn't like iPhone six to iPhone seven, but true innovation that's actually going to save lives. So those are just some points that I wanted to make after hearing the discussions.
Christopher Morten 59:00
Yeah, I thought in just a moment, I'm going to open the floor to Q and A from this larger group. This has been a really rich discussion. I want to create a little bit of space. I'll invite in just a second, Charles and Steve and Hilary to offer further reflections on the remarks you've heard, if you'd like. But I just thought, for me, I've been trying to keep up with some of the suggestions that the panelists came up with for reforms to law and policy, and just a few of the ones I jotted down, I think a theme I heard from both Fran and Hillary is this kind of fundamental premise that the patient should be at the center of innovation policy and healthcare, and should be like patient care. Patient health should be the goal, and that patients should be in the room at the table where decisions are made. Also really interested to see that play out kind of more concretely with patient participation and research on boards, as Fran described, really fascinating to hear about NBCC and other patient orgs. Developing sort of alternative research apparatuses, like the Artemis project and really fascinating friend, I hadn't known about the patent application that you all hold, and would love to talk more about that and the ways you're thinking of using it. Really interested. I think it was Hillary made the point that we have laws, for example, laws governing hospitals that can be paternalistic and deprive patients of autonomy and decision making, and that can undermine their health. Thinking about Steve's points about direct to consumer advertising and its harms and the need to get more and better information and information patients can actually use to patients about medical products, including drugs and devices. Thinking about Charles's points on IP law, making sure that IP doesn't thwart customization or prevent patients from accessing their own health data on their own devices, thinking about how FDA and regulatory law accommodate open source innovation and other sort of less traditional or less commercial innovation, and finally, thinking about how to foster patient communities and maybe even building infrastructures of data sharing and communication among patients. Those are just some of the ideas that I heard from this rich discussion. Okay with that, I would love to open the floor to any questions, and I see Jeannie, and I see Jessica and I see Rachelle move about order, and I keep your hands up if I admit if I've missed you and you have
Speaker 1 1:01:24
a question. Thank you so much for this inspiring and insightful panel. I'm wondering how you feel about the current federal political environment. On the one hand, I would think that, you know, there's some anti scientific evidence aspects right now, but on the other hand, there's a it seems like there's a real openness to listening to people on the ground and their experiences and that informing things. I'm wondering, how you think you know? Do you feel this is a favorable environment to push forward certain things or a less favorable environment than usual.
Fran Visco 1:02:04
Well, you know, as an organization that spends most of its time trying to generate evidence and high levels of evidence to save lives, I am not feeling very good about this administration at all. And I wanted to start also by saying my Presidential Citizens award came from Joe Biden. Okay, so it's, you know, I think it means something, but, but, you know, I think the, you know, the desire to do away with regulation, to make it easier, to move products forward, without really a lot of consideration as to what the evidence is that that is going to be beneficial. I think it's all very harmful. We're very involved in this safe science work that's going on now, and working with a number of different national organizations to bring sanity back to the discussion about the importance of science and what science actually is
Hilary Koch 1:03:06
I was just going to add my frustration has always been. I had a brief stint as the policy director for t1 international I don't know if we'd worked together at the time, and I had worked with a couple of congressional offices before there was any success with like an insulin Reduction Act or anything. And I was particularly frustrated, actually, with the insulin reduction act when everyone was so excited, because a reduction in price doesn't mean something's affordable, right? So while I'm incredibly disappointed with this administration, it doesn't matter. Diabetes doesn't distinguish whether you're a Democrat or Republican, right? And so this is, this is just one of those things where, again, I think patients have been in the oval office saying the same thing for a really, really, really long time. The technology is not what we need, and the price of medications and access to medications is still not what we need. And it really doesn't matter who's been in the office.
Fran Visco 1:04:15
No, I think it does matter, because right now you have a the head of the Department of Health and Human Services, is undermining so many of the medical advances that we've seen over the years that have actually saved lives. You know, any measles vaccines I agree with that matters a great deal.
Steve Woloshin 1:04:36
So I guess I would just say, I mean, I think the idea of questioning expert expertise, that's great. I mean, I think science is all about skepticism, right? But, but there are rules that you use that you need in order to move from a question to an answer, and that's, you know? So on the one hand, they're doing some things, right? They're advocating for. I'm sort of questioning and rethinking things but, but we really need evidence, and there are way there. We know, we know a lot about how to do science. They have to remember that, and I don't think they are. So I'm very, I'm very anxious about how things are. I mean, it could be the beginning of something good. That's the op ed that I wrote was about drug information. It's a nice start right now. You have to, you know, you have to keep going. So I'm, I don't know what's gonna happen, but I'm very anxious.
Charles Duan 1:05:33
I don't have anything interesting.
Jessica Silbey 1:05:37
I'm sure that's not true. I so my question is, focus on Fran and Hillary's work, but I do have a point to make for Steve. So I'm interested in your communities and how you resolve disputes within your communities. So as citizen scientists, or, you know, patient scientists, without with that are, you know, with a common problem, but not necessarily common metrics for solving the problem the way scientists might, for example, disciplinarily come up through a space and figure out how to test and do all these things. I'm just wondering there. I'm interested in how your community resolves disputes around process and accuracy and the best way forward, for example, and for Steve, I don't know if you've heard about the metal lab at Harvard, their data nutrition project. So they they're trying to create a graphic model for how to describe the data sets, what's in a data set to make sure the data set people, when they're using the data set, they can use it well. And they've come up with lots of different ways of visually imaging the ingredients of these very complex and I would just urge you to look at their website. It's called Data nutrition at meta lab at Harvard.
Fran Visco 1:07:00
So I mean, how we resolve disputes in our community? That is the National Breast Cancer Coalition. It took a long time to get to the point where we were able to do that. Well, when we started the coalition in 92 you know, having such a diverse group of individuals and perspectives at the table. There's a lot of mistrust in the beginning, and the very first thing we had to do was build trust among the players. And it took a few years for us to get to that point. And first we identified what are the issues that we can agree on now, you know, and what are the tactics that we can agree on now, we were fortunate in that when we brought people together in the beginning, we were all like women in our 40s, and we had been through the anti war movement, and we had been through the women's rights movement, And so we had developed a number of you know, tactics to get to a point of consensus, and we had to bring that to this new group of very diverse people. So it absolutely was not easy. There also still is a lot of, you know, we don't the entire breast cancer community is not on the same page. I would say the National Breast Cancer Coalition community is on the same page, but we're constantly combating people who want to make these issues pink and pretty and easy, and so it's an ongoing battle
Hilary Koch 1:08:40
for diabetes. There are two different communities, right? There's the one that's advocacy, that's trying to change laws. And within that community, there are, there are several sort of contingents, right? There's a really, there are a lot of discussions, because there are several organizations that you're probably familiar with, like JDRF Ada, maybe beyond type one. That's Nick Jonas organization, many of which are received funding from pharmaceutical companies, insulin manufacturers. And then there are other organizations, of which I've been involved with, beta cell Foundation, t1 international, that refuse to take money from pharmaceutical organizations. So those are, those are discussions where there's sometimes people can come together and agree with, maybe we are all kind of working towards the same goals, but we disagree with working with organizations that will accept money from insulin manufacturers. And then there's the looping community, totally different where it is just, I mean, answer your question, why do you do it? Because it makes life better. Why wouldn't you? Why wouldn't you help someone if you can? And and the answer is, when you disagree with someone, well, you do you your diabetes may vary. The great thing about this is you. You can download an algorithm, and I can customize it for my son, because I know how to Swift code, and you can do it for you, and we just work it out and talk about how to make it better for each other. I mean, it's just amazing. But to be clear again, my son is no longer looping, and I'm I'm talking about the community there. There are people who work together and they upload the code, but it's open source, so you can do it as you want to, and they don't make money off of it. So there's no kind of arguments like
Charles Duan 1:10:33
that. There is actually a really interesting story about that, with regard to the sleep apnea software. So I mentioned that this guy named Mark Watkins wrote the original program called sleepy head. He got into this very interesting argument with the rest of the community, which you can easily read about if you look online, because it's just in the forums, and the community just separated from Him. And so they made a fork of the software. The fork of the software is called Oscar, and they just built a new community that started developing the software separately. And that is something that you get to do when you're working in this sort of, in this sort of, like community based development. So if you're looking for an interesting story about how these sorts of communities resolve disputes that that's actually a really interesting one to look into. I don't know too much about it, but it would be a it would be a good study
Christopher Morten 1:11:26
in the interest of time. I think I'm going to ask Rachelle to ask her question, and then maybe wrap up.
Rochelle Dreyfuss 1:11:36
Doctors say that when somebody walks into the office and said, I did my own research that strikes ice in their hearts. There's so much misinformation now and purposely out there, and that's that's hurting us too. I mean, people aren't taking vaccines and all of that. So do you have anything on that course, those questions?
Fran Visco 1:12:00
I mean, that's why we have to educate the consumers, educate payments. I I represent an organization of organizations that, you know, a constituency. We're about system change. We're not about I walk in the office because I did my own internet research. We're about educating groups in communities about the importance of evidence and how to evaluate and understand information so that then they can bring the right information and abilities to their communities. That's the best we can do. You know, I don't really care if doctors have ice in their veins when I walk in and ask a question, I care about, how to get these individuals in communities understand how to evaluate evidence and information, so the decisions that they are making are actually having the right impact,
Hilary Koch 1:12:57
right And to be clear, what so this community is works often hand in hand with doctors. Nowadays, we're not making insulin, and again, we're not physically hacking pumps and changing anything in the pumps. There are scientific studies that have been published about this, and there are 1000s and 1000s of people around the world who do this now, any and does equal one, but when we started doing this, I went into the doctor's office and said, we're going to do this, and we'd like you to be partners and and our doctor was on board. Yeah, sometimes they sometimes they do it, and then they go into their doctor's office, and the doctor says, What are you doing? Tell me more. But that is just the way that diabetes works. It really is. Usually what you do is you look for patterns, you try one thing, you do one thing, and then when it doesn't work, you try something different. It's so hard to wrap your head around unless you do it yourself on a day to day basis. All it is, really, is an algorithm. And then you try to estimate carbs and how it's going to impact your body. You eat a slice of pizza, and you realize it sticks with you longer. Sometimes you figure that out yourself. If you don't have diabetes, you're like, wow, that that Chinese food just stuck with me a little longer. Or, you know, I went for a run, and I guess that banana didn't stick with me as long as I needed it to. It's the same kind of thing, only you put it in an algorithm so that your insulin doesn't last as long. That's really all it is. And so our doctor was just really, truly delighted that we did this.
Steve Woloshin 1:14:42
I guess I would just say to do your own research. Those are four potentially scary words, because especially now with AI, with chat GPT, it's really easy to ask questions natural language and get really reasonable sounding responses. But. They're often really crazy. So I mean, I couldn't agree more with Fran that. Think the answer is education, but the difficulty is, how you do it. Now, for full disclosure, I'm a huge fan of project lead. I think for me, we had more project leads in the world would be so much better. But the problem is, how you get these educational dimensions out to people? I mean, I've written books and done taught for years and years and years, but very narrow slice of the world, the question is how to do it, and it has to start early. Has to start elementary school. You can't wait to people are adults. So, you know, it's tough. And with AI. I think everything's being it's like misinformation on steroids. And so I'm not sure what the right thing to do, but I think that ultimately, the answer isn't regulation. The answer is education.
Charles Duan 1:15:55
Well, I'll just say something very, very quickly, a lot of this comes down to the way in which information flows through communities and how misinformation can flow through communities, but also how good information can flow through communities. So this is where I will actually plug something that I wrote, James groleman and I put together an edited volume on crowds and crowd dynamics, and we had a lot of great contributions from actually Rebecca and a couple of other folks, and we explore some of these questions, why do crowds spread bad information, but also, how are they able to correct that, and how are they able to spread good information? How do corrective mechanisms or moderation techniques within crowds work? And so I think that a lot of the answer to that question comes down to, how do we manage those sorts of crowd dynamics in what is a difficult world these days, with AI and with social media and everything
Christopher Morten 1:16:51
great, I think, I fear that's all the time we have. So please join me in thanking our speakers. Thank you
Announcer 1:17:07
all. The engelberg center live podcast is a production of the engelberg center on innovation Law and Policy at NYU Law, and is released under a Creative Commons Attribution, 4.0 International license. Our theme music is by Jessica batke, and is licensed under a Creative Commons Attribution, 4.0 International license. You.