Jonathan Warden 0:00 That. 2:28 Excellent. 2:28 All right. 2:30 Good morning, everyone. 2:32 Welcome to our next section in the media and civics track. 2:37 Very, very interesting talk coming up next with some, just a very interesting app that I've been following along. 2:42 So we have Jonathan Warden here to talk to us about his project, Blue Notes. 2:48 Take it away. Speaker B 2:49 Okay, thank you everyone for being here, and thanks everyone for helping to organize this event. 2:55 My name is Jonathan Warden. 2:57 I'm an independent researcher. 2:59 I am an engineer and co-founder of a startup small research organization called socialprotocols.org. 3:06 My focus is on the problem of how the design of platforms, social platforms and algorithms, has positive or negative social outcomes. 3:16 And my big thesis is that if platforms can inadvertently create all these negative social outcomes, couldn't they be intentionally designed to produce positive social outcomes? 3:28 So instead of misinformation and toxicity and division, couldn't they promote intelligent conversation and finding common ground? 3:36 I've been approaching this problem from the point of view of game theory, information theory, argumentation theory, collective intelligence, a whole lot of theory. 3:48 And the reason I'm working on Community Notes now is it's one of the only examples I know of where the theory has been put into practice successfully and at scale. 4:01 Community Notes on Twitter has actually helped to decrease the spread of misinformation on Twitter. 4:10 So I'm going to talk to you briefly about why I think Community Notes is great, both for fighting misinformation and as a unique and powerful scalable tool for pure moderation. 4:21 I'll give you a brief demonstration of Blue Notes, and then I'll talk mainly about the architecture— some of the architectural challenges with bringing community notes to @protocol and making it decentralized and composable. 4:35 Some of the challenges include making annotations work via labeler infrastructure, privacy and anonymity, and of course bots and manipulation. 4:47 300 years ago, Jonathan Swift said, "Falsehood flies and truth comes limping after it, so that when men come to be undeceived, it is too late." And he hadn't even heard of Twitter. 5:03 Rumors have always— misinformation or falsehood has always spread, and it spreads largely because people think that it's true. 5:12 Some people are spreading misinformation maliciously, but it spreads faster because people don't know the truth. 5:18 So what Community Notes does is it helps increase the velocity of truth in social networks. 5:25 It helps the truth sometimes catch up to people. 5:27 And so people, you know, where normally they just share something because they believe that it's true, it looks plausible, they kind of want to believe it's true, somebody just tells them, hey, this is actually not true, this is just a rumor, this image is fake, well then maybe they wouldn't share it. 5:45 Now besides being a great tool for combating misinformation, Community Notes also is a very unique and powerful peer moderation protocol. 5:56 It is decentralized in the sense that there are no privileged roles. 6:00 There are no administrators, there are no editors, there's just the community. 6:06 And so it is moderation without moderators. 6:09 And that means that instead of trusting fact-checkers or editors, instead of trusting people or in organizations, you place your trust in the community and in the protocol that that community uses to make decisions. 6:28 And there are some benefits in placing trust in the community. 6:31 One is that people trust the results of moderation decisions more that way. 6:36 People have a lot of trust in Community Notes across the political spectrum, and one of the reasons for that is that it's perceived as being fairly neutral. 6:47 Community Notes was developed before Elon Musk took over, and it's still— the algorithm is open source, all the data is open, and it's still running using that same algorithm. 6:58 And for the most part, people still trust community notes even though they don't trust necessarily the person that's running it. 7:04 And people perceive it as neutral because in a sense it really is neutral. 7:09 It is not just a naive majority voting-based crowdsourced fact-checking system. 7:15 It uses a very innovative bridging algorithm. 7:19 What this algorithm tries to do is not just trying to find what people think or what's most popular. 7:24 It tries to find common ground. 7:27 And so one way of looking at that is you can think of the algorithm as looking at people's votes and saying, okay, well, these votes— this person's upvoted this because they have a left-wing bias, and this person, all these upvotes because this person has a right-wing bias. 7:42 And maybe this person just upvotes everything or just downvotes everything, so they just have this, this upvote bias. 7:48 But these votes, there are some votes that cannot be explained away. 7:52 By bias. 7:53 And so this model discovers these latent factors that influence people's votes, and it says, look, this latent factor represents an area of common ground, and when a post or a community note has this factor, people are more likely to upvote that post or that community note across the political spectrum, across the spectrum of opinion. 8:17 So in a sense, it extracts signal from noise. 8:22 If you have, for example, a group where the majority of people voting on community notes is right-wing, which is actually what happened with what exists in X right now. 8:36 If a community note gets a lot of helpful ratings, what does that mean? 8:41 Does it mean that it's helpful or does it mean that it supports right-wing worldview? 8:46 Well, it could be either. 8:46 We don't know just by the fact that it's popular. 8:49 We don't know why people upvoted it. 8:52 But the Community Notes algorithm breaks that down into components, and it says maybe most of these upvotes are just because it supports a right-wing worldview, but there's also a bunch of upvotes that can't be explained away so easily. 9:04 So in sum, what it does is it's a protocol that scales peer moderation in a way that is robust to factual capture. 9:12 And I've written more about that on a blog post, jonathanwarden.com/understanding-community-notes, if you're interested in that. 9:19 So my goal is to bring @proto, to bring community notes to @proto and to make it decentralized. 9:27 I say pretty decentralized because I don't know what the word decentralized means anymore after talking to all of you. 9:34 But what I mean is that I want a system where the power of any one individual is limited so that no one can accumulate the power to censor the results. 9:47 A system that is transparent, so all the data is open. 9:51 Anyone can inspect, look at the data, and run the algorithm and verify that it's doing what it says it's supposed to be doing. 9:57 And then the @proto ethos of algorithmic freedom and composability. 10:02 The system I'm proposing allows you to mix and match your moderation apps, your algorithms, your labelers, and so on. 10:09 I've designed this to be a generalized peer moderation protocol, not just for community notes but for labels or any sort of moderation decision. 10:21 And I've designed this so that it can work with other social networks. 10:24 You can place labels or community notes on a Mastodon post. 10:27 For example. 10:30 So the Blue Sky team and a lot of people I've talked to here that have their own social apps have indicated that they want to integrate some sort of community notes-like feature. 10:45 But nobody knows exactly how they want that to work. 10:48 How exactly do annotations work in app proto? 10:50 How do you make this decentralized? 10:52 There's a lot of open questions, but I've gone ahead and just proposed some answers to those questions. 10:56 I have an informal proposal called Open Community Notes that I've published at that URL, this URL. 11:03 And then I've gone ahead and I've created an implementation that I call Blue Notes. 11:07 Blue Notes is based on a fork of the Blue Sky social app. 11:12 I did this as a fork just to demonstrate how community notes could be integrated into a social app. 11:18 I don't necessarily want people to use this except for as beta testers. 11:21 Because Community Notes is not useful unless there is some critical mass of users. 11:26 There needs to be enough people submitting and rating notes for the algorithm to actually produce any output at all. 11:35 And then of course people are not going to bother doing that unless the notes are actually going to be shown on posts, because that's the only way they do any good in fighting misinformation. 11:44 And so my goal, my plan, is to work with Blue Sky and other skies, other social apps, to refine this spec into something that they think is acceptable, that they could integrate into their social app. 12:00 And then have a launch where there's a little bit of publicity so that we can build this initial community of contributors and actually start producing some useful community notes. 12:10 However, in the meantime, anyone can integrate with this. 12:13 And it's fairly easy to integrate if you want your app to just show community notes when they start to show up. 12:20 So I'm going to do a quick demo if I can make it work. 12:24 Okay. 12:29 Okay, so— all right. 12:35 So fork of the Blue Sky app here, we have a section, a community notes section. 12:40 Where you can browse existing community notes. 12:43 So here I'll look at some community notes that have been rated helpful. 12:46 These have been rated helpful because I manually went in and rated them helpful. 12:49 Again, the algorithm doesn't produce any useful results yet. 12:54 So if a post has a helpful community note, you'll see that community note displayed underneath it. 13:01 This looks just like it looks on X. 13:04 You can then rate the note. 13:07 I've already rated this one, but I'm going to delete my rating and resubmit. 13:11 I'll say, yes, this is helpful. 13:13 I'll say why it's helpful. 13:16 Right now these checkboxes, this data is not actually used by the algorithm, or it is used a little bit, but it's not very important. 13:24 But I think it could be important in the future. 13:26 So I've rated a note. 13:28 Finally, I can write a note. 13:30 Actually, I'm gonna I'm going to pull up a post that's fairly recent. 13:36 I'm going to add a note to this post. 13:41 Here's a video of Kash Patel dancing. 13:43 I'm sure you all wanted to see that, but it's not actually him. 13:48 It was posted— it's misinformation. 13:52 So I've actually copied this community note from the community note that was posted on X on the same post. 14:00 You often see the same misinformation being repeated in both platforms. 14:06 Factual error. 14:07 This is not actually Kash Patel. 14:10 Community notes require at least one source. 14:12 You need to put in a URL. 14:13 It's actually just plain text, but the UI checks that you've actually got a URL. 14:20 You need to say, yes, I think this is a a source that people would find trustworthy. 14:25 And now I submit. 14:27 And now here's a note. 14:31 You'll see that this is a proposal, so it's just a proposed note. 14:34 And I'll talk a little bit more about the lexicon in a moment. 14:38 So that's it for the UI. 14:40 It's really quite a simple UI. 14:44 And what's, I think, a little bit more interesting about Community Notes is the protocol, is that the process for deciding how these notes or which notes are found helpful, which notes on which posts. 14:57 So first I'll talk about annotations. 15:01 Actually, I'm going to go back and show you one thing that I think will help you kind of see under the hood what's going on here. 15:08 If you look at the same post on the Bluesky app, what you'll see is it has a tag. 15:17 I've subscribed to the Blue Notes labeler. 15:19 Yeah, it's got a label. 15:21 I've subscribed to the Blue Notes labeler. 15:23 And when someone first submits a community note to Blue Notes, this label is emitted. 15:29 Now Blue Sky doesn't know how to show community notes. 15:31 It knows there's a label. 15:33 It doesn't know how to show a community note. 15:34 So what happens, Blue Notes social app, the only difference is that it recognizes this tag. 15:40 It recognizes the DID of the community notes labeler, and it says, oh, OK, this is a community note. 15:45 Let me go fetch the content of that community note. 15:47 And it makes an additional call. 15:49 You can actually see that in the network tab for the nerds here to get proposals. 15:56 Well, proposals. 16:03 Well, let's see. 16:04 That didn't work. 16:05 Oh, that's right, because it's not rated helpful. 16:07 But that's what you would see. 16:11 If a note has been submitted but has not been rated helpful, you'll just see this prompt to go rate the note. 16:20 OK, so that's how things work now. 16:23 So it's a bit of a hack to put annotations on posts using the Labeler infrastructure. 16:30 The way it should work— and I think this is kind of obvious— is that you should be able to add generalized labels. 16:38 So labels— @proto:labels are actually a type of annotation. 16:44 Annotation is just information that one person adds to somebody else's post. 16:48 And @proto:labels actually map really well to the W3C annotations data model. 16:54 An annotation in that data model is a target identified with a URI and then a body. 17:01 And that body can just be a semantic tag which is what an @proto label is, or it can be text, or it can be some other things. 17:10 And so it makes sense to generalize labels and to allow them to— the label infrastructure to support other types of annotations. 17:20 An obvious way to do that is to add a field. 17:23 A ref field is what I'm proposing here that points to the URI of an @proto record with the content of the annotation. 17:29 And then what seems to make sense is that AppViews, when they are hydrating posts— right now, as many of you know, the AppView will return a hydrated post that has all the labels for that post, or at least labels for the labelers that you've subscribed to. 17:46 Well, it would make sense if the hydrated post also included the actual content of the annotation. 17:52 This would look a lot like an embedded record, where if you Or if you quote tweet a post, you also get the content of the quoted post in the hydrated record. 18:02 And then of course, that means that the social apps need to know how to display these annotations. 18:07 And there might not be a lot of different types of annotations. 18:12 Some that I can think of are highlights, alt text, and then this context annotation is another type. 18:21 And so there's some interesting discussions to be had about how do social apps know how to display your annotation, especially if you're using custom lexicons. 18:28 There's some interesting discussions in the @proto-discourse about lexicon embed and embedding records from arbitrary custom lexicons in your posts. 18:39 But I won't say more about that now. 18:42 So the overall architecture— I actually based Open Community Notes on an existing project. 18:48 This year— last year at this conference, Drew McArthur, who was a student at UC Boulder, did a talk about Pimsky, pimsky.social. 18:58 PM is peer moderation. 19:01 And he created this system and this lexicon for peer moderation that was exactly what I was envisioning for Open Community Notes. 19:10 So I've actually adopted his lexicon. 19:11 He extended it a little bit for me. 19:15 So this lexicon has just two record types. 19:17 There's a proposal, and that's for a proposed moderation action. 19:22 It looks a lot like a label record, but it doesn't have to be necessarily a label. 19:29 It could be a proposed annotation or some other proposed action such as, I don't know, deleting a user or something like that. 19:36 And then there's a vote. 19:37 People can vote on these proposals. 19:39 They can agree or disagree, upvote or downvote. 19:42 And that's it for the lexicon. 19:45 So the overall architecture looks like this. 19:47 You have social apps that are able to display annotations. 19:51 They receive annotations as labelers. 19:53 So individuals can subscribe to annotators. 19:56 You can subscribe to the Blue Notes community notes annotator or some other. 20:00 Or it can be an app labeler or a built-in app annotator, default app annotator. 20:06 And then currently For community notes, you need to make this additional call to the notes API to fetch the content of the annotation. 20:14 So integrating this with your apps is really quite easy. 20:19 And then I'd like to talk to people that are interested in integrating with their apps on how I could make it easier, like for example creating, I don't know, a React component for these notes or something like that. 20:29 All right, so the other piece, moderation apps. 20:33 Their job is to just do those things that I showed you in the demo. 20:37 You can browse notes, you can submit community notes, and you can rate notes. 20:41 And of course, moderation ops can also be used to submit and rate proposed labels or other moderation actions. 20:50 There's a submission service, which is an important piece of the architecture. 20:54 All the proposal and vote records are submitted through the submission service, and it has the responsibility of anonymizing these records. 21:02 This is an important part of the architecture that I'll talk about in just a moment. 21:07 And then there are scoring services. 21:09 And you can— there's the community notes scoring service, but of course people can create their own algorithms for aggregating these votes and making decisions. 21:18 And when the scoring service— when a proposal reaches a certain threshold, when the score reaches a certain threshold, that proposal becomes a label or an annotation. 21:31 Now, privacy and anonymity or pseudonymity is a very important component of community notes. 21:41 Anonymity is a dangerous thing in social networks. 21:46 If people can create a bunch of fake accounts and post a lot of anonymous crap, you can really destroy a social an online community. 21:56 But in some instances anonymity can be a good thing. 21:59 And when it comes— anonymous voting, for example, is a good thing. 22:03 If everybody knows how you're going to vote, that may influence how you vote and may make it harder for you to vote your conscience. 22:08 And that's actually very important to the Community Notes algorithm. 22:13 It kind of depends on people saying, like, Ah, this time I kind of agree with those people, right? 22:21 Like, you know, I don't like it, but this really does look like it's misinformation, so I'm going to honestly say, yeah, I think this is a helpful community note. 22:30 And so anonymity is important, but the algorithm needs some concept of identity. 22:38 It needs to know these posts were made by this user. 22:42 And so the solution is— these anonymous IDs, or my proposed solution, these anonymous IDs. 22:49 So all these records are created by the submission service, the vote and proposal records. 22:53 They're created using that submission service's ID. 22:58 And then the records contain an additional field, an AID field or an anonymous ID field that corresponds to the user. 23:06 Now this field is generated as a hash of the user's actual DID, which the submission service knows, and a secret salt. 23:14 And that secret salt is critical because it makes it so that you cannot easily map— figure out somebody's actual user did based on their anonymous ID. 23:26 So another challenge is, of course, bots. 23:31 The Community Notes algorithm is naturally robust to bots. 23:35 It is not as easy as you might think to manipulate the algorithm just by creating a bunch of fake accounts and voting in one way. 23:42 The algorithm kind of filters that out. 23:45 But with enough bots, you can manipulate the algorithm. 23:49 And so for community notes to work, what you need is a contributor base that is mostly real people, mostly acting in good faith. 24:00 If you have too many bots or if you have too many people that really are not trying to rate things as helpful, it won't work. 24:07 But you can have a lot of bias, right? 24:09 You don't have to have a group of perfect people. 24:11 You just need to have mostly real people, mostly acting in good faith. 24:15 And so some possible solutions are proof of personhood, minimum account age requirements, for example, just general bot filtering tools. 24:25 There's a lot of solutions that people have been talking about in this conference. 24:30 Elements that we could apply. 24:32 Something called the rating impact score, which is what Twitter does, which requires you to have a certain reputation based on your ratings before you can actually write notes. 24:42 Okay, so finally, one of the reasons I'm excited about community notes is that it is so— there's so much unrealized potential. 24:50 Twitter underutilizes that so much because it is— I was inspired by Edith's opening remarks yesterday. 24:59 One of the things that she said is that social platforms should treat people as subjects with agency and not as objects to be manipulated. 25:10 And I think one thing that people with agency want is to use these platforms to help discover truth. 25:18 They want to share truth. 25:20 Right now, so many social algorithms will show you what you want to see, show you what they think you will believe. 25:26 But that's not what I want a social platform. 25:28 I don't want a social platform to show me what it thinks I will believe, especially if the platform knows or if there's a way to know that I wouldn't believe this if I knew the truth, if I had more information. 25:41 And so I think there are so many ways, if you think about how can we design, that we can use community notes as a base for a social platform that has a lot of affordances for helping people share and find accurate information. 25:57 There's a product called Roundabout that New Public has created, and they have, for example, a feature where when you submit a post, before you submit it, it says, "Hey, do you really want to submit this? 26:10 This violates some of our community norms." It's just a prompt. 26:14 Well, the same thing could happen for community notes. 26:16 You could write something and the system could say, "Hey, look, someone else has posted a similar claim and there is a community note saying that this might not be true. 26:24 Are you sure you want to post this? 26:27 I think the algorithm could be faster. 26:28 It could cause— notes could achieve a helpful or unhelpful status much, much faster, which could help increase the velocity of truth and help misinformation not spread as fast. 26:42 It could be more integrated with the ranking algorithms. 26:44 If the algorithm knows that this this post is likely misinformation or that this poster tends to spread a lot of misinformation, could downrank that content. 26:53 A lot of possibility with correction and retraction. 26:55 If you've shared misinformation, the system might prompt you, hey, look, you shared this, but now there's this community note to help undo some of the damage that happens when misinformation is already spread. 27:06 And then a lot of other possibilities when, for example, a community note may itself be inaccurate. 27:11 And what is the process for putting a note on a note, which I think opens up some really interesting possibilities for almost a deliberative argumentation type process for making these decisions. 27:23 OK, I took more time than I thought, and we've only got 4 minutes and 30 seconds left for questions. 27:28 So any questions, anyone? Jonathan Warden 27:32 Thank you very much for— Really super detailed and very thoughtful. 27:40 I just want to recognize the amount of thought that you put into this. 27:43 All right. 27:44 Any questions from the crowd here? 27:45 Oh, we got— Thank you. 27:48 Yeah, super interesting. 27:49 I'm curious if with the anonymity you could also integrate reputation because it seems like that also becomes important over time. Speaker B 27:57 Yes. 27:58 So the algorithm itself has a reputation score kind of built into it. 28:04 The way it works is every user is assigned two parameters. 28:10 These are two factors that kind of represent that person's bias. 28:16 And then this rating impact score is also a sort of reputation system. 28:21 And there's some other things, but in summary, yes, I think reputation could really improve the quality of the system, right? 28:28 Especially if you know like this person, when they rate a note as helpful, it always ends up being rated as helpful by the community. 28:35 And so you could, you could say, look, provisionally, we're just gonna go ahead and display this community note as a helpful community note just based on, you know, one or two people's ratings that have a, you know, high amount of trust. Jonathan Warden 28:48 Thanks, really cool presentation and great work. 28:52 It strikes me that some of the stuff that is worth building here that you've alluded to, like the anonymization service or reputation tracking algorithms, or maybe like AI for deduplicating proposals, this kind of thing, is a centralizing vector, which is susceptible to potential corruption, right? 29:17 You could have censorship of proposals during the anonymization phase or this kind of thing. 29:24 Do you have any thoughts on how to build resistance to that kind of potential corruption? Speaker B 29:30 Yes. 29:31 I actually had a slide about that issue, and I skipped because I thought nobody would notice. 29:38 And so the weakest— you've zoomed in right on the weakest piece of the system, the piece that requires the most trust. 29:46 It's the submission service. 29:48 So the submission service, you have to trust first of all, not to de-anonymize you. 29:52 And you have to trust that it's going to submit what you say that it's going to submit. 29:56 Now, some of that is verifiable. 29:58 You can go and submit something. 30:00 And then if you don't see it show up in the data, you can say, hey, look, these people are not doing what they say they're going to be doing. 30:05 But that is the weakest point of the system. 30:07 And it does require trust. 30:08 So for example, if anyone were to implement this in your own social app, I would say, run your own anonymizer service. 30:18 And that allows you also to control who can contribute and help control for bots and just, you know, general good faith contribution in that way. 30:27 In the long run, I think that weakness could be possibly addressed with cryptographic solutions, with zero-knowledge proofs. 30:36 But I don't think it's necessary, at least not for now. 30:39 I think it's important to recognize that weakness even though I was kind of trying to gloss over it. 30:44 But it's important to recognize that and deal with it maybe in the future. Jonathan Warden 30:49 Excellent. 30:49 All right. 30:49 Well, thank you once again, Jonathan, for an excellent presentation. 30:56 We're going to switch over here. 30:59 Evelyn, come on down. 31:03 That was really great. Speaker B 31:04 OK. 31:04 Thank you. 31:04 That was excellent. 31:05 Yeah.