MR. KOJO NNAMDIFrom WAMU 88.5 at American University in Washington welcome to "The Kojo Nnamdi Show," connecting your neighborhood with the world. Crowdsourcing has come a long way. When Wired writer, Jeff Howe coined the term in 2006, he envisioned a new pool of cheap labor, a future where amateurs would replace skilled professionals, companies would pay pennies for services that used to cost thousands and people would use their spare time to help solve complex problems beyond the scope of a single mind.

MR. KOJO NNAMDIFast forward to 2013 and some of that vision has already materialized, but new technologies are also creating unexpected opportunities to harness the power of crowds. Activists are designing platforms to document and respond to emergencies, collecting and archiving grainy YouTube videos of war crimes inside Syria or using text messages from storm-ravaged communities to figure out where to send humanitarian aid.

MR. KOJO NNAMDIWe're exploring new frontiers in crowdsourcing and the challenge of making sense of a world awash in user-generated data and joining us in studio to do that is Dan Russell. He's a senior research scientist at Google. He is known as Google's director of user happiness. Dan Russell, good to see you again.

MR. DAN RUSSELLGood to see you, Kojo.

NNAMDIAnd thank you for joining us in studio. Also with us in studio is Jennifer Chan. She is a director of Global Emergency Medicine at Northwestern University's Feinberg School of Medicine. She's also associate faculty and member of the Crisis Mapping and Early Warning Program at the Harvard Humanitarian Initiative. Jennifer Chan, thank you for joining us.

MS. JENNIFER CHANIt's great to be here.

NNAMDIAnd Sam Gregory is here. He is program director at WITNESS.org where he also leads the Cameras Everywhere Initiative. He's also a lecturer at the Harvard Kennedy School. Sam Gregory, thank you for joining us.

MR. SAM GREGORYThank you.

NNAMDIThey were all here for the Future of Information Alliance Conference that was held in this area. Dan Russell is futurist in residence with the Future of Information Alliance. Jennifer and Sam are visiting futurists.

NNAMDIIf you'd like to join this conversation about crowdsourcing, you can call us at 800-433-8850. How accurate do you feel is the wisdom of crowds? Do you trust information from a Wikipedia or YouTube? 800-433-8850. Dan Russell I'll start with you. What exactly is crowdsourcing? The term is thrown around a lot lately.

RUSSELLIt has been kicked around a lot, but it's really kind of an old idea. I mean, think about classical Athens. People would vote. You get the wisdom of the crowds through voting or you could more recently elect a president.

RUSSELLBut the more modern sense of crowdsourcing is really in the day of digital technology where people can contribute little bits of work. You mentioned, for example, labeling things. You can imagine people trying to fold and refold protein molecules to discover new configurations. That's deep science being done by people at home.

RUSSELLIt looks like a game, but it's actually valuable work. Or crowdsourcing can be thought of as 2,000 people singing together in a virtual choir. So Google, in a sense, is crowdsourced because we drive all that information and it makes Google a good search engine by looking at the way people build content on the web and looking at the links and the references and so on.

RUSSELLOr something like YouTube where millions of people upload content, tag it, label it, provide it to everybody for free. So it's kind of this combination of all of us working together to make something greater.

NNAMDISam Gregory, on sites like YouTube, you can find thousands of images and videos documenting, say, horrors in Syria. What does a human rights organization like WITNESS do with these kinds of materials and how is that crowdsourcing?

GREGORYWe work at both ends of the spectrum of how that video gets created and used so as an organization, we try and think about how you build a greater pool of credible, authentic video, which is about how you get the right skills and the right tools to people to do that safely and effectively. And I think that's the first point in a crowdsourcing arc in human rights, is how do more people document their first person witnessed experience in a powerful way.

GREGORYAnd at the other end of the spectrum, we try and think how do you get this information sorted, verified and presented to people so it can be actionable information? So, for example, we run a human rights channel on YouTube where we work with a group called Story Filter, verify and contextualize that information and try and push it as actionable information to decision makers. So I think there's two ends of the spectrum of crowdsourcing in human rights, more information, more authenticable information then turning that into action.

NNAMDIBecause we have grown used to the concept of citizen journalists and you seem to be advocating citizen witnessing.

GREGORYExactly, we think that there's an important skill that almost everyone at this point should have given, that we have increasing devices in our pockets that allow us to capture information, to tell a narrative and then to share it that it's often in the unexpected moments that you become someone who witnesses an event.

GREGORYIt could be a war crime, but it could also be an act of police brutality. Or it could be something that is wrong in your community. So it's an essential, almost a literacy of the citizen is to know how to capture that and share it and add it to this value chain that can then be sorted through.

NNAMDIJennifer Chan, when Hurricane Sandy hit, volunteers turned to crowdsourcing as a response to that disaster. You've been involved with multiple crowdsourcing projects that aid disaster relief. Tell us a little bit about what some of these projects look like.

CHANThe projects look different across different types of disasters and crises, but one thing that is common between them is some of the digital volunteers or digital humanitarians that get involved in these projects. And it’s the ability for communities, whether they're part of the Haitian Diaspora or part of people just in the United States or beyond, that that can get online and process small bits of information, but collectively create a large amount of informational knowledge.

CHANAnd we've seen it happen in Haiti with the Mission 4636 initiative, as well as some of the activities of OpenStreetMap and Future in the Forward, humanitarian OpenStreetMap and also the MapMill project, which was put together -- a public crowdsourcing project with many partners and that happened soon after Superstorm Sandy.

NNAMDITalk about incidents or earthquakes, disasters like Haiti for a minute because you mentioned volunteers used information to help there. They used tweets. They used texts. They used websites to create maps, the mapping of that disaster. As someone who has worked on the ground in those disaster relief missions, can you explain how that helps responders to help deliver aid?

CHANI'll start with a personal story. I was actually one of the first responders in Haiti a couple of weeks after the event. I was deputy of a field hospital, one of the largest rehabilitation hospitals at that time and as a responder, I soon came to realize that there was a real-time map that was being created by volunteers from the OpenStreetMap community.

CHANAnd I was lucky, I actually had electricity. We did have internet access. I happened to be part of the crisis mapping community and had sort of seen the sparks and heard maybe the wind of this going on. Once I realized it was there, the map was quite good that I could actually begin to think about ways to transfer patients from our hospital to others and use that map not only for myself for determining patient transport, but also to use that actual paper map from an online to an offline process to share those maps with the drivers that we used.

CHANBut at the same time, I think that there is a bigger, broader challenge that we face with regard to how do we use crowdsourcing projects in these very complicated dynamic settings. Sometimes they last a week or two. In acute disasters, sometimes they last ten years in a protracted crisis.

CHANBut how do we take that to people like myself or my colleagues all around the world? And how do we really get that to go from decision to action? And that's where I think the research part of me wants to explore with other people about really where we are in this field.

NNAMDIBack to Haiti for a second. How were Haitians involved in creating and using those crisis maps?

CHANThe one example I like to talk about is the Mission 4636 initiative and a lot of the crowdsourcing activities were with the Haitian Diaspora as well as Haitian nationals. And I speak to this from a report that comes from Rodman Rowe (sp?) that was published in 2012 in the Journal of Information Retrieval and actually the crowd there was really people who were tightly connected according to this report, to what was going on in the aftermath of Haiti.

CHANAnd actually in that report 95 percent of the workforce was actually Haitian nationals or the Haitian Diaspora.

NNAMDI800-433-8850 is the number to call if you'd like to join the conversation. We're talking about crowdsourcing with Jennifer Chan. She's the director of Global Emergency Medicine at Northwestern University's Feinberg School of Medicine. Sam Gregory is program director at WITNESS.org where he also leads the Cameras Everywhere Initiative and Dan Russell is a senior research scientist at Google. He's known as Google's director of user happiness.

NNAMDICrowdsourcing by nature seems to put quantity of information over quality Dan. How can you sift through the mass of information so that it is actually useful?

RUSSELLIt's an interesting question because you're right, the nature of crowdsourcing is that you get a lot of data, a lot of people contributing little bits and one of the key ideas about this is that people will vote effectively with their answers. So you might get 200 people commenting on a particular interpretation, a photo or piece of data that's out there, but if 199 of them agree on an interpretation, then you can safely ignore the one that's out there.

RUSSELLOne of the things that's great about this is then people can build on top of each other so it's not just simple voting, but you can get an effect called accelerated crowd innovation where people are not just doing interpretation, but then are building larger stories about what's going on and I think that is what's really interesting. It's not just the simple mass or wisdom of the crowds, but it's the smart interpretation by all those individuals, smart people doing it.

NNAMDISame question to you, Sam Gregory.

GREGORYYou know, I think when you're dealing with human rights, a lot of time we place this emphasis on the NGOs, on the professional organizations coming there to document and it's a very strange model because it relies on people turning up often weeks after an event to try and uncover what happened.

GREGORYAnd so there is a very definite emphasis on quantity in what I'm describing, but it actually improves the quality of human rights documentation because you want the person who is closest to it, who has the best sense of the context in which they work, who has the best sense of the other sources around and who is capturing it in real time to be a part of that volume of material.

GREGORYNow, I agree with Dan. I think there's then a process that can happen within the crowd and can also happen in relation to experts of then narrowing that down and thinking how that then gets translated into action. And I think that translation into action is a key challenge and quantity sometimes overwhelms us so we turn away because we're overwhelmed by that quantity of first-person experience.

GREGORYBut in principle, the first-person experience is far closer to what we need to make our case, to secure justice and so there's a process question we're dealing with, but that quantity is not what should worry us as the underlying question, it's what we have to deal with to translate that into action.

NNAMDIProcess question and maybe in a way a philosophical question also, Jennifer Chan. And that is how do you extract quality from quantity?

CHANI think that's an ongoing challenge and I do think it changes by project and by context and I think Sam speaks very nicely to some of the challenges and some of the issues there. Extracting it, I think, can be partly an automated process as a way and a step-wise pattern toward getting to quality because sometimes there is such a mass of information explosion coming from different sources that one must start somewhere to filter out.

CHANBut I think as a project begins to understand what its goals are and if it is connected to communities and needs and there's a heterogeneous group of needs out there, then the processing for too much quality becomes quite complicated and should be, I think, in my mind, a mix of more automated processing, expert analysis, a broader view of what we deem experts to be. And a lot of human perceptions that go into understanding what that quality is. And that really, in my mind, guides itself toward the decision to action importance and purpose in a lot of these projects.

NNAMDIIf you have questions or comments you can call us at 800-433-8850. How have you or your company used crowdsourcing? Have you contributed to any crowdsourcing projects like, oh, Wikipedia? Here is Tony in Bethesda, Md. Tony, you're on the air. Go ahead, please.

TONYYeah, a couple of points. I was just looking at the new map of North Korea which Google maps put out which had a lot of I guess crowdsource information. So if we could maybe talk about the accuracy and the motivations of the people who contributed to something like that. And then a second question, going back to motivation, would also be how do we deal with businesses who might add a lot of content to sort of things like Google maps and YouTube and so on. And again, I go back to motivation where you might skew crowdsourcing a little bit. Thank you.

NNAMDIThank you very much for your call. As our caller pointed out, Dan Russell, last week Google uploaded a more detailed map of North Korea to Google Maps. That information coming from so called citizen cartographers. I guess our caller wants to know, one question is how did Google go about vetting that information?

RUSSELLWell, the North Korean map, I don't know how accurate it is. I do know that we've done some reconciliation between contributions that people made through Google Map Maker and the actual physical imagery. So if there's a line and it looks like there's a road there then you probably think that's probably correct. Also if you get a lot of people agreeing, it's that sort of going back to the mass opinion again. If you get 500 people agreeing that that's a road or that's a city or that's a restaurant then it's probably correct.

RUSSELLNow the caller asked about motivations. It's a great question because people have, I think, especially in a large crowd many different motivations. There's not a single solitary motivation. And one of the surprising things is we found that people often do little jobs like this, like labeling a road or drawing a line or labeling a particular building in their spare time. So they do this in sort of the cognitive surplus moments.

RUSSELLSo often it's just something they do for fun but I think in the case, for example, of the Haitian Diaspora, it's people who genuinely want to contribute knowledge in the world. And the folks, for example, in the United States who are from Haiti could label those places correctly and accurately. I think the same thing goes on with North Korea. People have some experience there can contribute and their motivation then is to make the whole information space richer, better, more accurate.

NNAMDITony, why did you ask the question about motivation?

TONYYeah, the second question was about businesses who might be sort of loading a lot of content into something like Google Maps and, you know, skewing the sort of results. I mean, we're talking about, you know, sort of citizen participants who are doing altruistic things. But we may have people whose motivations may not quite be so pure.

NNAMDIDan, care to respond?

RUSSELLSure. People with business interests obviously want to put the locations of their businesses or their interests onto the map. So we see a lot of people, for example, annotating national parks in the United States or annotating oil regions in other parts of the world. So in those motivations it's clear they want to track some attention to that kind of thing. I don't know that that skews the results because the map is, for example, the collection of roads, the collection of things that are there and there's not a particular business motivation for that. You can't just make up a road, right? The crowd will not support that. So you can't invent things out of whole cloth.

RUSSELLBut layers on top of that, like business locations and so on, then that just annotates. It makes the whole collection of business results that much better.

NNAMDIThank you very much for your call. Dan, the paradox, if you will, of Google's map of North Korea is that most North Koreans won't ever see it because they don't have internet access. Is the power of information limited to those who can access it?

RUSSELLWell, I think that's sort of the story of this whole panel. It's not, in many cases, information solely for those that live in that spot. So, for example, in Syria, I don't know how many people in Syria actually have high enough bandwidth YouTube connections to watch all the video that's being uploaded, similarly for North Korea. And there's a lot of places around the world where that's true. But that influence is policy. That influence is decisions that are made elsewhere in the world. So I think it's a net good. It's not just for the people who live there.

NNAMDINevertheless, Jennifer Chan, how can you reach those who rarely have access to the internet?

CHANI would say that technology is a modality for communication. And that's why we see so much of explosion in a lot of things that we can visualize on the computer. And it's amazing. But for communities that may not always have electricity or internet access, it's still possible to take online information offline. And it's a really remarkable process that I think requires a lot more exploration. Not only among academicians or even international practitioners, but it's time spent with communities in these regions for which they can describe how and in what way and when and when safely to take online information that they may have with a partner who has access into the offline space.

CHANSome of the most powerful information exchanges I've seen has actually been voice to voice but coming from reports that were transferred from the capital via internet.

NNAMDIGot to take a short break. When we come back, we're going to continue this conversation on Crowdsourcing and continue taking your calls at 800-433-8850. How accurate do you feel is the wisdom of crowds? Do you trust information from crowdsources like Wikipedia or YouTube? What events prompted you to pull out your Smartphone and snap a picture? 800-433-8850 or send email to kojo@wamu.org. I'm Kojo Nnamdi.

NNAMDIWelcome back to our conversation on Crowdsourcing. We're talking with Sam Gregory. He is program director at WITNESS.org where he also leads the Camera's Everywhere Initiative. Jennifer Chan is director of Global Emergency Medicine at Northwestern University's Feinberg School of Medicine. And Dan Russell. Dan Russell is senior research scientist at Google. he's known as Google's director of User Happiness.

NNAMDII should tell you that WAMU has recently been dipping its toe into crowdsourcing, or jumping into the water really. For the last year we've used a tool called the Public Insight Network which allows WAMU listeners and members to volunteer to be a source about any topic or issue area that they have expertise or knowledge in. Over the past year WAMU News has generated roughly 40 new stories from topics suggested or directly influenced by sources through PIN. We've also used this tool on this show. You can get more info on the Public Insight Network at WAMU.org/pin.

NNAMDISam Gregory, how willing are people in other parts of the world to participate in crowdsourcing?

GREGORYI think there's a -- it's hard to generalize so I want to be careful. I think when people are in situations of crisis there is often a desire to contribute information that other people will act on. Now I think where the problems arise around this, and this is something that we rest with in both the human rights and the humanitarian sphere, is people don't necessarily know how that contribution they make when they place it up on YouTube or they Tweet it out or they share it on Facebook or others share it out online, how it will be used and what will be the risks to them.

GREGORYAnd I think this is a big question because you're a data point within a larger project often where you're contributing your testimony or the evidence of a violation. And so I think people are often very willing -- they're often not clear about the risks themselves or how to think about issues of anonymity. And they're also not often clear on what are the things that will make it useful to other people. And so we come up against questions of expectations of individuals who are adding that act of citizen witnessing into that stream of information that they hope will be acted upon by say the international criminal court or a human rights group.

GREGORYAnd yet this piece of media will not be useful in that kind of documentational evidence setting. And that's why we put a lot of emphasis on how do you help those people who are trying to become part of this great pool of citizen witnesses to do it in a way that is safer and more likely to be useable in that value chain of turning it into action.

NNAMDIThis question I guess for all of you. Do you think all global aid efforts should not have a digital component? Starting with you, Sam?

GREGORYI think you should consider it. I think I'd go back to Jennifer's point is think about what are the right modalities of communication for the situation you're in. And that might be what is relevant for the community. And so for example, radio may be a much better way of engaging in a particular situation for a local community than a digital-driven approach.

GREGORYAt the same time, if you're trying to communicate to a solidarity audience or engage that boomerang effect where you kind of throw the boomerang from Syria to try and hit Joe Biden on the head and get him to pay attention to Syria, then a digital element may be really important in order to mobilize people in the U.S. and the global North. So I think it is about choices in situations, though increasingly it is I would say almost essential if you're thinking beyond the very local and very non-digital societies.

NNAMDIJennifer, then Dan.

CHANI would say that I think most organizations should definitely consider it. But I look at -- from an organizational perspective and another space perspective of communities, projects and organizations as a whole. And organizations could be a community-based organization local. It could be a very large international organization. But as I would -- the view that I take is as one or an organization decides to look into what the options are it's very important to think about the capacity and resources that exist both with communities and organizations internally.

CHANAlso to think about what that information may mean from a security and privacy issue. So that as projects roll out or plans and strategies get put in place, that those ideas and those issues and those contingency plans are worked out or explored from the very beginning. And then those two together really speak toward an important issue with regard to funding and whether or not the ideas, as they go into an organization plan, whether or not their resources and funding capacities to make those projects move forward in a meaningful way.

NNAMDIAnd Dan Russell, Google has launched several digital projects aimed at social impacts from providing online crisis relief for the flooding in Jakarta to tracking the global spread of the flu. So I guess, do you think all global aid efforts should now have a digital component?

RUSSELLWell, absolutely. And I think the reason is something that's a little bit overlooked in the internet age is that the internet really provides a different way of publishing. It used to be that if you wanted to build a radio station like WAMU you had to actually invest in a lot of concrete, a lot of wires, a lot of power, a lot of transmitters and so on. Now if you want to create a podcast and reach a million people, you can do that in your bedroom on your computer in your spare time.

RUSSELLYou want to publish pictures of what's going wrong in Ethiopia, you want to see what's going on through say the medium of a video, that's easy to do. And so we've reduced the cost of capturing the experience. So for example, I know that some local community activists will take a video and put it on YouTube of the potholes in their street. If that works in my town I bet it works internationally as well. So yeah, I completely agree that this is the thing. And it's primarily because of the ease of getting this information out.

NNAMDICrowdsourcing is what we're talking about. Give us a call, 800-433-8850. What information sources do you trust, 800-433-8850? Or you can send email to kojo@wamu.org. Send us a Tweet at kojoshow. You know, people in one culture might be more than likely to pull out their Smartphone to document an event than people in another. How do media habits vary from country to country and how does that affect the results of crowdsourcing, Sam?

GREGORYI think there are differing expectations. And again, there's just a huge variance of witnesses worked in pretty much every continent except Antarctica, though I suspect there's some opportunities there eventually in the next few years. You know, I think that there are different expectations around how we tell stories. I think often those relate to the idea of long form storytelling. I think when you come back to it the basics of how we explain what is happening in a context, who is doing something, what is happening, their motivation, which are often the basics of witnessing an event, I think are broadly shared.

GREGORYI think once it comes to telling a longer narrative that there are very different expectations about how to tell narrative. I think there are also differing understandings and expectations around consent and privacy. A lot of those are driven by law. So is it legal to film in public spaces? Is it legal to film public figures? And of course, whether there are risks associated with it. And then there are also issues around graphic imagery, and that's sort of -- I think that's also driven by predominant media culture.

GREGORYSo suddenly people who are less familiar, for example, with the coverage of say Al Jazeera Arabic are often confused when they see how much graphic footage is seen on the television, if you watch Al Jazeera Arabic versus say watching the BBC or another Western news channel. And so I think that set some of the expectations around graphic footage. So I think there are differentials but I actually think that basic act of witnessing at the very fundamental level goes to some simple questions that are very universally shed.

NNAMDIAnd Jennifer Chan, I guess this is also a question about how accessibility affects culture. Because you can also look at cultures, I guess, and see different attitudes toward this within cultures, depending on whether people are urban or whether they're rural.

CHANYeah, I think there's definitely a very diverse landscape of how people look at the world around them, how they communicate it. There's actually even different dialects within one country or even actually within a region that's important to people. But it actually crosses national lines. And so how to manage that with crowdsourcing I think -- and if thinking about it during disasters or humanitarian crises, I like to think of it across actually the disaster cycle.

CHANIn times where people can talk about their perceptions, learn about others. People can figure out what may be the appropriate crowdsourcing approach, or maybe even realize that the storytelling may not even require crowdsourcing at all. And that can be done before a disaster or in a pre-disaster phase. So that if it does apply when there's a large volume of information that people have had a shared space in a potentially shared area of understanding. So that when a community or an event happens there's a level of necessary and probably valuable preparedness that can occur.

NNAMDIOn to the telephones again to Rebecca in Leesburg, Va. Rebecca, your turn. Go ahead, please.

REBECCAHi, Kojo. I'm happy to be on your show. I love it very much.

NNAMDIThank you.

REBECCAI have a question about the sort of human psychology base. How do we avoid the pseudo fact that happens when people will hear something and repeat it and hear something and repeat it? And before we know it, it's accepted fact and yet it may not be based on something. And I worry about that in a crowd situation, particularly in a crisis situation.

NNAMDIWell, I'll start with you, Dan Russell.

RUSSELLWell, the echo chamber's been with us forever. I mean, this is...

NNAMDIYeah, we've all had that speech class, yes.

RUSSELLYeah, exactly, right. And so this is not really a new phenomenon. It might be somewhat amplified by technology. So when I teach my classes about how to be a better more effective searcher, this is a key point I make. Look, you've got to check -- you've got to check this stuff. You can't just accept something because you heard it a hundred times. And, in fact, a great heuristic for this is often on the internet you'll find somebody saying some claim that sounds a little funny. And if you see exactly that same text or exactly that same picture or exactly that same audio clip a dozen times, start to worry. Somebody's starting to copy.

RUSSELLThey're not adding their own value. They're not adding their interpretation. So that's one trick to use when you're trying to see whether or not this is just an echo of something made long ago.

NNAMDISam Gregory, news organizations have struggled to verify the information and multimedia that pours in through YouTube and Twitter. For example, with the conflict in Syria, fake photos and video of the Syrian conflict have circulated around the web. How can journalists or researchers be sure that information from nontraditional sources is accurate?

GREGORYI don't think they can ever be sure. I think there's a key role for journalists and researchers for using expertise to analyze photos and videos. I think there are some steps we can take that move us toward making it easier. One part of that is looking at how you combine traditional journalistic skills of source verification, looking at multiple sources with the types of analysis you can do in terms of how media circulates.

GREGORYSo are people who are trustworthy sharing it on Facebook and Twitter? Are they the same people who shared trustworthy information before? Are they people who have a history on Twitter and in social media? Can you then map the imagery to what's on Google Maps or what is seen in other footage or combined multiple perspectives on a single incident, as one of our partners at U.C. Berkley are doing with a project called Rashomon.

GREGORYSo I think there's aspect of it, and there's also, you know, helping build a stronger pool of citizen witnesses who have the skills to contribute to this. Now, that means you have the journalist in a better place with that tool set and with that stronger pool to say, wait a second, this one looks like it's falsified, and I will pay my attention to try to put it through forensic analysis for example of whether the Photoshop -- whether it's been Photoshopped and there's an increasing number of tools to do that as well. So I think there are process tools, and there are also technologies which we need to apply to this.

NNAMDIWell, Jennifer Chan, can unverified information or images jeopardize the goals of human rights groups, or for that matter, journalists trying to report on conflict-ridden areas?

CHANI think that those types of images and information make the landscape more complex. Whether in the end it jeopardizes, I think is a matter of context in specific situation, and that's why I like to talk a lot about ways in which you can enable organizations or journalists or communities to be able to wrap themselves around different pieces of information and understand what it means to them. Because the ability to speak out and say I think it actually looks like this, or we are a trusted organization and we feel that this image is not reflective. Who else thinks this?

CHANOr if you're online on Twitter and you say, I think this is actually a rumor, but actually I know somebody who is working on the ground in the field or in the community, and you have that connection, then you come back and you say, actually. It's about repositioning and creating a tighter and a more comprehensive understanding around that piece of information that I think will mitigate some of the risks.

NNAMDIRebecca, thank you very much for your call. We move onto Carl in Washington. Carl, you're on the air. Go ahead, please.

CARLHi. I use Google Maps, and I certainly appreciate the ability to use them. However, in the District I -- D.C., I ran into a little problem, and that is the District boundary was Maryland and Virginia -- was Maryland rather, in all straight lines. However, in Google Maps, they dogleg left and right, they disappear and reappear, and they're just sometimes in the wrong place. For example, Eastern Avenue, the boundary is above it, the District owns the entire avenue, and then some portion of it, yet, according to Google Maps, it doesn't, it does, it goes this way and that.

CARLIt's just not useful. And when I submitted a comment, the response was, well, this goes beyond what we can do here. We have to talk to the developers and that was several months ago, and I haven't seen any change.

NNAMDIMy own driving experience suggests that Eastern Avenue does go this way and that. It is not, in fact, one straight line. But I don't know, Dan Russell, this is your question.

RUSSELLWell, I'm not the maps expert on the geopolitics of D.C. versus Maryland, and I certainly wasn't there when it was originally surveyed. But often in situations like this there are deeper, more complex issues going on, and if you file a comment, then you've done the right thing. And I don't know the details of this, like I said, but often this reflects kind of a deeper problem or set of issues that go on. I mean, there are all kinds of interesting gerrymandering issues that go on with different political boundaries and so on, sometimes reaching international proportions.

RUSSELLSo if it's on side of Eastern Ave. or not, you know, I don't actually know. That's the kind of thing that we really do have to try to resolve with higher powers who own those keys.

NNAMDIWell, I can tell you, Carl, if you're driving west along Eastern Avenue in Northeast, and you go to Sergeant Road, you have to make a right, and then you go a couple of blocks, and then you have to make a left to pick up Eastern Avenue again. So it does dogleg in certain places, and that's been my experience. What say you, Carl?

CARLThe line itself is a straight line.

NNAMDIGot you.

CARLBut Eastern Avenue for much of its part is a straight line. We recently had a big story in the Post about the circle at Colesville and 16th.

NNAMDIYeah.

CARLThe line goes directly through there. If you go to Google Maps and try to find the boundaries, you're not going to find the correct boundary at all.

NNAMDII know that line.

RUSSELLSo can I ask you, Carl, how do you know? And it's sort of actually to the point that we've all been discussing. How do you know what you believe to be true about the boundary?

CARLBecause I've looked at D.C. maps going back into the 1800s.

RUSSELLOkay.

CARLYou can find them in the Library of Congress.

RUSSELLRight. So part of the issue here will be how do you know those maps are right? Because...

CARLThe boundary markers by the original surveyors and the District of Columbia for many years was the only jurisdiction in the United States that had its own surveyor who was dedicated to finding such things. So yes we do know with some accuracy. However, the fact of the matter is, showing line doglegging around that particular area is simply inaccurate and incorrect because it must be straight. Even if it's misplaced, it would still be straight.

NNAMDIDan Russell, are we getting, even as we speak, an example of how crowdsourcing works?

RUSSELLYes. It's exactly right, and as we've mentioned a couple times here, there can be a variety of opinions about this, and this is actually where an expert would come into play. So what I would want to do now is actually get the surveyor from the city to actually come out and take a look at this, because...

NNAMDIWell, he's listening and he'll be popping by shortly.

RUSSELLWould you please give me a call? We'll figure this out.

NNAMDIBut Carl, thank you very much for your call, and for raising that issue. We've got to take a short break. When we come back we'll continue this conversation on crowdsourcing, but you can still call us at 800-433-8850 with your questions or comments. What information sources do you trust? I'm Kojo Nnamdi.

NNAMDIWelcome back to our conversation on outsourcing. We're talking with Jennifer Chan. She is director of global emergency medicine at Northwestern University's Feinberg School of Medicine. Sam Gregory is program director at witness.org where he also leads the Cameras Everywhere initiative, and Dan Russell is a senior research scientist at Google known as Google's director of user happiness.

NNAMDIIf you'd like to join the conversation, call us at 800-433-8850, or send email to kojo@wamu.org. What sources of information do you trust? Wikipedia? YouTube? We got an email from Joe who writes, "There's a health-related non-profit that's using what I think is the first example of crowdsourcing to research a disease state, the fight against Alzheimer's. People worldwide can register online at the Alzheimer's registry. Unlike cancer, Alzheimer's doesn't usually run in a family, so anyone registering can help in the effort." Good example of crowdsourcing here, Dan?

RUSSELLPotentially. There's -- as we were just discussing, there's sort of tension between the amateurs who are deeply interested in topic and often know a great deal, and the experts who have a great deal of sort of technical background in this area. But there multiple sites that actually, for patient support -- mutual patient support, are just fantastic. There's one site called "Patients Like Me," I think, where people with a particular syndrome or disease can get together and talk about what their experience is.

RUSSELLAnd so they are crowdsourcing their experience of the disease process, and I think that's an interesting example. I don't happen to know this one site, but there's a lot of great crowdsourced material there for patients.

NNAMDISam Gregory, "I Paid a Bribe" is a site in India that collects anonymous reports of bribery. That site and others like it shed light on injustices through anonymous reports and images. How does that information translate into meaningful change?

GREGORYI'm not super familiar with "I Paid a Bribe" overall, but, you know, what you're doing there is trying to give information that can be used really by two audiences. One is within a community of people, so you can see a concentration of data points that says, well, if you go to the police station in this part of Delhi, you're going to see that a lot of people have had to pay a bribe, so it can inform you and provide information to a citizen about the choices they may need to make.

GREGORYIt can also drive broader advocacy efforts, and here I'm less sure how "I Paid a Bribe" is being used to translate into that. I think there's a question there of how we take these maps and these sets of data and translate that into policy action. A lot of the time, you know, and I'm familiar with, for example, efforts in Syria like the Syria rape crisis tracker which is trying to gather all the incidents of reporting of that, and it's saying, look, the more information we can get out into the public space, the more likely we're going to keep discussing that.

GREGORYAnd I think that's also a really important aspect of this is how do we keep a public space for discussion on this type of issue, and then, you know, when you're paying a bribe, it's a very personal act that for many people in some societies can be something that is recurring and something that is not really sort of shared out. And so thinking about how we keep a public space around the discussion I think is really vital as well.

NNAMDIPublic space important. On the other hand, Jennifer Chan, how important is it for organizations using crowdsourcing to protect the privacy of those participating?

CHANI think it's a very important conversation to have.

NNAMDII don't want anybody to know I reported something on "I Paid a Bribe," especially not the guy I bribed.

CHANTrue. And I think that, you know, as projects come together, a discussion around privacy, anonymity, risk, is a really important core part of, you know, discussing how that project will go out, and is not only the organization I would say that is it's helpful for that organization to have that discussion, but it's about the outward communication to the crowd whether it's an unbounded mass public versus a bounded crowd, which is a group of people who know one another who will be providing information.

CHANIt's about that interchange of what that information means. People -- sometimes for projects there's an opt-in, opt-out. You may want to know about the project. You may want to have that conversation, or be part of the advocacy, but, you know, but may not want to actually be the information provider. And so I think it is very important to explore, understand, and then come up with action plans.

NNAMDISam Gregory, how does facial recognition technology threaten to turn crowdsourcing against citizen activists?

GREGORYThis issue of facial recognition, and I think it's a broader issuer as well of how governments and other parties other than activists or people with good intentions use technologies and use the same technologies to repress. So, for example, after the Green Revolution in Iran, there was a very public crowdsourcing effort by the Iranian government in putting out the faces of individuals who had been in YouTube or in photos uploaded, and asking people to identify them.

GREGORYAnd really, that's in some sense a rather superficial act. I think it was more about terrifying or sort of shaming the opposition. But if you think about it, our face is our unique identifier. It's the thing that links, you know, the one time that I speak to the hundred other times that I'm on Facebook, you know, posting about my family life, and often human rights activism is about that lone act within a broader pattern. You know, I become an activist because I've seen injustice, it's not because every act of my day has to do with that.

GREGORYAnd once you've got that unique identifier linked to your act of activism can be correlated. So we put a lot of focus in our practice on this, what Jennifer was saying about, you know, really helping people understand when they might want to put their visual image out into the ecosystem, and also on tools that can help people control that. So we developed a tool called ObscuraCam that helped people blur out faces, and also strip out the metadata which is the underlying data in an image, because, of course, you can be exposed because someone sees your face and that is either recognized or put through a facial recognition algorithm, or you can be recognized by the data that's imbedded in an image.

GREGORYAnd on that, we were also very glad to see YouTube build one of -- a similar tool into their platform. And the reason we were excited about that and would like to see, you know, companies like Facebook and Twitter and others building similar approaches is you want to make these tools available to everyone who might turn out to be a citizen witness, because you can't assume that people will track down the human rights tool. We can't assume they'll come to witness.org to find the face blurring tool.

GREGORYYou can assume that people will explore Facebook or YouTube to find things that look fun to use maybe first at a party because you don't want your keg party photos out there too much, or you don't want the photos of your friend's kids in the school photo out there, and then that becomes a useful tool at a later point where you actually have an act of political activism where you do have to make a calculated decision about whether to link your face to every other act -- link that act to activism to every other act you commit. And so I think that's where facial recognition becomes very important.

NNAMDIYou care to comment on that, Jennifer?

CHANI would have to say, you know, I agree with a lot of Sam's comments about the -- really, the amazing power of transition of learning how to embrace user communities even though they don't seem to intrinsically be the area of your -- area of interest of expertise or their primary engagement, because it is about just communication, and is about understanding what may happen at a keg party, but understanding where those pieces can actually be translated.

CHANAnd also coming to a recognition that sometimes where some pieces don't translate and being okay with the split between the two.

NNAMDIDan Russell, in the academic model, the experts hold all the information. Crowdsourcing supposes that information is diffused among the group. Could the crowdsourcing model change, say, how education works in the U.S.?

RUSSELLAnother part of crowdsourcing is that expertise also exists in many people in ways that you hadn't thought about it before. So for in fact, you could imagine a Wikipedia article about the insects of southeast Pennsylvania written by the world's expert in that, the world's expert entomologist on that topic who actually nobody knows about. And so one of the great things about having this very low cost of publication and this low cost of participation, is that experts can be found.

RUSSELLThe Internet's full of marvelous people, including some that you just -- are not recognized. And so this is true for entomology, this is true for any obscure topic, and it's true for any kind of participant science, so citizen scientists are great, in particular because we can now see them, where they were invisible before.

NNAMDIExcept the other side of that coin, because crowdsourcing depends on volunteers, is that people like even you may worry that these are volunteers who are replacing paid professionals like the aforementioned entomologist.

RUSSELLRight. That's right. Well, I think one of the great things that's happening with education, in particular you were just mentioning, is that I suspect -- this is now a prediction, I suspect that we're going to see a kind of a democratization of education. That is, really great teachers can now, through the medium of publication on the Internet, say, through a MOOK or through an online class, can rise up and be visible where before they were just obscure, that obscure entomologist.

RUSSELLAnd so this will inevitably change the nature of entomology and change the nature of education. But I think for all of us, this implies that we're going to get a rising tide lifting all our education boats together.

NNAMDIAnd on that hopeful note, we've come to the end of this broadcast. Dan Russell is a senior research scientist at Google. He is known as Google's director of user happiness. He is this year's futurist-in-residence with the Future of Information Alliance. Dan Russell, thank you for joining us.

RUSSELLThank you, Kojo.

NNAMDIJennifer Chan is director of global emergency medicine at Northwestern University's Feinberg School of Medicine. She is also associate faculty and member of the Crisis Mapping and Early Warning Program at the Harvard Humanitarian Initiative. Jennifer Chan, thank you for joining us.

CHANThank you, Kojo.

NNAMDIAnd Sam Gregory is program director at witness.org where he also leads the Cameras Everywhere initiative. He is also a lecturer at the Harvard-Kennedy School. Sam Gregory, thank you for joining us.

GREGORYThank you.

NNAMDI"The Kojo Nnamdi Show" is produced by Brendan Sweeney, Michael Martinez, Ingalisa Schrobsdorff, Tayla Burney, Kathy Goldgeier, and Elizabeth Weinstein, with help from Stephannie Stokes. Our engineer is Tobey Schreiner. Natalie Yuravlivker is on the phones. Thank you all for listening. I'm Kojo Nnamdi.