Mark Zuckerberg

Facebook talked privacy, Google actually built it

Posted by | Apps, artificial intelligence, Developer, Facebook, facebook privacy, Google, Google I/O 2019, google privacy, Mark Zuckerberg, Mobile, Opinion, Policy, privacy, Sundar Pichai, TC | No Comments

Mark Zuckerberg: “The future is private”. Sundar Pichai: ~The present is private~. While both CEO’s made protecting user data a central theme of their conference keynotes this month, Facebook’s product updates were mostly vague vaporware while Google’s were either ready to ship or ready to demo. The contrast highlights the divergence in strategy between the two tech giants.

For Facebook, privacy is a talking point meant to boost confidence in sharing, deter regulators, and repair its battered image. For Google, privacy is functional, going hand-in-hand with on-device data processing to make features faster and more widely accessible.

Everyone wants tech to be more private, but we must discern between promises and delivery. Like “mobile”, “on-demand”, “AI”, and “blockchain” before it, “privacy” can’t be taken at face value. We deserve improvements to the core of how our software and hardware work, not cosmetic add-ons and instantiations no one is asking for.

AMY OSBORNE/AFP/Getty Images

At Facebook’s F8 last week, we heard from Zuckerberg about how “Privacy gives us the freedom to be ourselves” and he reiterated how that would happen through ephemerality and secure data storage. He said Messenger and Instagram Direct will become encrypted…eventually…which Zuckerberg had already announced in January and detailed in March. We didn’t get the Clear History feature that Zuckerberg made the privacy centerpiece of his 2018 conference, or anything about the Data Transfer Project that’s been silent for the 10 months since it’s reveal.

What users did get was a clumsy joke from Zuckerberg about how “I get that a lot of people aren’t sure that we’re serious about this. I know that we don’t exactly have the strongest reputation on privacy right now to put it lightly. But I’m committed to doing this well.” No one laughed. At least he admitted that “It’s not going to happen overnight.”

But it shouldn’t have to. Facebook made its first massive privacy mistake in 2007 with Beacon, which quietly relayed your off-site ecommerce and web activity to your friends. It’s had 12 years, a deal with the FTC promising to improve, countless screwups and apologies, the democracy-shaking Cambridge Analytica scandal, and hours of being grilled by congress to get serious about the problem. That makes it clear that if “the future is private”, then the past wasn’t. Facebook is too late here to receive the benefit of the doubt.

At Google’s I/O, we saw demos from Pichai showing how “our work on privacy and security is never done. And we want to do more to stay ahead of constantly evolving user expectations.” Instead of waiting to fall so far behind that users demand more privacy, Google has been steadily working on it for the past decade since it introduced Chrome incognito mode. It’s changed directions away from using Gmail content to target ads and allowing any developer to request access to your email, though there are plenty of sins to atone for. Now when the company is hit with scandals, it’s typically over its frightening efficiency as with its cancelled Project Maven AI military tech, not its creepiness.

Google made more progress on privacy in low-key updates in the runup to I/O than Facebook did on stage. In the past month it launched the ability to use your Android device as a physical security key, and a new auto-delete feature rolling out in the coming weeks that erases your web and app activity after 3 or 18 months. Then in its keynote today, it published “privacy commitments” for Made By Google products like Nest detailing exactly how they use your data and your control over that. For example, the new Nest Home Max does all its Face Match processing on device so facial recognition data isn’t sent to Google. Failing to note there’s a microphone in its Nest security alarm did cause an uproar in February, but the company has already course-corrected

That concept of on-device processing is a hallmark of the new Android 10 Q operating system. Opening in beta to developers today, it comes with almost 50 new security and privacy features like TLS 1.3 support and Mac address randomization. Google Assistant will now be better protected, Pichai told a cheering crowd. “Further advances in deep learning have allowed us to combine and shrink the 100 gigabyte models down to half a gigabyte — small enough to bring it onto mobile devices.” This makes Assistant not only more private, but fast enough that it’s quicker to navigate your phone by voice than touch. Here, privacy and utility intertwine.

The result is that Google can listen to video chats and caption them for you in real-time, transcribe in-person conversations, or relay aloud your typed responses to a phone call without transmitting audio data to the cloud. That could be a huge help if you’re hearing or vision impaired, or just have your hands full. A lot of the new Assistant features coming to Google Pixel phones this year will even work in Airplane mode. Pichai says that “Gboard is already using federated learning to improve next word prediction, as well as emoji prediction across 10s of millions of devices” by using on-phone processing so only improvements to Google’s AI are sent to the company, not what you typed.

Google’s senior director of Android Stephanie Cuthbertson hammered the idea home, noting that “On device machine learning powers everything from these incredible breakthroughs like Live Captions to helpful everyday features like Smart Reply. And it does this with no user input ever leaving the phone, all of which protects user privacy.” Apple pioneered much of the on-device processing, and many Google features still rely on cloud computing, but it’s swiftly progressing.

When Google does make privacy announcements about things that aren’t about to ship, they’re significant and will be worth the wait. Chrome will implement anti-fingerprinting tech and change cookies to be more private so only the site that created them can use them. And Incognito Mode will soon come to the Google Maps and Search apps.

Pichai didn’t have to rely on grand proclamations, cringey jokes, or imaginary product changes to get his message across. Privacy isn’t just a means to an end for Google. It’s not a PR strategy. And it’s not some theoretical part of tomorrow like it is for Zuckerberg and Facebook. It’s now a natural part of building user-first technology…after 20 years of more cavalier attitudes towards data. That new approach is why the company dedicated to organizing the world’s information has been getting so little backlash lately.

With privacy, it’s all about show, don’t tell.

Powered by WPeMatico

Facebook pivots to what it wishes it was

Posted by | Apps, F8 2019, Facebook, facebook groups, Facebook Marketplace, facebook privacy, Facebook Watch, Mark Zuckerberg, Mobile, privacy, Social | No Comments

In Facebook’s dreams, it’s a clean and private place. People spend their time having thoughtful discussions in “meaningful” Groups, planning offline meetups with Events, or laughing together in a Facebook Watch party.

In reality, Facebook is a cluttered mess of features that seem to constantly leak user data. People waste their time viewing inane News Feed posts from “friends” they never talk to, enviously stalking through photos of peers, or chowing on click-bait articles and viral videos in isolation. Facebook will never shake this reputation if it just keeps polishing its old features.

That’s why Facebook is rolling out what could be called an “aspirational redesign” known as FB5. Rather than polishing what Facebook was, it tries to spotlight what it wants to be. “This is the biggest change we’ve made to the Facebook app and site in five years” CEO Mark Zuckerberg said to open Facebook’s F8 conference yesterday.

The New Facebook

Most noticeably, that starts with sucking much of the blue out of the Facebook interface to making it look sparse and calming — despite a More button that unveils the social network’s bloat into dozens of rarely used features. A new logo features a brighter blue bubble around Facebook’s distinctive white f, which attempts to but a more uplifting spin on a bruised brand.

Functionally, FB5 means placing Groups near the center of a freshly tabbed interface for the both Facebook’s website and app, and putting suggestions for new ones to join across the service. “Everywhere there are friends, there should be Groups” says the head of the Facebook app Fidji Simo. Groups already has 1 billion monthly users, so Facebook is following the behavior pattern and doubling down. But Facebook’s goal is not only to have 2.38 billion people using the feature — the same number as use its whole app — but to get them all into meaningful Groups that emblematize their identity. 400 million already are. And now Groups for specific interests like gaming or health support will get special features, and power users will get a dashboard of updates across all their communities.

Groups will be flanked by Marketplace, perhaps the Facebook feature with the most latent potential. It’s a rapidly emerging use case Facebook wants to fuel. Just a a year and a half after launch, Marketplace had 800 million monthly users. Zuckerberg took Craigslist, added real identity to thwart bad behavior, and now is bolting it to the navigation bar of the most-used app on earth. The result is a place where it’s easy to put things up for sale and get tons of viewers. I once sold a couch on Marketplace in 20 minutes. Now sellers can take payments directly in the app instead of with cash or Venmo, and they can offer to ship items anywhere at the buyer’s expense. By following Zuckerberg’s mandate that 2019 focus on commerce, Facebook has become a viable Shopify competitor.

If Groups is what’s already working about Facebook’s future, Watch is the opposite. It’s a product designed to capture the video viewing bonanza Facebook observes on Netflix and YouTube. But without tent pole content like a “Game Of Thrones” or “Stranger Things”, it’s failed to impact the cultural zeitgeist. The closest thing it has to must-see video is Buffy The Vampire Slayer re-runs and a docuseries on NBA star Steph Curry. Facebook claims 75 million people now Watch for at least one minute per day though those 60 seconds don’t have to be  sequential. That’s still just 4 percent of its users. And a Diffusion study found 50 percent of adult US Facebook users had never even heard of Watch. Sticking it front and center demonstrates Facebook commitment to making Watch a hit even if it has to cram it down our throats.

Not The Old Facebook

The products of the past got little love on stage at F8. Nothing new for News Feed, Facebook’s mint but also the source of its misinformation woes. In the age of Snapchat and Zuckerberg’s newfound insistence on ephemerality to prevent embarrassment, the Timeline profile chronicling your whole Facebook life got nary a mention. And Pages for businesses that were the center of its monetization strategy years ago didn’t find space in the keynote, similar to how they’ve been butted out of the News Feed by competition and Facebook’s philosophical shift from public content to friends and family.

The one thing we heard a lot about but didn’t actually see much of was privacy. Zuckerberg started the conference declaring “The future is private!” He spoke about how Facebook plans to make its messaging apps encrypted, how it wants to be a living room rather than just a town hall, and how it’s following the shift in user behavior away from broadcasting. But we didn’t see any new privacy protections for the developer platform, a replacement for its Chief Security Officer that’s been vacant for nine months, or the Clear History feature Zuckerberg announced last year.

“I get that a lot of people aren’t sure that we’re serious about this. I know that we don’t exactly have the strongest reputation on privacy right now, to put it lightly” Zuckerberg joked without seeming to generate a single laugh. Combined with having little to show to enhance privacy, making fun of such a dire situation doesn’t instill much confidence. When Zuckerberg does take things seriously, it quickly manifests itself in the product like with Facebook’s 2012 shift to mobile, or in the company like with 2018’s doubling of security headcount. He knew mobile and content moderation failures could kill his network. But does someone who told Time magazine in 2010 that “What people want isn’t complete privacy” truly see a loose stance on privacy as an existential threat?

Interoperable, encrypted messaging will boost privacy, but it’s also just good business logic given Zuckerberg’s intention to own chat — the heart of your phone. Facebook’s creepiness stems from it sucking in data to power ad targeting. Nothing new was announced to address that. Despite his words, perhaps Zuckerberg doesn’t aspire to make Facebook as private as he aspired to make it mobile and secure. 

Wired reported that Zuckerberg authored a strategy book given to all employees ahead of the IPO that noted “If we don’t create the thing that kills Facebook, someone else will.” But F8 offered a new interpretation. Maybe given the lack of direct competitors in its league, and the absence of a mass exodus over its constant privacy scandals, it was the outdated product itself that was killing Facebook. The permanent Facebook. The all-you-do-is-scroll Facebook. The bored-of-my-friends Facebook. Users were being neglected rather than pushed away or stolen. By ignoring the past and emphasizing the products it aspires to have dominate tomorrow — Groups, Marketplace, Watch — Facebook can start to unchain itself from the toxic brand poisoning its potential.

Powered by WPeMatico

Zuckerberg wants messages to auto-expire to make Facebook a ‘living room’

Posted by | Apps, encryption, end-to-end encryption, Facebook, facebook messenger, Facebook Policy, facebook privacy, instagram, Mark Zuckerberg, Mobile, Policy, privacy, Social, TC, WhatsApp | No Comments

On feed-based “broader social networks, where people can accumulate friends or followers until the services feel more public . . . it feels more like a town square than a more intimate space like a living room” Facebook CEO Mark Zuckerberg explained in a blog post today. With messaging, groups, and ephemeral stories as the fastest growing social features, Zuckerberg laid out why he’s rethinking Facebook as a private living room where people can be comfortable being themselves without fear of hackers, government spying, and embarrassment from old content — all without encryption allowing bad actors to hide their crimes.

Perhaps this will just be more lip service in a time of PR crisis for Facebook. But with the business imperative fueled by social networking’s shift away from permanent feed broadcasting, Facebook can espouse the philosophy of privacy while in reality servicing its shareholders and bottom line. It’s this alignment that actually spurs product change. We saw Facebook’s agility with last year’s realization that a misinformation- and hate-plagued platform wouldn’t survive long-term so it had to triple its security and moderation staff. And in 2017, recognizing the threat of Stories, it implemented them across its apps. Now Facebook might finally see the dollar signs within privacy.

The New York Times’ Mike Isaac recently reported that Facebook planned to unify its Facebook, WhatsApp, and Instagram messaging infrastructure to allow cross-app messaging and end-to-end encryption. And Zuckerberg discussed this and the value of ephemerality on the recent earnings call. But now Zuckerberg has roadmapped a clearer slate of changes and policies to turn Facebook into a living room:

-Facebook will let users opt in to the ability to send or receive messages across Facebook, WhatsApp, and Instagram

-Facebook wants to expand that interoperability to SMS on Android

-Zuckerberg wants to make ephemerality automatic on messaging threads, so chats disappear by default after a month or year, with users able to control that or put timers on individual messages.

-Facebook plans to limit how long it retains metadata on messages once it’s no longer needed for spam or safety protections

-Facebook will extend end-to-end encryption across its messaging apps but use metadata and other non-content signals to weed out criminals using privacy to hide their misdeeds.

-Facebook won’t store data in countries with a bad track record of privacy abuse such as Russia, even if that means having to shut down or postpone operations in a country

You can read the full blog post from Zuckerberg below:

A Privacy-Focused Vision for Social Networking

My focus for the last couple of years has been understanding and addressing the biggest challenges facing Facebook. This means taking positions on important issues concerning the future of the internet. In this note, I’ll outline our vision and principles around building a privacy-focused messaging and social networking platform. There’s a lot to do here, and we’re committed to working openly and consulting with experts across society as we develop this.

Over the last 15 years, Facebook and Instagram have helped people connect with friends, communities, and interests in the digital equivalent of a town square. But people increasingly also want to connect privately in the digital equivalent of the living room. As I think about the future of the internet, I believe a privacy-focused communications platform will become even more important than today’s open platforms. Privacy gives people the freedom to be themselves and connect more naturally, which is why we build social networks.

Today we already see that private messaging, ephemeral stories, and small groups are by far the fastest growing areas of online communication. There are a number of reasons for this. Many people prefer the intimacy of communicating one-on-one or with just a few friends. People are more cautious of having a permanent record of what they’ve shared. And we all expect to be able to do things like payments privately and securely.

Public social networks will continue to be very important in people’s lives — for connecting with everyone you know, discovering new people, ideas and content, and giving people a voice more broadly. People find these valuable every day, and there are still a lot of useful services to build on top of them. But now, with all the ways people also want to interact privately, there’s also an opportunity to build a simpler platform that’s focused on privacy first.

I understand that many people don’t think Facebook can or would even want to build this kind of privacy-focused platform — because frankly we don’t currently have a strong reputation for building privacy protective services, and we’ve historically focused on tools for more open sharing. But we’ve repeatedly shown that we can evolve to build the services that people really want, including in private messaging and stories.

I believe the future of communication will increasingly shift to private, encrypted services where people can be confident what they say to each other stays secure and their messages and content won’t stick around forever. This is the future I hope we will help bring about.

We plan to build this the way we’ve developed WhatsApp: focus on the most fundamental and private use case — messaging — make it as secure as possible, and then build more ways for people to interact on top of that, including calls, video chats, groups, stories, businesses, payments, commerce, and ultimately a platform for many other kinds of private services.

This privacy-focused platform will be built around several principles:

Private interactions. People should have simple, intimate places where they have clear control over who can communicate with them and confidence that no one else can access what they share.

Encryption. People’s private communications should be secure. End-to-end encryption prevents anyone — including us — from seeing what people share on our services.

Permanence. People should be comfortable being themselves, and should not have to worry about what they share coming back to hurt them later. So we won’t keep messages or stories around for longer than necessary to deliver the service or longer than people want it.

Safety. People should expect that we will do everything we can to keep them safe on our services within the limits of what’s possible in an encrypted service.

Interoperability. People should be able to use any of our apps to reach their friends, and they should be able to communicate across networks easily and securely.

Secure data storage. People should expect that we won’t store sensitive data in countries with weak records on human rights like privacy and freedom of expression in order to protect data from being improperly accessed.

Over the next few years, we plan to rebuild more of our services around these ideas. The decisions we’ll face along the way will mean taking positions on important issues concerning the future of the internet. We understand there are a lot of tradeoffs to get right, and we’re committed to consulting with experts and discussing the best way forward. This will take some time, but we’re not going to develop this major change in our direction behind closed doors. We’re going to do this as openly and collaboratively as we can because many of these issues affect different parts of society.

Private Interactions as a Foundation

For a service to feel private, there must never be any doubt about who you are communicating with. We’ve worked hard to build privacy into all our products, including those for public sharing. But one great property of messaging services is that even as your contacts list grows, your individual threads and groups remain private. As your friends evolve over time, messaging services evolve gracefully and remain intimate.

This is different from broader social networks, where people can accumulate friends or followers until the services feel more public. This is well-suited to many important uses — telling all your friends about something, using your voice on important topics, finding communities of people with similar interests, following creators and media, buying and selling things, organizing fundraisers, growing businesses, or many other things that benefit from having everyone you know in one place. Still, when you see all these experiences together, it feels more like a town square than a more intimate space like a living room.

There is an opportunity to build a platform that focuses on all of the ways people want to interact privately. This sense of privacy and intimacy is not just about technical features — it is designed deeply into the feel of the service overall. In WhatsApp, for example, our team is obsessed with creating an intimate environment in every aspect of the product. Even where we’ve built features that allow for broader sharing, it’s still a less public experience. When the team built groups, they put in a size limit to make sure every interaction felt private. When we shipped stories on WhatsApp, we limited public content because we worried it might erode the feeling of privacy to see lots of public content — even if it didn’t actually change who you’re sharing with.

In a few years, I expect future versions of Messenger and WhatsApp to become the main ways people communicate on the Facebook network. We’re focused on making both of these apps faster, simpler, more private and more secure, including with end-to-end encryption. We then plan to add more ways to interact privately with your friends, groups, and businesses. If this evolution is successful, interacting with your friends and family across the Facebook network will become a fundamentally more private experience.

Encryption and Safety

People expect their private communications to be secure and to only be seen by the people they’ve sent them to — not hackers, criminals, over-reaching governments, or even the people operating the services they’re using.

There is a growing awareness that the more entities that have access to your data, the more vulnerabilities there are for someone to misuse it or for a cyber attack to expose it. There is also a growing concern among some that technology may be centralizing power in the hands of governments and companies like ours. And some people worry that our services could access their messages and use them for advertising or in other ways they don’t expect.

End-to-end encryption is an important tool in developing a privacy-focused social network. Encryption is decentralizing — it limits services like ours from seeing the content flowing through them and makes it much harder for anyone else to access your information. This is why encryption is an increasingly important part of our online lives, from banking to healthcare services. It’s also why we built end-to-end encryption into WhatsApp after we acquired it.

In the last year, I’ve spoken with dissidents who’ve told me encryption is the reason they are free, or even alive. Governments often make unlawful demands for data, and while we push back and fight these requests in court, there’s always a risk we’ll lose a case — and if the information isn’t encrypted we’d either have to turn over the data or risk our employees being arrested if we failed to comply. This may seem extreme, but we’ve had a case where one of our employees was actually jailed for not providing access to someone’s private information even though we couldn’t access it since it was encrypted.

At the same time, there are real safety concerns to address before we can implement end-to-end encryption across all of our messaging services. Encryption is a powerful tool for privacy, but that includes the privacy of people doing bad things. When billions of people use a service to connect, some of them are going to misuse it for truly terrible things like child exploitation, terrorism, and extortion. We have a responsibility to work with law enforcement and to help prevent these wherever we can. We are working to improve our ability to identify and stop bad actors across our apps by detecting patterns of activity or through other means, even when we can’t see the content of the messages, and we will continue to invest in this work. But we face an inherent tradeoff because we will never find all of the potential harm we do today when our security systems can see the messages themselves.

Finding the right ways to protect both privacy and safety is something societies have historically grappled with. There are still many open questions here and we’ll consult with safety experts, law enforcement and governments on the best ways to implement safety measures. We’ll also need to work together with other platforms to make sure that as an industry we get this right. The more we can create a common approach, the better.

On balance, I believe working towards implementing end-to-end encryption for all private communications is the right thing to do. Messages and calls are some of the most sensitive private conversations people have, and in a world of increasing cyber security threats and heavy-handed government intervention in many countries, people want us to take the extra step to secure their most private data. That seems right to me, as long as we take the time to build the appropriate safety systems that stop bad actors as much as we possibly can within the limits of an encrypted service. We’ve started working on these safety systems building on the work we’ve done in WhatsApp, and we’ll discuss them with experts through 2019 and beyond before fully implementing end-to-end encryption. As we learn more from those experts, we’ll finalize how to roll out these systems.

Reducing Permanence

We increasingly believe it’s important to keep information around for shorter periods of time. People want to know that what they share won’t come back to hurt them later, and reducing the length of time their information is stored and accessible will help.

One challenge in building social tools is the “permanence problem”. As we build up large collections of messages and photos over time, they can become a liability as well as an asset. For example, many people who have been on Facebook for a long time have photos from when they were younger that could be embarrassing. But people also really love keeping a record of their lives. And if all posts on Facebook and Instagram disappeared, people would lose access to a lot of valuable knowledge and experiences others have shared.

I believe there’s an opportunity to set a new standard for private communication platforms — where content automatically expires or is archived over time. Stories already expire after 24 hours unless you archive them, and that gives people the comfort to share more naturally. This philosophy could be extended to all private content.

For example, messages could be deleted after a month or a year by default. This would reduce the risk of your messages resurfacing and embarrassing you later. Of course you’d have the ability to change the timeframe or turn off auto-deletion for your threads if you wanted. And we could also provide an option for you to set individual messages to expire after a few seconds or minutes if you wanted.

It also makes sense to limit the amount of time we store messaging metadata. We use this data to run our spam and safety systems, but we don’t always need to keep it around for a long time. An important part of the solution is to collect less personal data in the first place, which is the way WhatsApp was built from the outset.

Interoperability

People want to be able to choose which service they use to communicate with people. However, today if you want to message people on Facebook you have to use Messenger, on Instagram you have to use Direct, and on WhatsApp you have to use WhatsApp. We want to give people a choice so they can reach their friends across these networks from whichever app they prefer.

We plan to start by making it possible for you to send messages to your contacts using any of our services, and then to extend that interoperability to SMS too. Of course, this would be opt-in and you will be able to keep your accounts separate if you’d like.

There are privacy and security advantages to interoperability. For example, many people use Messenger on Android to send and receive SMS texts. Those texts can’t be end-to-end encrypted because the SMS protocol is not encrypted. With the ability to message across our services, however, you’d be able to send an encrypted message to someone’s phone number in WhatsApp from Messenger.

This could also improve convenience in many experiences where people use Facebook or Instagram as their social network and WhatsApp as their preferred messaging service. For example, lots of people selling items on Marketplace list their phone number so people can message them about buying it. That’s not ideal, because you’re giving strangers your phone number. With interoperability, you’d be able to use WhatsApp to receive messages sent to your Facebook account without sharing your phone number — and the buyer wouldn’t have to worry about whether you prefer to be messaged on one network or the other.

You can imagine many simple experiences — a person discovers a business on Instagram and easily transitions to their preferred messaging app for secure payments and customer support; another person wants to catch up with a friend and can send them a message that goes to their preferred app without having to think about where that person prefers to be reached; or you simply post a story from your day across both Facebook and Instagram and can get all the replies from your friends in one place.

You can already send and receive SMS texts through Messenger on Android today, and we’d like to extend this further in the future, perhaps including the new telecom RCS standard. However, there are several issues we’ll need to work through before this will be possible. First, Apple doesn’t allow apps to interoperate with SMS on their devices, so we’d only be able to do this on Android. Second, we’d need to make sure interoperability doesn’t compromise the expectation of encryption that people already have using WhatsApp. Finally, it would create safety and spam vulnerabilities in an encrypted system to let people send messages from unknown apps where our safety and security systems couldn’t see the patterns of activity.

These are significant challenges and there are many questions here that require further consultation and discussion. But if we can implement this, we can give people more choice to use their preferred service to securely reach the people they want.

Secure Data Storage

People want to know their data is stored securely in places they trust. Looking at the future of the internet and privacy, I believe one of the most important decisions we’ll make is where we’ll build data centers and store people’s sensitive data.

There’s an important difference between providing a service in a country and storing people’s data there. As we build our infrastructure around the world, we’ve chosen not to build data centers in countries that have a track record of violating human rights like privacy or freedom of expression. If we build data centers and store sensitive data in these countries, rather than just caching non-sensitive data, it could make it easier for those governments to take people’s information.

Upholding this principle may mean that our services will get blocked in some countries, or that we won’t be able to enter others anytime soon. That’s a tradeoff we’re willing to make. We do not believe storing people’s data in some countries is a secure enough foundation to build such important internet infrastructure on.

Of course, the best way to protect the most sensitive data is not to store it at all, which is why WhatsApp doesn’t store any encryption keys and we plan to do the same with our other services going forward.

But storing data in more countries also establishes a precedent that emboldens other governments to seek greater access to their citizen’s data and therefore weakens privacy and security protections for people around the world. I think it’s important for the future of the internet and privacy that our industry continues to hold firm against storing people’s data in places where it won’t be secure.

Next Steps

Over the next year and beyond, there are a lot more details and trade-offs to work through related to each of these principles. A lot of this work is in the early stages, and we are committed to consulting with experts, advocates, industry partners, and governments — including law enforcement and regulators — around the world to get these decisions right.

At the same time, working through these principles is only the first step in building out a privacy-focused social platform. Beyond that, significant thought needs to go into all of the services we build on top of that foundation — from how people do payments and financial transactions, to the role of businesses and advertising, to how we can offer a platform for other private services.

But these initial questions are critical to get right. If we do this well, we can create platforms for private sharing that could be even more important to people than the platforms we’ve already built to help people share and connect more openly.

Doing this means taking positions on some of the most important issues facing the future of the internet. As a society, we have an opportunity to set out where we stand, to decide how we value private communications, and who gets to decide how long and where data should be stored.

I believe we should be working towards a world where people can speak privately and live freely knowing that their information will only be seen by who they want to see it and won’t all stick around forever. If we can help move the world in this direction, I will be proud of the difference we’ve made.

Powered by WPeMatico

Highlights & transcript from Zuckerberg’s 20K-word ethics talk

Posted by | Advertising Tech, Apps, Developer, Facebook, Facebook Policy, Facebook Politics, facebook privacy, Government, Mark Zuckerberg, Media, Mobile, Policy, Social, Talent, TC | No Comments

Mark Zuckerberg says it might be right for Facebook to let people pay to not see ads, but that it would feel wrong to charge users for extra privacy controls. That’s just one of the fascinating philosophical views the CEO shared during the first of his public talks he’s promised as part of his 2019 personal challenge.

Talking to Harvard Law and computer science professor Jonathan Zittrain on the campus of the university he dropped out of, Zuckerberg managed to escape the 100-minute conversation with just a few gaffes. At one point he said “we definitely don’t want a society where there’s a camera in everyone’s living room watching the content of those conversations”. Zittrain swiftly reminded him that’s exactly what Facebook Portal is, and Zuckerberg tried to deflect by saying Portal’s recordings would be encrypted.

Later Zuckerberg mentioned “the ads, in a lot of places are not even that different from the organic content in terms of the quality of what people are being able to see” which is pretty sad and derisive assessment of the personal photos and status updates people share. And when he suggested crowdsourced fact-checking, Zittrain chimed in that this could become an avenue for “astroturfing” where mobs of users provide purposefully biased information to promote their interests, like a political group’s supporting voting that their opponents’ facts are lies. While sometimes avoiding hard stances on questions, Zuckerberg was otherwise relatively logical and coherent.

Policy And Cooperating With Governments

The CEO touched on his borderline content policy that quietly demotes posts that come close to breaking its policy against nudity, hate speech etc that otherwise are the most sensational and get the most distribution but don’t make people feel good. Zuckerberg noted some progress here, saying “a lot of the things that we’ve done in the last year were focused on that problem and it really improves the quality of the service and people appreciate that.”

This aligns with Zuckerberg contemplating Facebook’s role as a “data fiduciary” where rather than necessarily giving in to users’ urges or prioritizing its short-term share price, the company tries to do what’s in the best long-term interest of its communities. “There’s a hard balance here which is — I mean if you’re talking about what people want to want versus what they want– you know, often people’s revealed preferences of what they actually do shows a deeper sense of what they want than what they think they want to want” he said. Essentially, people might tap on clickbait even if it doesn’t make them feel good.

On working with governments, Zuckerberg explained how incentives weren’t always aligned, like when law enforcement is monitoring someone accidentally dropping clues about their crimes and collaborators. The government and society might benefit from that continued surveillance but Facebook might want to immediately suspend the account if it found out. “But as you build up the relationships and trust, you can get to that kind of a relationship where they can also flag for you, ‘Hey, this is where we’re at’”, implying Facebook might purposefully allow that person to keep incriminating themselves to assist the authorities.

But disagreements between governments can flare up, Zuckerberg notes that “we’ve had employees thrown in jail because we have gotten court orders that we have to turnover data that we wouldn’t probably anyway, but we can’t because it’s encrypted.” That’s likely a reference to the 2016 arrest of Facebook’s VP for Latin Amercia Diego Dzodan over WhatsApp’s encryption preventing the company from providing evidence for a drug case.

Decentralizing Facebook

The tradeoffs of encryption and decentralization were a central theme. He discussed how while many people fear how encryption could mask illegal or offensive activity, Facebook doesn’t have to peek at someone’s actual content to determine they’re violating policy. “One of the — I guess, somewhat surprising to me — findings of the last couple of years of working on content governance and enforcement is that it often is much more effective to identify fake accounts and bad actors upstream of them doing something bad by patterns of activity rather than looking at the content” Zuckerberg said.

With Facebook rapidly building out a blockchain team to potentially launch a cryptocurrency for fee-less payments or an identity layer for decentralized applications, Zittrain asked about the potential for letting users control which other apps they give their profile information to without Facebook as an intermediary.

SAN JOSE, CA – MAY 01: Facebook CEO Mark Zuckerberg (Photo by Justin Sullivan/Getty Images)

Zuckerberg stressed that at Facebook’s scale, moving to a less efficient distributed architecture would be extremely “computationally intense” though it might eventually be possible. Instead, he said “One of the things that I’ve been thinking about a lot is a use of blockchain that I am potentially interesting in– although I haven’t figured out a way to make this work out, is around authentication and bringing– and basically granting access to your information and to different services. So, basically, replacing the notion of what we have with Facebook Connect with something that’s fully distributed.” This might be attractive to developers who would know Facebook couldn’t cut them off from the users.

The problem is that if a developer was abusing users, Zuckerberg fears that “in a fully distributed system there would be no one who could cut off the developers’ access. So, the question is if you have a fully distributed system, it dramatically empowers individuals on the one hand, but it really raises the stakes and it gets to your questions around, well, what are the boundaries on consent and how people can really actually effectively know that they’re giving consent to an institution?”

No “Pay For Privacy”

But perhaps most novel and urgent were Zuckerberg’s comments on the secondary questions raised by where Facebook should let people pay to remove ads. “You start getting into a principle question which is ‘are we going to let people pay to have different controls on data use than other people?’ And my answer to that is a hard no.” Facebook has promised to always operate free version so everyone can have a voice. Yet some including myself have suggested that a premium ad-free subscription to Facebook could help ween it off maximizing data collection and engagement, though it might break Facebook’s revenue machine by pulling the most affluent and desired users out of the ad targeting pool.

“What I’m saying is on the data use, I don’t believe that that’s something that people should buy. I think the data principles that we have need to be uniformly available to everyone. That to me is a really important principle” Zuckerberg expands. “It’s, like, maybe you could have a conversation about whether you should be able to pay and not see ads. That doesn’t feel like a moral question to me. But the question of whether you can pay to have different privacy controls feels wrong.”

Back in May, Zuckerberg announced Facebook would build a Clear History button in 2018 that deletes all the web browsing data the social network has collected about you, but that data’s deep integration into the company’s systems has delayed the launch. Research suggests users don’t want the inconvenience of getting logged out of all their Facebook Connected services, though, they’d like to hide certain data from the company.

“Clear history is a prerequisite, I think, for being able to do anything like subscriptions. Because, like, partially what someone would want to do if they were going to really actually pay for a not ad supported version where their data wasn’t being used in a system like that, you would want to have a control so that Facebook didn’t have access or wasn’t using that data or associating it with your account. And as a principled matter, we are not going to just offer a control like that to people who pay.”

Of all the apologies, promises, and predictions Zuckerberg has made recently, this pledge might instill the most confidence. While some might think of Zuckerberg as a data tyrant out to absorb and exploit as much of our personal info as possible, there are at least lines he’s not willing to cross. Facebook could try to charge you for privacy, but it won’t. And given Facebook’s dominance in social networking and messaging plus Zuckerberg’s voting control of the company, a greedier man could make the internet much worse.

TRANSCRIPT – MARK ZUCKERBERG AT HARVARD / FIRST PERSONAL CHALLENGE 2019

Jonathan Zittrain: Very good. So, thank you, Mark, for coming to talk to me and to our students from the Techtopia program and from my “Internet and Society” course at Harvard Law School. We’re really pleased to have a chance to talk about any number of issues and we should just dive right in. So, privacy, autonomy, and information fiduciaries.

Mark Zuckerberg: All right!

Jonathan Zittrain: Love to talk about that.

Mark Zuckerberg: Yeah! I read your piece in The New York Times.

Jonathan Zittrain: The one with the headline that said, “Mark Zuckerberg can fix this mess”?

Mark Zuckerberg: Yeah.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: Although that was last year.

Jonathan Zittrain: That’s true! Are you suggesting it’s all fixed?

Mark Zuckerberg: No. No.

Jonathan Zittrain: Okay, good. So–

Jonathan Zittrain: I’m suggesting that I’m curious whether you still think that we can fix this mess?

Jonathan Zittrain: Ah!

Jonathan Zittrain: I hope–

Jonathan Zittrain: “Hope springs eternal”–

Mark Zuckerberg: Yeah, there you go.

Jonathan Zittrain: –is my motto. So, all right, let me give a quick characterization of this idea that the coinage and the scaffolding for it is from my colleague, Jack Balkin, at Yale. And the two of us have been developing it out further. There are a standard number of privacy questions with which you might have some familiarity, having to do with people conveying information that they know they’re conveying or they’re not so sure they are, but “mouse droppings” as we used to call them when they run in the rafters of the Internet and leave traces. And then the standard way of talking about that is you want to make sure that that stuff doesn’t go where you don’t want it to go. And we call that “informational privacy”. We don’t want people to know stuff that we want maybe our friends only to know. And on a place like Facebook, you’re supposed to be able to tweak your settings and say, “Give them to this and not to that.” But there’s also ways in which stuff that we share with consent could still sort of be used against us and it feels like, “Well, you consented,” may not end the discussion. And the analogy that my colleague Jack brought to bear was one of a doctor and a patient or a lawyer and a client or– sometimes in America, but not always– a financial advisor and a client that says that those professionals have certain expertise, they get trusted with all sorts of sensitive information from their clients and patients and, so, they have an extra duty to act in the interests of those clients even if their own interests conflict. And, so, maybe just one quick hypo to get us started. I wrote a piece in 2014, that maybe you read, that was a hypothetical about elections in which it said, “Just hypothetically, imagine that Facebook had a view about which candidate should win and they reminded people likely to vote for the favored candidate that it was Election Day,” and to others they simply sent a cat photo. Would that be wrong? And I find– I have no idea if it’s illegal; it does seem wrong to me and it might be that the fiduciary approach captures what makes it wrong.

Mark Zuckerberg: All right. So, I think we could probably spend the whole next hour just talking about that!

Mark Zuckerberg: So, I read your op-ed and I also read Balkin’s blogpost on information fiduciaries. And I’ve had a conversation with him, too.

Jonathan Zittrain: Great.

Mark Zuckerberg: And the– at first blush, kind of reading through this, my reaction is there’s a lot here that makes sense. Right? The idea of us having a fiduciary relationship with the people who use our services is kind of intuitively– it’s how we think about how we’re building what we’re building. So, reading through this, it’s like, all right, you know, a lot of people seem to have this mistaken notion that when we’re putting together news feed and doing ranking that we have a team of people who are focused on maximizing the time that people spend, but that’s not the goal that we give them. We tell people on the team, “Produce the service–” that we think is going to be the highest quality that– we try to ground it in kind of getting people to come in and tell us, right, of the content that we could potentially show what is going to be– they tell us what they want to see, then we build models that kind of– that can predict that, and build that service.

Jonathan Zittrain: And, by the way, was that always the case or–

Mark Zuckerberg: No.

Jonathan Zittrain: –was that a place you got to through some course adjustments?

Mark Zuckerberg: Through course adjustments. I mean, you start off using simpler signals like what people are clicking on in feed, but then you pretty quickly learn, “Hey, that gets you to local optimum,” right? Where if you’re focusing on what people click on and predicting what people click on, then you select for click bait. Right? So, pretty quickly you realize from real feedback, from real people, that’s not actually what people want. You’re not going to build the best service by doing that. So, you bring in people and actually have these panels of– we call it “getting to ground truth”– of you show people all the candidates for what can be shown to them and you have people say, “What’s the most meaningful thing that I wish that this system were showing us? So, all this is kind of a way of saying that our own self image of ourselves and what we’re doing is that we’re acting as fiduciaries and trying to build the best services for people. Where I think that this ends up getting interesting is then the question of who gets to decide in the legal sense or the policy sense of what’s in people’s best interest? Right? So, we come in every day and think, “Hey, we’re building a service where we’re ranking newsfeed trying to show people the most relevant content with an assumption that’s backed by data; that, in general, people want us to show them the most relevant content. But, at some level, you could ask the question which is “Who gets to decide that ranking newsfeed or showing relevant ads?” or any of the other things that we choose to work on are actually in people’s interest. And we’re doing the best that we can to try to build the services [ph?] that we think are the best. At the end of the day, a lot of this is grounded in “People choose to use it.” Right? Because, clearly, they’re getting some value from it. But then there are all these questions like you say about, you have– about where people can effectively give consent and not.

Jonathan Zittrain: Yes.

Mark Zuckerberg: So, I think that there’s a lot of interesting questions in this to unpack about how you’d implement a model like that. But, at a high level I think, you know, one of the things that I think about in terms of we’re running this big company; it’s important in society that people trust the institutions of society. Clearly, I think we’re in a position now where people rightly have a lot of questions about big internet companies, Facebook in particular, and I do think getting to a point there there’s the right regulation and rules in place just provides a kind of societal guardrail framework where people can have confidence that, okay, these companies are operating within a framework that we’ve all agreed. That’s better than them just doing whatever they want. And I think that that would give people confidence. So, figuring out what that framework is, I think, is a really important thing. And I’m sure we’ll talk about that as it relates–

Jonathan Zittrain: Yes.

Mark Zuckerberg: –to a lot of the content areas today. But getting to that question of how do you– “Who determines what’s in people’s best interest, if not people themselves?”Jonathan Zittrain: Yes.

Mark Zuckerberg: –is a really interesting question.

Jonathan Zittrain: Yes, so, we should surely talk about that. So, on our agenda is the “Who decides?” question.

Mark Zuckerberg: All right.

Jonathan Zittrain: Other agenda items include– just as you say, the fiduciary framework sounds nice to you– doctors, patients, Facebook users. And I hear you saying that’s pretty much where you’re wanting to end up anyway. There are some interesting questions about what people want, versus what they want to want.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: People will say “On January 1st, what I want–” New Year’s resolution– “is a gym membership.” And then on January 2nd, they don’t want to go to the gym. They want to want to go to the gym, but they never quite make it. And then, of course, a business model of pay for the whole year ahead of time and they know you’ll never turn up develops around that. And I guess a specific area to delve into for a moment on that might be on the advertising side of things, maybe the dichotomy between personalization and does it ever going into exploitation? Now, there might be stuff– I know Facebook, for example, bans payday loans as best it can.

Mark Zuckerberg: Mm-hm.

Jonathan Zittrain: That’s just a substantive area that it’s like, “All right, we don’t want to do that.”

Mark Zuckerberg: Mm-hm.

Jonathan Zittrain: But when we think about good personalization so that Facebook knows I have a dog and not a cat, and a targeter can then offer me dog food and not cat food. How about, if not now, a future day in which an advertising platform can offer to an ad targeter some sense of “I just lost my pet, I’m really upset, I’m ready to make some snap decisions that I might regret later, but when I make them–“

Mark Zuckerberg: Mm-hm.

Jonathan Zittrain: “–I’m going to make them.” So, this is the perfect time to tee up

Mark Zuckerberg: Yeah.

Jonathan Zittrain: –a Cubic Zirconia or whatever the thing is that– .

Mark Zuckerberg: Mm-hm.

Jonathan Zittrain: That seems to me a fiduciary approach would say, ideally– how we get there I don’t know, but ideally we wouldn’t permit that kind of approach to somebody using the information we’ve gleaned from them to know they’re in a tough spot–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: –and then to exploit them. But I don’t know. I don’t know how you would think about something like that. Could you write an algorithm to detect something like that?

Mark Zuckerberg: Well, I think one of the key principles is that we’re trying to run this company for the long term. And I think that people think that a lot of things that– if you were just trying to optimize the profits for next quarter or something like that, you might want to do things that people might like in the near term, but over the long term will come to resent. But if you actually care about building a community and achieving this mission and building the company for the long term, I think you’re just much more aligned than people often think companies are. And it gets back to the idea before, where I think our self image is largely acting as– in this kind of fiduciary relationship as you’re saying– and across– we could probably go through a lot of different examples. I mean, we don’t want to show people content that they’re going to click on and engage with, but then feel like they wasted their time afterwards. Where we don’t want to show them things that they’re going to make a decision based off of that and then regret later. I mean, there’s a hard balance here which is– I mean if you’re talking about what people want to want versus what they want– you know, often people’s revealed preferences of what they actually do shows a deeper sense of what they want than what they think they want to want. So, I think there’s a question between when something is exploitative versus when something is real, but isn’t what you would say that you want.

Jonathan Zittrain: Yes.

Mark Zuckerberg: And that’s a really hard thing to get at.

Jonathan Zittrain: Yes.

Mark Zuckerberg: But on a lot of these cases my experience of running the company is that you start off building a system, you have relatively unsophisticated signals to start, and you build up increasingly complex models over time that try to take into account more of what people care about. And there are all these examples that we can go through. I think probably newsfeed and ads are probably the two most complex ranking examples–

Jonathan Zittrain: Yes.

Mark Zuckerberg: –that we have. But it’s– like we were talking about a second ago, when we started off with the systems, I mean, just start with newsfeeds– but you could do this on ads, too– you know, the most naïve signals, right, are what people click on or what people “Like”. But then you just very quickly realize that that doesn’t– it approximates something, but it’s a very crude approximation of the ground truth of what people actually care about. So, what you really want to get to is as much as possible getting real people to look at the real candidates for content and tell you in a multi-dimensional way what matters to them and try to build systems that model that. And then you want to be kind of conservative on preventing downside. So, your example of the payday loans– and when we’ve talked about this in the past, your– you’ve put the question to me of “How do you know when a payday loan is going to be exploitative?” right? “If you’re targeting someone who is in a bad situation?” And our answer is, “Well, we don’t really know when it’s going to be exploitative, but we think that the whole category potentially has a massive risk of that, so we just ban it–

Jonathan Zittrain: Right. Which makes it an easy case.

Mark Zuckerberg: Yes. And I think that the harder cases are when there’s significant upside and significant downside and you want to weigh both of them. So, I mean, for example, once we started putting together a really big effort on preventing election interference, one of the initial ideas that came up was “Why don’t we just ban all ads that relate to anything that is political?” And they you pretty quickly get into, all right, well, what’s a political ad? The classic legal definition is things that are around elections and candidates, but that’s not actually what Russia and other folks were primarily doing. Right? It’s– you know, a lot of the issues that we’ve seen are around issue ads, right, and basically sewing division on what are social issues. So, all right, I don’t think you’re going to get in the way of people’s speech and ability to promote and do advocacy on issues that they care about. So, then the question is “All right, well, so, then what’s the right balance?” of how do you make sure that you’re providing the right level of controls, that people who aren’t supposed to be participating in these debates aren’t or that at least you’re providing the right transparency. But I think we’ve veered a little bit from the original questionJonathan Zittrain: Yes.

Mark Zuckerberg: –but the– but, yeah. So, let’s get back to where you were

Jonathan Zittrain: Well, here’s– and this is a way of maybe moving it forward, which is: A platform as complete as Facebook is these days offers lots of opportunities to shape what people see and possibly to help them with those nudges, that it’s time to go to the gym or to avoid them from falling into the depredations of the payday loan. And it is a question of so long as the platform to do it, does it now have an ethical obligation to do it, to help people achieve the good life?

Mark Zuckerberg: Mm-hm.

Jonathan Zittrain: And I worry that it is too great a burden for any company to bear to have to figure out, say, if not the perfect, the most reasonable newsfeed for every one of the– how many? Two and a half billion active users? Something like that.

Mark Zuckerberg: Yeah. On that order.

Jonathan Zittrain: All the time and there might be some ways that start a little bit to get into the engineering of the thing that would say, “Okay, with all hindsight, are there ways to architect this so that the stakes aren’t as high, aren’t as focused on just, “Gosh, is Facebook doing this right?” It’s as if there was only one newspaper in the whole world or one or two, and it’s like, “Well, then what The New York Times chooses to put on it’s home page, if it were the only newspaper, would have outsize importance.”

Mark Zuckerberg: Mm-hm.

Jonathan Zittrain: So, just as a technical matter, a number of the students in this room had a chance to hear from Tim Berners-Lee, inventor of the World Wide Web, and he has a new idea for something called “Solid”. I don’t know if you’ve heard of Solid. It’s a protocol more than it is a product. So, there’s no car to move off the lot today. But its idea is allowing people to have the data that they generate as they motor around the web end up in their own kind of data locker. Now, for somebody like Tim, it might mean literally in a locker under his desk and he could wake up in the middle of the night and see where his data is. For others, it might mean Iraq somewhere, guarded perhaps by a fiduciary who’s looking out for them, the way that we put money in a bank and then we can sleep at night knowing the bankers are– this is maybe not the best analogy in 2019, but watching.

Mark Zuckerberg: We’ll get there.

Jonathan Zittrain: We’ll get there. But Solid says if you did that, people would then– or their helpful proxies– be able to say, “All right, Facebook is coming along. It wants the following data from me and including that data that it has generated about me as I use it, but stored back in my locker and it kind of has to come back to my well to draw water each time. And that way if I want to switch to Schmacebook or something, it’s still in my well and I can just immediately grant permission to Schmacebook to see it and I don’t have to do a kind of data slurp and then re-upload it. It’s a fully distributed way of thinking about data. And I’m curious from an engineering perspective does this seem doable with something of the size and the number of spinning wheels that Facebook has and does it seem like a

Mark Zuckerberg: Yeah–

Jonathan Zittrain: –and I’m curious your reaction to an idea like that.

Mark Zuckerberg: So, I think it’s quite interesting. Certainly, the level of computation that Facebook is doing and all the services that we’re building is really intense to do in a distributed way. I mean, I think as a basic model I think we’re building out the data center capacity over the next five years and our plan for what we think we need to do that we think is on the order of all of what AWS and Google Cloud are doing for supporting all of their customers. So, okay, so, this is like a relatively computationally intense thing.

Over time you assume you’ll get more compute. So, decentralized things which are less efficient computationally will be harder– sorry, they’re harder to do computation on, but eventually maybe you have the compute resources to do that. I think the more interesting questions there are not feasibility in the near term, but are the philosophical questions of the goodness of a system like that.

So, one question if you want to– so, we can get into decentralization, one of the things that I’ve been thinking about a lot is a use of blockchain that I am potentially interesting in– although I haven’t figured out a way to make this work out, is around authentication and bringing– and basically granting access to your information and to different services. So, basically, replacing the notion of what we have with Facebook Connect with something that’s fully distributed.

Jonathan Zittrain: “Do you want to login with your Facebook account?” is the status quo

Mark Zuckerberg: Basically, you take your information, you store it on some decentralized system and you have the choice of whether to login to different places and you’re not going through an intermediary, which is kind of like what you’re suggesting here–

Jonathan Zittrain: Yes.

Mark Zuckerberg: –in a sense. Okay, now, there’s a lot of things that I think would be quite attractive about that. You know, for developers one of the things that is really troubling about working with our system, or Google’s system for that matter, or having your services through Apple’s app store, is that you don’t want to have an intermediary between serving your– the people who are using your service and you, right, where someone can just say, “Hey, we as a developer have to follow your policy and if we don’t, then you can cut off access to the people we’re serving.” That’s kind of a difficult and troubling position to be in. I think developers–

Jonathan Zittrain: –you’re referring to a recent incident.

Mark Zuckerberg: No, well, I was– well, sure

Mark Zuckerberg: But I think it underscores the– I think every developer probably feels this: People are using any app store but also login with Facebook, with Google; any of these services, you want a direct relationship with the people you serve.

Jonathan Zittrain: Yes.

Mark Zuckerberg: Now, okay, but let’s look at the flip side. So, what we saw in the last couple of years with Cambridge Analytica, was basically an example where people chose to take data that they– some of it was their data, some of it was data that they had seen from their friends, right? Because if you want to do things like making it so alternative services can build a competing newsfeed, then you need to be able to make it so that people can bring the data that they see you [ph?] within the system. Okay, theybasically, people chose to give their data to a developer who’s affiliated with Cambridge University, which is a really respected institution, and then that developer turned around and sold the data to the firm Cambridge Analytica, which is in violation of our policies. So, we cut off the developers’ access. And, of course, in a fully distributed system there would be no one who could cut off the developers’ access. So, the question is if you have a fully distributed system, it dramatically empowers individuals on the one hand, but it really raises the stakes and it gets to your questions around, well, what are the boundaries on consent and how people can really actually effectively know that they’re giving consent to an institution?

In some ways it’s a lot easier to regulate and hold accountable large companies like Facebook or Google, because they’re more visible, they’re more transparent than the long tail of services that people would chose to then go interact with directly. So, I think that this is a really interesting social question. To some degree I think this idea of going in the direction of blockchain authentication is less gated on the technology and capacity to do that. I think if you were doing fully decentralized Facebook, that would take massive computation, but I’m sure we could do fully decentralized authentication if we wanted to. I think the real question is do you really want that?

Jonathan Zittrain: Yes.

Mark Zuckerberg: Right? And I think you’d have more cases where, yes, people would be able to not have an intermediary, but you’d also have more cases of abuse and the recourse would be much harder.

Jonathan Zittrain: Yes. What I hear you saying is people as they go about their business online are generating data about themselves that’s quite valuable, if not to themselves, to others who might interact with them. And the more they are empowered, possibly through a distributed system, to decide where that data goes, with whom they want to share it, the more they could be exposed to exploitation. this is a genuine dilemma–

Mark Zuckerberg: Yeah, yeah.

Jonathan Zittrain: –because I’m a huge fan of decentralization.

Mark Zuckerberg: Yeah, yeah.

Jonathan Zittrain: But I also see the problem. And maybe one answer is there’s some data that’s just so toxic there’s no vessel we should put it in; it might eat a whole through it or something, metaphorically speaking. But, then again, innocuous data can so quickly be assembled into something scary. So, I don’t know if the next election–

Mark Zuckerberg: Yeah. [ph?] I mean, I think in general we’re talking about the large-scale of data being assembled into meaning something different from what the individual data points mean.

Jonathan Zittrain: Yes.

Mark Zuckerberg: And I think that’s the whole challenge here. But I philosophically agree with you thatI mean, I want to think about the– like, I do think about the work that we’re doing as a decentralizing force in the world, right? A lot of the reason why I think people of my generation got into technology is because we believe that technology gives individuals power and isn’t massively centralizing. Now you’ve built a bunch of big companies in the process, but I think what has largely happened is that individuals today have more voice, more ability to affiliate with who they want, and stay connected with people, ability to form communities in ways that they couldn’t before, and I think that’s massively empowering to individuals and that’s philosophically kind of the side that I tend to be on. So, that’s why I’m thinking about going back to decentralized or blockchain authentication. That’s why I’m kind of bouncing around how could you potentially make this work, because from my orientation is to try to go in that direction.

Jonathan Zittrain: Yes.

Mark Zuckerberg: An example where I think we’re generally a lot closer to going in that direction is encryption. I mean, this is, like, one of the really big debates today is basically what are the boundaries on where you would want a messaging service to be encrypted. And there are all these benefits from a privacy and security perspective, but, on the other hand, if what we’re trying to do– one of the big issues that we’re grappling with content governance and where is the line between free expression and, I suppose, privacy on one side, but safety on the other as people do really bad things, right, some of the time. And I think people rightfully have an expectation of us that we’re going to do everything we can to stop terrorists from recruiting people or people from exploiting children or doing different things. And moving in the direction of making these systems more encrypted certainly reduces some of the signals that we would have access to be able to do some of that really important work.

But here we are, right, we’re sitting in this position where we’re running WhatsApp, which is the largest end-to-end encrypting service in the world; we’re running messenger, which is another one of the largest messaging systems in the world where encryption is an option, but it isn’t the default. I don’t think long term it really makes sense to be running different systems with very different policies on this. I think this is sort of a philosophical question where you want to figure out where you want to be on it. And, so, my question for you– now,

I’ll talk about how I’m thinking about this– is all right, if you were in my position and you got to flip a switch is probably too glib, because there’s a lot of work that goes into this, and go in one direction for both of those services, who would you think about that?

Jonathan Zittrain: Well, the question you’re putting on the table, which is a hard one is “Is it okay,” and let’s just take the simple case, “for two people to communicate with each other in a way that makes it difficult for any third party to casually listen in?” Is that okay? And I think that the way we normally answer that question is kind of a form of what you might call status quo-ism, which is not satisfying. It’s whatever has been the case is—

Mark Zuckerberg: Yeah, yeah.

Jonathan Zittrain: –whatever has been the case is what should stay the case.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: And, so, for WhatsApp, it’s like right now WhatsApp, as I understand it, you could correct me if I’m wrong, is pretty hard to get into if–

Mark Zuckerberg: It’s fully end-to-end encrypted.

Jonathan Zittrain: Right. So, if Facebook gets handed a subpoena or a warrant or something from name-your-favorite-country–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: –and you’re just like, “Thank you for playing. We have nothing to–”

Mark Zuckerberg: Oh, yeah, we’ve had employees thrown in jail because we have gotten court orders that we have to turnover data that we wouldn’t probably anyway, but we can’t because it’s encrypted.

Jonathan Zittrain: Yes. And then, on the other hand, and this is not as clean as it could be in theory, but Messenger is sometimes encrypted, sometimes not. If it doesn’t happen to have been encrypted by the users, then that subpoena could work and, more than that, there could start to be some automated systems either on Facebook’s own initiative or under pressure from governments in the general case, not a specific warrant, to say, “Hey, if the following phrases appear, if there’s some telltale that says, “This is somebody going after a kid for exploitation,” it should be forwarded up. If that’s already happening and we can produce x-number of people who have been identified and a number of crimes averted that way, who wants to be the person to be like, “Lock it down!” Like, “We don’t want any more of that!” But I guess, to put myself now to your question, when I look out over years rather than just weeks or months, the ability to casually peek at any conversation going on between two people or among a small group of people or even to have a machine do it for you, so, you can just set your alert list, you know, crudely speaking, and get stuff back, that– it’s always trite to call something Orwellian, but it makes Orwell look like a piker. I mean, it seems like a classic case where you– the next sentence would be “What could possible go wrong?”

Jonathan Zittrain: And we can fill that in! And it does mean, though, I think that we have to confront the fact that if we choose to allow that kind of communication, then there’s going to be crimes unsolved that could’ve been solved. There’s going to be crimes not prevented that could have been prevented. And the only thing that kind of blunts it a little is it is not really all or nothing. The modern surveillance states of note in the world, have a lot of arrows in their quivers. And just being able to darken you door and demand surveillance of a certain kind, that might be a first thing they would go to, but they’ve got a Plan B, and Plan C, and a Plan D. And I guess it really gets to what’s your threat model? If you think everybody is kind of a threat, think about the battles of copyright 15 years ago. Everybody is a potential infringer. All they have to do is fire up Napster, then you’re wanting some massive technical infrastructure to prevent the bad thing. If what you’re thinking is instead, they are a few really bad apples and they tend to– when they congregate online or otherwise with one another– tend to identify themselves and then we might have to send somebody near their house to listen with a cup at the window, metaphorically speaking. That’s a different threat model and [sic] might not need it.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: Is that getting to an answer to your question?

Mark Zuckerberg: Yeah, and I think I generally agree. I mean, I’ve already said publically that my inclination is to move these services in the direction of being all encrypted, at least the private communication version. I basically think if you want to kind of talk in metaphors, messaging is like people’s living room, right? And I think we– you know, we definitely don’t want a society where there’s a camera in everyone’s living room watching the content of those conversations.

Jonathan Zittrain: Even as we’re now– I mean, it is 2019, people are happily are putting cameras in their living rooms.

Mark Zuckerberg: That’s their choice, but I guess they’re putting cameras in their living rooms, well, for a number of reasons, but–

Jonathan Zittrain: And Facebook has a camera that you can go into your living room- Mark Zuckerberg: That is, I guess–

Jonathan Zittrain: I just want to be clear.

Mark Zuckerberg: Yeah, although that would be encrypted in this world.

Jonathan Zittrain: Encrypted between you and Facebook!

Mark Zuckerberg: No, no, no. I think– but it also–

Jonathan Zittrain: Doesn’t it have like a little Alexa functionality, too?

Mark Zuckerberg: Well, Portal works over Messenger. So, if we go towards encryption on Messenger, then that’ll be fully encrypted, which I think, frankly, is probably what people want.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: The other model, beside the living room is the town square and that, I think, just has different social norms and different policies and norms that should be at play around that. But I do think that these things are very different. Right? You’re not going to– you may end up in a world where the town square is a fully decentralized or fully encrypted thing, but it’s not clear what value there is in encrypting something that’s public content anyway, or very broad.

Jonathan Zittrain: But, now, you were put to it pretty hard in that as I understand it there’s now a change to how WhatsApp works, that there’s only five forwards permitted.

Mark Zuckerberg: Yeah, so, this is a really interesting point, right? So, when people talk about how encryption will darken some of the signals that we’ll be able to use, you know, both for potentially providing better services and for preventing harm. One of the– I guess, somewhat surprising to me, findings of the last couple of years of working on content governance and enforcement is that it often is much more effective to identify fake accounts and bad actors upstream of them doing something bad by patterns of activity rather than looking at the content.

Jonathan Zittrain: So-called meta data.

Mark Zuckerberg: Sure.

Jonathan Zittrain: “I don’t know what they’re saying, but here’s who’s they’re calling” kind of thing.

Mark Zuckerberg: Yeah, or just like they– this account doesn’t seem to really act like a person, right?

And I guess as AI gets more advanced and you build these adversarial networks or generalized adversarial networks, you’ll get to a place where you have Ai that can probably more effectively

Jonathan Zittrain: Go under mimic [ph?] cover. Mimic act like another person–

Mark Zuckerberg: –for a while.

Mark Zuckerberg: Yeah. But, at the same time, you’ll be building up contrary AI on the other side, but is better at identifying AIs that are doing that. But this has certainly been the most effective tactic across a lot of the areas where we’ve needed to focus to preventing harm. You know, the ability to identify fake accounts, which, like, a huge amount of the– under any category of issue that you’re talking about, a lot of the issues downstream come from fake accounts or people who are clearly acting in some malicious or not normal way. You can identify a lot of that without necessarily even looking at the content itself. And if you have to look at a piece of content, then in some cases, you’re already late, because the content exists and the activity has already happened. So, that’s one of the things that makes me feel like encryption for these messaging services is really the right direction to go, because you’re– it’s a very proprivacy and per security move to give people that control and assurance and I’m relatively confident that even though you are losing some tools to– on the finding harmful content side of the ledger, I don’t think at the end of the day that those are going to end up being the most important tools

Jonathan Zittrain: Yes.

Mark Zuckerberg: –for finding the most of the–

Jonathan Zittrain: But now connect it up quickly to the five forwards thing.

Mark Zuckerberg: Oh, yeah, sure. So, that gets down to if you’re not operating on a piece of content directly, you need to operate on patterns of behavior in the network. And what we, basically found was there weren’t that many good uses for people forwarding things more than five times except to basically spam or blast stuff off. It was being disproportionately abused. So, you end up thinking about different tactics when you’re not operating on content specifically; you end up thinking about patterns of usage more.

Jonathan Zittrain: Well, spam, I get and that– I’m always in favor of things that reduce spam. However, you could also say the second category was just to spread content. You could have the classic, I don’t know, like Les Mis, or Paul Revere’s ride, or Arab Spring-esque in the romanticized vision of it: “Gosh, this is a way for people to do a tree,” and pass along a message that “you can’t stop the signal,” to use a Joss Whedon reference. You really want to get the word out. This would obviously stop that, too.

Mark Zuckerberg: Yeah, and then I think the question is you’re just weighing whether you want this private communication tool where the vast majority of the use and the reason why it was designed was the vast majority of just one-on-one; there’s a large amount of groups that people communicate into, but it’s a pretty small edge case of people operating this with, like– you have a lot of different groups and you’re trying to organize something and almost hack public content-type or public sharing- type utility into an encrypted space and, again, there I think you start getting into “Is this the living room or is this the town square?” And when people start trying to use tools that are designed for one thing to get around what I think the social norms are for the town square, that’s when I think you probably start to have some issues. This is not– we’re not done addressing these issues. There’s a lot more to think through on this

Jonathan Zittrain: Yeah.

Mark Zuckerberg: –but that’s the general shape of the problem that at least I perceive from the work that we’re doing.

Jonathan Zittrain: Well, without any particular segue, let’s talk about fake news.

Jonathan Zittrain: So, insert your favorite segue here. There’s some choice or at least some decision that gets made to figure out what’s going to be next in my newsfeed when I scroll up a little more.

Mark Zuckerberg: Mm-hm.

Jonathan Zittrain: And in the last conversation bit, we were talking about how much we’re looking at content versus telltales and metadata, things that surround the content.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: For knowing about what that next thing in the newsfeed should be, is it a valid desirable material consideration, do you think, for a platform like Facebook to say is the thing we are about to present true, whatever true means?

Mark Zuckerberg: Well, yes, because, again, getting at trying to serve people, people tell us that they don’t want fake content. Right. I mean, I don’t know anyone who wants fake content. I think the whole issue is, again, who gets to decide. Right. So broadly speaking, I don’t know any individual who would sit there and say, “Yes, please show me things that you know are false and that are fake.” People want good quality content and information. That said, I don’t really think that people want us to be deciding what is true for them and people disagree on what is true. And, like, truth is, I mean, there are different levels of when someone is telling a story, maybe the meta arc is talking about something that is true but the facts that were used in it are wrong in some nuanced way but, like, it speaks to some deeper experience. Well, was that true or not? And do people want that disqualified from to them? I think different people are going to come to different places on this.

Now, so I’ve been very sensitive, which, on, like, we really want to make sure that we’re showing people high quality content and information. We know that people don’t want false information. So we’re building quite advanced systems to be able to– to make sure that we’re emphasizing and showing stuff that is going to be high quality. But the big question is where do you get the signal on what the quality is? So the kind of initial v.1 of this was working with third party fact checkers.

Right, I believe very strongly that people do not want Facebook and that we should not be the arbiters of truth in deciding what is correct for everyone in the society. I think people already generally think that we have too much power in deciding what content is good. I tend to also be concerned about that and we should talk about some of the governance stuff that we’re working on separately to try to make it so that we can bring more independent oversight into that.

Jonathan Zittrain: Yes.

Mark Zuckerberg: But let’s put that in a box for now and just say that with those concerns in mind, I’m definitely not looking to try to take on a lot more in terms of also deciding in addition to enforcing all the content policies, also deciding what is true for everyone in the world. Okay, so v.1 of that is we’re going to work with–

Jonathan Zittrain: Truth experts.

Mark Zuckerberg: We’re working with fact checkers.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: And, and they’re experts and basically, there’s like a whole field of how you go and assess certain content. They’re accredited. People can disagree with the leaning of some of these organizations.

Jonathan Zittrain: Who does accredited fact checkers?

Mark Zuckerberg: The Poynter Institute for Journalism.

Jonathan Zittrain: I should apply for my certification.

Mark Zuckerberg: You may.

Jonathan Zittrain: Okay, good.

Mark Zuckerberg: You’d probably get it, but you have to– You’d have to go through the process.

Mark Zuckerberg: The issue there is there aren’t enough of them, right. So there’s a large content. There’s obviously a lot of information is shared every day and there just aren’t a lot of fact checkers. So then the question is okay, that is probably

Jonathan Zittrain: But the portion– You’re saying the food is good, it’s just the portions are small. But the food is good.

Mark Zuckerberg: I think in general, but so you build systems, which is what we’ve done especially leading up to elections where I think are some of the most fraught times around this where people really are aggressively trying to spread misinformation.

Jonathan Zittrain: Yes.

Mark Zuckerberg: You build systems that prioritize content that seems like it’s going viral because you want to reduce the prevalence of how widespread the stuff gets, so that way the fact checkers have tools to be able to, like, prioritize what they need to go– what they need to go look at. But it’s still getting to a relatively small percent of the content. So I think the real thing that we want to try to get to over time is more of a crowd sourced model where people, it’s not that people are trusting some sort, some basic set of experts who are accredited but are in some kid of lofty institution somewhere else. It’s like do you trust, yeah, like, if you get enough data points from within the community of people reasonably looking at something and assessing it over time, then the question is can you compound that together into something that is a strong enough signal that we can then use that?

Jonathan Zittrain: Kind of in the old school like a slash-dot moderating system

Mark Zuckerberg: Yeah.

Jonathan Zittrain: With only the worry that if the stakes get high enough, somebody wants to Astroturf that.

Mark Zuckerberg: Yes.

Jonathan Zittrain: I’d be–

Mark Zuckerberg: There are a lot of questions here, which is why I’m not sitting here and announcing a new program.

Mark Zuckerberg: But what I’m saying is this is, like,–

Jonathan Zittrain: Yeah,

Mark Zuckerberg: This is the general direction that I think we should be thinking about when we haveand I think that there’s a lot of questions and–

Jonathan Zittrain: Yes.

Mark Zuckerberg: And we’d like to run some tests in this area to see whether this can help out. Which would be upholding the principles which are that we want to stop–

Jonathan Zittrain: Yes.

Mark Zuckerberg: The spread of misinformation.

Jonathan Zittrain: Yes.

Mark Zuckerberg: Knowing that no one wants misinformation. And the other principle, which is that we do not want to be arbiters of truth.

Jonathan Zittrain: Want to be the decider, yes.

Mark Zuckerberg: And I think that that’s the basic– those are the basic contours I think of that, of that problem.

Jonathan Zittrain: So let me run an idea by you that you can process in real time and tell me the eight reasons I have not thought of why this is a terrible idea. And that would be people see something in their Facebook feed. They’re about to share it out because it’s got a kind of outrage factor to it. I think of the classic story from two years ago in The Denver Guardian about “FBI agent suspected in Hilary Clinton email leak implicated in murder-suicide.” I have just uttered fake news.

None of that was true if you clicked through The Denver Guardian. There was just that article. There is Denver Guardian. If you live in Denver, you cannot subscribe. Like, it is unambiguously fake. And it was shared more times than the most shared story during the election season of The Boston Globe. And so

Mark Zuckerberg: So, and this is actually an example, by the way, of where trying to figure out fake accounts is a much simpler solution.

Jonathan Zittrain: Yes.

Mark Zuckerberg: Than trying to down–

Jonathan Zittrain: So if newspaper has one article–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: Wait for ten more before you decide they’re a newspaper.

Mark Zuckerberg: Yeah. Or, you know, I mean, it’s there are any number of systems that you could build to basically detect, “Hey, this is–”

Jonathan Zittrain: A Potemkin.

Mark Zuckerberg: This is a fraudulent thing.

Jonathan Zittrain: Yes.

Mark Zuckerberg: And then you can take that down. And again, that ends up being a much less controversial decision because you’re doing it upstream based on the basis of inauthenticity.

Jonathan Zittrain: Yes.

Mark Zuckerberg: In a system where people are supposed to be their real and represent that they’re their real selves than downstream, trying to say, “Hey, is this true or false?”

Jonathan Zittrain: I made a mistake in giving you the easy case.

Mark Zuckerberg: Okay.

Jonathan Zittrain: So I should have not used that example.

Mark Zuckerberg: Too simple.

Jonathan Zittrain: You’re right and you knocked that one out of the park and, like, Denver Guardian, come up with more articles and be real and then come back and talk to us.

Jonathan Zittrain: So, here’s the harder case which is something that might be in an outlet that is, you know, viewed as legitimate, has a number of users, et cetera. So you can’t use the metadata as easily.

Imagine if somebody as they shared it out could say, “By the way, I want to follow this. I want to learn a little bit more about this.” They click a button that says that. And I also realized when I talked earlier to somebody at Facebook on this that adding a new button to the homepage is, like, everybody’s first idea

Mark Zuckerberg: Oh, yeah.

Jonathan Zittrain: And it’s–

Mark Zuckerberg: But it’s a reasonable thought experiment, even though it would lead to a very bad UI.

Jonathan Zittrain: Fair enough. I understand this is already–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: In the land of fantasy. So they add the button. They say, “I want to follow up on this.”

If enough people are clicking comparatively on the same thing to say, “I want to learn more about this. If anything else develops, let me know, Facebook,” that, then, if I have my pneumatic tube, it then goes to a convened virtually panel of three librarians. We go to the librarians of the nation and the world at public and private libraries across the land who agree to participate in this program. Maybe we set up a little foundation for it that’s endowed permanently and no long connected to whoever endowed it. And those librarians together discuss the piece and they come back with what they would tell a patron if somebody came up to them and said, “I’m about to cite this in my social studies paper. What do you think?” And librarians, like, live for questions like that.

Mark Zuckerberg: Mm-hmm, yeah.

Jonathan Zittrain: They’re like, “Wow. Let us tell you.” And they have a huge fiduciary notion of patron duty that says, “I may disapprove of you even studying this, whatever, but I’m here to serve you, the user.”

Mark Zuckerberg: Yeah.

Jonathan Zittrain: “And I just think you should know, this is why maybe it’s not such a good source.” And when they come up with that they can send it back and it gets pushed out to everybody who asks for follow-up–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: And they can do with it as they will. And last piece of the puzzle, we have high school students who apprentice as librarian number three for credit.

Jonathan Zittrain: And then they can get graded on how well they participated in this exercise which helps generate a new generation of librarian-themed people who are better off at reading things, so.

Mark Zuckerberg: All right, well, I think you have a side goal here which I haven’t been thinking about on the librarian thing.

Mark Zuckerberg: Which is the evil goal of promoting libraries.

Jonathan Zittrain: Well, it’s

Mark Zuckerberg: No, but I mean, look, I think solving– preventing misinformation or spreading misinformation is hard enough without also trying to develop high school students in a direction.

Jonathan Zittrain: Ah. My colleague Charlies Foote–

Mark Zuckerberg: So, that’s solving a problem with a problem.

Jonathan Zittrain: All right. Well, anyway, yes.

Mark Zuckerberg: So I actually think I agree with most of what you have in there. It doesn’t need to be a button on the home page, it can be– I mean, it turns out that there’s so many people using these services that even if you get– even if you put something that looks like it’s not super prominent, like, behind the three dots on a given newsfeed story, you have the options, yeah, you’re not– not everyone is going tois going to like something.

Jonathan Zittrain: If 1 out of 1000 do it, you still get 10,000 or 100,000 people, yeah.

Mark Zuckerberg: You get pretty good signal. But I actually think you could do even better, which is, it’s not even clear that you need that signal. I think that that’s super helpful. I think really what matters is looking at stuff that’s getting a lot of distribution. So, you know, I think that there’s kind of this notion, and I’m going back to the encryption conversation, which is all right, if I say something that’s wrong to you in a one-on-one conversation, I mean, does that need to be fact checked? I mean, it’s, yeah, it would be good if you got the most accurate information.

Jonathan Zittrain: I do have a personal librarian to accompany me for most conversations, yes. There you go.

Mark Zuckerberg: Well, you are–

Jonathan Zittrain: Unusual.

Mark Zuckerberg: Yeah, yeah. Yes.

Mark Zuckerberg: That’s the word I was looking for.

Jonathan Zittrain: I’m not sure I believe you, but yes.

Mark Zuckerberg: It’s– But I think that there’s limited– I don’t think anyone would say that every message that goes back and forth in especially an encrypted messaging service should be

Jonathan Zittrain: Fact checked.

Mark Zuckerberg: Should be fact checked.

Jonathan Zittrain: Correct.

Mark Zuckerberg: So I think the real question is all right, when something starts going viral or getting a lot of distribution, that’s when it becomes most socially important for it to be– have some level of validation or at least that we know where that the community in general thinks that this is a reasonable thing. So it’s actually, while it’s helpful to have the signal of whether people are flagging this as something that we should look at, I actually think increasingly you want to be designing systems that just prevent like alarming or sensational content from going viral in the first place. And making sure that that, that the stuff that is getting wide distribution is doing so because it’s high quality on whatever front you care about. So then, okay–

Jonathan Zittrain: And that quality is still generally from Poynter or some external party that

Mark Zuckerberg: Well, well quality has many dimensions.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: But certainly accuracy is one dimension of it. You also, I mean, you pointed out I think in one of your questions, is this piece of content prone to incite outrage. If you don’t mind, I’ll get to your panel of three things in a second, but as a slight detour on this.

Jonathan Zittrain: Yes.

Mark Zuckerberg: One of the findings that has been quite interesting is, you know, there’s this question about whether social media in general increases, basically makes it so that sensationalist content gets the most distribution. And what we’ve found is that, all right, so we’re going to have rules, right, about what content is allowed. And what we found is that generally within whatever rules you set up, as content approaches the line of what is allowed, it often gets more distribution. So if you’ll have some rule on, you know, what– And take a completely different example and our nudity policies. Right. It’s like, okay, you have to define what is unacceptable nudity in some way. As you get as close to that as possible it’s like, all right. Like, this is maybe a photo of someone–

Jonathan Zittrain: The skin to share ratio goes up until it gets banned at which point it goes to zero.

Mark Zuckerberg: Yes. Okay. So that is a bad property of a system, right, that I think you want to generally address. Or you don’t want to design a community where or systems for helping to build a community where things that get as close to the line as what is bad get the most distribution.

Jonathan Zittrain: So long as we have the premise, which in many cases is true, but I could probably try to think of some where it wouldn’t be true, that as you near the line, you are getting worse.

Mark Zuckerberg: That’s a good point. That’s a good point. There’s–

Jonathan Zittrain: You know, there might be humor that’s really edgy.

Mark Zuckerberg: That’s true.

Jonathan Zittrain: And that conveys a message that would be impossible to convey without the edginess, while not still–

Mark Zuckerberg: That is–

Jonathan Zittrain: But, I–

Mark Zuckerberg: That’s true.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: So but then you get the question of what’s the cost benefit of allowing that. And obviously, where you can accurately separate what’s good and bad which you, like in the case of misinformation I’m not sure you could do it fully accurately, but you can try to build systems that approximate that, there’s certainly the issue, which is that, I mean, there is misinformation which leads to massive public harm, right. So if it’s misinformation that is also spreading hate and leading to genocide or public attacks or, it’s like, okay, we’re not going to allow that. Right. That’s coming down. But then generally if you say something that’s wrong, we’re not going to try to block that.

Jonathan Zittrain: Yes.

Mark Zuckerberg: We’re just going to try to not show it to people widely because people don’t want content that is wrong. So then the question is as something is approaching the line, how do you assess that? This is a general theme in a lot of the content governance and enforcement work that we’re doing, which is there’s one piece of this which is just making sure that we can as effectively as possible enforce the policies that exist. Then there’s a whole other stream of work, which I called borderline content, which is basically this issue of as content approaches the line of being against the policies, how do you make sure that that isn’t the content that is somehow getting the most distribution? And a lot of the things that we’ve done in the last year were focused on that problem and it really improves the quality of the service and people appreciate that.

Jonathan Zittrain: So this idea would be stuff that you’re kind of letting down easy without banning and letting down easy as it’s going to somehow have a coefficient of friction for sharing that goes up. It’s going to be harder–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: For it to go viral.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: And–

Mark Zuckerberg: So it’s fascinating because it’s just against– Like, you can take almost any category of policy that we have, so I used nudity a second ago. You know, gore and violent imagery.

Jonathan Zittrain: Yes.

Mark Zuckerberg: Hate speech.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: Any of these things. I mean, there’s, like, hate speech, there’s content that you would just say is mean or toxic, but that did not violate– But that you would not want to have a society that banned being able to say that thing. But it’s, but you don’t necessarily want that to be the content that is getting the most distribution.

Jonathan Zittrain: So here’s a classic transparency question around exactly that system you described.

And when you described this, I think you did a post around this a few months ago. This was fascinating.

You had graphs in the post depicting this, which was great. How would you feel about sharing back to the person who posted or possibly to everybody who encounters it its coefficient of friction? Would that freak people out? Would it be, like, all right, I– And in fact, they would then probably start conforming their posts, for better or worse,–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: To try to maximize the sharability. But that rating is already somewhere in there by design. Would it be okay to surface it?

Mark Zuckerberg: So, as a principle, I think that that would be good, but I don’t– The way that the systems are designed isn’t that you get a score of how inflammatory or sensationalist a piece of content is. The way that it basically works is you can build classifiers that identify specific types of things. Right.

So we’re going down the list of, like, all right, there’s 20 categories of harmful content that you’re trying to identify. You know, everything from terrorist propaganda on the one hand to self-harm issues to hate speech and election interference. And basically, each of these things while it uses a lot of the same underlying machine learning infrastructure, you’re doing specific work for each of them. So if you go back to the example on Nudity for a second, you know, what you– you’re not necessarily scoring everything on a scale of not at all nude to nude. You[‘re basically enforcing specific policies. So, you know, you’re saying, “Okay, if–”

Jonathan Zittrain: So by machine learning it would just be give me an estimate of the odds by which if a human looked at it who was employed to enforce policy–

Mark Zuckerberg: Well, basically–

Jonathan Zittrain: Whether it violates the policy.

Mark Zuckerberg: And you have a sense of, okay, this is– So what are the things that are adjacent to the policy, right? So you night say, okay, well, if the person is completely naked, that is something that you can definitely build a classifier to be able to identify with relatively high accuracy. But even if they’re not, you know, then the question is you kind of need to be able to qualitatively describe what are the things that are adjacent to that. So maybe the person is wearing a bathing suit and is in a sexually suggestive position. Right. It’s not like any piece of content you’re going to score from not at all nude to nude. But you kind of have the cases for what you think are adjacent to the issues and, and again, you ground this and qualitatively, people, like, people might click on it, they might engage with it, but at the end, they don’t necessarily feel good about it. And you want to get at when you’re designing these systems not just what people do, but also you want to make sure we factor in, too, like is this the content that people say that they really want to be seeing? Do they–?

Jonathan Zittrain: The constitutional law, there’s a formal kind of definition that’s emerged for the word “prurient.” If something appeals to the prurient interest–

Mark Zuckerberg: Okay.

Jonathan Zittrain: As part of a definition of obscenity, the famous Miller test, which was not a beeroriented test. And part of a prurient interest is basically it excites me and yet it completely disgusts me.

And it sounds like you’re actually converging to the Supreme Court’s vision of prurience with this.

Mark Zuckerberg: Maybe.

Jonathan Zittrain: And it might be– Don’t worry, I’m not trying to nail you down on that. But it’s very interesting that machine learning, which you invoked, is both really good, I gather, at something like this.

It’s the kind of thing that’s like just have some people tell me with their expertise, does this come near to violating the policy or not and I’ll just through a Spidey sense start to tell you whether it would.

Mark Zuckerberg: Mm-hmm.

Jonathan Zittrain: Rather than being able to throw out exactly what the factors are. I know the person’s fully clothed, but it still is going to invoke that quality. So all of the benefits of machine learning and all of, of course, all the drawbacks where it classifies something and somebody’s like, “Wait a minute. That was me doing a parody of blah, blah, blah.” That all comes to the fore.

Mark Zuckerberg: Yeah and I mean, when you ask people what they want to see in addition to looking at what they actually engage with, you do get a completely different sense of what people value and you can build systems that approximate that. But going back to your question, I think rather than giving people a score of the friction–

Jonathan Zittrain: Yes.

Mark Zuckerberg: I think you can probably give people feedback of, “Hey, this might make people uncomfortable in this way, in this specific way.” And this fits your–

Jonathan Zittrain: It might affect how much it gets– how much it gets shared.

Mark Zuckerberg: Yeah. And this gets down to a different– There’s a different AI ethics question which I think is really important here, which is designing AI systems to be understandable by people

Jonathan Zittrain: Right.

Mark Zuckerberg: Right and to some degree, you don’t just want it to spit out a score of how offensive or, like, where it scores on any given policy. You want it to be able to map to specific things that might be problematic.

Jonathan Zittrain: Yes.

Mark Zuckerberg: And that’s the way that we’re trying to design the systems overall.

Jonathan Zittrain: Yes. Now we have something parked in the box we should take out, which is the external review stuff. But before we do, one other just transparency thing maybe to broach. It basically just occurred to me, I imagine it might be possible to issue me a score of how much I’ve earned for Facebook this year. It could simply say, “This is how much we collected on the basis of you in particular being exposed to an ad.” And I know sometimes people, I guess, might compete to get their numbers up. But I’m just curious, would that be a figure? I’d kind of be curious to know, in part because it might even lay the groundwork of being like, “Look, Mark, I’ll double it. You can have double the money and then don’t show me any ads.” Can we get a car off of that lot today?

Mark Zuckerberg: Okay, well, there’s a lot–

Mark Zuckerberg: There’s a lot in there.

Jonathan Zittrain: It was a quick question.

Mark Zuckerberg: So there’s a question in what you’re saying which is so we built an ad-supported system. Should we have an option for people to pay to not see ads.

Jonathan Zittrain: Right.

Mark Zuckerberg: I think is kind of what you’re saying. I mean, just as the basic primer from first principles on this. You know, we’re building this service. We want to give everyone a voice. We want everyone to be able to connect with who they care about. If you’re trying to build a service for everyone,

Jonathan Zittrain: Got to be free. That’s just

Mark Zuckerberg: If you want them to use it, that’s just going to be the argument. Yes, yes.

Jonathan Zittrain: Okay. All right.

Mark Zuckerberg: So then, so this is a kind of a tried and true thing. There are a lot of companies over time that have been ad supported. In general what we find is that if people are going to see ads, they want them to be relevant. They don’t want them to be junk. Right. So then within that you give people control over how their data is used to show them ads. But the vast majority of people say, like, show me the most relevant ads that you can because I get that I have to see ads. This is a free service. So now the question is, all right, so there’s a whole set of questions around that that we could get into, but but then

Jonathan Zittrain: For which we did talk about enough to reopen it, the personalization exploitation.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: Or even just philosophical question. Right now, Uber or Lyft are not funded that way.

We could apply this ad model to Uber or Lyft, “Free rides. Totally free. It’s just every fifth ride takes you to Wendy’s and idles outside the drive through window.”

Jonathan Zittrain: “Totally up to you what you want to do, but you’re going to sit here for a while,” and then you go on your way. I don’t know how– and status quo-ism would probably say people would have a problem with that, but it would give people rides that otherwise wouldn’t get rides.

Mark Zuckerberg: I have not thought about that case in their–

Mark Zuckerberg: In their business, so, so–

Jonathan Zittrain: Well, that’s my patent, damn it, so don’t you steal it.

Mark Zuckerberg: But certainly some services, I think tend themselves better towards being ad supported than others.

Jonathan Zittrain: Okay.

Mark Zuckerberg: Okay and I think generally information-based ones tend to–

Jonathan Zittrain: Than my false imprisonment hypo, I’d– Okay, fair enough.

Mark Zuckerberg: I mean, that seems

Jonathan Zittrain: Yeah.

Mark Zuckerberg: There might be, you know, more– more issues there. But okay, but go to the subscription thing.

Jonathan Zittrain: Yes.

Mark Zuckerberg: When people have questions about the ad model on Facebook, I don’t think the questions are just about the ad model, I think they’re about both seeing ads and data use around ads.

And the thing that I think, so when I think about this it’s, I don’t just think you want to let people pay to not see ads because I actually think then the question is the questions are around ads and data use and I don’t think people are going to be that psyched about not seeing ads but then not having different controls over how their data is used. Okay, but now you start getting into a principle question which is are we going to let people pay to have different controls on data use than other people. And my answer to that is a hard no, right. So the prerequisite–

Jonathan Zittrain: What’s an example of data use that isn’t ad-based, just so we know what we’re talking about?

Mark Zuckerberg: That isn’t ad-based?

Jonathan Zittrain: Yeah.

Mark Zuckerberg: Like what do you mean?

Jonathan Zittrain: You were saying, I don’t want to see ads. But you’re saying that’s kind of just the wax on the car. What’s underneath is how the data gets used.

Mark Zuckerberg: So, well, look– Maybe– let me keep going with this explanation and then I think this’ll be clear.

Jonathan Zittrain: Yeah, sure.

Mark Zuckerberg: So one of the things that we’ve been working on is this tool that we call clear history. And the basic idea is it is you can kind of analogize it to a web browser where you can clear your cookies. That’s kind of a normal thing. You know that when you clear your cookies you’re going to get logged out of a bunch of stuff. A bunch of stuff might get more annoying.

Jonathan Zittrain: Which is why my guess is, am I right, probably nobody clears their cookies.

Mark Zuckerberg: I don’t know.

Jonathan Zittrain: They might use incognito mode or something, but.

Mark Zuckerberg: I think– I don’t know. How many of you guys clear your cookies every once in a while, right?

Jonathan Zittrain: This is not a representative group, damn it.

Mark Zuckerberg: Okay. Like, maybe once a year or something I’ll clear my cookies.

Jonathan Zittrain:

Mark Zuckerberg: But no, it’s, I think–

Jonathan Zittrain: Happy New Year.

Mark Zuckerberg: No, over some period of time, all right, but–

Jonathan Zittrain: Yeah, okay.

Mark Zuckerberg: But not necessarily every day. But it’s important that people have that tool even though it might in a local sense make their experience worse.

Jonathan Zittrain: Yes.

Mark Zuckerberg: Okay. So that kind of content of what different services, websites and apps send to Facebook that, you know, we use to help measure the ads in effectiveness there, right, so things like, you know, if you’re an app developer and you’re trying to pay for ads to help grow your app, we want to only charge you when we actually, when something that we show leads to an install, not just whether someone sees the ad or clicks on it, but if they add–

Jonathan Zittrain: That requires a whole infrastructure to, yeah.

Mark Zuckerberg: Okay, so then, yeah, so you build that out. It helps us show people more relevant ads.

It can help show more relevant content. Often a lot of these signals are super useful also on the security side for some of the other things that we’ve talked about, so that ends up being important. But fundamentally, you know, looking at the model today, it seems like you should have something like this ability to clear history. It turns out that it’s a much more complex technical project. I’d talked about this at our developer conference last year, about how I’d hoped that we’d roll it out by the end of 2018 and just, the plumbing goes so deep into all the different systems that it’s, that– But we’re still working on it and we’re going to do it. It’s just it’s taking a little bit longer.

Jonathan Zittrain: So clear history basically means I am as if a newb, I just show

Mark Zuckerberg: Yes.

Jonathan Zittrain: Even though I’ve been using Facebook for a while, it’s as if it knows nothing about me and it starts accreting again.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: And I’m just trying to think just as a plain old citizen, how would I make an informed judgment about how often to do that or when I should do it? What–?

Mark Zuckerberg: Well, hold on. Let’s go to that in a second.

Jonathan Zittrain: Okay.

Mark Zuckerberg: But one thing, just to connect the dots on the last conversation.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: Clear history is a prerequisite, I think, for being able to do anything like subscriptions.

Right. Because, like, partially what someone would want to do if they were going to really actually pay for a not ad supported version where their data wasn’t being used in a system like that, you would want to have a control so that Facebook didn’t have access or wasn’t using that data or associating it with your account. And as a principled matter, we are not going to just offer a control like that to people who pay.

Right. That’s going to, if we’re going to give controls over data use, we’re going to do that for everyone in the community. So that’s the first thing that I think we need to go do.

Mark Zuckerberg: So that’s, so that’s kind of– This is sort of the how we’re thinking about the projects and this is a really deep and big technical project but we’re committed to doing it because I think it’s that’s what it’s there for. [ph?] +++

Jonathan Zittrain: And I guess like an ad block or somebody could then write a little script for your browser that would just clear your history every time you visit or something.

Mark Zuckerberg: Oh, yeah, no, but the plan would also be to offer something that’s an ongoing thing.

Jonathan Zittrain: I see.

Mark Zuckerberg: In your browser, but I think the analogy here is you kind of have, in your browser you have the ability to clear your cookies. And then, like, in some other place you have under your, like, nuclear settings, like, don’t ever accept any cookies in my browser. And it’s like, all right, your browser’s not really going to work that well.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: But, but you can do that if you want because you should have that control. I think that these are part and parcel, right. It’s I think a lot of people might go and clear their history on a periodic basis because they– Or, or actually in the research that we’ve done on this as we’ve been developing it, the real thing that people have told us that they want is similar to cookie management, not necessarily wiping everything, because that ends in inconvenience of getting logged out of a bunch of things, but there are just certain services or apps that you don’t want that data to be connected to your Facebook account. So having the ability on an ad hoc basis to go through and say, “Hey, stop associating this thing,” is going to end up being a quite important thing that I think we want to try to deliver. So that’s, this is partially as we’re getting into this, it’s a more complex thing but I think it’s very valuable. And I think if any conversation around the– around subscriptions, I think you would want to start with giving people these, make sure that everyone has these kind of controls. So that’s, we’re kind of in the early phases of doing that. The philosophical downstream question of whether you also let people pay to not have ads, I don’t know. There were a bunch of questions around whether that’s actually a good thing, but I personally don’t believe that very many people would like to pay to not have ads. That all of the research that we have, it’s it may still end up being the right thing to offer that as a choice down the line, but all of the data that I’ve seen suggests that the vast, vast, vast majority of people want a free service and that the ads, in a lot of places are not even that different from the organic content in terms of the quality of what people are being able to see.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: People like being able to get information from local businesses and things like that too, so. So there’s a lot of good there.

Jonathan Zittrain: Yeah. Forty years ago it would have been the question of ABC versus HBO and the answer turned out to be yes.

Jonathan Zittrain: So you’re right. And people might have different things.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: There’s a little paradox lingering in there about if something’s so important and vital that we wouldn’t want to deprive anybody of access to it but therefore nobody gets it until we figured out how to remove it for everybody.

Mark Zuckerberg: What we– [ph?] +++

Jonathan Zittrain: In other words, if I could buy my way out of ads and data collection it wouldn’t be fair to those who can’t and therefore we all subsist with it until the advances you’re talking about.

Mark Zuckerberg: Yeah, but I guess what I’m saying is on the data use, I don’t believe that that’s something that people should buy. I think the data principles that we have need to be uniformly available to everyone. That to me is a really important principle. It’s, like, maybe you could have a conversation about whether you should be able to pay and not see ads. That doesn’t feel like a moral question to me.

Jonathan Zittrain: Yes.

Mark Zuckerberg: But the question of whether you can pay to have different privacy controls feels wrong. So that to me is something that in any conversation about whether we’d evolve towards having a subscription service, I think you have to have these controls first and it’s a very deep thing. A technical problem to go do, but we’re– that’s why we’re working through that.

Jonathan Zittrain: Yes. So long as the privacy controls that we’re not able to buy our way into aren’t controls that people ought to have. You know, it’s just the kind of underlying question of is the system as it is that we can’t opt out of a fair system. And that’s of course, you know, you have to go into the details to figure out what you mean by it. But let’s in the remaining time we have left

Mark Zuckerberg: How are we doing on time?

Jonathan Zittrain: We’re good. We’re 76 minutes in.

Mark Zuckerberg: All right, into–

Mark Zuckerberg: We’re going to get through maybe half the topics.

Jonathan Zittrain: Yeah, yeah, yeah.

Mark Zuckerberg: And I’ll come back and do another one later.

Jonathan Zittrain: I’m going to bring this in for a landing soon. On my agenda left includes such things as taking out of the box the independent review stuff, chat a little bit about that. I’d be curious, and this might be a nice thing, really, as we wrap up, which would be a sense of any vision you have for what would Facebook look like in 10 or 15 years and how different would it be than the Facebook of 10 years ago is compared to today. So that’s something I’d want to talk about. Is there anything big on your list that you want to make sure we talk about?

Mark Zuckerberg: Those are good. Those are good topics.

Jonathan Zittrain: Fair enough.

Mark Zuckerberg:

Jonathan Zittrain: So all right, the external review board.

Mark Zuckerberg: Yeah. So one of the big questions that I have just been thinking about is, you know, we make a lot of decisions around content enforcement and what stays up and what comes down. And having gone through this process over the last few years of working on the systems, one of the themes that I feel really strongly about is that we shouldn’t be making so many of these decisions ourselves. You know, one of the ways that I try to reason about this stuff is take myself out of the position of being CEO of the company, almost like a Rawlsian perspective. If I was a different person, what would I want the CEO of the company to be able to do? And I would not want so many decisions about content to be concentrated with any individual. So–

Jonathan Zittrain: It is weird to see big impactful, to use a terrible word, decisions about what a huge swath of humanity does or doesn’t see inevitably handled as, like, a customer service issue. It does feel like a mismatch, which is what I hear you saying.

Mark Zuckerberg: So let’s, yeah, so I actually think the customer service analogy is a really interesting one. Right. So when you email Amazon, because they don’t, they make a mistake with your package, that’s customer support. Right. I mean, they are trying to provide a service and generally, they can invest more in customer support and make people happier. We’re doing something completely different, right.

When someone emails us with an issue or flags some content, they’re basically complaining about something that someone else in the community did. So it’s more like it’s almost more like a court system in that sense. Doing more of that does not make people happy because in every one of those transactions one person ends up the winner and one is the loser. Either you said that that content, that the content was fine, in which case the person complaining is upset, or you the someone’s content down, in which case the person is really upset because you’re now telling them that they don’t have the ability to express something that they feel is a valid thing that they should be able to express.

So in some deep sense while some amount of what we do is customer support, people get locked out of their account, et cetera, you know, we now have, like, more than 30,000 people working on content review and safety review, doing the kind of judgments that, you know, it’s basically a lot of the stuff, we have machine learning systems that flag things that could be problematic in addition to people in the community flagging things, but making these assessments of whether the stuff is right or not. So one of the questions that I just think about, it’s like, okay, well, you have many people doing this.

Regardless of how much training they have, we’re going to make mistakes, right. So you want to start building in principles around, you know, what you would kind of think of as due process, right. So we’re building in an ability to have an appeal, right, which already is quite good in that we are able to overturn a bunch of mistakes that the first line people make in making these assessments. But at some level I think you also want a level of kind of independent appeal, right, where if, okay, let’s say, so the appeals go to maybe a higher level of Facebook employee who is a little more trained in the nuances of the policies; but at some point, I think you also need an appeal to an independent groups, which is, like, is this policy fair? Was this–? Like is this piece of content really getting on the wrong side of the balance of free expression and safety? And I just don’t think at the end of the day that that’s something that you want centralized in a single company. So now the question is how do you design that system and that’s a real question, right, so that we don’t pretend to have the answers on this. What we’re basically working through is we have a draft proposal and we’re working with a lot of experts around the world to run a few pilots in the first half of this year that can hopefully we can codify into something that’s a longer term thing. But I just, I believe that this is just an incredibly important thing. As a person and if I take aside the role that I have as CEO of the company, I do not want the company being able to make all of those final decisions without a check and balance and accountability, so I want to use the position that I’m in to help build that kind of an institution.

Jonathan Zittrain: Yes. And when we talk about an appeal, then, it sounds like you could appeal two distinct things. One is this was the rule but it was applied wrong to me. This, in fact, was parody [ph?] so it shouldn’t be seen as near the line.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: And I want the independent body to look at that. The other would be the rule is wrong. The rule should change because–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: And you’re thinking the independent body could weigh in on both of those?

Mark Zuckerberg: Yeah. Over time, I would like the role of the independent oversight board to be able to expand to do additional things as well. I think the question is it’s hard enough to even set something up that’s going to codify the values that we have around expression and safety on a relatively defined topic.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: So I think the question is if you kind of view this as an experiment in institution building where we’re trying to build this thing that is going to have real power toJonathan Zittrain: Yes.

Mark Zuckerberg: I mean, like, I will not be able to make a decision that overturns what they say. Which I think is good. I think also just it raises the stakes. You need to make sure we get this right, so.

Jonathan Zittrain: It’s fascinating. It’s huge. I think the way you’re describing it, I wouldn’t want to understate–

Mark Zuckerberg: Yeah.

Jonathan Zittrain: That this is not a usual way of doing business.

Mark Zuckerberg: Yeah, but I think it– I think this is– I really care about getting this right.

Jonathan Zittrain: Yeah.

Mark Zuckerberg: But I think you want to start with something that’s relatively well-defined and then hopefully expand it to be able to cover more things over time. So in the beginning I think one question that could come up is my understanding, I mean, it’s always dangerous talking about legal precedence when I’m, this might be one of my first times at Harvard Law School. I did not spend a lot of time here

Mark Zuckerberg: When I was an undergrad. But, you know what I mean, the, if the Supreme Court overturns something, they don’t tell Congress what the law should be, they just say there’s an issue here, right. And then basically there’s a process. All right. So if I’m getting that wrong

Mark Zuckerberg: All right. I shouldn’t have done that.

Jonathan Zittrain: No, no. That’s quite honest. [ph?]

Mark Zuckerberg: I knew that was dangerous.

Mark Zuckerberg: And that that was a mistake.

Jonathan Zittrain: There are people who do agree with you.

Mark Zuckerberg: Okay. Oh, so that’s an open question that that’s how it works.

Jonathan Zittrain: It’s a highly debated question, yes.

Mark Zuckerberg: All right.

Jonathan Zittrain: There’s the I’m just the umpire calling balls and strikes and in fact, the first type of question we brought up, which was, “Hey, we get this is the standard. Does it apply here?” lends itself a little more to, you know, you get three swings and if you miss them all, like, you can’t keep playing. The umpire can usher you away from the home plate. This is, I’m really digging deep into my knowledge now of baseball. There’s another thing about, like,–

Mark Zuckerberg: That’s okay. I’m not the person who’s going to call you out on getting something wrong there.

Jonathan Zittrain: I appreciate that.

Mark Zuckerberg: That’s why I also need to have a librarian next to me at all times.

Jonathan Zittrain: Very good. I wonder how much librarians tend to know about baseball.

Mark Zuckerberg: Aww.

Jonathan Zittrain: But we digress. Ah, we’re going to get letters, mentions.

Mark Zuckerberg: Yeah.

Jonathan Zittrain: But whether or not the game is actually any good with a three strikes rule, maybe there should be two or four or whatever, starts to ask of the umpire more than just, you know, your best sense of how that play just went. Both may be something. Both are surely beyond standard customer service issues, so both could maybe be usefully externalized. What you’d ask the board to do in the category one kind of stuff maybe it’s true that, like, professional umpirage [ph?] could help us and there are people who are jurists who can do that worldwide. For the other, whether it’s the Supreme

Jonathan Zittrain: –court, or the so-called common law and state courts where often a state supreme court will be like, “Henceforth, 50 feet needs to be the height of a baseball net,” and like, “If you don’t agree, Legislature, we’ll hear from you, but until then it’s 50 feet.” They really do kind of get into the weeds. They derive maybe some legitimacy for decisions like that from being close to their communities, and it really regresses them to a question of: Is Facebook a global community, a community of 2.X billion people worldwide, transcending any national boundaries, and for which I think so far on these issues, it’s meant to be, “The rule is the rule,” it doesn’t really change in terms of service from one place to anotherversus how much do we think of it as somehow localized– whether or not localized through governmentbut where different local communities make their own judgments?

Mark Zuckerberg: That is one of the big questions. I mean, right now we have community standards that are global. We follow local laws, as you say. But I think the idea is– I don’t think we want to end up in a place where we have very different norms in different places, but you want to have some sense of representation and making sure that the body that can deliberate on this has a good diversity of views. So these are a lot of the things that we’re trying to figure out, is like: Well, how big is the body? When decisions are made, are they made by the whole body, or do you have panels of people that are smaller sets? If there are panels, how do you make sure that you’re not just getting a random sample that kind of skews in the values perspective towards one thing? So then there a bunch of mechanisms like, okay, maybe one panel that’s randomly constituted decides on whether the board will take up a question or one of the issues, but then a separate random panel of the group actually does the decisions, so that way you eliminate some risk that any given panel is going to be too ideologically skewed. So there’s a bunch of things that I think we need to think through and work through, but the goal on this is to, over time, have it grow into something that can provide greater accountability and oversight to potentially more of the hard questions that we face, but I think it’s so high-stakes that starting with something that’s relatively defined is going to be the right way to go in the beginning. So regardless of the fact that I was unaware of the controversy around the legal point that I made a second ago, I do think in our case it makes sense to start with not having this group say what the policies are going to be, but just have there be– have it be able to say, “Hey, we think that you guys are on the wrong side on this, and maybe you should rethink where the policy is because we think you’re on the wrong side.” There’s one other thing that I think is worth calling out, which is in a typical kind of judicial analog, or at least here in the U.S., my understanding, is there’s the kind of appeal route to the independent board considering an issue, but I also think that we want to have an avenue where we as the company can also just raise hard issues that come up to the board without having– which I don’t actually know if there’s any mechanism for that.

Jonathan Zittrain: It’s called an advisory opinion.

Jonathan Zittrain: But under U.S. federal law, it’s not allowed because of Article III Case or Controversy requirement, but state courts do it all the time. You’ll have a federal court sometimes say– because it’s a federal court but it’s deciding something under state law. It’ll be like, “I don’t know, ask Florida.” And they’ll be like, “Hey Florida,” and then Florida is just Florida.

Mark Zuckerberg: Sure. So I think that–

Jonathan Zittrain: So you can do an advisory opinion.

Mark Zuckerberg: –that’ll end up being an important part of this too. We’re never going to be able to get out of the business of making frontline judgments. We’ll have the AI systems flag content that they think is against policies or could be, and then we’ll have people– this set of 30 thousand people, which is growing– that is trained to basically understand what the policies are. We have to make the frontline decisions, because a lot of this stuff needs to get handled in a timely way, and a more deliberative process that’s thinking about the fairness and the policies overall should happen over a different timeframe than what is often relevant, which is the enforcement of the initial policy. But I do think overall for a lot of the biggest questions, I just want to build a more independent process.

Jonathan Zittrain: Well, as you say, it’s an area with fractal complexity in the best of ways, and it really is terra incognito, and it’d be exciting to see how it might be built out. I imagine there’s a number of law professors around the world, including some who come from civil rather than common law jurisdictions, who are like, “This is how it works over here,” from which you could draw. Another lingering question would be– lawyers often have a bad reputation. I have no idea why. But they often are the glue for a system like this so that a judge does not have to be oracular or omniscient. There’s a process where the lawyer for one side does a ton of work and looks at prior decisions of this board and says, “Well, this is what would be consistent,” and the other lawyer comes back, and then the judge just gets to decide between the two, rather than having to just know everything. There’s a huge tradeoff here for every appealed content decision, how much do we want to build it into a case, and you need experts to help the parties, versus they each just sort of come before Solomon and say, “This kind of happened,” and– or Judge Judy maybe is a more contemporary reference.

Mark Zuckerberg: Somewhere between the two, yeah.

Jonathan Zittrain: Yeah. So it’s a lot of stuff– and for me, I both find myself– I don’t know if this is the definition of prurient– both excited by it and somewhat terrified by it, but very much saying that it’s better than a status quo, which is where I think you and I are completely agreeing, and maybe a model for other firms out there. So that’s the last question in this area that pops to my mind, which is: What part of what you’re developing at Facebook– a lot of which is really resource-intensive– is best thought of as a public good to be shared, including among basically competitors, versus, “That’s part of our comparative advantage and our secret sauce”? If you develop a particularly good algorithm that can really well detect fake news or spammers or bad actors– you’ve got the PhDs, you’ve got the processors– is that like, “In your face, Schmitter [ph?],” or is like, “We should have somebody that– some body– that can help democratize that advance”? And it could be the same to be said for these content decisions. How do you think about that?

Mark Zuckerberg: Yeah, so certainly the threat-sharing and security work that you just referenced is a good area where there’s much better collaboration now than there was historically. I think that that’s just because everyone recognizes that it’s such a more important issue. And by the way, there’s much better collaboration with governments now too on this, and not just our own here in the U.S., and law enforcement, but around the world with election commissions and law enforcement, because there’s just a broad awareness that these are issues and that–

Jonathan Zittrain: Especially if you have state actors in the mix as the adversary.

Mark Zuckerberg: Yes. So that’s certainly an area where there’s much better collaboration now, and that’s good. There’s still issues. For example, if you’re law enforcement or intelligence and you have developed a– “source” is not the right word– but basically if you’ve identified someone as a source of signals that you can watch and learn about, then you may not want to come to us and tell us, “Hey, we’ve identified that this state actor is doing this bad thing,” because then the natural thing that we’re going to want to do is make sure that they’re not on our system doing bad things, or that they’re not– either they’re not in the system at all or that we’re interfering with the bad things that they’re trying to do. So there’s some mismatch of incentives, but as you build up the relationships and trust, you can get to that kind of a relationship where they can also flag for you, “Hey, this is what we’re at.” So I just think having that kind of baseline where you build that up over time is helpful. And I think on security and safety is probably the biggest area of that kind of collaboration now, across all the different types of threats; not just election and democratic process type stuff, but any kind of safety issue. The other area where I tend to think about what we’re doing is– it should be open– is just technical infrastructure overall. I mean, that is probably a less controversial piece, but we open-source a lot of the basic stuff that runs our systems, and I think that that is a– that’s a contribution that I’m quite proud of that we do.

We have sort of pioneered this way of thinking about how people connect, and the data model around that is more of a graph, and the idea of graph database and a lot of the infrastructure for being able to efficiently access that kind of content I think is broadly applicable beyond the context of a social network.

When I was here as an undergrad, even though I wasn’t here for very long, I studied psychology and computer science, and to me– I mean, my grounding philosophy on this stuff is that basically people should be at the center of more of the technology that we build. I mean, one of the early things that I kind of recognized when I was a student was like– at the time, there were internet sites for finding almost anything you cared about, whether it’s books or music or news or information or businesses– but as people, we think about the world primarily in terms of other people, not in terms of other objects, not cutting things up in terms of content or commerce or politics or different things, but it’s like– the stuff should be organized around the connections that people have, where people are at the centerpiece of that, and one of the missions that I care about is over time just pushing more technology development in the tech industry overall to develop things with that mindset. I think– and this is a little bit of a tangentbut the way that our phones work today, and all computing systems, organized around apps and tasks is fundamentally not how people– how our brains work and how we approach the world. It’s not– so that’s one of the reasons why I’m just very excited longer-term about especially things like augmented reality, because it’ll give us a platform that I think actually is how we think about stuff. We’ll be able to bring the computational objects into the world but fundamentally we’ll be interacting as people around them. The whole thing won’t be organized around an app or a task; it’ll be organized around people, and that I think is a much more natural and human system for how our technology should be organized. So opensourcing all of that infrastructure– to do that, and enabling not just us but other companies to kind of get that mindset into more of their thinking and the technical underpinning of that, is just something that I care really deeply about.

Jonathan Zittrain: Well, this is nice, and this is bringing us in for our landing, because we’re talking about 10, 20, 30 years ahead. As a term of art, I understand augmented reality to mean, “I’ve got a visor”version 0.1 was Google Glass– something where I’m kind of out in the world but I’m literally online at the same time because there’s data coming at me in some– that’s what you’re talking about, correct?

Mark Zuckerberg: Yeah, although it really should be glasses like what you have. I think we’ll probablymaybe they’ll have to be a little bigger, but not too much bigger or else it would start to get weird.

Mark Zuckerberg: So I don’t think a visor is going to catch. I don’t think anyone is psyched about that feature.

Jonathan Zittrain: And anything involving surgery starts to sound a little bad too.

Mark Zuckerberg: No, no, we’re definitely focused on–

Mark Zuckerberg: –on external things. Although–

Jonathan Zittrain: Like, “Don’t make news, don’t make news, don’t make news.”

Mark Zuckerberg: No, no, no. Although we have showed this demo of basically can someone type by thinking, and of course when you’re talking about brain-computer interfaces, there’s two dimensions of that work. There’s the external stuff, and there’s the internal stuff, and invasive, and yes, of course if you’re actually trying to build things that everyone is going to use, you’re going to want to focus on the noninvasive things.

Jonathan Zittrain: Yes. Can you type by thinking?

Mark Zuckerberg: You can.

Jonathan Zittrain: It’s called a Ouija Board. No. But you’re subvocalizing enough or there’s enough of a read of–

Mark Zuckerberg: No, no, no. So there’s actually a bunch of the research here– there’s a question of throughput and how quickly can you type and how many bits can you express efficiently, but the basic foundation for the research is someone– a bunch of folks who are doing this research showed a bunch of people images– I think it was animals– so, “Here’s an elephant, here’s a giraffe”– while having kind of a net on their head, noninvasive, but shining light and therefore looking at the level of blood activity andjust blood flow and activity in the brain– trained a machine learning basically on what the pattern of that imagery looked like when the person was looking at different animals, then told the person to think about an animal, right? So think about– just pick one of the animals to think about, and can predict what the person was thinking about in broad strokes just based on matching the neural activity. So the question is, so you can use that to type.

Jonathan Zittrain: Fifth amendment implications are staggering.

Jonathan Zittrain: Sorry.

Mark Zuckerberg: Well, yes. I mean, presumably this would be something that someone would choose to use a product. I’m not– yeah, yeah. I mean, yes, there’s of course all the other implications, but yeah, I think that this is going to be– that’s going to be an interesting thing down the line.

Jonathan Zittrain: But basically your vision then for a future–

Mark Zuckerberg: I don’t know how we got onto that.

Jonathan Zittrain: You can’t blame me. I think you brought this up.

Mark Zuckerberg: I did, but of all the things that– I mean, this is exciting, but we haven’t even covered yet how we should talk about– tech regulation and all this stuff I figured we’d get into. I mean, we’ll be here for like six or seven hours. I don’t know how many days you want to spend here to talking about this, but–

Jonathan Zittrain: “We’re here at the Zuckerberg Center and hostage crisis.”

Jonathan Zittrain: “The building is surrounded.”

Mark Zuckerberg: Yeah. But I think a little bit on future tech and research is interesting too, so.

Jonathan Zittrain: Please.

Mark Zuckerberg: Yeah, we’re good.

Jonathan Zittrain: Oh, we did cover it, is what you’re saying.

Mark Zuckerberg: I mean, but going back to your question about what– if this is the last topic– what I’m excited about for the next 10 or 20 years– I do think over the long term, reshaping our computing platforms to be fundamentally more about people and how we process the world is a really fundamental thing. Over the nearer term– so call it five years– I think the clear trend is towards more private communication. If you look at all of the different ways that people want to share and communicate across the internet– but we have a good sense of the cross-strength, everything from one-on-one messages to kind of broadcasting publicly– the thing that is growing the fastest is private communication. Right?

So between WhatsApp and Messenger, and Instagram now, just the number of private messages– it’s about 100 billion a day through those systems alone, growing very quickly, growing much faster than the amount that people want to share or broadcast into a feed-type system. Of the type of broadcast content that people are doing, the thing that is growing by far the fastest is stories. Right?

So ephemeral sharing of, “I’m going to put this out, but I want to have a timeframe after which the data goes away.” So I think that that just gives you a sense of where the hub of social activity is going. It also is how we think about the strategy of the company. I mean, people– when we talk about privacy, I think a lot of the questions are often about privacy policies and legal or policy-type things, and privacy as a thing not to be breached, and making sure that you’re within the balance of what is good. But I actually think that there’s a much more– there’s another element of this that’s really fundamental, which is that people want tools that give them new contexts to communicate, and that’s also fundamentally about giving people power through privacy, not just not violating privacy, right? So not violating privacy is a backstop, but actually– you can kind of think about all the success that Facebook has had– this is kind of a counterintuitive thing– has been because we’ve given people new private or semi-private ways to communicate things that they wouldn’t have had before.

So thinking about Facebook as an innovator in privacy is certainly not the mainstream view, but going back to the very first thing that we did, making it so Harvard students could communicate in a way that they had some confidence that their content and information would be shared with only people within that community, there was no way that people had to communicate stuff at that scale, but not have it either be completely public or with just a small set of people before. And people’s desire to be understood and express themselves and be able to communicate with all different kinds of groups is, in the experience that I’ve had, nearly unbounded, and if you can give people new ways to be able to communicate safely and express themselves, then that is something that people just have a deep thirst and desire for.

So encryption is really important, because I mean, we take for granted in the U.S. that there’s good rule of law, and that the government isn’t too much in our business, but in a lot of places around the world, especially where WhatsApp is the biggest, people can’t take that for granted. So having it so that you really have confidence that you’re sharing something one-on-one and it’s not– and it really is one-on-one, it’s not one-on-one and the government there– actually makes it so people can share things that they wouldn’t be comfortable otherwise doing it. That’s power that you’re giving people through building privacy innovations.

Stories I just think is another example of this, where there are a lot of things that people don’t want as part of the permanent record but want to express, and it’s not an accident that that is becoming the primary way that people want to share with all of their friends, not putting something in a feed that goes on their permanent record. There will always be a use for that too– people want to have a record and there’s a lot of value that you can build around that– you can have longer-term discussions– it’s harder to do that around stories. There’s different value for these things. But over the next five years, I think we’re going to see all of social networking kind of be reconstituted around this base of private communication, and that’s something that I’m just very excited about. I think that that’s– it’s going to unlock a lot of people’s ability to express themselves and communicate things that they haven’t had the tools to do before, and it’s going to be the foundation for building a lot of really important tools on top of that too.

Jonathan Zittrain: That’s so interesting to me. I would not have predicted that direction for the next five years. I would have figured, “Gosh, if you already know with whom you want to speak, there are so many tools to speak with them,” some of which are end-to-end, some of which aren’t, some of which are rollyourown and open-source, and there’s always a way to try to make that easier and better, but that feels a little bit to me like a kind of crowded space, not yet knowing of the innovations that might lie ahead and means of communicating with the people you already know you want to talk to. And for that, as you say, if that’s where it’s at, you’re right that encryption is going to be a big question, and otherwise technical design so that if the law comes knocking on the door, what would the company be in a position to say.

This is the Apple iPhone Cupertino– sorry, San Bernardino case– and it also calls to mind will there be peer-to-peer implementations of the things you’re thinking about that might not even need the server at all, and it’s basically just an app that people use, and if it’s going to deliver an ad, it can still do that appside, and how much governments will abide it. They have not, for the most part, demanded technology mandates to reshape how the technology works. They’re just saying, “If you’ve got it”– in part you’ve got it because you want to serve ads– “we want it.” But if you don’t even have it, it’s been rare for the governments to say, “Well, you’ve got to build your system to do it.” It did happen with the telephone system back in the day. CALEA, the Communications Assistance to Law Enforcement Act, did have federal law in the United States saying, “If you’re in the business of building a phone network, AT&T, you’ve got to make it so we can plug in as you go digital,” and we haven’t yet seen those mandates in the internet software side so much. So we can see that coming up again. But it’s so funny, because if you’d asked me, I would have figured it’s encountering people you haven’t met before and interacting with them, for which all of the stuff about air traffic control of what goes into your feed and how much your stuff gets shared– all of those issues start to rise to the fore, and it gets me thinking about, “I ought to be able to make a feed recipe that’s my recipe, and fills it according to Facebook variables, but I get to say what the variables are.” But I could see that if you’re just thinking about people communicating with the people they already know and like, that is a very different realm.

Mark Zuckerberg: It’s not necessarily– it’s not just the people that you already know. I do think– we’ve really focused on friends and family for the last 10 or 15 years, and I think a big part of what we’re going to focus on now is around building communities in different ways and all the utility that you can build on top of, once you have a network like this in place. So everything from how people can do commerce better to things like dating, which is– a lot of dating happens on our services, but we haven’t built any tools specifically for that.

Jonathan Zittrain: I do remember the Facebook joint experiment– “experiment” is such a terrible wordstudy, by which one could predict when two Facebook members are going to declare themselves in a relationship, months ahead of the actual declaration. I was thinking some of the ancillary products were in-laws.

Mark Zuckerberg: That was very early. Yeah. So you’re right that a lot of this is going to be about utility that you can build on top of it, but a lot of these things are fundamentally private, right? So if you’re thinking about commerce, that people have a higher expectation for privacy, and the question is: Is the right context for that going to be around an app like Facebook, which is broad, or an app like Instagram?

I think part of it is– the discovery part of it, I think we’ll be very well served there– but then we’ll also transition to something that people want to be more private and secure. Anyhow, we could probably go on for many hours on this, but maybe we should save this for the Round 2 of this that we’ll do in the future.

Jonathan Zittrain: Indeed. So thanks so much for coming out, for talking at such length, for covering such a kaleidoscopic range of topics, and we look forward to the next time we see you.

Mark Zuckerberg: Yeah. Thanks.

Jonathan Zittrain: Thanks.

Powered by WPeMatico

Facebook now lets everyone unsend messages for 10 minutes

Posted by | Apps, Facebook, facebook messenger, Facebook unsend, Mark Zuckerberg, Mobile, Policy, Social, TC | No Comments

Facebook has finally made good on its promise to let users unsend chats after TechCrunch discovered Mark Zuckerberg had secretly retracted some of his Facebook Messages from recipients. Today Facebook Messenger globally rolls out “Remove for everyone” to help you pull back typos, poor choices, embarrassing thoughts or any other message.

For up to 10 minutes after sending a Facebook Message, the sender can tap on it and they’ll find the delete button has been replaced by “Remove for you,” but there’s now also a “Remove for everyone” option that pulls the message from recipients’ inboxes. They’ll see an alert that you removed a message in its place, and can still flag the message to Facebook, which will retain the content briefly to see if it’s reported. The feature could make people more comfortable having honest conversations or using Messenger for flirting since they can second-guess what they send, but it won’t let people change ancient history.

The company abused its power by altering the history of Zuckerberg’s Facebook’s messages in a way that email or other communication mediums wouldn’t allow. Yet Facebook refused to say if it will now resume removing executives’ messages from recipients even long after they’re delivered after telling TechCrunch in April that “until this feature is ready, we will no longer be deleting any executives’ messages.”

For a quick recap, here’s how Facebook got to Unsend:

-Facebook Messenger never had an Unsend option, except in its encrypted Secret messaging product where you can set an expiration timer on chats, or in Instagram Direct.

-In April 2018, TechCrunch reported that some of Mark Zuckerberg’s messages had been removed from the inboxes of recipients, including non-employees. There was no trace of the chats in the message thread, leaving his conversation partners looking like they were talking to themselves, but email receipts proved the messages had been sent but later disappeared.

-Facebook claimed this was partly because it was “limiting the retention period for Mark’s messages” for security purposes in the wake of the Sony Pictures hack, yet it never explained why only some messages to some people had been removed.

-The next morning, Facebook changed its tune and announced it’d build an Unsend button for everyone, providing this statement: “We have discussed this feature several times . . . We will now be making a broader delete message feature available. This may take some time. And until this feature is ready, we will no longer be deleting any executives’ messages. We should have done this sooner — and we’re sorry that we did not.”

-Six months later in October 2018, Facebook still hadn’t launched Unsend, but then TechCrunch found Facebook had been prototyping the feature.

-In November, Facebook started to roll out the feature with the current “Remove for everyone” design and 10-minute limit.

-Now every iOS and Messenger user globally will get the Unsend feature.

So will Facebook start retracting executives’ messages again? It’d only say that the new feature would be available to both users and employees. But in Zuckerberg’s case, messages from years ago were removed in a way users still aren’t allowed to. Remove for everyone could make messaging on Facebook a little less anxiety-inducing. But it shouldn’t have taken Facebook being caught stealing from the inboxes of its users to get it built.

Powered by WPeMatico

Apple bans Facebook’s Research app that paid users for data

Posted by | Apple, Apps, Facebook, facebook research, Mark Zuckerberg, Mobile, Policy, privacy, Social, TC, Teens, Tim Cook, vpn | No Comments

In the wake of TechCrunch’s investigation yesterday, Apple blocked Facebook’s Research VPN app before the social network could voluntarily shut it down. The Research app asked users for root network access to all data passing through their phone in exchange for $20 per month. Apple tells TechCrunch that yesterday evening it revoked the Enterprise Certificate that allows Facebook to distribute the Research app without going through the App Store.

TechCrunch had reported that Facebook was breaking Apple’s policy that the Enterprise system is only for distributing internal corporate apps to employees, not paid external testers. That was actually before Facebook released a statement last night saying that it had shut down the iOS version of the Research program without mentioning that it was forced by Apple to do so.

TechCrunch’s investigation discovered that Facebook has been quietly operated the Research program on iOS and Android since 2016, recently under the name Project Atlas. It recruited 13 to 35 year olds, 5 percent of which were teenagers, with ads on Instagram and Snapchat and paid them a monthly fee plus referral bonuses to install Facebook’s Research app, the included VPN app that routes traffic to Facebook, and to ‘Trust’ the company with root network access to their phone. That lets Facebook pull in a user’s web browsing activity, what apps are on their phone and how they use them, and even decrypt their encrypted traffic. Facebook went so far as to ask users to screenshot and submit their Amazon order history. Facebook uses all this data to track competitors, assess trends, and plan its product roadmap.

Facebook was forced to remove its similar Onavo Protect app in August last year after Apple changed its policies to prohibit the VPN app’s data collection practices. But Facebook never shut down the Research app with the same functionality it was running in parallel. In fact, TechCrunch commissioned security expert Will Strafach to dig into the Facebook Research app, and we found that it featured tons of similar code and references to Onavo Protect. That means Facebook was purposefully disobeying the spirit of Apple’s 2018 privacy policy change while also abusing the Enterprise Certificate program.

Sources tell us that Apple revoking Facebook’s Enterprise Certificate has broken all of the company’s legitimate employee-only apps. Those include pre-launch internal-testing versions of Facebook and Instagram, as well as the employee apps for coordinating office collaboration, commutes, seeing the day’s lunch schedule, and more. That’s causing mayhem at Facebook, disrupting their daily work flow and ability to do product development. We predicted yesterday that Apple could take this drastic step to punish Facebook much harder than just removing its Research app. The disruption will translate into a huge loss of productivity for Facebook’s 33,000 employees.

[Update: Facebook later confirmed to TechCrunch that its internal apps were broken by Apple’s punishment and that it’s in talks with Apple to try to resolve the issue and get their employee tools running again.]

For reference, Facebook’s main iOS app still functions normally. Also, you can’t get paid for installing Onavo Protect on Android, only for the Facebook Research app. And Facebook isn’t the only one violating Apple’s Enterprise Certificate policy, as TechCrunch discovered Google’s Screenwise Meter surveillance app breaks the rules too.

This morning, Apple informed us it had banned Facebook’s Research app yesterday before the social network seemingly pulled it voluntarily. Apple provided us with this strongly worded statement condemning the social network’s behavior:

“We designed our Enterprise Developer Program solely for the internal distribution of apps within an organization. Facebook has been using their membership to distribute a data-collecting app to consumers, which is a clear breach of their agreement with Apple. Any developer using their enterprise certificates to distribute apps to consumers will have their certificates revoked, which is what we did in this case to protect our users and their data.”

That comes in direct contradiction to Facebook’s initial response to our investigation. Facebook claimed it was in alignment with Apple’s Enterprise Certificate policy and that the program was no different than a focus group.

Seven hours later, a Facebook spokesperson said it was pulling its Research program from iOS without mentioning that Apple forced it to do so, and issued this statement disputing the characterization of our story:

“Key facts about this market research program are being ignored. Despite early reports, there was nothing ‘secret’ about this; it was literally called the Facebook Research App. It wasn’t ‘spying’ as all of the people who signed up to participate went through a clear on-boarding process asking for their permission and were paid to participate. Finally, less than 5 percent of the people who chose to participate in this market research program were teens. All of them with signed parental consent forms.”

We refute those accusations by Facebook. As we wrote yesterday night, Facebook did not publicly promote the Research VPN itself and used intermediaries that often didn’t disclose Facebook’s involvement until users had begun the signup process. While users were given clear instructions and warnings, the program never stresses nor mentions the full extent of the data Facebook can collect through the VPN. A small fraction of the users paid may have been teens, but we stand by the newsworthiness of its choice not to exclude minors from this data collection initiative.

Senator Mark Warner has since called on Facebook CEO Mark Zuckerberg to support legislation requiring individual informed consent for market research initiatives like Facebook Research. Meanwhile, Senator Richard Blumenthal issued a fierce statement that “Wiretapping teens is not research, and it should never be permissible.”

The situation will surely worsen the relationship between Facebook and Apple after years of mounting animosity between the tech giants. Apple’s Tim Cook has repeatedly criticized Facebook’s data collection practices, and Zuckerberg has countered that it offers products for free for everyone rather than making products few can afford like Apple. Flared tensions could see Facebook receive less promotion in the App Store, fewer integrations into iOS, and more jabs from Cook. Meanwhile, the world sees Facebook as having been caught red-handed threatening user privacy and breaking Apple policy.

Powered by WPeMatico

Seized cache of Facebook docs raise competition and consent questions

Posted by | Android, api, competition, Damian Collins, data protection law, DCMS committee, Developer, Europe, european union, Facebook, Mark Zuckerberg, Onavo, Policy, privacy, Six4Three, Social, social network, terms of service, United Kingdom, vpn | No Comments

A UK parliamentary committee has published the cache of Facebook documents it dramatically seized last week.

The documents were obtained by a legal discovery process by a startup that’s suing the social network in a California court in a case related to Facebook changing data access permissions back in 2014/15.

The court had sealed the documents but the DCMS committee used rarely deployed parliamentary powers to obtain them from the Six4Three founder, during a business trip to London.

You can read the redacted documents here — all 250 pages of them.

In a series of tweets regarding the publication, committee chair Damian Collins says he believes there is “considerable public interest” in releasing them.

“They raise important questions about how Facebook treats users data, their policies for working with app developers, and how they exercise their dominant position in the social media market,” he writes.

“We don’t feel we have had straight answers from Facebook on these important issues, which is why we are releasing the documents. We need a more public debate about the rights of social media users and the smaller businesses who are required to work with the tech giants. I hope that our committee investigation can stand up for them.”

The committee has been investigating online disinformation and election interference for the best part of this year, and has been repeatedly frustrated in its attempts to extract answers from Facebook.

But it is protected by parliamentary privilege — hence it’s now published the Six4Three files, having waited a week in order to redact certain pieces of personal information.

Collins has included a summary of key issues, as the committee sees them after reviewing the documents, in which he draws attention to six issues.

Here is his summary of the key issues:

  • White Lists Facebook have clearly entered into whitelisting agreements with certain companies, which meant that after the platform changes in 2014/15 they maintained full access to friends data. It is not clear that there was any user consent for this, nor how Facebook decided which companies should be whitelisted or not.

Facebook responded

  • Value of friends data It is clear that increasing revenues from major app developers was one of the key drivers behind the Platform 3.0 changes at Facebook. The idea of linking access to friends data to the financial value of the developers relationship with Facebook is a recurring feature of the documents.

In their response Facebook contends that this was essentially another “cherrypicked” topic and that the company “ultimately settled on a model where developers did not need to purchase advertising to access APIs and we continued to provide the developer platform for free.”

  • Reciprocity Data reciprocity between Facebook and app developers was a central feature in the discussions about the launch of Platform 3.0.
  • Android Facebook knew that the changes to its policies on the Android mobile phone system, which enabled the Facebook app to collect a record of calls and texts sent by the user would be controversial. To mitigate any bad PR, Facebook planned to make it as hard of possible for users to know that this was one of the underlying features of the upgrade of their app.
  • Onavo Facebook used Onavo to conduct global surveys of the usage of mobile apps by customers, and apparently without their knowledge. They used this data to assess not just how many people had downloaded apps, but how often they used them. This knowledge helped them to decide which companies to acquire, and which to treat as a threat.
  • Targeting competitor Apps The files show evidence of Facebook taking aggressive positions against apps, with the consequence that denying them access to data led to the failure of that business.

Update: 11:40am

Facebook has posted a lengthy response (read it here) positing that the “set of documents, by design, tells only one side of the story and omits important context.” They give a blow-by-blow response to Collins’ points below though they are ultimately pretty selective in what they actually address.

Generally they suggest that some of the issues being framed as anti-competitive were in fact designed to prevent “sketchy apps” from operating on the platform. Furthermore, Facebook details that they delete some old call logs on Android, that using “market research” data from Onava is essentially standard practice and that users had the choice whether data was shared reciprocally between FB and developers. In regard to specific competitors’ apps, Facebook appears to have tried to get ahead of this release with their announcement yesterday that it was ending its platform policy of banning apps that “replicate core functionality.” 

The publication of the files comes at an awkward moment for Facebook — which remains on the back foot after a string of data and security scandals, and has just announced a major policy change — ending a long-running ban on apps copying its own platform features.

Albeit the timing of Facebook’s policy shift announcement hardly looks incidental — given Collins said last week the committee would publish the files this week.

The policy in question has been used by Facebook to close down competitors in the past, such as — two years ago — when it cut off style transfer app Prisma’s access to its live-streaming Live API when the startup tried to launch a livestreaming art filter (Facebook subsequently launched its own style transfer filters for Live).

So its policy reversal now looks intended to diffuse regulatory scrutiny around potential antitrust concerns.

But emails in the Six4Three files suggesting that Facebook took “aggressive positions” against competing apps could spark fresh competition concerns.

In one email dated January 24, 2013, a Facebook staffer, Justin Osofsky, discusses Twitter’s launch of its short video clip app, Vine, and says Facebook’s response will be to close off its API access.

As part of their NUX, you can find friends via FB. Unless anyone raises objections, we will shut down their friends API access today. We’ve prepared reactive PR, and I will let Jana know our decision,” he writes. 

Osofsky’s email is followed by what looks like a big thumbs up from Zuckerberg, who replies: “Yup, go for it.”

Also of concern on the competition front is Facebook’s use of a VPN startup it acquired, Onavo, to gather intelligence on competing apps — either for acquisition purposes or to target as a threat to its business.

The files show various Onavo industry charts detailing reach and usage of mobile apps and social networks — with each of these graphs stamped ‘highly confidential’.

Facebook bought Onavo back in October 2013. Shortly after it shelled out $19BN to acquire rival messaging app WhatsApp — which one Onavo chart in the cache indicates was beasting Facebook on mobile, accounting for well over double the daily message sends at that time.

Onavo charts are quite an insight into facebook’s commanding view of the app-based attention marketplace pic.twitter.com/Ezdaxk6ffC

— David Carroll 🦅 (@profcarroll) December 5, 2018

The files also spotlight several issues of concern relating to privacy and data protection law, with internal documents raising fresh questions over how or even whether (in the case of Facebook’s whitelisting agreements with certain developers) it obtained consent from users to process their personal data.

The company is already facing a number of privacy complaints under the EU’s GDPR framework over its use of ‘forced consent‘, given that it does not offer users an opt-out from targeted advertising.

But the Six4Three files look set to pour fresh fuel on the consent fire.

Collins’ fourth line item — related to an Android upgrade — also speaks loudly to consent complaints.

Earlier this year Facebook was forced to deny that it collects calls and SMS data from users of its Android apps without permission. But, as we wrote at the time, it had used privacy-hostile design tricks to sneak expansive data-gobbling permissions past users. So, put simple, people clicked ‘agree’ without knowing exactly what they were agreeing to.

The Six4Three files back up the notion that Facebook was intentionally trying to mislead users.

In one email dated November 15, 2013, from Matt Scutari, manager privacy and public policy, suggests ways to prevent users from choosing to set a higher level of privacy protection, writing: “Matt is providing policy feedback on a Mark Z request that Product explore the possibility of making the Only Me audience setting unsticky. The goal of this change would be to help users avoid inadvertently posting to the Only Me audience. We are encouraging Product to explore other alternatives, such as more aggressive user education or removing stickiness for all audience settings.”

Another awkward trust issue for Facebook which the documents could stir up afresh relates to its repeat claim — including under questions from lawmakers — that it does not sell user data.

In one email from the cache — sent by Mark Zuckerberg, dated October 7, 2012 — the Facebook founder appears to be entertaining the idea of charging developers for “reading anything, including friends”.

Yet earlier this year, when he was asked by a US lawmaker how Facebook makes money, Zuckerberg replied: “Senator, we sell ads.”

He did not include a caveat that he had apparently personally entertained the idea of liberally selling access to user data.

Responding to the publication of the Six4Three documents, a Facebook spokesperson told us:

As we’ve said many times, the documents Six4Three gathered for their baseless case are only part of the story and are presented in a way that is very misleading without additional context. We stand by the platform changes we made in 2015 to stop a person from sharing their friends’ data with developers. Like any business, we had many of internal conversations about the various ways we could build a sustainable business model for our platform. But the facts are clear: we’ve never sold people’s data.

Zuckerberg has repeatedly refused to testify in person to the DCMS committee.

At its last public hearing — which was held in the form of a grand committee comprising representatives from nine international parliaments, all with burning questions for Facebook — the company sent its policy VP, Richard Allan, leaving an empty chair where Zuckerberg’s bum should be.

Powered by WPeMatico

Tech giants offer empty apologies because users can’t quit

Posted by | Amazon, Apple, Apps, Cambridge Analytica, Drama, Elliot Schrage, Facebook, Facebook Policy, facebook privacy, GDPR, Google, Government, Mark Zuckerberg, Microsoft, Mobile, Policy, privacy, project maven, Security, Social, Talent, TC | No Comments

A true apology consists of a sincere acknowledgement of wrong-doing, a show of empathic remorse for why you wronged and the harm it caused, and a promise of restitution by improving ones actions to make things right. Without the follow-through, saying sorry isn’t an apology, it’s a hollow ploy for forgiveness.

That’s the kind of “sorry” we’re getting from tech giants — an attempt to quell bad PR and placate the afflicted, often without the systemic change necessary to prevent repeated problems. Sometimes it’s delivered in a blog post. Sometimes it’s in an executive apology tour of media interviews. But rarely is it in the form of change to the underlying structures of a business that caused the issue.

Intractable Revenue

Unfortunately, tech company business models often conflict with the way we wish they would act. We want more privacy but they thrive on targeting and personalization data. We want control of our attention but they subsist on stealing as much of it as possible with distraction while showing us ads. We want safe, ethically built devices that don’t spy on us but they make their margins by manufacturing them wherever’s cheap with questionable standards of labor and oversight. We want groundbreaking technologies to be responsibly applied, but juicy government contracts and the allure of China’s enormous population compromise their morals. And we want to stick to what we need and what’s best for us, but they monetize our craving for the latest status symbol or content through planned obsolescence and locking us into their platforms.

The result is that even if their leaders earnestly wanted to impart meaningful change to provide restitution for their wrongs, their hands are tied by entrenched business models and the short-term focus of the quarterly earnings cycle. They apologize and go right back to problematic behavior. The Washington Post recently chronicled a dozen times Facebook CEO Mark Zuckerberg has apologized, yet the social network keeps experiencing fiasco after fiasco. Tech giants won’t improve enough on their own.

Addiction To Utility

The threat of us abandoning ship should theoretically hold the captains in line. But tech giants have evolved into fundamental utilities that many have a hard time imagining living without. How would you connect with friends? Find what you needed? Get work done? Spend your time? What hardware or software would you cuddle up with in the moments you feel lonely? We live our lives through tech, have become addicted to its utility, and fear the withdrawal.

If there were principled alternatives to switch to, perhaps we could hold the giants accountable. But the scalability, network effects, and aggregation of supply by distributors has led to near monopolies in these core utilities. The second-place solution is often distant. What’s the next best social network that serves as an identity and login platform that isn’t owned by Facebook? The next best premium mobile and PC maker behind Apple? The next best mobile operating system for the developing world beyond Google’s Android? The next best ecommerce hub that’s not Amazon? The next best search engine? Photo feed? Web hosting service? Global chat app? Spreadsheet?

Facebook is still growing in the US & Canada despite the backlash, proving that tech users aren’t voting with their feet. And if not for a calculation methodology change, it would have added 1 million users in Europe this quarter too.

One of the few tech backlashes that led to real flight was #DeleteUber. Workplace discrimination, shady business protocols, exploitative pricing and more combined to spur the movement to ditch the ridehailing app. But what was different here is that US Uber users did have a principled alternative to switch to without much hassle: Lyft. The result was that “Lyft benefitted tremendously from Uber’s troubles in 2018” eMarketer’s forecasting director Shelleen Shum told the USA Today in May. Uber missed eMarketer’s projections while Lyft exceeded them, narrowing the gap between the car services. And meanwhile, Uber’s CEO stepped down as it tried to overhaul its internal policies.

This is why we need regulation that promotes competition by preventing massive mergers and giving users the right to interoperable data portability so they can easily switch away from companies that treat them poorly

But in the absence of viable alternatives to the giants, leaving these mainstays is inconvenient. After all, they’re the ones that made us practically allergic to friction. Even after massive scandals, data breaches, toxic cultures, and unfair practices, we largely stick with them to avoid the uncertainty of life without them. Even Facebook added 1 million monthly users in the US and Canada last quarter despite seemingly every possible source of unrest. Tech users are not voting with their feet. We’ve proven we can harbor ill will towards the giants while begrudgingly buying and using their products. Our leverage to improve their behavior is vastly weakened by our loyalty.

Inadequate Oversight

Regulators have failed to adequately step up either. This year’s congressional hearings about Facebook and social media often devolved into inane and uninformed questioning like how does Facebook earn money if its doesn’t charge? “Senator, we run ads” Facebook CEO Mark Zuckerberg said with a smirk. Other times, politicians were so intent on scoring partisan points by grandstanding or advancing conspiracy theories about bias that they were unable to make any real progress. A recent survey commissioned by Axios found that “In the past year, there has been a 15-point spike in the number of people who fear the federal government won’t do enough to regulate big tech companies — with 55% now sharing this concern.”

When regulators do step in, their attempts can backfire. GDPR was supposed to help tamp down on the dominance of Google and Facebook by limiting how they could collect user data and making them more transparent. But the high cost of compliance simply hindered smaller players or drove them out of the market while the giants had ample cash to spend on jumping through government hoops. Google actually gained ad tech market share and Facebook saw the littlest loss while smaller ad tech firms lost 20 or 30 percent of their business.

Europe’s GDPR privacy regulations backfired, reinforcing Google and Facebook’s dominance. Chart via Ghostery, Cliqz, and WhoTracksMe.

Even the Honest Ads act, which was designed to bring political campaign transparency to internet platforms following election interference in 2016, has yet to be passed even despite support from Facebook and Twitter. There’s hasn’t been meaningful discussion of blocking social networks from acquiring their competitors in the future, let alone actually breaking Instagram and WhatsApp off of Facebook. Governments like the U.K. that just forcibly seized documents related to Facebook’s machinations surrounding the Cambridge Analytica debacle provide some indication of willpower. But clumsy regulation could deepen the moats of the incumbents, and prevent disruptors from gaining a foothold. We can’t depend on regulators to sufficiently protect us from tech giants right now.

Our Hope On The Inside

The best bet for change will come from the rank and file of these monolithic companies. With the war for talent raging, rock star employees able to have huge impact on products, and compensation costs to keep them around rising, tech giants are vulnerable to the opinions of their own staff. It’s simply too expensive and disjointing to have to recruit new high-skilled workers to replace those that flee.

Google declined to renew a contract with the government after 4000 employees petitioned and a few resigned over Project Maven’s artificial intelligence being used to target lethal drone strikes. Change can even flow across company lines. Many tech giants including Facebook and Airbnb have removed their forced arbitration rules for harassment disputes after Google did the same in response to 20,000 of its employees walking out in protest.

Thousands of Google employees protested the company’s handling of sexual harassment and misconduct allegations on Nov. 1.

Facebook is desperately pushing an internal communications campaign to reassure staffers it’s improving in the wake of damning press reports from the New York Times and others. TechCrunch published an internal memo from Facebook’s outgoing VP of communications Elliot Schrage in which he took the blame for recent issues, encouraged employees to avoid finger-pointing, and COO Sheryl Sandberg tried to reassure employees that “I know this has been a distraction at a time when you’re all working hard to close out the year — and I am sorry.” These internal apologizes could come with much more contrition and real change than those paraded for the public.

And so after years of us relying on these tech workers to build the product we use every day, we must now rely that will save us from them. It’s a weighty responsibility to move their talents where the impact is positive, or commit to standing up against the business imperatives of their employers. We as the public and media must in turn celebrate when they do what’s right for society, even when it reduces value for shareholders. If apps abuse us or unduly rob us of our attention, we need to stay off of them.

And we must accept that shaping the future for the collective good may be inconvenient for the individual. There’s an oppprtunity here not just to complain or wish, but to build a social movement that holds tech giants accountable for delivering the change they’ve promised over and over.

For more on this topic:

Powered by WPeMatico

Facebook Messenger starts rolling out Unsend; here’s how it works

Posted by | Apps, Facebook, facebook messenger, Facebook unsend, Mark Zuckerberg, Mobile, Social, Stan Chudnovsky, TC | No Comments

Facebook secretly retracted messages sent by CEO Mark Zuckerberg, TechCrunch reported seven months ago. Now for the first time, Facebook Messenger users will get the power to unsend too so they can remove their sent messages from the recipient’s inbox. Messages can only be unsent for the first 10 minutes after they’re delivered so that you can correct a mistake or remove something you accidentally pushed, but you won’t be able to edit ancient history. Formally known as “Remove for Everyone,” the button also leaves a “tombstone” indicating a message was retracted. And to prevent bullies from using the feature to cover their tracks, Facebook will retain unsent messages for a short period of time so if they’re reported, it can review them for policy violations.

The Remove feature rolls out in Poland, Bolivia, Colombia and Lithuania today on Messenger for iOS and Android. A Facebook spokesperson tells me the plan is to roll it out globally as soon as possible, though that may be influenced by the holiday App Store update cut-off. In the meantime, it’s also working on more unsend features, potentially including the ability to preemptively set an expiration date for specific messages or entire threads.

“The pros are that users want to be in control . . . and if you make a mistake you can correct it. There are a lot of legitimate use cases out there that we wanted to enable,” Facebook’s head of Messenger Stan Chudnovsky tells me in an exclusive interview. But conversely, he says, “We need to make sure we don’t open up any new venues for bullying. We need to make sure people aren’t sending you bad messages and then removing them because if you report them and the messages aren’t there we can’t do anything.”

Zuckerberg did it; soon you can, too

Facebook first informed TechCrunch it would build an unsend feature back in April after I reported that six sources told me some of Mark Zuckerberg’s Facebook messages had been silently removed from the inboxes of recipients, including non-employees with no tombstone left in their place. We saw that as a violation of user trust and an abuse of the company’s power, given the public had no way to unsend their own messages.

Facebook claimed this was to protect the privacy of its executives and the company’s trade secrets, telling me that “After Sony Pictures’ emails were hacked in 2014 we made a number of changes to protect our executives’ communications. These included limiting the retention period for Mark’s messages in Messenger.” But it seems likely that Facebook also wanted to avoid another embarrassing situation like when Zuckerberg’s old instant messages from 2004 leaked. One damning exchange saw Zuckerberg tell a friend “if you ever need info about anyone at harvard . . . just ask . . . i have over 4000 emails, pictures, addresses, sns.” “what!? how’d you manage that one?”  the friend replied. “People just submitted it . .  i don’t know why . . . they ‘trust me’ . . . dumb fucks” Zuckerberg replied.

The company told me it was actually already working on an Unsend button for everyone, and wouldn’t delete any more executives’ messages until it launched. Chudnovsky tells me he felt like “I wish we launched this sooner” when the news broke. But then six months went by without progress or comment from Facebook before TechCrunch broke the news that tipster Jane Manchun Wong had spotted Facebook prototyping the Remove feature. Then a week ago, Facebook Messenger’s App Store release notes accidentally mentioned that a 10-minute Unsend button was coming soon.

So why the seven-month wait? Especially given Instagram already allows users to unsend messages no matter how old? “The reason why it took so long is because on the server side, it’s actually much harder. All the messages are stored on the server, and that goes into the core transportation layer of our how our messaging system was built,” Chudnovsky explains. “It was hard to do given how we were architected, but we were always worried about the integrity concerns it would open up.” Now the company is confident it’s surmounted the engineering challenge to ensure an Unsent message reliably disappears from the recipient.

“The question becomes ‘who owns that message?’ Before that message is delivered to your Messenger app, it belongs to me. But when it actually arrives, it probably belongs to both of us,” Chudnovsky pontificates.

How Facebook Messenger’s “Remove for Everyone” button works

Facebook settled on the ability to let you remove any kind of message — including text, group chats, photos, videos, links and more — within 10 minutes of sending. You can still delete any message on just your side of the conversation, but only messages you sent can be removed from their recipients. You can’t delete from someone else what they sent you, the feature’s PR manager Kat Chui tells me. And Facebook will keep a private copy of the message for a short while after it’s deleted to make sure it can review if it’s reported for harassment.

To use the unsend feature, tap and hold on a message you sent, then select “Remove.” You’ll get options to “Remove for Everyone” which will retract the message, or “Remove for you,” which replaces the old delete option and leaves the message in the recipient’s inbox. You’ll get a warning that explains “You’ll permanently remove this message for all chat members. They can see that you removed a message and still report it.” If you confirm the removal, a line of text noting “you [or the sender’s name] removed a message” (known as a tombstone) will appear in the thread where the message was. If you want to report a removed message for abuse or another issue, you’ll tap the person’s name, scroll to “Something’s Wrong” and select the proper category such as harassment or that they were pretending to be someone else.

Why the 10-minute limit specifically? “We looked at how the existing delete functionality works. It turns out that when people are deleting messages because it’s a mistake or they sent something they didn’t want to send, it’s under a minute. We decided to extend it to 10, but decided we didn’t need to do more,” Chudnovsky reveals.

He says he’s not sure if Facebook’s security team will now resume removing executive messages. However, he stresses that the Unsend button Facebook is launching “is definitely not the same feature” as what was used on Zuckerberg’s messages. If Facebook wanted to truly respect its users, it would at least insert the tombstone when it erases old messages from executives.

Messenger is also building more unsend functionality. Taking a cue from encrypted messaging app Signal’s customizable per thread expiration date feature, Chudnovsky tells me “hypothetically, if I want all the messages to be deleted after six months, they get purged. This is something that can be set up on a per thread level,” though Facebook is still tinkering with the details. Another option would be for Facebook to extend to all chats the per message expiration date option from its encrypted Secret messages feature.

“It’s one of those things that feels very simple on the surface. And it would be very easy if the servers were built one way or another from the very beginning,” Chudnovsky concludes. “But it’s one of those things philosophically and technologically that once you get to the scale of 1.3 billion people using it, changing from one model to another is way more complicated.” Hopefully in the future, Facebook won’t give its executives extrajudicial ways to manipulate communications… or at least not until it’s sorted out the consequences of giving the public the same power.

Powered by WPeMatico

10 critical points from Zuckerberg’s epic security manifesto

Posted by | Apps, Election Interference, Facebook, Facebook Election Interference, Facebook Policy, Facebook Security, Government, Mark Zuckerberg, Mobile, Personnel, Policy, Talent, TC | No Comments

Mark Zuckerberg wants you to know he’s trying his damnedest to fix Facebook before it breaks democracy. Tonight he posted a 3,260-word battle plan for fighting election interference. Amidst drilling through Facebook’s strategy and progress, he slips in several notable passages revealing his own philosophy.

Zuckerberg has cast off his premature skepticism and is ready to command the troops. He sees Facebook’s real identity policy as a powerful weapon for truth other social networks lack, but that would be weakened if Instagram and WhatsApp were split off by regulators. He’s done with the finger-pointing and wants everyone to work together on solutions. And he’s adopted a touch of cynicism that could open his eyes and help him predict how people will misuse his creation.

Here are the most important parts of Zuckerberg’s security manifesto:

Zuckerberg embraces his war-time tactician role

“While we want to move quickly when we identify a threat, it’s also important to wait until we uncover as much of the network as we can before we take accounts down to avoid tipping off our adversaries, who would otherwise take extra steps to cover their remaining tracks. And ideally, we time these takedowns to cause the maximum disruption to their operations.”

The fury he unleashed on Google+, Snapchat, and Facebook’s IPO-killer is now aimed at election attackers

“These are incredibly complex and important problems, and this has been an intense year. I am bringing the same focus and rigor to addressing these issues that I’ve brought to previous product challenges like shifting our services to mobile.”

Balancing free speech and security is complicated and expensive

“These issues are even harder because people don’t agree on what a good outcome looks like, or what tradeoffs are acceptable to make. When it comes to free expression, thoughtful people come to different conclusions about the right balances. When it comes to implementing a solution, certainly some investors disagree with my approach to invest so much in security.”

Putting Twitter and YouTube on blast for allowing pseudonymity…

“One advantage Facebook has is that we have a principle that you must use your real identity. This means we have a clear notion of what’s an authentic account. This is harder with services like Instagram, WhatsApp, Twitter, YouTube, iMessage, or any other service where you don’t need to provide your real identity.”

…While making an argument for why the Internet is more secure if Facebook isn’t broken up

“Fortunately, our systems are shared, so when we find bad actors on Facebook, we can also remove accounts linked to them on Instagram and WhatsApp as well. And where we can share information with other companies, we can also help them remove fake accounts too.”‘

Political ads aren’t a business, they’re supposedly a moral duty

“When deciding on this policy, we also discussed whether it would be better to ban political ads altogether. Initially, this seemed simple and attractive. But we decided against it — not due to money, as this new verification process is costly and so we no longer make any meaningful profit on political ads — but because we believe in giving people a voice. We didn’t want to take away an important tool many groups use to engage in the political process.”

Zuckerberg overruled staff to allow academic research on Facebook

“As a result of these controversies [like Cambridge Analytica], there was considerable concern amongst Facebook employees about allowing researchers to access data. Ultimately, I decided that the benefits of enabling this kind of academic research outweigh the risks. But we are dedicating significant resources to ensuring this research is conducted in a way that respects people’s privacy and meets the highest ethical standards.”

Calling on law enforcement to step up

“There are certain critical signals that only law enforcement has access to, like money flows. For example, our systems make it significantly harder to set up fake accounts or buy political ads from outside the country. But it would still be very difficult without additional intelligence for Facebook or others to figure out if a foreign adversary had set up a company in the US, wired money to it, and then registered an authentic account on our services and bought ads from the US.”

Instead of minimizing their own blame, the major players must unite forces

“Preventing election interference is bigger than any single organization. It’s now clear that everyone — governments, tech companies, and independent experts such as the Atlantic Council — need to do a better job sharing the signals and information they have to prevent abuse . . . The last point I’ll make is that we’re all in this together. The definition of success is that we stop cyberattacks and coordinated information operations before they can cause harm.”

The end of Zuckerberg’s utopic idealism

“One of the important lessons I’ve learned is that when you build services that connect billions of people across countries and cultures, you’re going to see all of the good humanity is capable of, and you’re also going to see people try to abuse those services in every way possible.”

Powered by WPeMatico