How Facebook's Mark Zuckerberg plans to take over the world

It’s late afternoon on a blustery spring day on the waterfront at San Francisco’s Fort Mason, a former military base that’s now hired out for corporate functions. Vast warehouses, once used to store army supplies, are awash with sleek signs, shimmering lights and endless snacks. Behind them is an Instagram-ready view of Alcatraz island. In front, a fleet of Uber and Lyft cars lines up in the car park, while inside one of the warehouses Scottish synthpop band Chvrches take the stage.

For the first few songs there’s only a small group of hardcore vocal fans at the front of the stage, flanked by a subdued mix of backpack-wearing dad types politely bobbing their heads, drinking cocktails out of plastic cups.

The shindig has been put on by Facebook for the benefit of delegates attending its F8 conference . The event, which has run most years since 2007, began as a means to win over the developer community and has now become a comprehensive and highly engineered launchpad for the company’s annual plans. Many of the 2,600 attendees have paid $595 to find out how they can integrate their own digital products with Facebook to carve out some kind of presence among its enormous audience – and there’s booze and entertainment thrown in.

The band is performing on the same stage from where Mark Zuckerberg has delivered the conference’s opening keynote speech. The lead singer makes a joke comparing the 31-year-old CEO to Star Wars villain Kylo Ren, and it seems to break the ice. From that point on the cheers and claps get louder and it starts to change into a more recognisable gig – but there is nothing as rapturous as when, hours earlier, “Zuck” had pledged a free virtual reality headset and Samsung smartphone to every attendee.

For this audience, it’s clear who the real rockstar is.

When Zuckerberg addresses the F8 audience it is with the composure and conviction of a president addressing his citizens . “We’ve gone from a world of isolated communities to one global community, and we are all better off for it,” he says as he hammers home his “mission” to connect the world.

He warns of “ people and nations turning inwards – against this idea of a connected world and community”, a position that fits both with his ideology and that of Facebook. This is not a speech about technical tweaks, but a state of the union address.

Related: Facebook's Mark Zuckerberg meets China propaganda chief in Beijing

“It takes courage to choose hope over fear,” he adds. Behind the rhetoric and the casual clothes, the message is clear: Facebook is one of the big boys now, taking on huge global challenges and planning for prosperity.

The scale of Facebook’s audience is unprecedented. More than 1.6 billion people use Facebook at least once a month, or half of all internet users. That’s before you count users on other Facebook-owned sites including WhatsApp, which has more than 1 billion monthly active users, and photo-sharing site Instagram, which has 400 million.

Facebook has also introduced its free basics service to 37 countries , offering a free but limited package of apps to mobile phone users, but which some critics say allows Facebook to tightly control the online experience of potentially the next billion people to come online.

“You hear all the platitudes about Facebook connecting the planet, but to say they are doing it for benevolent reasons is absolute nonsense. It’s about connecting commerce, not people,” says venture capitalist and former journalist Om Malik, who reminds us of the hidden agenda of social networking firms: if you’re not paying, you’re the product.

Facebook – which made $5.8bn of revenue in the last three months of 2015 – is able to make money from its users not just because of that unprecedented audience, but the amount of time they spend on the service. In the US the average 18- to 34-year-old spends 30 hours per month on social networking services, and 26 of those are on Facebook, according to analysts at ComScore.

Every click, every like, every comment and every connection is used to build up a rich profile of each user. Brands can then pay Facebook to target users based on their age, location, relationship status and interests. This is how Facebook makes its money – profiles of us that advertisers adore.

On the Facebook platform, the glue that keeps everyone hanging around is “content”. The first phase was personal – our status updates, thoughts, feelings and witty punditry, but we quickly learned that our friends weren’t as interesting as we thought. The second phase was photos. The rise of smartphones meant that everyone had a camera in their pocket and a newfound belief that a single image could tell a thousand words about their latte or hotdog legs. But not everyone is a great photographer.

Related: Facebook is worried about users sharing less – but it only has itself to blame

So Facebook evolved. It’s this power for transformation that has allowed the company to thrive where others (MySpace, Friends Reunited, even Twitter) have withered, moving from a digital directory for college kids to a communications behemoth.

When the company recognised that we often want to keep our conversations private it turned Messenger into a standalone app – with huge success. When photo-sharing startup Instagram was gaining traction Facebook bought it , but instead of incorporating it and killing the brand, it allowed it to continue as a satellite.

The platitudes about Facebook connecting the planet? To say they're doing it for benevolent reasons is absolute nonsense

Om Malik, investor

And now we find ourselves in the third phase: sharing articles, videos and images created by media organisations and seeded on to the platform. It’s appealing to the media because that’s where everyone is already. They don’t have to lure people to their websites; they can simply deliver stories to people’s news feeds. The strategy has been extremely successful at driving traffic to publishers’ websites – typically around a quarter of visitors will come from the social network.

Facebook’s latest move is “instant articles”, which allow publishers to show smartphone users a fast-loading view of entire articles without ever having to leave the social networking app. It’s good for the reader, but the creator of the content loses the traffic to its own website and control of how it’s presented. Publishers can try to sell advertising on instant articles themselves or let Facebook do it for them – for a fee, of course.

The newly launched live video tool – stress-tested by BuzzFeed, who drew an audience of 800,000 for a demonstration of how to explode a watermelon using rubber bands (the footage has since been watched more than 10m times) – is likely to present a similar dilemma for broadcasters.

“Facebook has incredible power and influence over all news brands now,” a developer from a major global media company tells me at F8. “Most of our growth over the last two years has been driven by Facebook and Google.”

Facebook has such sway that it has started to dictate the innovation strategies of media companies. When Facebook announces new features, publishers scramble to rearrange their own priorities for fear of being relegated by Facebook’s algorithms.

Having gobbled up a hefty chunk of the news industry, Facebook has also set its sights on digital services that can be brought into the nest and regurgitated for hungry Facebook users: bookmarking, 360-degree video, customer service robots , payments and virtual reality. For each of these things, there are free simple-to-use tools to let businesses plug in. They don’t have to use them, but it’s easier to piggyback on Facebook’s prowess than build a system from scratch.

Computers will start to get closer to the way we experience the real world with our fingers, mouth and with no keyboard

Facebook chief product officer Chris Cox

The company is slowly removing the reasons for leaving its site – whether it’s to book a taxi, watch video content, make phone calls, pay for things or read news articles. And the Facebook experience is sanitised, shielding users from the big bad internet with strict rules about what content it deems appropriate.

If live video is Facebook’s phase four, then artificial intelligence and virtual reality look like being big parts of phase five. Both of these fit its strategy of monetising as many of our social interactions as possible.

“I think it makes sense that computers start to get closer and closer to the way we experience the real world with our fingers, mouth and with no keyboard,” says the company’s chief product officer, Chris Cox.

Facebook already uses artificial intelligence to personalise your newsfeed, identify you in photos and translate your posts. The company has built technology that can recognise objects – even different dog breeds – in photos and videos.

The ultimate aim is to develop algorithms that can understand the nuances of people’s physical interactions. This is particularly important if social VR, which would allow us to have conversations with distant loved ones in which they feel as though they are in the same space as you – imagine Skype in 3D – is going to take off.

Facebook demonstrated a crude prototype at F8 to illustrate how people might hang out with friends in virtual spaces viewed through Oculus Rift VR headsets , but using cartoonish avatars with only very basic facial features and hand gestures. Achieving a real sense that the person is right there with you will require a system that can capture a person’s movements down to the subtlest gesture, and turn that into a digital doppelganger inside a virtual world.

Yaser Sheikh, who leads the Oculus Rift research team, is using a camera-lined dome called the panoptic studio to record the movements of people placed inside from all angles. He is categorising the thousands of gestures and expressions that people use to interact. This library of gestures is what American anthropologist-linguist Edward Sapir described as “an elaborate and secret code that is written nowhere, known by none and understood by all”.

“We need a computational understanding of Sapir’s code,” Sheikh says. In other words, Facebook wants to be able to read our intentions and emotions. It’s not hard to imagine how this could be exploited in areas beyond VR. Chief technology officer Mike Schroepfer says an augmented reality device – perhaps smart glasses or earbuds – could give you “additional information” about someone while you were having a conversation with them, without them consciously offering it up.

“Maybe I get a warning indicating that you’re really sceptical about what I’m talking about and so maybe I should stop and clarify,” he says.

Zuckerberg says in his keynote speech: “Our goal with artificial intelligence is to build systems that are better than people at perception — seeing, hearing, language and so on.”

So Facebook could lift itself out of the confines of its apps and into the realm of ubiquitous virtual assistant that can help people navigate complex social situations.

Sheikh says: “We are a long way from producing a computational model of social interaction, but a lot of progress has been made and I’m confident we’ll get there.”

In the meantime, there’s plenty more money to be made with Facebook’s existing offering, particularly as the company commercialises properties that have been hitherto ad-free. “They haven’t turned on the monetisation switch for WhatsApp and Messenger yet; it wouldn’t take much to make a lot of money from those audiences,” says Jan Dawson, chief analyst at Jackdaw Research.

This is where the bots come in. Messenger bots , launched at F8, are lightweight apps with conversational capabilities that sit in the messaging app. You don’t download them, you just send queries to them through message threads as if they were another contact.

“Our experiences with businesses, services and brands are super fragmented,” says David Marcus, head of Messenger. “We call them, fill out forms on websites, email and download apps.” Bots, he says, can help automate many of the simple requests we have, whether tracking an order, checking in for a flight or ordering flowers.

Bringing these bots into Messenger also allows Facebook to eavesdrop on people’s one-to-one conversations with brands, providing more data-mining and monetisation opportunities. Marcus mentions sponsored chatbot messages as one way to make money, but it’s not difficult to see how, when combined with a credit card payment system or PayPal (where Marcus came from), it could help Facebook to become a powerful retail engine.

It’s a model that’s worked well for Tencent’s WeChat service in China, where users can interact with more than 10m branded “official accounts” to order food, buy movie tickets, send money to friends and pay bills – as well as follow news, recognise music and book appointments.

Dawson believes that this is part of a strategy to take on Amazon. “They seem to be building a commerce platform by stealth. Every company has a Facebook page, advertising already takes place on Facebook, you can talk to customer service bots. It’s everything except the process of selling you a product,” he says.

It’s unlikely Facebook would get into the logistics of fulfilling orders, but it could certainly create a customer relationship, and then push for a slice of each sale. And thus retailers follow publishers and marketers into the same Faustian pact.

It’s easy to forget that Facebook faced its only real existential threat when it failed to anticipate how quickly its users would migrate from desktop to mobile. After years of foot-dragging, the company spent $1bn buying Instagram and focused on working out how to make money from people using its apps on smartphones. Since then it’s been incredibly aggressive at buying or cloning challenger technologies.

It’s unlikely – but not impossible – that a competitor could step in with a Facebook substitute. Facebook simply has too many users and too many tentacles into different parts of our lives. Anyone that looks like a threat gets acquired, as we saw with Instagram and WhatsApp .

Disrupting Facebook would be like trying to disrupt telephone calls, it’s so ubiquitous

Paul Adams, former Facebook staffer

There remains, however, a thorn in Facebook’s side: Snapchat. Facebook tried to buy the messaging service, which has 100 million daily active users, for $3bn in 2013, but Snapchat turned it down. Facebook’s response was to build a clone called Slingshot, which didn’t take off. It’s since been experimenting with incorporating many of Snapchat’s popular features, such as ephemeral messages and photo-editing tools. More recently Facebook launched face-altering filters and scannable personal identifiers called messenger codes, almost identical to Snapchat’s snapcodes.

It might be irksome for Facebook, but analysts don’t think Snapchat is a major threat. “Snapchat is clearly a competitor, but it’s not yet clear to me how it will make the leap from focusing on a single generation. There’s a network effect that comes from being really big that’s really hard to overcome,” Dawson says.

“It might be a threat to Messenger and WhatsApp but it’s not a threat to Facebook, which is a system,” adds former Facebook employee Paul Adams, who now works at Intercom.

“It would be like trying to disrupt telephone calls, it’s so ubiquitous.”

Whichever way you look at it, Facebook is booming; publishers, businesses and brands are clinging on for the ride.

guardian.co.uk © Guardian News and Media 2016