Does anyone have a good explanation for Meta's strategy with AI?
The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?
They also don't have the same economic setup and DNA as MS/OpenAI. Large corporate customers don't pay for access to the FB cloud, nor are they likely to -- Ellison has spent years building out Oracle Cloud, and he's on the FB board, for example. And I bet you didn't think of using Oracle's Cloud for your last project.
So, your company DNA is free-to-all social based on ad monetization, with a large bet on metaverse / AR / experiential social compute being next. You aren't a trusted corporate partner for anything but gatekeeping your immense community through ad sales.
And, it's clear you a) have some of the most interesting private social data in the world, including photos and DMs and texts, and b) this AI thing is huge.
A play that doesn't f with your existing corporate structure too much is to build this stuff, give it away, keep publishing, build your AI team internally, and see where it takes you.
This isn't the only play, but I think it's reasonable. It's pretty clear large enterprises are going to need their own, internally built / owned, Foundation models to be competitive in a bunch of arenas in the next decade. In this case, if FB can get a little mindshare, keep the conversation going, and as a sidenote, be a disruptor by lowering Azure/OpenAI revs with open releases at-the-edge, that's probably a strategy win.
If I were in charge of AI strategy at FB, I'd probably double down more on generative AI, and I'd be working hard on realtime multimodal stuff -- their recent very large multimodal speech to text in multiple languages work is good. If a team could eyeball realtime-ish video chat with translations, that would be something the platform has a natural advantage in pushing out. Generative hits existing customers, and metaverse asset creation, which is going to experience radical changes in costs and productivity over the next few years, and impact Oculus 100% no matter what anybody wishes were true.
That’s interesting. I tend to lump FB, Amazon, Google, and MS in my head when thinking about the tech giants, but you’re right, FB is the only one of those not offering a commercial platform. For them, building out the capabilities of the LLMs is something to be done in the open with community involvement, because they’re not going to monetize the models themselves.
They’re also getting a fantastic amount of press from all this, which is good for attracting talent and helping improve their image, at least among the nerd set.
FB is unlike the other BigTech(tm) since Zuck never sold out and has a controlling equity stake. Amazon, Google, and MS are all controlled by and beholden to institutional investors.
FB can release these for no other reason than Zuck’s ego or desire to kill OpenAI. Same deal as him going off on a tangent with the Metaverse thing.
Given that OpenAI finished training GPT4 a year ago, and no models today (including these) can beat it, I highly doubt anyone is capable of killing Open AI in the near future. I’m guessing by the time GPT5 is out, someone will finally catch up with GPT4.
Depends what you mean by platform and depends what you mean by FB. If by FB you mean Meta, they have also https://www.workplace.com/ (which is like an internal facebook), instagram, whatsapp and some others. Integration of LLMs technology in those "platform" might give them some advantage.
Right, but they’re not competing directly on offering the LLM - they benefit from having a better LLM as a feature, but their value add is elsewhere in the product.
Meta absolutely could not overcome the barriers to entry and technical mismatch for any sort of traditional IAS style product, and it would be foolish for them to try. They might be able to pull off some sort of next generation Heroku style service aimed at smaller shops with built in facebook integration and authn/z management, but that's tangential.
I don’t believe they’re going for the same hosted monetization as Oracle or Google. I’m sure they’ll play around with assistant AIs but you can imagine them leveraging their graph and data for this.
Who is better positioned to answer a question like, “What should I get my friend Sophia for her birthday?” Facebook/Instagram already have huge volumes of data to specifically target ads. They can feed those into a chat interface pretty easily.
Customers would then buy per impression by describing their product and trusting Facebook to place it correctly. They already do this today, it’s just a different medium.
> Who is better positioned to answer a question like, “What should I get my friend Sophia for her birthday?” Facebook/Instagram already have huge volumes of data to specifically target ads. They can feed those into a chat interface pretty easily.
Interesting idea but sounds risky and intrusive in practice.
I think this suggestion lacks subtlety. More likely, around the time leading up to Sophia's birthday, you will see more ads for things (maybe even gift idea ads) that just so happen to be things Sophia would love (at least, according to their data).
Yes. But it also needs a very different org structure to support that. Their internal infra from what I heard is dated (monolithic PHP binary deployment, no federated authorization management etc.). It is doable (FAIR's org structure was very different in the first a few years), but would also be a distraction for a long time.
I would add that having open source gen AI will enable the creation of content for metaverse / AR / VR, which will improve the chances that all of that will take off.
Right, exactly this. Ratcheting the costs down two orders of magnitude in both dollar and expertise/human costs is going to make huge changes. You better believe FB is thinking about this hard.
Yeah, there are tons of opportunities for AI to do something with facebooks private user data and sell new services. For users to create engagement - and for ad companies to get very good targeted ads delivered. It is of course a challenge, to update the models on the fly, to include the latest private data, but then you can tailor an ad, that has subtil references to the latest shared wishes of the user. Probably quite effective.
So for now they mainly need top talent, to make some of it work. And open source is the best bet, for creating a ecosystem they can control and get talents who already trained on their tools. And they loose allmost nothing, because yes, they ain't in the cloud buisness.
So I will continue to not use facebook. But the models I will try.
You ought to think about using Oracle Cloud for your next LLM/GPU project, because they sell access to A100/H100s for cheap and they actually have them in stock!
Commercially it's not clear if there is a reliable "ahead", I'd be surprised if copyright lawsuits don't start hitting MS/OAI when publishers wake up and if you take out that training data where does it leave their models?
Countries putting copyright above AI progress will just fall behind. It's one thing to demand no exact replication of copyrighted content, another to forbid training on copyrighted works. Ideas were not supposed to be under copyright, only expression, from what I remember.
The argument that copyright abuse is required for "AI progress" is sus. It is required for quick easy buck to be made by the likes of Microsoft-- that I agree...
Clearly the research team at Meta knows the domain as well anybody, has access to a data trove as large as anybody and their distribution capability is as large scale as anyone's.
If their choice right now is not to try to overtly monetize these capabilities but instead commoditize and "democratize" what others are offering it suggests they think that a proprietary monetization route is not available to them. In other words they do not leave money on the table. They think that (at least right now) there is no money on the table that they can get to.
Rather than remaining quiet and isolated, the best alternative - their conjectured thinking goes - is to show up as they do, buying up good will with various stakeholders, maintaining mindshare internally and externally etc.
Assuming that the above reading is correct it still leaves various options as to why they may have come to that conclusion: For example reasoning about the future of this sector they might be thinking that there is no real technical moat and they simply accelerate that reality to gain some brownie points.
It may be also idiosyncratic reasons specific to their own business model (data privacy challenges and how any AI monetization will mesh with all that). The drawback of being the elephant in the room is that there is not much room to move.
The nature of their long game depends on which of the decision branches carries more weight. Maybe it is wait-and-see until others clear up the regulatory hurdles. Or keep the engines running until the real and irreducible added value of LLM algos and the like becomes clear.
There really is no technical moat. Any new architectures are going to be published because that's 100% the culture and AI folks won't work somewhere where that's not true. Training details/hyperparameters/model "build-ons" aren't published but those are a very weak moat.
The only moat that is meaningful is data and they've got that more than any other player save maybe google. Publishing models doesn't erode that moat, and it's not going anywhere as long as facebook/whatsapp/instagram rule "interactive" social.
Well Facebook is a walled garden, perhaps the board hopes free highly capable LLMs will continue degrading the internet outside those walls thus acting as a moat for their money printer.
Retention project to keep their top ML/AI staff engaged and not straying away?
Working towards NLU that can solve content moderation once and for all? Contrast with tiktok which is clearly using word filters that are easily worked around with phrases like "un-alived" or "corn".
They want to replace influencers and your friends with chatbots and keep you scrolling through an infinite feed of ads and AI generated content?
There has been some shuffling of seats but from what I am hearing FAIR is the best setup as far as staffing and funding that they have been in quite some time. Mark is pivoting hard to stay competitive in AI and is providing the resourcing to do so, the results speak for themselves.
As a business strategy I would see it as preventing themselves from being hemmed in from the market leaders. By open sourcing and raising the bar for commodity AI, they get to crowd source improvement to their models and techniques to get ahead in their own uses by co-opting open source improvement. I would say to sage this is working amazingly well - the amount of interest around open source models from meta is immense. I also think, as do I, the majority of uses in the future will be from fine tuned RAG capable models embedded in devices, not pangalactic planet sized computers running septillion parameter models. Llamacpp is a perfect illustration of where that’s working.
We followed a similar model under more duress at Netscape. When you use Firefox that’s the fruit of that effort. It didn’t save Netscape, but Meta has a better and more diversified revenue base.
I watched a good talk from Yann LeCun who is Chief AI Scientist at Meta, and he explained that the thinking is that open source AI models will be the long-term winner, so it's best for them to work in that arena.
Yann wants to cement his position as a leader in AI and while he clearly does not appreciate LLMS at all, he realizes that he needs to make waves in this area.
Mark needs a generative product and has invested tremendously in the infrastructure for AI in general (for recommendation). He needs researchers to use that infrastructure to create a generative product(s).
Yann sees this going on, realizes that he has a very powerful (research+recruiting) position and tells mark that he will only sign on if Meta gives away a good deal of research and Mark concedes, with the condition that he wants his generative product by end of 2023 or start of 2024.
It’s not just ego. It’s accelerationism. Giving this stuff away from free is probably going to accelerate AI a decade faster than if it was kept locked up behind closed doors at Google, OpenAI, etc. And if you’re an optimist then that actually might make the world a better place much faster.
I wish that Meta would release models like SeamlessM4T[0] under the same license as llama2, or an even better one. I don't understand the rationale for keeping it under a completely non-commercial license, but I agree that is better than not releasing anything at all.
There seem to be opportunities for people to use technology like SeamlessM4T to improve lives, if it were licensed correctly, and I don't see how any commercial offering from smaller companies would compete with anything that Meta does. Last I checked, Meta has never offered any kind of translation or transcription API that third parties can use.
Whisper is licensed more permissively and does a great job with speech to text in some languages, and it can translate to English only. However, it can't translate between a large number of languages, and it doesn't have any kind of text to speech or speech to speech capabilities. SeamlessM4T seems like it would be an all-around upgrade.
Yeah - different projects have different goals and licenses aren't one size fits all. Depending on the project, type of technology, goals, etc.. we will select or even develop the right license that aligns with those goals. Hope this helps :)
Facebook Connect is what used to be called Oculus Connect. Kinda their equivalent of Apple's WWDC, I guess. It's when and where the Quest 3 will be officially unveiled in full, for example.
Probably just talent acquisition. As Google and OpenAI start sharing and publishing less, they become less attractive to scientists. No scientist wants to fall into a black hole and not publish for 8 years.
Exactly. The Google and OpenAI engineers who published their groundbreaking research 5 years ago are now rockstars. Those who create great research but can't share it often get frustrated.
The problem is also companies bragging about AI, but not releasing anything behind (like most of the recent Google announcements).
If nobody except the researcher can reproduce an AI paper, and there is no source-code, and no demos that the public can access, then it's almost like if it doesn't exist.
I wouldn't want to work in a company that would throw away my research and just use it for PR purposes.
Meta has a clear channel to leverage generative AI in profitable ways in their ads. At some point in the probably not so far future, everybody's going to have custom ads generated for them that are optimized to get that particular person to click/buy/etc. Those will convert well, and the better ads convert, the more businesses will be willing to pay Meta for a given ad.
This compares favorably with Google, which is as likely to cannibalize its search business with generative AI as to create new value for itself.
Thus, for all the gen AI stuff like this, for which Meta doesn't have an obvious path to commercialization, it makes sense to release it publicly. They get plenty of benefits from this - for one, engineers (and smart people generally) who are working on really complex problems like to be able to talk about the work they're doing. If you're picking between jobs at Meta and Google, the fact that Meta's going to release your stuff publicly might well be the deciding factor.
I would also argue that there's an economic incentive. Right now, being seen as an AI company is definitely a positive for your multiple. I think the movement of Meta's stock price over the last 12 months relative to their change in profit and revenue is certainly driven in part by the perception that they're a leader in AI.
It makes sense to me that Facebook is releasing these models similarly to the way that Google releases Android OS. Google's advertising model benefits from as many people being online as possible and their mobile operating system furthers that aim. Similarly, Facebook's advertising model benefits from having loads of content being generated to then be posted in their various products' feeds.
> Does anyone have a good explanation for Meta's strategy with AI?
Yes. I said it many times. Meta is already at the finish line in the AI race to zero. All the other cloud-based AI models cannot increase their prices given that a $0 free AI model is available to be self-hosted or used on-device for private / compliance reasons.
Cloud-based AI models cannot afford to compete with free or close to free. It costs Meta close to nothing to release a readily available $0 AI model which is good enough for most use-cases that ChatGPT has already done.
> The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?
Mostly benefits the PyTorch ecosystem which Meta has an active community around it.
vessenes and rvz kind of sum the idea I think they're going for to me.
AI has no moat, but many players are in denial about this still. Microsoft and other might have tight enough control they can use a product dumping strategy to get people dependent upon their implementation such they can start charging, but that isn't a delusion Meta can have.
That max revenue license they used with the models seemed fairly clever to me. It will seed the environment with players that base their product on Meta tech in return for them being born with a poison pill preventing their use by big players (other than Meta) buying them. This is a long term play that may not really work but it creates the potential for big opportunities. And even if it doesn't work out, denying easy wins for their powerful competitors might be worth the price on its own.
I posit it is similar to how Adobe lets students pirate Photoshop, because when they join the workforce that is what they know and need their employers to buy Adobe services, which for corporate is very expensive.
Meta by democratizing AI access is generating more capable developers which will make the Metaverse a reality, where FB leads. They have already realized they have a losing gambit with Google, Apple, Microsoft (also X?) having an antagonistic monopoly against Meta product advancement
the only beneficiary of this are the hardware vendors.. nvidia and amd. and startups which get these foundation models for free.
because language models are a complementary product, and the complement must be commoditized as a strategy.
I see AMD as a bigger beneficiary, since, very soon, amd will equal nvidia for inference and fine-tuning, but amd has a long way to go to equal in foundation model training.
> and startups which get these foundation models for free.
It's licensed non-commercially, so I'm not sure what those startups stand to gain.
> since, very soon, amd will equal nvidia for inference and fine-tuning
Source? If you're referring to Olive, it is indeed impressive but also has caveats:
1. It is just as proprietary as CUDA or CoreML.
2. You need a copy of Windows and licensed DirectX to use those optimizations.
3. AMD only matches Nvidia's inferencing performance when comparing Olive to Pytorch. Olive-to-Olive comparisons will still reflect an Nvidia lead.
I don't think AMD has the capability to equal Nvidia in the short-term. It will take longtime software investments from across the industry to shake Nvidia's yoke.
Microsoft is a hosting partner, there's an Azure service for hosted private LLaMa inference for business. Being a go-to hosting provider for SoTA AI is of course a very good thing for Microsoft.
The only thing I've been able to think is they're trying to commoditize this new category before Microsoft and Google can lock it in, but where to from there? Is it just to block the others from a new revenue source, or do they have a longer game they're playing?