Hi. I’m not an AI expert and had not done much ML hands on. I’m just wondering, in your opinion, whether the first party data that google et al holds actually confer significant advantages to them. If so , how significant?
These points make sense to me. The only innovation coming to mind that may change your thoughts around “The limits of on-device inference for LMs” and the supposed moat from big tech’s compute resources is decentralized cloud computing.
Storj is crushing it by competing with AWS’ Simple Storage Service and Microsoft Azure’s Blob storage on cost, scalability, security, fault tolerance, and even sustainability: https://www.storj.io/solutions/big-data. Then there are new entrants like Together getting funding to build a decentralized cloud for artificial intelligence: https://www.together.xyz/blog/seed-funding. These services allow the open source community to scale LLM training. Imagine what would happen if we all donated our spare compute to help train and operate an open source LLM competitor? I’d have my company contribute a section of its data center to such an initiative.
Regardless, I feel this competition conversation along with all the doomsday and AI governance rhetoric is just a distraction from the most pressing issue: wealth and power distribution. While large incumbent enterprises and countries compete, working class people have no control over their lives. AI’s gift is increasing our productivity and saving us our most valuable asset: time. Unfortunately, these benefits and profits from AI will not be distributed equitably. The wealth gap will only widen and I don’t see anyone doing anything about it.
For example, my argument may break if a big tech company buys those competing decentralized tech companies like they did with OpenAI and DeepMind. Power corrupts.
I think you're correct, but I also believe that OpenAI, Google, and Microsoft are concerned about the rise of open source models. I believe that Sam Altman's recent trip to congress where he begged them to create a regulatory agency that could issue licenses for LLMs above a certain number of parameters was a calculated attempt to discourage new entrants into the field because these licenses would presumably be very expensive to obtain and require a lot of bureaucratic steps to keep. I could be wrong about that, but at present I believe this desire for regulation is more about creating a government-enforced moat than it is about any concerns for what AI might do to society.
That’s very illuminating. Thanks. So the war is over? Or there never really was a war? Or there’s no prospect of any future war? These monopolies will simply be our principle drivers?
Interesting thoughts thanks. I was however lacking the actual explanation for why OS couldn't compete. I mean I get the points but I am not sure they are as ironclad as you seem to assume.
Obviously in the short term OpenAI is way ahead, but to stay ahead require that there isn't diminishing returns on new language models which I think is unclear at this point whether there will be.
Also the need for chips might actually put a natural damper on OpenAI's ability to implement their models and it might even be an advantage for models trying to do more with less regardless of them being inferior in quality.
However I admit thats is also pure speculation. :)
Thanks for writing this! I agree that open source AI models are not as good as proprietary models right now. However, I believe that open source models will continue to improve over time. Especially if you consider the last 12 months. Additionally, the gaps in factuality may not be as important for some use cases. Ultimately, open source AI models will be everywhere in some form, and they will not need to look or perform the same.
I'm not nearly as bearish as you on the promise of open source AI.
The revelations brought forth in this article are eye opening and illustrate the true nature of the generative AI and how it will turned out to be. In particular, the emergence of OpenAI offered a window of opportunity for the big tech companies like Google, Microsoft, Meta, and Nvidia to reassess their products because AI was inevitably becoming unresistable disruptor that would not have been underestimated. It is evident that generative AI seemed to cement the market leaders in different industries as it already backed up existing products. However, as you mentioned, it showed that the first creator and distributor of generative AI would control the market. Microsoft has taken over OpenAI by being the largest shareholder and this makes it get exclusive rights of the advanced technology. Similarly, with its Bard, Google has responded and it seems the battle for the AI market will not end soon due to the fact that the market is still new and it is gaining mainstream attention.
Meanwhile, check my latest article on AI and look at how I explored its strengths and Weaknesses through my one-on-one interaction.
For now it is hard to imagine that LLM development won’t be an arms race with unfathomable resource expenditures in the next couple of years.
That being said, it is equally hard to guess how many applications and use cases will really need the mattest and greatest models.
Think iPhones, WEIRD (western, educated, ...) people tend to have them but the majority of the planet in terms of numbers use different phones. It is plausible that in the future most LLMs won’t be cutting edge but specialized agents.
Also, bar patents or regulatory constraints there isn’t anything yet(!) in principle that can’t be done by a group of motivated people (side note: this is likely the reason why synthetic biology is having a hard time to advance as fast as people had hoped).
Hi. I’m not an AI expert and had not done much ML hands on. I’m just wondering, in your opinion, whether the first party data that google et al holds actually confer significant advantages to them. If so , how significant?
I liked the article and it's really well-written. I just wish I liked the conclusion it reaches!
But what about the fact that Python and other OS tooling is relied on by the ‘Incumbents’ ? Doesn’t that figure for a lot?
AI models MUST NOT be be "Open-Sourced" (Beer),
but training methods, error rates, data cleansing, filters and model spec should be.
AI models MUST be shared with Public Control Agency (in EU, as EU agency), and freely licensed for public utility
These points make sense to me. The only innovation coming to mind that may change your thoughts around “The limits of on-device inference for LMs” and the supposed moat from big tech’s compute resources is decentralized cloud computing.
Storj is crushing it by competing with AWS’ Simple Storage Service and Microsoft Azure’s Blob storage on cost, scalability, security, fault tolerance, and even sustainability: https://www.storj.io/solutions/big-data. Then there are new entrants like Together getting funding to build a decentralized cloud for artificial intelligence: https://www.together.xyz/blog/seed-funding. These services allow the open source community to scale LLM training. Imagine what would happen if we all donated our spare compute to help train and operate an open source LLM competitor? I’d have my company contribute a section of its data center to such an initiative.
Regardless, I feel this competition conversation along with all the doomsday and AI governance rhetoric is just a distraction from the most pressing issue: wealth and power distribution. While large incumbent enterprises and countries compete, working class people have no control over their lives. AI’s gift is increasing our productivity and saving us our most valuable asset: time. Unfortunately, these benefits and profits from AI will not be distributed equitably. The wealth gap will only widen and I don’t see anyone doing anything about it.
For example, my argument may break if a big tech company buys those competing decentralized tech companies like they did with OpenAI and DeepMind. Power corrupts.
I think you're correct, but I also believe that OpenAI, Google, and Microsoft are concerned about the rise of open source models. I believe that Sam Altman's recent trip to congress where he begged them to create a regulatory agency that could issue licenses for LLMs above a certain number of parameters was a calculated attempt to discourage new entrants into the field because these licenses would presumably be very expensive to obtain and require a lot of bureaucratic steps to keep. I could be wrong about that, but at present I believe this desire for regulation is more about creating a government-enforced moat than it is about any concerns for what AI might do to society.
That’s very illuminating. Thanks. So the war is over? Or there never really was a war? Or there’s no prospect of any future war? These monopolies will simply be our principle drivers?
Interesting thoughts thanks. I was however lacking the actual explanation for why OS couldn't compete. I mean I get the points but I am not sure they are as ironclad as you seem to assume.
Obviously in the short term OpenAI is way ahead, but to stay ahead require that there isn't diminishing returns on new language models which I think is unclear at this point whether there will be.
Also the need for chips might actually put a natural damper on OpenAI's ability to implement their models and it might even be an advantage for models trying to do more with less regardless of them being inferior in quality.
However I admit thats is also pure speculation. :)
Thanks for writing this! I agree that open source AI models are not as good as proprietary models right now. However, I believe that open source models will continue to improve over time. Especially if you consider the last 12 months. Additionally, the gaps in factuality may not be as important for some use cases. Ultimately, open source AI models will be everywhere in some form, and they will not need to look or perform the same.
I'm not nearly as bearish as you on the promise of open source AI.
The revelations brought forth in this article are eye opening and illustrate the true nature of the generative AI and how it will turned out to be. In particular, the emergence of OpenAI offered a window of opportunity for the big tech companies like Google, Microsoft, Meta, and Nvidia to reassess their products because AI was inevitably becoming unresistable disruptor that would not have been underestimated. It is evident that generative AI seemed to cement the market leaders in different industries as it already backed up existing products. However, as you mentioned, it showed that the first creator and distributor of generative AI would control the market. Microsoft has taken over OpenAI by being the largest shareholder and this makes it get exclusive rights of the advanced technology. Similarly, with its Bard, Google has responded and it seems the battle for the AI market will not end soon due to the fact that the market is still new and it is gaining mainstream attention.
Meanwhile, check my latest article on AI and look at how I explored its strengths and Weaknesses through my one-on-one interaction.
https://thestartupglobal.substack.com/p/my-encounter-with-ai-assisted-chatbot?sd=pf
For now it is hard to imagine that LLM development won’t be an arms race with unfathomable resource expenditures in the next couple of years.
That being said, it is equally hard to guess how many applications and use cases will really need the mattest and greatest models.
Think iPhones, WEIRD (western, educated, ...) people tend to have them but the majority of the planet in terms of numbers use different phones. It is plausible that in the future most LLMs won’t be cutting edge but specialized agents.
Also, bar patents or regulatory constraints there isn’t anything yet(!) in principle that can’t be done by a group of motivated people (side note: this is likely the reason why synthetic biology is having a hard time to advance as fast as people had hoped).