The democratization of large language models is key for the healthy advancement of AI, as I’ve previously argued. Now, the AI community is starting to make it a reality through open-source AI models— code, weights, and datasets — for training and open platforms for inference. Notable institutions like EleutherAI or Hugging Face and projects like BigScience (they’re training BLOOM, a 176B open multilingual model) are growing in prominence due to their unique role in this regard.
Even Meta — often despised as being uncaring of the effects of its tech — has open-sourced a pre-trained large language model the size of GPT-3, called OPT (Open Pretrained Transformer), as well as the code to train it. And just three days ago, Google announced it’s also entering the open-source space by sharing the trained weights of the Switch Transformer models, including the Switch-C, a 1.6T-parameter sparse model.
Delegating the power to decide who uses or not these incredible tools to a few powerful and resourceful companies like OpenAI was not sustainable (it’s worth remembering that although Meta and Google are open-sourcing a couple of models, most of their AI remains private).
It’s ironic that is precisely OpenAI the latest to feel the consequences of proprietary AI. DALL·E mini, a Hugging Face-hosted open-source cousin of DALL·E 2, is now significantly more viral than the original. To give you some numbers, DALL·E 2 generated 3 million images from April 6th to May 18th whereas DALL·E mini is generating 50 million a day. That’s a 700x increase. Although OpenAI is now adding 1000 users per week, they can’t compete against open-source, even if we factor in the obvious difference in quality (DALL·E 2 produces way better images than DALL·E mini).
It now seems clear that the advantages of breakthrough AI, like DALL·E 2 or GPT-3, should benefit and be available to all. However, defending open-source AI isn’t as simple as it seems. And today I bring you a very recent case that reflects this very fact.
I’ve considered not publishing this article because I could bring unwanted attention to this specific example just by writing about it. While the thesis I’m defending is precisely to help avoid cases like this in the future, publishing it could achieve the contrary. On second thought — and the reason why I’ve finally decided to publish it — I realize that while the downsides will be limited to this particular case, the benefits of raising awareness will be evergreen (if I achieve the goal of making you, the reader, think about this).
Keep reading with a 7-day free trial
Subscribe to The Algorithmic Bridge to keep reading this post and get 7 days of free access to the full post archives.