Regarding regulation, let's be clear that regulation only applies to those willing to follow the law. We should never assume that regulation can make AI safe. Laws in general are kind of like the lock on your front door. The lock keeps your nosy neighbors out of your house, but it's worthless against anyone willing to break a window.
Great writeup, but I should note that Midjourney is 10000% training on top tier artists, and to stunning results.
If "Respecting artists" is Stability's main motive, then we must ask: Why isn't Midjourney pressured to do the same?
I see several reasons.
1. Midjourney is completely dependent on subscriber income, so it has to please its paying retail consumers. It does not take outside investment from my observations. Eliminating the ability to do high quality art will be a deathblow for the business, so they'd rather take the legal risk.
Stability on the other hand, is currently running on VC money, and in the future, offering model-customization services to other companies. Therefore it has to please investors and other reputation conscious businesses, so much more averse to legal and reputational concerns.
2. Midjourney is closed source AND heavily community moderated, making it much less of a target to regulators. Stability being open sourced makes it a terror to politicians, who fear such a model being flinged around for infinite deepfakes, with no way to 'cease-and-desist' them, so therefore they must target stability directly.
Incidentally, Midjourney can implement prompt level filtering for NSFW content, therefore it feels free to train on NSFW data. Stability being open sourced cannot possibly moderate the prompts, so has to do training-level filtering, which much worse impacts on end-image quality.
The drama of SD2.0, is not merely about whether artist data should be included or not. But also about the future of open vs closed source models dominating the market. The previous hope is that everyone can get free access to good open source models that can compete with closed source ones.
Now, it appears that closed source business models will dominate, because less sensitive to regulatory pressure and censorship.
Emad states that SD2.0 will serve as a clean base for fine-tuning the model for more specific uses (Adding back art and NSFW), but that's an expensive training process that only companies can afford. NovelAI is the most famous example of SD finetuning, they are closed source and charge subscriptions to access the model (Their version 1 model got leaked, but their version 2 won't)
Although I dislike the removal of NSFW content (it restricts freedom and I have no reasons to consider it a good thing), I am happy that the new models remove the names of specific people and artists because that makes the technology less easy to criticise in terms of copyright while not actually removing anything from its training data. This means that its results are more of its own, rather than imitating anyone in particular and potentially making them angry. And if one still does want to imitate a particular style, they can still fine-tune the training data to make this possible (which is often necessary anyways if the style one wants does not have many examples in the dataset). For my own use case, of either imitating my own style (which I am still developing) or perhaps doing style transfer to some generic style like "anime", this does not hurt it at all.
If one wants AI to remain powerful and able to make amazing images, I think it is a good idea to support changes like this because it could, hopefully, reduce criticism around the technology and make it less likely people create laws against it. If you want AI to have a certain cool style, you can still custom-train it to do so. Using custom images, ideally ones which one has copyright permission to use, in order to get the AI to learn a style, I think, is a much better method than copying the style of some random artist who may or may not like you doing that. And most importantly, it can still enable art to be super super cheap! Imagine an AI company, commissioning an art in a certain style, and having a custom, paid (but still cheap!) model that generates things in that signature style! Or as an individual artist, having a custom model that either creates art for you in your style, or creates a base that you can polish the details of, as an extremely cheap tier for routine, high-quantity images. I want to see a world, where custom art can be everywhere! And not being able to use "Greg Rutkowski" does not hinder that goal at all, while hopefully making criticism that could end it all a bit less likely.
That said, I do think it would be nice if artists and people could opt-in to their names being included in the data, if they wanted to be in it. It could potentially be good advertising, if prompting using your name resulted in very good images.
Regarding regulation, let's be clear that regulation only applies to those willing to follow the law. We should never assume that regulation can make AI safe. Laws in general are kind of like the lock on your front door. The lock keeps your nosy neighbors out of your house, but it's worthless against anyone willing to break a window.
Great writeup, but I should note that Midjourney is 10000% training on top tier artists, and to stunning results.
If "Respecting artists" is Stability's main motive, then we must ask: Why isn't Midjourney pressured to do the same?
I see several reasons.
1. Midjourney is completely dependent on subscriber income, so it has to please its paying retail consumers. It does not take outside investment from my observations. Eliminating the ability to do high quality art will be a deathblow for the business, so they'd rather take the legal risk.
Stability on the other hand, is currently running on VC money, and in the future, offering model-customization services to other companies. Therefore it has to please investors and other reputation conscious businesses, so much more averse to legal and reputational concerns.
2. Midjourney is closed source AND heavily community moderated, making it much less of a target to regulators. Stability being open sourced makes it a terror to politicians, who fear such a model being flinged around for infinite deepfakes, with no way to 'cease-and-desist' them, so therefore they must target stability directly.
Incidentally, Midjourney can implement prompt level filtering for NSFW content, therefore it feels free to train on NSFW data. Stability being open sourced cannot possibly moderate the prompts, so has to do training-level filtering, which much worse impacts on end-image quality.
The drama of SD2.0, is not merely about whether artist data should be included or not. But also about the future of open vs closed source models dominating the market. The previous hope is that everyone can get free access to good open source models that can compete with closed source ones.
Now, it appears that closed source business models will dominate, because less sensitive to regulatory pressure and censorship.
Emad states that SD2.0 will serve as a clean base for fine-tuning the model for more specific uses (Adding back art and NSFW), but that's an expensive training process that only companies can afford. NovelAI is the most famous example of SD finetuning, they are closed source and charge subscriptions to access the model (Their version 1 model got leaked, but their version 2 won't)
Although I dislike the removal of NSFW content (it restricts freedom and I have no reasons to consider it a good thing), I am happy that the new models remove the names of specific people and artists because that makes the technology less easy to criticise in terms of copyright while not actually removing anything from its training data. This means that its results are more of its own, rather than imitating anyone in particular and potentially making them angry. And if one still does want to imitate a particular style, they can still fine-tune the training data to make this possible (which is often necessary anyways if the style one wants does not have many examples in the dataset). For my own use case, of either imitating my own style (which I am still developing) or perhaps doing style transfer to some generic style like "anime", this does not hurt it at all.
If one wants AI to remain powerful and able to make amazing images, I think it is a good idea to support changes like this because it could, hopefully, reduce criticism around the technology and make it less likely people create laws against it. If you want AI to have a certain cool style, you can still custom-train it to do so. Using custom images, ideally ones which one has copyright permission to use, in order to get the AI to learn a style, I think, is a much better method than copying the style of some random artist who may or may not like you doing that. And most importantly, it can still enable art to be super super cheap! Imagine an AI company, commissioning an art in a certain style, and having a custom, paid (but still cheap!) model that generates things in that signature style! Or as an individual artist, having a custom model that either creates art for you in your style, or creates a base that you can polish the details of, as an extremely cheap tier for routine, high-quantity images. I want to see a world, where custom art can be everywhere! And not being able to use "Greg Rutkowski" does not hinder that goal at all, while hopefully making criticism that could end it all a bit less likely.
That said, I do think it would be nice if artists and people could opt-in to their names being included in the data, if they wanted to be in it. It could potentially be good advertising, if prompting using your name resulted in very good images.