Hi everyone! I want to do something different this Sunday.
It’s been 3 months since I did my first AMA. We were 1,500 subscribers back then.
We’re very close to 3,500 now—thank you so much for your support!!
I’ll be doing another AMA today: You can ask me about AI (current trends, predictions, companies, models, etc.), about writing, Substack, my background, future plans, thoughts on any topic… whatever!
So, bring in the questions! (I’ll be around for the next few hours)
Oct 16, 2022·edited Oct 16, 2022Liked by Alberto Romero
It feels like things have exploded this year (especially the past few months) when it comes to text-to-image / text-to-video consumer-level releases and announcements.
Question: Is this likely to be an odd blip in an otherwise more gradual pace of AI progress (i.e. a few similar competing models being completed at roughly the same time, making it seem like an avalanche of new stuff)? Or is this an accelerating trend that might hold/increase exponentially as more companies get in the game across more disciplines?
I realize this is a very speculative question, but as someone who is quite removed from all the ongoing behind-the-scenes AI work, it is sure hard to keep track or project where we're heading.
I have a question regarding writing! When did you start writing? Was it always Substack or did you start somewhere else? And how long did it take to build a loyal follower base for your newsletter?
In your opinion what is the most accurate speech recognition system for dictation and transcription currently available in Spanish for a journalist? Can you share some comparative data?
With respect to level 5 autonomous driving , do you you think it could be solve with more data or edge cases or do you think we need done breakthroughs in AI itself.
Also , how useful you think training in simulation would be? Any limitations in these simulation trainings?
Do you see any efforts in openness in industrialised applications of ML/AI. I'm looking for a hugging face for the industry (including training dataset)?
Oct 16, 2022·edited Oct 16, 2022Liked by Alberto Romero
Where are the open source projects for building/improving training datasets for LaLM? As far as I can see, The Pile (EleutherAI) is SotA and it appears to have been frozen at the beginning of 2021. I keep hearing how important the training datasets are for these models yet I can't find any projects that are focused on building/improving what is available. Do you know of any projects, or where people discuss this topic? I've spent a short amount of time on EleutherAI's Discord but all discussion appears to be on the model code and none on The Pile but I may have missed it.
What can you say about NLP? GPT3 was huge at the time. But, it seems it does not perform equally on languages other than English. My goals are on Domain Specific Corpora, so fine tuning is a must. On the other side of large models, we are seeing newcomers. Pathways Language Model (PaLM) is one of them. I do not know much about it, but it seems interesting enough. I have invested quite a few hours on GPT3. Since my target language is not English, should I move on?
What do you think is a bigger bottleneck to a general-use home robot (e.g. Optimus) - the robotics side of things or the AI side of things? Any guess when we'll get one?
It feels like things have exploded this year (especially the past few months) when it comes to text-to-image / text-to-video consumer-level releases and announcements.
Question: Is this likely to be an odd blip in an otherwise more gradual pace of AI progress (i.e. a few similar competing models being completed at roughly the same time, making it seem like an avalanche of new stuff)? Or is this an accelerating trend that might hold/increase exponentially as more companies get in the game across more disciplines?
I realize this is a very speculative question, but as someone who is quite removed from all the ongoing behind-the-scenes AI work, it is sure hard to keep track or project where we're heading.
Thank you for your always fascinating takes!
Anything to achieve anomaly detection, even with little training data, predictive maintenance and autonomous operation.
When you follow what is shared around AI it is mainly about the fields you mentioned (language, images, social...)
I know that the industry has difficulties sharing data to allow rapid growth as for AI which are trained on images or language.
I'm keen to learn if there is anything on the horizon what may could change this.
First of all - love your work!
I have a question regarding writing! When did you start writing? Was it always Substack or did you start somewhere else? And how long did it take to build a loyal follower base for your newsletter?
In your opinion what is the most accurate speech recognition system for dictation and transcription currently available in Spanish for a journalist? Can you share some comparative data?
With respect to level 5 autonomous driving , do you you think it could be solve with more data or edge cases or do you think we need done breakthroughs in AI itself.
Also , how useful you think training in simulation would be? Any limitations in these simulation trainings?
Do you see any efforts in openness in industrialised applications of ML/AI. I'm looking for a hugging face for the industry (including training dataset)?
Where do you see the state of the art in AI in the next 10-20 years? Worst and best case.
Where are the open source projects for building/improving training datasets for LaLM? As far as I can see, The Pile (EleutherAI) is SotA and it appears to have been frozen at the beginning of 2021. I keep hearing how important the training datasets are for these models yet I can't find any projects that are focused on building/improving what is available. Do you know of any projects, or where people discuss this topic? I've spent a short amount of time on EleutherAI's Discord but all discussion appears to be on the model code and none on The Pile but I may have missed it.
What can you say about NLP? GPT3 was huge at the time. But, it seems it does not perform equally on languages other than English. My goals are on Domain Specific Corpora, so fine tuning is a must. On the other side of large models, we are seeing newcomers. Pathways Language Model (PaLM) is one of them. I do not know much about it, but it seems interesting enough. I have invested quite a few hours on GPT3. Since my target language is not English, should I move on?
What do you think is a bigger bottleneck to a general-use home robot (e.g. Optimus) - the robotics side of things or the AI side of things? Any guess when we'll get one?
-Alex