I think the part that annoys me the most is the hype around it, just like blockchain. People who don’t know any better claiming magic.

We’ve had a few sequence specific architectures over the years. GRU, LSTM and now Transformers. They were all better than the last at the task of sequence specific transformations, and at least for the last one the specific task was language translation. We eventually figured out these guys have a bit of clairvoyance too, they could make accurate predictions based on past data, or at least accurate enough to bet on, and you can bet traders of various stripes have already made billions off that fact. I’ve even seen a transformer based weather model. It did OK, but transformers are better at language.

And that’s all it is! ChatGPT is a Transformer in the predictive stance. It looks at a transcript of a conversation and thinks what a human is most likely to say next. It’s a very complex transformation of historical data. If you give it the exact same transcript, it gives the exact same answer. It is in the literally mathematically rigorous sense entirely incapable of an original thought. Any perceived sentience is a shadow of OpenAI’s army of annotators or the corpus it was trained on, and I have a hard time assigning sentience to tomorrow’s forecast, which may well have used similar technology. It’s just an ultra fancy search engine index.

Anyways, that’s my rant done I guess. Call it a cynical engineer’s opinion. To be clear I think it’s a fantastic and useful technology, and it WILL change how we interact with machines. It can do fancy things with the combination of “shell” code driving it’s UI like multi-step “agents” or running code, and I actually hope OpenAI extends it far into the future, but I sincerely think any form of AGI will be something entirely different to LLMs, or at least they’ll only form a small part of it as an encoder/decoder for it’s thoughts.

EDIT: Added some paragraph spacing. Sorry, went into a more broad AI rant rather than staying on topic about coding specifically lol

Yeah, in my mind I thought of it more as a “why not” in addition to vision. Like why make it only as capable as the humans its trying to replace when it can have even more data to work with? Probably would have been even more expensive though

True that, he did good on that front for a while though. He got too confident

I remember hearing a while back that Musk made an executive decision at Tesla to not use LIDAR. I thought: “That’s a stupid decision. At least invest in making it better if you think its not sufficient” and I had a quite negative view of his engineering abilities ever since. Seeing as a Tesla can be fooled by a projector these days, I’m willing to die on that hill. I will admit that he is an exceptional businessman, most people would piss away a fortune if given one, but an engineer he is not, not by a loooooooong way.