Skip to content

Instantly share code, notes, and snippets.

@cedrickchee

cedrickchee/rebutting_devin_ai_claim.md

Last active April 13, 2024 13:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cedrickchee/588a55cbcaeb2d0faba694ae1fa560dd to your computer and use it in GitHub Desktop.
Save cedrickchee/588a55cbcaeb2d0faba694ae1fa560dd to your computer and use it in GitHub Desktop.
Download ZIP
Rebutting Devin: "First AI Software Engineer" Claim is Not True
Raw
rebutting_devin_ai_claim.md

Rebutting Devin: "First AI Software Engineer" Claim is Not True

A human software engineer, Carl (aka. "InternetOfBugs") looked closer and exposed Cognition Labs's Devin "First AI Software Engineer" Upwork lie. InternetOfBugs is an AI enthusiast and uses coding AI himself. InternetOfBugs is not anti-AI, but anti-hype.

Debunking Devin: "First AI Software Engineer" Upwork lie exposed!

The company lied and said that their video showed Devin completing and getting paid for freelance jobs on Upwork, but it didn't show that at all. On the whole that's not surprising given the current state of Generative AI, and I wouldn't be bothering to debunk it, except:

  1. The company lied about what Devin could do in the video description, and
  2. a lot of people uncritically parroted the lie all over the Internet, and
  3. That caused a lot of non-technical people to believe that AI might replace programmers soon.

A convincing rebuttal.

The demo video of it completing an Upwork task is quite far from the truth of what it actually did.

It did not do what the Upwork task specifically asked. The Upwork task asked for instructions on how to do this on AWS. Devin generated nothing of the sort. Because it was not even given that input (prompt).

Devin is nowhere near 100% automated. But we always knew that given the current state of Generative AI.

Devin looked to me it's far more marketing and hype than reality.

Devin actually does some cool task in the process, but over-hyping or fabricating the results of the work, actually does a disservice to the overall development of the AI agent.

Generative AI is cool, but demos around it should be honest and transparent. Have they learning anything from Google's Gemini blunder?

Carl's plea:

Please tell people the truth about your products. If you're a journalist, blogger or influencer, please don't blindly repeat and amplify things that people say on the Internet without doing some due diligence, without looking to see if they're actually true. If you don't understand if they're true own your own, ask someone or don't amplify it. Please for the love of all that's holy, be skeptical of everything you see on the Internet, especially anything that might possibly be AI related.

I guess InternetOfBugs is inspired by Gary Marcus. Haha...

"Trust, but verify". Seek the truth.

#busting-hype, #grounding-hype

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment