mikethompson:
I've found the AIs to be amazing. Staggering really. But I do regard them like a 12 year old savant. Most problems come from them not having enough information. There's stuff in your head that they don't know.
Hence being spec driven. I have to iterate before I ask them to do anything substantive.
Spec driven just means "prompt engineering driven"
The imperative code that didibus found was because I never said "prefer a functional solution unless it impacts performance critical code". The moment I did, we were good.
didibus:
I'm going to make two comments. But I don't want them to feel like criticism, because I actually applaud this attempt. We need to experiment with stuff like this, how else would we know the limits and possibilities otherwise.
In my opinion, the entire 60k is slop. Slop is the over abundance of mostly good code, text, images, etc. It would take more than the 5 days it took to AI generate them to actually have someone assess if it's good, has issues, and how it could have been done better. And slop is when someone is telling you how awesome that enlarged artifact they generated is, but they didn't spend the necessary time and effort to assess it's truly good, of course not or it would eat into all the time savings, and instead are pushing that onus on you.
I think this is the reality we're all reckoning with. In one year we will have 20+ different AI generated Clojure web frameworks. How do we navigate all that? When you're flooded with knockoffs, who curates?
And I don't say that to mean it's bad. There's a quality to quantity. We might all prefer more features even if they're all cheaper made. But it does cause fatigue, and it will require us to address new problems, how do you develop good taste, how do you identify good from bad?
How do you imagine a user uses reframe2?
Do I consume the implementation as a library and have my AI use it to write an app?
Do I fork the spec and add to it my own app specifications and then have my AI generate the full app?
There's a part of me that's skeptical we need reframe2. My AI can directly implement the full app, the framework it should use is embedded in its learned weights. Aren't frameworks of the future prompt frameworks?
mikethompson:
I guess all I know is that I appear to have created re-frame-2 with the design I always wanted. The one I've thought about for 10 years, but never had time to get too. Perhaps that design is slop, we'll see, but if it is then that's on me not the Ais. (edited) [1:39 AM]My guess is that there is a few man-years in implementing it by hand. [1:41 AM]And given there a bunch of re-frame clones out there (which I like!), JS-land, etc, I figured I'd give them the ability to roll their own.
...
didibus:
Again want to reiterate, I support this endeavor 100%
all I know is that I appear to have created re-frame-2
"appear" is where a lot of the problem comes from. It's where things get hard with AI in my experience. Everything appears good, professional, bug free, and that it understood the spec. Going from appearing as such, to being sure of it is what's missing. I think it's where a lot of the people pushing back on AI native projects come from. In the past, you could announce something grand, and it would carry more weight, because someone spent a lot of time behind it, it was worth a look. Now I think the bar is raised, where is the example of a full complex app using it, two more simpler apps, and so on.
At my work, it's been where the challenge has lived as well. A lot of things get done faster, but once they reach integration and QA, more issues are found, and less people know how to solve for them. It's made it projects don't always reach customers sooner, depending how much you are okay pushing issues to your users or not.
Perhaps that design is slop, we'll see, but if it is then that's on me not the Ais
In this case, since you spent so much time on the design, the ideas, I wouldn't say that's slop. The implementation is though. I mean, you probably didn't read it all yourself. At this point, slop is like assembly, would you want to look at compiled assembly? But if someone hand-crafted assembly I'd maybe be interested to look at it. The issue here is, we don't know if the model understood the spec and implemented it as it should. And we don't have that human confidence of someone having written it who knew the spec.
mikethompson:
@didibus I appreciate the support. (And the bug report :-))
And the rest is all valid concerns. I've gone though various cycles with AI myself. Some of them unpleasant. Let's see how this plays out.
Possible outcomes:
- Its all annoying, fragile slop
- It delivers some benefits, but has uncomfortable trade-offs.
- AI is unusually effective in this one particular usecase because I'd spent 10 years thinking carefully about what I wanted and had, say, 20 pages of iterated notes before I started.
At this point, I believe we are somewhere between 1.5 and 2.5 (edited)