reply to: https://news.ycombinator.com/item?id=11245209
I think this you can view this from a different point of view.
In my opinion this is because what people think software is.
So if you see software as code which expresses what you want, the question is what do you do when it does not do what you want wrong, or do you want something additionally.
So software really is our desire for some specific thing. But it is also a tool which can express arbitrary things. So its a mirror which reflects back on us to discover our real intentions and desires.
Eventually its more of a conversation in which you expand and direct your intentions. And programming or software is just one way to do that.
I think eventually AI will be able to deliver such reflecting conversations to us, the question would be which medium (hardware, operating system, programming language) will it use.
I do not think it will use building blocks (hardware, operating system, programming language) created by humans. Because those are to incomplete and arbitrary.
remember the building blocks allow for plenty of room to allow bootstrapping on multiple levels. An AI could create blocks to create a solution that is so very simple we can't even imagine, yet is unthinkable for humans right now.