Today, I bring to you a thought-provoking concept: the amalgamation of technology and psychology, or more specifically, the application of System 1 and System 2 LLMs in AI programming. This idea was born from two main observations I made.
Firstly, there is the revolutionary technology of "embeddings" that has seen its usage skyrocket since 2022. It allows us to create a programmable memory for AI, thus eliminating the need to grapple with prompt engineering. I began to ponder if there could be a more efficient alternative to embeddings, one that utilizes fast and economical LLMs.
Secondly, the theory of System 1 and System 2 in psychology, which speaks about two different decision-making processes in our brain, caught my attention. System 1 corresponds to our intuitive, automatic, often subconscious thought process, like recognizing facial expressions. On the other hand, System 2 is deliberate, slower, and more logical, like solving complex math problems.
Inspired by this theory, I began to explore the possibility of using different sizes of LLMs in different stages of the programming journey. Much like how our brains switch between System 1 and System 2 depending on the complexity of the task, could we also employ smaller, faster LLMs for simpler programming tasks and larger, more powerful LLMs for complex tasks?
To simplify, can we use fast and slow LLMs in a way that mimics the cognitive processes of System 1 and System 2? This is the intriguing concept that I am currently investigating. By blending the realms of psychology and technology, I believe we can uncover new, more efficient ways of programming AI.
This would be interesting