WIL-ProgrammingLanguage.md

Why?

• It's fun.
• It's useful.
  • Knowledge of how languages work is useful when you hit a really weird bug.
  • It gives you an appreciation of how languages are designed for the use case. It reduces WTFs.
• I can say I've built a programming language; looks great on the CV!
• It lets me experiment - are there features that could be better? Is there a combination we haven't tried?
• It's hard. I like hard challenges.

How did you learn?

• Coursera course on compilers (Stanford, Alex Aiken) - available all the time!
• Kaleidoscope tutorial - uses LLVM, but goes into detail about lexers and parsers.
• Wikipedia has lots of background reading on language features and techniques.

What does a programming language consist of?

• Lexer/Tokeniser reads through the file string and outputs a list of tokens. Examples of tokens are 'identifier', 'string' or 'open function definition'. Very basic, not a lot of syntax checking here (but it does reject invalid characters).
• Parser takes this token stream and turns it into an expression tree - this knows about the syntax of the language, so it knows that () { 1 + 2 } is a function prototype and turns it into an expression FunctionPrototypeExpression.
• Runtime takes the expression tree and evaluates it in the underlying programming language.
• Between the parser and runtime you might get a compiler, for statically compiled languages. This converts the expression tree into a set of instructions for the underlying language. For example, I might write a compiler that generates a Ruby file, instead of pushing it directly into the Runtime.
• There are lots of generators for the Lexer/Parser out there, but this was a learning exercise so I wrote it myself!

Show us your programming language then.

Here we go. (Do the tour!)

What does it look like on the inside?

Here we go. (Go through the tokeniser, parser and runtime.)

What did you learn?

• You can make a programming language in less than 24 hours.
• You can get confused really easily. Splitting it up into little steps is important!
• Building a deterministic language is hard.
• It feels really cool when you build it.

What's next?

• The language doesn't have some basic things like loops in it, so it would be good to get them in.
• Error handling has bitten me in the bum. I often can't figure out which line of Fn is causing the problem...
• Working on 'everything is a scope' pull request to improve layering and have less special cases.
• Write it in a different programming language (C, Go). Write as much as possible in Fn, less in C/Go.
• Type systems are interesting; it would be cool to build one.
• Write another one! I have an idea for one that closely resembles traditional languages (nouns, verbs, adjectives, etc.), but let's start here first.