template_literals.js

//For the following:
var a = `foo ${b}`, b = `bar ${a}`;
console.log(a);
//Why would console.log output 'foo' and then 'undefined'? 
//Is it because b is defined after a?

//If that's true, then why does the following code output 'foo bar undefined'?:
var b = `bar ${a}`, a = `foo ${b}`;
console.log(a);

After the hoisting well explained by Hadi.
Your code will run like this:
01 - Assign value to a:
b value is undefined, so the string 'foo undefined' was made.
02 - Assign value to b:
a value is 'foo undefined', so the string 'bar foo undefined' was made.
03 - logs the value of a (it logs 'foo undefined')
04 - Reassign value to b:
a value is 'foo undefined', so b now is 'bar foo undefined'.
05 - Reassign value to a:
b value is 'bar foo undefined', so a now is 'foo bar foo undefined'.
06 - logs the value of a (it logs 'foo bar foo undefined' and not 'foo bar undefined')
With your template literals, the word undefined is be creating as string. You can check with: typeof b and b.length for example.