Hi, everybody! So let's have some fun with the Nintendo, which was a fine piece of hardware. Even today. So what is this thing that I'm going to be talking about, I guess? It's the latest in 1980s music technology. This is controlling an Nintendo's sound. Like, the sound stuff that makes the... Sound stuff that is in the microprocessor, like... Compiling down to the assembler through JavaScript. So this piece of hardware right here is what we're going to be running everything on. And there's no trickery involved or whatever. It's actual Nintendo programs running. Just want to make -- some people are like -- hey, you have some crazy thing. But no, it's real. So when I first started this project, the goal was to make it easy to write music for a Nintendo. And that's been done a lot in the past also. There's trackers, which are basically sequencers. That you can use to sequence music on Nintendo itself, or you can have programs that will output actual sound of the Nintendo, or emulate the sound chip. But I wanted to do it a little bit differently. I wanted to do it using JavaScript. Because JavaScript and whatever. So why? Why do you want to do this in JavaScript?
To me, it's self-evident. I mean, JavaScript is awesome, right? It's a language that I really like. But the whole project itself -- like, I really like this sort of music. This chiptune-y sound. I grew up in the '80s, playing a lot of Nintendo. Four or five-year-old, playing super Mario brothers all the time, Zelda, all those games. Over time, there's a nostalgia factor. Those tunes -- I didn't pay attention to the music, but they're ingrained in my brain now. So I love it. I don't know how else to explain it. I think that probably a lot of people have that nostalgia factor. They look back to their childhood and whatever they did a lot or played a lot -- as far as video games or music or whatever, they gravitate towards that.
So I think that's a big thing. But the JavaScript part is because of browsers. I knew that if I could get something working in JavaScript, I would have a path to get to the browser. And that's important, because once you do that, if you provide a low level JavaScript API to this sound engine, you can create or you can give the opportunity for people to create fancy UIs for trackers, sequencers, that are all web-based. And I think that's really cool. And I actually want to do that. But I think this is, like, the first step to that point. So that's primarily why. And also it's just a lot of fun to hack on a Nintendo with JavaScript. It's crazy. So the next question is... How is this working? Because it seems sort of magical to have a Nintendo in JavaScript and stuff. So to get this work, I had to read a lot about how the processor works, how the 6502 assembler is written. I didn't graduate with a computer science degree, so I have no idea how this stuff works. It's just all magical to me. But I read a lot and I learned a lot of assembler. Which is crazy. No one should ever have to do that. Right?
But then along in the process, I wasn't just reading about assembler and how the Nintendo works. I read a lot about music too, because a lot of that stuff... I had no idea how it worked. And then throughout the whole process, I would go, like, write assembler or write JavaScript and figure out how this stuff was going to work, and figure out the API and all that stuff. And then what I would end up doing was... Once I had something working, I would write music. And try to, like, figure out, like... Does this make sense? Does this work? Does this API... No. And then I would go back and be like... That part's stupid. Get rid of it. So I'm at a place now where there's some stuff that I don't like, but I think the API is decent enough to present. So what I have here that I'm going to demo in just a little bit after I get through the basis of what this is -- is an assembler sound engine. So what happened was I was like -- oh, I want to make, like, this thing make sound.
So I went and I read a lot, and figured out how to do that in assembler. I made a square channel beep. It just kind of went beep. Like square channel. Which I'll demo kind of in a second. Then after I went through these tutorials on these NES developer forums, and at the end of the tutorials, you kind of have a basic working sound engine in assembler. But assembler is really difficult to work with. And it's not very fun to write music in assembler. It just doesn't... Like, I don't know. It's assembler.
So once I had that, though, it was easy to write a JavaScript API to that. So that it could compile down to what a song would look like in assembler. So this has... Let's see. How many are there? Five channels. Two squares. A triangle noise, which is just random noise, and a sampler, which... I did not do the sampler, because it takes up a lot of memory, and I didn't really figure out how it works. So I'm just going to be working with these four now. So the square channel... The features that it has is... Duty cycle. I'll show a demo of it. Because a lot of this stuff, you can't really visualize it. I guess you can visualize it, if I had an oscilloscope, but I don't. But it's better just to listen to it. So we'll listen to it in a second. These are the features. Didn't get frequency sweeps working. But here we go. I'll demo the square. Can we get the demo thing going? Okay. So this is the little file system thing on the Nintendo. That I have going. Okay. So here's a square. This is going to play a C note, and it's going to go through the octave range of a C note on a square channel.
(beeping up several octaves)
That was just doing a for loop with a number or something and just... Passing that into song.square, and it's just an array of notes. And it plays it.
(beeping up several octaves)
All right. I've got to reset this sucker to get back. Oh, one other thing that I kind of skipped over. Duty cycle and volume envelopes. So duty cycle... It kind of spaces out, like, did we get this back? Sorry. Can we get this... Yeah. Back. Sorry. It spaces out or narrows the actual square. How it looks. And it creates a different sound. So just to kind of... This is the same note, same octave, just manipulating the duty cycle of the square channel.
(higher timbre notes)
Okay. So I got square stuff working and it was beeping. The next thing I did was, like, actually, once I got the square channel working, made that beep, then I went to the sound engine and wrote the API to it. And then I wrote this thing, which I don't... It's just some notes in changing octaves. This is just what it sounds like. This is literally the first thing I wrote, besides the beep. So it doesn't sound awesome at all.
(ponderous organ-like musical notes in a minor mode)
All right.
(applause)
Okay, cool. So... Yeah, that was the first thing I wrote. And then I started implementing all these features. Volume envelopes -- if you're not familiar with them, like, imagine a note is broken out into a bunch of frames, and you have the ability to say -- on frame one of the length of the note, make the volume zero. And on a scale of zero to, say, ten. Zero being off, ten being the loudest. So zero, it's off. You gradually increase it. Frame one, one. Two, two. Five, ten. And then it's the loudest. So you have created a fade-in effect. So you can manipulate these notes with volume envelopes to create, like, blip effects, like echo or staccato, fade-ins, fade-outs, all kinds of things. I don't think I have a fade-in demo. But let's see. Oh, one other note is like square channel is the only one that can do that. So with triangle, you have the ability to turn volume on or off. So you're pretty limited. I don't know why this is the case. But you can either say -- hey, this is on. It's just binary on or off. And you can use volume envelopes with them, but it will just interpret any value that's not zero as a one. So triangle... It's made for, like, bass sounds. For, like, the lower octaves. The really high octaves, it's the flute sounds that you hear in, like, Zelda and stuff. So here's an example of... I don't remember... I think this is just the same as the square. Just going through the octaves of C.
(notes going up octaves)
So there we go. Triangle. Noise. I don't really know how to explain this. Because it doesn't really map to music very well. It sounds like drums. Sometimes you can get, like, certain ones sound like hi-hats or snares or bass drums and stuff. But for musical notation, I didn't know actually how to represent this. So this was one of the problems I had. But here's the full noise range that you have.
(sounds resembling hi-hat and harpsichord)
So... Then with those sounds, you can create drums. And this is kind of a really dumb simple drum beat.
(four on the floor drum beat)
Okay. So techniques. So... All the channels on here are monophonic. So you can only play one note at a time. You can't make chords. So what people would do, composers of the '80s, they would fake it. Like, they would make arpeggios, and speed them up super fast. Increase the timing of the note. Or speed the tempo up. A lot of the Japanese games and stuff, they were just written like... If you listen to a lot of video game music, it's just written monophonically. Some of the Commodore 64 people would use the arpeggio technique. But let's go through some songs, before I go forward. So I was having trouble, like, I was trying to do some '80s for JSConf, given the theme. So I did Billie Jean. Just the bass line, though. Because the square doesn't really sound... I couldn't get it to sound super good. This is just the bass line. It's really simple. Let's see if it sounds decent.
(bass line of Billie Jean)
That's it. That's the whole song, pretty much. And the square channel would be that bermp-bermp. Yeah. Okay, let's go... Then I kind of went through a bunch of songs. I tried a bunch of different kinds of music before I talked to actual people that composed music, and they gave me some pointers. I was like -- I like Slayer. I like Raining Blood. That must be an awesome song to do. Not really, because it's actually really boring.
(Slayer, Raining Blood)
This is the whole song, pretty much. I...
(drums come in)
It's just drop D over and over again. And there's that little part in the middle.
(bars repeat)
Okay. So Slayer was boring. Like, a lot of... Yeah. I mean, dropped D over and over again does not sound good on the Nintendo. So I was like... Okay. I'm going to take a crack at rewriting a Super Mario track. So I came up with this.
(main Super Mario theme)
(applause)
So that one... That one actually worked out really well. I was much happier with that than Slayer. And yeah. So then after talking to some people and showing them what I had, though, I was kind of like... But here's the problem I'm having. All these songs have chords, and it doesn't, like, it usually doesn't sound good. There's a couple cases where I did get chords to kind of sound good, but... I don't know. What I did, instead of doing the arpeggio technique, because I could almost never get that to sound okay, I was talking to my brother -- and he was like just use power chords. You have two square channels, so you can just use the two... And that kind of worked, but it still didn't seem right. But then I was talking to a composer friend of mine, and he was like... You should check out the... And I forgot what they're called now. But these Bach pieces. BWV847 I think is the prelude in C minor. And 851 is D minor. I didn't finish these, and they're kind of long, so I won't play them all, but they're really great, because they're mostly monophonic. So they map really well to the Nintendo. And I have volume envelopes, so I can make them play staccato, which is what Glenn Gould plays them at.
(Bach playing staccato)
Okay. It's more of that. It's just... Yeah. So really quickly, this is just the first couple bars of the D minor one.
(rapid counterpoint)
I think the timing is off. So those work really well, because they're monophonic. Since there's no chords, it makes sense. If you're reading music, for the bass clef, you can just use the triangle channel, and then if you have notes up top, you can use square, or even mix them up or whatever. Okay. Great. So... That's kind of like what I have. So moving forward, though, going back over, I guess, to here, is the future. Like, what do I want to do with this? I have this API. I have this module that's on NPM, that can take JavaScript and produce these songs in the end. They're still... It's mostly JavaScript. There's this last piece -- the assembler itself is actually C++ assembler. But I think that even can be moved to JavaScript. And there's already people that have JavaScript assemblers for the Nintendo. So it's not that much of a jump to get there. So I think we can get an NES assembler in JS that's one module that you can just say -- hey, here's some assembler. Give me a Nintendo program. And that will be awesome.
And then I think also with this API, we can build web UI tools that, since you have an assembler in JS, and you have this stuff that I wrote in JS, you can say -- here's a sequence of sounds. Click a button or whatever. It builds it. It outputs your assembler. It assembles it, gives you an actual Nintendo program, and then you have things like -- people have written emulators in JavaScript for Nintendo, right? I'm sure... We see these things pop up on all of the news sites and whatever, over the last five years. So you have your whole, like, build environment in JS. And you just need to piece it together and get it in the browser, and we can have this really awesome, like, authentic-sounding thing to make 8-bit music. So yeah. I don't even know how much time I have. Because this thing is not going. But... I can keep talking about this stuff. But... Okay. Cool. So... Let me go back to the why of, like, how I got started into this stuff. So I had previously written, like, graphical stuff for this, and I gave some talks. Like, writing slides and stuff. On the Nintendo itself. And that was super hard, and I didn't do a very good job at, like, the graphical stuff. I had a lot of problems with it, and it was a lot of stuff that was hacked together.
And then I can't remember where it was, but I gave... I was giving a talk, and the AV person came up to me and they said... Oh, are you going to need sound to the Nintendo? And I thought... Oh, no, I won't need sound. I don't have any sound. And I was thinking about it a little bit later, and I was like... Why don't I have sound? Why didn't I read that part about sound? So... I thought, like, oh, I should go back and figure out how sound works. And it was just that curiosity. Like, in the back of my mind. Like, how does this work and stuff. So I went back, and there's a lot of stuff that's written about the Nintendo, that's out there. But there's not a whole lot of practical examples. So finding, like, this sound tutorial, and kind of standing on the shoulders of giants, and taking this stuff that these other people have written, and using that as a basis for, like, saying -- hey, I could just write a JavaScript API to this, and I know JavaScript well enough that I can expose this, so that other people can use this stuff.
And yeah. So... I got to that point, and it was actually way, way easier than all the graphical stuff. The sound stuff was a piece of cake, compared to the graphical stuff. And I think, like, a lot of this stuff is still relevant today. I mean, you have bands like Anamanaguchi that actually use a Nintendo in their music. They use it as an instrument. It's an instrument that has these channels that you're able to manipulate sound and get a real authentic sound. And there's actually a game that... For Nintendo, that was released a couple months ago, called Star Versus. You can buy this game for $40 or whatever, and they give you a real Nintendo cartridge. And that is so cool, that people are still hacking on it and selling games and stuff. So I think this stuff is really, really awesome. And so yeah. I want everybody to just, like, you know, if you're interested in this stuff, like... I don't know. Hit me up, and talk to me. And we'll geek out about Nintendos and building musical stuff. Cool. Thank you.
(applause)
I think the most important question that you have to ask is... Now, when you're using this, do you have to blow into the NES in cases of reset?
You don't. The reason why you had to blow to get this to work or jiggle it around or whatever is because... Right here, these pins were usually worn out on your super old cartridges. And they're not... You know, they wear out after probably a little bit. Because I remember being super young and they would be like... I would be like jiggling it and whatever. This thing is like... Just a couple years old, and it's probably higher quality than your average Nintendo cartridge. So this works every single time you put it in.