Is that working? Great. Hi! I'm super excited to be here and speaking at JSConf. My name is Mariko. That's my Twitter handle. I should be automatically tweeting this slide, linked to this slide, so if you're in the back and you want to see that, you can find it there. I will be joining a company called (inaudible) next week, which -- I'm super excited about that. But today I'm not going to talk about software engineering. I'm going to talk about knitting. And just in case you're wondering, like if you are at the wrong conference, I will assure you that I will have enough math and code examples for you to be interested. Just for the guideline, I have a visual relation down on the slide. So the blue dots are indications that the slide has some kind of math reference, computer science reference. The radius of the circle is how many scoops of ice cream I served myself after I figured that out. The yellow dots are some random GIF animations, because why not? So I like knitting. I've been doing knitting for quite a while. I think the first time I was doing it I was seven years old. It's something I've always done. Once a year, come winter, I knit a scarf or mitten. Something very simple. Nothing crazy like having an Etsy store or anything. So here's a few of my projects that I've done in recent years. One winter, I think three years ago, was a very special winter for me, because I learned to JavaScript while I was also working on this green scarf project. And I started to notice really interesting power of computer programming and the process of knitting. So, for example, when you get started -- like, if you've never knit something, knitting might look like this. Seven-year-old me was like -- I'm never going to be able to figure this out, this thing that mom and Grandma is doing. So intimidating. My experiences programming, the first time, was just installing NPM install on a terminal. It was so confusing. Words flying around, and is my laptop being hacked? And like what's going on?
But we all know that once you've done it, once, and know, like, what exactly it's doing, you know, it's not so scary at all. The other thing -- computer programming is very binary. Knitting is very binary. The atomic unit, the smallest unit that you can do is two stitches, which is knit stitch and purl stitch. Knit stitch is the stitch that you pull the yarn from the back of the loop, and purl stitch is the yarn -- you push the yarn in front of the loop. And those two stitches create two distinct faces of fabric. And the combination of those two, much like we say -- programming is ones and zeroes -- creates a pattern in the fabric. So the green side of the fabric, which is 2x2 ribbing, which is commonly used for the cuff of your sweater or the neckline of your sweater, is two knit stitch, two purl stitch, alternating each other, and then going row by row. The other side of the pattern is the one I took from my sweater, which is a little more intricate, making a diagonal line. But if you look closely, it's just knit and purl stitch. Just one stitch by one stitch, shifting to the left to create a diagonal line. Knitters have a Github, actually. It's called Ravelry. People log in, you can put your patterns, sell your patterns, star somebody's projects, and show your photos. It's a central repository of knowledge that you do with knitting. And speaking of a central repository, code for knitting -- the knitting pattern -- looks like this. And I thought this was totally -- looked like code. Because there is, like, some letters that are kind of repeating, and maybe kind of like a variable, that's accessing the same information again and again. And there's more explicit loop indications. Like knit until two stitches remaining. Or repeat row one to four, for ten times. This brings me to -- I could pseudoknit in JavaScript. So I took this code, which was a scarf pattern, very simple scarf pattern, put it into 30 lines of JavaScript, to visualize what this would look like, virtually. And this is what I came out. You know, it is... It turned out it's only using a knit stitch. It's not a purl stitch. So it's okay. And it's a knit scarf. So making a triangular shape. Which is all right. I can understand that. So I actually knit that pattern myself, and I have it here.
But it turns out it didn't quite become triangular. Instead, it became like an arc. So in my code, those yellow stripes should be straight, flat lines. But then it became an arc. And I was like... This is only 30 lines of JavaScript. And I write JavaScript professionally. Like, where is my bug? I can't figure that out. So I have it here. Where my four stitches started here, two inches or so, and I knit it down. And if I force the fabric to see it, you could say this is like a straight line, but in fact, it ended up being an arc of a scarf. So why? I did a little bit of research and found out that what I was discovering was something called hyperbolic geometry in math. And I was like... Okay.
In knitting, we call it ruffles. So I did some more research, and there were three kinds of geometry to describe, and they both use one perfectly straight line. The one we're familiar is Euclidean space, which -- you have a baseline, and then parallel to that, you have another line that will never meet the baseline again. So it's very 2D. The other one is spherical space, where the straight lines are on the sphere. However you place it, it will always meet each other. And the third one is a hyperbolic space, which -- there is a baseline, and the straight line, infinite number of straight lines, can be drawn. And it turns out this hyperbolic geometry is a hard problem to solve in computer graphics, because it's hard to model.
Researcher at Cornell actually created a physical model of hyperbolic geometry in knitted fabric. In her case, created in crotchet. And here's her paper about how she created the hyperbolic plane. So when I discovered this, I just stumbled upon a problem making ruffles. But when I discovered this is an actual math problem behind my ruffle, I was like -- this is great. I want to do more of this. I want to discover more about pattern making, and I want to do this. The problem was that knitting is a very slow process. Like, this scarf -- itself, take maybe five days of solid work. So... Eight hours a day? And if you think about software engineer salary -- this is a very expensive piece of scarf. So I did more research about -- how can I test out my patterns? And stumbled upon this thing, called a knitting machine. Which I have never knew that this existed. But apparently it existed for home use, available to your house, since 1890.
And this is the ad from that era. And furthermore, interesting, as a native Japanese, was that the knitting machine was a very popular thing to own in the household, in the '70s and '80s Japan. And people were offering local classes, people owned a knitting machine like you would own a sewing machine. I found one TV commercial on YouTube about this knitting machine from 1979. If you can get the audio.
(song in Japanese with female vocalist)
(male narrator speaking Japanese)
(female vocalist singing in Japanese)
All right. So I am as confused as you are to see this TV commercial for one different reason. The song that was in the background, the lyrics said that creating a special original pattern knitwear is like sending a secret love letter to your loved one. And I thought that was kind of creepy, but apparently... Things to do in the '70s! I lived in New York City when I discovered this, so I couldn't really go down to local classes. So I don't even know if that's offered in Tokyo now. But I made a short trip to eBay. Spent a couple hundred dollars to buy this machine. And I got it, I set it up, and I was like... Okay. I'm going to set this up. And create all of the patterns, and I'm going to be awesome, and create sweaters. Turns out... Operating the machine, even before going into creating a pattern, was a lot of work.
The machine was made in the '80s. It was sitting in somebody's closet for a long time. It was missing the manuals. So I ended up -- the next two months, just researching all kinds of stuff on Japanese blog posts about machine knitting. Possibly watched every single YouTube video about machine knitting that's posted. Many of them are Japanese. Occasional trip to a free preview version of Google books, because I couldn't go to a Japanese bookstore down the street from my parents' house and pick up the book. I never thanked myself enough to be able to speak Japanese. Because all of the stuff, great stuff, was available in Japanese. And I just get to angry, when software only offers documentation in English. Certain things I could not understand. But in knitting, at least I got my turn to just do things in Japanese. On top of figuring out how to operate the machine, I actually needed to fix the machine. So I figured out how to operate this machine. Moving back and forth. But it wasn't knitting quite right. Something was missing. And basically I flipped the carriers, looking at examining what exactly it's supposed to do, and what each path is doing. And after looking at maybe four hours of, like, mechanics of things, I found out that one piece of thing was stuck in there. And just a jiggle -- it worked perfectly! And so finally, I got to knit a fabric out of the machine. And I was like... This is great. One thing about this machine is it's known for something called Jacquard knit or Fair Isle knit, or pattern knit, if you're familiar with those terms. The term Jacquard comes directly from the Jacquard loom, which is a different kind of machine that creates a fabric, and uses a punch card to input the data. And this punch card system, the punch card loom, became the base of the modern computer. Which -- I thought this was a great discovery. When I did the research and learned about it, I was so excited. So... The machine is based on that. And you see on the '70s ad -- the machine actually had a punch card feeder. So you create your own punch card, and then feed it into the punch card feeder, and that will translate the pattern for you. My machine is coming from the '80s, so it is an Electroknit. It's actually the name of the model. It's called the Electroknit. And what does that mean? It no longer has a feeder for a bunch card. But it has a mother board and chip and magnetic strip. You use an electromagnet to push and pull the needles automatically.
It's a lot like dot matrix printing, if you are familiar with a dot matrix printer. One pixel becomes one stitch. So in this machine's case, I feed it a yellow and blue yarn, the needle that's pushed forward is grabbing a contrasting color, which is blue, and then the needle that was left behind is grabbing a base color, which is yellow. That's how they create a pattern. So that means I can create some kind of graphic -- anything. In this case, the JSConf logo. And then put it into knitted fabric. And I was like... I found out how to operate this machine, I fixed this machine. This shouldn't be a problem. I'm going to do this. Turns out it was not that easy. Not even... So there was so many things that you needed to figure out. Which is indicated in that blue dot.
Even before you actually put the data into the machine. So first question was like -- how do I even make bitmap image? Because if you're using Photoshop, you just load the colored photo in, select grayscale, and select the bitmap method, and it will automatically feed the bitmap for you. But I need to do this in code. How do I programmatically do that? Let's start with just learning about images, and loading the image. I used a library called graphics magic and then NPM package GM, which wraps that library. I actually started by using PhantomJS, creating a canvas in Phantom, loading the image, and using the Phantom API to extract all of the data, which worked fine, but it was very slow. And GM was suggested to me as a faster alternative. And it worked perfectly.
So when you load a file into GM as a text file, this is what you get. XY coordinates of the pixel. RGB value of the color, and then also the hex value for that color. So all you do is a handy-dandy regexing, and create an array of arrays for each pixel. So now I have RGB color for each pixel. Let's get on it and make it grayscale? But how do I grayscale? I spent two hours just looking at a single Wikipedia page about grayscaling, and the engineer from Adobe confirmed that's okay. So I'm confident about my explanation here. You have RGB value for each color. In this case, pink and green. You find the highest value for each color. In this case, pink, red is the highest value. Green, green is the highest value. You repeat that number, and then you get the grayscale color.
So now you load the image in GM. You get all of the color. And you have three color -- RGB value array. You just find the max value out of the array, and then you have an array of just the max value. And you can use that to create a grayscale image. So I was like... This is black and white photography. So I can just load it in, because it's black and white. But the grey scale image is actually black and white and many shades of grey in between. And my machine only has two colors. So it has to be just two colors. Nothing in the shades of grey. So my first upload was to split it in half. Anything closer to white side gets white, and anything closer to black side gets black. And it turns out like this. Something like... What came out of a fax machine. I wasn't really happy with that. So I was like... This is my favorite game when I grew up.
When I was growing up. And this only used, like, black and white, and it operated beautifully in my tiny Apple computer. So I literally Googled Cosmic Osmo graphics and figured out that I needed to do something called dithering. Which -- the Wikipedia article said it was an intentionally applied form of noise. I don't really understand, but okay. GM has built-in methods for it. So I did that. It turns out better than my method. But still... Somewhere around the area is missing a definition. And I wasn't quite happy with it. I did more research, and something -- discovered that the thing I needed to do was something called ordered dithering, or half tone screening, which is used for newspapers to translate that picture into a newspaper, which is just one color of ink. So you have a base image. Which you extracted the value for the grayscale. And then you have this noise matrix part. You laid it over. You compare the overlapping pixels. And if the base image has a lower value than your noise matrix, then it gets zero. If it's over, then it gets one. And that's how you make it into two color images. And of course, if it's like... One and two, you can put it into ones and zeroes -- matrix.
Here's a few examples of that noise matrix. And a lot of people explore their own matrix to create a better dithered image. I ended up using Bayer, because I like the output most. So now I loaded the JSConf logo and outputted it as ones and zero pictures. And if I replace the zero with no value and one with asterisk, you can see something like this. And it's starting to look like something on the knitted fabric. So I'm like -- great. I've got files. Let's just put it into the machine and just knit. Let's make knitting pattern files. The file format for this is something called .dat. Which I think stands for digital audio tape? No idea. If you open it up, you get ones and zeroes. Okay. Great. Binary file. Just replace my one and zero matrix and put it in. But I found out if the pattern is in there, it's actually hex. So I'm like... Something needs to be done in my one and zero matrix. Into this file. What do I do?
There was one guy, Steve Conklin, who is writing the same kind of project in Python. So I studied his code, and discovered this line. And at the time, I was a newbie Python developer. So I was learning Python. And I'm like... What is this? I just went through Python's tutorial. Nothing was mentioned about bitwise operator. How could I not have heard of this? I just went through training. And I felt like this. I also found out that the bitwise operator existed in JavaScript, and read an article, and still couldn't really get what it is. So what I concluded, after two days of, like, you know... Throwing balls against walls, was that this is like magic. And this is my interpretation of what bitwise operator and bit shifting is. So I'm going to give you two examples. Two operations that I use for this machine. One is a bitwise left shift. Left shift is a guy or fellow who pushes you to the left. And he's a magician, because it's a bitwise operator. So he has a magic hat. So A is the small value of one. And let's bit shift forward to the left. So the representation of this small one is that. And this left shift guy comes in, pushes you four times to the left, and because he is a magician, he makes it to 16. Like, how... What... How could that happen?
Okay. And the bitwise OR operator -- the same thing. This person favors one over zero, and he's also a magician. So let's do bitwise OR, with 1 and 5. Representation of the 1 and 5 is that. First compares 1 and 1. There is only 1, so he chooses 1. There's 0 and 0. So there's no 1, so okay. 0. And next one is 1 and 0. So okay... 1. And because it's magic, it stays as 5, even though you just did something. But it stays as 5. So... I needed to do bitshifting, and if you're interested, I'm happy to share the code, but my one and zero matrix shifted into one set of the decimal value of A, shifted again, became one number, and then turned that decimal number into hex, and I can finally put it into my knitting machine. So now I have the file prepared. And put it into the machine. And now I want to demo the machine.
Okay. So I just turned it in. I pre-loaded a file this morning. I used USB to serial port cable. Which the tutorial of how to make is available online. And... Just select... You might not be able to see it. But I'm selecting a custom pattern that I created. It says one time. Stitch of 90. Okay. Start. I changed the setting. Make sure this is okay. And now... Something interesting should happen.
(grinding noises)
See? These needles are pushed forward. This means that this carriage uses magnetics to pull those needles, and now these needles that are pushed forward are picking up this blue yarn. And...
(grinding noises)
All right! Now the pattern is done. I can just discharge this. Take this out. Actually, just leave it there. You may not be able to see it from the front. But it actually printed out the JSConf 2015 logo. And after that, I'll finish this scarf, and I can show it to you in person, if you're interested.
(applause)
Woo!
Oh, how kind! Thank you! So going into this project, I had no idea what I was doing. I just wanted to do knitting. And I discovered that knitting and JavaScript are kind of related. That turned into, like, finding out about this machine, and I wanted to do this. I had no reservation about trying to sound smart. This wasn't my job. So whenever I go into a problem, I just tweet out or send emails about it, and so many people just jumped right in, sending me code samples, and giving me encouragement and all of that. So I just want to give a shout out. Thank you! And most of the code will be available on Electroknit repo, which right now is empty, but I promise I will put it up. And a subset from that project -- the part I was doing, the color math -- I was super interested in it. And I added more features about translation between color spaces and created a small library for you to... What do you call it? Explore the color. And that's available on the color mixer repo, or NPM module. So thank you!