Neural Network - I

.txt

INTRO TO NEURAL NETWORKS
WEEK - I
INSTRUCTOR: Shreya V Shetty
shreyav4@gmail.com

Inspiration
As the title suggests, the concept of neural network was loosely inspired by neurons of the nervous system. What’s so great about this nervous system? A single neuron has incredible capacity. The nervous system on a whole can aid humans in recognizing patterns and detecting objects. Let’s look into some details of the nervous system architecture.
How does the Human Brain Learn?
Much is still unknown about how the brain trains itself. In the human brain, a typical neuron collects signals from others dendrites. The neuron sends out spikes of electrical activity through axon, which splits into thousands of branches. At the end of each branch, a structure called a synapse converts the activity from the axon into electrical effects that inhibit or excite activity from the axon into electrical effects that inhibit or excite activity in the connected neurons. Learning occurs by changing the effectiveness of the synapses so that the influence of one neuron on another changes.

Reference : https://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html

Artificial Neural Network


Andrew Ng, once stated :  “A single neuron in the brain is an incredibly complex machine that even today we don’t understand. A single “neuron” in a neural network is an incredibly simple mathematical function that captures a minuscule fraction of the complexity of a biological neuron. So to say neural networks mimic the brain, that is true at the level of loose inspiration, but really artificial neural networks are nothing like what the biological brain does.”
Note that neural network is loosely modelled on the nervous system.
Similar to neurons is the perceptron (also called neuron or unit or node). Perceptron is the basic unit of the nervous system.

Architecture Of Perceptron

A perceptron consists of a series of inputs-x0+x1+x2+....+xn associated with it. Here, we generally equate x0 with 1. Associated with each input, is weight- w0 associated with x0, w1 associated with x1 and so on.These weights are assigned based on the basis of importance of the inputs.So, inputs along with their weights are given to the perceptron. The perceptron now performs a summation of inputs * weights - x0*w0+x1*w1+.....+xn*wn or i=0nxi*wi ---> net value
On obtaining this net value, a function is applied on this in order to determine the output. Here, the function is a step function. A step function looks like this:

In simpler words, a step function is the function, f(x) is such that output = 1 if x>0 and output =0 if x<=0. So, here we can say that the threshold value is 0.
Let me give an example so that it’s easy to understand. This perceptron is a device that makes decisions by weighing up evidence-here evidence is the data. Suppose there is a BFC match coming up. You love football, and are trying to decide whether or not to go. You might make your decision by weighing up three factors:
1)If it clashes with the IPL match for which you have already booked a ticket?
2)If you friends are coming with you or not?
3)Is it going to rain?
We can represent these three factors by corresponding binary variables x1,x2 and x3. For instance, we'd have x1=1 if it does not clash and x1=0 if is clashes. Similarly, x2=1 if your friends are going, and x2=0 if not. And similarly again, x3=1 if it’s not going to rain and x3=0 if it’s going to rain.
So, let us assume that you would definitely not go if is going to rain. So, let’s give it a high weight, w3 = 8. If it clashes with the IPL match, you might be half disinterested to go, so let’s give it a weight, w1 = 5. Since, it's your favorite team playing, you don’t mind going there alone, so w2 = 1.
Kindly note that these weights are give randomly based on intuition. But, we will get the correct weights using updation methods which we’ll talk in next class.
If the output = 1, then you are going for the match. If output = 0, then you are not going for the match.
So, now let’s assume that it’s not clashing with the IPL match -> x1 = 1, your friend’s are not coming -> x2 = 0 and it’s not raining -> x3 = 1.  
Now, instead of using 0 as the threshold value let’s use 8 as the threshold value. Let’s ignore w0 and x0 for now - these are bias. We’ll see later.
x1 = 1,x2 = 0,x3 =1,w1 = 5,w2 = 1,w3 =8, threshold =8
So, net value here is x1*w1+x2*w2 + x3*w3 = 1*5+0*1+1*8 = 13
Can you guess what’s the function here:
Output = 1 if net value > 8 
Output = 0 if net value < 8
Since, our net value = 13 which means Output = 1. Hence, you are going for the match. :D
Network Of Neurons
A perceptron is not capable of performing complex computation. It’s necessary to create a network of neurons. It consists of 3 layers:
1)Input Layer
2)Hidden Layer
3)Output Layer

Install Keras with backend and TensorFlow as frontend

Keras is a simple, high-level neural networks library, written in Python that works as a wrapper to Tensorflow. It’s very easy to build neural network models using these two.
Kindly, follow what’s recommended on the site 
Step 1:
https://www.tensorflow.org/install/ 
Step 2:
https://keras.io/ - jump to installation. 




Any queries just drop in a mail- shreyav4@gmail.com with subject “Neural Networks Coursework”