Perceptron Learning Algorithm Explained | What is Perceptron Learning Algorithm

1. Perceptron Learning Algorithm
2. What is a Neural Network?
3. Application of Neural Network
4. Neural Network Implementation
5. What are hidden layers?
6. Simple Model of Neural Networks- The Perceptron
7. How it Works
8. Introducing a Bias and Sigmoid Function
9. Primary component of a Perceptron
10. Conclusion

Contributed by: Arun Dixit Sharma
LinkedIn Profile: https://www.linkedin.com/in/arundixitsharma/

Introduction 

In Machine learning, the Perceptron Learning Algorithm is the supervised learning algorithm which has binary classes. Even it is a part of the Neural Network. The concept of the Neural Network is not difficult to understand by humans. The concept of artificial neural networks draws inspiration from and is found to be a small but accurate representation of the biological neural networks of our brain. 

We now have machines that replicate the working of a brain – at least of a few functions. Artificial intelligence has given us machines that could classify objects, communicate with us, foresee the future, and play games better than us. In this blog, we will discuss the below-mentioned topics.

• What is a Neural Network?
• Application of  Neural Network
• Neural Network Implementation
• Hidden Layers
• Simple Model of Neural Network- The Perceptron
• Primary component of a Perceptron
• Final Notes
• Conclusion and summary

Also Read: Convolutional Neural Network

What is a Neural Network?

A Neural Network is a computing system that is based on the biological neural network that makes up the human brain. Neural networks are not based on any specific computer program written for it, but it can progressively learn and improve its performance over time.

A neural network is made up of a collection of units or nodes called neurons. These neurons are associated with methods for an association called a synapse. Using the synapse, a neuron can transmit signals or information to another neuron nearby. The receiving neuron can receive the signal, process it, and signal the next one. The process continues until an output signal is produced.

Application of Neural Networks

Note that Neural Networks are a part of Artificial Intelligence. So the application area has to do with systems that try to mimic the human way of doing things. There are many modern application areas of neural networks which includes:

Computer Vision: Since no program can be composed to cause the PC to perceive all the items in presence, the main route is to utilize neural systems with the end goal that as time goes, the PC could all alone perceive new things based on what it has already realized.

Pattern Recognition/Matching: This could be applied in looking through a storehouse of pictures to coordinate say, a face with a known face. It is utilized in criminal examination.

Natural Language Processing: System that allows the computer to recognize spoken human language by learning and listening progressively with time.

Neural Network Implementation

So how can we implement an artificial neural network in a real system? The first step would be to have a network of nodes that would represent the neurons. We assign a real number to each of the neurons. These genuine numbers would speak to the sign held by that neuron.

The output of each neuron is calculated by a nonlinear function. This function would take the sum of all the inputs of that neuron.

Now, both neurons and synapses usually have a weight that continually adjusts as the learning progresses. This weight controls the strength of the signal the neuron sends out across the synapse to the next neuron. Neurons are normally arranged in layers. Various layers may perform distinctive sorts of changes on its information. Signals move through different layers including hidden layers to the output.

What are the Hidden Layers?

The layers between input and output layers are called hidden layers. There can be many layers until we get an output. Neurons are normally arranged in layers. Different layers may perform different kinds of transformation on its input, or it can adjust as per output result. Signals move through different layers including hidden layers to the outputs.

Simple Model of Neural Networks- The Perceptron

The perceptron learning algorithm is the simplest model of a neuron that illustrates how a neural network works. The perceptron is a machine learning algorithm developed in 1957 by Frank Rosenblatt and first implemented in IBM 704.

How it Works

How the perceptron learning algorithm functions are represented in the above figure.  In the above example, the perceptron has three inputs x1, x2, and x3 and one output.

This input variable’s importance is determined by the respective weights w1, w2, and w3 assigned to these inputs. The yield could be a 0 or a 1 relying upon the weighted entirety of the data sources.

Output = w₁x₁ + w₂x₂ + w₃x₃ 

If Output is below threshold then result will be 0 otherwise it will be 1. This edge could be a genuine number and a boundary of the neuron.

This operation of the perceptron clearly explains the basics of Neural Networks.

Introduction to learning neural networks. Presently we would look at an increasing point by point model of a neural system, yet that would be to a limited extent 2 since I have to keep this exercise as basic as could be expected under the circumstances.

Introducing a Bias and Sigmoid Function

Let’s not consider a general example, this time we have not just 3 inputs but n inputs. Just as you know, the formula now becomes:

Output = w₁x₁ + w₂x₂ + w₃x₃ +,,, + w_nx_n

Which is not much different from the one we previously had. But if we use a function like this one, the output could be any number. However, we want the output to be a number between 0 and 1.
So what we would do is to pass this weighted sum into a function that would act on the data to produce values between 0 and 1. What function would that be? Yes, that is the sigmoid function! Which is also known as a logistic curve.

For the sigmoid function, very negative inputs get close to zero and very positive inputs gets close to 1 and it sharply increases at the zero point.

So if we use the symbol σ, we would have:

σ (w₁x₁ + w₂x₂ + w₃x₃ + ,,, + w_nx_n)

Now, suppose, we want the neuron to activate when the value of this output is greater than some threshold, that is, below this threshold, the neuron does not activate, above the threshold, it activates. At that point we call this limit, inclination and remember it for the capacity.

We would then have-

σ (w₁x₁ + w₂x₂ + w₃x₃ + ,,, + w_nx_{n+  bias)}

The bias is a measure of how high the weighted sum needs to be before the neuron activates.

 Only activate when weighted sum > bias.

What we have considered is something like what appeared above, with only two layers. In any case, neural systems really could contain a few layers and that is the thing that we will consider in ensuing exercises on AI.

Primary component of a Perceptron Learning Algorithm

• Neurons- A neural network is made up of a collection of units or nodes which are called neurons.
• Synapse- The getting neuron can get the sign, process it and sign the following one. By means of the synapse, a neuron can transmit signals or information to another neuron nearby. Process it and signal the next one. The process continues, until an output signal is produced.
• Input- Neurons are used as input vectors in an equation which is also donated as X1, X2….Xn .
• Weight- This value is updated in every occurrence and changed in every second till the result does not come. It occurs during hidden layers which is donated by W1, W2 …..Wn.
• Bias- It is a special type of classifier   which is also known as synapse. Even it helps input vectors to get the correct answer.
• Activation/step Function- This is a process of reaching to result or outcome. There are 3 main types of activation or step function: Linear, Heaviside step and sigmoid functions. The Heaviside step function is the most used form in AI.
• Weighted Summation- This is the last step in which result or output has been calculated and expected results come out. In this step X1W1+X2W2+………….+XnWn has been added. Input vector and related weight has been summed up to get final output.

Final Notes

• However complex the Neural Network idea shows up, you presently have the hidden rule.
• Set of inputs combined with weights (plus a bias or error to be discussed in the next lesson) to provide an output. 
• Neurons are connected to each other by means of synapses.
• Neurons send signals(output) to the next neuron.
• The network undergoes a learning process over time to become more efficient.

Conclusion and Summary

The most noteworthy consequence of our trials is that running the perceptron calculation in a higher-dimensional space utilizing portion capacities creates critical upgrades in execution, yielding practically identical exactness levels. However, still, the second rate, to those possible with help vector machines. Then again, our calculation is a lot quicker and simpler to execute than the last strategy.

Moreover, the hypothetical investigation of the normal mistake of the perceptron calculation yields fundamentally the same as limits to those of help vector machines. It is an open issue to build up a superior hypothetical comprehension of the exact predominance of help vector machines.

We additionally think that it’s noteworthy that casting a ballot and averaging work better than simply utilizing the last speculation. This shows the hypothetical investigation, which proposes utilizing casting a ballot, is catching a portion of reality. Then again, we don’t have a hypothetical clarification for the improvement in execution following the main age.