MLP from Scratch

I made this project in order to get a better understanding in Deep Learning.

About

• The model is coded in Python
• Only Numpy, MpMath, and Matplotlib libraries are used during the proccess
• As task I choosed "XOR Gate Learning", which is one of the basis task for deep learning models
• The activation function is the "Sigmoid" function for all layers
• The model contains:
  • 1 input layer with 2 neurons
  • 1 hidden layer with 4 neurons
  • 1 output layer with 1 neuron
• Weights are initiliazed respect to "Normal Distribution"
  

Results

• Sample number: 80 (4*20)
• Training duration: 28 mins 15 secs
• Epoch number: 5000
• Learning rate: 0.003
• Loss: 0.0002100859
• Accuracy: % 97.87174999
• Loss / Accuracy Graph:
  

Reference

I have benefited greatly from the following resources in this process:

• Medium - Neural Networks, Multilayer Perceptron and the Backpropagation Algorithm by Tiago M. Leite

• Youtube - Backpropagation calculus | Chapter 4, Deep learning

• Youtube - Gradient Descent, Step-by-Step

• Youtube - Gradient descent, how neural networks learn | Chapter 2, Deep learning

• Book - Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville