Artificial Neural Networks (ANNs) - XOR - Step-By-Step
•
6 likes•1,542 views
Artificial neural networks (ANNs) or connectionist systems are a computational model used in machine learning, computer science and other research disciplines, which is based on a large collection of connected simple units called artificial neurons, loosely analogous to axons in a biological brain. Connections between neurons carry an activation signal of varying strength. If the combined incoming signals are strong enough, the neuron becomes activated and the signal travels to other neurons connected to it. Such systems can be trained from examples, rather than explicitly programmed, and excel in areas where the solution or feature detection is difficult to express in a traditional computer program. Like other machine learning methods, neural networks have been used to solve a wide variety of tasks, like computer vision and speech recognition, that are difficult to solve using ordinary rule-based programming. Find me on: AFCIT http://www.afcit.xyz YouTube https://www.youtube.com/channel/UCuewOYbBXH5gwhfOrQOZOdw Google Plus https://plus.google.com/u/0/+AhmedGadIT SlideShare https://www.slideshare.net/AhmedGadFCIT LinkedIn https://www.linkedin.com/in/ahmedfgad/ ResearchGate https://www.researchgate.net/profile/Ahmed_Gad13 Academia https://www.academia.edu/ Google Scholar https://scholar.google.com.eg/citations?user=r07tjocAAAAJ&hl=en Mendelay https://www.mendeley.com/profiles/ahmed-gad12/ ORCID https://orcid.org/0000-0003-1978-8574 StackOverFlow http://stackoverflow.com/users/5426539/ahmed-gad Twitter https://twitter.com/ahmedfgad Facebook https://www.facebook.com/ahmed.f.gadd Pinterest https://www.pinterest.com/ahmedfgad/
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Artificial Neural Networks (ANNs) - XOR - Step-By-Step
Similar to Artificial Neural Networks (ANNs) - XOR - Step-By-Step (20)
More from Ahmed Gad
More from Ahmed Gad (20)
Recently uploaded
Recently uploaded (20)
Artificial Neural Networks (ANNs) - XOR - Step-By-Step
- 1. Artificial Neural Networks (ANNs) XOR Step-By-Step MENOUFIA UNIVERSITY FACULTY OF COMPUTERS AND INFORMATION ALL DEPARTMENTS ARTIFICIAL INTELLIGENCE المنوفية جامعة والمعلومات الحاسبات كلية األقسام جميع الذكاءاإلصطناعي المنوفية جامعة Ahmed Fawzy Gad ahmed.fawzy@ci.menofia.edu.eg
- 3. Neural Networks Input Hidden Output BA 01 1 10 00 0 11
- 4. Neural Networks BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2
- 5. Neural Networks BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Can`t be Solved Linearly. Single Layer Perceptron Can`t Work. Use Hidden Layer.
- 7. Input Layer Input Output A B BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Hidden
- 8. Hidden Layer Start by Two Neurons Input Output BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Hidden A B
- 9. Output Layer Input Output BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Hidden 1/0 𝒀𝒋 A B
- 10. Weights Input Output BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Hidden Weights=𝑾𝒊 A B 1/0 𝒀𝒋
- 11. Weights Input Layer – Hidden Layer Input Output BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Hidden Weights=𝑾𝒊 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 A B 1/0 𝒀𝒋
- 12. Weights Hidden Layer – Output Layer Input Output BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Hidden Weights=𝑾𝒊 𝑾 𝟓 𝑾 𝟔 A B 1/0 𝒀𝒋 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 13. All Layers Input Output BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Hidden Weights=𝑾𝒊 𝑾 𝟓 𝑾 𝟔 1/0 𝒀𝒋 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 14. Activation Function Output 𝒀𝒋 BA 01 1 10 00 0 11 Input Hidden 1/0 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 15. Activation Function Output 𝒀𝒋 BA 01 1 10 00 0 11 Input Hidden 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 16. Activation Function Output 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 17. Activation Function Components Output 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 18. Activation Function Inputs Output s 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 19. Activation Function Inputs Output s 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 s=SOP(𝑿𝒊, 𝑾𝒊)
- 20. Activation Function Inputs Output s 𝑿𝒊=Inputs 𝑾𝒊=Weights 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 s=SOP(𝑿𝒊, 𝑾𝒊)
- 21. Activation Function Inputs Output s 𝑿𝒊=Inputs 𝑾𝒊=Weights 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 s=SOP(𝑿𝒊, 𝑾𝒊)
- 22. 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 Activation Function Inputs Output s 𝑿𝒊=Inputs 𝑾𝒊=Weights 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊 s=SOP(𝑿𝒊, 𝑾𝒊) 1/0
- 23. Activation Function Inputs Output s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊 1/0 Each Hidden/Output Layer Neuron has its SOP.
- 24. Activation Function Inputs Output s 𝑿 𝟏 𝑿 𝟐 𝑺 𝟏=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊 1/0
- 25. Activation Function Inputs Output s 𝑿 𝟏 𝑿 𝟐 𝑺 𝟐=(𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊 1/0
- 26. Activation Function Inputs Output s 𝑿 𝟏 𝑿 𝟐 𝑺 𝟑=(𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊 1/0
- 27. Activation Function Outputs Output F(s)s 𝑿 𝟏 𝑿 𝟐 Class Label 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 28. Activation Function Outputs Output F(s)s 𝑿 𝟏 𝑿 𝟐 Class Label 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 Input Hidden 1/0 Each Hidden/Output Layer Neuron has its Activation Function. 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 30. Activation Functions Which activation function to use? Outputs Class Labels Activation Function TWO Class Labels TWO Outputs One that gives two outputs. Which activation function to use? 𝑪𝒋𝒀𝒋 BA 01 1 10 00 0 11 BA 01 1 10 00 0 11
- 31. Activation Functions Piecewise Linear Sigmoid BinaryBinary
- 32. Activation Function Output F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 BA 01 1 10 00 0 11 0 0.2 0.4 0.6 0.8 1 1.2 0 0.2 0.4 0.6 0.8 1 1.2 1/0 Input Hidden 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 33. Bias Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 34. Bias Hidden Layer Neurons Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0
- 35. Bias Output Layer Neurons Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟑 1/0 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 36. All Bias Values Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 37. Bias Add Bias to SOP Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 𝑺 𝟏=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) 𝑺 𝟐=(𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) 𝑺 𝟑=(𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 38. Bias Add Bias to SOP Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 𝑺 𝟏=(𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) 1/0 +1 𝒃 𝟑 𝑺 𝟏=(+𝟏𝒃 𝟏+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 39. Bias Add Bias to SOP Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 𝑺 𝟐=(𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) 1/0 +1 𝒃 𝟑 𝑺 𝟐=(+𝟏𝒃 𝟐+𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 40. Bias Add Bias to SOP Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 𝑺 𝟑=(𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) 1/0 +1 𝒃 𝟑 𝑺 𝟑=(+𝟏𝒃 𝟑+𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 41. Bias Importance Input Output BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 42. Bias Importance Input Output X Y BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 43. Bias Importance Input Output X Y y=ax+b BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 44. Bias Importance Input Output X Y y=ax+b BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 45. Bias Importance Input Output X Y y=ax+b Y-Intercept BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 46. Bias Importance Input Output X Y y=ax+b Y-Intercept b=0 BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 47. Bias Importance Input Output X Y Y-Intercept b=0 BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 y=ax+b
- 48. Bias Importance Input Output X Y Y-Intercept b=+v BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 y=ax+b
- 49. Bias Importance Input Output X Y Y-Intercept b=+v BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 y=ax+b
- 50. Bias Importance Input Output X Y Y-Intercept b=-v BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 y=ax+b
- 51. Bias Importance Input Output X Y Y-Intercept b=+v BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 y=ax+b
- 52. Bias Importance Input Output Same Concept Applies to Bias S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 53. Bias Importance Input Output S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 54. Bias Importance Input Output S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 55. Bias Importance Input Output S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 56. Bias Importance Input Output S= 𝟏 𝒎 𝑿𝒊 𝑾𝒊+BIAS BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 57. Learning Rate 𝟎 ≤ η ≤ 𝟏 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 58. Summary of Parameters Inputs 𝑿 𝒎 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 59. Summary of Parameters Weights 𝑾 𝒎 𝟎 ≤ η ≤ 𝟏 W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …) 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 60. Summary of Parameters Bias 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …) F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 61. Summary of Parameters Sum Of Products (SOP) 𝒔 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐+…) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …) F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 62. Summary of Parameters Activation Function 𝒃𝒊𝒏 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐+…) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …) F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒
- 63. Summary of Parameters Outputs 𝒀𝒋 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐+…) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …)
- 64. Summary of Parameters Learning Rate η 𝟎 ≤ η ≤ 𝟏 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐+…) 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …)
- 65. Other Parameters Step n 𝒏 = 𝟎, 𝟏, 𝟐, … F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐+…) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …)
- 66. Other Parameters Desired Output 𝒅𝒋 𝒏 = 𝟎, 𝟏, 𝟐, … 𝒅 𝒏 = 𝟏, 𝒙 𝒏 𝒃𝒆𝒍𝒐𝒏𝒈𝒔 𝒕𝒐 𝑪𝟏 (𝟏) 𝟎, 𝒙 𝒏 𝒃𝒆𝒍𝒐𝒏𝒈𝒔 𝒕𝒐 𝑪𝟐 (𝟎) BA 01 1 10 00 0 11 F(s)s 𝑿 𝟏 𝑿 𝟐 bin 𝒀𝒋 +1 𝒃 𝟏 +1 𝒃 𝟐 1/0 +1 𝒃 𝟑 𝑾 𝟓 𝑾 𝟔 A B 𝑾 𝟏 𝑾 𝟐 𝑾 𝟑 𝑾 𝟒 s=(𝑿 𝟎 𝑾 𝟎+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟐+…) 𝟎 ≤ η ≤ 𝟏 𝑿(𝒏)=(𝑿 𝟎, 𝑿 𝟏,𝑿 𝟐, …) W(𝒏)=(𝑾 𝟎, 𝑾 𝟏,𝑾 𝟐, …)
- 67. Neural Networks Training Steps Weights Initialization Inputs Application Sum of Inputs-Weights Products Activation Function Response Calculation Weights Adaptation Back to Step 2 1 2 3 4 5 6
- 68. Regarding 5th Step: Weights Adaptation • If the predicted output Y is not the same as the desired output d, then weights are to be adapted according to the following equation: 𝑾 𝒏 + 𝟏 = 𝑾 𝒏 + η 𝒅 𝒏 − 𝒀 𝒏 𝑿(𝒏) Where 𝑾 𝒏 = [𝒃 𝒏 , 𝑾 𝟏(𝒏), 𝑾 𝟐(𝒏), 𝑾 𝟑(𝒏), … , 𝑾 𝒎(𝒏)]
- 69. Neural Networks Training Example Step n=0 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=0: η = .001 𝑋 𝑛 = 𝑋 0 = +1, +1, +1,1, 0 𝑊 𝑛 = 𝑊 0 = 𝑏1, 𝑏2, 𝑏3, 𝑤1, 𝑤2, 𝑤3, 𝑤4, 𝑤5, 𝑤6 = −1.5, −.5, −.5, 1, 1, 1, 1, −2, 1 𝑑 𝑛 = 𝑑 0 = 1 BA 01 1 => 1 10 00 0 => 0 11
- 70. Neural Networks Training Example Step n=0 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin BA 01 1 => 1 10 00 0 => 0 11
- 71. Neural Networks Training Example Step n=0 – SOP – 𝑺 𝟏 𝑺 𝟏=(+𝟏𝒃 𝟏+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) =+1*-1.5+1*1+0*1 =-.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 72. Neural Networks Training Example Step n=0 – Output – 𝑺 𝟏 𝒀 𝑺 𝟏 = = 𝑩𝑰𝑵 𝑺 𝟏 = 𝑩𝑰𝑵 −. 𝟓 = 𝟎 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 73. Neural Networks Training Example Step n=0 – SOP – 𝑺 𝟐 𝑺 𝟐=(+𝟏𝒃 𝟐+𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) =+1*-.5+1*1+0*1 =.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 74. Neural Networks Training Example Step n=0 – Output – 𝑺 𝟐 𝒀 𝑺2 = = 𝑩𝑰𝑵 𝑺2 = 𝑩𝑰𝑵 . 𝟓 = 1 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 75. Neural Networks Training Example Step n=0 – SOP – 𝑺 𝟑 𝑺 𝟑=(+𝟏𝒃 𝟑+𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) =+1*-.5+0*-2+1*1 =.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 76. Neural Networks Training Example Step n=0 – Output – 𝑺 𝟑 𝒀 𝑺3 = = 𝑩𝑰𝑵 𝑺3 = 𝑩𝑰𝑵 . 𝟓 = 1 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 77. Neural Networks Training Example Step n=0 - Output 𝒀 𝒏 = 𝒀 𝟎 = 𝒀 𝑺3 = 1 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 78. Neural Networks Training Example Step n=0 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟎 = 1 𝐝 𝒏 = 𝒅 𝟎 = 1 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 79. Neural Networks Training Example Step n=1 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=1: η = .001 𝑋 𝑛 = 𝑋 1 = +1, +1, +1,0, 1 𝑊 𝑛 = 𝑊 1 = 𝑊 0 = 𝑏1, 𝑏2, 𝑏3, 𝑤1, 𝑤1, 𝑤2, 𝑤3, 𝑤4, 𝑤5, 𝑤6 = −1.5, −.5, −.5, 1, 1, 1, 1, −2, 1 𝑑 𝑛 = 𝑑 1 = +1 BA 01 1 => 1 10 00 0 => 0 11
- 80. Neural Networks Training Example Step n=1 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin BA 01 1 => 1 10 00 0 => 0 11
- 81. Neural Networks Training Example Step n=1 – SOP – 𝑺 𝟏 𝑺 𝟏=(+𝟏𝒃 𝟏+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) =+1*-1.5+0*1+1*1 =-.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 82. Neural Networks Training Example Step n=1 – Output – 𝑺 𝟏 𝒀 𝑺 𝟏 = = 𝑩𝑰𝑵 𝑺 𝟏 = 𝑩𝑰𝑵 −. 𝟓 = 𝟎 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 83. Neural Networks Training Example Step n=1 – SOP – 𝑺 𝟐 𝑺 𝟐=(+𝟏𝒃 𝟐+𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) =+1*-.5+0*1+1*1 =.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 84. Neural Networks Training Example Step n=1 – Output – 𝑺 𝟐 𝒀 𝑺2 = = 𝑩𝑰𝑵 𝑺2 = 𝑩𝑰𝑵 . 𝟓 = 1 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 85. Neural Networks Training Example Step n=1 – SOP – 𝑺 𝟑 𝑺 𝟑=(+𝟏𝒃 𝟑+𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) =+1*-.5+0*-2+1*1 =.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 86. Neural Networks Training Example Step n=1 – Output – 𝑺 𝟑 𝒀 𝑺3 = = 𝑩𝑰𝑵 𝑺3 = 𝑩𝑰𝑵 . 𝟓 = 1 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 87. Neural Networks Training Example Step n=1 - Output 𝒀 𝒏 = 𝒀 𝟏 = 𝒀 𝑺3 = 1 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 88. Neural Networks Training Example Step n=1 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟏 = 1 𝐝 𝒏 = 𝒅 𝟏 = 1 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −2 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 89. Neural Networks Training Example Step n=2 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=2: η = .001 𝑋 𝑛 = 𝑋 2 = +1, +1, +1,0, 0 𝑊 𝑛 = 𝑊 2 = 𝑊 1 = 𝑏1, 𝑏2, 𝑏3, 𝑤1, 𝑤1, 𝑤2, 𝑤3, 𝑤4, 𝑤5, 𝑤6 = −1.5, −.5, −.5, 1, 1, 1, 1, −2, 1 𝑑 𝑛 = 𝑑 2 = 0 BA 01 1 => 1 10 00 0 => 0 11
- 90. Neural Networks Training Example Step n=2 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin BA 01 1 => 1 10 00 0 => 0 11
- 91. Neural Networks Training Example Step n=2 – SOP – 𝑺 𝟏 𝑺 𝟏=(+𝟏𝒃 𝟏+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) =+1*-1.5+0*1+0*1 =-1.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 92. Neural Networks Training Example Step n=2 – Output – 𝑺 𝟏 𝒀 𝑺 𝟏 = = 𝑩𝑰𝑵 𝑺 𝟏 = 𝑩𝑰𝑵 −𝟏. 𝟓 = 𝟎 𝒃𝒊n 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 93. Neural Networks Training Example Step n=2 – SOP – 𝑺 𝟐 𝑺 𝟐=(+𝟏𝒃 𝟐+𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) =+1*-.5+0*1+0*1 =-.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 94. Neural Networks Training Example Step n=2 – Output – 𝑺 𝟐 𝒀 𝑺2 = = 𝑺𝑮𝑵 𝑺2 = 𝑺𝑮𝑵 −. 𝟓 =0 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 95. Neural Networks Training Example Step n=2 – SOP – 𝑺 𝟑 𝑺 𝟑=(+𝟏𝒃 𝟑+𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) =+1*-.5+0*-2+0*1 =-.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 96. Neural Networks Training Example Step n=2 – Output – 𝑺 𝟑 𝒀 𝑺3 = = 𝑩𝑰𝑵 𝑺3 = 𝑩𝑰𝑵 −. 𝟓 = 𝟎 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 97. Neural Networks Training Example Step n=2 - Output 𝒀 𝒏 = 𝒀 𝟐 = 𝒀 𝑺3 = 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 98. Neural Networks Training Example Step n=2 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟐 = 𝟎 𝐝 𝒏 = 𝒅 𝟐 = 𝟎 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 99. Neural Networks Training Example Step n=3 • In each step in the solution, the parameters of the neural network must be known. • Parameters of step n=3: η = .001 𝑋 𝑛 = 𝑋 3 = +1, +1, +1,1, 1 𝑊 𝑛 = 𝑊 3 = 𝑊 2 = 𝑏1, 𝑏2, 𝑏3, 𝑤1, 𝑤1, 𝑤2, 𝑤3, 𝑤4, 𝑤5, 𝑤6 = −1.5, −.5, −.5, 1, 1, 1, 1, −2, 1 𝑑 𝑛 = 𝑑 3 = 0 BA 01 1 => 1 10 00 0 => 0 11
- 100. Neural Networks Training Example Step n=3 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin BA 01 1 => 1 10 00 0 => 0 11
- 101. Neural Networks Training Example Step n=3 – SOP – 𝑺 𝟏 𝑺 𝟏=(+𝟏𝒃 𝟏+𝑿 𝟏 𝑾 𝟏+𝑿 𝟐 𝑾 𝟑) =+1*-1.5+1*1+1*1 =.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 102. Neural Networks Training Example Step n=3 – Output – 𝑺 𝟏 𝒀 𝑺 𝟏 = = 𝑩𝑰𝑵 𝑺 𝟏 = 𝑩𝑰𝑵 . 𝟓 = 1 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 103. Neural Networks Training Example Step n=3 – SOP – 𝑺 𝟐 𝑺 𝟐=(+𝟏𝒃 𝟐+𝑿 𝟏 𝑾 𝟐+𝑿 𝟐 𝑾 𝟒) =+1*-.5+1*1+1*1 =1.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 104. Neural Networks Training Example Step n=3 – Output – 𝑺 𝟐 𝒀 𝑺2 = = 𝑩𝑰𝑵 𝑺2 = 𝑩𝑰𝑵 𝟏. 𝟓 = 1 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 −𝟏, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 105. Neural Networks Training Example Step n=3 – SOP – 𝑺 𝟑 𝑺 𝟑=(+𝟏𝒃 𝟑+𝑺 𝟏 𝑾 𝟓+𝑺 𝟐 𝑾 𝟔) =+1*-.5+1*-2+1*1 =-1.5 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 106. Neural Networks Training Example Step n=3 – Output – 𝑺 𝟑 𝒀 𝑺3 = = 𝑩𝑰𝑵 𝑺3 = 𝑩𝑰𝑵 −𝟏. 𝟓 = 𝟎 𝒃𝒊𝒏 𝒔 = +𝟏, 𝒔 ≥ 𝟎 𝟎, 𝒔 < 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 107. Neural Networks Training Example Step n=3 - Output 𝒀 𝒏 = 𝒀 𝟑 = 𝒀 𝑺3 = 𝟎 BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 108. Neural Networks Training Example Step n=3 Predicted Vs. Desired 𝒀 𝒏 = 𝒀 𝟑 = 𝟎 𝐝 𝒏 = 𝒅 𝟑 = 𝟎 ∵ 𝒀 𝒏 = 𝒅 𝒏 ∴ Weights are Correct. No Adaptation BA 01 1 => 1 10 00 0 => 0 11 s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin
- 109. Final Weights s 𝑿 𝟏 𝑿 𝟐 𝒀𝒋 +1 −𝟏. 𝟓 +1 −. 𝟓 1/0 +1 −. 𝟓 −𝟐 +𝟏 A B +𝟏 +𝟏 +𝟏 +𝟏 bin Current weights predicted the desired outputs.