Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
NeuroDimension Neuro Solutions HELP
1. Table of Contents
Preface 32
About On-line Help ................................................................................. 32
Acknowledgments .................................................................................. 32
Product Information 33
Contacting NeuroDimension.................................................................... 33
NeuroSolutions Technical Support ........................................................... 34
NeuroDimension Products and Services .................................................. 35
Level Restrictions ................................................................................... 39
Evaluation Mode..................................................................................... 40
NeuroSolutions Pricing ........................................................................... 40
NeuroSolutions University Site License Pricing ......................................... 41
Ordering Information ............................................................................... 41
Getting Started 42
System Requirements............................................................................. 42
Running the Demos ................................................................................ 42
What to do after Running the Demos ....................................................... 45
Frequently Asked Questions (FAQ) ......................................................... 45
Terms to Know....................................................................................... 50
Main Window ................................................................................... 50
Inspector.......................................................................................... 51
Breadboards .................................................................................... 52
Neural Components.......................................................................... 53
Toolbars and Palettes ....................................................................... 53
Selection and Stamping Modes ......................................................... 54
Temporary License........................................................................... 54
Menus & Toolbars .................................................................................. 55
File Menu & Toolbar Commands ....................................................... 55
Edit Menu & Toolbar Commands ....................................................... 56
Alignment Menu & Toolbar Commands .............................................. 57
Windows Menu & Toolbar Commands .............................................. 58
Component Menu ............................................................................. 59
Tools ............................................................................................... 59
Tools Menu Commands .............................................................. 59
Control Menu & Toolbar Commands ............................................ 61
Macro Menu & Toolbar Commands ............................................. 62
Customize Toolbars Page ........................................................... 63
Component Palettes......................................................................................................................................63
Command Toolbars .......................................................................................................................................64
Customize Buttons Page............................................................. 65
View ................................................................................................ 66
View Menu ................................................................................. 66
Macro Bars ................................................................................ 66
Status Bar.................................................................................. 68
Help................................................................................................. 69
Help Menu & Toolbar Commands ................................................ 69
Activate Software Panel .............................................................. 70
User Options .......................................................................................... 71
Options Window ............................................................................... 71
Options Workspace Page.................................................................. 72
1
2. Options Save Page........................................................................... 73
Examples ............................................................................................... 74
Example 1 - Toolbar Manipulation...................................................... 74
Example 2 - Component Manipulation................................................ 74
Example 3 - Inspecting a Component's Parameters ............................ 75
Simulations 76
Simulations ............................................................................................ 76
Introduction to Neural Network Simulations .............................................. 77
What Are Artificial Neural Networks ................................................... 77
A Prototype Problem......................................................................... 78
Ingredients of a Simulation ...................................................................... 79
Formulation of the problem................................................................ 79
Data Collection and Coding ............................................................... 79
Getting Data into the Network ............................................................ 80
Cross Validation ............................................................................... 81
Network Topology ............................................................................ 82
Network Training .............................................................................. 84
Probing ............................................................................................ 86
Running the Simulation ..................................................................... 86
Concepts 86
Concepts ............................................................................................... 86
NeuroSolutions Structure ........................................................................ 87
NeuroSolutions Structure .................................................................. 87
Palettes ........................................................................................... 87
Breadboard...................................................................................... 88
NeuroSolutions Graphical User Interface (GUI) ........................................ 89
NeuroSolutions Graphical User Interface (GUI) .................................. 89
Logic of the Interface........................................................................ 89
Components .................................................................................... 90
The Inspector................................................................................... 90
Single-Click vs. Double-Click............................................................. 91
File Open Dialog Box ........................................................................ 92
Save As Dialog Box .......................................................................... 92
Toolbars and Palettes ....................................................................... 92
Title Bar........................................................................................... 93
Scroll Bars ....................................................................................... 93
Network Construction ....................................................................... 94
Network Construction ................................................................. 94
Stamping ................................................................................... 94
Manipulating Components........................................................... 94
Replacing Axons and Synapses .................................................. 95
Connectors ................................................................................ 95
Cabling ...................................................................................... 96
Stacking..................................................................................... 97
Network Access ............................................................................... 97
Network Access ......................................................................... 97
Probes ....................................................................................... 98
Data Input/Output ....................................................................... 99
Transmitters and Receivers ........................................................100
Network Simulation..........................................................................100
Network Simulation....................................................................100
Application Window Commands .......................................................100
2
3. Size command (System menu) 100
Move command (Control menu) 101
Minimize command (application Control menu)............................101
Maximize command (System menu) 101
Close command (Control menus) 101
Restore command (Control menu) 102
Switch to command (application Control menu) 102
Generating Source Code .......................................................................103
Generating Source Code .................................................................103
Customized Components .......................................................................103
Customized Components .................................................................103
Testing the Network ...............................................................................104
The TestingWizard ..........................................................................104
Freezing the Network Weights..........................................................104
Cross Validation ..............................................................................104
Production Data Set.........................................................................105
Sensitivity Analysis..........................................................................105
Confusion Matrix .............................................................................105
Correlation Coefficient .....................................................................106
ROC Matrix .....................................................................................107
Performance Measures ....................................................................108
Practical Simulation Issues.....................................................................109
Practical Simulation Issues...............................................................109
Associating a File Extension with an Editor........................................110
Data Preparation .............................................................................110
Normalization File............................................................................111
Forms of Backpropagation ...............................................................112
Probing ...........................................................................................113
Saving and Fixing Network Weights..................................................113
Weights File ....................................................................................113
Saving Network Data .......................................................................118
Stop Criteria ....................................................................................118
Constructing Learning Dynamics ......................................................118
Simulating Recurrent Networks ........................................................119
Component Naming Conventions .....................................................119
Coordinating Unsupervised and Supervised Learning ........................121
Organization of NeuroSolutions ..............................................................121
Organization of NeuroSolutions ........................................................121
Activation Family .............................................................................122
Activation Family .............................................................................122
Axon Family ..............................................................................124
MemoryAxon Family ..................................................................125
FuzzyAxon Family .....................................................................127
ErrorCriteria Family .........................................................................128
Synapse Family.........................................................................130
Backprop Family..............................................................................131
Backprop Family..............................................................................131
BackAxon Family.......................................................................132
BackMemoryAxon Family...........................................................134
BackSynapse Family .................................................................136
3
4. GradientSearch Family ....................................................................137
GradientSearch Family ..............................................................137
Controls Family ...............................................................................139
Controls Family .........................................................................139
ActivationControl Family ............................................................140
BackpropControl Family.............................................................142
Unsupervised Family .......................................................................143
Unsupervised Family .................................................................143
Hebbian Family .........................................................................144
Competitive Family ....................................................................145
Kohonen Family ........................................................................146
Probe Family...................................................................................147
Probe Family.............................................................................147
Input Family ....................................................................................148
Input Family ....................................................................................148
Transmitter Family...........................................................................149
Transmitter Family...........................................................................149
Schedule Family..............................................................................150
Schedule Family..............................................................................150
Introduction to Neural Computation 151
Introduction to NeuroComputation ..........................................................151
Introduction to Neural Computation.........................................................152
Introduction to NeuroComputation ....................................................152
History of Neural Networks...............................................................152
What are Artificial Neural Networks...................................................153
Neural Network Solutions .................................................................154
Neural Network Analysis ........................................................................155
Neural Network Analysis ..................................................................155
Neural Network Taxonomies ............................................................157
Learning Paradigms.........................................................................159
Learning Paradigms...................................................................159
Cost Function ............................................................................160
Gradient Descent ......................................................................161
Constraining the Learning Dynamics.................................................166
Constraining the Learning Dynamics...........................................166
Practical Issues of Learning ...................................................................168
Practical Issues of Learning .............................................................168
Training Set ....................................................................................168
Network Size...................................................................................169
Learning Parameters .......................................................................169
Stop Criteria ....................................................................................170
Unsupervised Learning ..........................................................................171
Unsupervised learning .....................................................................171
Support Vector Machines .......................................................................174
Support Vector Machines .................................................................174
Dynamic Networks.................................................................................176
Dynamic Networks...........................................................................176
Famous Neural Topologies ....................................................................177
Famous Neural Topologies ..............................................................177
Perceptron ......................................................................................177
Multilayer Perceptron .......................................................................178
Madaline .........................................................................................180
Radial Basis Function Networks .......................................................180
Associative Memories ......................................................................181
Jordan/Elman Networks ...................................................................182
4
5. Hopfield Network .............................................................................183
Principal Component Analysis Networks ...........................................184
Kohonen Self-Organizing Maps (SOFM) ...........................................185
Adaptive Resonance Theory (ART) ..................................................187
Fukushima ......................................................................................187
Time Lagged Recurrent Networks.....................................................187
Tutorials 190
Tutorials Chapter ...................................................................................190
Running NeuroSolutions ........................................................................190
Signal Generator Example .....................................................................191
Signal Generator Example ...............................................................191
Construction Rules ..........................................................................192
Stamping Components ....................................................................193
On-line Help ....................................................................................193
Connectors .....................................................................................193
Selecting and Configuring a Component ...........................................194
Arranging Icons ...............................................................................194
Connecting Components..................................................................195
The Cursor......................................................................................195
Component Compatibility .................................................................195
Bringing in the Function Generators ..................................................196
Stacking Components......................................................................196
Accessing the Component Hierarchy ................................................197
Access Points .................................................................................198
Displaying the Output Waveform ......................................................198
Opening the Display Window............................................................199
Controlling Data Flow ......................................................................199
Configuring the Controller ................................................................199
Running the Signal Generator Example ............................................200
Things to Try with the Signal Generator ............................................201
What You have Learned from the Signal Generator Example .............202
Combination of Data Sources Example ...................................................203
Combination of Data Sources Example .............................................203
Constructing a McCulloch-Pitts Processing Element ..........................203
Preparing Files for Input into NeuroSolutions .....................................204
Things to Try with the Combination of Data Sources Example ............206
What You have Learned from the Combination of Data Sources Example207
The Perceptron and Multilayer Perceptron ..............................................207
Perceptron and Multilayer Perceptron Example .................................207
Perceptron Topology .......................................................................207
Constructing the Learning Dynamics of a Perceptron .........................208
Alternate Procedure for Constructing the Learning Dynamics of a Perceptron 210
Selecting the Learning Paradigm......................................................211
Running the Perceptron ...................................................................212
MLP Construction ............................................................................213
Running the MLP .............................................................................214
Things to Try with the Perceptron and Multilayer Perceptron Example 215
What You have Learned from the Perceptron and Multilayer Perceptron Example
......................................................................................................216
Associator Example ...............................................................................217
Associator Example .........................................................................217
Building the Associator ....................................................................217
Things to Try with the Associator ......................................................221
What you have Learned from the Associator Example .......................221
Filtering Example...................................................................................222
5
6. Filtering Example.............................................................................222
Constructing A Linear Filter..............................................................222
Things to Try with the Linear Filter....................................................224
Adaptive Network Construction.........................................................225
Running the Adaptive Network .........................................................225
Things to Try with the Adaptive Network ...........................................226
What You have Learned from the Filter Example ...............................229
Recurrent Neural Network Example ........................................................229
Recurrent Neural Network Example ..................................................229
Creating the Recurrent Topology ......................................................229
Fixed Point Learning ........................................................................231
Running the Recurrent Network ........................................................232
Things to Try with the Recurrent Network ..........................................234
What You have Learned from the Recurrent Network Example...........237
Frequency Doubler Example ..................................................................237
Frequency Doubler Example ............................................................237
Creating the Frequency Doubler Network ..........................................237
Configuration of the Trajectory..........................................................239
Running the Frequency Doubler Network ..........................................239
Using the Gamma Model to Double the Frequency ............................240
Visualizing the State Space..............................................................242
Things to Try with the Frequency Doubler Network ............................244
What You have Learned from the Frequency Doubler Example ..........246
Unsupervised Learning Example ............................................................247
Unsupervised Learning Example ......................................................247
Introduction to Unsupervised Learning ..............................................247
Noise Reduction with Oja's or Sanger's Learning ...............................248
Things to Try with the Unsupervised Network ....................................249
What You have Learned from the Unsupervised Learning Example ....251
Principle Component Analysis Example ..................................................251
Principal Component Analysis Example ............................................251
Introduction to Principal Component Analysis....................................251
Running the PCA Network ...............................................................251
Things to Try with the PCA Network..................................................253
What You have Learned from the Principal Component Analysis Example 253
Competitive Learning Example...............................................................253
Competitive Learning Example.........................................................253
Introduction to Competitive Learning .................................................253
Constructing the Competitive Network ..............................................254
Things to Try with the Competitive Network .......................................256
What You have Learned from the Competitive Learning Example.......258
Kohonen Self Organizing Feature Map (SOFM) Example .........................259
Kohonen Self Organizing Feature Map (SOFM) Example ...................259
Introduction to SOFM Example .........................................................259
SOFM Network Construction ............................................................259
Running the SOFM Network .............................................................260
Things to Try with the SOFM Network ...............................................261
What you have Learned from the Kohonen SOFM Example ...............261
Character Recognition Example .............................................................261
Character Recognition Example .......................................................261
Introduction to Character Recognition Example .................................262
Constructing the Counterpropagation Network...................................262
Running the Counterpropagation Network .........................................263
Things to Try with the Counterpropagation Network ...........................264
What You have Learned from the Character Recognition Example .....265
Pattern Recognition Example .................................................................265
6
7. Pattern Recognition Example ...........................................................265
Introduction to Pattern Recognition Example .....................................266
Constructing the Pattern Recognition Network ...................................266
Running the Pattern Recognition Network .........................................268
What you have Learned from the Pattern Recognition Example..........269
Time Series Prediction Example.............................................................269
Time Series Prediction Example.......................................................269
Introduction to Time Series Prediction Example .................................269
Constructing the TLRN Network .......................................................270
Running the TLRN Network ..............................................................271
What You have Learned from the Time Series Prediction Example .....271
Neural Network Components 271
Components .........................................................................................271
Engine Family .......................................................................................272
Activation Family ...................................................................................272
Axon Family ....................................................................................272
Axon.........................................................................................272
BiasAxon ..................................................................................273
CombinerAxon ..........................................................................274
GaussianAxon...........................................................................275
LinearAxon................................................................................276
LinearSigmoidAxon ...................................................................277
LinearTanhAxon........................................................................278
NormalizedAxon........................................................................279
NormalizedSigmoidAxon ............................................................280
SigmoidAxon.............................................................................281
SoftMaxAxon.............................................................................282
TanhAxon .................................................................................283
ThresholdAxon..........................................................................284
WinnerTakeAllAxon ...................................................................285
Access Points ...........................................................................286
Axon Family Access Points........................................................................................................................286
DLL Implementation...................................................................287
Axon DLL Implementation..........................................................................................................................287
BiasAxon DLL Implementation...................................................................................................................288
GaussianAxon DLL Implementation.........................................................................................................288
LinearAxon DLL Implementation...............................................................................................................289
LinearSigmoidAxon DLL Implementation.................................................................................................290
LinearTanhAxon DLL Implementation......................................................................................................291
SigmoidAxon DLL Implementation............................................................................................................292
SoftMaxAxon DLL Implementation............................................................................................................292
TanhAxon DLL Implementation.................................................................................................................293
ThresholdAxon DLL Implementation........................................................................................................294
WinnerTakeAllAxon DLL Implementation ................................................................................................295
Examples ..................................................................................296
Axon Example...............................................................................................................................................296
BiasAxon Example.......................................................................................................................................297
GaussianAxon Example..............................................................................................................................298
LinearAxon Example...................................................................................................................................299
LinearSigmoidAxon Example.....................................................................................................................300
LinearTanhAxon Example..........................................................................................................................301
SigmoidAxon Example................................................................................................................................302
SoftMaxAxon Example................................................................................................................................303
TanhAxon Example.....................................................................................................................................304
ThresholdAxon Example.............................................................................................................................305
WinnerTakeAllAxon Example.....................................................................................................................306
Macro Actions ...........................................................................307
7