We are almost there! We can nearly put all the code together. All we need is some test data to call UpdateWeights with (for the training) and some test data (for the testing).
At this point it would be a good idea copy the source code into the Arduino IDE and follow this article while looking at the sources.
For one of the tests I'm going to use the same one as I used here in the Python implementation by Tariq Rashid. The neural network is presented with a spike in a graph and must return the position of that spike.
Here is how I create the test data:
// This creates a test input vector and the ideal output for that input
void CreatePositionTrainingSample (mtx_type* TrainInputVector,
mtx_type* TrainTargetVector)
{
// A check to stop silly errors as you meddle with the code...
if (NUM_OUTPUTS != 2) {
Serial.print ("Training sample error, I want 2 outputs!") ;
}
// Choose a place from 0 to (NUM_INPUTS-1)
int iPos = random (0,NUM_INPUTS) ;
// Make a vector with a spike at the randomly chosen iPos
for (int i = 0 ; i < NUM_INPUTS ; i++) {
if (i == iPos) {
TrainInputVector[i] = 1.0 ;
} else {
TrainInputVector[i] = 0.0 ;
}
}
// Now we have an input vector with a single non zero value
// What is the expected output number?
// We want one output to be at the "value of the position" so to speak
double OutputValue1 = (double)iPos/double(NUM_INPUTS-1) ;
// Just to have more than one output...
// ...make other output to be at the "opposite of the value of the position"
double OutputValue2 = 1.0 - OutputValue1 ;
// This is the idea correct answer...
TrainTargetVector[0] = OutputValue1 ;
TrainTargetVector[1] = OutputValue2 ;
}
So if the function creates an input vector with a spike in the middle:
{0,0,1,0,0}
I get an output value of 0.5.
And if the function creates an input vector with a spike at the end of the vector.
{0,0,0,0,1}
I get an output vector 1.0.
And so on.
Actually, to keep things interesting, I have two outputs, the position of the spike and its mirror image, that is what OutputValue2 is in the above function.
Be careful with the use of random(). The second parameter is exclusive.
That position test is actually a simple problem for neural networks, a harder one is XOR. The XOR problem is what prompted the use of the hidden layer. The XOR problem cannot be solved with a single layer neural network. There's plenty on the internet about the XOR problem so I won't go into it here, but to make sure we've got things right in our two layer Arduino neural network we must test it with the XOR problem.
Basically if there are two inputs when both are active (=1) the output should be inactive (0). And if only one of the two inputs are active then the output should be active.
Looking at it another way, the neural network should detect if the two inputs are different. If they are both 0 or both 1 then the output should be 0.
Here's a table which illustrates that:
inputs XOR XNOR
0 0 0 1
0 1 1 0
1 0 1 0
1 1 0 1
I've added XNOR an output as well, just to make it interesting. XNOR is simply the opposite of XOR, as you can see in the above table.
Here's how I create a sample for the XOR training and testing:
void CreateXORTrainingSample (mtx_type* TrainInputVector, mtx_type* TrainTargetVector)
{
// A check to stop silly errors as you meddle with the code...
if (NUM_OUTPUTS != 2) {
Serial.print ("Training sample error, I want 2 outputs!") ;
}
if (NUM_INPUTS != 2) {
Serial.print ("Training sample error, I want 2 inputs!") ;
}
// Choose a row in the truth table...
int iWhichRow = random (0,4) ; // will give me a number from 0 to 3 inclusive
if (iWhichRow == 0) {
TrainInputVector[0] = 0 ; // INPUT 1
TrainInputVector[1] = 0 ; // INPUT 2
TrainTargetVector[0] = 0 ; // XOR
TrainTargetVector[1] = 1 ; // XNOR
} else if (iWhichRow == 1) {
TrainInputVector[0] = 1 ;
TrainInputVector[1] = 0 ;
TrainTargetVector[0] = 1 ;
TrainTargetVector[1] = 0 ;
} else if (iWhichRow == 2) {
TrainInputVector[0] = 0 ;
TrainInputVector[1] = 1 ;
TrainTargetVector[0] = 1 ;
TrainTargetVector[1] = 0 ;
} else {
TrainInputVector[0] = 1 ;
TrainInputVector[1] = 1 ;
TrainTargetVector[0] = 0 ;
TrainTargetVector[1] = 1 ;
}
}
Now, in the whole source code I've made it possible for you to use either of these tests by simply setting a define to 1:
// To decide the type of test one of these should be 1 and the other 0
#define XOR_TEST 0
#define POSITION_TEST 1
The number of inputs outputs and hidden nodes will change according to what sort of test you use.
Note also that in the same place in the code I set the NUM_TRAINING_SAMPLES to depend on if we are doing XOR_TEST or POSITION_TEST. NUM_TRAINING_SAMPLES is much higher for the XOR test because XOR is a harder problem.
The top level of the program starts at void loop() near the end of the sources. First the network is trained, and then it is tested.
You need to run the serial monitor to see the output of the training and of the testing. Here is the output when POSITION_TEST is used:
TrainOnOneSample is called repeatedly (from inside RunMultipleTrain) in order to slowly modify the weights.
Inside TrainOnOneSample is the define QUERY_AS_YOU_UPDATE. If that is set to 1 then you will test the neural network after every single train and print the results. If it is set to 0 you will only see the final test results.
Now it is up to you to decide what...
- light inputs
- light outputs
- motors
- servos
- solenoids
- switches
- relays
- varistors
- transistors
- robots
- robot arms
- robot legs
- touch sensors
- pressure sensors
- meteorological sensors
- ...
...you want to use this program with!
Whatever you decide I think you'll probably need to increase the number of hidden nodes.
No comments:
Post a Comment