Simultaneous control of multiple servos with the Arduino

 It is easy enough to control multiple servo motors with the Arduino, but I wanted a way to have them all going to different positions and at different speeds simultaneously. For example I did not want to move Servo A to 77° and afterwards Servo B to 25°. I wanted both servos to move (or appear to move) at the same time.

(For the basics of controlling a servo with the Arduino there's a ton of stuff on the web, this link seems like a good article. Read that first before continuing with this post.)

I'd seen some articles on multitasking on the Arduino and I thought: "Aha, I can use a similar technique for driving multiple servos..."

The main action of an Arduino program takes place in a function called loop, which is called automatically again and again. In our case the loop looks like this:

At the head of the loop you capture the command. The command can come over the serial link from a program running on the PC, or maybe from functions which read sensors. 

If a command comes in which says "move Servo A to 33°" the last block in the above loop will start to do that. But the next time round there could be a command which says to "move Servo B to 90°". And again that last block will move both servos towards their final position, so they appear to be moving simultaneously.

Of course once the servos have reached their positions they do not move until another command arrives for them.

What I also wanted to do was to have each servo have acceleration and deceleration. I do this by having a curve which specifies the speed with respect to time. Time being measured with 0 at the start of the movement. So here is what I mean graphically:

 Notice that the curve above does not reach a speed of 0, else the servo might stop before it gets to its target. That last horizontal line guarantees that movement will continue even if it takes a long time to get to the target.

In the program I don't actually do a curve, I have some discreet steps, look at the comments inside the kSpeedCurves part. Each speed curve has an id, I use ids 'A', 'B' and 'C', which means you can move one servo following one speed curve, and another servo following a different curve. Some faster, some slower, some with more acceleration, some with less.

As far as the program is concerned the speed of the servo is actually simply the angular size of the step to take in each loop.

The command from the PC are not gathered together in one iteration of the loop. A character is read in each iteration until a 0 terminator arrives, then the command is interpreted. Remember that commands do not move the servo, but specify a target for a given servo. The servo moves towards that target. It may be interrupted by another command even before it gets to the target.

The format of the commands contain which servo and which curve to use and the degrees target to move to, see the comments about "VA123" in the sources.

Here is the code, it was written for a specific need of mine, but maybe you'll find something useful in there and can adapt it to you own requirements. Have fun!

// Moving multiple servos at different rates simultanesouly...

#include <Servo.h>

/***************************************************************************************************************/

typedef struct {
    Servo* pServoObj ; // Which servo will be driven by this structure
    int iServoPin ; // On which pin is the servo connected
    double CurrPos ; // Where we are now
    double TargetPos ; // Where we want to go
    unsigned long imsRealStartTime ; // The time when we started going
    char chCurveId ; // For example 'A' which curve in kSpeedCurves to use
} ServoCurver_t ;

/***************************************************************************************************************/

// Each one of these is an entry in the speed curve
typedef struct {
    unsigned long iTimeOrPcent ; // x-axis in ms or in %
    double Speed ; // y-axis, actually a delta angle to apply
} SpeedAt_t ;


// Here is a list of speed curves....
#define NUM_X_COORDS 4
typedef struct {
    char chId ;  
    SpeedAt_t Curve [NUM_X_COORDS] ;
} CurverData_t ;

const CurverData_t kSpeedCurves[] = {
    {
        'A', // the id has to e unique inside this array
        {
            {     0,  1.00},  // at 0 ms, the beginning of the move the delta to apply is 1 degrees
            {   500,  2.00},   // at 0.5s the delta to apply is 2 degrees
            {  2000,  1.50},   // at 2s the delta to apply is 1.5 degrees
            { 15000,  1.25},   // at 15s the delta to apply is 1.25 degrees
        }
    } ,
    {  
        'B',
        {
            {    0, 1.0},
            { 500,  3.75},
            { 7000, 1.5},
            {10000, 0.5},
        }
    },
    {  
        'C',
        {
            {    0, 6.0},
            { 500,  5.0},
            { 900,  4.0},
            {10000, 0.75},
        }
    },
};

const size_t ikTotalNumCurves = sizeof(kSpeedCurves)/sizeof(kSpeedCurves[0]) ;

/***************************************************************************************************************/

int LedPin = 13 ;

/***************************************************************************************************************/

// This stores communications from the PC...
int iSerialCharsRead = 0 ;
#define CMD_BUF_SIZE 100
char szCommand[CMD_BUF_SIZE] = "" ;

void ClearCmdBuffer ()
{
    iSerialCharsRead = 0 ;
    memset (szCommand,CMD_BUF_SIZE,0) ;  
}

/***************************************************************************************************************/

// Create the servo objects
const int ikHeadVerticalPin = 3 ;
ServoCurver_t scHeadVertical  ;
Servo srvHeadVertical;  

const int ikHeadHorizontalPin = 10 ;
ServoCurver_t scHeadHorizontal  ;
Servo srvHeadHorizontal;  

Servo srvRightArm ;
const int ikRightArmPin = 5 ;
ServoCurver_t scRightArm ;

Servo srvLeftArm ;
const int ikLeftArmPin = 6 ;
ServoCurver_t scLeftArm ;

/***************************************************************************************************************/

// Look up which CurverData_t corresponds to the given id
const CurverData_t& GetCurveFromId (char chId)
{
    for (size_t c = 0 ; c < ikTotalNumCurves ; ++c) {
        if (kSpeedCurves[c].chId == chId) {
            return kSpeedCurves[c] ;
        }
    }

    Serial.print ("GetCurveFromId unknown id: ") ;
    Serial.print (chId) ;
    Serial.println() ;  

    // Return something
    return kSpeedCurves[0] ;
}

/***************************************************************************************************************/

// First attempt no interpolation
// Calculates and returns the speed we must apply to the servo given its
// position and the speed curve it is using
// imsDeltaTime is how far into the speed curve the servo is
double GetSpeedFromTimeCurve (char chCurve, unsigned long imsDeltaTime)
{
    // Get hold of that speed curve in an an easy to use way...
    const CurverData_t& kThisTimeSpeedCurve = GetCurveFromId(chCurve) ;

    if (imsDeltaTime == 0) {
        // We are at the very beginning of the curve, return the first value
        //Serial.print (" at time start") ;
        //Serial.println (kThisTimeSpeedCurve.Curve[0].Speed) ;
        return kThisTimeSpeedCurve.Curve[0].Speed ;  
    }

    // Look at every section in the time speed curve...
    for (int x = 0 ; x < NUM_X_COORDS-1 ; ++x) {
        // Are we in the range x to x+1 in the graph?
        const bool kbInRange = (imsDeltaTime >= kThisTimeSpeedCurve.Curve[x].iTimeOrPcent) && (imsDeltaTime <  kThisTimeSpeedCurve.Curve[x+1].iTimeOrPcent) ;
        if (kbInRange) {
            const double kSpeed = kThisTimeSpeedCurve.Curve[x].Speed ;
           
            return kSpeed ;
        }
    }


    // if we get here we are beyond the end of the x-axis of the curve,
    // Assume he curve carries on to infinity from the last value

    const double kSpeed = kThisTimeSpeedCurve.Curve[NUM_X_COORDS-1].Speed ;

    return kSpeed ;
}

/*****************************************************************************************************/

// Get direction required to move towards the target of this servo...
int GetSignForServoMovement (const ServoCurver_t& ServoCurver)
{
    const double kOnTargetMargin = 1.0 ;

    const double kError = abs (ServoCurver.CurrPos - ServoCurver.TargetPos) ;
    if (kError < kOnTargetMargin ) {
        // In this case the servo is already at the target and we don't need to
        // do anything
        return 0 ;
    }

    int iSign ; // In which direction to move the servo

    if (ServoCurver.CurrPos < ServoCurver.TargetPos) {
        // need to move in a positive direction to get to the target
        iSign = +1 ;

    } else if (ServoCurver.CurrPos > ServoCurver.TargetPos) {
        // need to move in a negative direction to get to the target      
        iSign = -1 ;
       
    } else {
        // In this case the servo is already at the target and we don't need to
        // do anything
        iSign = 0 ;
    }

    return iSign ;
}

void MoveServoOnCurveByTime (ServoCurver_t& ServoCurver)
{
    const int ikSign = GetSignForServoMovement (ServoCurver) ;
    if (ikSign == 0) {
        // Servo is at target, no movement to do
        return ;
    }  

    // Get the time now. unsigned long is important
    const unsigned long imsNow = millis() ;
 
    // Get the delta time, how far into the curve you are
    unsigned long imsDelta = imsNow - ServoCurver.imsRealStartTime ;

    // See what speed the curve gives us at this time.  
    const double kSpeed = GetSpeedFromTimeCurve (ServoCurver.chCurveId,imsDelta) ;
   
    if (kSpeed > 0) {
        // Update the current pos...
        ServoCurver.CurrPos = ServoCurver.CurrPos + (ikSign*kSpeed) ;
     
    } else if (kSpeed == 0) {
        // Speed 0 means that time is beyond the end of the curve
        // so we must be at the target position. Update vars and move servo
        // to the final target position
        ServoCurver.CurrPos = ServoCurver.TargetPos ;
    }

    // ...and the actual server
    const int ikIntPos = int(round (ServoCurver.CurrPos)) ;
    ServoCurver.pServoObj->write (ikIntPos) ;
}

/********************************************************************************************/

// This should be called everytime we need a new target, i.e every time a new command comes along
void SetServoCurveAndTarget (ServoCurver_t& ServoCurve, const char chCurveToUse, const double kNewTarget)
{
    // ServoCurve.CurrPos should not be changed, the servo is where it is, wherever that
    // may be, but we want to give it a new *target*.
    ServoCurve.TargetPos = kNewTarget ;

    ServoCurve.chCurveId = chCurveToUse ;  // This is which graph to use

    // Only one of these will be used, depends on what sort of curve it is
    ServoCurve.imsRealStartTime = millis() ; // This is the start time of the command, now
}

// This should be called only once per servo in the setup
void InitServoCurverAndHardware (ServoCurver_t& ServoCurve, Servo* pServoObject, int const ikServoPin, char chCurveToUse)
{
    // Set everything to 90° initially, servos in a halfway position
    const double kDefaultPos = 90.0 ;
    ServoCurve.CurrPos = kDefaultPos ; // this is where it is
 
    SetServoCurveAndTarget (ServoCurve,chCurveToUse,kDefaultPos) ;  

    // Remember what hardware this ServoCurver is attached to...
    ServoCurve.pServoObj = pServoObject ;
    ServoCurve.iServoPin = ikServoPin ;  

    // Attach and set the position
    ServoCurve.pServoObj->attach(ServoCurve.iServoPin) ;  
    ServoCurve.pServoObj->write (int(round(ServoCurve.CurrPos))) ;  
}

/*********************************************************************************************************************************/

void setup() {
    Serial.begin (9600) ;

    InitServoCurverAndHardware (scHeadVertical,
                                &srvHeadVertical,
                                ikHeadVerticalPin,
                                'A') ;

    InitServoCurverAndHardware (scHeadHorizontal,
                                &srvHeadHorizontal,
                                ikHeadHorizontalPin,
                                'A') ;

    InitServoCurverAndHardware (scLeftArm,
                                &srvLeftArm,
                                ikLeftArmPin,
                                'A') ;
 
    InitServoCurverAndHardware (scRightArm,
                                &srvRightArm,
                                ikRightArmPin,
                                'A') ;
   
    pinMode(LedPin, OUTPUT);    

    ClearCmdBuffer () ;
}

unsigned long iTimeToMoveServos = 0 ;
const unsigned long imskServoMoveDelta = 200 ;

void loop() {

    // Read a command character from the PC...
    char c ;
    bool bReadingSerial = false ;
    if (Serial.available() > 0) {
        // read next character
        c = Serial.read();

        bReadingSerial = true ;

        // newline is end of command
        if (c != '\n') {
            // not a newline so still collecting chars from the serial
            szCommand[iSerialCharsRead] = c;
            iSerialCharsRead++;
            digitalWrite(LedPin, LOW);
        }
    }

    // If we have some characters and the last char recieved was a newline...
    // ...we got a command, do it
    // Commands set targets and let the servos get on with it
    // Commands are in the form "VA123", "HB90"
    // V is which servo
    // A is which curve to use
    // 123 is the target angle
    if ((iSerialCharsRead > 0) && (c == '\n')) {
        // We have a command...
        szCommand[iSerialCharsRead] = 0 ; // zero terminate the command
        Serial.println(szCommand);

        // Gather data from the command string...
        char chServo = szCommand[0] ;  // 'H' or 'V' etc
        char chCurve = szCommand[1] ;  // 'A' for example
        double kNewTarget = atof(szCommand+2) ;// Degrees position of new target
        if ((kNewTarget >= 0.0) && (kNewTarget <= 180.0)) {
        // Act on the command...
        switch (chServo) {            
            case 'H':
                // Horizontally turn the head
                SetServoCurveAndTarget (scHeadHorizontal,chCurve,kNewTarget) ;
                break ;
           
            case 'V':
                // Vertically nod the head
                SetServoCurveAndTarget (scHeadVertical,chCurve,kNewTarget) ;
                break ;
           
            case 'R':
                // Move the right arm...
                SetServoCurveAndTarget (scRightArm,chCurve,kNewTarget) ;
                break ;
           
            case 'L':
                // Move the leftt arm...
                SetServoCurveAndTarget (scLeftArm,chCurve,kNewTarget) ;
                break ;
           
            default:
               // Don't understand the command, ignore it
               break ;
            }
        }
     
        // get ready to read another command
        ClearCmdBuffer () ;
    }  

    // Whethere there has been a command or not the servos carry on moving till
    // they get to their targets
    if (!bReadingSerial) {
        if (millis() > iTimeToMoveServos) {
            MoveServoOnCurveByTime (scHeadVertical) ;      
            MoveServoOnCurveByTime (scHeadHorizontal) ;      
            MoveServoOnCurveByTime (scLeftArm) ;
            MoveServoOnCurveByTime (scRightArm) ;
            iTimeToMoveServos = millis() + imskServoMoveDelta ;
        }
    }
}
    
 Note, the code above can oscillate for a while near the target position. This versione of MoveServoOnCurveByTime solves that problem, see kbSameSign...

static void MoveServoOnCurveByTime (CServoCurver& ServoCurver)
{
    const int ikSign = GetSignForServoMovement (ServoCurver) ;
    if (ikSign == 0) {
        // Servo is at target, no movement to do
        return ;
    }  

    // Get the time now. unsigned long is important
    const unsigned long imsNow = millis() ;
 
    // Get the delta time, how far into the curve you are
    unsigned long imsDelta = imsNow - ServoCurver.m_imsRealStartTime ;

    // See what speed the curve gives us at this time.  
    const double kSpeed = GetSpeedFromTimeCurve (ServoCurver.m_chCurveId,imsDelta) ;

    // Where are we now?
    const double kCurrPos = ServoCurver.m_CurrPos ;
    const double kCurrDelta = kCurrPos - ServoCurver.m_TargetPos ;

    // Where would we move to?
    const double kNewPos = ServoCurver.m_CurrPos + (ikSign*kSpeed) ;
    const double kNewDelta = kNewPos - ServoCurver.m_TargetPos ;

    // 2021-07-11 This change stops oscillations around the target
    const bool kbBothNegative = (kNewDelta < 0.0) && (kCurrDelta < 0.0) ;
    const bool kbBothPositive = (kNewDelta > 0.0) && (kCurrDelta > 0.0) ;
    const bool kbSameSign = kbBothNegative || kbBothPositive ;
 
    if (kbSameSign && (kSpeed > 0)) {
        // Update the current pos...
        ServoCurver.m_CurrPos = kNewPos ;
        
    } else {
        // Moving to the new position we would overshoot the target, so just move
        // to the target
        ServoCurver.m_CurrPos = ServoCurver.m_TargetPos ;
    }

    // Now move the actual servo
    const int ikIntPos = int(round (ServoCurver.m_CurrPos)) ;
    ServoCurver.m_ServoObj.write (ikIntPos) ;
}

    
 

 

 

 

 

 

 

 

A diabolical for-loop

Older people sometimes think that experience is knowledge. Sometimes it is, and sometimes it isn’t. In my experience.

Look at this:

Ooops now, sorry, that's it for those of you who are not technically minded. You can stop reading now, just after you consider this post sent to me by my daughter. Who must have thought, for some reason that I would appreciate the sentiment:

 Bye. Now for those remaining this...

...caused me some grief. It is the last part of a for loop:

    for (int e = 0 ; e < ikNumDegSymbols ; e = e++) {
        if (eIn == kDegsDescs[e].e) {
            return kDegsDescs[e].pszDesc;
        }
    }

What is strange is that I normally use e++, or i++ or whatever....

    for (int e = 0 ; e < ikNumDegSymbols ; e++) {

I must have been tired or dreaming when I used e = e++. But it has worked for years, this unnoticed glitch, in this one for-loop.

It worked until it didn't, one day the behaviour changed, suddenly the for loop never exited, and debugging revealed that e was stuck at was some random value, 23567 for example. At most it should have done 3 loops.

I asked some programmer friends, and they enlightened me. In the C++ language the behaviour of e=e++ is undefined! I'd never even considered that to be a possibility. Uninitialized variables are one thing, an increment and assignment being undefined is another.

Undefined behaviour means that the compiler can do WTF it wants. And an upgrade to the compiler had changed how it interpreted e = e++. I have no idea what it thinks of that operation now, seems almost like an unitialised variable.

One programmer in particular, Alessio Nava, much younger than me, said he always used ++e, the behaviour of which is defined. So now I always use ++e.

Oh well, older people sometimes think that experience is knowledge. Sometimes it is, and sometimes it isn’t. In my experience.

Using a C# EXE to interact with AutoCAD

I'm updating this because I've found out that the method explained below has problems, and that you'd do better making a C# DLL to run inside AutoCAD, or use C# with ObjectDBX to run as an EXE which talks to AutoCAD.

The problem is that Microsoft changed the way .NET works, details here, which means that Documents.Open  can sometimes fail. The error code is HRESULT: 0x80010001 (RPC_E_CALL_REJECTED). It is to do with the fact that AutoCAD is still initializing when the call is made, so you need to add some sort of delay, or catch an exception, or maybe wait till it is visible. This is my solution, though I don't like having to do this:

            AcadDocument AcadDoc = null;

            int iNumTries = 0;
            const int ikMaxTries = 400;
            while (iNumTries < ikMaxTries)
            {
                bool bError = false ;
                try
                {
                    AcadDoc = AcadApp.Documents.Open(@"C:\DisegniSviluppati\04.DWG");
                    AcadDoc.Activate();
                }

                catch (Exception e1)
                {
                    Debug.WriteLine("Catch on try " + iNumTries.ToString() + ", exception: " + e1.Message + "\n");
                    bError = true;
                }

                if (!bError)
                {
                    break;
                }

                if (iNumTries == ikMaxTries)
                {
                    break;
                }

                iNumTries++;
            }

            if (iNumTries == ikMaxTries)
            {
                // Failure
                return;
            }

            // Now we will loop over all the entities in the drawing we have just saved and re-opened...
            var ModelSpace = AcadDoc.ModelSpace;
            int iNumEntities = ModelSpace.Count;

Essentially I stay in a loop trying to open the drawing until I don't get an exception. The source below will work generally, but be ready to strange and random exceptions, depending on the timing of your program.

Here's the old article:

When cool beats useful, the NETGEAR N300 WNR2000v5 router

You'd think after all these years of studies on design for usability companies would have learned something. Netgear has not. Here is their NETGEAR N300 WNR2000v5 router.

The electronics work decently (I have to do a reset every two days or so), but why have they hidden the WPS button in plain site? If you do not do network setups as your job then you do it at most once a year. So the icons mean nothing. And WPS is meant to make things easy for ordinary users.

Where on earth is the WPS button which would allow me to connect simply and quickly to a wi-fi booster? I've already given you a clue by showing you only 1 of the 6 sides...there are other buttons hidden and unlabeled on the other sides of the device.

It is the triangle next to the padlock. And here you can see where "cool" beats useful. The designer wanted to be "cool" so the icon is tiny and non standard. The buttons, instead of being round are "coolly" triangular and look like LEDs. The icon is actually only 3mm wide, so anybody with vision problems is not going to be able to see it clearly.

The "cool" designer needs to go back to school and learn about usability. Oddly enough an earlier design (2017?) was much clearer:

The button is round, there is the icon (useful if you know what it means, and not the standard for WPS) but there is the acronym WPS in clear lettering underneath it.

 

The Visualization of Points of View

I'm a fan of Edward Tufte and have all three of his books on Visualization of Data (see bottom of this page). It occurred to me that you could show the two "extreme" points of view of Covod-19 in a single info-graphic.

So there are those who say Covid-19 is a hoax or not really a problem, and there are those who say it is a catastrophe for humanity. I think the two points of view are summed up by this image:

The population of England and Wales is 57 million, and the number of deaths from Covid-19 this year is about 42,000 (so far). So the doubters say: "Deaths caused by Covid are less than 0.074% of the population." 

You can hardly see the thin red line under the red text of the bar graph.

Personally I think that, yes, the number of people directly affected is very very tiny, but the effect on the nurses, doctors, actual patients, our healthcare system is huge, and we must respect that fact.

Anyway this info graphic came to my mind so I made it.

Here are the wonderful books of Edward Tufte, full of examples of good and bad info-graphics, a joy to see, read and understand. Anybody who thinks that Excel's graphing functions are all you need to know, should read at least one of these.

Neural Networks on the Arduino Part 6: Putting it all together, training and testing your neural network

We are almost there! We can nearly put all the code together. All we need is some test data to call UpdateWeights with (for the training) and some test data (for the testing).

At this point it would be a good idea copy the source code into the Arduino IDE and follow this article while looking at the sources.

For one of the tests I'm going to use the same one as I used here in the Python implementation by Tariq Rashid. The neural network is presented with a spike in a graph and must return the position of that spike.

Here is how I create the test data:

// This creates a test input vector and the ideal output for that input
void CreatePositionTrainingSample (mtx_type* TrainInputVector,
                                   mtx_type* TrainTargetVector)
{
    // A check to stop silly errors as you meddle with the code...
    if (NUM_OUTPUTS != 2) {
        Serial.print ("Training sample error, I want 2 outputs!") ;
    }
 
    // Choose a place from 0 to (NUM_INPUTS-1)
    int iPos = random (0,NUM_INPUTS) ;  
   
    // Make a vector with a spike at the randomly chosen iPos
    for (int i = 0 ; i < NUM_INPUTS ; i++) {
        if (i == iPos) {
            TrainInputVector[i] = 1.0 ;      
        } else {
            TrainInputVector[i] = 0.0 ;
        }
    }
    // Now we have an input vector with a single non zero value

    // What is the expected output number?
    // We want one output to be at the "value of the position" so to speak
    double OutputValue1 = (double)iPos/double(NUM_INPUTS-1) ;

    // Just to have more than one output...
    // ...make other output to be at the "opposite of the value of the position"
    double OutputValue2 = 1.0 - OutputValue1 ;

    // This is the idea correct answer...
    TrainTargetVector[0] = OutputValue1 ;
    TrainTargetVector[1] = OutputValue2 ;
}

So if the function creates an input vector with a spike in the middle:

{0,0,1,0,0}  

I get an output value of 0.5.

And if the function creates an input vector with a spike at the end of the vector.

{0,0,0,0,1}  

I get an output vector 1.0.

And so on.


Actually, to keep things interesting, I have two outputs, the position of the spike and its mirror image, that is what OutputValue2 is in the above function.

Be careful with the use of random(). The second parameter is exclusive.

That position test is actually a simple problem for neural networks, a harder one is XOR. The XOR problem is what prompted the use of the hidden layer. The XOR problem cannot be solved with a single layer neural network. There's plenty on the internet about the XOR problem so I won't go into it here, but to make sure we've got things right in our two layer Arduino neural network we must test it with the XOR problem.

Basically if there are two inputs when both are active (=1) the output should be inactive (0). And if only one of the two inputs are active then the output should be active.

Looking at it another way, the neural network should detect if the two inputs are different. If they are both 0 or both 1 then the output should be 0.

Here's a table which illustrates that:


inputs       XOR    XNOR
0    0        0      1
0    1        1      0
1    0        1      0
1    1        0      1



I've added XNOR an output as well, just to make it interesting. XNOR is simply the opposite of XOR, as you can see in the above table.

Here's how I create a sample for the XOR training and testing:

void CreateXORTrainingSample (mtx_type* TrainInputVector, mtx_type* TrainTargetVector)
{
    // A check to stop silly errors as you meddle with the code...
    if (NUM_OUTPUTS != 2) {
        Serial.print ("Training sample error, I want 2 outputs!") ;
    }
    if (NUM_INPUTS != 2) {
        Serial.print ("Training sample error, I want 2 inputs!") ;
    }
    
    // Choose a row in the truth table...
    int iWhichRow = random (0,4) ;  // will give me a number from 0 to 3 inclusive

    if (iWhichRow == 0) {
        TrainInputVector[0] = 0 ;  // INPUT 1   
        TrainInputVector[1] = 0 ;  // INPUT 2
        TrainTargetVector[0] = 0 ; // XOR  
        TrainTargetVector[1] = 1 ; // XNOR  
       
    } else if (iWhichRow == 1) {
        TrainInputVector[0] = 1 ;     
        TrainInputVector[1] = 0 ;     
        TrainTargetVector[0] = 1 ;  
        TrainTargetVector[1] = 0 ;  

    } else if (iWhichRow == 2) {
        TrainInputVector[0] = 0 ;     
        TrainInputVector[1] = 1 ;     
        TrainTargetVector[0] = 1 ;  
        TrainTargetVector[1] = 0 ;  
   
    } else {
        TrainInputVector[0] = 1 ;     
        TrainInputVector[1] = 1 ;     
        TrainTargetVector[0] = 0 ;  
        TrainTargetVector[1] = 1 ;  
    }
}

Now, in the whole source code I've made it possible for you to use either of these tests by simply setting a define to 1:

// To decide the type of test one of these should be 1 and the other 0
#define XOR_TEST 0
#define POSITION_TEST 1


The number of inputs outputs and hidden nodes will change according to what sort of test you use.

Note also that in the same place in the code I set the  NUM_TRAINING_SAMPLES to depend on if we are doing XOR_TEST or POSITION_TEST. NUM_TRAINING_SAMPLES is much higher for the XOR test because XOR is a harder problem.

The top level of the program starts at void loop() near the end of the sources. First the network is trained, and then it is tested.

You need to run the serial monitor to see the output of the training and of the testing. Here is the output when POSITION_TEST is used:

TrainOnOneSample is called repeatedly (from inside RunMultipleTrain) in order to slowly modify the weights.

Inside TrainOnOneSample is the define QUERY_AS_YOU_UPDATE. If that is set to 1 then you will test the neural network after every single train and print the results. If it is set to 0 you will only see the final test results.

Now it is up to you to decide what...

1. light inputs
2. light outputs
3. motors
4. servos
5. solenoids
6. switches
7. relays
8. varistors
9. transistors
10. robots
11. robot arms
12. robot legs
13. touch sensors
14. pressure sensors
15. meteorological sensors
16. ...

...you want to use this program with!

Whatever you decide I think you'll probably need to increase the number of hidden nodes.

Part 7 the complete source code to a neural network on the Arduino

Neural Networks on the Arduino Part 5: How to update the weights

Previously I said that since we have to sets of weights to update (= two matrices to change) we may as well write a single function and call it twice.
 

So what are the inputs and outputs of our back propagation weight adjusting C like function? I'd suggest...

1. iNumLocalInputNodes
2. InputValues (a vector  iNumLocalInputNodes big)
3. iNumLocalOutputNodes
4. OutputValues (a vector iNumLocalOutputNodes big)
5. ErrorValues (a vector iNumLocalOutputNodes big) 
6. WeightMatrix (with iNumLocalInputNodes rows and iNumLocalOutputNodes columns)

I like to draw (pencil and paper away from the computer!) diagrams of functions I'm going to write, just to make sure I have them correct in my head. If I don't have them clear in my head I will have a snowflake's chance in hell of writing the function correctly.

Note that the inputs and outputs are not necessarily the inputs and outputs of the whole network. The above function will be called first for the hidden to output part and then again for the input to hidden part. That is the point of writing the function, so it can be used in more than one place.

Here is the code, note that I use the WeightMatrix as both an input and output.

// Change the weights matrix so that it produces less errors
void UpdateWeights (int iNumInputNodes,  mtx_type* InputValues, // a col vector
                    int iNumOutputNodes, mtx_type* OutputValues, // a row vector
                    mtx_type* ErrorValues, // same size as output values
                    mtx_type* WeightMatrix) // This is an input and output
{
    // This is just to keep sizes of matrices in mind
    int iNumRows = iNumInputNodes ;
    int iNumCols = iNumOutputNodes ;
   
    // The f "horizontal" row vector is formed from
    // alfa*(error*(output*(1-output)) in each column
    // Initialised from errors and outputs of this layer, and so has the same
    // size as the error vector and output vector
    mtx_type f[iNumOutputNodes] ;

    for (int col = 0 ; col < iNumOutputNodes ; col++) {
        // The outouts have been created using the sigmoid function.
        // The derivative of the sigmnoid is used to modify weights.
        // Fortunately, because we have the outputs values, the derivative
        // is easy to calculate...Look up derivative of sigmoid
        const double SigmoidDeriv = OutputValues[col]*(1.0-OutputValues[col]) ;
        f[col] = Alpha*ErrorValues[col]*SigmoidDeriv ;
    }

    // The "vertical" column vector is the inputs to the current layer

    // Now we can do the outer product to form a matrix from a
    // a column vector multiplied by a row vector...
    // to get a matrix of delta weights
    mtx_type ErrorDeltasMat [iNumRows*iNumCols] ;
    OuterProduct((mtx_type*)InputValues,

                  f,

                  iNumRows, 

                  iNumCols,

                  (mtx_type*)ErrorDeltasMat) ;

    // Now we have the deltas to add to the current matrix
    // We are simply doing OldWeight = OldWeight+DeltaWeight here
    for (int row = 0 ; row < iNumRows ; row++) {
        for (int col = 0 ; col < iNumCols ; col++) {
            int iIndex = (row*iNumCols)+col ;
            WeightMatrix[iIndex] = WeightMatrix[iIndex] +

                                   ErrorDeltasMat[iIndex] ;
        }
    }

}

In the above code Alpha is the learning rate, often between 0.1 and 0.2, and a constant defined elsewhere in the Arduino program.

The SigmoidDeriv is the derivative of the sigmoid function, and is calculated like this:

S' = S(1-S)

Now the outputs of our layers are the sigmoids of the inputs. So the derivative is simply (theoutputs*(1-theoutputs)) as shown in the code above. Simples!

Part 6 Putting It All Together