Neural Network Backpropagation? -


can recommend website or give me brief of how backpropagation implemented in nn? understand basic concept, i'm unsure of how go writing code.

many of sources i've found show equations without giving explanation of why they're doing it, , variable names make difficult find out.

example:

void bpnn_output_error(delta, target, output, nj, err) double *delta, *target, *output, *err; int nj; {   int j;   double o, t, errsum;    errsum = 0.0;   (j = 1; j <= nj; j++) {     o = output[j];     t = target[j];     delta[j] = o * (1.0 - o) * (t - o);     errsum += abs(delta[j]);   }   *err = errsum;  } 

in example, can explain purpose of

delta[j] = o * (1.0 - o) * (t - o); 

thanks.

the purpose of

delta[j] = o * (1.0 - o) * (t - o);

is find error of output node in backpropagation network.

o represents output of node, t expected value of output node.

the term, (o * (1.0 - o), derivative of common transfer function used, sigmoid function. (other transfer functions not uncommon, , require rewrite of code has sigmoid first derivative instead. mismatch between function , derivative mean training not converge.) node has "activation" value fed through transfer function obtain output o, like

o = f(activation)

the main thing backpropagation uses gradient descent, , error gets backward-propagated application of chain rule. problem 1 of credit assignment, or blame if will, hidden nodes output not directly comparable expected value. start known , comparable, output nodes. error taken proportional first derivative of output times raw error value between expected output , actual output.

so more symbolically, we'd write line as

delta[j] = f'(activation_j) * (t_j - o_j)

where f transfer function, , f' first derivative of it.

further in hidden layers, error @ node estimated contribution errors found @ next layer. deltas succeeding layer multiplied connecting weights, , products summed. sum multiplied first derivative of activation of hidden node delta hidden node, or

delta[j] = f'(activation_j) * sum(delta[k] * w_jk)

where j references hidden node , k node in succeeding layer.


Comments

Popular posts from this blog

c++ - Convert big endian to little endian when reading from a binary file -

C#: Application without a window or taskbar item (background app) that can still use Console.WriteLine() -

unicode - Are email addresses allowed to contain non-alphanumeric characters? -