Backpropagation neural network

OCR using a tiny backpropagation neural network of 80 neurons

by Jason Tiscione

HTML

<body style='text-align:left; font-family: Calibri'>
    <div style='display:block'>
      <div style='float:left;margin-right:20px;margin-bottom:20px'>
        <div style='text-align:center; font-family: '>
          <p style='text-align:center'><b>OPTICAL CHARACTER RECOGNITION</b></p>
          <p>Draw a letter or a digit in the grid below.</p>
        </div>
        <canvas id='canvas' width='370' height='310'>
          Canvas not supported
        </canvas>
        <div>
          <input id='eraser_box' type='checkbox' id='eraser_box'/>
          <label for='eraser_box'>eraser mode</label>
          <input id='clear_button' type='button' value='CLEAR' style='margin-left:90px;margin-right:110px'/>
        </div>
        <script src='abc.js'></script>
      </div>
      <p><br><br><br><br></p>
      <p>This is a small experiment, using a feedforward neural network of 80 neurons, pre-trained using a backpropagation algorithm on handwriting inputs drawn inside a training application written in Java.</p>
      <p>Performance of this particular network isn't terribly impressive, since I crammed the network's base-94 encoded string literal into 720 lines of JavaScript.</p>
      <p>After a backpropation network has been trained, using it is fast enough to do synchronously within a JavaScript mouse event handler. It only requires an input vector to undergo two array multiplications, with a nonlinear sigmoid function applied to each element of the intermediate vector product to perturb what would otherwise be a brain-dead linear system.</p>
      <p>The resemblance to biological neurons is weak. Real neurons don't use backpropagation during learning, and they just barely resemble feedforward artificial neural networks at all. They are smoothly arranged into columnar structures and can't be simply classified as occupying discrete layers  (input, hidden, output) required of an artificial feedforward network.</p>
      <p>In general, backpropagation training does...

JavaScript

var MIN_BRIGHTNESS = 0, MAX_BRIGHTNESS = 1, GRID_HUE = 0, LETTER_HUE = 60, SATURATION = 100,
  clear_button = document.getElementById('clear_button'),
  eraser_box = document.getElementById('eraser_box'),
  canvas = document.getElementById('canvas'),
  context = canvas.getContext('2d'),
  outputSymbols = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789",
  bbox = canvas.getBoundingClientRect(),
  dragging = false, demo_letter = true,
  lastDraggedX = undefined, lastDraggedY = undefined,
  brush_mask = [[1.0, 0.9, 0.5, 0.3, 0.2],
                [0.9, 0.7, 0.4, 0.2, 0.1],
                [0.5, 0.4, 0.3, 0.2, 0.1],
                [0.3, 0.2, 0.2, 0.1, 0.0],
                [0.2, 0.1, 0.1, 0.0, 0.0]],
  HORIZ_TILE_COUNT, VERT_TILE_COUNT, TILE_WIDTH, TILE_HEIGHT,
  R, S1, S2, S3, inputs, W1, b1, W2, b2, W3, b3, responses;
  
deserialize();
setupDemoLetter();
draw();

function multiply (a, b) {
  var c = new Array(a.length);
  for (var i = 0; i < a.length; i++) {
    c[i] = new Array(b[0].length);
    for (var j=0; j < b[0].length; j++) {
      c[i][j] = 0;
      for (var k = 0; k < b.length; k++) {
        c[i][j] += a[i][k] * b[k][j];
      }
    }
  }
  return c;
}

function logsigmoid(ip, b) {
  var numRows = ip.length, numCols = ip[0].length;
  var out = new Array(numRows);
  for (var i = 0; i < numRows; i++) {
    out[i] = new Array(numCols);
    for (var j = 0; j < numCols; j++) {
      out[i][j] = 1.0 / (1.0 + Math.exp(-b[j]-ip[i][j]));
    }
  }
  return out;
}

function draw() {
  drawGrid();
  ocr();
  drawLetters();
}

function drawGrid() {
  context.fillStyle = '#202020';
  context.beginPath();
  context.rect(0, 0, canvas.width, canvas.height);
  context.fill();
  context.strokeStyle = 'gray';
  context.lineWidth = 0.5;
  var r = 0, c = 0;
  for (var i = 0; i < R; i++) {
    context.beginPath();
    context.fillStyle = 'hsl(' + GRID_HUE + ',' + SATURATION + '%,' + 50 * inputs[i] + '%)';
    context.rect(c * TILE_WIDTH + 1, r * TILE_HEIGHT + 1, TILE_WIDTH,...