Can we get the ball rolling on some open neural network stuff? I have a neural net fetish but there aren’t many hobbyist tools out there to play with.
Introduction:
The great thing about neural nets is that instead of programing a new algorithm for each system you want to control, you program one algorithm that generates algorithms to solve any given problem. [In nature, this approach has proven to be remarkably efficient and robust even when [compared to the best custom designed algorithms produced by decades of research. They’re especially handy for modeling non-linear problems.
Here’s an over-simplified explanation of how they work:
Each sensor (i.e. each pixel on a CMOS camera chip) has a corresponding “neuron” in the input layer. The number of neurons in a layer corresponds to the number of dimensions that that layer maps data to. So, the input from each sensor is initially assumed to be linearly independent from all other sensors. Of course, the input from pixels on a camera sensor are not completely independent, so the inputs to this sensor layer will map to a lower dimensional manifold. If you feed the output from the sensor layer into a second layer with fewer neurons (fewer dimensions to map data to), the second layer will reduce the dimensionality of the data. In doing so, it will model relationships between the inputs.
[Here’s a decent introduction to the field
[Here’s a great video on the state of the art (circa 2007)
In the above video, Dr. Hinton invites the audience to visit [his website where all the Matlab code for his demonstrations is freely available. I believe a good starting point (on the software side) would be to translate this code to Octave so that you don’t have to take out a second mortgage for a copy of Matlab.
On the hardware side, I’d really like to see a heterogeneous computing platform for hobbyists. A cortex M3 connected to an FPGA with some on-board RAM would be great. Sadly, each new generation of FPGAs seems to have fewer and fewer non-BGA packaged devices.
[Altera’s Cyclone III with 40K Logic cells is the largest such device, but it’s a little pricey and doesn’t have cascading DSP blocks with accumulators or distributed RAM for registers etc. which kind of cripples it.
[Lattice’s ECP2 with 20K logic cells has cascading DSP blocks and distributed ram but the built in DDR memory interfaces are only available on BGA-packaged devices. Also, at less than half the price of the Cyclone, it has about 1/4 the embedded RAM and 1/5 the embedded multipliers (though they run about twice as fast).
Finally, there’s the 9K logic cell [Spartan-6 from Xilinx. It’s pretty much the same story as the lattice part but with half the resources and a little nicer DSP blocks. It will probably end up being more cost effective (more resources/$) than the Lattice ECP2 because the ECP2 is lagging the Spartan-6 by about two process nodes (ECP2 is made on 90 nm silicon, Spartan-6 is made on 45 nm), however; I don’t think the Spartan-6 goes into full production until mid-2011.
There, I’ve said my piece. Anyone else interested in this stuff?](http://www.xilinx.com/products/spartan6/index.htm)](http://www.latticesemi.com/products/fpga/ecp2/index.cfm?source=topnav)](http://www.altera.com/products/devices/cyclone3/overview/cy3-overview.html)](Home Page of Geoffrey Hinton)](http://www.youtube.com/watch?v=AyzOUbkUf3M)](Introduction to Neural Networks)](Press Center | MIT News | Massachusetts Institute of Technology)](http://www.wired.com/wiredscience/2009/11/fly-eyes/)
