New IBM brain-like chip: 4096 cores, 1 million neurons, 5.4 billion transistors

[witeken]

IBM cracks open a new era of computing with brain-like chip: 4096 cores, 1 million neurons, 5.4 billion transistors

Scientists at IBM Research have created by far the most advanced neuromorphic (brain-like) computer chip to date. The chip, called TrueNorth, consists of 1 million programmable neurons and 256 million programmable synapses across 4096 individual neurosynaptic cores. Built on Samsung’s 28nm process and with a monstrous transistor count of 5.4 billion, this is one of the largest and most advanced computer chips ever made. Perhaps most importantly, though, TrueNorth is incredibly efficient: The chip consumes just 72 milliwatts at max load, which equates to around 400 billion synaptic operations per second per watt — or about 176,000 times more efficient than a modern CPU running the same brain-like workload, or 769 times more efficient than other state-of-the-art neuromorphic approaches.

This chip would be great for 3D stacking. With a few thousand layers, neuron count could be greatly increased. The article also says this thing is production ready.

 

[monstercameron]

how does the start and stop uarch work, no clock speeds?

 

[jdubs03]

This is really cool, I hope IBM can commercialize this quickly. The trickle-down effects from this could be absolutely tremendous, big data is really going to benefit. I didn't realize how slow software simulation of neural networks was, even on Xeon-class CPUs. This steps it up to a whole new level.

In the article it states that the fastest supercomputer is 4.5 GFLOPs/W (Tesla K40), but to note AMD just announced the FirePro S9150 gets 10.8 GFLOPs/W putting out 2.53 TFLOPS of Double-Precision performance. (Knight's Landing will be ~15 GFLOPs/W.

Here is a paper from Cornell back in 2013: http://vlsi.cornell.edu/~ni49/Imam_IJCNN_2013.pdf

The paper is: A million spiking-neuron integrated circuit with a scalable communication network and interface, its behind a paywall.

 

[DigDog]

it should be able to run minecraft maxed out

 

[biostud]

Good for face recognition and video surveillance.

 

[Bubbleawsome]

it should be able to run minecraft maxed out

To be fair, even my 4670k doesn't always play nice even at 4.3Ghz. They really didn't do much in optimization there. Then again it is java.

 

[jhu]

In other news, IBM has, for some reason, changed their name to Cyberdyne Systems.

 

[ninaholic37]

And all employees got upgraded to 8K monitors (judging by the image size).

 

[MisterMac]

It's amazing how much IBM Research in general does in providing some sort of "POC" for certain theories.


I wonder how Intel\TSMC etc, would be without indications from IBM that certain technologies are technicly possible - so their engineers can figure out how to make it profitable.

 

[witeken]

I didn't realize how slow software simulation of neural networks was, even on Xeon-class CPUs. This steps it up to a whole new level.

That's the problem with simulation. A CPU is very different from an animal brain. You need an EFLOPS supercomputer to simulate the whole human brain, with a power consumption of ~30MW (projected in 2018-2020), which is a million times more than what an actual brain consumes, although we have tiny 22nm transistors. Some people say that human brains are very efficient, but you really can't compare them with computers.

 

[bakalu]

this is gpu or cpu ?

 

[witeken]

this is gpu or cpu ?

More like a GPU because it has loads of cores (neurons). But it isn't really comparable because it's neither.

 

[cytg111]

http://www.theverge.com/2014/8/3/5965099/elon-musk-compares-artificial-intelligence-to-nukes

 

[cytg111]

Also, you dont go and make such a chip just cause u can, you have specific jobs that there is a market for, yet there is little mention of that outside of "neural nets" .. Wonder what the real application is.

 

[Roland00Address]

Making a computer act like a human brain is very hard.

Human brains use a lot of intuition. A cpu or gpu is not designed to do intuition, you can brute force it with hardware, and if you use intelligently designed software you can use intuitive algorithms, but to use intuition organically, that is asking for a whole lot.

Computers use procedure and sequence to do math and other types of problems. They can do this without limit, never getting tired, never making a mistake, and at a damn fast rate. It is easier to design software that uses deduction instead of induction, and it is easier to design software to use induction over intuition.

 

[witeken]

http://www.theverge.com/2014/8/3/5965099/elon-musk-compares-artificial-intelligence-to-nukes

I'm not as pessimistic as him, rather optimistic. Computers thus far are only used to compute. It would be poor human use that could make computers dangerous, but that would be because humans are dangerous, not computers. I'm not afraid of computer intelligence because that's real intelligence, not the evolutionary biological pseudo intelligence of humans.

 

[witeken]

Making a computer act like a human brain is very hard.

Human brains use a lot of intuition. A cpu or gpu is not designed to do intuition, you can brute force it with hardware, and if you use intelligently designed software you can use intuitive algorithms, but to use intuition organically, that is asking for a whole lot.

But in the end this intuition is just the result of physical brain activity. If it's possible to make a brain out of silicon, and apparently it is, making the chip compute as a brain could be possible.

 

[cytg111]

Making a computer act like a human brain is very hard.

Human brains use a lot of intuition...

Intuition isnt a "thing", intuition is just logic that you havent put words to yet, rest assured at the bottom level intuition is just 1's and 0's.

 

[cytg111]

But in the end this intuition is just the result of physical brain activity. If it's possible to make a brain out of silicon, and apparently it is, making the chip compute as a brain could be possible.

There is one thing that the biological construct has going for it vs silicon and its concurrency. Ie having neuron x fire 2.5 times the rate of neuron y and all the aftermath of that several layers down is hard to virtualize.

 

[videogames101]

There is one thing that the biological construct has going for it vs silicon and its concurrency. Ie having neuron x fire 2.5 times the rate of neuron y and all the aftermath of that several layers down is hard to virtualize.

read: impossible at this juncture

 

[Roland00Address]

Intuition isnt a "thing", intuition is just logic that you havent put words to yet, rest assured at the bottom level intuition is just 1's and 0's.

True. Intuition is just recognizing their is a pattern (or falsely recognizing their is a pattern when there really is not one), you recognize the pattern but you do not understand the why their is a pattern yet.

Intuition is kinda logical and kinda isn't. I am not trying to diss intuition, no I am in complete awe of intuition. It is one of my strongest skills, but it is also so hard to understand. Intuition is not like deduction, but we use intuition, deduction, and induction all together to make rational decisions.

In the BBC sherlock episode the Sign of Three. There is a wonderful scene where Sherlock notices something subconsciously due to intuition, but he can't figure it out. He switches between using deduction and induction (induction represented via his brother in a mindscape) until he figures out the crime.

-----

Intuition pattern recognizing is why the human brain is so much better at recognizing faces than a computer. Most computer software that recognizes images tries to look at faces in some form of sequential matter. But change enough details such as color, angles, shadows etc and the sequential matter is useless. Things like facial recognition would in theory be something this type of computer chip would be good at.

 

[witeken]

I don't think it has anything to do with intuition. Our brain just has a dedicated area for recognizing faces.

 

[Ajay]

Intuition isnt a "thing", intuition is just logic that you havent put words to yet, rest assured at the bottom level intuition is just 1's and 0's.

Not really, since it's analog and synapses are also modulated by a host of neurotransmitters. This added to the massively asynchronous behavior neurons, that you mention, makes developing a digital equivalent of the brain pretty, well, mind boggling.

 

[Roland00Address]

I don't think it has anything to do with intuition. Our brain just has a dedicated area for recognizing faces.

Yes our brain does have dedicated areas for certain types of problems. Wikipedia article on face perception. We use lots of our occipital and temporal lobe with the recognizing of faces it is even speculated that a specific part of the temporal lobe is specialized for this purpose called the Fusiform face area.

The algorithm we use though is probably very different than the algorithm computers use for the processing of information. Our brains are much more abstract than computers. Furthermore not all the parts of our brain "think" the same way, our frontal cortex is different than our temporal lobe, which is different than the sensory part of our brain in the parietal lobe, etc. We have specialized parts of our brain and since these parts are specialized they probably use different algorithms, different styles of thinking.

Some parts of our frontal lobe, work more like a computer does with the traditional type of computer algorithms. They are much more concrete, they are much more sequential, the break down tasks from big tasks to small tasks etc. The use of math by contrasts is more of parietal lobe part.

----

What I am trying to say is computers are much concrete in their thinking than the human brain is. The human brain is much more abstract. Intuition is highly linked with being abstract. The ability to create new types of variables, with new parameters, and new rules is very much abstract thinking.

 

[Roland00Address]

Not really, since it's analog and synapses are also modulated by a host of neurotransmitters. This added to the massively asynchronous behavior neurons, that you mention, makes developing a digital equivalent of the brain pretty, well, mind boggling.

Even neurotransmitters and brain cells in the end are justs 1:0. One neurotransmitter may influences dozens of other cells and how it affects one receptor may produce X reaction, but when exposed to a different receptor it produces a different Y reaction.

Sure we may not not obtain a completely perfect 1:1 overlap with a digital to analog conversion or vice versa. But we can get a very close overlap, and that overlap may be within a certain "percentile" such as 90% confidence, 99% confidence, 99.9% confidence, 99.99% confidence, etc with the limit of that confidence being 1.00. With the more of the human brain we learn about we can be sure we are getting much closer to how the brain really works. We may not know everything but we will certainly know a whole lot more.

-----

There is a quote that according to the internet came from a mathematician Ian Stewart. "
“If our brains were simple enough for us to understand them, we'd be so simple that we couldn't.”