Beta
×

Welcome to the Slashdot Beta site -- learn more here. Use the link in the footer or click here to return to the Classic version of Slashdot.

Thank you!

Before you choose to head back to the Classic look of the site, we'd appreciate it if you share your thoughts on the Beta; your feedback is what drives our ongoing development.

Beta is different and we value you taking the time to try it out. Please take a look at the changes we've made in Beta and  learn more about it. Thanks for reading, and for making the site better!

Reading Guide To AI Design & Neural Networks?

kdawson posted more than 5 years ago | from the open-the-library-doors-hal dept.

Books 266

Raistlin84 writes "I'm a PhD student in theoretical physics who's recently gotten quite interested in AI design. During my high school days, I spent most of my spare time coding various stuff, so I have a good working knowledge of some application programming languages (C/C++, Pascal/Delphi, Assembler) and how a computer works internally. Recently, I was given the book On Intelligence, where Jeff Hawkins describes numerous interesting ideas on how one would actually design a brain. As I have no formal background in computer science, I would like to broaden my knowledge in the direction of neural networks, pattern recognition, etc., but don't really know where to start reading. Due to my background, I figure that the 'abstract' theory would be mostly suited for me, so I would like to ask for a few book suggestions or other directions."

cancel ×

266 comments

Sorry! There are no comments related to the filter you selected.

PDP (5, Informative)

kahizonaki (1226692) | more than 5 years ago | (#25957457)

Parallel Distributed Processing (both books) by Rumelhart, McClelland, and the PDP research group, 1986. "THE" classic neural network resource--and still somewhat relevant.

Re:PDP (3, Informative)

agravier (1411419) | more than 5 years ago | (#25957615)

For a somewhat more up-to-date and maybe complementary book, I advise you Computational Explorations in Cognitive Neuroscience by Randall C. O'Reilly and Yuko Munakata (The MIT Press). The simulator intends to extend and replace PDP++ and is quite pleasant to use. It is on http://grey.colorado.edu/emergent/index.php/Main_Page [colorado.edu]

Re:PDP (4, Interesting)

babbs (1403837) | more than 5 years ago | (#25957757)

I prefer James Anderson's "An Introduction to Neural Networks". I think it is better suited for someone coming from the physical, mathematical, or neuro- sciences.

Re:PDP (2, Interesting)

kahizonaki (1226692) | more than 5 years ago | (#25957859)

The great thing about the PDP books is that they make almost NO assumption as to what the reader's background is. There's no code, a bunch of pictures, and something in there for everyone. Each chapter is written with a specific goal in mind, and by leaders in the field--there are chapters on the mathematics of the networks, the dynamical properties of them (i.e. how they can be thought of as boltzmann's machines), as well as lots of ideas for applications and specific studies of how real experiments worked. In addition, of course, there is the chapters which actually introduce the different types of networks--and there are equations (and appendices of equations--in case one likes them even more) which can be ignored if one wishes. Overall, in addition to an interesting read in general, by offering the opportunity to just pick-and-choose what one's interested in after reading the initial bit, these books are extremely dynamic and I recommend them strongly. Not to mention you can buy the full set in hardback used (off of amazon or whatever) for ten dollars (what a deal!).

Re:PDP (2, Informative)

Schwarzchild (225794) | more than 5 years ago | (#25958109)

Cosma Shalizi is also a Physicist. I don't think he is actually doing research in machine learning or AI but he likes to read a lot and he tends to have fairly extensive reading lists.

Machine Learning [umich.edu]

AI [umich.edu]

You may also want to get familiar with Geoffrey Hinton's current work in neural networks [youtube.com] .

The Resistance (5, Funny)

Anonymous Coward | more than 5 years ago | (#25957483)

Due to the possibility of a robot army rising up, I refuse to help.

Re:The Resistance (0)

Anonymous Coward | more than 5 years ago | (#25957913)

robot army is already...ANY cop.

Re:The Resistance (1, Funny)

Anonymous Coward | more than 5 years ago | (#25958125)

You can die a virgin or you can die after having sex with a Summer Glau lookalike killing machine.
Hard choice.

Normally I'm pro-Jew (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#25958217)

But fuck Al Franken. He fits the profile of the stereotypical Jew perfectly - angry, self-hating, and greedy. Hell, he didn't even write his own book, "Liars..." Like most celebrity authors, it was ghosted for him.

AIMA (5, Informative)

omuls are tasty (1321759) | more than 5 years ago | (#25957487)

Artificial Intelligence: A Modern Approach by Rusell and Norvig is more or less the standard AI textbook and the book I'd suggest to get an overview of AI and its different methodologies. Mind you, it's over 1000 pages, but a very interesting read.

Re:AIMA (3, Interesting)

xtracto (837672) | more than 5 years ago | (#25957619)

I must second that, Russel and Norvig book is one of the most important books.

I would also recommend:

Artificial Intelligence: A new Synthesis [google.com] from Nills J. Nilson [wikipedia.org] , who is considered one of the founders of A.I.

Re:AIMA (1)

backwardMechanic (959818) | more than 5 years ago | (#25957927)

If it's the book I think it is, it gives a good overview of 'traditional' AI (rules, logic systems, planning) but not really anything about 'soft' approaches like neural nets. I found it rather disappointing. Read any of the classic Rob Brooks papers. If nothing else, they are certainly inspiring - they always make me want to build robots.

Re:AIMA (2, Informative)

Anonymous Coward | more than 5 years ago | (#25957989)

I'd like to add to this. AIMA gives you a very broad and moderately deep overview of the state of AI ten years ago. As such, it is a truly excellent introduction introduction to the subject.

If you want a more recent, much more thorough and narrow introduction to neural networks in particular and machine learning in general, I'd recommend Chris Bishop's book: Pattern Recognition and Machine Learning (http://research.microsoft.com/~cmbishop/prml/), which focuses on learning rather than searching and planning. An outstanding more broad, shallow and dated book on machine learning is Tom Mitchell's book, Machine Learning (http://www.cs.cmu.edu/~tom/mlbook.html)

(Posting AC for the obvious reason that I can't be bothered to create an account)

Re:AIMA (0)

Anonymous Coward | more than 5 years ago | (#25958041)

Also seconded.

Re:AIMA (2, Informative)

Yvanhoe (564877) | more than 5 years ago | (#25958145)

Agreed. All the basic knowledge about the field is in this book. Part of these are available freely online. You can be judge : http://aima.cs.berkeley.edu/ [berkeley.edu]

Re:AIMA (1)

Der PC (1026194) | more than 5 years ago | (#25958223)

I agree. Russell&Norvig is THE introductory AI book to read.

I'll add to your reading list: Reinforcement Learning: An introduction by Sutton & Barto. It's a very well written book which should come as a natural follow-up for R&N.

AI != design brain (4, Insightful)

Kupfernigk (1190345) | more than 5 years ago | (#25957499)

There is a very big difference between AI - which is based on guesses about how "intelligence" works, and studies of brain function. I'm going to make a totally unjustified sweeping generalisation and suggest that one reason that AI has generally been a failure is because we have had quite wrong ideas about how the brain actually works. That's to say, the focus has been on how the brain seems to be like a distributed computer (neurons and the axons that relay their output) because up till now nobody has really understood how the brain stores and organises memory in parallel- which seems to be the key to it all, and is all about the software.

So my feeling is that the first people really to get anywhere with AI will either work for Google or be the neurobiologists who finally crack what is actually going on in there. If I wasn't close to retirement, and wanted to build a career in AI, I'd be looking at how mapreduce works, and the work being done building on that, rather than robotics. I'd also be looking as seriously parallel processing.

So my initial suggestion is nothing to do with conventional AI at all - look at Programming Erlang, and anything you can find about how Google does its stuff.

Re:AI != design brain (2, Funny)

Anonymous Coward | more than 5 years ago | (#25957691)

The human brain does not use anything that even remotely resembles software. The brain is hardwired.

Software in brains... that a paddlin'

Re:AI != design brain (3, Funny)

dmbasso (1052166) | more than 5 years ago | (#25957791)

The universe is software, the brain workings are just a tiny side-effect, but can still be considered software.

From universe.c:

int main()
{
      [...]
      return 42;
}

Re:AI != design brain (2, Informative)

Dan East (318230) | more than 5 years ago | (#25957717)

http://www.databasecolumn.com/2008/01/mapreduce-a-major-step-back.html [databasecolumn.com]

As both educators and researchers, we are amazed at the hype that the MapReduce proponents have spread about how it represents a paradigm shift in the development of scalable, data-intensive applications. MapReduce may be a good idea for writing certain types of general-purpose computations, but to the database community, it is:

      1. A giant step backward in the programming paradigm for large-scale data intensive applications

      2. A sub-optimal implementation, in that it uses brute force instead of indexing

      3. Not novel at all -- it represents a specific implementation of well known techniques developed nearly 25 years ago

      4. Missing most of the features that are routinely included in current DBMS

      5. Incompatible with all of the tools DBMS users have come to depend on

Theres nothing magical about parallel computation (3, Interesting)

Viol8 (599362) | more than 5 years ago | (#25957853)

.. as applied to normal computers. In this case its simply speeded up serial computation - ie the algorithm could be run serially so Programming Erlang is irrelevant. With the brain , parallel computation is *vital* to how it works - it couldn't work serially - some things MUST happen at the same time - eg different inputs to the same neuron, so studying parallel computation in ordinary computers is a complete waste of time if you want to learn how biological brains work. Its comparing apples and oranges.

Re:AI != design brain (1)

drfireman (101623) | more than 5 years ago | (#25957895)

You may be right, but it's never been a major goal of AI researchers to duplicate how the brain works. AI has been steadfastly interested in building machines that do what the brain does, but not how the brain does it. So while I'm sure that many AI researchers keep an eye on these things, I don't think that "wrong ideas about how the brain actually works" is the problem, since ideas about how the brain works have relatively little influence on AI.

As an aside, MapReduce is not that complicated, nor is it particularly novel except in scale. Many people who are interested in AI, the brain, or both understand it pretty thoroughly and don't get much insight from it. So if you're otherwise right about things, I'll put my money on the neurobiologists, the systems neuroscientists, and all the other groups of researchers trying to understand memory and other brain functions.

Start with Dennet's "Consciousness Explained" (1)

anw (42556) | more than 5 years ago | (#25957905)

Firstly it will get you thinking about the relationship between brains and minds, and how the later might be built out of the former. Secondly, Dennet is very interested in the technical aspects of all this and provides lots of suggestions for further reading.

Re:AI != design brain (1)

Black Parrot (19622) | more than 5 years ago | (#25957975)

There is a very big difference between AI - which is based on guesses about how "intelligence" works, and studies of brain function. I'm going to make a totally unjustified sweeping generalisation and suggest that one reason that AI has generally been a failure is because we have had quite wrong ideas about how the brain actually works. That's to say, the focus has been on how the brain seems to be like a distributed computer (neurons and the axons that relay their output) because up till now nobody has really understood how the brain stores and organises memory in parallel- which seems to be the key to it all, and is all about the software.

A lot of the brain's function is architectural, rather than merely a matter of 'software'.

I don't know if you can say "AI has generally been a failure", but traditional AI has actually been guided by the non-biological notion of a "physical symbol system" rather than by conceptions about how the brain actually works. And even in the biologically inspired side of the field, only the most ignorant would think that artificial neural networks have much in common with the brain.

The field of AI, with few execptions, has given up - or at least indefinitely postponed - attempts to create a HAL 9000 style intelligence. With few exceptions, everyone works on methods applicable to a single problem, or at best a very narrow range of problems. It's not possible to draw clean line between the fields of AI and algorithms.

So my feeling is that the first people really to get anywhere with AI will either work for Google or be the neurobiologists who finally crack what is actually going on in there.

We do have some very accurate cortical simulators. AFAIK, they still only model a very small chunk of the cortex, and not the whole brain at all. I'm not aware that they're telling us much about "intelligence" yet either.

If I wasn't close to retirement, and wanted to build a career in AI, I'd be looking at how mapreduce works, and the work being done building on that, rather than robotics. I'd also be looking as seriously parallel processing.

Here the reader begins to wonder whether you know anything about what you're talking about.

Re:AI != design brain (1)

ion.simon.c (1183967) | more than 5 years ago | (#25957985)

If I wasn't close to retirement, and wanted to build a career in AI, I'd be looking at how mapreduce works...

Why not do this stuff during your retirement? What else are you going to do with the time between now and your death?

Re:AI != design brain (1)

Have Brain Will Rent (1031664) | more than 5 years ago | (#25958237)

What else are you going to do with the time between now and your death?

Revenge?

Re:AI != design brain (1)

khallow (566160) | more than 5 years ago | (#25958249)

No kidding. Retirement sound tailor-made for pie-in-the-sky projects like this. Course maybe Kupfernigk has something else in mind.

Re:AI != design brain (1)

Xest (935314) | more than 5 years ago | (#25958181)

"There is a very big difference between AI - which is based on guesses about how "intelligence" works, and studies of brain function."

Yes, there most certainly is. AI is a far broader topic than study of the brain for starters, it extends to the study of swarm intelligence and emergent properties in evolution for example. The field of AI generally uses nature as inspiration and builds useful techniques from there. The human brain is but one of these items that has been studied for inspiration and has led to the idea of neural networks which in no way aim to recreate the brain, but simply mimic parts of it that we understand to perform certain tasks.

"I'm going to make a totally unjustified sweeping generalisation and suggest that one reason that AI has generally been a failure is because we have had quite wrong ideas about how the brain actually works."

It'd be a start if you even got as far as your generalisation before being rather wrong. The premise that AI has been a failure is a stumbling block in your argument before you even reach the reason why you believe it's been a failure. Suggesting AI has been a failure is akin to suggesting physics has been a failure because they haven't yet nailed down that elusive grand unified theory of everything. The fruits of AI research are used in everything from search engines to spelling/grammar checks, to voice recognition, to expert systems for medical and mechanical fault diagnosis, to optimisation of vehicle design, to convincing computer game opponents, to intelligent and fault tolerant network and telecomms routing. The results of AI research are far reaching and extend throughout nearly all areas of computing and are used by us daily without us even realising it. To me that is far from a failure, unless again you define failure as not reaching your absolute end goal as per my physics example.

Again, the brain is only a small part of AI research and neural networks have been quite good for pattern recognition so it's hard to argue that the parts of AI that are relevant to study of the brain have been a failure, let alone the subject as a whole.

"So my feeling is that the first people really to get anywhere with AI will either work for Google or be the neurobiologists who finally crack what is actually going on in there."

Google already has many good AI practitioners because AI research covers so many areas as stated above, many of which are relevant to their business- AI is extremely useful in data mining for example. That said, I'm not sure why Google's AI practitioners would be anymore likely to produce strong intelligence, which is what I assume you're after, than any other AI practitioners. I'd say these guys will have a good head start for example:

http://news.bbc.co.uk/2/hi/science/nature/7740484.stm [bbc.co.uk]

"If I wasn't close to retirement, and wanted to build a career in AI, I'd be looking at how mapreduce works, and the work being done building on that, rather than robotics. I'd also be looking as seriously parallel processing."

Robotics huh? Where did that come from? Are you suggesting all of AI is related to robotics? Your starting points are not those that I would recommend to someone who is truly interested in advancing AI research and knowledge.

The future of AI is undoubtedly going to be in higher performance computers, modern systems simply can't process as efficiently as natural systems such as the brain so we certainly need advances there. Parallel processing is somewhat of an option but I'd argue it's only somewhat of a bandaid fix. Quantum computing and biological computers are the best bet, I'm personally placing my money on biological computers because I don't think we'll end up producing strong AI on the types of computers we have sat on our desks or even super computers- I think we'll likely just end up learning how to program physical perhaps man-made brains themselves.

Re:AI != design brain (3, Informative)

TapeCutter (624760) | more than 5 years ago | (#25958263)

"I'd also be looking as seriously parallel processing."

If you haven't seen this [bluebrain.epfl.ch] it might interest you. Note that it's a simulation for use in studying the physiology of the mammalian brain, not an AI experiment. Any ghost in the machine would have to emerge by itself in pretty much the same way mind emerges from brain function.

Re:AI != design brain (0)

Anonymous Coward | more than 5 years ago | (#25958351)

That made me cream my panties! And that is neither artificial or intelligent.

Heard of AGI? (2, Informative)

QuantumG (50515) | more than 5 years ago | (#25957503)

http://www.opencog.org/wiki/OpenCogPrime:WikiBook [opencog.org]

Some interesting stuff.

Re:Heard of AGI? (1)

OeLeWaPpErKe (412765) | more than 5 years ago | (#25957729)

Only philosophical bullshit. AI is making way too many simplifications in how the brain works, but this book contains even less material. It makes sweeping conclusions based on almost no data.

It is very, very probably flat out wrong.

Re:Heard of AGI? (1)

QuantumG (50515) | more than 5 years ago | (#25957779)

"this book" .. by that do you mean "On Intelligence".. in which case I agree, but umm.. maybe you weren't trying to reply to me.

Slashdot's comment system is fucked, I recommend you switch to "classic" view as soon as possible.

It's a lot like Vista......

If AI Design was any Good (-1, Troll)

giafly (926567) | more than 5 years ago | (#25957507)

AI's would be able to design themselves by now. They can't, therefore it's not, so don't waste your time.

Re:If AI Design was any Good (0)

Anonymous Coward | more than 5 years ago | (#25957539)

Those who cannot remember the past are condemned to repeat it.

Re:If AI Design was any Good (0)

Anonymous Coward | more than 5 years ago | (#25957795)

and if fire was any good, we would all breath fire by now ...

Re:If AI Design was any Good (1)

OeLeWaPpErKe (412765) | more than 5 years ago | (#25958119)

They are. Ever heard of having genetic algorithms design neural-network controlled players ?

That's one non-interactive AI designing another interactive AI in order to improve a certain function.

And if your criterium is actual reproduction, let's keep in mind that no single humans are capable of even making a C64-level computer from scratch. Even a simple calculator would be pushing it too far for all but a few engineers.

The only way humans are capable of "improving their own design" according to darwin is to have lots of kids, then kill most of them.

You understand, we do expect AI's to do better than that. Because that, they can do today.

Re:If AI Design was any Good (1, Informative)

CnlPepper (140772) | more than 5 years ago | (#25958177)

Sorry, no. Genetic algorithms are optimisation algorithms that use a parallel, quasi-historical method to explore parameter space. They can not an artificial intelligence.

Assuming you've started it (-1, Flamebait)

Anonymous Coward | more than 5 years ago | (#25957509)

Dude, you're never going to finish your thesis if you keep getting distracted like that.

Russell & Norvig (4, Interesting)

Gazzonyx (982402) | more than 5 years ago | (#25957517)

In my AI class, last semester, we used Stuart Russell and Peter Norvig's Artificial Intelligence A Modern Approach, 2nd Ed.. It's fairly dry, but good for theory nonetheless. If you're a physics geek, it should be right up your alley; they approach everything from a mathematical angle and then have a bit of commentary on the theory, but never seem to get to the practical uses for the theory.

If you're in the US, send me an email and I'll send you my copy. They charge an arm and a leg for these books and then buy them back for 1/10 the price. I usually don't even bother selling them back.

Re:Russell & Norvig (1)

Gazzonyx (982402) | more than 5 years ago | (#25957555)

Oh... yeah, my email is moc.liamg@grebnevol.ttocs (reversed for spam protection).

Re:Russell & Norvig (1)

Deus.1.01 (946808) | more than 5 years ago | (#25957559)

Damn, you beat me to it.

Re:Russell & Norvig (1)

Deus.1.01 (946808) | more than 5 years ago | (#25957617)

Say....can i still get some mod points anyway?

Re:Russell & Norvig (0)

Anonymous Coward | more than 5 years ago | (#25957595)

I have the PDF, any bites and I'll rapidshare that shit.

Re:Russell & Norvig (-1)

Anonymous Coward | more than 5 years ago | (#25957621)

I doubt I'm the only one who would like to see that. So *nibble nibble*

Re:Russell & Norvig (0)

Anonymous Coward | more than 5 years ago | (#25958215)

Seconded.

Re:Russell & Norvig (1)

Raistlin84 (1420993) | more than 5 years ago | (#25957871)

Just a short comment (I'm at work right now): Thanks for the offer, but I'm actually from Germany. But I've access to a really huge university library with essentially unlimited borrowing, so my point of asking for books was to actually get a reading list. So far & thanks again, R.

Re:Russell & Norvig (0)

Anonymous Coward | more than 5 years ago | (#25958205)

If you're in the US, send me an email and I'll send you my copy. They charge an arm and a leg for these books and then buy them back for 1/10 the price. I usually don't even bother selling them back.

I thought you'd be glad to get a finger back at the end of it.

machine learning resources (4, Informative)

Anonymous Coward | more than 5 years ago | (#25957521)

Following Books are must have for machine learning enthusiasts:

Christopher Bishop
http://research.microsoft.com/~cmbishop/prml/

Richard Duda
http://rii.ricoh.com/~stork/DHS.html

There you will get an insight how machine learning methods (like neural networks, SVM, boosting, bayes classificator) work

for general AI (not so much in direction of statistical learning as the books above, but more into higher level learning like inference rules) I can recommend published work done by

Drew McDermott
http://cs-www.cs.yale.edu/homes/dvm/

Re:machine learning resources (2, Informative)

DocDJ (530740) | more than 5 years ago | (#25957981)

+1 for the book by Bishop (don't know about the others). In addition, have a look at Information Theory by David Mackay which I found stunningly good. There is a free on-line version available, but you should buy it: http://www.inference.phy.cam.ac.uk/itprnn/book.html [cam.ac.uk]

Re:machine learning resources (2, Informative)

Beezlebub33 (1220368) | more than 5 years ago | (#25958121)

I'll second Duda and Hart, though I guess it's Duda, Hart, and Stork now.

It's probably the most widely used pattern classification book that I've seen, and covers most of the techniques that you'll find. The coverage of neural networks is limited to Backprop though, so you'll need to look elsewhere for more in-depth on those.

Re:machine learning resources (1)

CnlPepper (140772) | more than 5 years ago | (#25958209)

I'd second the recommendation of Bishops book, it's superb if your interest is in using neural nets for pattern recognition.

Google? (0)

Anonymous Coward | more than 5 years ago | (#25957523)

How about google?

Ask an Eliza (4, Funny)

MosesJones (55544) | more than 5 years ago | (#25957537)

Question: Where can I find a Reading Guide to AI Design & Neural Networks

Answer: Why do you want to AI design & Neural Networks?

Question: Because I want to learn.

Answer: Will learn AI design & neural networks make you happy

Question: Yes

There you go. Now the question is whether Slashdot beats the Turing test on this one.

stochastic discrimination (1)

devonbowen (231626) | more than 5 years ago | (#25957639)

Adding another point to your feature space, I'll put in a plug for a technique called Stochastic Discrimination. It's not well known but is quite good at pattern recognition and avoids a lot of the weaknesses of neural networks such as over-training. Since it's not so well known, you have to go to the few academic papers to read up on it. Or visit the website http://kappa.math.buffalo.edu/ [buffalo.edu] . But it's got a very solid mathematical foundation (developed by a former math professor if mine) and isn't as "hacky" as other techniques.

Devon

You basically have to read papers.. (1)

wanax (46819) | more than 5 years ago | (#25957651)

On Neural Nets at least.. The only text book that I can think of offhand which is decent is Duda, Hart and Stork [ricoh.com]

Hawkins, like many others, has ripped off many of his ideas from Steve Grossberg [bu.edu] (in this case, the ART model). Although he's not very easy to read, especially if you start much earlier than say, Ellias and Grossberg, 1975. You should also check out the work of people like Jack Cowan [uchicago.edu] , Rajesh Rao [washington.edu] , Christof Koch [caltech.edu] , Tom Poggio [mit.edu] , David McLaughlin [nyu.edu] , Bard Ermentrout [pitt.edu] , among many, many others. I think the above names are sufficient to start a survey.

since you are still in school (1)

zome (546331) | more than 5 years ago | (#25957655)

start at your school library. Search a few AI books and read a few pages.

I bet your school has access to ACM and IEEE database. You will find good AI papers there too.

If you still want to buy something, try "Machine Learning" by Tom Mitchell. I think it fits for what you are looking for (lot of theoretical stuff, with pseudo code, and tons of references).

resources on ai and machine learning (0)

Anonymous Coward | more than 5 years ago | (#25957667)

machine learning:

http://research.microsoft.com/~cmbishop/index.htm
http://rii.ricoh.com/~stork/DHS.html

ai:
http://cs-www.cs.yale.edu/homes/dvm/

choose your subjects wisely (2, Interesting)

Gearoid_Murphy (976819) | more than 5 years ago | (#25957697)

be careful before committing to a large scale neural network project. Aside from the intuition that the brain is a massively interconnected network, no one is really sure what aspect of neural network functionality is necessary for intelligence. My advice to you is to spend time coming to terms with the abstract nature of intelligence rather than coding up elaborate projects. This link [uh.edu] is a philosophical discussion on directed behaviour which I found quite interesting (if a bit vague, which is the mark of philosophy). Also, as you become familiar with the literature, you will see many examples of algorithms which claim to model certain aspects of intelligence. These algorithms work because they have a reliable and unambiguous artificial environment from which they draw their sensory information. The problem with practical artificial intelligence is that the real world is extremely ambiguous and noisy (in the signal sense). Therefore the problem is not creating an algorithm which can emulate intelligent behaviour but solving the problem of taking the empirical information of the sensory input and producing from that data a reliable abstract representation which is easily processed by the AI algorithms (whatever they may be, neural networks, genetic programming, decision trees etc) Good luck.

Re:choose your subjects wisely (1)

Black Parrot (19622) | more than 5 years ago | (#25958003)

My advice to you is to spend time coming to terms with the abstract nature of intelligence rather than coding up elaborate projects. This link is a philosophical discussion on directed behaviour which I found quite interesting (if a bit vague, which is the mark of philosophy).

I wouldn't recommend for anyone to waste their time reading philosophers' opinions about AI research. Might as well read a used car salesman's treatise on automotive design.

At least used car salesmen actually have cars to sell...

This book was very useful for me (0)

Anonymous Coward | more than 5 years ago | (#25957723)

Jacek M. Zurada, Introduction to Artificial Neural Systems (see http://www.amazon.com/Introduction-Artificial-Neural-Systems-Zurada/dp/053495460X)

Nice book... (0)

Anonymous Coward | more than 5 years ago | (#25957735)

I started with:
- Laurene Fausset. Fundamentals of Neural Networks - Architectures. Algorithms, and Applications, Prentice Hall, 1994

It's pretty old, but still good to consulting about the algorithms and guide to implementation. :)

A complete guide to neural network would be:
- Simon Haykin. Neural Networks - A Comprehensive Foudation, Pretice Hall, 1999.

The best blue book I have :P

Thiago F Pappacena

Not as OT as it sounds at first blush (1)

$RANDOMLUSER (804576) | more than 5 years ago | (#25957741)

Christoph Adami's Introduction to Artificial Life. He's a closet physicist and it shows. Do at least read the TOC before you dismiss it.

Weka (1)

davekor (170997) | more than 5 years ago | (#25957747)

If you just want to experiment with some machine learning/pattern recognition stuff without too much programming, give Weka [waikato.ac.nz] a try. It is a suite of open source machine learning algorithms packed in a pretty usable interface.

Machine Learning (0)

Anonymous Coward | more than 5 years ago | (#25957755)

By Tom Mitchel

maybe this would interest you (0)

Anonymous Coward | more than 5 years ago | (#25957759)

http://www.ibm.com/developerworks/library/l-neural/

http://www-128.ibm.com/developerworks/library/l-neurnet/?ca=dgr-lnxw961NeuralNet

Try to code it yourself. (-1, Offtopic)

Anonymous Coward | more than 5 years ago | (#25957781)

Get a compilter of your choice and start experimenting,google snippets,write templates,etc.
Books shouldn't be read as guide,but as reference:when you need something,you look it up(like googling),which is far more effective then absorbing all book content.

Cognitive Psychology (2, Interesting)

tgv (254536) | more than 5 years ago | (#25957783)

I would strongly recommend starting with a text book on Cognitive Psychology, or reading it in parallel. AI tends to overlook the fact that intelligence is a human trait, not the most efficient algorithm for solving a logic puzzle. Anderson's book can be recommended: http://bcs.worthpublishers.com/anderson6e/default.asp?s=&n=&i=&v=&o=&ns=0&uid=0&rau=0 [worthpublishers.com] .

Re:Cognitive Psychology (1)

Darth_Ramirez (1098767) | more than 5 years ago | (#25957893)

Anderson is ok, but I would also recommend "The Scientist in the Crib" by Allison Gopnik et al. Less formal, but very clear and inspiring.

Re:Cognitive Psychology (1)

khallow (566160) | more than 5 years ago | (#25958203)

AI tends to overlook the fact that intelligence is a human trait

That's incorrect unless one wants to claim other intelligent creatures such as some cetaceans and octopi, to give a couple examples, are human. And once we develope actual artificial intelligences, are they now human as well?

Haykin (1)

gcaridakis (772392) | more than 5 years ago | (#25957835)

i would suggest Haykin's Neural Networks: A Comprehensive Foundation although you might look into a more cognitive approach...

Re:Haykin (1)

racz (799291) | more than 5 years ago | (#25957855)

Hey, you stole my comment!

Re:Haykin (1)

gcaridakis (772392) | more than 5 years ago | (#25957967)

<quote>Hey, you stole my comment!</quote>

by gcaridakis (772392)  on Tuesday December 02, @02:08PM
<br>
by racz (799291) Alter Relationship   on Tuesday December 02, @02:08PM
<br>
coincidence? :D
<br>

Haykin (1)

racz (799291) | more than 5 years ago | (#25957841)

I read and liked very much:

Neural Networks: A Comprehensive Foundation (2nd Edition) by Simon Haykin

ISBN-13: 978-0132733502

Funny you should say that... (1, Funny)

Anonymous Coward | more than 5 years ago | (#25957845)

I'm a PhD neural hypernetwork studying theoretical physics that's recently gotten quite interested in human design...

formalisms (1)

hjf (703092) | more than 5 years ago | (#25957869)

you said you don't have any formal knowledge on CS. then don't think about neural networks yet, you have to build from the ground up. you need to take algorithms (doesn't matter if you're a programmer) and language theory (languages, regex, ... turing machines) at the very least. after that you can start experimenting with AI.

Last one to make a God is a dummy! (1)

Randomly (858836) | more than 5 years ago | (#25957899)

Ready! Steady! Go!

Try an overview book, first, like (1)

Zsub (1365549) | more than 5 years ago | (#25957903)

"Cognitive Science, an introduction to the study of mind" by Friedenberg and Silverman

[sarcasm]Surely anyone could just pick this up? (1)

Viol8 (599362) | more than 5 years ago | (#25957907)

Haven't we had a number of stories recently questioning the validity of CS degrees with lots of (usually sys admins) waffling on about how degrees are a waste of time and how anyone can pick up computer skills? Ok all you "I don't need no degree , I can do it all on my own" , show us how you've all conquered the world of AI where so many others doing BScs, MSCs and PHds degrees have failed?

What? Is that the sound of silence I can hear?

Re:[sarcasm]Surely anyone could just pick this up? (0)

Anonymous Coward | more than 5 years ago | (#25958057)

I don't recall claiming to be able to design neural networks with my industry certs. Those discussions were about people trying to get into SA jobs without degrees.

Looks like you missed the point of that one, still at least you get to lord it over us non-uni educated peons, GJ!

Neural Gas (1)

Black Parrot (19622) | more than 5 years ago | (#25957911)

I think 'neural gas' is the area of neural networks research inspired by statistical physics. Don't know if there are any books about it, but you may find a chapter in an ANN textbook, and can certainly find papers vial Google.

Contrary to what others are suggesting, you probably aren't looking for the Russell & Norvig book, which is in fact good and almost qualifies as "the standard AI textbook". I counterrecommend it only because it's about Good Old Fashioned AI, which is interesting stuff, but completely different from what you are asking about.

Read up on neural gas, or pickup a textbook on neural networks. Be forewarned that few ANN reseachers are trying to build brains... like almost everyone else in AI, most ANN researchers are trying to build intelligent solutions to narrow problem sets, rather than trying to build general purpose intelligences.

You can find books on pattern recognition too, though ANN is only one of many approaches in that field.

The Conscious Mind: In Search of a Fundamental The (0)

Anonymous Coward | more than 5 years ago | (#25957923)

http://www.amazon.com/Conscious-Mind-Search-Fundamental-Philosophy/dp/0195117891/ref=sr_1_1?ie=UTF8&s=books&qid=1228220299&sr=8-1

worth reading

This is getting scary... (1)

macshome (818789) | more than 5 years ago | (#25957945)

We seem to be reading a lot of Skynet related posts these days.

I better get the drapes for the bunker finished!

finish your PhD first (plus a book recommendation) (1)

drfireman (101623) | more than 5 years ago | (#25957973)

Without knowing the details about where you stand with things, my advice would be to concentrate on finishing your PhD first. There's no limit to the number of distractions during that final push, but big new areas of study are usually a bad idea.

Assuming that's not an issue (nor or eventually), as a beginner in the field, you don't need to start with articles, there are books that will help for a while. But you may find quickly that you need to place yourself in one of two camps: people who want to develop artificial brains that work just like the real brain, and people who want to develop artificial intelligence that does some/all of the things real intelligence does but isn't constrained to do it the same way people do it. As a quick and dirty litmus test, would you consider your project successful if it had near-perfect memory for names and numbers (like computers do) or flawed memory for names and numbers (like people).

Beyond that, I will recommend the following book some friends of mine wrote:

Computational Explorations in Cognitive Neuroscience [colorado.edu]

Turing test? (1)

GloomE (695185) | more than 5 years ago | (#25957987)

This post sounds like a Turing test to me.
Could be the C2H5OH^H^H^H^H^H^H lateness of the night tho'.

Information theory, ... by Mac Kay (0)

Anonymous Coward | more than 5 years ago | (#25958023)

Information Theory, Pattern Recognition and Neural Network by David MacKay.
The book is available online: http://www.inference.phy.cam.ac.uk/itprnn/book.html

Chris Bishop (1)

axedog (991609) | more than 5 years ago | (#25958073)

I did my PhD in neural networks, and have read (and written) widely on the topic. My First recommendation is Chris Bishop's book "Neural Networks for Pattern Recognition" [microsoft.com] . It is somewhat out of date now, but it covers all the widely known methods. Simon Haykin's book, which others have recommended, is also good, but Bishop's is more concise, and better if you don't need to know every detail of every technique. It's also worth investigating the Generative Topographic Mapping, which is not covered by either book.
As a PhD student, you should approach the topic of neural networks with caution! Be prepared to spend a lot of time training networks, re-training, adjusting ad hoc parameters, re-training. Almost all of the time, a neural network can be replaced by a standard statistical method, which will perform better and have a lower computational cost.

Assembler? (1)

Yacoby (1295064) | more than 5 years ago | (#25958101)

Does this annoy anyone else as much as me? Saying I know Assembler is like saying I know Compiler when you mean that you know C++
An Assembler is a program than converts Assembly into machine code. It is not a language.

[/rant]

Practical neural networks implementations (1)

ogrisel (1168023) | more than 5 years ago | (#25958141)

First start by Norvig's book for a general overview of Machine Learning. Then the best practical guide to implement backpropagation training for feed forward neural networks is by Le Cun and Bottou: http://leon.bottou.org/papers/lecun-98x [bottou.org] (PDF or DjVu versions - 44 pages). However backprop will only reach interesting convergence for 2 to 3 layers NN with labeled data as input which is not the type of architectures presented in On Intelligence. To explore deep architectures such as the Hierarchical Temporal Memory introduced by Jeff Hawkins you should read recent papers on Deep Belief Networks by G. Hinton and Y. Bengio. They share interesting similarities with HTMs among which is the general architecture of the layered cortex as described by the mathematical models of the brain by Karl Friston. DBNs however lacks the temporal / sequencial aspect of HTMs. My personnal take is to use local predictive models such as 2-layers feed forward neural network trained using backprop to predict the future observed data and stack them into a deep structure similar to DBNs and HTMs.

Neural Networks and Kernel Methods (1)

nickruiz (1185947) | more than 5 years ago | (#25958155)

When I studied Neural Networks in my undergrad program, we read Neural Network Design [tinyurl.com] by Hagan, Bemuth, and Deale (ISBN 0971732108). At that time, we had several Physics students in the class as well, with minimal CS backgrounds. The Physics students did a great job of keeping up to speed with the concepts, since they had all of the mathematical background behind the theory.

If you want to go much further in some of the more recent theory behind pattern recognition, I could recommend Kernel Methods for Pattern Analysis [tinyurl.com] by Shawe-Taylor and Christianini (ISBN 0521813972). This book is very challenging, but greatly describes the theory.

AI research is kind of like alchemy (1)

circletimessquare (444983) | more than 5 years ago | (#25958169)

that is, its complete bullshit, but as a dream forever out of reach, it drives a lot of important and accidental discoveries, like databases or optical character recognition

so we need lots of bright minds working in AI. none of them will ever actually achieve the goal. but along the way, they will spin off fantastic new technology

so i applaud your focus, but you should be aware that anything you do of any import will be orthogonal to your goals

Duda/Hart (1)

leonbloy (812294) | more than 5 years ago | (#25958175)

The venerable Duda & Hart book on pattern clasification: its old first edition was focused on probabilistic (bayesian) aproach, but new edition [amazon.com] is very different, gives a broad view of pattern clasification and learning techniques, including neural networks.

The Emperor's New Mind (1)

CountBrass (590228) | more than 5 years ago | (#25958221)

By Prof Penrose.

Your PhD should stand you in good stead for the math required.

Another vote for this one... (1)

bbroerman (715822) | more than 5 years ago | (#25958247)

I had to read this for work. Very good book. You can find the previous version on Amazon for a reasonable price.

Read a Book on Discrete Mathematics (1)

aaaaaaargh! (1150173) | more than 5 years ago | (#25958273)

I can only recommend some literature for the classic AI approach that probably isn't your primary interest, since you've mentioned connectionism. Just in case you aren't familiar with it yet, get up to date in discrete mathematics with a focus on logic and model theory first and learn some abstract algebra and topology. That's for the formal stuff that you will encounter in classic AI. Then take a look at Russell & Norvig for an overview. With your background it will be fairly easy reading and you can skip some of the chapters. If for some reason you happen to become interested in knowledge representation (my domain), I'd recommend Friedman & Halpern's "Reasoning About Knowledge" and Halpern's "Reasoning About Uncertainty". As for connectionism and pattern recognition, I suppose you could jump into the primary literature (articles, etc.) immediately, given your theoretical physics background. But are you sure that, say, string theory isn't more interesting and rewarding than neural network programming in the long run?
Load More Comments
Slashdot Login

Need an Account?

Forgot your password?

Submission Text Formatting Tips

We support a small subset of HTML, namely these tags:

  • b
  • i
  • p
  • br
  • a
  • ol
  • ul
  • li
  • dl
  • dt
  • dd
  • em
  • strong
  • tt
  • blockquote
  • div
  • quote
  • ecode

"ecode" can be used for code snippets, for example:

<ecode>    while(1) { do_something(); } </ecode>