Well, the last post generated a few comments – which is always nice to see even if they mostly had little to do with the actual post – so thanks to my readers for that. One of the comments brought up the topic of AGI input. For those who don’t care to re-read the more sordid parts, here’s a summary as i recall it (since i don’t care to re-read those parts either)… I claimed that the von Neumann architecture (VNA) is unsuitable for AGI development. Reader said that the computers can accept any input. I magnanimously resisted an all out flame and responded flippantly instead.
(To clarify, well of course any non-spiritual data can somehow be reduced to zeroes and ones and then somehow dumped into a computer to be somehow processed. But that’s a lot of somehows. Reader suggested it’s a matter of the AGI book club getting together for beers and discussing, and was surprisingly resistant to assurances – and not just from me – that it would take a fair bit more effort. Regardless, my main problem with the VNA is that it lacks an inherent notion of time. It focuses on procedural logic, which is great for financial transactions, downloading movies, and posting blog entries, but less so for physical simulation. Brains work differently (i believe i’ve discussed this at length already, but if any readers have specific questions, ask away), and so developing an AGI for a VNA means the developer has to account for all matters to do with time. Possible, of course, but decidedly non-trivial. Further, if early and popular computer architectures did have a notion of time built in, generations of software developers would have already explored its various aspects and built tools and such that would compare to the very powerful tools we currently have at our fingertips for procedural logic. I can go on about this but i’m wandering from the point of this post, so i’ll leave it for another time.)
Ok, so anyway, i got on a train of thought about AGI input. This is a vast area since pretty much any data imaginable has been used to train an intelligent system. Chess boards, subway schedules, logic statements (my favourite: Dracula is a type of Vampire), music, … In my own research i tried quite a number of input types myself, but consistently found that shortly after deciding on the information format examples arose that didn’t fit. Since the biological brain has always been my muse (it being the only proof that intelligence is possible), i eventually conceded that a suitable format would be what the brain uses. I characterize this format as a massively multi-channel stream of events, as in the spiking of neurons. An event is then simply the time stamp of when it occurred – it contains no further information. This information format is definitely robust in that a great deal of information can be reduce to it. But it is also a bear to work with, not least of which due to the processing power required, but also its non-intuitive nature, at least to my brain. It’s just baffling to keep track of what is going on.
The format can arguably be simplified – both for performance and intuition – by quantifying the events into time buckets. Instead of having a few thousand events scattered over the course of 10 seconds, you could have 100 sums of events, each representing the number of events that occurred in a 0.1s bucket. This results in a stream of integers, which is so very much easier to work with. (For one, you can now chart the stream on a graph, which is much easier to mentally digest than a line of event densities.) It’s not clear how a chess board can be reduced to this format, but i’d argue it doesn’t really matter. The majority of things people would like an AGI to do these days involves working with data that does reduce to this format relatively easily.
But the problem is finding the data. I have a customer that sent me some real data recently, within which is a real problem that needs to be solved. The problem is in fact fairly simple, and a straightforward narrow AI approach would probably do the trick nicely. But who wants to do that? So naturally, i cast a wide net and wrote some code that should be usable in a great number of situations, which could be quite divergent from my customer’s issue. But i can’t test it to know for sure because i don’t have any more data.
The point of this post is two-fold. First, it’s to try and describe the hopelessness of building an AGI without a definite problem to solve. Brains evolved in an environment of limited resources where survival meant solving the problem of getting what it needed before some other brain. A system without a goal has no motivation. The input data (which should consist of training and operation portions) must present a problem to be solved.
Second, may i respectfully ask readers to respond if they know of any repositories of data that would challenge an AGI. I’ve heard of such things, but have never found anything near decent. (GoID is supposed to be such an input data source, of course, so no need to mention that.) And, if there is a dearth, perhaps we all can create such a thing?
This is also something I’ve encountered. I’ve thought about pixel and audio data from video files, and I’ve done work attempting to break down audio data into a form more akin to what you mention above, but nothing really successful.
Geoffrey Hinton’s lab has done lots of work with the MNIST database of handwritten digits (and I’ve had some success following them on it), but it’s pretty limited and shallow. I know he’s also done work with motion-capture data and natural image patches, but I don’t know where those might be on the web.
And of course, MLComp (http://mlcomp.org/) has a big collection of datasets for narrow learning.
I had a look at MLComp. It’s a great setup, but as you said, narrow learning datasets. (I wrote a little code to solve the binary/multi-classification stuff. It took about 3 hours or so, but i didn’t bother uploading out of laziness. The solving code was much more interesting than the wrapper code.)
Anyway, i’m starting to think now that GoiD was in fact the correct approach. Not only does it produce temporal data, but it also provides a dynamic environment where the actions of the agent can change the environment. This is something that at least MLComp doesn’t currently seem to be able to do, although i might follow up with them to see if they are interested in collaborating to provide such.
BTW, Jeff Hawkins mentioned kaggle.com in a note. A different perspective than MLComp, but it seems that the nature of the data is roughly the same, i.e. appropriate for narrow AI.