# HG changeset patch # User Robert McIntyre # Date 1395965931 14400 # Node ID bb81cef09ad7579f86c0032eefe4ab9b0a36f1f4 # Parent b345650a0baa426a0e8cfe2bdd6136be9eafacbb stuff about simulation vs reality. diff -r b345650a0baa -r bb81cef09ad7 org/intro.org --- a/org/intro.org Thu Mar 27 18:17:23 2014 -0400 +++ b/org/intro.org Thu Mar 27 20:18:51 2014 -0400 @@ -195,36 +195,3 @@ out of all the open projects I looked at, and because I could then write my code in Clojure, an implementation of LISP that runs on the JVM. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff -r b345650a0baa -r bb81cef09ad7 thesis/cortex.org --- a/thesis/cortex.org Thu Mar 27 18:17:23 2014 -0400 +++ b/thesis/cortex.org Thu Mar 27 20:18:51 2014 -0400 @@ -174,9 +174,9 @@ #+ATTR_LaTeX: :width 15cm [[./images/worm-intro-white.png]] - #+caption: =EMPATH= recognized and classified each of these poses by - #+caption: inferring the complete sensory experience from - #+caption: proprioceptive data. + #+caption: =EMPATH= recognized and classified each of these + #+caption: poses by inferring the complete sensory experience + #+caption: from proprioceptive data. #+name: worm-recognition-intro #+ATTR_LaTeX: :width 15cm [[./images/worm-poses.png]] @@ -221,8 +221,8 @@ (let [worm-touch (:touch (peek experiences)) tail-touch (worm-touch 0) head-touch (worm-touch 4)] - (and (< 0.55 (contact worm-segment-bottom-tip tail-touch)) - (< 0.55 (contact worm-segment-top-tip head-touch)))))) + (and (< 0.2 (contact worm-segment-bottom-tip tail-touch)) + (< 0.2 (contact worm-segment-top-tip head-touch)))))) #+end_src #+end_listing @@ -232,9 +232,9 @@ I built =CORTEX= to be a general AI research platform for doing experiments involving multiple rich senses and a wide variety and number of creatures. I intend it to be useful as a library for many - more projects than just this one. =CORTEX= was necessary to meet a - need among AI researchers at CSAIL and beyond, which is that people - often will invent neat ideas that are best expressed in the + more projects than just this thesis. =CORTEX= was necessary to meet + a need among AI researchers at CSAIL and beyond, which is that + people often will invent neat ideas that are best expressed in the language of creatures and senses, but in order to explore those ideas they must first build a platform in which they can create simulated creatures with rich senses! There are many ideas that @@ -331,11 +331,116 @@ * Building =CORTEX= -** To explore embodiment, we need a world, body, and senses + I intend for =CORTEX= to be used as a general purpose library for + building creatures and outfitting them with senses, so that it will + be useful for other researchers who want to test out ideas of their + own. To this end, wherver I have had to make archetictural choices + about =CORTEX=, I have chosen to give as much freedom to the user as + possible, so that =CORTEX= may be used for things I have not + forseen. + +** Simulation or Reality? + + The most important archetictural decision of all is the choice to + use a computer-simulated environemnt in the first place! The world + is a vast and rich place, and for now simulations are a very poor + reflection of its complexity. It may be that there is a significant + qualatative difference between dealing with senses in the real + world and dealing with pale facilimilies of them in a + simulation. What are the advantages and disadvantages of a + simulation vs. reality? + +*** Simulation + + The advantages of virtual reality are that when everything is a + simulation, experiments in that simulation are absolutely + reproducible. It's also easier to change the character and world + to explore new situations and different sensory combinations. + + If the world is to be simulated on a computer, then not only do + you have to worry about whether the character's senses are rich + enough to learn from the world, but whether the world itself is + rendered with enough detail and realism to give enough working + material to the character's senses. To name just a few + difficulties facing modern physics simulators: destructibility of + the environment, simulation of water/other fluids, large areas, + nonrigid bodies, lots of objects, smoke. I don't know of any + computer simulation that would allow a character to take a rock + and grind it into fine dust, then use that dust to make a clay + sculpture, at least not without spending years calculating the + interactions of every single small grain of dust. Maybe a + simulated world with today's limitations doesn't provide enough + richness for real intelligence to evolve. + +*** Reality + + The other approach for playing with senses is to hook your + software up to real cameras, microphones, robots, etc., and let it + loose in the real world. This has the advantage of eliminating + concerns about simulating the world at the expense of increasing + the complexity of implementing the senses. Instead of just + grabbing the current rendered frame for processing, you have to + use an actual camera with real lenses and interact with photons to + get an image. It is much harder to change the character, which is + now partly a physical robot of some sort, since doing so involves + changing things around in the real world instead of modifying + lines of code. While the real world is very rich and definitely + provides enough stimulation for intelligence to develop as + evidenced by our own existence, it is also uncontrollable in the + sense that a particular situation cannot be recreated perfectly or + saved for later use. It is harder to conduct science because it is + harder to repeat an experiment. The worst thing about using the + real world instead of a simulation is the matter of time. Instead + of simulated time you get the constant and unstoppable flow of + real time. This severely limits the sorts of software you can use + to program the AI because all sense inputs must be handled in real + time. Complicated ideas may have to be implemented in hardware or + may simply be impossible given the current speed of our + processors. Contrast this with a simulation, in which the flow of + time in the simulated world can be slowed down to accommodate the + limitations of the character's programming. In terms of cost, + doing everything in software is far cheaper than building custom + real-time hardware. All you need is a laptop and some patience. ** Because of Time, simulation is perferable to reality + I envision =CORTEX= being used to support rapid prototyping and + iteration of ideas. Even if I could put together a well constructed + kit for creating robots, it would still not be enough because of + the scourge of real-time processing. Anyone who wants to test their + ideas in the real world must always worry about getting their + algorithms to run fast enough to process information in real + time. The need for real time processing only increases if multiple + senses are involved. In the extreme case, even simple algorithms + will have to be accelerated by ASIC chips or FPGAs, turning what + would otherwise be a few lines of code and a 10x speed penality + into a multi-month ordeal. For this reason, =CORTEX= supports + /time-dialiation/, which scales back the framerate of the + simulation in proportion to the amount of processing each + frame. From the perspective of the creatures inside the simulation, + time always appears to flow at a constant rate, regardless of how + complicated the envorimnent becomes or how many creatures are in + the simulation. The cost is that =CORTEX= can sometimes run slower + than real time. This can also be an advantage, however --- + simulations of very simple creatures in =CORTEX= generally run at + 40x on my machine! + ** Video game engines are a great starting point + + I did not need to write my own physics simulation code or shader to + build =CORTEX=. Doing so would lead to a system that is impossible + for anyone but myself to use anyway. Instead, I use a video game + engine as a base and modify it to accomodate the additional needs + of =CORTEX=. Video game engines are an ideal starting point to + build =CORTEX=, because they are not far from being creature + building systems themselves. + + First off, general purpose video game engines come with a physics + engine and lighting / sound system. The physics system provides + tools that can be co-opted to serve as touch, proprioception, and + muscles. Since some games support split screen views, a good video + game engine will allow you to efficiently create multiple cameras + in the simulated world that can be used as eyes. ** Bodies are composed of segments connected by joints