1.1 --- a/thesis/cortex.org	Thu Mar 27 18:17:23 2014 -0400
     1.2 +++ b/thesis/cortex.org	Thu Mar 27 20:18:51 2014 -0400
     1.3 @@ -174,9 +174,9 @@
     1.4     #+ATTR_LaTeX: :width 15cm
     1.5     [[./images/worm-intro-white.png]]
     1.6  
     1.7 -   #+caption: =EMPATH= recognized and classified each of these poses by
     1.8 -   #+caption: inferring the complete sensory experience from 
     1.9 -   #+caption: proprioceptive data.
    1.10 +   #+caption: =EMPATH= recognized and classified each of these 
    1.11 +   #+caption: poses by inferring the complete sensory experience 
    1.12 +   #+caption: from proprioceptive data.
    1.13     #+name: worm-recognition-intro
    1.14     #+ATTR_LaTeX: :width 15cm
    1.15     [[./images/worm-poses.png]]
    1.16 @@ -221,8 +221,8 @@
    1.17         (let [worm-touch (:touch (peek experiences))
    1.18               tail-touch (worm-touch 0)
    1.19               head-touch (worm-touch 4)]
    1.20 -         (and (< 0.55 (contact worm-segment-bottom-tip tail-touch))
    1.21 -              (< 0.55 (contact worm-segment-top-tip    head-touch))))))
    1.22 +         (and (< 0.2 (contact worm-segment-bottom-tip tail-touch))
    1.23 +              (< 0.2 (contact worm-segment-top-tip    head-touch))))))
    1.24     #+end_src
    1.25     #+end_listing
    1.26  
    1.27 @@ -232,9 +232,9 @@
    1.28     I built =CORTEX= to be a general AI research platform for doing
    1.29     experiments involving multiple rich senses and a wide variety and
    1.30     number of creatures. I intend it to be useful as a library for many
    1.31 -   more projects than just this one. =CORTEX= was necessary to meet a
    1.32 -   need among AI researchers at CSAIL and beyond, which is that people
    1.33 -   often will invent neat ideas that are best expressed in the
    1.34 +   more projects than just this thesis. =CORTEX= was necessary to meet
    1.35 +   a need among AI researchers at CSAIL and beyond, which is that
    1.36 +   people often will invent neat ideas that are best expressed in the
    1.37     language of creatures and senses, but in order to explore those
    1.38     ideas they must first build a platform in which they can create
    1.39     simulated creatures with rich senses! There are many ideas that
    1.40 @@ -331,11 +331,116 @@
    1.41     
    1.42  * Building =CORTEX=
    1.43  
    1.44 -** To explore embodiment, we need a world, body, and senses
    1.45 +  I intend for =CORTEX= to be used as a general purpose library for
    1.46 +  building creatures and outfitting them with senses, so that it will
    1.47 +  be useful for other researchers who want to test out ideas of their
    1.48 +  own. To this end, wherver I have had to make archetictural choices
    1.49 +  about =CORTEX=, I have chosen to give as much freedom to the user as
    1.50 +  possible, so that =CORTEX= may be used for things I have not
    1.51 +  forseen.
    1.52 +
    1.53 +** Simulation or Reality?
    1.54 +   
    1.55 +   The most important archetictural decision of all is the choice to
    1.56 +   use a computer-simulated environemnt in the first place! The world
    1.57 +   is a vast and rich place, and for now simulations are a very poor
    1.58 +   reflection of its complexity. It may be that there is a significant
    1.59 +   qualatative difference between dealing with senses in the real
    1.60 +   world and dealing with pale facilimilies of them in a
    1.61 +   simulation. What are the advantages and disadvantages of a
    1.62 +   simulation vs. reality?
    1.63 +
    1.64 +*** Simulation
    1.65 +
    1.66 +    The advantages of virtual reality are that when everything is a
    1.67 +    simulation, experiments in that simulation are absolutely
    1.68 +    reproducible. It's also easier to change the character and world
    1.69 +    to explore new situations and different sensory combinations.
    1.70 +
    1.71 +    If the world is to be simulated on a computer, then not only do
    1.72 +    you have to worry about whether the character's senses are rich
    1.73 +    enough to learn from the world, but whether the world itself is
    1.74 +    rendered with enough detail and realism to give enough working
    1.75 +    material to the character's senses. To name just a few
    1.76 +    difficulties facing modern physics simulators: destructibility of
    1.77 +    the environment, simulation of water/other fluids, large areas,
    1.78 +    nonrigid bodies, lots of objects, smoke. I don't know of any
    1.79 +    computer simulation that would allow a character to take a rock
    1.80 +    and grind it into fine dust, then use that dust to make a clay
    1.81 +    sculpture, at least not without spending years calculating the
    1.82 +    interactions of every single small grain of dust. Maybe a
    1.83 +    simulated world with today's limitations doesn't provide enough
    1.84 +    richness for real intelligence to evolve.
    1.85 +
    1.86 +*** Reality
    1.87 +
    1.88 +    The other approach for playing with senses is to hook your
    1.89 +    software up to real cameras, microphones, robots, etc., and let it
    1.90 +    loose in the real world. This has the advantage of eliminating
    1.91 +    concerns about simulating the world at the expense of increasing
    1.92 +    the complexity of implementing the senses. Instead of just
    1.93 +    grabbing the current rendered frame for processing, you have to
    1.94 +    use an actual camera with real lenses and interact with photons to
    1.95 +    get an image. It is much harder to change the character, which is
    1.96 +    now partly a physical robot of some sort, since doing so involves
    1.97 +    changing things around in the real world instead of modifying
    1.98 +    lines of code. While the real world is very rich and definitely
    1.99 +    provides enough stimulation for intelligence to develop as
   1.100 +    evidenced by our own existence, it is also uncontrollable in the
   1.101 +    sense that a particular situation cannot be recreated perfectly or
   1.102 +    saved for later use. It is harder to conduct science because it is
   1.103 +    harder to repeat an experiment. The worst thing about using the
   1.104 +    real world instead of a simulation is the matter of time. Instead
   1.105 +    of simulated time you get the constant and unstoppable flow of
   1.106 +    real time. This severely limits the sorts of software you can use
   1.107 +    to program the AI because all sense inputs must be handled in real
   1.108 +    time. Complicated ideas may have to be implemented in hardware or
   1.109 +    may simply be impossible given the current speed of our
   1.110 +    processors. Contrast this with a simulation, in which the flow of
   1.111 +    time in the simulated world can be slowed down to accommodate the
   1.112 +    limitations of the character's programming. In terms of cost,
   1.113 +    doing everything in software is far cheaper than building custom
   1.114 +    real-time hardware. All you need is a laptop and some patience.
   1.115  
   1.116  ** Because of Time, simulation is perferable to reality
   1.117  
   1.118 +   I envision =CORTEX= being used to support rapid prototyping and
   1.119 +   iteration of ideas. Even if I could put together a well constructed
   1.120 +   kit for creating robots, it would still not be enough because of
   1.121 +   the scourge of real-time processing. Anyone who wants to test their
   1.122 +   ideas in the real world must always worry about getting their
   1.123 +   algorithms to run fast enough to process information in real
   1.124 +   time. The need for real time processing only increases if multiple
   1.125 +   senses are involved. In the extreme case, even simple algorithms
   1.126 +   will have to be accelerated by ASIC chips or FPGAs, turning what
   1.127 +   would otherwise be a few lines of code and a 10x speed penality
   1.128 +   into a multi-month ordeal. For this reason, =CORTEX= supports
   1.129 +   /time-dialiation/, which scales back the framerate of the
   1.130 +   simulation in proportion to the amount of processing each
   1.131 +   frame. From the perspective of the creatures inside the simulation,
   1.132 +   time always appears to flow at a constant rate, regardless of how
   1.133 +   complicated the envorimnent becomes or how many creatures are in
   1.134 +   the simulation. The cost is that =CORTEX= can sometimes run slower
   1.135 +   than real time. This can also be an advantage, however ---
   1.136 +   simulations of very simple creatures in =CORTEX= generally run at
   1.137 +   40x on my machine!
   1.138 +
   1.139  ** Video game engines are a great starting point
   1.140 +   
   1.141 +   I did not need to write my own physics simulation code or shader to
   1.142 +   build =CORTEX=. Doing so would lead to a system that is impossible
   1.143 +   for anyone but myself to use anyway. Instead, I use a video game
   1.144 +   engine as a base and modify it to accomodate the additional needs
   1.145 +   of =CORTEX=. Video game engines are an ideal starting point to
   1.146 +   build =CORTEX=, because they are not far from being creature
   1.147 +   building systems themselves. 
   1.148 +   
   1.149 +   First off, general purpose video game engines come with a physics
   1.150 +   engine and lighting / sound system. The physics system provides
   1.151 +   tools that can be co-opted to serve as touch, proprioception, and
   1.152 +   muscles. Since some games support split screen views, a good video
   1.153 +   game engine will allow you to efficiently create multiple cameras
   1.154 +   in the simulated world that can be used as eyes. 
   1.155  
   1.156  ** Bodies are composed of segments connected by joints
   1.157