cortex: thesis/cortex.org comparison

comparison thesis/cortex.org @ 462:bb81cef09ad7

stuff about simulation vs reality.

author	Robert McIntyre <rlm@mit.edu>
date	Thu, 27 Mar 2014 20:18:51 -0400
parents	b345650a0baa
children	6d55ac73bc6f

comparison

equal deleted inserted replaced

-:b345650a0baa
+:bb81cef09ad7
 #+caption: curling, wiggling, and resting.
 #+name: worm-intro
 #+ATTR_LaTeX: :width 15cm
 [[./images/worm-intro-white.png]]
-#+caption: =EMPATH= recognized and classified each of these poses by
+#+caption: =EMPATH= recognized and classified each of these
-#+caption: inferring the complete sensory experience from
+#+caption: poses by inferring the complete sensory experience
-#+caption: proprioceptive data.
+#+caption: from proprioceptive data.
 #+name: worm-recognition-intro
 #+ATTR_LaTeX: :width 15cm
 [[./images/worm-poses.png]]
 One powerful advantage of empathic problem solving is that it
 [experiences]
 (and (curled? experiences)
 (let [worm-touch (:touch (peek experiences))
 tail-touch (worm-touch 0)
 head-touch (worm-touch 4)]
-(and (< 0.55 (contact worm-segment-bottom-tip tail-touch))
+(and (< 0.2 (contact worm-segment-bottom-tip tail-touch))
-(< 0.55 (contact worm-segment-top-tip    head-touch))))))
+(< 0.2 (contact worm-segment-top-tip    head-touch))))))
 #+end_src
 #+end_listing
 **  =CORTEX= is a toolkit for building sensate creatures
 I built =CORTEX= to be a general AI research platform for doing
 experiments involving multiple rich senses and a wide variety and
 number of creatures. I intend it to be useful as a library for many
-more projects than just this one. =CORTEX= was necessary to meet a
+more projects than just this thesis. =CORTEX= was necessary to meet
-need among AI researchers at CSAIL and beyond, which is that people
+a need among AI researchers at CSAIL and beyond, which is that
-often will invent neat ideas that are best expressed in the
+people often will invent neat ideas that are best expressed in the
 language of creatures and senses, but in order to explore those
 ideas they must first build a platform in which they can create
 simulated creatures with rich senses! There are many ideas that
 would be simple to execute (such as =EMPATH=), but attached to them
 is the multi-month effort to make a good creature simulator. Often,
 - I built =EMPATH=, which uses =CORTEX= to identify the actions of
 a worm-like creature using a computational model of empathy.
 * Building =CORTEX=
-** To explore embodiment, we need a world, body, and senses
+I intend for =CORTEX= to be used as a general purpose library for
+building creatures and outfitting them with senses, so that it will
+be useful for other researchers who want to test out ideas of their
+own. To this end, wherver I have had to make archetictural choices
+about =CORTEX=, I have chosen to give as much freedom to the user as
+possible, so that =CORTEX= may be used for things I have not
+forseen.
+** Simulation or Reality?
+The most important archetictural decision of all is the choice to
+use a computer-simulated environemnt in the first place! The world
+is a vast and rich place, and for now simulations are a very poor
+reflection of its complexity. It may be that there is a significant
+qualatative difference between dealing with senses in the real
+world and dealing with pale facilimilies of them in a
+simulation. What are the advantages and disadvantages of a
+simulation vs. reality?
+*** Simulation
+The advantages of virtual reality are that when everything is a
+simulation, experiments in that simulation are absolutely
+reproducible. It's also easier to change the character and world
+to explore new situations and different sensory combinations.
+If the world is to be simulated on a computer, then not only do
+you have to worry about whether the character's senses are rich
+enough to learn from the world, but whether the world itself is
+rendered with enough detail and realism to give enough working
+material to the character's senses. To name just a few
+difficulties facing modern physics simulators: destructibility of
+the environment, simulation of water/other fluids, large areas,
+nonrigid bodies, lots of objects, smoke. I don't know of any
+computer simulation that would allow a character to take a rock
+and grind it into fine dust, then use that dust to make a clay
+sculpture, at least not without spending years calculating the
+interactions of every single small grain of dust. Maybe a
+simulated world with today's limitations doesn't provide enough
+richness for real intelligence to evolve.
+*** Reality
+The other approach for playing with senses is to hook your
+software up to real cameras, microphones, robots, etc., and let it
+loose in the real world. This has the advantage of eliminating
+concerns about simulating the world at the expense of increasing
+the complexity of implementing the senses. Instead of just
+grabbing the current rendered frame for processing, you have to
+use an actual camera with real lenses and interact with photons to
+get an image. It is much harder to change the character, which is
+now partly a physical robot of some sort, since doing so involves
+changing things around in the real world instead of modifying
+lines of code. While the real world is very rich and definitely
+provides enough stimulation for intelligence to develop as
+evidenced by our own existence, it is also uncontrollable in the
+sense that a particular situation cannot be recreated perfectly or
+saved for later use. It is harder to conduct science because it is
+harder to repeat an experiment. The worst thing about using the
+real world instead of a simulation is the matter of time. Instead
+of simulated time you get the constant and unstoppable flow of
+real time. This severely limits the sorts of software you can use
+to program the AI because all sense inputs must be handled in real
+time. Complicated ideas may have to be implemented in hardware or
+may simply be impossible given the current speed of our
+processors. Contrast this with a simulation, in which the flow of
+time in the simulated world can be slowed down to accommodate the
+limitations of the character's programming. In terms of cost,
+doing everything in software is far cheaper than building custom
+real-time hardware. All you need is a laptop and some patience.
 ** Because of Time, simulation is perferable to reality
+I envision =CORTEX= being used to support rapid prototyping and
+iteration of ideas. Even if I could put together a well constructed
+kit for creating robots, it would still not be enough because of
+the scourge of real-time processing. Anyone who wants to test their
+ideas in the real world must always worry about getting their
+algorithms to run fast enough to process information in real
+time. The need for real time processing only increases if multiple
+senses are involved. In the extreme case, even simple algorithms
+will have to be accelerated by ASIC chips or FPGAs, turning what
+would otherwise be a few lines of code and a 10x speed penality
+into a multi-month ordeal. For this reason, =CORTEX= supports
+/time-dialiation/, which scales back the framerate of the
+simulation in proportion to the amount of processing each
+frame. From the perspective of the creatures inside the simulation,
+time always appears to flow at a constant rate, regardless of how
+complicated the envorimnent becomes or how many creatures are in
+the simulation. The cost is that =CORTEX= can sometimes run slower
+than real time. This can also be an advantage, however ---
+simulations of very simple creatures in =CORTEX= generally run at
+40x on my machine!
 ** Video game engines are a great starting point
+I did not need to write my own physics simulation code or shader to
+build =CORTEX=. Doing so would lead to a system that is impossible
+for anyone but myself to use anyway. Instead, I use a video game
+engine as a base and modify it to accomodate the additional needs
+of =CORTEX=. Video game engines are an ideal starting point to
+build =CORTEX=, because they are not far from being creature
+building systems themselves.
+First off, general purpose video game engines come with a physics
+engine and lighting / sound system. The physics system provides
+tools that can be co-opted to serve as touch, proprioception, and
+muscles. Since some games support split screen views, a good video
+game engine will allow you to efficiently create multiple cameras
+in the simulated world that can be used as eyes.
 ** Bodies are composed of segments connected by joints
 ** Eyes reuse standard video game components

Mercurial > cortex

comparison thesis/cortex.org @ 462:bb81cef09ad7