Mercurial > cortex

comparison org/intro.org @ 22:157b416152ea

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
continuing splitting
author Robert McIntyre <rlm@mit.edu>
date Sun, 23 Oct 2011 23:35:04 -0700
parents
children e965675ec4d0
comparison
equal deleted inserted replaced
21:01e1427126af 22:157b416152ea
1 #+title: Simulated Senses
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+description: Simulating senses for AI research using JMonkeyEngine3
5 #+SETUPFILE: ../../aurellem/org/setup.org
6 #+INCLUDE: ../../aurellem/org/level-0.org
7 #+babel: :mkdirp yes :noweb yes
8
9 * Background
10 Artificial Intelligence has tried and failed for more than half a
11 century to produce programs as flexible, creative, and "intelligent"
12 as the human mind itself. Clearly, we are still missing some important
13 ideas concerning intelligent programs or we would have strong AI
14 already. What idea could be missing?
15
16 When Turing first proposed his famous "Turing Test" in the
17 groundbreaking paper [[./sources/turing.pdf][/Computing Machines and Intelligence/]], he gave
18 little importance to how a computer program might interact with the
19 world:
20
21 #+BEGIN_QUOTE
22 \ldquo{}We need not be too concerned about the legs, eyes, etc. The example of
23 Miss Helen Keller shows that education can take place provided that
24 communication in both directions between teacher and pupil can take
25 place by some means or other.\rdquo{}
26 #+END_QUOTE
27
28 And from the example of Hellen Keller he went on to assume that the
29 only thing a fledgling AI program could need by way of communication
30 is a teletypewriter. But Hellen Keller did possess vision and hearing
31 for the first few months of her life, and her tactile sense was far
32 more rich than any text-stream could hope to achieve. She possessed a
33 body she could move freely, and had continual access to the real world
34 to learn from her actions.
35
36 I believe that our programs are suffering from too little sensory
37 input to become really intelligent. Imagine for a moment that you
38 lived in a world completely cut off form all sensory stimulation. You
39 have no eyes to see, no ears to hear, no mouth to speak. No body, no
40 taste, no feeling whatsoever. The only sense you get at all is a
41 single point of light, flickering on and off in the void. If this was
42 your life from birth, you would never learn anything, and could never
43 become intelligent. Actual humans placed in sensory deprivation
44 chambers experience hallucinations and can begin to loose their sense
45 of reality in as little as 15 minutes[sensory-deprivation]. Most of
46 the time, the programs we write are in exactly this situation. They do
47 not interface with cameras and microphones, and they do not control a
48 real or simulated body or interact with any sort of world.
49
50
51 * Simulation vs. Reality
52 I want demonstrate that multiple senses are what enable
53 intelligence. There are two ways of playing around with senses and
54 computer programs:
55
56 The first is to go entirely with simulation: virtual world, virtual
57 character, virtual senses. The advantages are that when everything is
58 a simulation, experiments in that simulation are absolutely
59 reproducible. It's also easier to change the character and world to
60 explore new situations and different sensory combinations.
61
62
63 ** Issues with Simulation
64
65 If the world is to be simulated on a computer, then not only do you
66 have to worry about whether the character's senses are rich enough to
67 learn from the world, but whether the world itself is rendered with
68 enough detail and realism to give enough working material to the
69 character's senses. To name just a few difficulties facing modern
70 physics simulators: destructibility of the environment, simulation of
71 water/other fluids, large areas, nonrigid bodies, lots of objects,
72 smoke. I don't know of any computer simulation that would allow a
73 character to take a rock and grind it into fine dust, then use that
74 dust to make a clay sculpture, at least not without spending years
75 calculating the interactions of every single small grain of
76 dust. Maybe a simulated world with today's limitations doesn't provide
77 enough richness for real intelligence to evolve.
78
79 ** Issues with Reality
80
81 The other approach for playing with senses is to hook your software up
82 to real cameras, microphones, robots, etc., and let it loose in the
83 real world. This has the advantage of eliminating concerns about
84 simulating the world at the expense of increasing the complexity of
85 implementing the senses. Instead of just grabbing the current rendered
86 frame for processing, you have to use an actual camera with real
87 lenses and interact with photons to get an image. It is much harder to
88 change the character, which is now partly a physical robot of some
89 sort, since doing so involves changing things around in the real world
90 instead of modifying lines of code. While the real world is very rich
91 and definitely provides enough stimulation for intelligence to develop
92 as evidenced by our own existence, it is also uncontrollable in the
93 sense that a particular situation cannot be recreated perfectly or
94 saved for later use. It is harder to conduct science because it is
95 harder to repeat an experiment. The worst thing about using the real
96 world instead of a simulation is the matter of time. Instead of
97 simulated time you get the constant and unstoppable flow of real
98 time. This severely limits the sorts of software you can use to
99 program the AI because all sense inputs must be handled in real
100 time. Complicated ideas may have to be implemented in hardware or may
101 simply be impossible given the current speed of our
102 processors. Contrast this with a simulation, in which the flow of time
103 in the simulated world can be slowed down to accommodate the
104 limitations of the character's programming. In terms of cost, doing
105 everything in software is far cheaper than building custom real-time
106 hardware. All you need is a laptop and some patience.
107
108 * Choose a Simulation Engine
109
110 Mainly because of issues with controlling the flow of time, I chose to
111 simulate both the world and the character. I set out to make a minimal
112 world in which I could embed a character with multiple senses. My main
113 goal is to make an environment where I can perform further experiments
114 in simulated senses.
115
116 As Carl Sagan once said, "If you wish to make an apple pie from
117 scratch, you must first invent the universe." I examined many
118 different 3D environments to try and find something I would use as the
119 base for my simulation; eventually the choice came down to three
120 engines: the Quake II engine, the Source Engine, and jMonkeyEngine.
121
122 ** Quake II/Jake2
123
124 I spent a bit more than a month working with the Quake II Engine from
125 ID software to see if I could use it for my purposes. All the source
126 code was released by ID software into the Public Domain several years
127 ago, and as a result it has been ported and modified for many
128 different reasons. This engine was famous for its advanced use of
129 realistic shading and had decent and fast physics
130 simulation. Researchers at Princeton [[http://www.nature.com/nature/journal/v461/n7266/pdf/nature08499.pdf][used this code]] to study spatial
131 information encoding in the hippocampal cells of rats. Those
132 researchers created a special Quake II level that simulated a maze,
133 and added an interface where a mouse could run around inside a ball in
134 various directions to move the character in the simulated maze. They
135 measured hippocampal activity during this exercise to try and tease
136 out the method in which spatial data was stored in that area of the
137 brain. I find this promising because if a real living rat can interact
138 with a computer simulation of a maze in the same way as it interacts
139 with a real-world maze, then maybe that simulation is close enough to
140 reality that a simulated sense of vision and motor control interacting
141 with that simulation could reveal useful information about the real
142 thing. It happens that there is a Java port of the original C source
143 code called Jake2. The port demonstrates Java's OpenGL bindings and
144 runs anywhere from 90% to 105% as fast as the C version. After
145 reviewing much of the source of Jake2, I eventually rejected it
146 because the engine is too tied to the concept of a first-person
147 shooter game. One of the problems I had was that there does not seem
148 to be any easy way to attach multiple cameras to a single
149 character. There are also several physics clipping issues that are
150 corrected in a way that only applies to the main character and does
151 not apply to arbitrary objects. While there is a large community of
152 level modders, I couldn't find a community to support using the engine
153 to make new things.
154
155 ** Source Engine
156
157 The Source Engine evolved from the Quake II and Quake I engines and is
158 used by Valve in the Half-Life series of games. The physics simulation
159 in the Source Engine is quite accurate and probably the best out of
160 all the engines I investigated. There is also an extensive community
161 actively working with the engine. However, applications that use the
162 Source Engine must be written in C++, the code is not open, it only
163 runs on Windows, and the tools that come with the SDK to handle models
164 and textures are complicated and awkward to use.
165
166 ** jMonkeyEngine
167
168 jMonkeyEngine is a new library for creating games in Java. It uses
169 OpenGL to render to the screen and uses screengraphs to avoid drawing
170 things that do not appear on the screen. It has an active community
171 and several games in the pipeline. The engine was not built to serve
172 any particular game but is instead meant to be used for any 3D
173 game. After experimenting with each of these three engines and a few
174 others for about 2 months I settled on jMonkeyEngine. I chose it
175 because it had the most features out of all the open projects I looked
176 at, and because I could then write my code in Clojure, an
177 implementation of LISP that runs on the JVM...