Mercurial > cortex

comparison org/intro.org @ 333:f4ef73370da1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
add file.
author Robert McIntyre <rlm@mit.edu>
date Fri, 20 Jul 2012 11:21:04 -0500
parents b84701e4f6ae
children bb81cef09ad7
comparison
equal deleted inserted replaced
332:698d48b91cd5 333:f4ef73370da1
6 #+SETUPFILE: ../../aurellem/org/setup.org 6 #+SETUPFILE: ../../aurellem/org/setup.org
7 #+INCLUDE: ../../aurellem/org/level-0.org 7 #+INCLUDE: ../../aurellem/org/level-0.org
8 #+babel: :mkdirp yes :noweb yes 8 #+babel: :mkdirp yes :noweb yes
9 9
10 * Background 10 * Background
11 Artificial Intelligence has tried and failed for more than half a 11
12 century to produce programs as flexible, creative, and "intelligent" 12 Artificial Intelligence has tried and failed for more than
13 as the human mind itself. Clearly, we are still missing some important 13 half a century to produce programs as flexible, creative,
14 ideas concerning intelligent programs or we would have strong AI 14 and "intelligent" as the human mind itself. Clearly, we are
15 already. What idea could be missing? 15 still missing some important ideas concerning intelligent
16 programs or we would have strong AI already. What idea could
17 be missing?
16 18
17 When Turing first proposed his famous "Turing Test" in the 19 When Turing first proposed his famous "Turing Test" in the
18 groundbreaking paper [[../sources/turing.pdf][/Computing Machines and Intelligence/]], he gave 20 groundbreaking paper [[../sources/turing.pdf][/Computing Machines and Intelligence/]],
19 little importance to how a computer program might interact with the 21 he gave little importance to how a computer program might
20 world: 22 interact with the world:
21 23
22 #+BEGIN_QUOTE 24 #+BEGIN_QUOTE
23 \ldquo{}We need not be too concerned about the legs, eyes, etc. The example of 25 \ldquo{}We need not be too concerned about the legs, eyes,
24 Miss Helen Keller shows that education can take place provided that 26 etc. The example of Miss Helen Keller shows that education
25 communication in both directions between teacher and pupil can take 27 can take place provided that communication in both
26 place by some means or other.\rdquo{} 28 directions between teacher and pupil can take place by some
29 means or other.\rdquo{}
27 #+END_QUOTE 30 #+END_QUOTE
28 31
29 And from the example of Hellen Keller he went on to assume that the 32 And from the example of Hellen Keller he went on to assume
30 only thing a fledgling AI program could need by way of communication 33 that the only thing a fledgling AI program could need by way
31 is a teletypewriter. But Hellen Keller did possess vision and hearing 34 of communication is a teletypewriter. But Hellen Keller did
32 for the first few months of her life, and her tactile sense was far 35 possess vision and hearing for the first few months of her
33 more rich than any text-stream could hope to achieve. She possessed a 36 life, and her tactile sense was far more rich than any
34 body she could move freely, and had continual access to the real world 37 text-stream could hope to achieve. She possessed a body she
35 to learn from her actions. 38 could move freely, and had continual access to the real
36 39 world to learn from her actions.
37 I believe that our programs are suffering from too little sensory 40
38 input to become really intelligent. Imagine for a moment that you 41 I believe that our programs are suffering from too little
39 lived in a world completely cut off form all sensory stimulation. You 42 sensory input to become really intelligent. Imagine for a
40 have no eyes to see, no ears to hear, no mouth to speak. No body, no 43 moment that you lived in a world completely cut off form all
41 taste, no feeling whatsoever. The only sense you get at all is a 44 sensory stimulation. You have no eyes to see, no ears to
42 single point of light, flickering on and off in the void. If this was 45 hear, no mouth to speak. No body, no taste, no feeling
43 your life from birth, you would never learn anything, and could never 46 whatsoever. The only sense you get at all is a single point
44 become intelligent. Actual humans placed in sensory deprivation 47 of light, flickering on and off in the void. If this was
45 chambers experience hallucinations and can begin to loose their sense 48 your life from birth, you would never learn anything, and
46 of reality. Most of the time, the programs we write are in exactly 49 could never become intelligent. Actual humans placed in
47 this situation. They do not interface with cameras and microphones, 50 sensory deprivation chambers experience hallucinations and
48 and they do not control a real or simulated body or interact with any 51 can begin to loose their sense of reality. Most of the time,
49 sort of world. 52 the programs we write are in exactly this situation. They do
53 not interface with cameras and microphones, and they do not
54 control a real or simulated body or interact with any sort
55 of world.
50 56
51 * Simulation vs. Reality 57 * Simulation vs. Reality
58
52 I want demonstrate that multiple senses are what enable 59 I want demonstrate that multiple senses are what enable
53 intelligence. There are two ways of playing around with senses and 60 intelligence. There are two ways of playing around with
54 computer programs: 61 senses and computer programs:
55
56 62
57 ** Simulation 63 ** Simulation
58 The first is to go entirely with simulation: virtual world, virtual 64
59 character, virtual senses. The advantages are that when everything is 65 The first is to go entirely with simulation: virtual world,
60 a simulation, experiments in that simulation are absolutely 66 virtual character, virtual senses. The advantages are that
61 reproducible. It's also easier to change the character and world to 67 when everything is a simulation, experiments in that
62 explore new situations and different sensory combinations. 68 simulation are absolutely reproducible. It's also easier to
63 69 change the character and world to explore new situations and
64 If the world is to be simulated on a computer, then not only do you 70 different sensory combinations.
65 have to worry about whether the character's senses are rich enough to 71
66 learn from the world, but whether the world itself is rendered with 72 If the world is to be simulated on a computer, then not only
67 enough detail and realism to give enough working material to the 73 do you have to worry about whether the character's senses
68 character's senses. To name just a few difficulties facing modern 74 are rich enough to learn from the world, but whether the
69 physics simulators: destructibility of the environment, simulation of 75 world itself is rendered with enough detail and realism to
70 water/other fluids, large areas, nonrigid bodies, lots of objects, 76 give enough working material to the character's senses. To
71 smoke. I don't know of any computer simulation that would allow a 77 name just a few difficulties facing modern physics
72 character to take a rock and grind it into fine dust, then use that 78 simulators: destructibility of the environment, simulation
73 dust to make a clay sculpture, at least not without spending years 79 of water/other fluids, large areas, nonrigid bodies, lots of
74 calculating the interactions of every single small grain of 80 objects, smoke. I don't know of any computer simulation that
75 dust. Maybe a simulated world with today's limitations doesn't provide 81 would allow a character to take a rock and grind it into
82 fine dust, then use that dust to make a clay sculpture, at
83 least not without spending years calculating the
84 interactions of every single small grain of dust. Maybe a
85 simulated world with today's limitations doesn't provide
76 enough richness for real intelligence to evolve. 86 enough richness for real intelligence to evolve.
77 87
78 ** Reality 88 ** Reality
79 89
80 The other approach for playing with senses is to hook your software up 90 The other approach for playing with senses is to hook your
81 to real cameras, microphones, robots, etc., and let it loose in the 91 software up to real cameras, microphones, robots, etc., and
82 real world. This has the advantage of eliminating concerns about 92 let it loose in the real world. This has the advantage of
83 simulating the world at the expense of increasing the complexity of 93 eliminating concerns about simulating the world at the
84 implementing the senses. Instead of just grabbing the current rendered 94 expense of increasing the complexity of implementing the
85 frame for processing, you have to use an actual camera with real 95 senses. Instead of just grabbing the current rendered frame
86 lenses and interact with photons to get an image. It is much harder to 96 for processing, you have to use an actual camera with real
87 change the character, which is now partly a physical robot of some 97 lenses and interact with photons to get an image. It is much
88 sort, since doing so involves changing things around in the real world 98 harder to change the character, which is now partly a
89 instead of modifying lines of code. While the real world is very rich 99 physical robot of some sort, since doing so involves
90 and definitely provides enough stimulation for intelligence to develop 100 changing things around in the real world instead of
91 as evidenced by our own existence, it is also uncontrollable in the 101 modifying lines of code. While the real world is very rich
92 sense that a particular situation cannot be recreated perfectly or 102 and definitely provides enough stimulation for intelligence
93 saved for later use. It is harder to conduct science because it is 103 to develop as evidenced by our own existence, it is also
94 harder to repeat an experiment. The worst thing about using the real 104 uncontrollable in the sense that a particular situation
95 world instead of a simulation is the matter of time. Instead of 105 cannot be recreated perfectly or saved for later use. It is
96 simulated time you get the constant and unstoppable flow of real 106 harder to conduct science because it is harder to repeat an
97 time. This severely limits the sorts of software you can use to 107 experiment. The worst thing about using the real world
98 program the AI because all sense inputs must be handled in real 108 instead of a simulation is the matter of time. Instead of
99 time. Complicated ideas may have to be implemented in hardware or may 109 simulated time you get the constant and unstoppable flow of
100 simply be impossible given the current speed of our 110 real time. This severely limits the sorts of software you
101 processors. Contrast this with a simulation, in which the flow of time 111 can use to program the AI because all sense inputs must be
102 in the simulated world can be slowed down to accommodate the 112 handled in real time. Complicated ideas may have to be
103 limitations of the character's programming. In terms of cost, doing 113 implemented in hardware or may simply be impossible given
104 everything in software is far cheaper than building custom real-time 114 the current speed of our processors. Contrast this with a
115 simulation, in which the flow of time in the simulated world
116 can be slowed down to accommodate the limitations of the
117 character's programming. In terms of cost, doing everything
118 in software is far cheaper than building custom real-time
105 hardware. All you need is a laptop and some patience. 119 hardware. All you need is a laptop and some patience.
106 120
107 * Choose a Simulation Engine 121 * Choose a Simulation Engine
108 122
109 Mainly because of issues with controlling the flow of time, I chose to 123 Mainly because of issues with controlling the flow of time,
110 simulate both the world and the character. I set out to make a world 124 I chose to simulate both the world and the character. I set
111 in which I could embed a character with multiple senses. My main goal 125 out to make a world in which I could embed a character with
112 is to make an environment where I can perform further experiments in 126 multiple senses. My main goal is to make an environment
113 simulated senses. 127 where I can perform further experiments in simulated senses.
114 128
115 I examined many different 3D environments to try and find something I 129 I examined many different 3D environments to try and find
116 would use as the base for my simulation; eventually the choice came 130 something I would use as the base for my simulation;
117 down to three engines: the Quake II engine, the Source Engine, and 131 eventually the choice came down to three engines: the Quake
118 jMonkeyEngine. 132 II engine, the Source Engine, and jMonkeyEngine.
119 133
120 ** [[http://www.idsoftware.com][Quake II]]/[[http://www.bytonic.de/html/jake2.html][Jake2]] 134 ** [[http://www.idsoftware.com][Quake II]]/[[http://www.bytonic.de/html/jake2.html][Jake2]]
121 135
122 I spent a bit more than a month working with the Quake II Engine from 136 I spent a bit more than a month working with the Quake II
123 ID software to see if I could use it for my purposes. All the source 137 Engine from ID software to see if I could use it for my
124 code was released by ID software into the Public Domain several years 138 purposes. All the source code was released by ID software
125 ago, and as a result it has been ported and modified for many 139 into the Public Domain several years ago, and as a result it
126 different reasons. This engine was famous for its advanced use of 140 has been ported and modified for many different
141 reasons. This engine was famous for its advanced use of
127 realistic shading and had decent and fast physics 142 realistic shading and had decent and fast physics
128 simulation. Researchers at Princeton [[http://papers.cnl.salk.edu/PDFs/Intracelllular%20Dynamics%20of%20Virtual%20Place%20Cells%202011-4178.pdf][used this code]] ([[http://brainwindows.wordpress.com/2009/10/14/playing-quake-with-a-real-mouse/][video]]) to study 143 simulation. Researchers at Princeton [[http://papers.cnl.salk.edu/PDFs/Intracelllular%20Dynamics%20of%20Virtual%20Place%20Cells%202011-4178.pdf][used this code]] ([[http://brainwindows.wordpress.com/2009/10/14/playing-quake-with-a-real-mouse/][video]])
129 spatial information encoding in the hippocampal cells of rats. Those 144 to study spatial information encoding in the hippocampal
130 researchers created a special Quake II level that simulated a maze, 145 cells of rats. Those researchers created a special Quake II
131 and added an interface where a mouse could run on top of a ball in 146 level that simulated a maze, and added an interface where a
132 various directions to move the character in the simulated maze. They 147 mouse could run on top of a ball in various directions to
133 measured hippocampal activity during this exercise to try and tease 148 move the character in the simulated maze. They measured
134 out the method in which spatial data was stored in that area of the 149 hippocampal activity during this exercise to try and tease
135 brain. I find this promising because if a real living rat can interact 150 out the method in which spatial data was stored in that area
136 with a computer simulation of a maze in the same way as it interacts 151 of the brain. I find this promising because if a real living
137 with a real-world maze, then maybe that simulation is close enough to 152 rat can interact with a computer simulation of a maze in the
138 reality that a simulated sense of vision and motor control interacting 153 same way as it interacts with a real-world maze, then maybe
139 with that simulation could reveal useful information about the real 154 that simulation is close enough to reality that a simulated
140 thing. There is a Java port of the original C source code called 155 sense of vision and motor control interacting with that
141 Jake2. The port demonstrates Java's OpenGL bindings and runs anywhere 156 simulation could reveal useful information about the real
142 from 90% to 105% as fast as the C version. After reviewing much of the 157 thing. There is a Java port of the original C source code
143 source of Jake2, I eventually rejected it because the engine is too 158 called Jake2. The port demonstrates Java's OpenGL bindings
144 tied to the concept of a first-person shooter game. One of the 159 and runs anywhere from 90% to 105% as fast as the C
145 problems I had was that there do not seem to be any easy way to attach 160 version. After reviewing much of the source of Jake2, I
146 multiple cameras to a single character. There are also several physics 161 rejected it because the engine is too tied to the concept of
147 clipping issues that are corrected in a way that only applies to the 162 a first-person shooter game. One of the problems I had was
148 main character and does not apply to arbitrary objects. While there is 163 that there does not seem to be any easy way to attach
149 a large community of level modders, I couldn't find a community to 164 multiple cameras to a single character. There are also
150 support using the engine to make new things. 165 several physics clipping issues that are corrected in a way
166 that only applies to the main character and do not apply to
167 arbitrary objects. While there is a large community of level
168 modders, I couldn't find a community to support using the
169 engine to make new things.
151 170
152 ** [[http://source.valvesoftware.com/][Source Engine]] 171 ** [[http://source.valvesoftware.com/][Source Engine]]
153 172
154 The Source Engine evolved from the Quake II and Quake I engines and is 173 The Source Engine evolved from the Quake II and Quake I
155 used by Valve in the Half-Life series of games. The physics simulation 174 engines and is used by Valve in the Half-Life series of
156 in the Source Engine is quite accurate and probably the best out of 175 games. The physics simulation in the Source Engine is quite
157 all the engines I investigated. There is also an extensive community 176 accurate and probably the best out of all the engines I
158 actively working with the engine. However, applications that use the 177 investigated. There is also an extensive community actively
159 Source Engine must be written in C++, the code is not open, it only 178 working with the engine. However, applications that use the
160 runs on Windows, and the tools that come with the SDK to handle models 179 Source Engine must be written in C++, the code is not open,
161 and textures are complicated and awkward to use. 180 it only runs on Windows, and the tools that come with the
181 SDK to handle models and textures are complicated and
182 awkward to use.
162 183
163 ** [[http://jmonkeyengine.com/][jMonkeyEngine3]] 184 ** [[http://jmonkeyengine.com/][jMonkeyEngine3]]
164 185
165 jMonkeyEngine is a new library for creating games in Java. It uses 186 jMonkeyEngine is a new library for creating games in
166 OpenGL to render to the screen and uses screengraphs to avoid drawing 187 Java. It uses OpenGL to render to the screen and uses
167 things that do not appear on the screen. It has an active community 188 screengraphs to avoid drawing things that do not appear on
168 and several games in the pipeline. The engine was not built to serve 189 the screen. It has an active community and several games in
169 any particular game but is instead meant to be used for any 3D 190 the pipeline. The engine was not built to serve any
170 game. After experimenting with each of these three engines and a few 191 particular game but is instead meant to be used for any 3D
171 others for about 2 months I settled on jMonkeyEngine. I chose it 192 game. After experimenting with each of these three engines
172 because it had the most features out of all the open projects I looked 193 and a few others for about 2 months I settled on
173 at, and because I could then write my code in Clojure, an 194 jMonkeyEngine. I chose it because it had the most features
174 implementation of LISP that runs on the JVM. 195 out of all the open projects I looked at, and because I
175 196 could then write my code in Clojure, an implementation of
176 197 LISP that runs on the JVM.
177 198
178 199
179 200
180 201
181 202
182 203
183 204
184 205
185 206
186 207
187 208
188 209
189 210
190 211
191 212
192 213
193 214
194 215
195 216
196 217
197 218
198 219
199 220
200 221
201 222
202 223
203 224
204 225
205 226
206 227
207 228
229
230