# HG changeset patch # User Robert McIntyre # Date 1396156039 14400 # Node ID c11d3fc3e6f000f22468e19bac7df16489968459 # Parent f2f029e1a6a9b96927627f7991bb8aaf2c906e80 inspecting cortex section. diff -r f2f029e1a6a9 -r c11d3fc3e6f0 thesis/cortex.bib --- a/thesis/cortex.bib Sun Mar 30 00:53:14 2014 -0400 +++ b/thesis/cortex.bib Sun Mar 30 01:07:19 2014 -0400 @@ -12,7 +12,10 @@ year = 2013, addendum = {\why{All complicated creatures in {\tt CORTEX} are described using Blender's extensive 3D modeling - capabilities.}} + capabilities. Blender is a very sophistaced 3D + modeling environment and has been used to create a + short movie called Sintel + \url{http://www.sintel.org/}}} } @inproceedings{winston-directed-perception, diff -r f2f029e1a6a9 -r c11d3fc3e6f0 thesis/cortex.org --- a/thesis/cortex.org Sun Mar 30 00:53:14 2014 -0400 +++ b/thesis/cortex.org Sun Mar 30 01:07:19 2014 -0400 @@ -641,11 +641,11 @@ #+ATTR_LaTeX: :width 10cm [[./images/empty-sense-nodes.png]] -** COMMENT Bodies are composed of segments connected by joints +** Bodies are composed of segments connected by joints Blender is a general purpose animation tool, which has been used in the past to create high quality movies such as Sintel - \cite{sintel}. Though Blender can model and render even complicated + \cite{blender}. Though Blender can model and render even complicated things like water, it is crucual to keep models that are meant to be simulated as creatures simple. =Bullet=, which =CORTEX= uses though jMonkeyEngine3, is a rigid-body physics system. This offers @@ -957,7 +957,7 @@ #+ATTR_LaTeX: :width 15cm [[./images/physical-hand.png]] -** COMMENT Eyes reuse standard video game components +** Eyes reuse standard video game components Vision is one of the most important senses for humans, so I need to build a simulated sense of vision for my AI. I will do this with @@ -991,9 +991,9 @@ #+caption: =ViewPorts= are cameras in the world. During each frame, #+caption: the =RenderManager= records a snapshot of what each view #+caption: is currently seeing; these snapshots are =FrameBuffer= objects. - #+name: name + #+name: rendermanagers #+ATTR_LaTeX: :width 10cm - [[../images/diagram_rendermanager2.png]] + [[./images/diagram_rendermanager2.png]] Each =ViewPort= can have any number of attached =SceneProcessor= objects, which are called every time a new frame is rendered. A @@ -1165,6 +1165,7 @@ #+caption: information from the simulation. #+name: vision-kernel #+begin_listing clojure + #+BEGIN_SRC clojure (defn vision-kernel "Returns a list of functions, each of which will return a color channel's worth of visual information when called inside a running @@ -1205,6 +1206,7 @@ (.getRGB @vision-image x y)))))) register-eye!))) retinal-map)))) + #+END_SRC #+end_listing Note that since each of the functions generated by =vision-kernel= @@ -1224,6 +1226,7 @@ #+caption: eyes. #+name: vision! #+begin_listing clojure + #+BEGIN_SRC clojure (defn vision! "Returns a list of functions, each of which returns visual sensory data when called inside a running simulation." @@ -1232,6 +1235,7 @@ concat (for [eye (eyes creature)] (vision-kernel creature eye)))) + #+END_SRC #+end_listing #+caption: Simulated vision with a test creature and the @@ -1254,7 +1258,7 @@ community and is now (in modified form) part of a system for capturing in-game video to a file. -** COMMENT Hearing is hard; =CORTEX= does it right +** Hearing is hard; =CORTEX= does it right At the end of this section I will have simulated ears that work the same way as the simulated eyes in the last section. I will be able to @@ -1375,6 +1379,7 @@ #+caption: listeners via context copying/switching. #+name: sync-openal-sources #+begin_listing C + #+BEGIN_SRC C void syncSources(ALsource *masterSource, ALsource *slaveSource, ALCcontext *masterCtx, ALCcontext *slaveCtx){ ALuint master = masterSource->source; @@ -1438,6 +1443,7 @@ // Restore whatever context was previously active. alcMakeContextCurrent(current); } + #+END_SRC #+end_listing With this special context-switching device, and some ugly JNI @@ -1449,6 +1455,7 @@ #+caption: all sensory data to a continuation function. #+name: add-ear #+begin_listing clojure + #+BEGIN_SRC clojure (defn add-ear! "Create a Listener centered on the current position of 'ear which follows the closest physical node in 'creature and @@ -1464,8 +1471,8 @@ (update-listener-velocity! target lis) (.addListener audio-renderer lis) (.registerSoundProcessor audio-renderer lis sp))) + #+END_SRC #+end_listing - The =Send= device, unlike most of the other devices in =OpenAL=, does not render sound unless asked. This enables the system to @@ -1478,6 +1485,7 @@ #+caption: Program to enable arbitrary hearing in =CORTEX= #+name: hearing #+begin_listing clojure +#+BEGIN_SRC clojure (defn hearing-kernel "Returns a function which returns auditory sensory data when called inside a running simulation." @@ -1504,6 +1512,7 @@ [#^Node creature] (for [ear (ears creature)] (hearing-kernel creature ear))) + #+END_SRC #+end_listing Armed with these functions, =CORTEX= is able to test possibly the @@ -1515,6 +1524,7 @@ #+caption: goes over a threshold. #+name: sound-test #+begin_listing java + #+BEGIN_SRC java /** * Respond to sound! This is the brain of an AI entity that * hears its surroundings and reacts to them. @@ -1539,6 +1549,7 @@ else { entity.getMaterial().setColor("Color", ColorRGBA.Gray); } + #+END_SRC #+end_listing #+caption: First ever simulation of multiple listerners in =CORTEX=. @@ -2358,7 +2369,7 @@ #+ATTR_LaTeX: :width 16cm [[./images/integration.png]] -** COMMENT =CORTEX= enables many possiblities for further research +** =CORTEX= enables many possiblities for further research Often times, the hardest part of building a system involving creatures is dealing with physics and graphics. =CORTEX= removes diff -r f2f029e1a6a9 -r c11d3fc3e6f0 thesis/images/diagram_rendermanager2.png Binary file thesis/images/diagram_rendermanager2.png has changed diff -r f2f029e1a6a9 -r c11d3fc3e6f0 thesis/images/goldeneye-4-player.png Binary file thesis/images/goldeneye-4-player.png has changed