cortex: thesis/cortex.org comparison

comparison thesis/cortex.org @ 508:c11d3fc3e6f0

inspecting cortex section.

author	Robert McIntyre <rlm@mit.edu>
date	Sun, 30 Mar 2014 01:07:19 -0400
parents	f2f029e1a6a9
children	81c845a91378

comparison

equal deleted inserted replaced

-:f2f029e1a6a9
+:c11d3fc3e6f0
 #+caption: of muscles, ears, eyes, or joints.
 #+name: sense-nodes
 #+ATTR_LaTeX: :width 10cm
 [[./images/empty-sense-nodes.png]]
-** COMMENT Bodies are composed of segments connected by joints
+** Bodies are composed of segments connected by joints
 Blender is a general purpose animation tool, which has been used in
 the past to create high quality movies such as Sintel
-\cite{sintel}. Though Blender can model and render even complicated
+\cite{blender}. Though Blender can model and render even complicated
 things like water, it is crucual to keep models that are meant to
 be simulated as creatures simple. =Bullet=, which =CORTEX= uses
 though jMonkeyEngine3, is a rigid-body physics system. This offers
 a compromise between the expressiveness of a game level and the
 speed at which it can be simulated, and it means that creatures
 #+caption: simulation environment.
 #+name: name
 #+ATTR_LaTeX: :width 15cm
 [[./images/physical-hand.png]]
-** COMMENT Eyes reuse standard video game components
+** Eyes reuse standard video game components
 Vision is one of the most important senses for humans, so I need to
 build a simulated sense of vision for my AI. I will do this with
 simulated eyes. Each eye can be independently moved and should see
 its own version of the world depending on where it is.
 is a =FrameBuffer= which represents the rendered image in the GPU.
 #+caption: =ViewPorts= are cameras in the world. During each frame,
 #+caption: the =RenderManager= records a snapshot of what each view
 #+caption: is currently seeing; these snapshots are =FrameBuffer= objects.
-#+name: name
+#+name: rendermanagers
 #+ATTR_LaTeX: :width 10cm
-[[../images/diagram_rendermanager2.png]]
+[[./images/diagram_rendermanager2.png]]
 Each =ViewPort= can have any number of attached =SceneProcessor=
 objects, which are called every time a new frame is rendered. A
 =SceneProcessor= receives its =ViewPort's= =FrameBuffer= and can do
 whatever it wants to the data.  Often this consists of invoking GPU
 #+caption: This is the core of vision in =CORTEX=. A given eye node
 #+caption: is converted into a function that returns visual
 #+caption: information from the simulation.
 #+name: vision-kernel
 #+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn vision-kernel
 "Returns a list of functions, each of which will return a color
 channel's worth of visual information when called inside a running
 simulation."
 [#^Node creature #^Spatial eye & {skip :skip :or {skip 0}}]
 (pixel-sense
 sensitivity
 (.getRGB @vision-image x y))))))
 register-eye!)))
 retinal-map))))
+#+END_SRC
 #+end_listing
 Note that since each of the functions generated by =vision-kernel=
 shares the same =register-eye!= function, the eye will be
 registered only once the first time any of the functions from the
 #+caption: With =vision!=, =CORTEX= is already a fine simulation
 #+caption: environment for experimenting with different types of
 #+caption: eyes.
 #+name: vision!
 #+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn vision!
 "Returns a list of functions, each of which returns visual sensory
 data when called inside a running simulation."
 [#^Node creature & {skip :skip :or {skip 0}}]
 (reduce
 concat
 (for [eye (eyes creature)]
 (vision-kernel creature eye))))
+#+END_SRC
 #+end_listing
 #+caption: Simulated vision with a test creature and the
 #+caption: human-like eye approximation. Notice how each channel
 #+caption: of the eye responds differently to the differently
 This vision code has already been absorbed by the jMonkeyEngine
 community and is now (in modified form) part of a system for
 capturing in-game video to a file.
-** COMMENT Hearing is hard; =CORTEX= does it right
+** Hearing is hard; =CORTEX= does it right
 At the end of this section I will have simulated ears that work the
 same way as the simulated eyes in the last section. I will be able to
 place any number of ear-nodes in a blender file, and they will bind to
 the closest physical object and follow it as it moves around. Each ear
 #+caption: Program for extending =OpenAL= to support multiple
 #+caption: listeners via context copying/switching.
 #+name: sync-openal-sources
 #+begin_listing C
+#+BEGIN_SRC C
 void syncSources(ALsource *masterSource, ALsource *slaveSource,
 		 ALCcontext *masterCtx, ALCcontext *slaveCtx){
 ALuint master = masterSource->source;
 ALuint slave = slaveSource->source;
 ALCcontext *current = alcGetCurrentContext();
 }
 }
 // Restore whatever context was previously active.
 alcMakeContextCurrent(current);
 }
+#+END_SRC
 #+end_listing
 With this special context-switching device, and some ugly JNI
 bindings that are not worth mentioning, =CORTEX= gains the ability
 to access multiple sound streams from =OpenAL=.
 #+caption: Program to create an ear from a blender empty node. The ear
 #+caption: follows around the nearest physical object and passes
 #+caption: all sensory data to a continuation function.
 #+name: add-ear
 #+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn add-ear!
 "Create a Listener centered on the current position of 'ear
 which follows the closest physical node in 'creature and
 sends sound data to 'continuation."
 [#^Application world #^Node creature #^Spatial ear continuation]
 (.setRotation lis (.getWorldRotation ear))
 (bind-sense target lis)
 (update-listener-velocity! target lis)
 (.addListener audio-renderer lis)
 (.registerSoundProcessor audio-renderer lis sp)))
+#+END_SRC
 #+end_listing
 The =Send= device, unlike most of the other devices in =OpenAL=,
 does not render sound unless asked. This enables the system to
 slow down or speed up depending on the needs of the AIs who are
 using it to listen. If the device tried to render samples in
 all of the sound in its environment!
 #+caption: Program to enable arbitrary hearing in =CORTEX=
 #+name: hearing
 #+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn hearing-kernel
 "Returns a function which returns auditory sensory data when called
 inside a running simulation."
 [#^Node creature #^Spatial ear]
 (let [hearing-data (atom [])
 hearing. Will return a sequence of functions, one for each ear,
 which when called will return the auditory data from that ear."
 [#^Node creature]
 (for [ear (ears creature)]
 (hearing-kernel creature ear)))
+#+END_SRC
 #+end_listing
 Armed with these functions, =CORTEX= is able to test possibly the
 first ever instance of multiple listeners in a video game engine
 based simulation!
 #+caption: Here a simple creature responds to sound by changing
 #+caption: its color from gray to green when the total volume
 #+caption: goes over a threshold.
 #+name: sound-test
 #+begin_listing java
+#+BEGIN_SRC java
 /**
 * Respond to sound!  This is the brain of an AI entity that
 * hears its surroundings and reacts to them.
 */
 public void process(ByteBuffer audioSamples,
 	entity.getMaterial().setColor("Color", ColorRGBA.Green);
 }
 else {
 	entity.getMaterial().setColor("Color", ColorRGBA.Gray);
 }
+#+END_SRC
 #+end_listing
 #+caption: First ever simulation of multiple listerners in =CORTEX=.
 #+caption: Each cube is a creature which processes sound data with
 #+caption: the =process= function from listing \ref{sound-test}.
 #+caption: The hand then uses its muscles to launch the cube!
 #+name: integration
 #+ATTR_LaTeX: :width 16cm
 [[./images/integration.png]]
-** COMMENT =CORTEX= enables many possiblities for further research
+** =CORTEX= enables many possiblities for further research
 Often times, the hardest part of building a system involving
 creatures is dealing with physics and graphics. =CORTEX= removes
 much of this initial difficulty and leaves researchers free to
 directly pursue their ideas. I hope that even undergrads with a

Mercurial > cortex

comparison thesis/cortex.org @ 508:c11d3fc3e6f0