1.1 --- a/org/vision.org	Fri Feb 10 11:37:19 2012 -0700
     1.2 +++ b/org/vision.org	Fri Feb 10 12:06:41 2012 -0700
     1.3 @@ -14,15 +14,15 @@
     1.4  simulated eyes. Each eye can be independely moved and should see its
     1.5  own version of the world depending on where it is.
     1.6  
     1.7 -Making these simulated eyes a reality is fairly simple bacause
     1.8 -jMonkeyEngine already conatains extensive support for multiple views
     1.9 -of the same 3D simulated world. The reason jMonkeyEngine has this
    1.10 -support is because the support is necessary to create games with
    1.11 -split-screen views. Multiple views are also used to create efficient
    1.12 +Making these simulated eyes a reality is simple bacause jMonkeyEngine
    1.13 +already conatains extensive support for multiple views of the same 3D
    1.14 +simulated world. The reason jMonkeyEngine has this support is because
    1.15 +the support is necessary to create games with split-screen
    1.16 +views. Multiple views are also used to create efficient
    1.17  pseudo-reflections by rendering the scene from a certain perspective
    1.18  and then projecting it back onto a surface in the 3D world.
    1.19  
    1.20 -#+caption: jMonkeyEngine supports multiple views to enable split-screen games, like GoldenEye
    1.21 +#+caption: jMonkeyEngine supports multiple views to enable split-screen games, like GoldenEye, which was one of the first games to use split-screen views.
    1.22  [[../images/goldeneye-4-player.png]]
    1.23  
    1.24  * Brief Description of jMonkeyEngine's Rendering Pipeline
    1.25 @@ -49,7 +49,8 @@
    1.26  function may perform both CPU and GPU operations on the data. To make
    1.27  this easy for the continuation function, the =SceneProcessor=
    1.28  maintains appropriatly sized buffers in RAM to hold the data.  It does
    1.29 -not do any copying from the GPU to the CPU itself.
    1.30 +not do any copying from the GPU to the CPU itself because it is a slow
    1.31 +operation.
    1.32  
    1.33  #+name: pipeline-1
    1.34  #+begin_src clojure
    1.35 @@ -88,9 +89,9 @@
    1.36  
    1.37  The continuation function given to =(vision-pipeline)= above will be
    1.38  given a =Renderer= and three containers for image data. The
    1.39 -=FrameBuffer= references the GPU image data, but it can not be used
    1.40 -directly on the CPU.  The =ByteBuffer= and =BufferedImage= are
    1.41 -initially "empty" but are sized to hold to data in the
    1.42 +=FrameBuffer= references the GPU image data, but the pixel data can
    1.43 +not be used directly on the CPU.  The =ByteBuffer= and =BufferedImage=
    1.44 +are initially "empty" but are sized to hold to data in the
    1.45  =FrameBuffer=. I call transfering the GPU image data to the CPU
    1.46  structures "mixing" the image data. I have provided three functions to
    1.47  do this mixing.
    1.48 @@ -172,10 +173,10 @@
    1.49          cam (Camera. cam-width cam-height)
    1.50          rot (.getWorldRotation eye)]
    1.51      (.setLocation cam (.getWorldTranslation eye))
    1.52 -    (.lookAtDirection cam (.mult rot Vector3f/UNIT_X)
    1.53 -                      ;; this part is consistent with using Z in
    1.54 -                      ;; blender as the UP vector.
    1.55 -                      (.mult rot Vector3f/UNIT_Y))
    1.56 +    (.lookAtDirection
    1.57 +     cam                                ; this part is not a mistake and
    1.58 +     (.mult rot Vector3f/UNIT_X)        ; is consistent with using Z in
    1.59 +     (.mult rot Vector3f/UNIT_Y))       ; blender as the UP vector.
    1.60      (.setFrustumPerspective
    1.61       cam 45 (/ (.getWidth cam) (.getHeight cam)) 1 1000)
    1.62      (bind-sense target cam) cam))
    1.63 @@ -188,7 +189,7 @@
    1.64  ** The Retina
    1.65  
    1.66  An eye is a surface (the retina) which contains many discrete sensors
    1.67 -to detect light. These sensors have can have different-light sensing
    1.68 +to detect light. These sensors have can have different light-sensing
    1.69  properties.  In humans, each discrete sensor is sensitive to red,
    1.70  blue, green, or gray. These different types of sensors can have
    1.71  different spatial distributions along the retina. In humans, there is
    1.72 @@ -227,8 +228,8 @@
    1.73  
    1.74  The numbers that serve as keys in the map determine a sensor's
    1.75  relative sensitivity to the channels red, green, and blue. These
    1.76 -sensitivity values are packed into an integer in the order _RGB in
    1.77 -8-bit fields. The RGB values of a pixel in the image are added
    1.78 +sensitivity values are packed into an integer in the order =|_|R|G|B|=
    1.79 +in 8-bit fields. The RGB values of a pixel in the image are added
    1.80  together with these sensitivities as linear weights. Therfore,
    1.81  0xFF0000 means sensitive to red only while 0xFFFFFF means sensitive to
    1.82  all colors equally (gray).
    1.83 @@ -267,8 +268,8 @@
    1.84        load-image
    1.85        (eval (read-string eye-map)))))
    1.86  
    1.87 -(defn eye-dimensions
    1.88 -  "Returns [width, height] specified in the metadata of the eye"
    1.89 +(defn eye-dimensions 
    1.90 +  "Returns [width, height] determined by the metadata of the eye."
    1.91    [#^Spatial eye]
    1.92    (let [dimensions
    1.93            (map #(vector (.getWidth %) (.getHeight %))
    1.94 @@ -277,9 +278,7 @@
    1.95       (apply max (map second dimensions))]))
    1.96  #+end_src
    1.97  
    1.98 -
    1.99  * Eye Creation 
   1.100 -
   1.101  First off, get the children of the "eyes" empty node to find all the
   1.102  eyes the creature has.
   1.103  #+name: eye-node
   1.104 @@ -312,13 +311,13 @@
   1.105  #+end_src
   1.106  
   1.107  
   1.108 -The continuation function registers the viewport with the simulation
   1.109 -the first time it is called, and uses the CPU to extract the
   1.110 +The eye's continuation function should register the viewport with the
   1.111 +simulation the first time it is called, use the CPU to extract the
   1.112  appropriate pixels from the rendered image and weight them by each
   1.113 -sensors sensitivity. I have the option to do this filtering in native
   1.114 -code for a slight gain in speed. I could also do it in the GPU for a
   1.115 -massive gain in speed. =(vision-kernel)= generates a list of such
   1.116 -continuation functions, one for each channel of the eye.
   1.117 +sensor's sensitivity. I have the option to do this processing in
   1.118 +native code for a slight gain in speed. I could also do it in the GPU
   1.119 +for a massive gain in speed. =(vision-kernel)= generates a list of
   1.120 +such continuation functions, one for each channel of the eye.
   1.121  
   1.122  #+name: kernel
   1.123  #+begin_src clojure 
   1.124 @@ -408,7 +407,7 @@
   1.125  
   1.126  ** Vision!
   1.127  
   1.128 -All the hard work has been done, all that remains is to apply
   1.129 +All the hard work has been done; all that remains is to apply
   1.130  =(vision-kernel)= to each eye in the creature and gather the results
   1.131  into one list of functions.
   1.132  
   1.133 @@ -416,7 +415,7 @@
   1.134  #+begin_src clojure
   1.135  (defn vision!
   1.136    "Returns a function which returns visual sensory data when called
   1.137 -   inside a running simulation"
   1.138 +   inside a running simulation."
   1.139    [#^Node creature & {skip :skip :or {skip 0}}]
   1.140    (reduce
   1.141     concat 
   1.142 @@ -517,7 +516,7 @@
   1.143  
   1.144  ** Adding Vision to the Worm
   1.145  
   1.146 -To the worm from the last post, we add a new node that describes its
   1.147 +To the worm from the last post, I add a new node that describes its
   1.148  eyes.
   1.149  
   1.150  #+attr_html: width=755
   1.151 @@ -525,17 +524,19 @@
   1.152  [[../images/worm-with-eye.png]]
   1.153  
   1.154  The node highlighted in yellow is the root level "eyes" node.  It has
   1.155 -a single node, highlighted in orange, which describes a single
   1.156 -eye. This is the "eye" node. The two nodes which are not highlighted describe the single joint
   1.157 -of the worm.
   1.158 +a single child, highlighted in orange, which describes a single
   1.159 +eye. This is the "eye" node. It is placed so that the worm will have
   1.160 +an eye located in the center of the flat portion of its lower
   1.161 +hemispherical section.
   1.162 +
   1.163 +The two nodes which are not highlighted describe the single joint of
   1.164 +the worm.
   1.165  
   1.166  The metadata of the eye-node is:
   1.167  
   1.168  #+begin_src clojure :results verbatim :exports both
   1.169  (cortex.sense/meta-data
   1.170 - (.getChild
   1.171 -  (.getChild (cortex.test.body/worm)
   1.172 -            "eyes") "eye") "eye")
   1.173 + (.getChild (.getChild (cortex.test.body/worm) "eyes") "eye") "eye")
   1.174  #+end_src
   1.175  
   1.176  #+results:
   1.177 @@ -608,7 +609,25 @@
   1.178               (fix-display world)))))
   1.179  #+end_src
   1.180  
   1.181 -** Methods to Generate the Worm Video
   1.182 +The world consists of the worm and a flat gray floor. I can shoot red,
   1.183 +green, blue and white cannonballs at the worm. The worm is initially
   1.184 +looking down at the floor, and there is no gravity. My perspective
   1.185 +(the Main View), the worm's perspective (Worm View) and the 4 sensor
   1.186 +channels that comprise the worm's eye are all saved frame-by-frame to
   1.187 +disk.
   1.188 +
   1.189 +* Demonstration of Vision
   1.190 +#+begin_html
   1.191 +<div class="figure">
   1.192 +<video controls="controls" width="755">
   1.193 +  <source src="../video/worm-vision.ogg" type="video/ogg"
   1.194 +	  preload="none" poster="../images/aurellem-1280x480.png" />
   1.195 +</video>
   1.196 +<p>Simulated Vision in a Virtual Environment</p>
   1.197 +</div>
   1.198 +#+end_html
   1.199 +
   1.200 +** Generate the Worm Video from Frames
   1.201  #+name: magick2
   1.202  #+begin_src clojure
   1.203  (ns cortex.video.magick2
   1.204 @@ -660,17 +679,6 @@
   1.205  ffmpeg -r 25 -b 9001k -i out/%07d.png -vcodec libtheora worm-vision.ogg 
   1.206  #+end_src
   1.207  
   1.208 -* Demonstration of Vision
   1.209 -#+begin_html
   1.210 -<div class="figure">
   1.211 -<video controls="controls" width="755">
   1.212 -  <source src="../video/worm-vision.ogg" type="video/ogg"
   1.213 -	  preload="none" poster="../images/aurellem-1280x480.png" />
   1.214 -</video>
   1.215 -<p>Simulated Vision in a Virtual Environment</p>
   1.216 -</div>
   1.217 -#+end_html
   1.218 -
   1.219  * Headers
   1.220  
   1.221  #+name: vision-header