1.1 --- a/thesis/cortex.org	Fri Mar 28 21:05:12 2014 -0400
     1.2 +++ b/thesis/cortex.org	Fri Mar 28 21:48:53 2014 -0400
     1.3 @@ -27,6 +27,8 @@
     1.4      #+caption: 
     1.5      #+name: name
     1.6      #+begin_listing clojure
     1.7 +    #+BEGIN_SRC clojure
     1.8 +    #+END_SRC
     1.9      #+end_listing
    1.10  
    1.11      #+caption: 
    1.12 @@ -1586,7 +1588,7 @@
    1.13     of the feelers is determined by a UV-mapped image which shows where
    1.14     each feeler should be on the 3D surface of the body.
    1.15  
    1.16 -*** Defining Touch Meta-Data in Blender
    1.17 +*** COMMENT Defining Touch Meta-Data in Blender
    1.18  
    1.19      Each geometry can have a single UV map which describes the
    1.20      position of the feelers which will constitute its sense of touch.
    1.21 @@ -1595,8 +1597,11 @@
    1.22      feeler is present) and white meaning a feeler length of =scale=,
    1.23      which is a float stored under the key "scale".
    1.24  
    1.25 -#+name: meta-data
    1.26 -#+begin_src clojure
    1.27 +    #+caption: Touch does not use empty nodes, to store metadata, 
    1.28 +    #+caption: because the metadata of each solid part of a 
    1.29 +    #+caption: creature's body is sufficient.
    1.30 +    #+name: touch-meta-data
    1.31 +    #+begin_listing clojure
    1.32  (defn tactile-sensor-profile
    1.33    "Return the touch-sensor distribution image in BufferedImage format,
    1.34     or nil if it does not exist."
    1.35 @@ -1610,16 +1615,21 @@
    1.36    [#^Geometry obj]
    1.37    (if-let [scale (meta-data obj "scale")]
    1.38      scale 0.1))
    1.39 -#+end_src
    1.40 +    #+end_listing
    1.41  
    1.42 -    Here is an example of a UV-map which specifies the position of touch
    1.43 -    sensors along the surface of the upper segment of the worm.
    1.44 +    Here is an example of a UV-map which specifies the position of
    1.45 +    touch sensors along the surface of the upper segment of a fingertip.
    1.46  
    1.47 -    #+attr_html: width=755
    1.48 -    #+caption: This is the tactile-sensor-profile for the upper segment of the worm. It defines regions of high touch sensitivity (where there are many white pixels) and regions of low sensitivity (where white pixels are sparse).
    1.49 +
    1.50 +    #+caption: This is the tactile-sensor-profile for the upper segment 
    1.51 +    #+caption: of a fingertip. It defines regions of high touch sensitivity 
    1.52 +    #+caption: (where there are many white pixels) and regions of low 
    1.53 +    #+caption: sensitivity (where white pixels are sparse).
    1.54 +    #+name: fimgertip-UV
    1.55 +    #+ATTR_LaTeX: :width 10cm
    1.56      [[../images/finger-UV.png]]
    1.57  
    1.58 -*** Implementation Summary
    1.59 +*** COMMENT Implementation Summary
    1.60    
    1.61      To simulate touch there are three conceptual steps. For each solid
    1.62      object in the creature, you first have to get UV image and scale
    1.63 @@ -1636,33 +1646,35 @@
    1.64      Translating the positions and orientations of the feelers from the
    1.65      UV-map to world-space is itself a three-step process.
    1.66  
    1.67 -  - Find the triangles which make up the mesh in pixel-space and in
    1.68 -    world-space. =triangles= =pixel-triangles=.
    1.69 +    - Find the triangles which make up the mesh in pixel-space and in
    1.70 +      world-space. (=triangles= =pixel-triangles=).
    1.71  
    1.72 -  - Find the coordinates of each feeler in world-space. These are the
    1.73 -    origins of the feelers. =feeler-origins=.
    1.74 +    - Find the coordinates of each feeler in world-space. These are
    1.75 +      the origins of the feelers. (=feeler-origins=).
    1.76      
    1.77 -  - Calculate the normals of the triangles in world space, and add
    1.78 -    them to each of the origins of the feelers. These are the
    1.79 -    normalized coordinates of the tips of the feelers. =feeler-tips=.
    1.80 +    - Calculate the normals of the triangles in world space, and add
    1.81 +      them to each of the origins of the feelers. These are the
    1.82 +      normalized coordinates of the tips of the feelers.
    1.83 +      (=feeler-tips=).
    1.84  
    1.85 -*** Triangle Math
    1.86 +*** COMMENT Triangle Math
    1.87  
    1.88 -The rigid objects which make up a creature have an underlying
    1.89 -=Geometry=, which is a =Mesh= plus a =Material= and other important
    1.90 -data involved with displaying the object.
    1.91 +    The rigid objects which make up a creature have an underlying
    1.92 +    =Geometry=, which is a =Mesh= plus a =Material= and other
    1.93 +    important data involved with displaying the object.
    1.94 +    
    1.95 +    A =Mesh= is composed of =Triangles=, and each =Triangle= has three
    1.96 +    vertices which have coordinates in world space and UV space.
    1.97 +    
    1.98 +    Here, =triangles= gets all the world-space triangles which
    1.99 +    comprise a mesh, while =pixel-triangles= gets those same triangles
   1.100 +    expressed in pixel coordinates (which are UV coordinates scaled to
   1.101 +    fit the height and width of the UV image).
   1.102  
   1.103 -A =Mesh= is composed of =Triangles=, and each =Triangle= has three
   1.104 -vertices which have coordinates in world space and UV space.
   1.105 - 
   1.106 -Here, =triangles= gets all the world-space triangles which comprise a
   1.107 -mesh, while =pixel-triangles= gets those same triangles expressed in
   1.108 -pixel coordinates (which are UV coordinates scaled to fit the height
   1.109 -and width of the UV image).
   1.110 -
   1.111 -#+name: triangles-2
   1.112 -#+begin_src clojure
   1.113 -(in-ns 'cortex.touch)
   1.114 +    #+caption: Programs to extract triangles from a geometry and get 
   1.115 +    #+caption: their verticies in both world and UV-coordinates.
   1.116 +    #+name: get-triangles
   1.117 +    #+begin_listing clojure
   1.118  (defn triangle
   1.119    "Get the triangle specified by triangle-index from the mesh."
   1.120    [#^Geometry geo triangle-index]
   1.121 @@ -1684,7 +1696,7 @@
   1.122      (.getTriangle mesh triangle-index indices)
   1.123      (vec indices)))
   1.124  
   1.125 -(defn vertex-UV-coord
   1.126 +    (defn vertex-UV-coord
   1.127    "Get the UV-coordinates of the vertex named by vertex-index"
   1.128    [#^Mesh mesh vertex-index]
   1.129    (let [UV-buffer
   1.130 @@ -1711,48 +1723,49 @@
   1.131          width (.getWidth image)]
   1.132      (map (partial pixel-triangle geo image)
   1.133           (range (.getTriangleCount (.getMesh geo))))))
   1.134 -#+end_src
   1.135 -
   1.136 +    #+end_listing
   1.137 +    
   1.138  *** The Affine Transform from one Triangle to Another
   1.139  
   1.140 -=pixel-triangles= gives us the mesh triangles expressed in pixel
   1.141 -coordinates and =triangles= gives us the mesh triangles expressed in
   1.142 -world coordinates. The tactile-sensor-profile gives the position of
   1.143 -each feeler in pixel-space. In order to convert pixel-space
   1.144 -coordinates into world-space coordinates we need something that takes
   1.145 -coordinates on the surface of one triangle and gives the corresponding
   1.146 -coordinates on the surface of another triangle.
   1.147 +    =pixel-triangles= gives us the mesh triangles expressed in pixel
   1.148 +    coordinates and =triangles= gives us the mesh triangles expressed
   1.149 +    in world coordinates. The tactile-sensor-profile gives the
   1.150 +    position of each feeler in pixel-space. In order to convert
   1.151 +    pixel-space coordinates into world-space coordinates we need
   1.152 +    something that takes coordinates on the surface of one triangle
   1.153 +    and gives the corresponding coordinates on the surface of another
   1.154 +    triangle.
   1.155 +    
   1.156 +    Triangles are [[http://mathworld.wolfram.com/AffineTransformation.html ][affine]], which means any triangle can be transformed
   1.157 +    into any other by a combination of translation, scaling, and
   1.158 +    rotation. The affine transformation from one triangle to another
   1.159 +    is readily computable if the triangle is expressed in terms of a
   1.160 +    $4x4$ matrix.
   1.161 +    
   1.162 +    \begin{bmatrix}
   1.163 +    x_1 & x_2 & x_3 & n_x \\
   1.164 +    y_1 & y_2 & y_3 & n_y \\ 
   1.165 +    z_1 & z_2 & z_3 & n_z \\
   1.166 +    1 & 1 & 1 & 1 
   1.167 +    \end{bmatrix}
   1.168 +    
   1.169 +    Here, the first three columns of the matrix are the vertices of
   1.170 +    the triangle. The last column is the right-handed unit normal of
   1.171 +    the triangle.
   1.172 +    
   1.173 +    With two triangles $T_{1}$ and $T_{2}$ each expressed as a matrix
   1.174 +    like above, the affine transform from $T_{1}$ to $T_{2}$ is
   1.175 +    
   1.176 +    $T_{2}T_{1}^{-1}$
   1.177 +    
   1.178 +    The clojure code below recapitulates the formulas above, using
   1.179 +    jMonkeyEngine's =Matrix4f= objects, which can describe any affine
   1.180 +    transformation.
   1.181  
   1.182 -Triangles are [[http://mathworld.wolfram.com/AffineTransformation.html ][affine]], which means any triangle can be transformed into
   1.183 -any other by a combination of translation, scaling, and
   1.184 -rotation. The affine transformation from one triangle to another
   1.185 -is readily computable if the triangle is expressed in terms of a $4x4$
   1.186 -matrix.
   1.187 -
   1.188 -\begin{bmatrix}
   1.189 -x_1 & x_2 & x_3 & n_x \\
   1.190 -y_1 & y_2 & y_3 & n_y \\
   1.191 -z_1 & z_2 & z_3 & n_z \\
   1.192 -1 & 1 & 1 & 1
   1.193 -\end{bmatrix}
   1.194 -
   1.195 -Here, the first three columns of the matrix are the vertices of the
   1.196 -triangle. The last column is the right-handed unit normal of the
   1.197 -triangle.
   1.198 -
   1.199 -With two triangles $T_{1}$ and $T_{2}$ each expressed as a matrix like
   1.200 -above, the affine transform from $T_{1}$ to $T_{2}$ is 
   1.201 -
   1.202 -$T_{2}T_{1}^{-1}$
   1.203 -
   1.204 -The clojure code below recapitulates the formulas above, using
   1.205 -jMonkeyEngine's =Matrix4f= objects, which can describe any affine
   1.206 -transformation.
   1.207 -
   1.208 -#+name: triangles-3
   1.209 -#+begin_src clojure
   1.210 -(in-ns 'cortex.touch)
   1.211 -
   1.212 +    #+caption: Program to interpert triangles as affine transforms.
   1.213 +    #+name: triangle-affine
   1.214 +    #+begin_listing clojure
   1.215 +    #+BEGIN_SRC clojure
   1.216  (defn triangle->matrix4f
   1.217    "Converts the triangle into a 4x4 matrix: The first three columns
   1.218     contain the vertices of the triangle; the last contains the unit
   1.219 @@ -1777,9 +1790,10 @@
   1.220    (.mult 
   1.221     (triangle->matrix4f tri-2)
   1.222     (.invert (triangle->matrix4f tri-1))))
   1.223 -#+end_src
   1.224 +    #+END_SRC
   1.225 +    #+end_listing
   1.226  
   1.227 -*** Triangle Boundaries
   1.228 +*** COMMENT Triangle Boundaries
   1.229    
   1.230  For efficiency's sake I will divide the tactile-profile image into
   1.231  small squares which inscribe each pixel-triangle, then extract the
   1.232 @@ -1791,8 +1805,11 @@
   1.233  =inside-triangle?= determines whether a point is inside a triangle
   1.234  in 2D pixel-space.
   1.235  
   1.236 -#+name: triangles-4
   1.237 -#+begin_src clojure
   1.238 +    #+caption: Program to efficiently determine point includion 
   1.239 +    #+caption: in a triangle.
   1.240 +    #+name: in-triangle
   1.241 +    #+begin_listing clojure
   1.242 +    #+BEGIN_SRC clojure
   1.243  (defn convex-bounds
   1.244    "Returns the smallest square containing the given vertices, as a
   1.245     vector of integers [left top width height]."
   1.246 @@ -1824,17 +1841,20 @@
   1.247       (same-side? vert-1 vert-2 vert-3 p)
   1.248       (same-side? vert-2 vert-3 vert-1 p)
   1.249       (same-side? vert-3 vert-1 vert-2 p))))
   1.250 -#+end_src
   1.251 +    #+END_SRC
   1.252 +    #+end_listing
   1.253  
   1.254 -*** Feeler Coordinates
   1.255 +*** COMMENT Feeler Coordinates
   1.256  
   1.257 -The triangle-related functions above make short work of calculating
   1.258 -the positions and orientations of each feeler in world-space.
   1.259 +    The triangle-related functions above make short work of
   1.260 +    calculating the positions and orientations of each feeler in
   1.261 +    world-space.
   1.262  
   1.263 -#+name: sensors
   1.264 -#+begin_src clojure
   1.265 -(in-ns 'cortex.touch)
   1.266 -
   1.267 +    #+caption: Program to get the coordinates of ``feelers '' in 
   1.268 +    #+caption: both world and UV-coordinates.
   1.269 +    #+name: feeler-coordinates
   1.270 +    #+begin_listing clojure
   1.271 +    #+BEGIN_SRC clojure
   1.272  (defn feeler-pixel-coords
   1.273   "Returns the coordinates of the feelers in pixel space in lists, one
   1.274    list for each triangle, ordered in the same way as (triangles) and
   1.275 @@ -1862,7 +1882,14 @@
   1.276     (map (fn [transform coords]
   1.277            (map #(.mult transform (->vector3f %)) coords))
   1.278          transforms (feeler-pixel-coords geo image))))
   1.279 +    #+END_SRC
   1.280 +    #+end_listing
   1.281  
   1.282 +    #+caption: Program to get the position of the base and tip of 
   1.283 +    #+caption: each ``feeler''
   1.284 +    #+name: feeler-tips
   1.285 +    #+begin_listing clojure
   1.286 +    #+BEGIN_SRC clojure
   1.287  (defn feeler-origins
   1.288    "The world space coordinates of the root of each feeler."
   1.289    [#^Geometry geo image]
   1.290 @@ -1884,21 +1911,24 @@
   1.291              world-coords normals)))
   1.292  
   1.293  (defn touch-topology
   1.294 -  "touch-topology? is not a function."
   1.295    [#^Geometry geo image]
   1.296    (collapse (reduce concat (feeler-pixel-coords geo image))))
   1.297 -#+end_src
   1.298 -*** Simulated Touch
   1.299 +    #+END_SRC
   1.300 +    #+end_listing
   1.301  
   1.302 -=touch-kernel= generates functions to be called from within a
   1.303 -simulation that perform the necessary physics collisions to collect
   1.304 -tactile data, and =touch!= recursively applies it to every node in
   1.305 -the creature. 
   1.306 +*** COMMENT Simulated Touch
   1.307  
   1.308 -#+name: kernel
   1.309 -#+begin_src clojure
   1.310 -(in-ns 'cortex.touch)
   1.311 +    Now that the functions to construct feelers are complete,
   1.312 +    =touch-kernel= generates functions to be called from within a
   1.313 +    simulation that perform the necessary physics collisions to
   1.314 +    collect tactile data, and =touch!= recursively applies it to every
   1.315 +    node in the creature.
   1.316  
   1.317 +    #+caption: Efficient program to transform a ray from 
   1.318 +    #+caption: one position to another.
   1.319 +    #+name: set-ray
   1.320 +    #+begin_listing clojure
   1.321 +    #+BEGIN_SRC clojure
   1.322  (defn set-ray [#^Ray ray #^Matrix4f transform
   1.323                 #^Vector3f origin #^Vector3f tip]
   1.324    ;; Doing everything locally reduces garbage collection by enough to
   1.325 @@ -1907,9 +1937,15 @@
   1.326    (.mult transform tip (.getDirection ray))
   1.327    (.subtractLocal (.getDirection ray) (.getOrigin ray))
   1.328    (.normalizeLocal (.getDirection ray)))
   1.329 +    #+END_SRC
   1.330 +    #+end_listing
   1.331  
   1.332 -(import com.jme3.math.FastMath)
   1.333 - 
   1.334 +    #+caption: This is the core of touch in =CORTEX= each feeler 
   1.335 +    #+caption: follows the object it is bound to, reporting any 
   1.336 +    #+caption: collisions that may happen.
   1.337 +    #+name: touch-kernel
   1.338 +    #+begin_listing clojure
   1.339 +    #+BEGIN_SRC clojure
   1.340  (defn touch-kernel
   1.341    "Constructs a function which will return tactile sensory data from
   1.342     'geo when called from inside a running simulation"
   1.343 @@ -1922,7 +1958,6 @@
   1.344            current-rays (map (fn [_] (Ray.)) ray-reference-origins)
   1.345            topology (touch-topology geo profile)
   1.346            correction (float (* ray-length -0.2))]
   1.347 -
   1.348        ;; slight tolerance for very close collisions.
   1.349        (dorun
   1.350         (map (fn [origin tip]
   1.351 @@ -1960,7 +1995,19 @@
   1.352                             (float 0.0)
   1.353                             limit)))
   1.354                       limit])))))))))))
   1.355 +    #+END_SRC
   1.356 +    #+end_listing
   1.357  
   1.358 +    Armed with the =touch!= function, =CORTEX= becomes capable of
   1.359 +    giving creatures a sense of touch. A simple test is to create a
   1.360 +    cube that is outfitted with a uniform distrubition of touch
   1.361 +    sensors. It can feel the ground and any balls that it touches.
   1.362 +
   1.363 +    #+caption: =CORTEX= interface for creating touch in a simulated
   1.364 +    #+caption: creature.
   1.365 +    #+name: touch
   1.366 +    #+begin_listing clojure
   1.367 +    #+BEGIN_SRC clojure
   1.368  (defn touch! 
   1.369    "Endow the creature with the sense of touch. Returns a sequence of
   1.370     functions, one for each body part with a tactile-sensor-profile,
   1.371 @@ -1971,16 +2018,29 @@
   1.372     (map touch-kernel
   1.373          (filter #(isa? (class %) Geometry)
   1.374                  (node-seq creature)))))
   1.375 -#+end_src
   1.376 +    #+END_SRC
   1.377 +    #+end_listing
   1.378 +    
   1.379 +    The tactile-sensor-profile image for the touch cube is a simple
   1.380 +    cross with a unifom distribution of touch sensors:
   1.381  
   1.382 +    #+caption: The touch profile for the touch-cube. Each pure white 
   1.383 +    #+caption: pixel defines a touch sensitive feeler.
   1.384 +    #+name: touch-cube-uv-map
   1.385 +    #+ATTR_LaTeX: :width 10cm
   1.386 +    [[./images/touch-profile.png]]
   1.387  
   1.388 -Armed with the =touch!= function, =CORTEX= becomes capable of giving
   1.389 -creatures a sense of touch. A simple test is to create a cube that is
   1.390 -outfitted with a uniform distrubition of touch sensors. It can feel
   1.391 -the ground and any balls that it touches.
   1.392 -
   1.393 -# insert touch cube image; UV map
   1.394 -# insert video
   1.395 +    #+caption: The touch cube reacts to canonballs. The black, red, 
   1.396 +    #+caption: and white cross on the right is a visual display of 
   1.397 +    #+caption: the creature's touch. White means that it is feeling 
   1.398 +    #+caption: something strongly, black is not feeling anything,
   1.399 +    #+caption: and gray is in-between. The cube can feel both the 
   1.400 +    #+caption: floor and the ball. Notice that when the ball causes 
   1.401 +    #+caption: the cube to tip, that the bottom face can still feel 
   1.402 +    #+caption: part of the ground.
   1.403 +    #+name: touch-cube-uv-map
   1.404 +    #+ATTR_LaTeX: :width 15cm
   1.405 +    [[./images/touch-cube.png]]
   1.406  
   1.407  ** Proprioception is the sense that makes everything ``real''
   1.408