cortex: thesis/cortex.org comparison

comparison thesis/cortex.org @ 475:3ec428e096e5

most of the way to getting touch integrated.

author	Robert McIntyre <rlm@mit.edu>
date	Fri, 28 Mar 2014 21:48:53 -0400
parents	57c7d5aec8d5
children	5a15611fbb9f

comparison

equal deleted inserted replaced

-:57c7d5aec8d5
+:3ec428e096e5
 #+caption:
 #+caption:
 #+caption:
 #+name: name
 #+begin_listing clojure
+#+BEGIN_SRC clojure
+#+END_SRC
 #+end_listing
 #+caption:
 #+caption:
 #+caption:
 simulate touch, I use jMonkeyEngine's physics system to execute
 many small collision detections, one for each feeler. The placement
 of the feelers is determined by a UV-mapped image which shows where
 each feeler should be on the 3D surface of the body.
-*** Defining Touch Meta-Data in Blender
+*** COMMENT Defining Touch Meta-Data in Blender
 Each geometry can have a single UV map which describes the
 position of the feelers which will constitute its sense of touch.
 This image path is stored under the ``touch'' key. The image itself
 is black and white, with black meaning a feeler length of 0 (no
 feeler is present) and white meaning a feeler length of =scale=,
 which is a float stored under the key "scale".
-#+name: meta-data
+#+caption: Touch does not use empty nodes, to store metadata,
-#+begin_src clojure
+#+caption: because the metadata of each solid part of a
+#+caption: creature's body is sufficient.
+#+name: touch-meta-data
+#+begin_listing clojure
 (defn tactile-sensor-profile
 "Return the touch-sensor distribution image in BufferedImage format,
 or nil if it does not exist."
 [#^Geometry obj]
 (if-let [image-path (meta-data obj "touch")]
 "Return the length of each feeler. Default scale is 0.01
 jMonkeyEngine units."
 [#^Geometry obj]
 (if-let [scale (meta-data obj "scale")]
 scale 0.1))
-#+end_src
+#+end_listing
-Here is an example of a UV-map which specifies the position of touch
+Here is an example of a UV-map which specifies the position of
-sensors along the surface of the upper segment of the worm.
+touch sensors along the surface of the upper segment of a fingertip.
-#+attr_html: width=755
-#+caption: This is the tactile-sensor-profile for the upper segment of the worm. It defines regions of high touch sensitivity (where there are many white pixels) and regions of low sensitivity (where white pixels are sparse).
+#+caption: This is the tactile-sensor-profile for the upper segment
+#+caption: of a fingertip. It defines regions of high touch sensitivity
+#+caption: (where there are many white pixels) and regions of low
+#+caption: sensitivity (where white pixels are sparse).
+#+name: fimgertip-UV
+#+ATTR_LaTeX: :width 10cm
 [[../images/finger-UV.png]]
-*** Implementation Summary
+*** COMMENT Implementation Summary
 To simulate touch there are three conceptual steps. For each solid
 object in the creature, you first have to get UV image and scale
 parameter which define the position and length of the feelers.
 Then, you use the triangles which comprise the mesh and the UV
 Extracting the meta-data has already been described. The third
 step, physics collision detection, is handled in =touch-kernel=.
 Translating the positions and orientations of the feelers from the
 UV-map to world-space is itself a three-step process.
 - Find the triangles which make up the mesh in pixel-space and in
-world-space. =triangles= =pixel-triangles=.
+world-space. (=triangles= =pixel-triangles=).
-- Find the coordinates of each feeler in world-space. These are the
+- Find the coordinates of each feeler in world-space. These are
-origins of the feelers. =feeler-origins=.
+the origins of the feelers. (=feeler-origins=).
 - Calculate the normals of the triangles in world space, and add
 them to each of the origins of the feelers. These are the
-normalized coordinates of the tips of the feelers. =feeler-tips=.
+normalized coordinates of the tips of the feelers.
+(=feeler-tips=).
-*** Triangle Math
+*** COMMENT Triangle Math
-The rigid objects which make up a creature have an underlying
-=Geometry=, which is a =Mesh= plus a =Material= and other important
+The rigid objects which make up a creature have an underlying
-data involved with displaying the object.
+=Geometry=, which is a =Mesh= plus a =Material= and other
+important data involved with displaying the object.
-A =Mesh= is composed of =Triangles=, and each =Triangle= has three
-vertices which have coordinates in world space and UV space.
+A =Mesh= is composed of =Triangles=, and each =Triangle= has three
+vertices which have coordinates in world space and UV space.
-Here, =triangles= gets all the world-space triangles which comprise a
-mesh, while =pixel-triangles= gets those same triangles expressed in
+Here, =triangles= gets all the world-space triangles which
-pixel coordinates (which are UV coordinates scaled to fit the height
+comprise a mesh, while =pixel-triangles= gets those same triangles
-and width of the UV image).
+expressed in pixel coordinates (which are UV coordinates scaled to
+fit the height and width of the UV image).
-#+name: triangles-2
-#+begin_src clojure
+#+caption: Programs to extract triangles from a geometry and get
-(in-ns 'cortex.touch)
+#+caption: their verticies in both world and UV-coordinates.
+#+name: get-triangles
+#+begin_listing clojure
 (defn triangle
 "Get the triangle specified by triangle-index from the mesh."
 [#^Geometry geo triangle-index]
 (triangle-seq
 (let [scratch (Triangle.)]
 [#^Mesh mesh triangle-index]
 (let [indices (int-array 3)]
 (.getTriangle mesh triangle-index indices)
 (vec indices)))
 (defn vertex-UV-coord
 "Get the UV-coordinates of the vertex named by vertex-index"
 [#^Mesh mesh vertex-index]
 (let [UV-buffer
 (.getData
 (.getBuffer
 [#^Geometry geo image]
 (let [height (.getHeight image)
 width (.getWidth image)]
 (map (partial pixel-triangle geo image)
 (range (.getTriangleCount (.getMesh geo))))))
-#+end_src
+#+end_listing
 *** The Affine Transform from one Triangle to Another
 =pixel-triangles= gives us the mesh triangles expressed in pixel
-coordinates and =triangles= gives us the mesh triangles expressed in
+coordinates and =triangles= gives us the mesh triangles expressed
-world coordinates. The tactile-sensor-profile gives the position of
+in world coordinates. The tactile-sensor-profile gives the
-each feeler in pixel-space. In order to convert pixel-space
+position of each feeler in pixel-space. In order to convert
-coordinates into world-space coordinates we need something that takes
+pixel-space coordinates into world-space coordinates we need
-coordinates on the surface of one triangle and gives the corresponding
+something that takes coordinates on the surface of one triangle
-coordinates on the surface of another triangle.
+and gives the corresponding coordinates on the surface of another
+triangle.
-Triangles are [[http://mathworld.wolfram.com/AffineTransformation.html ][affine]], which means any triangle can be transformed into
-any other by a combination of translation, scaling, and
+Triangles are [[http://mathworld.wolfram.com/AffineTransformation.html ][affine]], which means any triangle can be transformed
-rotation. The affine transformation from one triangle to another
+into any other by a combination of translation, scaling, and
-is readily computable if the triangle is expressed in terms of a $4x4$
+rotation. The affine transformation from one triangle to another
-matrix.
+is readily computable if the triangle is expressed in terms of a
+$4x4$ matrix.
-\begin{bmatrix}
-x_1 & x_2 & x_3 & n_x \\
+\begin{bmatrix}
-y_1 & y_2 & y_3 & n_y \\
+x_1 & x_2 & x_3 & n_x \\
-z_1 & z_2 & z_3 & n_z \\
+y_1 & y_2 & y_3 & n_y \\
-1 & 1 & 1 & 1
+z_1 & z_2 & z_3 & n_z \\
-\end{bmatrix}
+1 & 1 & 1 & 1
+\end{bmatrix}
-Here, the first three columns of the matrix are the vertices of the
-triangle. The last column is the right-handed unit normal of the
+Here, the first three columns of the matrix are the vertices of
-triangle.
+the triangle. The last column is the right-handed unit normal of
+the triangle.
-With two triangles $T_{1}$ and $T_{2}$ each expressed as a matrix like
-above, the affine transform from $T_{1}$ to $T_{2}$ is
+With two triangles $T_{1}$ and $T_{2}$ each expressed as a matrix
+like above, the affine transform from $T_{1}$ to $T_{2}$ is
-$T_{2}T_{1}^{-1}$
+$T_{2}T_{1}^{-1}$
-The clojure code below recapitulates the formulas above, using
-jMonkeyEngine's =Matrix4f= objects, which can describe any affine
+The clojure code below recapitulates the formulas above, using
-transformation.
+jMonkeyEngine's =Matrix4f= objects, which can describe any affine
+transformation.
-#+name: triangles-3
-#+begin_src clojure
+#+caption: Program to interpert triangles as affine transforms.
-(in-ns 'cortex.touch)
+#+name: triangle-affine
+#+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn triangle->matrix4f
 "Converts the triangle into a 4x4 matrix: The first three columns
 contain the vertices of the triangle; the last contains the unit
 normal of the triangle. The bottom row is filled with 1s."
 [#^Triangle t]
 triangle."
 [#^Triangle tri-1 #^Triangle tri-2]
 (.mult
 (triangle->matrix4f tri-2)
 (.invert (triangle->matrix4f tri-1))))
-#+end_src
+#+END_SRC
+#+end_listing
-*** Triangle Boundaries
+*** COMMENT Triangle Boundaries
 For efficiency's sake I will divide the tactile-profile image into
 small squares which inscribe each pixel-triangle, then extract the
 points which lie inside the triangle and map them to 3D-space using
 =triangle-transform= above. To do this I need a function,
 triangle.
 =inside-triangle?= determines whether a point is inside a triangle
 in 2D pixel-space.
-#+name: triangles-4
+#+caption: Program to efficiently determine point includion
-#+begin_src clojure
+#+caption: in a triangle.
+#+name: in-triangle
+#+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn convex-bounds
 "Returns the smallest square containing the given vertices, as a
 vector of integers [left top width height]."
 [verts]
 (let [xs (map first verts)
 (let [[vert-1 vert-2 vert-3] [(.get1 tri) (.get2 tri) (.get3 tri)]]
 (and
 (same-side? vert-1 vert-2 vert-3 p)
 (same-side? vert-2 vert-3 vert-1 p)
 (same-side? vert-3 vert-1 vert-2 p))))
-#+end_src
+#+END_SRC
+#+end_listing
-*** Feeler Coordinates
+*** COMMENT Feeler Coordinates
-The triangle-related functions above make short work of calculating
-the positions and orientations of each feeler in world-space.
+The triangle-related functions above make short work of
+calculating the positions and orientations of each feeler in
-#+name: sensors
+world-space.
-#+begin_src clojure
-(in-ns 'cortex.touch)
+#+caption: Program to get the coordinates of ``feelers '' in
+#+caption: both world and UV-coordinates.
+#+name: feeler-coordinates
+#+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn feeler-pixel-coords
 "Returns the coordinates of the feelers in pixel space in lists, one
 list for each triangle, ordered in the same way as (triangles) and
 (pixel-triangles)."
 [#^Geometry geo image]
 (pixel-triangles geo image)
 (triangles geo))]
 (map (fn [transform coords]
 (map #(.mult transform (->vector3f %)) coords))
 transforms (feeler-pixel-coords geo image))))
+#+END_SRC
+#+end_listing
+#+caption: Program to get the position of the base and tip of
+#+caption: each ``feeler''
+#+name: feeler-tips
+#+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn feeler-origins
 "The world space coordinates of the root of each feeler."
 [#^Geometry geo image]
 (reduce concat (feeler-world-coords geo image)))
 (mapcat (fn [origins normal]
 (map #(.add % normal) origins))
 world-coords normals)))
 (defn touch-topology
-"touch-topology? is not a function."
 [#^Geometry geo image]
 (collapse (reduce concat (feeler-pixel-coords geo image))))
-#+end_src
+#+END_SRC
-*** Simulated Touch
+#+end_listing
-=touch-kernel= generates functions to be called from within a
+*** COMMENT Simulated Touch
-simulation that perform the necessary physics collisions to collect
-tactile data, and =touch!= recursively applies it to every node in
+Now that the functions to construct feelers are complete,
-the creature.
+=touch-kernel= generates functions to be called from within a
+simulation that perform the necessary physics collisions to
-#+name: kernel
+collect tactile data, and =touch!= recursively applies it to every
-#+begin_src clojure
+node in the creature.
-(in-ns 'cortex.touch)
+#+caption: Efficient program to transform a ray from
+#+caption: one position to another.
+#+name: set-ray
+#+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn set-ray [#^Ray ray #^Matrix4f transform
 #^Vector3f origin #^Vector3f tip]
 ;; Doing everything locally reduces garbage collection by enough to
 ;; be worth it.
 (.mult transform origin (.getOrigin ray))
 (.mult transform tip (.getDirection ray))
 (.subtractLocal (.getDirection ray) (.getOrigin ray))
 (.normalizeLocal (.getDirection ray)))
+#+END_SRC
-(import com.jme3.math.FastMath)
+#+end_listing
+#+caption: This is the core of touch in =CORTEX= each feeler
+#+caption: follows the object it is bound to, reporting any
+#+caption: collisions that may happen.
+#+name: touch-kernel
+#+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn touch-kernel
 "Constructs a function which will return tactile sensory data from
 'geo when called from inside a running simulation"
 [#^Geometry geo]
 (if-let
 ray-reference-tips (feeler-tips geo profile)
 ray-length (tactile-scale geo)
 current-rays (map (fn [_] (Ray.)) ray-reference-origins)
 topology (touch-topology geo profile)
 correction (float (* ray-length -0.2))]
 ;; slight tolerance for very close collisions.
 (dorun
 (map (fn [origin tip]
 (.addLocal origin (.mult (.subtract tip origin)
 correction)))
 (if (> response limit) (float 0.0)
 (+ response correction)))
 (float 0.0)
 limit)))
 limit])))))))))))
+#+END_SRC
+#+end_listing
+Armed with the =touch!= function, =CORTEX= becomes capable of
+giving creatures a sense of touch. A simple test is to create a
+cube that is outfitted with a uniform distrubition of touch
+sensors. It can feel the ground and any balls that it touches.
+#+caption: =CORTEX= interface for creating touch in a simulated
+#+caption: creature.
+#+name: touch
+#+begin_listing clojure
+#+BEGIN_SRC clojure
 (defn touch!
 "Endow the creature with the sense of touch. Returns a sequence of
 functions, one for each body part with a tactile-sensor-profile,
 each of which when called returns sensory data for that body part."
 [#^Node creature]
 (filter
 (comp not nil?)
 (map touch-kernel
 (filter #(isa? (class %) Geometry)
 (node-seq creature)))))
-#+end_src
+#+END_SRC
+#+end_listing
-Armed with the =touch!= function, =CORTEX= becomes capable of giving
+The tactile-sensor-profile image for the touch cube is a simple
-creatures a sense of touch. A simple test is to create a cube that is
+cross with a unifom distribution of touch sensors:
-outfitted with a uniform distrubition of touch sensors. It can feel
-the ground and any balls that it touches.
+#+caption: The touch profile for the touch-cube. Each pure white
+#+caption: pixel defines a touch sensitive feeler.
-# insert touch cube image; UV map
+#+name: touch-cube-uv-map
-# insert video
+#+ATTR_LaTeX: :width 10cm
+[[./images/touch-profile.png]]
+#+caption: The touch cube reacts to canonballs. The black, red,
+#+caption: and white cross on the right is a visual display of
+#+caption: the creature's touch. White means that it is feeling
+#+caption: something strongly, black is not feeling anything,
+#+caption: and gray is in-between. The cube can feel both the
+#+caption: floor and the ball. Notice that when the ball causes
+#+caption: the cube to tip, that the bottom face can still feel
+#+caption: part of the ground.
+#+name: touch-cube-uv-map
+#+ATTR_LaTeX: :width 15cm
+[[./images/touch-cube.png]]
 ** Proprioception is the sense that makes everything ``real''
 ** Muscles are both effectors and sensors

Mercurial > cortex

comparison thesis/cortex.org @ 475:3ec428e096e5