# HG changeset patch # User Robert McIntyre # Date 1329012819 25200 # Node ID 6f1be9525e4059f78d7b70c2dea1b8b5e66674a1 # Parent 0589c35f04f23ab4e32692cff648831ae5f8a75f fleshing out text in touch.org diff -r 0589c35f04f2 -r 6f1be9525e40 org/touch.org --- a/org/touch.org Sat Feb 11 18:42:27 2012 -0700 +++ b/org/touch.org Sat Feb 11 19:13:39 2012 -0700 @@ -6,6 +6,8 @@ #+SETUPFILE: ../../aurellem/org/setup.org #+INCLUDE: ../../aurellem/org/level-0.org + + * Touch Touch is critical to navigation and spatial reasoning and as such I @@ -35,59 +37,47 @@ than vision and hearing. Those two senses piggybacked off jMonkeyEngine's 3D audio and video rendering subsystems. To simulate Touch, I use jMonkeyEngine's physics system to execute many small -collision detections, one for each "hair". +collision detections, one for each "hair". The placement of the +"hairs" is determined by a UV-mapped image which shows where each hair +should be on the 3D surface of the body. -* Sensor Related Functions + +* Defining Touch Meta-Data in Blender + +Each geometry can have a single UV map which describes the position +and length of the "hairs" which will constitute its sense of +touch. This image path is stored under the "touch" key. The image +itself is grayscale, with black meaning a hair length of 0 (no hair is +present) and white meaning a hair length of =scale=, which is a float +stored under the key "scale". If the pixel is gray then the resultant +hair length is linearly interpolated between 0 and =scale=. + #+begin_src clojure -(defn sensors-in-triangle - "Locate the touch sensors in the triangle, returning a map of their - UV and geometry-relative coordinates." - [image mesh tri-index] - (let [width (.getWidth image) - height (.getHeight image) - UV-vertex-coords (triangle-UV-coord mesh width height tri-index) - bounds (convex-bounds UV-vertex-coords) - - cutout-triangle (points->triangle UV-vertex-coords) - UV-sensor-coords - (filter (comp (partial inside-triangle? cutout-triangle) - (fn [[u v]] (Vector3f. u v 0))) - (white-coordinates image bounds)) - UV->geometry (triangle-transformation - cutout-triangle - (mesh-triangle mesh tri-index)) - geometry-sensor-coords - (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0))) - UV-sensor-coords)] - {:UV UV-sensor-coords :geometry geometry-sensor-coords})) - -(defn-memo locate-feelers - "Search the geometry's tactile UV profile for touch sensors, - returning their positions in geometry-relative coordinates." - [#^Geometry geo] - (let [mesh (.getMesh geo) - num-triangles (.getTriangleCount mesh)] - (if-let [image (tactile-sensor-profile geo)] - (map - (partial sensors-in-triangle image mesh) - (range num-triangles)) - (repeat (.getTriangleCount mesh) {:UV nil :geometry nil})))) - -(defn-memo touch-topology - "Return a sequence of vectors of the form [x y] describing the - \"topology\" of the tactile sensors. Points that are close together - in the touch-topology are generally close together in the simulation." - [#^Gemoetry geo] - (vec (collapse (reduce concat (map :UV (locate-feelers geo)))))) - -(defn-memo feeler-coordinates - "The location of the touch sensors in world-space coordinates." - [#^Geometry geo] - (vec (map :geometry (locate-feelers geo)))) +(defn tactile-sensor-profile + "Return the touch-sensor distribution image in BufferedImage format, + or nil if it does not exist." + [#^Geometry obj] + (if-let [image-path (meta-data obj "touch")] + (load-image image-path))) #+end_src + +** TODO add image showing example touch-uv map +** TODO add metadata display for worm + * Triangle Manipulation Functions +The rigid bodies which make up a creature have an underlying +=Geometry=, which is a =Mesh= plus a =Material= and other important +data involved with displaying the body. + +A =Mesh= is composed of =Triangles=, and each =Triangle= has three +verticies which have coordinates in XYZ space and UV space. + +Here, =(triangles)= gets all the triangles which compose a mesh, and +=(triangle-UV-coord)= returns the the UV coordinates of the verticies +of a triangle. + #+begin_src clojure (defn triangles "Return a sequence of all the Triangles which compose a given @@ -141,6 +131,12 @@ #+end_src * Schrapnel Conversion Functions + +It is convienent to treat a =Triangle= as a sequence of verticies, and +a =Vector2f= and =Vector3f= as a sequence of floats. These conversion +functions make this easy. If these classes implemented =Iterable= then +this code would not be necessary. Hopefully they will in the future. + #+begin_src clojure (defn triangle-seq [#^Triangle tri] [(.get1 tri) (.get2 tri) (.get3 tri)]) @@ -174,6 +170,13 @@ * Triangle Affine Transforms +The position of each hair is stored in a 2D image in UV +coordinates. To place the hair in 3D space we must convert from UV +coordinates to XYZ coordinates. Each =Triangle= has coordinates in +both UV-space and XYZ-space, which defines a unique [[http://mathworld.wolfram.com/AffineTransformation.html ][Affine Transform]] +for translating any coordinate within the UV triangle to the +cooresponding coordinate in the XYZ triangle. + #+begin_src clojure (defn triangle->matrix4f "Converts the triangle into a 4x4 matrix: The first three columns @@ -202,15 +205,103 @@ (.invert (triangle->matrix4f tri-1)))) #+end_src -* Blender Meta-Data +* Triangle Boundaries + +For efficiency's sake I will divide the UV-image into small squares +which inscribe each UV-triangle, then extract the points which lie +inside the triangle and map them to 3D-space using +=(triangle-transform)= above. To do this I need a function, +=(inside-triangle?)=, which determines whether a point is inside a +triangle in 2D UV-space. #+begin_src clojure -(defn tactile-sensor-profile - "Return the touch-sensor distribution image in BufferedImage format, - or nil if it does not exist." - [#^Geometry obj] - (if-let [image-path (meta-data obj "touch")] - (load-image image-path))) +(defn convex-bounds + "Returns the smallest square containing the given vertices, as a + vector of integers [left top width height]." + [uv-verts] + (let [xs (map first uv-verts) + ys (map second uv-verts) + x0 (Math/floor (apply min xs)) + y0 (Math/floor (apply min ys)) + x1 (Math/ceil (apply max xs)) + y1 (Math/ceil (apply max ys))] + [x0 y0 (- x1 x0) (- y1 y0)])) + +(defn same-side? + "Given the points p1 and p2 and the reference point ref, is point p + on the same side of the line that goes through p1 and p2 as ref is?" + [p1 p2 ref p] + (<= + 0 + (.dot + (.cross (.subtract p2 p1) (.subtract p p1)) + (.cross (.subtract p2 p1) (.subtract ref p1))))) + +(defn inside-triangle? + "Is the point inside the triangle?" + {:author "Dylan Holmes"} + [#^Triangle tri #^Vector3f p] + (let [[vert-1 vert-2 vert-3] (triangle-seq tri)] + (and + (same-side? vert-1 vert-2 vert-3 p) + (same-side? vert-2 vert-3 vert-1 p) + (same-side? vert-3 vert-1 vert-2 p)))) +#+end_src + + + +* Sensor Related Functions + +These functions analyze the touch-sensor-profile image convert the +location of each touch sensor from pixel coordinates to UV-coordinates +and XYZ-coordinates. + +#+begin_src clojure +(defn sensors-in-triangle + "Locate the touch sensors in the triangle, returning a map of their + UV and geometry-relative coordinates." + [image mesh tri-index] + (let [width (.getWidth image) + height (.getHeight image) + UV-vertex-coords (triangle-UV-coord mesh width height tri-index) + bounds (convex-bounds UV-vertex-coords) + + cutout-triangle (points->triangle UV-vertex-coords) + UV-sensor-coords + (filter (comp (partial inside-triangle? cutout-triangle) + (fn [[u v]] (Vector3f. u v 0))) + (white-coordinates image bounds)) + UV->geometry (triangle-transformation + cutout-triangle + (mesh-triangle mesh tri-index)) + geometry-sensor-coords + (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0))) + UV-sensor-coords)] + {:UV UV-sensor-coords :geometry geometry-sensor-coords})) + +(defn-memo locate-feelers + "Search the geometry's tactile UV profile for touch sensors, + returning their positions in geometry-relative coordinates." + [#^Geometry geo] + (let [mesh (.getMesh geo) + num-triangles (.getTriangleCount mesh)] + (if-let [image (tactile-sensor-profile geo)] + (map + (partial sensors-in-triangle image mesh) + (range num-triangles)) + (repeat (.getTriangleCount mesh) {:UV nil :geometry nil})))) + +(defn-memo touch-topology + "Return a sequence of vectors of the form [x y] describing the + \"topology\" of the tactile sensors. Points that are close together + in the touch-topology are generally close together in the simulation." + [#^Gemoetry geo] + (vec (collapse (reduce concat (map :UV (locate-feelers geo)))))) + +(defn-memo feeler-coordinates + "The location of the touch sensors in world-space coordinates." + [#^Geometry geo] + (vec (map :geometry (locate-feelers geo)))) #+end_src * Physics Collision Objects @@ -238,40 +329,6 @@ (.subtract n+c (get-ray-origin geom tri)))) #+end_src -* Triangle Boundaries -#+begin_src clojure -(defn same-side? - "Given the points p1 and p2 and the reference point ref, is point p - on the same side of the line that goes through p1 and p2 as ref is?" - [p1 p2 ref p] - (<= - 0 - (.dot - (.cross (.subtract p2 p1) (.subtract p p1)) - (.cross (.subtract p2 p1) (.subtract ref p1))))) - -(defn inside-triangle? - "Is the point inside the triangle?" - {:author "Dylan Holmes"} - [#^Triangle tri #^Vector3f p] - (let [[vert-1 vert-2 vert-3] (triangle-seq tri)] - (and - (same-side? vert-1 vert-2 vert-3 p) - (same-side? vert-2 vert-3 vert-1 p) - (same-side? vert-3 vert-1 vert-2 p)))) - -(defn convex-bounds - "Returns the smallest square containing the given vertices, as a - vector of integers [left top width height]." - [uv-verts] - (let [xs (map first uv-verts) - ys (map second uv-verts) - x0 (Math/floor (apply min xs)) - y0 (Math/floor (apply min ys)) - x1 (Math/ceil (apply max xs)) - y1 (Math/ceil (apply max ys))] - [x0 y0 (- x1 x0) (- y1 y0)])) -#+end_src * Skin Creation @@ -366,8 +423,6 @@ (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f))) #+end_src -;; Every Mesh has many triangles, each with its own index. -;; Every vertex has its own index as well. * Source Listing * Next