Mercurial > cortex
diff org/touch.org @ 229:6f1be9525e40
fleshing out text in touch.org
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Sat, 11 Feb 2012 19:13:39 -0700 |
parents | 0589c35f04f2 |
children | f9b7d674aed8 |
line wrap: on
line diff
1.1 --- a/org/touch.org Sat Feb 11 18:42:27 2012 -0700 1.2 +++ b/org/touch.org Sat Feb 11 19:13:39 2012 -0700 1.3 @@ -6,6 +6,8 @@ 1.4 #+SETUPFILE: ../../aurellem/org/setup.org 1.5 #+INCLUDE: ../../aurellem/org/level-0.org 1.6 1.7 + 1.8 + 1.9 * Touch 1.10 1.11 Touch is critical to navigation and spatial reasoning and as such I 1.12 @@ -35,59 +37,47 @@ 1.13 than vision and hearing. Those two senses piggybacked off 1.14 jMonkeyEngine's 3D audio and video rendering subsystems. To simulate 1.15 Touch, I use jMonkeyEngine's physics system to execute many small 1.16 -collision detections, one for each "hair". 1.17 +collision detections, one for each "hair". The placement of the 1.18 +"hairs" is determined by a UV-mapped image which shows where each hair 1.19 +should be on the 3D surface of the body. 1.20 1.21 -* Sensor Related Functions 1.22 + 1.23 +* Defining Touch Meta-Data in Blender 1.24 + 1.25 +Each geometry can have a single UV map which describes the position 1.26 +and length of the "hairs" which will constitute its sense of 1.27 +touch. This image path is stored under the "touch" key. The image 1.28 +itself is grayscale, with black meaning a hair length of 0 (no hair is 1.29 +present) and white meaning a hair length of =scale=, which is a float 1.30 +stored under the key "scale". If the pixel is gray then the resultant 1.31 +hair length is linearly interpolated between 0 and =scale=. 1.32 + 1.33 #+begin_src clojure 1.34 -(defn sensors-in-triangle 1.35 - "Locate the touch sensors in the triangle, returning a map of their 1.36 - UV and geometry-relative coordinates." 1.37 - [image mesh tri-index] 1.38 - (let [width (.getWidth image) 1.39 - height (.getHeight image) 1.40 - UV-vertex-coords (triangle-UV-coord mesh width height tri-index) 1.41 - bounds (convex-bounds UV-vertex-coords) 1.42 - 1.43 - cutout-triangle (points->triangle UV-vertex-coords) 1.44 - UV-sensor-coords 1.45 - (filter (comp (partial inside-triangle? cutout-triangle) 1.46 - (fn [[u v]] (Vector3f. u v 0))) 1.47 - (white-coordinates image bounds)) 1.48 - UV->geometry (triangle-transformation 1.49 - cutout-triangle 1.50 - (mesh-triangle mesh tri-index)) 1.51 - geometry-sensor-coords 1.52 - (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0))) 1.53 - UV-sensor-coords)] 1.54 - {:UV UV-sensor-coords :geometry geometry-sensor-coords})) 1.55 - 1.56 -(defn-memo locate-feelers 1.57 - "Search the geometry's tactile UV profile for touch sensors, 1.58 - returning their positions in geometry-relative coordinates." 1.59 - [#^Geometry geo] 1.60 - (let [mesh (.getMesh geo) 1.61 - num-triangles (.getTriangleCount mesh)] 1.62 - (if-let [image (tactile-sensor-profile geo)] 1.63 - (map 1.64 - (partial sensors-in-triangle image mesh) 1.65 - (range num-triangles)) 1.66 - (repeat (.getTriangleCount mesh) {:UV nil :geometry nil})))) 1.67 - 1.68 -(defn-memo touch-topology 1.69 - "Return a sequence of vectors of the form [x y] describing the 1.70 - \"topology\" of the tactile sensors. Points that are close together 1.71 - in the touch-topology are generally close together in the simulation." 1.72 - [#^Gemoetry geo] 1.73 - (vec (collapse (reduce concat (map :UV (locate-feelers geo)))))) 1.74 - 1.75 -(defn-memo feeler-coordinates 1.76 - "The location of the touch sensors in world-space coordinates." 1.77 - [#^Geometry geo] 1.78 - (vec (map :geometry (locate-feelers geo)))) 1.79 +(defn tactile-sensor-profile 1.80 + "Return the touch-sensor distribution image in BufferedImage format, 1.81 + or nil if it does not exist." 1.82 + [#^Geometry obj] 1.83 + (if-let [image-path (meta-data obj "touch")] 1.84 + (load-image image-path))) 1.85 #+end_src 1.86 1.87 + 1.88 +** TODO add image showing example touch-uv map 1.89 +** TODO add metadata display for worm 1.90 + 1.91 * Triangle Manipulation Functions 1.92 1.93 +The rigid bodies which make up a creature have an underlying 1.94 +=Geometry=, which is a =Mesh= plus a =Material= and other important 1.95 +data involved with displaying the body. 1.96 + 1.97 +A =Mesh= is composed of =Triangles=, and each =Triangle= has three 1.98 +verticies which have coordinates in XYZ space and UV space. 1.99 + 1.100 +Here, =(triangles)= gets all the triangles which compose a mesh, and 1.101 +=(triangle-UV-coord)= returns the the UV coordinates of the verticies 1.102 +of a triangle. 1.103 + 1.104 #+begin_src clojure 1.105 (defn triangles 1.106 "Return a sequence of all the Triangles which compose a given 1.107 @@ -141,6 +131,12 @@ 1.108 #+end_src 1.109 1.110 * Schrapnel Conversion Functions 1.111 + 1.112 +It is convienent to treat a =Triangle= as a sequence of verticies, and 1.113 +a =Vector2f= and =Vector3f= as a sequence of floats. These conversion 1.114 +functions make this easy. If these classes implemented =Iterable= then 1.115 +this code would not be necessary. Hopefully they will in the future. 1.116 + 1.117 #+begin_src clojure 1.118 (defn triangle-seq [#^Triangle tri] 1.119 [(.get1 tri) (.get2 tri) (.get3 tri)]) 1.120 @@ -174,6 +170,13 @@ 1.121 1.122 * Triangle Affine Transforms 1.123 1.124 +The position of each hair is stored in a 2D image in UV 1.125 +coordinates. To place the hair in 3D space we must convert from UV 1.126 +coordinates to XYZ coordinates. Each =Triangle= has coordinates in 1.127 +both UV-space and XYZ-space, which defines a unique [[http://mathworld.wolfram.com/AffineTransformation.html ][Affine Transform]] 1.128 +for translating any coordinate within the UV triangle to the 1.129 +cooresponding coordinate in the XYZ triangle. 1.130 + 1.131 #+begin_src clojure 1.132 (defn triangle->matrix4f 1.133 "Converts the triangle into a 4x4 matrix: The first three columns 1.134 @@ -202,15 +205,103 @@ 1.135 (.invert (triangle->matrix4f tri-1)))) 1.136 #+end_src 1.137 1.138 -* Blender Meta-Data 1.139 +* Triangle Boundaries 1.140 + 1.141 +For efficiency's sake I will divide the UV-image into small squares 1.142 +which inscribe each UV-triangle, then extract the points which lie 1.143 +inside the triangle and map them to 3D-space using 1.144 +=(triangle-transform)= above. To do this I need a function, 1.145 +=(inside-triangle?)=, which determines whether a point is inside a 1.146 +triangle in 2D UV-space. 1.147 1.148 #+begin_src clojure 1.149 -(defn tactile-sensor-profile 1.150 - "Return the touch-sensor distribution image in BufferedImage format, 1.151 - or nil if it does not exist." 1.152 - [#^Geometry obj] 1.153 - (if-let [image-path (meta-data obj "touch")] 1.154 - (load-image image-path))) 1.155 +(defn convex-bounds 1.156 + "Returns the smallest square containing the given vertices, as a 1.157 + vector of integers [left top width height]." 1.158 + [uv-verts] 1.159 + (let [xs (map first uv-verts) 1.160 + ys (map second uv-verts) 1.161 + x0 (Math/floor (apply min xs)) 1.162 + y0 (Math/floor (apply min ys)) 1.163 + x1 (Math/ceil (apply max xs)) 1.164 + y1 (Math/ceil (apply max ys))] 1.165 + [x0 y0 (- x1 x0) (- y1 y0)])) 1.166 + 1.167 +(defn same-side? 1.168 + "Given the points p1 and p2 and the reference point ref, is point p 1.169 + on the same side of the line that goes through p1 and p2 as ref is?" 1.170 + [p1 p2 ref p] 1.171 + (<= 1.172 + 0 1.173 + (.dot 1.174 + (.cross (.subtract p2 p1) (.subtract p p1)) 1.175 + (.cross (.subtract p2 p1) (.subtract ref p1))))) 1.176 + 1.177 +(defn inside-triangle? 1.178 + "Is the point inside the triangle?" 1.179 + {:author "Dylan Holmes"} 1.180 + [#^Triangle tri #^Vector3f p] 1.181 + (let [[vert-1 vert-2 vert-3] (triangle-seq tri)] 1.182 + (and 1.183 + (same-side? vert-1 vert-2 vert-3 p) 1.184 + (same-side? vert-2 vert-3 vert-1 p) 1.185 + (same-side? vert-3 vert-1 vert-2 p)))) 1.186 +#+end_src 1.187 + 1.188 + 1.189 + 1.190 +* Sensor Related Functions 1.191 + 1.192 +These functions analyze the touch-sensor-profile image convert the 1.193 +location of each touch sensor from pixel coordinates to UV-coordinates 1.194 +and XYZ-coordinates. 1.195 + 1.196 +#+begin_src clojure 1.197 +(defn sensors-in-triangle 1.198 + "Locate the touch sensors in the triangle, returning a map of their 1.199 + UV and geometry-relative coordinates." 1.200 + [image mesh tri-index] 1.201 + (let [width (.getWidth image) 1.202 + height (.getHeight image) 1.203 + UV-vertex-coords (triangle-UV-coord mesh width height tri-index) 1.204 + bounds (convex-bounds UV-vertex-coords) 1.205 + 1.206 + cutout-triangle (points->triangle UV-vertex-coords) 1.207 + UV-sensor-coords 1.208 + (filter (comp (partial inside-triangle? cutout-triangle) 1.209 + (fn [[u v]] (Vector3f. u v 0))) 1.210 + (white-coordinates image bounds)) 1.211 + UV->geometry (triangle-transformation 1.212 + cutout-triangle 1.213 + (mesh-triangle mesh tri-index)) 1.214 + geometry-sensor-coords 1.215 + (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0))) 1.216 + UV-sensor-coords)] 1.217 + {:UV UV-sensor-coords :geometry geometry-sensor-coords})) 1.218 + 1.219 +(defn-memo locate-feelers 1.220 + "Search the geometry's tactile UV profile for touch sensors, 1.221 + returning their positions in geometry-relative coordinates." 1.222 + [#^Geometry geo] 1.223 + (let [mesh (.getMesh geo) 1.224 + num-triangles (.getTriangleCount mesh)] 1.225 + (if-let [image (tactile-sensor-profile geo)] 1.226 + (map 1.227 + (partial sensors-in-triangle image mesh) 1.228 + (range num-triangles)) 1.229 + (repeat (.getTriangleCount mesh) {:UV nil :geometry nil})))) 1.230 + 1.231 +(defn-memo touch-topology 1.232 + "Return a sequence of vectors of the form [x y] describing the 1.233 + \"topology\" of the tactile sensors. Points that are close together 1.234 + in the touch-topology are generally close together in the simulation." 1.235 + [#^Gemoetry geo] 1.236 + (vec (collapse (reduce concat (map :UV (locate-feelers geo)))))) 1.237 + 1.238 +(defn-memo feeler-coordinates 1.239 + "The location of the touch sensors in world-space coordinates." 1.240 + [#^Geometry geo] 1.241 + (vec (map :geometry (locate-feelers geo)))) 1.242 #+end_src 1.243 1.244 * Physics Collision Objects 1.245 @@ -238,40 +329,6 @@ 1.246 (.subtract n+c (get-ray-origin geom tri)))) 1.247 #+end_src 1.248 1.249 -* Triangle Boundaries 1.250 -#+begin_src clojure 1.251 -(defn same-side? 1.252 - "Given the points p1 and p2 and the reference point ref, is point p 1.253 - on the same side of the line that goes through p1 and p2 as ref is?" 1.254 - [p1 p2 ref p] 1.255 - (<= 1.256 - 0 1.257 - (.dot 1.258 - (.cross (.subtract p2 p1) (.subtract p p1)) 1.259 - (.cross (.subtract p2 p1) (.subtract ref p1))))) 1.260 - 1.261 -(defn inside-triangle? 1.262 - "Is the point inside the triangle?" 1.263 - {:author "Dylan Holmes"} 1.264 - [#^Triangle tri #^Vector3f p] 1.265 - (let [[vert-1 vert-2 vert-3] (triangle-seq tri)] 1.266 - (and 1.267 - (same-side? vert-1 vert-2 vert-3 p) 1.268 - (same-side? vert-2 vert-3 vert-1 p) 1.269 - (same-side? vert-3 vert-1 vert-2 p)))) 1.270 - 1.271 -(defn convex-bounds 1.272 - "Returns the smallest square containing the given vertices, as a 1.273 - vector of integers [left top width height]." 1.274 - [uv-verts] 1.275 - (let [xs (map first uv-verts) 1.276 - ys (map second uv-verts) 1.277 - x0 (Math/floor (apply min xs)) 1.278 - y0 (Math/floor (apply min ys)) 1.279 - x1 (Math/ceil (apply max xs)) 1.280 - y1 (Math/ceil (apply max ys))] 1.281 - [x0 y0 (- x1 x0) (- y1 y0)])) 1.282 -#+end_src 1.283 1.284 * Skin Creation 1.285 1.286 @@ -366,8 +423,6 @@ 1.287 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f))) 1.288 #+end_src 1.289 1.290 -;; Every Mesh has many triangles, each with its own index. 1.291 -;; Every vertex has its own index as well. 1.292 1.293 * Source Listing 1.294 * Next