comparison org/touch.org @ 247:4e220c8fb1ed

first pass at rough draft complete
author Robert McIntyre <rlm@mit.edu>
date Sun, 12 Feb 2012 15:46:01 -0700
parents 63da037ce1c5
children 267add63b168
comparison
equal deleted inserted replaced
246:63da037ce1c5 247:4e220c8fb1ed
18 Human skin has a wide array of touch sensors, each of which speciliaze 18 Human skin has a wide array of touch sensors, each of which speciliaze
19 in detecting different vibrational modes and pressures. These sensors 19 in detecting different vibrational modes and pressures. These sensors
20 can integrate a vast expanse of skin (i.e. your entire palm), or a 20 can integrate a vast expanse of skin (i.e. your entire palm), or a
21 tiny patch of skin at the tip of your finger. The hairs of the skin 21 tiny patch of skin at the tip of your finger. The hairs of the skin
22 help detect objects before they even come into contact with the skin 22 help detect objects before they even come into contact with the skin
23 proper. 23 proper.
24 24
25 Instead of measuring deformation or vibration, I surround each rigid 25 Instead of measuring deformation or vibration, I surround each rigid
26 part with a plenitude of hair-like objects which do not interact with 26 part with a plenitude of hair-like objects (/feelers/) which do not
27 the physical world. Physical objects can pass through them with no 27 interact with the physical world. Physical objects can pass through
28 effect. The hairs are able to measure contact with other objects, and 28 them with no effect. The feelers are able to measure contact with
29 constantly report how much of their extent is covered. So, even though 29 other objects, and constantly report how much of their extent is
30 the creature's body parts do not deform, the hairs create a margin 30 covered. So, even though the creature's body parts do not deform, the
31 around those body parts which achieves a sense of touch which is a 31 feelers create a margin around those body parts which achieves a sense
32 hybrid between a human's sense of deformation and sense from hairs. 32 of touch which is a hybrid between a human's sense of deformation and
33 sense from hairs.
33 34
34 Implementing touch in jMonkeyEngine follows a different techinal route 35 Implementing touch in jMonkeyEngine follows a different techinal route
35 than vision and hearing. Those two senses piggybacked off 36 than vision and hearing. Those two senses piggybacked off
36 jMonkeyEngine's 3D audio and video rendering subsystems. To simulate 37 jMonkeyEngine's 3D audio and video rendering subsystems. To simulate
37 Touch, I use jMonkeyEngine's physics system to execute many small 38 touch, I use jMonkeyEngine's physics system to execute many small
38 collision detections, one for each "hair". The placement of the 39 collision detections, one for each feeler. The placement of the
39 "hairs" is determined by a UV-mapped image which shows where each hair 40 feelers is determined by a UV-mapped image which shows where each
40 should be on the 3D surface of the body. 41 feeler should be on the 3D surface of the body.
41 42
42 * Defining Touch Meta-Data in Blender 43 * Defining Touch Meta-Data in Blender
43 44
44 Each geometry can have a single UV map which describes the position of 45 Each geometry can have a single UV map which describes the position of
45 the "hairs" which will constitute its sense of touch. This image path 46 the feelers which will constitute its sense of touch. This image path
46 is stored under the "touch" key. The image itself is black and white, 47 is stored under the "touch" key. The image itself is black and white,
47 with black meaning a hair length of 0 (no hair is present) and white 48 with black meaning a feeler length of 0 (no feeler is present) and
48 meaning a hair length of =scale=, which is a float stored under the 49 white meaning a feeler length of =scale=, which is a float stored
49 key "scale". I call these "hairs" /feelers/. 50 under the key "scale".
50 51
51 #+name: meta-data 52 #+name: meta-data
52 #+begin_src clojure 53 #+begin_src clojure
53 (defn tactile-sensor-profile 54 (defn tactile-sensor-profile
54 "Return the touch-sensor distribution image in BufferedImage format, 55 "Return the touch-sensor distribution image in BufferedImage format,
56 [#^Geometry obj] 57 [#^Geometry obj]
57 (if-let [image-path (meta-data obj "touch")] 58 (if-let [image-path (meta-data obj "touch")]
58 (load-image image-path))) 59 (load-image image-path)))
59 60
60 (defn tactile-scale 61 (defn tactile-scale
61 "Return the maximum length of a hair. All hairs are scalled between 62 "Return the length of each feeler. Default scale is 0.01
62 0.0 and this length, depending on their color. Black is 0, and 63 jMonkeyEngine units."
63 white is maximum length, and everything in between is scalled
64 linearlly. Default scale is 0.01 jMonkeyEngine units."
65 [#^Geometry obj] 64 [#^Geometry obj]
66 (if-let [scale (meta-data obj "scale")] 65 (if-let [scale (meta-data obj "scale")]
67 scale 0.1)) 66 scale 0.1))
68 #+end_src 67 #+end_src
69 68
70 Here is an example of a UV-map which specifies the position of touch 69 Here is an example of a UV-map which specifies the position of touch
71 sensors along the surface of the worm. 70 sensors along the surface of the upper segment of the worm.
72 71
73 #+attr_html: width=755 72 #+attr_html: width=755
74 #+caption: This is the tactile-sensor-profile for the upper segment of the worm. It defines regions of high touch sensitivity (where there are many white pixels) and regions of low sensitivity (where white pixels are sparse). 73 #+caption: This is the tactile-sensor-profile for the upper segment of the worm. It defines regions of high touch sensitivity (where there are many white pixels) and regions of low sensitivity (where white pixels are sparse).
75 [[../images/finger-UV.png]] 74 [[../images/finger-UV.png]]
76 75
77 * Skin Creation 76 * Implementation Summary
78 77
79 =(touch-kernel)= generates the functions which implement the sense of 78 To simulate touch there are three conceptual steps. For each solid
80 touch for a creature. These functions must do 6 things to obtain touch 79 object in the creature, you first have to get UV image and scale
81 data. 80 paramater which define the position and length of the feelers. Then,
82 81 you use the triangles which compose the mesh and the UV data stored in
83 - Get the tactile profile image and scale paramaters which describe 82 the mesh to determine the world-space position and orientation of each
84 the layout of feelers along the object's surface. 83 feeler. Once every frame, update these positions and orientations to
85 =(tactile-sensor-profile)=, =(tactile-scale)= 84 match the current position and orientation of the object, and use
85 physics collision detection to gather tactile data.
86
87 Extracting the meta-data has already been described. The third step,
88 physics collision detection, is handled in =(touch-kernel)=.
89 Translating the positions and orientations of the feelers from the
90 UV-map to world-space is also a three-step process.
86 91
87 - Find the triangles which make up the mesh in pixel-space and in 92 - Find the triangles which make up the mesh in pixel-space and in
88 world-space. 93 world-space. =(triangles)= =(pixel-triangles)=.
89 =(triangles)= =(pixel-triangles)= 94
90 95 - Find the coordinates of each feeler in world-space. These are the
91 - Find the coordinates of each pixel in pixel space. These 96 origins of the feelers. =(feeler-origins)=.
92 coordinates are used to make the touch-topology.
93 =(feeler-pixel-coords)=
94
95 - Find the coordinates of each pixel in world-space. These
96 coordinates are the origins of the feelers. =(feeler-origins)=
97 97
98 - Calculate the normals of the triangles in world space, and add 98 - Calculate the normals of the triangles in world space, and add
99 them to each of the origins of the feelers. These are the 99 them to each of the origins of the feelers. These are the
100 normalized coordinates of the tips of the feelers. 100 normalized coordinates of the tips of the feelers. =(feeler-tips)=.
101 For both of these, =(feeler-tips)= 101
102 102 * Triangle Math
103 - Generate some sort of topology for the sensors. 103 ** Schrapnel Conversion Functions
104 =(touch-topology)= 104
105 105 #+name: triangles-1
106 #+begin_src clojure
107 (defn vector3f-seq [#^Vector3f v]
108 [(.getX v) (.getY v) (.getZ v)])
109
110 (defn triangle-seq [#^Triangle tri]
111 [(vector3f-seq (.get1 tri))
112 (vector3f-seq (.get2 tri))
113 (vector3f-seq (.get3 tri))])
114
115 (defn ->vector3f
116 ([coords] (Vector3f. (nth coords 0 0)
117 (nth coords 1 0)
118 (nth coords 2 0))))
119
120 (defn ->triangle [points]
121 (apply #(Triangle. %1 %2 %3) (map ->vector3f points)))
122 #+end_src
123
124 It is convienent to treat a =Triangle= as a sequence of verticies, and
125 a =Vector2f= and =Vector3f= as a sequence of floats. These conversion
126 functions make this easy. If these classes implemented =Iterable= then
127 =(seq)= would work on them automitacally.
128 ** Decomposing a 3D shape into Triangles
129
130 The rigid bodies which make up a creature have an underlying
131 =Geometry=, which is a =Mesh= plus a =Material= and other important
132 data involved with displaying the body.
133
134 A =Mesh= is composed of =Triangles=, and each =Triangle= has three
135 verticies which have coordinates in world space and UV space.
136
137 Here, =(triangles)= gets all the world-space triangles which compose a
138 mesh, while =(pixel-triangles)= gets those same triangles expressed in
139 pixel coordinates (which are UV coordinates scaled to fit the height
140 and width of the UV image).
141
142 #+name: triangles-2
143 #+begin_src clojure
144 (in-ns 'cortex.touch)
145 (defn triangle
146 "Get the triangle specified by triangle-index from the mesh."
147 [#^Geometry geo triangle-index]
148 (triangle-seq
149 (let [scratch (Triangle.)]
150 (.getTriangle (.getMesh geo) triangle-index scratch) scratch)))
151
152 (defn triangles
153 "Return a sequence of all the Triangles which compose a given
154 Geometry."
155 [#^Geometry geo]
156 (map (partial triangle geo) (range (.getTriangleCount (.getMesh geo)))))
157
158 (defn triangle-vertex-indices
159 "Get the triangle vertex indices of a given triangle from a given
160 mesh."
161 [#^Mesh mesh triangle-index]
162 (let [indices (int-array 3)]
163 (.getTriangle mesh triangle-index indices)
164 (vec indices)))
165
166 (defn vertex-UV-coord
167 "Get the UV-coordinates of the vertex named by vertex-index"
168 [#^Mesh mesh vertex-index]
169 (let [UV-buffer
170 (.getData
171 (.getBuffer
172 mesh
173 VertexBuffer$Type/TexCoord))]
174 [(.get UV-buffer (* vertex-index 2))
175 (.get UV-buffer (+ 1 (* vertex-index 2)))]))
176
177 (defn pixel-triangle [#^Geometry geo image index]
178 (let [mesh (.getMesh geo)
179 width (.getWidth image)
180 height (.getHeight image)]
181 (vec (map (fn [[u v]] (vector (* width u) (* height v)))
182 (map (partial vertex-UV-coord mesh)
183 (triangle-vertex-indices mesh index))))))
184
185 (defn pixel-triangles [#^Geometry geo image]
186 (let [height (.getHeight image)
187 width (.getWidth image)]
188 (map (partial pixel-triangle geo image)
189 (range (.getTriangleCount (.getMesh geo))))))
190 #+end_src
191 ** The Affine Transform from one Triangle to Another
192
193 =(pixel-triangles)= gives us the mesh triangles expressed in pixel
194 coordinates and =(triangles)= gives us the mesh triangles expressed in
195 world coordinates. The tactile-sensor-profile gives the position of
196 each feeler in pixel-space. In order to convert pixel-dpace
197 coordinates into world-space coordinates we need something that takes
198 coordinates on the surface of one triangle and gives the corresponding
199 coordinates on the surface of another triangle.
200
201 Triangles are [[http://mathworld.wolfram.com/AffineTransformation.html ][affine]], which means any triangle can be transformed into
202 any other by a combination of translation, scaling, and
203 rotation. jMonkeyEngine's =Matrix4f= objects can describe any affine
204 transformation. The affine transformation from one triangle to another
205 is readily computable if the triangle is expressed in terms of a $4x4$
206 matrix.
207
208 \begin{bmatrix}
209 x_1 & x_2 & x_3 & n_x \\
210 y_1 & y_2 & y_3 & n_y \\
211 z_1 & z_2 & z_3 & n_z \\
212 1 & 1 & 1 & 1
213 \end{bmatrix}
214
215 Here, the first three columns of the matrix are the verticies of the
216 triangle. The last column is the right-handed unit normal of the
217 triangle.
218
219 With two triangles $T_{1}$ and $T_{2}$ each expressed as a matrix like
220 above, the affine transform from $T_{1}$ to $T_{2}$ is
221
222 $T_{2}T_{1}^{-1}$
223
224 The clojure code below recaptiulates the formulas above.
225
226 #+name: triangles-3
227 #+begin_src clojure
228 (in-ns 'cortex.touch)
229
230 (defn triangle->matrix4f
231 "Converts the triangle into a 4x4 matrix: The first three columns
232 contain the vertices of the triangle; the last contains the unit
233 normal of the triangle. The bottom row is filled with 1s."
234 [#^Triangle t]
235 (let [mat (Matrix4f.)
236 [vert-1 vert-2 vert-3]
237 ((comp vec map) #(.get t %) (range 3))
238 unit-normal (do (.calculateNormal t)(.getNormal t))
239 vertices [vert-1 vert-2 vert-3 unit-normal]]
240 (dorun
241 (for [row (range 4) col (range 3)]
242 (do
243 (.set mat col row (.get (vertices row) col))
244 (.set mat 3 row 1)))) mat))
245
246 (defn triangles->affine-transform
247 "Returns the affine transformation that converts each vertex in the
248 first triangle into the corresponding vertex in the second
249 triangle."
250 [#^Triangle tri-1 #^Triangle tri-2]
251 (.mult
252 (triangle->matrix4f tri-2)
253 (.invert (triangle->matrix4f tri-1))))
254 #+end_src
255 ** Triangle Boundaries
256
257 For efficiency's sake I will divide the tactile-profile image into
258 small squares which inscribe each pixel-triangle, then extract the
259 points which lie inside the triangle and map them to 3D-space using
260 =(triangle-transform)= above. To do this I need a function,
261 =(convex-bounds)= which finds the smallest box which inscribes a 2D
262 triangle.
263
264 =(inside-triangle?)= determines whether a point is inside a triangle
265 in 2D pixel-space.
266
267 #+name: triangles-4
268 #+begin_src clojure
269 (defn convex-bounds
270 "Returns the smallest square containing the given vertices, as a
271 vector of integers [left top width height]."
272 [verts]
273 (let [xs (map first verts)
274 ys (map second verts)
275 x0 (Math/floor (apply min xs))
276 y0 (Math/floor (apply min ys))
277 x1 (Math/ceil (apply max xs))
278 y1 (Math/ceil (apply max ys))]
279 [x0 y0 (- x1 x0) (- y1 y0)]))
280
281 (defn same-side?
282 "Given the points p1 and p2 and the reference point ref, is point p
283 on the same side of the line that goes through p1 and p2 as ref is?"
284 [p1 p2 ref p]
285 (<=
286 0
287 (.dot
288 (.cross (.subtract p2 p1) (.subtract p p1))
289 (.cross (.subtract p2 p1) (.subtract ref p1)))))
290
291 (defn inside-triangle?
292 "Is the point inside the triangle?"
293 {:author "Dylan Holmes"}
294 [#^Triangle tri #^Vector3f p]
295 (let [[vert-1 vert-2 vert-3] [(.get1 tri) (.get2 tri) (.get3 tri)]]
296 (and
297 (same-side? vert-1 vert-2 vert-3 p)
298 (same-side? vert-2 vert-3 vert-1 p)
299 (same-side? vert-3 vert-1 vert-2 p))))
300 #+end_src
301
302 * Feeler Coordinates
303
304 The triangle-related functions above make short work of calculating
305 the positions and orientations of each feeler in world-space.
306
307 #+name: sensors
308 #+begin_src clojure
309 (in-ns 'cortex.touch)
310
311 (defn feeler-pixel-coords
312 "Returns the coordinates of the feelers in pixel space in lists, one
313 list for each triangle, ordered in the same way as (triangles) and
314 (pixel-triangles)."
315 [#^Geometry geo image]
316 (map
317 (fn [pixel-triangle]
318 (filter
319 (fn [coord]
320 (inside-triangle? (->triangle pixel-triangle)
321 (->vector3f coord)))
322 (white-coordinates image (convex-bounds pixel-triangle))))
323 (pixel-triangles geo image)))
324
325 (defn feeler-world-coords
326 "Returns the coordinates of the feelers in world space in lists, one
327 list for each triangle, ordered in the same way as (triangles) and
328 (pixel-triangles)."
329 [#^Geometry geo image]
330 (let [transforms
331 (map #(triangles->affine-transform
332 (->triangle %1) (->triangle %2))
333 (pixel-triangles geo image)
334 (triangles geo))]
335 (map (fn [transform coords]
336 (map #(.mult transform (->vector3f %)) coords))
337 transforms (feeler-pixel-coords geo image))))
338
339 (defn feeler-origins
340 "The world space coordinates of the root of each feeler."
341 [#^Geometry geo image]
342 (reduce concat (feeler-world-coords geo image)))
343
344 (defn feeler-tips
345 "The world space coordinates of the tip of each feeler."
346 [#^Geometry geo image]
347 (let [world-coords (feeler-world-coords geo image)
348 normals
349 (map
350 (fn [triangle]
351 (.calculateNormal triangle)
352 (.clone (.getNormal triangle)))
353 (map ->triangle (triangles geo)))]
354
355 (mapcat (fn [origins normal]
356 (map #(.add % normal) origins))
357 world-coords normals)))
358
359 (defn touch-topology
360 "touch-topology? is not a function."
361 [#^Geometry geo image]
362 (collapse (reduce concat (feeler-pixel-coords geo image))))
363 #+end_src
364 * Simulated Touch
365
366 =(touch-kernel)= generates functions to be called from within a
367 simulation that perform the necessary physics collisions to collect
368 tactile data, and =(touch!)= recursively applies it to every node in
369 the creature.
106 370
107 #+name: kernel 371 #+name: kernel
108 #+begin_src clojure 372 #+begin_src clojure
109 (in-ns 'cortex.touch) 373 (in-ns 'cortex.touch)
110 374
161 (map touch-kernel 425 (map touch-kernel
162 (filter #(isa? (class %) Geometry) 426 (filter #(isa? (class %) Geometry)
163 (node-seq creature))))) 427 (node-seq creature)))))
164 #+end_src 428 #+end_src
165 429
166 * Sensor Related Functions
167
168 These functions analyze the touch-sensor-profile image convert the
169 location of each touch sensor from pixel coordinates to UV-coordinates
170 and XYZ-coordinates.
171
172 #+name: sensors
173 #+begin_src clojure
174 (in-ns 'cortex.touch)
175
176 (defn feeler-pixel-coords
177 "Returns the coordinates of the feelers in pixel space in lists, one
178 list for each triangle, ordered in the same way as (triangles) and
179 (pixel-triangles)."
180 [#^Geometry geo image]
181 (map
182 (fn [pixel-triangle]
183 (filter
184 (fn [coord]
185 (inside-triangle? (->triangle pixel-triangle)
186 (->vector3f coord)))
187 (white-coordinates image (convex-bounds pixel-triangle))))
188 (pixel-triangles geo image)))
189
190 (defn feeler-world-coords [#^Geometry geo image]
191 (let [transforms
192 (map #(triangles->affine-transform
193 (->triangle %1) (->triangle %2))
194 (pixel-triangles geo image)
195 (triangles geo))]
196 (map (fn [transform coords]
197 (map #(.mult transform (->vector3f %)) coords))
198 transforms (feeler-pixel-coords geo image))))
199
200 (defn feeler-origins [#^Geometry geo image]
201 (reduce concat (feeler-world-coords geo image)))
202
203 (defn feeler-tips [#^Geometry geo image]
204 (let [world-coords (feeler-world-coords geo image)
205 normals
206 (map
207 (fn [triangle]
208 (.calculateNormal triangle)
209 (.clone (.getNormal triangle)))
210 (map ->triangle (triangles geo)))]
211
212 (mapcat (fn [origins normal]
213 (map #(.add % normal) origins))
214 world-coords normals)))
215
216 (defn touch-topology [#^Geometry geo image]
217 (collapse (reduce concat (feeler-pixel-coords geo image))))
218 #+end_src
219
220 * Visualizing Touch 430 * Visualizing Touch
431
221 #+name: visualization 432 #+name: visualization
222 #+begin_src clojure 433 #+begin_src clojure
223 (in-ns 'cortex.touch) 434 (in-ns 'cortex.touch)
224 435
225 (defn touch->gray 436 (defn touch->gray
237 (dorun 448 (dorun
238 (for [i (range (count coords))] 449 (for [i (range (count coords))]
239 (.setRGB image ((coords i) 0) ((coords i) 1) 450 (.setRGB image ((coords i) 0) ((coords i) 1)
240 (apply touch->gray (sensor-data i))))) image)))) 451 (apply touch->gray (sensor-data i))))) image))))
241 #+end_src 452 #+end_src
242 453 * Adding Touch to the Worm
243 454
244 455 #+name: test-touch
245 * Triangle Manipulation Functions 456 #+begin_src clojure
246 457 (ns cortex.test.touch
247 The rigid bodies which make up a creature have an underlying 458 (:use (cortex world util sense body touch))
248 =Geometry=, which is a =Mesh= plus a =Material= and other important 459 (:use cortex.test.body))
249 data involved with displaying the body. 460
250 461 (cortex.import/mega-import-jme3)
251 A =Mesh= is composed of =Triangles=, and each =Triangle= has three 462
252 verticies which have coordinates in XYZ space and UV space. 463 (defn test-touch []
253 464 (let [the-worm (doto (worm) (body!))
254 Here, =(triangles)= gets all the triangles which compose a mesh, and 465 touch (touch! the-worm)
255 =(triangle-UV-coord)= returns the the UV coordinates of the verticies 466 touch-display (view-touch)]
256 of a triangle. 467 (world (nodify [the-worm (floor)])
257 468 standard-debug-controls
258 #+name: triangles-1 469
259 #+begin_src clojure 470 (fn [world]
260 (in-ns 'cortex.touch) 471 (speed-up world)
261 472 (light-up-everything world))
262 (defn vector3f-seq [#^Vector3f v] 473
263 [(.getX v) (.getY v) (.getZ v)]) 474 (fn [world tpf]
264 475 (touch-display
265 (defn triangle-seq [#^Triangle tri] 476 (map #(% (.getRootNode world)) touch))))))
266 [(vector3f-seq (.get1 tri)) 477 #+end_src
267 (vector3f-seq (.get2 tri))
268 (vector3f-seq (.get3 tri))])
269
270 (defn ->vector3f
271 ([coords] (Vector3f. (nth coords 0 0)
272 (nth coords 1 0)
273 (nth coords 2 0))))
274
275 (defn ->triangle [points]
276 (apply #(Triangle. %1 %2 %3) (map ->vector3f points)))
277
278 (defn triangle
279 "Get the triangle specified by triangle-index from the mesh."
280 [#^Geometry geo triangle-index]
281 (triangle-seq
282 (let [scratch (Triangle.)]
283 (.getTriangle (.getMesh geo) triangle-index scratch) scratch)))
284
285 (defn triangles
286 "Return a sequence of all the Triangles which compose a given
287 Geometry."
288 [#^Geometry geo]
289 (map (partial triangle geo) (range (.getTriangleCount (.getMesh geo)))))
290
291 (defn triangle-vertex-indices
292 "Get the triangle vertex indices of a given triangle from a given
293 mesh."
294 [#^Mesh mesh triangle-index]
295 (let [indices (int-array 3)]
296 (.getTriangle mesh triangle-index indices)
297 (vec indices)))
298
299 (defn vertex-UV-coord
300 "Get the UV-coordinates of the vertex named by vertex-index"
301 [#^Mesh mesh vertex-index]
302 (let [UV-buffer
303 (.getData
304 (.getBuffer
305 mesh
306 VertexBuffer$Type/TexCoord))]
307 [(.get UV-buffer (* vertex-index 2))
308 (.get UV-buffer (+ 1 (* vertex-index 2)))]))
309
310 (defn pixel-triangle [#^Geometry geo image index]
311 (let [mesh (.getMesh geo)
312 width (.getWidth image)
313 height (.getHeight image)]
314 (vec (map (fn [[u v]] (vector (* width u) (* height v)))
315 (map (partial vertex-UV-coord mesh)
316 (triangle-vertex-indices mesh index))))))
317
318 (defn pixel-triangles [#^Geometry geo image]
319 (let [height (.getHeight image)
320 width (.getWidth image)]
321 (map (partial pixel-triangle geo image)
322 (range (.getTriangleCount (.getMesh geo))))))
323
324 #+end_src
325
326 * Triangle Affine Transforms
327
328 The position of each hair is stored in a 2D image in UV
329 coordinates. To place the hair in 3D space we must convert from UV
330 coordinates to XYZ coordinates. Each =Triangle= has coordinates in
331 both UV-space and XYZ-space, which defines a unique [[http://mathworld.wolfram.com/AffineTransformation.html ][Affine Transform]]
332 for translating any coordinate within the UV triangle to the
333 cooresponding coordinate in the XYZ triangle.
334
335 #+name: triangles-3
336 #+begin_src clojure
337 (in-ns 'cortex.touch)
338
339 (defn triangle->matrix4f
340 "Converts the triangle into a 4x4 matrix: The first three columns
341 contain the vertices of the triangle; the last contains the unit
342 normal of the triangle. The bottom row is filled with 1s."
343 [#^Triangle t]
344 (let [mat (Matrix4f.)
345 [vert-1 vert-2 vert-3]
346 ((comp vec map) #(.get t %) (range 3))
347 unit-normal (do (.calculateNormal t)(.getNormal t))
348 vertices [vert-1 vert-2 vert-3 unit-normal]]
349 (dorun
350 (for [row (range 4) col (range 3)]
351 (do
352 (.set mat col row (.get (vertices row)col))
353 (.set mat 3 row 1)))) mat))
354
355 (defn triangles->affine-transform
356 "Returns the affine transformation that converts each vertex in the
357 first triangle into the corresponding vertex in the second
358 triangle."
359 [#^Triangle tri-1 #^Triangle tri-2]
360 (.mult
361 (triangle->matrix4f tri-2)
362 (.invert (triangle->matrix4f tri-1))))
363 #+end_src
364
365
366 * Schrapnel Conversion Functions
367
368 It is convienent to treat a =Triangle= as a sequence of verticies, and
369 a =Vector2f= and =Vector3f= as a sequence of floats. These conversion
370 functions make this easy. If these classes implemented =Iterable= then
371 this code would not be necessary. Hopefully they will in the future.
372
373 * Triangle Boundaries
374
375 For efficiency's sake I will divide the UV-image into small squares
376 which inscribe each UV-triangle, then extract the points which lie
377 inside the triangle and map them to 3D-space using
378 =(triangle-transform)= above. To do this I need a function,
379 =(inside-triangle?)=, which determines whether a point is inside a
380 triangle in 2D UV-space.
381
382 #+name: triangles-4
383 #+begin_src clojure
384 (defn convex-bounds
385 "Returns the smallest square containing the given vertices, as a
386 vector of integers [left top width height]."
387 [verts]
388 (let [xs (map first verts)
389 ys (map second verts)
390 x0 (Math/floor (apply min xs))
391 y0 (Math/floor (apply min ys))
392 x1 (Math/ceil (apply max xs))
393 y1 (Math/ceil (apply max ys))]
394 [x0 y0 (- x1 x0) (- y1 y0)]))
395
396 (defn same-side?
397 "Given the points p1 and p2 and the reference point ref, is point p
398 on the same side of the line that goes through p1 and p2 as ref is?"
399 [p1 p2 ref p]
400 (<=
401 0
402 (.dot
403 (.cross (.subtract p2 p1) (.subtract p p1))
404 (.cross (.subtract p2 p1) (.subtract ref p1)))))
405
406 (defn inside-triangle?
407 "Is the point inside the triangle?"
408 {:author "Dylan Holmes"}
409 [#^Triangle tri #^Vector3f p]
410 (let [[vert-1 vert-2 vert-3] [(.get1 tri) (.get2 tri) (.get3 tri)]]
411 (and
412 (same-side? vert-1 vert-2 vert-3 p)
413 (same-side? vert-2 vert-3 vert-1 p)
414 (same-side? vert-3 vert-1 vert-2 p))))
415 #+end_src
416
417 * Physics Collision Objects
418
419 The "hairs" are actually =Rays= which extend from a point on a
420 =Triangle= in the =Mesh= normal to the =Triangle's= surface.
421 478
422 * Headers 479 * Headers
423 480
424 #+name: touch-header 481 #+name: touch-header
425 #+begin_src clojure 482 #+begin_src clojure
436 (:import com.jme3.collision.CollisionResults) 493 (:import com.jme3.collision.CollisionResults)
437 (:import com.jme3.scene.VertexBuffer$Type) 494 (:import com.jme3.scene.VertexBuffer$Type)
438 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f))) 495 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f)))
439 #+end_src 496 #+end_src
440 497
441 * Adding Touch to the Worm
442
443 #+name: test-touch
444 #+begin_src clojure
445 (ns cortex.test.touch
446 (:use (cortex world util sense body touch))
447 (:use cortex.test.body))
448
449 (cortex.import/mega-import-jme3)
450
451 (defn test-touch []
452 (let [the-worm (doto (worm) (body!))
453 touch (touch! the-worm)
454 touch-display (view-touch)]
455 (world (nodify [the-worm (floor)])
456 standard-debug-controls
457
458 (fn [world]
459 (speed-up world)
460 (light-up-everything world))
461
462 (fn [world tpf]
463 (touch-display
464 (map #(% (.getRootNode world)) touch))))))
465 #+end_src
466 * Source Listing 498 * Source Listing
467 * Next 499 * Next
468 500
469 501
470 * COMMENT Code Generation 502 * COMMENT Code Generation
471 #+begin_src clojure :tangle ../src/cortex/touch.clj 503 #+begin_src clojure :tangle ../src/cortex/touch.clj
472 <<touch-header>> 504 <<touch-header>>
473 <<meta-data>> 505 <<meta-data>>
474 <<triangles-1>> 506 <<triangles-1>>
507 <<triangles-2>>
475 <<triangles-3>> 508 <<triangles-3>>
476 <<triangles-4>> 509 <<triangles-4>>
477 <<sensors>> 510 <<sensors>>
478 <<kernel>> 511 <<kernel>>
479 <<visualization>> 512 <<visualization>>