Mercurial > cortex
comparison org/touch.org @ 247:4e220c8fb1ed
first pass at rough draft complete
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Sun, 12 Feb 2012 15:46:01 -0700 |
parents | 63da037ce1c5 |
children | 267add63b168 |
comparison
equal
deleted
inserted
replaced
246:63da037ce1c5 | 247:4e220c8fb1ed |
---|---|
18 Human skin has a wide array of touch sensors, each of which speciliaze | 18 Human skin has a wide array of touch sensors, each of which speciliaze |
19 in detecting different vibrational modes and pressures. These sensors | 19 in detecting different vibrational modes and pressures. These sensors |
20 can integrate a vast expanse of skin (i.e. your entire palm), or a | 20 can integrate a vast expanse of skin (i.e. your entire palm), or a |
21 tiny patch of skin at the tip of your finger. The hairs of the skin | 21 tiny patch of skin at the tip of your finger. The hairs of the skin |
22 help detect objects before they even come into contact with the skin | 22 help detect objects before they even come into contact with the skin |
23 proper. | 23 proper. |
24 | 24 |
25 Instead of measuring deformation or vibration, I surround each rigid | 25 Instead of measuring deformation or vibration, I surround each rigid |
26 part with a plenitude of hair-like objects which do not interact with | 26 part with a plenitude of hair-like objects (/feelers/) which do not |
27 the physical world. Physical objects can pass through them with no | 27 interact with the physical world. Physical objects can pass through |
28 effect. The hairs are able to measure contact with other objects, and | 28 them with no effect. The feelers are able to measure contact with |
29 constantly report how much of their extent is covered. So, even though | 29 other objects, and constantly report how much of their extent is |
30 the creature's body parts do not deform, the hairs create a margin | 30 covered. So, even though the creature's body parts do not deform, the |
31 around those body parts which achieves a sense of touch which is a | 31 feelers create a margin around those body parts which achieves a sense |
32 hybrid between a human's sense of deformation and sense from hairs. | 32 of touch which is a hybrid between a human's sense of deformation and |
33 sense from hairs. | |
33 | 34 |
34 Implementing touch in jMonkeyEngine follows a different techinal route | 35 Implementing touch in jMonkeyEngine follows a different techinal route |
35 than vision and hearing. Those two senses piggybacked off | 36 than vision and hearing. Those two senses piggybacked off |
36 jMonkeyEngine's 3D audio and video rendering subsystems. To simulate | 37 jMonkeyEngine's 3D audio and video rendering subsystems. To simulate |
37 Touch, I use jMonkeyEngine's physics system to execute many small | 38 touch, I use jMonkeyEngine's physics system to execute many small |
38 collision detections, one for each "hair". The placement of the | 39 collision detections, one for each feeler. The placement of the |
39 "hairs" is determined by a UV-mapped image which shows where each hair | 40 feelers is determined by a UV-mapped image which shows where each |
40 should be on the 3D surface of the body. | 41 feeler should be on the 3D surface of the body. |
41 | 42 |
42 * Defining Touch Meta-Data in Blender | 43 * Defining Touch Meta-Data in Blender |
43 | 44 |
44 Each geometry can have a single UV map which describes the position of | 45 Each geometry can have a single UV map which describes the position of |
45 the "hairs" which will constitute its sense of touch. This image path | 46 the feelers which will constitute its sense of touch. This image path |
46 is stored under the "touch" key. The image itself is black and white, | 47 is stored under the "touch" key. The image itself is black and white, |
47 with black meaning a hair length of 0 (no hair is present) and white | 48 with black meaning a feeler length of 0 (no feeler is present) and |
48 meaning a hair length of =scale=, which is a float stored under the | 49 white meaning a feeler length of =scale=, which is a float stored |
49 key "scale". I call these "hairs" /feelers/. | 50 under the key "scale". |
50 | 51 |
51 #+name: meta-data | 52 #+name: meta-data |
52 #+begin_src clojure | 53 #+begin_src clojure |
53 (defn tactile-sensor-profile | 54 (defn tactile-sensor-profile |
54 "Return the touch-sensor distribution image in BufferedImage format, | 55 "Return the touch-sensor distribution image in BufferedImage format, |
56 [#^Geometry obj] | 57 [#^Geometry obj] |
57 (if-let [image-path (meta-data obj "touch")] | 58 (if-let [image-path (meta-data obj "touch")] |
58 (load-image image-path))) | 59 (load-image image-path))) |
59 | 60 |
60 (defn tactile-scale | 61 (defn tactile-scale |
61 "Return the maximum length of a hair. All hairs are scalled between | 62 "Return the length of each feeler. Default scale is 0.01 |
62 0.0 and this length, depending on their color. Black is 0, and | 63 jMonkeyEngine units." |
63 white is maximum length, and everything in between is scalled | |
64 linearlly. Default scale is 0.01 jMonkeyEngine units." | |
65 [#^Geometry obj] | 64 [#^Geometry obj] |
66 (if-let [scale (meta-data obj "scale")] | 65 (if-let [scale (meta-data obj "scale")] |
67 scale 0.1)) | 66 scale 0.1)) |
68 #+end_src | 67 #+end_src |
69 | 68 |
70 Here is an example of a UV-map which specifies the position of touch | 69 Here is an example of a UV-map which specifies the position of touch |
71 sensors along the surface of the worm. | 70 sensors along the surface of the upper segment of the worm. |
72 | 71 |
73 #+attr_html: width=755 | 72 #+attr_html: width=755 |
74 #+caption: This is the tactile-sensor-profile for the upper segment of the worm. It defines regions of high touch sensitivity (where there are many white pixels) and regions of low sensitivity (where white pixels are sparse). | 73 #+caption: This is the tactile-sensor-profile for the upper segment of the worm. It defines regions of high touch sensitivity (where there are many white pixels) and regions of low sensitivity (where white pixels are sparse). |
75 [[../images/finger-UV.png]] | 74 [[../images/finger-UV.png]] |
76 | 75 |
77 * Skin Creation | 76 * Implementation Summary |
78 | 77 |
79 =(touch-kernel)= generates the functions which implement the sense of | 78 To simulate touch there are three conceptual steps. For each solid |
80 touch for a creature. These functions must do 6 things to obtain touch | 79 object in the creature, you first have to get UV image and scale |
81 data. | 80 paramater which define the position and length of the feelers. Then, |
82 | 81 you use the triangles which compose the mesh and the UV data stored in |
83 - Get the tactile profile image and scale paramaters which describe | 82 the mesh to determine the world-space position and orientation of each |
84 the layout of feelers along the object's surface. | 83 feeler. Once every frame, update these positions and orientations to |
85 =(tactile-sensor-profile)=, =(tactile-scale)= | 84 match the current position and orientation of the object, and use |
85 physics collision detection to gather tactile data. | |
86 | |
87 Extracting the meta-data has already been described. The third step, | |
88 physics collision detection, is handled in =(touch-kernel)=. | |
89 Translating the positions and orientations of the feelers from the | |
90 UV-map to world-space is also a three-step process. | |
86 | 91 |
87 - Find the triangles which make up the mesh in pixel-space and in | 92 - Find the triangles which make up the mesh in pixel-space and in |
88 world-space. | 93 world-space. =(triangles)= =(pixel-triangles)=. |
89 =(triangles)= =(pixel-triangles)= | 94 |
90 | 95 - Find the coordinates of each feeler in world-space. These are the |
91 - Find the coordinates of each pixel in pixel space. These | 96 origins of the feelers. =(feeler-origins)=. |
92 coordinates are used to make the touch-topology. | |
93 =(feeler-pixel-coords)= | |
94 | |
95 - Find the coordinates of each pixel in world-space. These | |
96 coordinates are the origins of the feelers. =(feeler-origins)= | |
97 | 97 |
98 - Calculate the normals of the triangles in world space, and add | 98 - Calculate the normals of the triangles in world space, and add |
99 them to each of the origins of the feelers. These are the | 99 them to each of the origins of the feelers. These are the |
100 normalized coordinates of the tips of the feelers. | 100 normalized coordinates of the tips of the feelers. =(feeler-tips)=. |
101 For both of these, =(feeler-tips)= | 101 |
102 | 102 * Triangle Math |
103 - Generate some sort of topology for the sensors. | 103 ** Schrapnel Conversion Functions |
104 =(touch-topology)= | 104 |
105 | 105 #+name: triangles-1 |
106 #+begin_src clojure | |
107 (defn vector3f-seq [#^Vector3f v] | |
108 [(.getX v) (.getY v) (.getZ v)]) | |
109 | |
110 (defn triangle-seq [#^Triangle tri] | |
111 [(vector3f-seq (.get1 tri)) | |
112 (vector3f-seq (.get2 tri)) | |
113 (vector3f-seq (.get3 tri))]) | |
114 | |
115 (defn ->vector3f | |
116 ([coords] (Vector3f. (nth coords 0 0) | |
117 (nth coords 1 0) | |
118 (nth coords 2 0)))) | |
119 | |
120 (defn ->triangle [points] | |
121 (apply #(Triangle. %1 %2 %3) (map ->vector3f points))) | |
122 #+end_src | |
123 | |
124 It is convienent to treat a =Triangle= as a sequence of verticies, and | |
125 a =Vector2f= and =Vector3f= as a sequence of floats. These conversion | |
126 functions make this easy. If these classes implemented =Iterable= then | |
127 =(seq)= would work on them automitacally. | |
128 ** Decomposing a 3D shape into Triangles | |
129 | |
130 The rigid bodies which make up a creature have an underlying | |
131 =Geometry=, which is a =Mesh= plus a =Material= and other important | |
132 data involved with displaying the body. | |
133 | |
134 A =Mesh= is composed of =Triangles=, and each =Triangle= has three | |
135 verticies which have coordinates in world space and UV space. | |
136 | |
137 Here, =(triangles)= gets all the world-space triangles which compose a | |
138 mesh, while =(pixel-triangles)= gets those same triangles expressed in | |
139 pixel coordinates (which are UV coordinates scaled to fit the height | |
140 and width of the UV image). | |
141 | |
142 #+name: triangles-2 | |
143 #+begin_src clojure | |
144 (in-ns 'cortex.touch) | |
145 (defn triangle | |
146 "Get the triangle specified by triangle-index from the mesh." | |
147 [#^Geometry geo triangle-index] | |
148 (triangle-seq | |
149 (let [scratch (Triangle.)] | |
150 (.getTriangle (.getMesh geo) triangle-index scratch) scratch))) | |
151 | |
152 (defn triangles | |
153 "Return a sequence of all the Triangles which compose a given | |
154 Geometry." | |
155 [#^Geometry geo] | |
156 (map (partial triangle geo) (range (.getTriangleCount (.getMesh geo))))) | |
157 | |
158 (defn triangle-vertex-indices | |
159 "Get the triangle vertex indices of a given triangle from a given | |
160 mesh." | |
161 [#^Mesh mesh triangle-index] | |
162 (let [indices (int-array 3)] | |
163 (.getTriangle mesh triangle-index indices) | |
164 (vec indices))) | |
165 | |
166 (defn vertex-UV-coord | |
167 "Get the UV-coordinates of the vertex named by vertex-index" | |
168 [#^Mesh mesh vertex-index] | |
169 (let [UV-buffer | |
170 (.getData | |
171 (.getBuffer | |
172 mesh | |
173 VertexBuffer$Type/TexCoord))] | |
174 [(.get UV-buffer (* vertex-index 2)) | |
175 (.get UV-buffer (+ 1 (* vertex-index 2)))])) | |
176 | |
177 (defn pixel-triangle [#^Geometry geo image index] | |
178 (let [mesh (.getMesh geo) | |
179 width (.getWidth image) | |
180 height (.getHeight image)] | |
181 (vec (map (fn [[u v]] (vector (* width u) (* height v))) | |
182 (map (partial vertex-UV-coord mesh) | |
183 (triangle-vertex-indices mesh index)))))) | |
184 | |
185 (defn pixel-triangles [#^Geometry geo image] | |
186 (let [height (.getHeight image) | |
187 width (.getWidth image)] | |
188 (map (partial pixel-triangle geo image) | |
189 (range (.getTriangleCount (.getMesh geo)))))) | |
190 #+end_src | |
191 ** The Affine Transform from one Triangle to Another | |
192 | |
193 =(pixel-triangles)= gives us the mesh triangles expressed in pixel | |
194 coordinates and =(triangles)= gives us the mesh triangles expressed in | |
195 world coordinates. The tactile-sensor-profile gives the position of | |
196 each feeler in pixel-space. In order to convert pixel-dpace | |
197 coordinates into world-space coordinates we need something that takes | |
198 coordinates on the surface of one triangle and gives the corresponding | |
199 coordinates on the surface of another triangle. | |
200 | |
201 Triangles are [[http://mathworld.wolfram.com/AffineTransformation.html ][affine]], which means any triangle can be transformed into | |
202 any other by a combination of translation, scaling, and | |
203 rotation. jMonkeyEngine's =Matrix4f= objects can describe any affine | |
204 transformation. The affine transformation from one triangle to another | |
205 is readily computable if the triangle is expressed in terms of a $4x4$ | |
206 matrix. | |
207 | |
208 \begin{bmatrix} | |
209 x_1 & x_2 & x_3 & n_x \\ | |
210 y_1 & y_2 & y_3 & n_y \\ | |
211 z_1 & z_2 & z_3 & n_z \\ | |
212 1 & 1 & 1 & 1 | |
213 \end{bmatrix} | |
214 | |
215 Here, the first three columns of the matrix are the verticies of the | |
216 triangle. The last column is the right-handed unit normal of the | |
217 triangle. | |
218 | |
219 With two triangles $T_{1}$ and $T_{2}$ each expressed as a matrix like | |
220 above, the affine transform from $T_{1}$ to $T_{2}$ is | |
221 | |
222 $T_{2}T_{1}^{-1}$ | |
223 | |
224 The clojure code below recaptiulates the formulas above. | |
225 | |
226 #+name: triangles-3 | |
227 #+begin_src clojure | |
228 (in-ns 'cortex.touch) | |
229 | |
230 (defn triangle->matrix4f | |
231 "Converts the triangle into a 4x4 matrix: The first three columns | |
232 contain the vertices of the triangle; the last contains the unit | |
233 normal of the triangle. The bottom row is filled with 1s." | |
234 [#^Triangle t] | |
235 (let [mat (Matrix4f.) | |
236 [vert-1 vert-2 vert-3] | |
237 ((comp vec map) #(.get t %) (range 3)) | |
238 unit-normal (do (.calculateNormal t)(.getNormal t)) | |
239 vertices [vert-1 vert-2 vert-3 unit-normal]] | |
240 (dorun | |
241 (for [row (range 4) col (range 3)] | |
242 (do | |
243 (.set mat col row (.get (vertices row) col)) | |
244 (.set mat 3 row 1)))) mat)) | |
245 | |
246 (defn triangles->affine-transform | |
247 "Returns the affine transformation that converts each vertex in the | |
248 first triangle into the corresponding vertex in the second | |
249 triangle." | |
250 [#^Triangle tri-1 #^Triangle tri-2] | |
251 (.mult | |
252 (triangle->matrix4f tri-2) | |
253 (.invert (triangle->matrix4f tri-1)))) | |
254 #+end_src | |
255 ** Triangle Boundaries | |
256 | |
257 For efficiency's sake I will divide the tactile-profile image into | |
258 small squares which inscribe each pixel-triangle, then extract the | |
259 points which lie inside the triangle and map them to 3D-space using | |
260 =(triangle-transform)= above. To do this I need a function, | |
261 =(convex-bounds)= which finds the smallest box which inscribes a 2D | |
262 triangle. | |
263 | |
264 =(inside-triangle?)= determines whether a point is inside a triangle | |
265 in 2D pixel-space. | |
266 | |
267 #+name: triangles-4 | |
268 #+begin_src clojure | |
269 (defn convex-bounds | |
270 "Returns the smallest square containing the given vertices, as a | |
271 vector of integers [left top width height]." | |
272 [verts] | |
273 (let [xs (map first verts) | |
274 ys (map second verts) | |
275 x0 (Math/floor (apply min xs)) | |
276 y0 (Math/floor (apply min ys)) | |
277 x1 (Math/ceil (apply max xs)) | |
278 y1 (Math/ceil (apply max ys))] | |
279 [x0 y0 (- x1 x0) (- y1 y0)])) | |
280 | |
281 (defn same-side? | |
282 "Given the points p1 and p2 and the reference point ref, is point p | |
283 on the same side of the line that goes through p1 and p2 as ref is?" | |
284 [p1 p2 ref p] | |
285 (<= | |
286 0 | |
287 (.dot | |
288 (.cross (.subtract p2 p1) (.subtract p p1)) | |
289 (.cross (.subtract p2 p1) (.subtract ref p1))))) | |
290 | |
291 (defn inside-triangle? | |
292 "Is the point inside the triangle?" | |
293 {:author "Dylan Holmes"} | |
294 [#^Triangle tri #^Vector3f p] | |
295 (let [[vert-1 vert-2 vert-3] [(.get1 tri) (.get2 tri) (.get3 tri)]] | |
296 (and | |
297 (same-side? vert-1 vert-2 vert-3 p) | |
298 (same-side? vert-2 vert-3 vert-1 p) | |
299 (same-side? vert-3 vert-1 vert-2 p)))) | |
300 #+end_src | |
301 | |
302 * Feeler Coordinates | |
303 | |
304 The triangle-related functions above make short work of calculating | |
305 the positions and orientations of each feeler in world-space. | |
306 | |
307 #+name: sensors | |
308 #+begin_src clojure | |
309 (in-ns 'cortex.touch) | |
310 | |
311 (defn feeler-pixel-coords | |
312 "Returns the coordinates of the feelers in pixel space in lists, one | |
313 list for each triangle, ordered in the same way as (triangles) and | |
314 (pixel-triangles)." | |
315 [#^Geometry geo image] | |
316 (map | |
317 (fn [pixel-triangle] | |
318 (filter | |
319 (fn [coord] | |
320 (inside-triangle? (->triangle pixel-triangle) | |
321 (->vector3f coord))) | |
322 (white-coordinates image (convex-bounds pixel-triangle)))) | |
323 (pixel-triangles geo image))) | |
324 | |
325 (defn feeler-world-coords | |
326 "Returns the coordinates of the feelers in world space in lists, one | |
327 list for each triangle, ordered in the same way as (triangles) and | |
328 (pixel-triangles)." | |
329 [#^Geometry geo image] | |
330 (let [transforms | |
331 (map #(triangles->affine-transform | |
332 (->triangle %1) (->triangle %2)) | |
333 (pixel-triangles geo image) | |
334 (triangles geo))] | |
335 (map (fn [transform coords] | |
336 (map #(.mult transform (->vector3f %)) coords)) | |
337 transforms (feeler-pixel-coords geo image)))) | |
338 | |
339 (defn feeler-origins | |
340 "The world space coordinates of the root of each feeler." | |
341 [#^Geometry geo image] | |
342 (reduce concat (feeler-world-coords geo image))) | |
343 | |
344 (defn feeler-tips | |
345 "The world space coordinates of the tip of each feeler." | |
346 [#^Geometry geo image] | |
347 (let [world-coords (feeler-world-coords geo image) | |
348 normals | |
349 (map | |
350 (fn [triangle] | |
351 (.calculateNormal triangle) | |
352 (.clone (.getNormal triangle))) | |
353 (map ->triangle (triangles geo)))] | |
354 | |
355 (mapcat (fn [origins normal] | |
356 (map #(.add % normal) origins)) | |
357 world-coords normals))) | |
358 | |
359 (defn touch-topology | |
360 "touch-topology? is not a function." | |
361 [#^Geometry geo image] | |
362 (collapse (reduce concat (feeler-pixel-coords geo image)))) | |
363 #+end_src | |
364 * Simulated Touch | |
365 | |
366 =(touch-kernel)= generates functions to be called from within a | |
367 simulation that perform the necessary physics collisions to collect | |
368 tactile data, and =(touch!)= recursively applies it to every node in | |
369 the creature. | |
106 | 370 |
107 #+name: kernel | 371 #+name: kernel |
108 #+begin_src clojure | 372 #+begin_src clojure |
109 (in-ns 'cortex.touch) | 373 (in-ns 'cortex.touch) |
110 | 374 |
161 (map touch-kernel | 425 (map touch-kernel |
162 (filter #(isa? (class %) Geometry) | 426 (filter #(isa? (class %) Geometry) |
163 (node-seq creature))))) | 427 (node-seq creature))))) |
164 #+end_src | 428 #+end_src |
165 | 429 |
166 * Sensor Related Functions | |
167 | |
168 These functions analyze the touch-sensor-profile image convert the | |
169 location of each touch sensor from pixel coordinates to UV-coordinates | |
170 and XYZ-coordinates. | |
171 | |
172 #+name: sensors | |
173 #+begin_src clojure | |
174 (in-ns 'cortex.touch) | |
175 | |
176 (defn feeler-pixel-coords | |
177 "Returns the coordinates of the feelers in pixel space in lists, one | |
178 list for each triangle, ordered in the same way as (triangles) and | |
179 (pixel-triangles)." | |
180 [#^Geometry geo image] | |
181 (map | |
182 (fn [pixel-triangle] | |
183 (filter | |
184 (fn [coord] | |
185 (inside-triangle? (->triangle pixel-triangle) | |
186 (->vector3f coord))) | |
187 (white-coordinates image (convex-bounds pixel-triangle)))) | |
188 (pixel-triangles geo image))) | |
189 | |
190 (defn feeler-world-coords [#^Geometry geo image] | |
191 (let [transforms | |
192 (map #(triangles->affine-transform | |
193 (->triangle %1) (->triangle %2)) | |
194 (pixel-triangles geo image) | |
195 (triangles geo))] | |
196 (map (fn [transform coords] | |
197 (map #(.mult transform (->vector3f %)) coords)) | |
198 transforms (feeler-pixel-coords geo image)))) | |
199 | |
200 (defn feeler-origins [#^Geometry geo image] | |
201 (reduce concat (feeler-world-coords geo image))) | |
202 | |
203 (defn feeler-tips [#^Geometry geo image] | |
204 (let [world-coords (feeler-world-coords geo image) | |
205 normals | |
206 (map | |
207 (fn [triangle] | |
208 (.calculateNormal triangle) | |
209 (.clone (.getNormal triangle))) | |
210 (map ->triangle (triangles geo)))] | |
211 | |
212 (mapcat (fn [origins normal] | |
213 (map #(.add % normal) origins)) | |
214 world-coords normals))) | |
215 | |
216 (defn touch-topology [#^Geometry geo image] | |
217 (collapse (reduce concat (feeler-pixel-coords geo image)))) | |
218 #+end_src | |
219 | |
220 * Visualizing Touch | 430 * Visualizing Touch |
431 | |
221 #+name: visualization | 432 #+name: visualization |
222 #+begin_src clojure | 433 #+begin_src clojure |
223 (in-ns 'cortex.touch) | 434 (in-ns 'cortex.touch) |
224 | 435 |
225 (defn touch->gray | 436 (defn touch->gray |
237 (dorun | 448 (dorun |
238 (for [i (range (count coords))] | 449 (for [i (range (count coords))] |
239 (.setRGB image ((coords i) 0) ((coords i) 1) | 450 (.setRGB image ((coords i) 0) ((coords i) 1) |
240 (apply touch->gray (sensor-data i))))) image)))) | 451 (apply touch->gray (sensor-data i))))) image)))) |
241 #+end_src | 452 #+end_src |
242 | 453 * Adding Touch to the Worm |
243 | 454 |
244 | 455 #+name: test-touch |
245 * Triangle Manipulation Functions | 456 #+begin_src clojure |
246 | 457 (ns cortex.test.touch |
247 The rigid bodies which make up a creature have an underlying | 458 (:use (cortex world util sense body touch)) |
248 =Geometry=, which is a =Mesh= plus a =Material= and other important | 459 (:use cortex.test.body)) |
249 data involved with displaying the body. | 460 |
250 | 461 (cortex.import/mega-import-jme3) |
251 A =Mesh= is composed of =Triangles=, and each =Triangle= has three | 462 |
252 verticies which have coordinates in XYZ space and UV space. | 463 (defn test-touch [] |
253 | 464 (let [the-worm (doto (worm) (body!)) |
254 Here, =(triangles)= gets all the triangles which compose a mesh, and | 465 touch (touch! the-worm) |
255 =(triangle-UV-coord)= returns the the UV coordinates of the verticies | 466 touch-display (view-touch)] |
256 of a triangle. | 467 (world (nodify [the-worm (floor)]) |
257 | 468 standard-debug-controls |
258 #+name: triangles-1 | 469 |
259 #+begin_src clojure | 470 (fn [world] |
260 (in-ns 'cortex.touch) | 471 (speed-up world) |
261 | 472 (light-up-everything world)) |
262 (defn vector3f-seq [#^Vector3f v] | 473 |
263 [(.getX v) (.getY v) (.getZ v)]) | 474 (fn [world tpf] |
264 | 475 (touch-display |
265 (defn triangle-seq [#^Triangle tri] | 476 (map #(% (.getRootNode world)) touch)))))) |
266 [(vector3f-seq (.get1 tri)) | 477 #+end_src |
267 (vector3f-seq (.get2 tri)) | |
268 (vector3f-seq (.get3 tri))]) | |
269 | |
270 (defn ->vector3f | |
271 ([coords] (Vector3f. (nth coords 0 0) | |
272 (nth coords 1 0) | |
273 (nth coords 2 0)))) | |
274 | |
275 (defn ->triangle [points] | |
276 (apply #(Triangle. %1 %2 %3) (map ->vector3f points))) | |
277 | |
278 (defn triangle | |
279 "Get the triangle specified by triangle-index from the mesh." | |
280 [#^Geometry geo triangle-index] | |
281 (triangle-seq | |
282 (let [scratch (Triangle.)] | |
283 (.getTriangle (.getMesh geo) triangle-index scratch) scratch))) | |
284 | |
285 (defn triangles | |
286 "Return a sequence of all the Triangles which compose a given | |
287 Geometry." | |
288 [#^Geometry geo] | |
289 (map (partial triangle geo) (range (.getTriangleCount (.getMesh geo))))) | |
290 | |
291 (defn triangle-vertex-indices | |
292 "Get the triangle vertex indices of a given triangle from a given | |
293 mesh." | |
294 [#^Mesh mesh triangle-index] | |
295 (let [indices (int-array 3)] | |
296 (.getTriangle mesh triangle-index indices) | |
297 (vec indices))) | |
298 | |
299 (defn vertex-UV-coord | |
300 "Get the UV-coordinates of the vertex named by vertex-index" | |
301 [#^Mesh mesh vertex-index] | |
302 (let [UV-buffer | |
303 (.getData | |
304 (.getBuffer | |
305 mesh | |
306 VertexBuffer$Type/TexCoord))] | |
307 [(.get UV-buffer (* vertex-index 2)) | |
308 (.get UV-buffer (+ 1 (* vertex-index 2)))])) | |
309 | |
310 (defn pixel-triangle [#^Geometry geo image index] | |
311 (let [mesh (.getMesh geo) | |
312 width (.getWidth image) | |
313 height (.getHeight image)] | |
314 (vec (map (fn [[u v]] (vector (* width u) (* height v))) | |
315 (map (partial vertex-UV-coord mesh) | |
316 (triangle-vertex-indices mesh index)))))) | |
317 | |
318 (defn pixel-triangles [#^Geometry geo image] | |
319 (let [height (.getHeight image) | |
320 width (.getWidth image)] | |
321 (map (partial pixel-triangle geo image) | |
322 (range (.getTriangleCount (.getMesh geo)))))) | |
323 | |
324 #+end_src | |
325 | |
326 * Triangle Affine Transforms | |
327 | |
328 The position of each hair is stored in a 2D image in UV | |
329 coordinates. To place the hair in 3D space we must convert from UV | |
330 coordinates to XYZ coordinates. Each =Triangle= has coordinates in | |
331 both UV-space and XYZ-space, which defines a unique [[http://mathworld.wolfram.com/AffineTransformation.html ][Affine Transform]] | |
332 for translating any coordinate within the UV triangle to the | |
333 cooresponding coordinate in the XYZ triangle. | |
334 | |
335 #+name: triangles-3 | |
336 #+begin_src clojure | |
337 (in-ns 'cortex.touch) | |
338 | |
339 (defn triangle->matrix4f | |
340 "Converts the triangle into a 4x4 matrix: The first three columns | |
341 contain the vertices of the triangle; the last contains the unit | |
342 normal of the triangle. The bottom row is filled with 1s." | |
343 [#^Triangle t] | |
344 (let [mat (Matrix4f.) | |
345 [vert-1 vert-2 vert-3] | |
346 ((comp vec map) #(.get t %) (range 3)) | |
347 unit-normal (do (.calculateNormal t)(.getNormal t)) | |
348 vertices [vert-1 vert-2 vert-3 unit-normal]] | |
349 (dorun | |
350 (for [row (range 4) col (range 3)] | |
351 (do | |
352 (.set mat col row (.get (vertices row)col)) | |
353 (.set mat 3 row 1)))) mat)) | |
354 | |
355 (defn triangles->affine-transform | |
356 "Returns the affine transformation that converts each vertex in the | |
357 first triangle into the corresponding vertex in the second | |
358 triangle." | |
359 [#^Triangle tri-1 #^Triangle tri-2] | |
360 (.mult | |
361 (triangle->matrix4f tri-2) | |
362 (.invert (triangle->matrix4f tri-1)))) | |
363 #+end_src | |
364 | |
365 | |
366 * Schrapnel Conversion Functions | |
367 | |
368 It is convienent to treat a =Triangle= as a sequence of verticies, and | |
369 a =Vector2f= and =Vector3f= as a sequence of floats. These conversion | |
370 functions make this easy. If these classes implemented =Iterable= then | |
371 this code would not be necessary. Hopefully they will in the future. | |
372 | |
373 * Triangle Boundaries | |
374 | |
375 For efficiency's sake I will divide the UV-image into small squares | |
376 which inscribe each UV-triangle, then extract the points which lie | |
377 inside the triangle and map them to 3D-space using | |
378 =(triangle-transform)= above. To do this I need a function, | |
379 =(inside-triangle?)=, which determines whether a point is inside a | |
380 triangle in 2D UV-space. | |
381 | |
382 #+name: triangles-4 | |
383 #+begin_src clojure | |
384 (defn convex-bounds | |
385 "Returns the smallest square containing the given vertices, as a | |
386 vector of integers [left top width height]." | |
387 [verts] | |
388 (let [xs (map first verts) | |
389 ys (map second verts) | |
390 x0 (Math/floor (apply min xs)) | |
391 y0 (Math/floor (apply min ys)) | |
392 x1 (Math/ceil (apply max xs)) | |
393 y1 (Math/ceil (apply max ys))] | |
394 [x0 y0 (- x1 x0) (- y1 y0)])) | |
395 | |
396 (defn same-side? | |
397 "Given the points p1 and p2 and the reference point ref, is point p | |
398 on the same side of the line that goes through p1 and p2 as ref is?" | |
399 [p1 p2 ref p] | |
400 (<= | |
401 0 | |
402 (.dot | |
403 (.cross (.subtract p2 p1) (.subtract p p1)) | |
404 (.cross (.subtract p2 p1) (.subtract ref p1))))) | |
405 | |
406 (defn inside-triangle? | |
407 "Is the point inside the triangle?" | |
408 {:author "Dylan Holmes"} | |
409 [#^Triangle tri #^Vector3f p] | |
410 (let [[vert-1 vert-2 vert-3] [(.get1 tri) (.get2 tri) (.get3 tri)]] | |
411 (and | |
412 (same-side? vert-1 vert-2 vert-3 p) | |
413 (same-side? vert-2 vert-3 vert-1 p) | |
414 (same-side? vert-3 vert-1 vert-2 p)))) | |
415 #+end_src | |
416 | |
417 * Physics Collision Objects | |
418 | |
419 The "hairs" are actually =Rays= which extend from a point on a | |
420 =Triangle= in the =Mesh= normal to the =Triangle's= surface. | |
421 | 478 |
422 * Headers | 479 * Headers |
423 | 480 |
424 #+name: touch-header | 481 #+name: touch-header |
425 #+begin_src clojure | 482 #+begin_src clojure |
436 (:import com.jme3.collision.CollisionResults) | 493 (:import com.jme3.collision.CollisionResults) |
437 (:import com.jme3.scene.VertexBuffer$Type) | 494 (:import com.jme3.scene.VertexBuffer$Type) |
438 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f))) | 495 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f))) |
439 #+end_src | 496 #+end_src |
440 | 497 |
441 * Adding Touch to the Worm | |
442 | |
443 #+name: test-touch | |
444 #+begin_src clojure | |
445 (ns cortex.test.touch | |
446 (:use (cortex world util sense body touch)) | |
447 (:use cortex.test.body)) | |
448 | |
449 (cortex.import/mega-import-jme3) | |
450 | |
451 (defn test-touch [] | |
452 (let [the-worm (doto (worm) (body!)) | |
453 touch (touch! the-worm) | |
454 touch-display (view-touch)] | |
455 (world (nodify [the-worm (floor)]) | |
456 standard-debug-controls | |
457 | |
458 (fn [world] | |
459 (speed-up world) | |
460 (light-up-everything world)) | |
461 | |
462 (fn [world tpf] | |
463 (touch-display | |
464 (map #(% (.getRootNode world)) touch)))))) | |
465 #+end_src | |
466 * Source Listing | 498 * Source Listing |
467 * Next | 499 * Next |
468 | 500 |
469 | 501 |
470 * COMMENT Code Generation | 502 * COMMENT Code Generation |
471 #+begin_src clojure :tangle ../src/cortex/touch.clj | 503 #+begin_src clojure :tangle ../src/cortex/touch.clj |
472 <<touch-header>> | 504 <<touch-header>> |
473 <<meta-data>> | 505 <<meta-data>> |
474 <<triangles-1>> | 506 <<triangles-1>> |
507 <<triangles-2>> | |
475 <<triangles-3>> | 508 <<triangles-3>> |
476 <<triangles-4>> | 509 <<triangles-4>> |
477 <<sensors>> | 510 <<sensors>> |
478 <<kernel>> | 511 <<kernel>> |
479 <<visualization>> | 512 <<visualization>> |