Mercurial > cortex

annotate org/touch.org @ 242:a7f26a074071

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
saving progress...
author Robert McIntyre <rlm@mit.edu>
date Sun, 12 Feb 2012 13:48:28 -0700
parents f2e583be8584
children f33fec68f775
rev   line source
rlm@37 1 #+title: Simulated Sense of Touch
rlm@0 2 #+author: Robert McIntyre
rlm@0 3 #+email: rlm@mit.edu
rlm@37 4 #+description: Simulated touch for AI research using JMonkeyEngine and clojure.
rlm@37 5 #+keywords: simulation, tactile sense, jMonkeyEngine3, clojure
rlm@4 6 #+SETUPFILE: ../../aurellem/org/setup.org
rlm@4 7 #+INCLUDE: ../../aurellem/org/level-0.org
rlm@0 8
rlm@229 9
rlm@229 10
rlm@37 11 * Touch
rlm@0 12
rlm@226 13 Touch is critical to navigation and spatial reasoning and as such I
rlm@226 14 need a simulated version of it to give to my AI creatures.
rlm@0 15
rlm@228 16 However, touch in my virtual can not exactly correspond to human touch
rlm@228 17 because my creatures are made out of completely rigid segments that
rlm@228 18 don't deform like human skin.
rlm@228 19
rlm@228 20 Human skin has a wide array of touch sensors, each of which speciliaze
rlm@228 21 in detecting different vibrational modes and pressures. These sensors
rlm@228 22 can integrate a vast expanse of skin (i.e. your entire palm), or a
rlm@228 23 tiny patch of skin at the tip of your finger. The hairs of the skin
rlm@228 24 help detect objects before they even come into contact with the skin
rlm@228 25 proper.
rlm@228 26
rlm@228 27 Instead of measuring deformation or vibration, I surround each rigid
rlm@228 28 part with a plenitude of hair-like objects which do not interact with
rlm@228 29 the physical world. Physical objects can pass through them with no
rlm@228 30 effect. The hairs are able to measure contact with other objects, and
rlm@228 31 constantly report how much of their extent is covered. So, even though
rlm@228 32 the creature's body parts do not deform, the hairs create a margin
rlm@228 33 around those body parts which achieves a sense of touch which is a
rlm@228 34 hybrid between a human's sense of deformation and sense from hairs.
rlm@228 35
rlm@228 36 Implementing touch in jMonkeyEngine follows a different techinal route
rlm@228 37 than vision and hearing. Those two senses piggybacked off
rlm@228 38 jMonkeyEngine's 3D audio and video rendering subsystems. To simulate
rlm@228 39 Touch, I use jMonkeyEngine's physics system to execute many small
rlm@229 40 collision detections, one for each "hair". The placement of the
rlm@229 41 "hairs" is determined by a UV-mapped image which shows where each hair
rlm@229 42 should be on the 3D surface of the body.
rlm@228 43
rlm@229 44
rlm@229 45 * Defining Touch Meta-Data in Blender
rlm@229 46
rlm@229 47 Each geometry can have a single UV map which describes the position
rlm@229 48 and length of the "hairs" which will constitute its sense of
rlm@229 49 touch. This image path is stored under the "touch" key. The image
rlm@229 50 itself is grayscale, with black meaning a hair length of 0 (no hair is
rlm@229 51 present) and white meaning a hair length of =scale=, which is a float
rlm@229 52 stored under the key "scale". If the pixel is gray then the resultant
rlm@238 53 hair length is linearly interpolated between 0 and =scale=. I call
rlm@238 54 these "hairs" /feelers/.
rlm@229 55
rlm@231 56 #+name: meta-data
rlm@0 57 #+begin_src clojure
rlm@229 58 (defn tactile-sensor-profile
rlm@229 59 "Return the touch-sensor distribution image in BufferedImage format,
rlm@229 60 or nil if it does not exist."
rlm@229 61 [#^Geometry obj]
rlm@229 62 (if-let [image-path (meta-data obj "touch")]
rlm@229 63 (load-image image-path)))
rlm@233 64
rlm@233 65 (defn tactile-scale
rlm@233 66 "Return the maximum length of a hair. All hairs are scalled between
rlm@233 67 0.0 and this length, depending on their color. Black is 0, and
rlm@233 68 white is maximum length, and everything in between is scalled
rlm@233 69 linearlly. Default scale is 0.01 jMonkeyEngine units."
rlm@233 70 [#^Geometry obj]
rlm@233 71 (if-let [scale (meta-data obj "scale")]
rlm@233 72 scale 0.1))
rlm@228 73 #+end_src
rlm@156 74
rlm@229 75 ** TODO add image showing example touch-uv map
rlm@229 76 ** TODO add metadata display for worm
rlm@229 77
rlm@234 78
rlm@233 79 * Skin Creation
rlm@234 80 * TODO get the actual lengths for each hair
rlm@234 81
rlm@234 82 #+begin_src clojure
rlm@234 83 pixel-triangles
rlm@234 84 xyz-triangles
rlm@234 85 conversions (map triangles->affine-transform pixel-triangles
rlm@234 86 xyz-triangles)
rlm@234 87
rlm@234 88 #+end_src
rlm@234 89
rlm@238 90
rlm@238 91 =(touch-kernel)= generates the functions which implement the sense of
rlm@238 92 touch for a creature. These functions must do 6 things to obtain touch
rlm@238 93 data.
rlm@238 94
rlm@238 95 - Get the tactile profile image and scale paramaters which describe
rlm@238 96 the layout of feelers along the object's surface.
rlm@239 97 =(tactile-sensor-profile)=, =(tactile-scale)=
rlm@239 98
rlm@238 99 - Get the lengths of each feeler by analyzing the color of the
rlm@238 100 pixels in the tactile profile image.
rlm@239 101 NOT IMPLEMENTED YET
rlm@239 102
rlm@238 103 - Find the triangles which make up the mesh in pixel-space and in
rlm@238 104 world-space.
rlm@239 105 =(triangles)= =(pixel-triangles)=
rlm@239 106
rlm@239 107 - Find the coordinates of each pixel in pixel space. These
rlm@239 108 coordinates are used to make the touch-topology.
rlm@240 109 =(feeler-pixel-coords)=
rlm@239 110
rlm@238 111 - Find the coordinates of each pixel in world-space. These
rlm@240 112 coordinates are the origins of the feelers. =(feeler-origins)=
rlm@239 113
rlm@238 114 - Calculate the normals of the triangles in world space, and add
rlm@238 115 them to each of the origins of the feelers. These are the
rlm@238 116 normalized coordinates of the tips of the feelers.
rlm@240 117 For both of these, =(feeler-tips)=
rlm@239 118
rlm@238 119 - Generate some sort of topology for the sensors.
rlm@239 120 =(touch-topology)=
rlm@239 121
rlm@239 122 #+begin_src clojure
rlm@239 123
rlm@239 124
rlm@239 125
rlm@239 126
rlm@239 127 #+end_src
rlm@239 128
rlm@239 129
rlm@238 130
rlm@233 131 #+name: kernel
rlm@233 132 #+begin_src clojure
rlm@233 133 (in-ns 'cortex.touch)
rlm@233 134
rlm@239 135 (declare touch-topology feelers set-ray)
rlm@234 136
rlm@242 137 (defn set-ray [#^Ray ray #^Matrix4f transform
rlm@242 138 #^Vector3f origin #^Vector3f tip
rlm@242 139 length]
rlm@242 140 (.setOrigin ray (.mult transform origin))
rlm@242 141 (.setDirection ray (.subtract
rlm@242 142 (.mult transform tip)
rlm@242 143 (.getOrigin ray)))
rlm@242 144 (.setLimit ray length)
rlm@242 145 ray)
rlm@242 146
rlm@242 147
rlm@233 148 (defn touch-kernel
rlm@234 149 "Constructs a function which will return tactile sensory data from
rlm@234 150 'geo when called from inside a running simulation"
rlm@234 151 [#^Geometry geo]
rlm@242 152 (let [profile (tactile-sensor-profile geo)
rlm@242 153 ray-reference-origins (feeler-origins geo profile)
rlm@242 154 ray-reference-tips (feeler-tips geo profile)
rlm@242 155 ray-lengths (repeat 9000 0.1)
rlm@234 156 current-rays (map (fn [] (Ray.)) ray-reference-origins)
rlm@242 157 topology (touch-topology geo profile)]
rlm@234 158 (if (empty? ray-reference-origins) nil
rlm@234 159 (fn [node]
rlm@234 160 (let [transform (.getWorldMatrix geo)]
rlm@234 161 (dorun
rlm@234 162 (map (fn [ray ref-origin ref-tip length]
rlm@234 163 (set-ray ray transform ref-origin ref-tip length))
rlm@234 164 current-rays ray-reference-origins
rlm@234 165 ray-reference-tips ray-lengths))
rlm@234 166 (vector
rlm@234 167 topology
rlm@234 168 (vec
rlm@234 169 (for [ray current-rays]
rlm@234 170 (do
rlm@234 171 (let [results (CollisionResults.)]
rlm@234 172 (.collideWith node ray results)
rlm@234 173 (let [touch-objects
rlm@234 174 (filter #(not (= geo (.getGeometry %)))
rlm@234 175 results)]
rlm@234 176 [(if (empty? touch-objects)
rlm@234 177 (.getLimit ray)
rlm@234 178 (.getDistance (first touch-objects)))
rlm@234 179 (.getLimit ray)])))))))))))
rlm@234 180
rlm@234 181 (defn touch-kernel*
rlm@233 182 "Returns a function which returns tactile sensory data when called
rlm@233 183 inside a running simulation."
rlm@233 184 [#^Geometry geo]
rlm@233 185 (let [feeler-coords (feeler-coordinates geo)
rlm@233 186 tris (triangles geo)
rlm@233 187 limit (tactile-scale geo)]
rlm@233 188 (if (empty? (touch-topology geo))
rlm@233 189 nil
rlm@233 190 (fn [node]
rlm@233 191 (let [sensor-origins
rlm@233 192 (map
rlm@233 193 #(map (partial local-to-world geo) %)
rlm@233 194 feeler-coords)
rlm@233 195 triangle-normals
rlm@233 196 (map (partial get-ray-direction geo)
rlm@233 197 tris)
rlm@233 198 rays
rlm@233 199 (flatten
rlm@233 200 (map (fn [origins norm]
rlm@233 201 (map #(doto (Ray. % norm)
rlm@233 202 (.setLimit limit)) origins))
rlm@233 203 sensor-origins triangle-normals))]
rlm@233 204 (vector
rlm@233 205 (touch-topology geo)
rlm@233 206 (vec
rlm@233 207 (for [ray rays]
rlm@233 208 (do
rlm@233 209 (let [results (CollisionResults.)]
rlm@233 210 (.collideWith node ray results)
rlm@233 211 (let [touch-objects
rlm@233 212 (filter #(not (= geo (.getGeometry %)))
rlm@233 213 results)]
rlm@233 214 [(if (empty? touch-objects)
rlm@233 215 limit (.getDistance (first touch-objects)))
rlm@233 216 limit])))))))))))
rlm@233 217
rlm@233 218 (defn touch!
rlm@233 219 "Endow the creature with the sense of touch. Returns a sequence of
rlm@233 220 functions, one for each body part with a tactile-sensor-proile,
rlm@233 221 each of which when called returns sensory data for that body part."
rlm@233 222 [#^Node creature]
rlm@233 223 (filter
rlm@233 224 (comp not nil?)
rlm@233 225 (map touch-kernel
rlm@233 226 (filter #(isa? (class %) Geometry)
rlm@233 227 (node-seq creature)))))
rlm@233 228 #+end_src
rlm@233 229
rlm@238 230 * Sensor Related Functions
rlm@238 231
rlm@238 232 These functions analyze the touch-sensor-profile image convert the
rlm@238 233 location of each touch sensor from pixel coordinates to UV-coordinates
rlm@238 234 and XYZ-coordinates.
rlm@238 235
rlm@238 236 #+name: sensors
rlm@238 237 #+begin_src clojure
rlm@240 238 (in-ns 'cortex.touch)
rlm@240 239
rlm@240 240 (defn feeler-pixel-coords
rlm@239 241 "Returns the coordinates of the feelers in pixel space in lists, one
rlm@239 242 list for each triangle, ordered in the same way as (triangles) and
rlm@239 243 (pixel-triangles)."
rlm@239 244 [#^Geometry geo image]
rlm@240 245 (map
rlm@240 246 (fn [pixel-triangle]
rlm@240 247 (filter
rlm@240 248 (fn [coord]
rlm@240 249 (inside-triangle? (->triangle pixel-triangle)
rlm@240 250 (->vector3f coord)))
rlm@240 251 (white-coordinates image (convex-bounds pixel-triangle))))
rlm@240 252 (pixel-triangles geo image)))
rlm@239 253
rlm@242 254 (defn feeler-world-coords [#^Geometry geo image]
rlm@240 255 (let [transforms
rlm@240 256 (map #(triangles->affine-transform
rlm@240 257 (->triangle %1) (->triangle %2))
rlm@240 258 (pixel-triangles geo image)
rlm@240 259 (triangles geo))]
rlm@242 260 (map (fn [transform coords]
rlm@240 261 (map #(.mult transform (->vector3f %)) coords))
rlm@240 262 transforms (feeler-pixel-coords geo image))))
rlm@239 263
rlm@242 264 (defn feeler-origins [#^Geometry geo image]
rlm@242 265 (reduce concat (feeler-world-coords geo image)))
rlm@242 266
rlm@240 267 (defn feeler-tips [#^Geometry geo image]
rlm@242 268 (let [world-coords (feeler-world-coords geo image)
rlm@241 269 normals
rlm@241 270 (map
rlm@241 271 (fn [triangle]
rlm@241 272 (.calculateNormal triangle)
rlm@241 273 (.clone (.getNormal triangle)))
rlm@241 274 (map ->triangle (triangles geo)))]
rlm@242 275
rlm@242 276 (mapcat (fn [origins normal]
rlm@242 277 (map #(.add % normal) origins))
rlm@242 278 world-coords normals)))
rlm@242 279
rlm@241 280
rlm@241 281 (defn touch-topology [#^Geometry geo image]
rlm@241 282 (collapse (feeler-pixel-coords geo image)))
rlm@240 283
rlm@239 284
rlm@238 285 (defn sensors-in-triangle
rlm@238 286 "Locate the touch sensors in the triangle, returning a map of their
rlm@238 287 UV and geometry-relative coordinates."
rlm@238 288 [image mesh tri-index]
rlm@238 289 (let [width (.getWidth image)
rlm@238 290 height (.getHeight image)
rlm@238 291 UV-vertex-coords (triangle-UV-coord mesh width height tri-index)
rlm@238 292 bounds (convex-bounds UV-vertex-coords)
rlm@238 293
rlm@238 294 cutout-triangle (points->triangle UV-vertex-coords)
rlm@238 295 UV-sensor-coords
rlm@238 296 (filter (comp (partial inside-triangle? cutout-triangle)
rlm@238 297 (fn [[u v]] (Vector3f. u v 0)))
rlm@238 298 (white-coordinates image bounds))
rlm@238 299 UV->geometry (triangle-transformation
rlm@238 300 cutout-triangle
rlm@238 301 (mesh-triangle mesh tri-index))
rlm@238 302 geometry-sensor-coords
rlm@238 303 (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0)))
rlm@238 304 UV-sensor-coords)]
rlm@238 305 {:UV UV-sensor-coords :geometry geometry-sensor-coords}))
rlm@238 306
rlm@238 307 (defn-memo locate-feelers
rlm@238 308 "Search the geometry's tactile UV profile for touch sensors,
rlm@238 309 returning their positions in geometry-relative coordinates."
rlm@238 310 [#^Geometry geo]
rlm@238 311 (let [mesh (.getMesh geo)
rlm@238 312 num-triangles (.getTriangleCount mesh)]
rlm@238 313 (if-let [image (tactile-sensor-profile geo)]
rlm@238 314 (map
rlm@238 315 (partial sensors-in-triangle image mesh)
rlm@238 316 (range num-triangles))
rlm@238 317 (repeat (.getTriangleCount mesh) {:UV nil :geometry nil}))))
rlm@238 318
rlm@238 319 (defn-memo touch-topology
rlm@238 320 "Return a sequence of vectors of the form [x y] describing the
rlm@238 321 \"topology\" of the tactile sensors. Points that are close together
rlm@238 322 in the touch-topology are generally close together in the simulation."
rlm@238 323 [#^Gemoetry geo]
rlm@238 324 (vec (collapse (reduce concat (map :UV (locate-feelers geo))))))
rlm@238 325
rlm@238 326 (defn-memo feeler-coordinates
rlm@238 327 "The location of the touch sensors in world-space coordinates."
rlm@238 328 [#^Geometry geo]
rlm@238 329 (vec (map :geometry (locate-feelers geo))))
rlm@238 330 #+end_src
rlm@238 331
rlm@238 332
rlm@238 333
rlm@238 334
rlm@233 335 * Visualizing Touch
rlm@233 336 #+name: visualization
rlm@233 337 #+begin_src clojure
rlm@233 338 (in-ns 'cortex.touch)
rlm@233 339
rlm@233 340 (defn touch->gray
rlm@233 341 "Convert a pair of [distance, max-distance] into a grayscale pixel"
rlm@233 342 [distance max-distance]
rlm@233 343 (gray
rlm@233 344 (- 255
rlm@233 345 (rem
rlm@233 346 (int
rlm@233 347 (* 255 (/ distance max-distance)))
rlm@233 348 256))))
rlm@233 349
rlm@233 350 (defn view-touch
rlm@233 351 "Creates a function which accepts a list of touch sensor-data and
rlm@233 352 displays each element to the screen."
rlm@233 353 []
rlm@233 354 (view-sense
rlm@233 355 (fn
rlm@233 356 [[coords sensor-data]]
rlm@233 357 (let [image (points->image coords)]
rlm@233 358 (dorun
rlm@233 359 (for [i (range (count coords))]
rlm@233 360 (.setRGB image ((coords i) 0) ((coords i) 1)
rlm@233 361 (apply touch->gray (sensor-data i)))))
rlm@233 362 image))))
rlm@233 363 #+end_src
rlm@233 364
rlm@233 365
rlm@233 366
rlm@228 367 * Triangle Manipulation Functions
rlm@228 368
rlm@229 369 The rigid bodies which make up a creature have an underlying
rlm@229 370 =Geometry=, which is a =Mesh= plus a =Material= and other important
rlm@229 371 data involved with displaying the body.
rlm@229 372
rlm@229 373 A =Mesh= is composed of =Triangles=, and each =Triangle= has three
rlm@229 374 verticies which have coordinates in XYZ space and UV space.
rlm@229 375
rlm@229 376 Here, =(triangles)= gets all the triangles which compose a mesh, and
rlm@229 377 =(triangle-UV-coord)= returns the the UV coordinates of the verticies
rlm@229 378 of a triangle.
rlm@229 379
rlm@231 380 #+name: triangles-1
rlm@228 381 #+begin_src clojure
rlm@239 382 (in-ns 'cortex.touch)
rlm@239 383
rlm@239 384 (defn vector3f-seq [#^Vector3f v]
rlm@239 385 [(.getX v) (.getY v) (.getZ v)])
rlm@239 386
rlm@239 387 (defn triangle-seq [#^Triangle tri]
rlm@239 388 [(vector3f-seq (.get1 tri))
rlm@239 389 (vector3f-seq (.get2 tri))
rlm@239 390 (vector3f-seq (.get3 tri))])
rlm@239 391
rlm@240 392 (defn ->vector3f
rlm@240 393 ([coords] (Vector3f. (nth coords 0 0)
rlm@240 394 (nth coords 1 0)
rlm@240 395 (nth coords 2 0))))
rlm@239 396
rlm@239 397 (defn ->triangle [points]
rlm@239 398 (apply #(Triangle. %1 %2 %3) (map ->vector3f points)))
rlm@239 399
rlm@239 400 (defn triangle
rlm@239 401 "Get the triangle specified by triangle-index from the mesh within
rlm@239 402 bounds."
rlm@239 403 [#^Geometry geo triangle-index]
rlm@239 404 (triangle-seq
rlm@239 405 (let [scratch (Triangle.)]
rlm@239 406 (.getTriangle (.getMesh geo) triangle-index scratch) scratch)))
rlm@239 407
rlm@228 408 (defn triangles
rlm@228 409 "Return a sequence of all the Triangles which compose a given
rlm@228 410 Geometry."
rlm@239 411 [#^Geometry geo]
rlm@239 412 (map (partial triangle geo) (range (.getTriangleCount (.getMesh geo)))))
rlm@228 413
rlm@228 414 (defn triangle-vertex-indices
rlm@228 415 "Get the triangle vertex indices of a given triangle from a given
rlm@228 416 mesh."
rlm@228 417 [#^Mesh mesh triangle-index]
rlm@228 418 (let [indices (int-array 3)]
rlm@228 419 (.getTriangle mesh triangle-index indices)
rlm@228 420 (vec indices)))
rlm@228 421
rlm@228 422 (defn vertex-UV-coord
rlm@228 423 "Get the UV-coordinates of the vertex named by vertex-index"
rlm@228 424 [#^Mesh mesh vertex-index]
rlm@228 425 (let [UV-buffer
rlm@228 426 (.getData
rlm@228 427 (.getBuffer
rlm@228 428 mesh
rlm@228 429 VertexBuffer$Type/TexCoord))]
rlm@228 430 [(.get UV-buffer (* vertex-index 2))
rlm@228 431 (.get UV-buffer (+ 1 (* vertex-index 2)))]))
rlm@228 432
rlm@239 433 (defn pixel-triangle [#^Geometry geo image index]
rlm@239 434 (let [mesh (.getMesh geo)
rlm@239 435 width (.getWidth image)
rlm@239 436 height (.getHeight image)]
rlm@239 437 (vec (map (fn [[u v]] (vector (* width u) (* height v)))
rlm@239 438 (map (partial vertex-UV-coord mesh)
rlm@239 439 (triangle-vertex-indices mesh index))))))
rlm@228 440
rlm@239 441 (defn pixel-triangles [#^Geometry geo image]
rlm@239 442 (let [height (.getHeight image)
rlm@239 443 width (.getWidth image)]
rlm@239 444 (map (partial pixel-triangle geo image)
rlm@239 445 (range (.getTriangleCount (.getMesh geo))))))
rlm@229 446
rlm@228 447 #+end_src
rlm@228 448
rlm@228 449 * Triangle Affine Transforms
rlm@228 450
rlm@229 451 The position of each hair is stored in a 2D image in UV
rlm@229 452 coordinates. To place the hair in 3D space we must convert from UV
rlm@229 453 coordinates to XYZ coordinates. Each =Triangle= has coordinates in
rlm@229 454 both UV-space and XYZ-space, which defines a unique [[http://mathworld.wolfram.com/AffineTransformation.html ][Affine Transform]]
rlm@229 455 for translating any coordinate within the UV triangle to the
rlm@229 456 cooresponding coordinate in the XYZ triangle.
rlm@229 457
rlm@231 458 #+name: triangles-3
rlm@228 459 #+begin_src clojure
rlm@228 460 (defn triangle->matrix4f
rlm@228 461 "Converts the triangle into a 4x4 matrix: The first three columns
rlm@228 462 contain the vertices of the triangle; the last contains the unit
rlm@228 463 normal of the triangle. The bottom row is filled with 1s."
rlm@228 464 [#^Triangle t]
rlm@228 465 (let [mat (Matrix4f.)
rlm@228 466 [vert-1 vert-2 vert-3]
rlm@228 467 ((comp vec map) #(.get t %) (range 3))
rlm@228 468 unit-normal (do (.calculateNormal t)(.getNormal t))
rlm@228 469 vertices [vert-1 vert-2 vert-3 unit-normal]]
rlm@228 470 (dorun
rlm@228 471 (for [row (range 4) col (range 3)]
rlm@228 472 (do
rlm@228 473 (.set mat col row (.get (vertices row)col))
rlm@228 474 (.set mat 3 row 1))))
rlm@228 475 mat))
rlm@228 476
rlm@240 477 (defn triangles->affine-transform
rlm@228 478 "Returns the affine transformation that converts each vertex in the
rlm@228 479 first triangle into the corresponding vertex in the second
rlm@228 480 triangle."
rlm@228 481 [#^Triangle tri-1 #^Triangle tri-2]
rlm@228 482 (.mult
rlm@228 483 (triangle->matrix4f tri-2)
rlm@228 484 (.invert (triangle->matrix4f tri-1))))
rlm@228 485 #+end_src
rlm@228 486
rlm@239 487
rlm@239 488 * Schrapnel Conversion Functions
rlm@239 489
rlm@239 490 It is convienent to treat a =Triangle= as a sequence of verticies, and
rlm@239 491 a =Vector2f= and =Vector3f= as a sequence of floats. These conversion
rlm@239 492 functions make this easy. If these classes implemented =Iterable= then
rlm@239 493 this code would not be necessary. Hopefully they will in the future.
rlm@239 494
rlm@239 495 #+name: triangles-2
rlm@239 496 #+begin_src clojure
rlm@239 497 (defn point->vector2f [[u v]]
rlm@239 498 (Vector2f. u v))
rlm@239 499
rlm@239 500 (defn vector2f->vector3f [v]
rlm@239 501 (Vector3f. (.getX v) (.getY v) 0))
rlm@239 502
rlm@239 503 (defn map-triangle [f #^Triangle tri]
rlm@239 504 (Triangle.
rlm@239 505 (f 0 (.get1 tri))
rlm@239 506 (f 1 (.get2 tri))
rlm@239 507 (f 2 (.get3 tri))))
rlm@239 508
rlm@239 509 (defn points->triangle
rlm@239 510 "Convert a list of points into a triangle."
rlm@239 511 [points]
rlm@239 512 (apply #(Triangle. %1 %2 %3)
rlm@239 513 (map (fn [point]
rlm@239 514 (let [point (vec point)]
rlm@239 515 (Vector3f. (get point 0 0)
rlm@239 516 (get point 1 0)
rlm@239 517 (get point 2 0))))
rlm@239 518 (take 3 points))))
rlm@239 519 #+end_src
rlm@239 520
rlm@239 521
rlm@229 522 * Triangle Boundaries
rlm@229 523
rlm@229 524 For efficiency's sake I will divide the UV-image into small squares
rlm@229 525 which inscribe each UV-triangle, then extract the points which lie
rlm@229 526 inside the triangle and map them to 3D-space using
rlm@229 527 =(triangle-transform)= above. To do this I need a function,
rlm@229 528 =(inside-triangle?)=, which determines whether a point is inside a
rlm@229 529 triangle in 2D UV-space.
rlm@228 530
rlm@231 531 #+name: triangles-4
rlm@228 532 #+begin_src clojure
rlm@229 533 (defn convex-bounds
rlm@229 534 "Returns the smallest square containing the given vertices, as a
rlm@229 535 vector of integers [left top width height]."
rlm@240 536 [verts]
rlm@240 537 (let [xs (map first verts)
rlm@240 538 ys (map second verts)
rlm@229 539 x0 (Math/floor (apply min xs))
rlm@229 540 y0 (Math/floor (apply min ys))
rlm@229 541 x1 (Math/ceil (apply max xs))
rlm@229 542 y1 (Math/ceil (apply max ys))]
rlm@229 543 [x0 y0 (- x1 x0) (- y1 y0)]))
rlm@229 544
rlm@229 545 (defn same-side?
rlm@229 546 "Given the points p1 and p2 and the reference point ref, is point p
rlm@229 547 on the same side of the line that goes through p1 and p2 as ref is?"
rlm@229 548 [p1 p2 ref p]
rlm@229 549 (<=
rlm@229 550 0
rlm@229 551 (.dot
rlm@229 552 (.cross (.subtract p2 p1) (.subtract p p1))
rlm@229 553 (.cross (.subtract p2 p1) (.subtract ref p1)))))
rlm@229 554
rlm@229 555 (defn inside-triangle?
rlm@229 556 "Is the point inside the triangle?"
rlm@229 557 {:author "Dylan Holmes"}
rlm@229 558 [#^Triangle tri #^Vector3f p]
rlm@240 559 (let [[vert-1 vert-2 vert-3] [(.get1 tri) (.get2 tri) (.get3 tri)]]
rlm@229 560 (and
rlm@229 561 (same-side? vert-1 vert-2 vert-3 p)
rlm@229 562 (same-side? vert-2 vert-3 vert-1 p)
rlm@229 563 (same-side? vert-3 vert-1 vert-2 p))))
rlm@229 564 #+end_src
rlm@229 565
rlm@240 566 #+results: triangles-4
rlm@240 567 : #'cortex.touch/inside-triangle?
rlm@240 568
rlm@229 569
rlm@228 570 * Physics Collision Objects
rlm@230 571
rlm@234 572 The "hairs" are actually =Rays= which extend from a point on a
rlm@230 573 =Triangle= in the =Mesh= normal to the =Triangle's= surface.
rlm@230 574
rlm@231 575 #+name: rays
rlm@228 576 #+begin_src clojure
rlm@228 577 (defn get-ray-origin
rlm@228 578 "Return the origin which a Ray would have to have to be in the exact
rlm@228 579 center of a particular Triangle in the Geometry in World
rlm@228 580 Coordinates."
rlm@228 581 [geom tri]
rlm@228 582 (let [new (Vector3f.)]
rlm@228 583 (.calculateCenter tri)
rlm@228 584 (.localToWorld geom (.getCenter tri) new) new))
rlm@228 585
rlm@228 586 (defn get-ray-direction
rlm@228 587 "Return the direction which a Ray would have to have to be to point
rlm@228 588 normal to the Triangle, in coordinates relative to the center of the
rlm@228 589 Triangle."
rlm@228 590 [geom tri]
rlm@228 591 (let [n+c (Vector3f.)]
rlm@228 592 (.calculateNormal tri)
rlm@228 593 (.calculateCenter tri)
rlm@228 594 (.localToWorld
rlm@228 595 geom
rlm@228 596 (.add (.getCenter tri) (.getNormal tri)) n+c)
rlm@228 597 (.subtract n+c (get-ray-origin geom tri))))
rlm@228 598 #+end_src
rlm@226 599 * Headers
rlm@231 600
rlm@231 601 #+name: touch-header
rlm@226 602 #+begin_src clojure
rlm@226 603 (ns cortex.touch
rlm@226 604 "Simulate the sense of touch in jMonkeyEngine3. Enables any Geometry
rlm@226 605 to be outfitted with touch sensors with density determined by a UV
rlm@226 606 image. In this way a Geometry can know what parts of itself are
rlm@226 607 touching nearby objects. Reads specially prepared blender files to
rlm@226 608 construct this sense automatically."
rlm@226 609 {:author "Robert McIntyre"}
rlm@226 610 (:use (cortex world util sense))
rlm@226 611 (:use clojure.contrib.def)
rlm@226 612 (:import (com.jme3.scene Geometry Node Mesh))
rlm@226 613 (:import com.jme3.collision.CollisionResults)
rlm@226 614 (:import com.jme3.scene.VertexBuffer$Type)
rlm@226 615 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f)))
rlm@226 616 #+end_src
rlm@37 617
rlm@232 618 * Adding Touch to the Worm
rlm@232 619
rlm@232 620 #+name: test-touch
rlm@232 621 #+begin_src clojure
rlm@232 622 (ns cortex.test.touch
rlm@232 623 (:use (cortex world util sense body touch))
rlm@232 624 (:use cortex.test.body))
rlm@232 625
rlm@232 626 (cortex.import/mega-import-jme3)
rlm@232 627
rlm@232 628 (defn test-touch []
rlm@232 629 (let [the-worm (doto (worm) (body!))
rlm@232 630 touch (touch! the-worm)
rlm@232 631 touch-display (view-touch)]
rlm@232 632 (world (nodify [the-worm (floor)])
rlm@232 633 standard-debug-controls
rlm@232 634
rlm@232 635 (fn [world]
rlm@232 636 (light-up-everything world))
rlm@232 637
rlm@232 638 (fn [world tpf]
rlm@232 639 (touch-display (map #(% (.getRootNode world)) touch))))))
rlm@232 640 #+end_src
rlm@228 641 * Source Listing
rlm@228 642 * Next
rlm@228 643
rlm@228 644
rlm@226 645 * COMMENT Code Generation
rlm@39 646 #+begin_src clojure :tangle ../src/cortex/touch.clj
rlm@231 647 <<touch-header>>
rlm@231 648 <<meta-data>>
rlm@231 649 <<triangles-1>>
rlm@231 650 <<triangles-2>>
rlm@231 651 <<triangles-3>>
rlm@231 652 <<triangles-4>>
rlm@231 653 <<sensors>>
rlm@231 654 <<rays>>
rlm@231 655 <<kernel>>
rlm@231 656 <<visualization>>
rlm@0 657 #+end_src
rlm@0 658
rlm@232 659
rlm@68 660 #+begin_src clojure :tangle ../src/cortex/test/touch.clj
rlm@232 661 <<test-touch>>
rlm@39 662 #+end_src
rlm@39 663
rlm@0 664
rlm@0 665
rlm@0 666
rlm@32 667
rlm@32 668
rlm@226 669