Mercurial > cortex

annotate org/test-creature.org @ 151:aaacf087504c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
refactored vision code
author Robert McIntyre <rlm@mit.edu>
date Fri, 03 Feb 2012 05:52:18 -0700
parents 1e6beed24cec
children c95179907951
rev   line source
rlm@73 1 #+title: First attempt at a creature!
rlm@73 2 #+author: Robert McIntyre
rlm@73 3 #+email: rlm@mit.edu
rlm@73 4 #+description:
rlm@73 5 #+keywords: simulation, jMonkeyEngine3, clojure
rlm@73 6 #+SETUPFILE: ../../aurellem/org/setup.org
rlm@73 7 #+INCLUDE: ../../aurellem/org/level-0.org
rlm@73 8
rlm@129 9
rlm@129 10
rlm@129 11 * Brainstorming different sensors and effectors.
rlm@129 12
rlm@129 13 Every sense that we have should have an effector that changes what
rlm@129 14 that sense (or others who have that sense) experiences.
rlm@129 15
rlm@129 16 ** Classic Senses
rlm@129 17 | Sense | Effector |
rlm@129 18 |------------------------------+---------------------------------|
rlm@129 19 | Vision | Variable Coloration |
rlm@129 20 | Hearing | Speech |
rlm@129 21 | Proprioception | Movement |
rlm@129 22 | Smell/Taste (Chemoreception) | Pheremones |
rlm@129 23 | Touch | Movement / Controllable Texture |
rlm@129 24 | Acceleration | Movement |
rlm@129 25 | Balance (sense gravity) | Movement |
rlm@129 26 | | |
rlm@129 27
rlm@129 28 - New Senses/Effectors
rlm@129 29 - Levitation
rlm@129 30 - Telekenesis
rlm@139 31 - control of gravity within a certain radius
rlm@139 32 - speed up/slow time
rlm@139 33 - object creation/destruction
rlm@129 34
rlm@129 35 - Symbol Sense
rlm@129 36 Where objects in the world can be queried for description /
rlm@129 37 symbols.
rlm@129 38
rlm@129 39 - Symbol Marking
rlm@129 40 The ability to mark objects in the world with your own descriptions
rlm@129 41 and symbols.
rlm@129 42
rlm@129 43 - Vision
rlm@129 44 Distinguish the polarization of light
rlm@129 45 Color
rlm@129 46 Movement
rlm@129 47
rlm@129 48 * project ideas
rlm@129 49 - HACKER for writing muscle-control programs : Presented with
rlm@129 50 low-level muscle control/ sense API, generate higher level programs
rlm@129 51 for accomplishing various stated goals. Example goals might be
rlm@129 52 "extend all your fingers" or "move your hand into the area with
rlm@129 53 blue light" or "decrease the angle of this joint". It would be
rlm@129 54 like Sussman's HACKER, except it would operate with much more data
rlm@129 55 in a more realistic world. Start off with "calestanthics" to
rlm@129 56 develop subrouitines over the motor control API. This would be the
rlm@129 57 "spinal chord" of a more intelligent creature. The low level
rlm@129 58 programming code might be a turning machine that could develop
rlm@129 59 programs to iterate over a "tape" where each entry in the tape
rlm@129 60 could control recruitment of the fibers in a muscle.
rlm@129 61 - Make a virtual computer in the virtual world which with which the
rlm@129 62 creature interacts using its fingers to press keys on a virtual
rlm@129 63 keyboard. The creature can access the internet, watch videos, take
rlm@129 64 over the world, anything it wants.
rlm@129 65 - Make virtual insturments like pianos, drumbs, etc that it learns to
rlm@129 66 play.
rlm@129 67 - make a joint that figures out what type of joint it is (range of
rlm@129 68 motion)
rlm@129 69
rlm@129 70
rlm@129 71
rlm@129 72
rlm@129 73
rlm@129 74 * goals
rlm@125 75
rlm@125 76 ** have to get done before winston
rlm@126 77 - [ ] write an explination for why greyscale bitmaps for senses is
rlm@126 78 appropiate -- 1/2 day
rlm@149 79 - [X] muscle control -- day
rlm@146 80 - [X] proprioception sensor map in the style of the other senses -- day
rlm@125 81 - [ ] refactor integration code to distribute to each of the senses
rlm@126 82 -- day
rlm@147 83 - [ ] create video showing all the senses for Winston -- 2 days
rlm@147 84 - [ ] send package to friends for critiques -- 2 days
rlm@126 85 - [ ] write summary of project for Winston \
rlm@126 86 - [ ] project proposals for Winston \
rlm@126 87 - [ ] additional senses to be implemented for Winston | -- 2 days
rlm@126 88 - [ ] send Winston package /
rlm@125 89
rlm@125 90 ** would be cool to get done before winston
rlm@126 91 - [X] enable greyscale bitmaps for touch -- 2 hours
rlm@126 92 - [X] use sawfish to auto-tile sense windows -- 6 hours
rlm@126 93 - [X] sawfish keybinding to automatically delete all sense windows
rlm@126 94 - [ ] directly change the UV-pixels to show sensor activation -- 2
rlm@126 95 days
rlm@126 96 - [ ] proof of concept C sense manipulation -- 2 days
rlm@126 97 - [ ] proof of concept GPU sense manipulation -- week
rlm@126 98 - [ ] fourier view of sound -- 2 or 3 days
rlm@146 99 - [ ] dancing music listener -- 1 day, depends on fourier
rlm@125 100
rlm@125 101 ** don't have to get done before winston
rlm@126 102 - [ ] write tests for integration -- 3 days
rlm@126 103 - [ ] usertime/gametime clock HUD display -- day
rlm@126 104 - [ ] find papers for each of the senses justifying my own
rlm@126 105 representation -- week
rlm@126 106 - [ ] show sensor maps in HUD display? -- 4 days
rlm@126 107 - [ ] show sensor maps in AWT display? -- 2 days
rlm@124 108
rlm@99 109
rlm@73 110 * Intro
rlm@73 111 So far, I've made the following senses --
rlm@73 112 - Vision
rlm@73 113 - Hearing
rlm@73 114 - Touch
rlm@73 115 - Proprioception
rlm@73 116
rlm@73 117 And one effector:
rlm@73 118 - Movement
rlm@73 119
rlm@73 120 However, the code so far has only enabled these senses, but has not
rlm@73 121 actually implemented them. For example, there is still a lot of work
rlm@73 122 to be done for vision. I need to be able to create an /eyeball/ in
rlm@73 123 simulation that can be moved around and see the world from different
rlm@73 124 angles. I also need to determine weather to use log-polar or cartesian
rlm@73 125 for the visual input, and I need to determine how/wether to
rlm@73 126 disceritise the visual input.
rlm@73 127
rlm@73 128 I also want to be able to visualize both the sensors and the
rlm@104 129 effectors in pretty pictures. This semi-retarted creature will be my
rlm@73 130 first attempt at bringing everything together.
rlm@73 131
rlm@73 132 * The creature's body
rlm@73 133
rlm@73 134 Still going to do an eve-like body in blender, but due to problems
rlm@104 135 importing the joints, etc into jMonkeyEngine3, I'm going to do all
rlm@73 136 the connecting here in clojure code, using the names of the individual
rlm@73 137 components and trial and error. Later, I'll maybe make some sort of
rlm@73 138 creature-building modifications to blender that support whatever
rlm@73 139 discreitized senses I'm going to make.
rlm@73 140
rlm@73 141 #+name: body-1
rlm@73 142 #+begin_src clojure
rlm@73 143 (ns cortex.silly
rlm@73 144 "let's play!"
rlm@73 145 {:author "Robert McIntyre"})
rlm@73 146
rlm@73 147 ;; TODO remove this!
rlm@73 148 (require 'cortex.import)
rlm@73 149 (cortex.import/mega-import-jme3)
rlm@151 150 (use '(cortex world util body hearing touch vision sense))
rlm@73 151
rlm@73 152 (rlm.rlm-commands/help)
rlm@99 153 (import java.awt.image.BufferedImage)
rlm@99 154 (import javax.swing.JPanel)
rlm@99 155 (import javax.swing.SwingUtilities)
rlm@99 156 (import java.awt.Dimension)
rlm@99 157 (import javax.swing.JFrame)
rlm@99 158 (import java.awt.Dimension)
rlm@106 159 (import com.aurellem.capture.RatchetTimer)
rlm@99 160 (declare joint-create)
rlm@108 161 (use 'clojure.contrib.def)
rlm@73 162
rlm@83 163
rlm@83 164 (defn load-bullet []
rlm@84 165 (let [sim (world (Node.) {} no-op no-op)]
rlm@102 166 (doto sim
rlm@102 167 (.enqueue
rlm@102 168 (fn []
rlm@102 169 (.stop sim)))
rlm@102 170 (.start))))
rlm@83 171
rlm@73 172 (defn load-blender-model
rlm@73 173 "Load a .blend file using an asset folder relative path."
rlm@73 174 [^String model]
rlm@73 175 (.loadModel
rlm@73 176 (doto (asset-manager)
rlm@73 177 (.registerLoader BlenderModelLoader (into-array String ["blend"])))
rlm@73 178 model))
rlm@73 179
rlm@78 180 (defn blender-to-jme
rlm@78 181 "Convert from Blender coordinates to JME coordinates"
rlm@78 182 [#^Vector3f in]
rlm@78 183 (Vector3f. (.getX in)
rlm@78 184 (.getZ in)
rlm@78 185 (- (.getY in))))
rlm@74 186
rlm@79 187
rlm@133 188
rlm@133 189
rlm@133 190
rlm@87 191 (defn world-to-local
rlm@87 192 "Convert the world coordinates into coordinates relative to the
rlm@87 193 object (i.e. local coordinates), taking into account the rotation
rlm@87 194 of object."
rlm@87 195 [#^Spatial object world-coordinate]
rlm@87 196 (let [out (Vector3f.)]
rlm@88 197 (.worldToLocal object world-coordinate out) out))
rlm@87 198
rlm@96 199 (defn local-to-world
rlm@96 200 "Convert the local coordinates into coordinates into world relative
rlm@96 201 coordinates"
rlm@96 202 [#^Spatial object local-coordinate]
rlm@96 203 (let [world-coordinate (Vector3f.)]
rlm@96 204 (.localToWorld object local-coordinate world-coordinate)
rlm@96 205 world-coordinate))
rlm@96 206
rlm@87 207 (defmulti joint-dispatch
rlm@87 208 "Translate blender pseudo-joints into real JME joints."
rlm@88 209 (fn [constraints & _]
rlm@87 210 (:type constraints)))
rlm@87 211
rlm@87 212 (defmethod joint-dispatch :point
rlm@87 213 [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87 214 (println-repl "creating POINT2POINT joint")
rlm@130 215 ;; bullet's point2point joints are BROKEN, so we must use the
rlm@130 216 ;; generic 6DOF joint instead of an actual Point2Point joint!
rlm@130 217
rlm@130 218 ;; should be able to do this:
rlm@130 219 (comment
rlm@130 220 (Point2PointJoint.
rlm@130 221 control-a
rlm@130 222 control-b
rlm@130 223 pivot-a
rlm@130 224 pivot-b))
rlm@130 225
rlm@130 226 ;; but instead we must do this:
rlm@130 227 (println-repl "substuting 6DOF joint for POINT2POINT joint!")
rlm@130 228 (doto
rlm@130 229 (SixDofJoint.
rlm@130 230 control-a
rlm@130 231 control-b
rlm@130 232 pivot-a
rlm@130 233 pivot-b
rlm@130 234 false)
rlm@130 235 (.setLinearLowerLimit Vector3f/ZERO)
rlm@130 236 (.setLinearUpperLimit Vector3f/ZERO)
rlm@130 237 ;;(.setAngularLowerLimit (Vector3f. 1 1 1))
rlm@130 238 ;;(.setAngularUpperLimit (Vector3f. 0 0 0))
rlm@130 239
rlm@130 240 ))
rlm@130 241
rlm@87 242
rlm@87 243 (defmethod joint-dispatch :hinge
rlm@87 244 [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87 245 (println-repl "creating HINGE joint")
rlm@87 246 (let [axis
rlm@87 247 (if-let
rlm@87 248 [axis (:axis constraints)]
rlm@87 249 axis
rlm@87 250 Vector3f/UNIT_X)
rlm@87 251 [limit-1 limit-2] (:limit constraints)
rlm@87 252 hinge-axis
rlm@87 253 (.mult
rlm@87 254 rotation
rlm@87 255 (blender-to-jme axis))]
rlm@87 256 (doto
rlm@87 257 (HingeJoint.
rlm@87 258 control-a
rlm@87 259 control-b
rlm@87 260 pivot-a
rlm@87 261 pivot-b
rlm@87 262 hinge-axis
rlm@87 263 hinge-axis)
rlm@87 264 (.setLimit limit-1 limit-2))))
rlm@87 265
rlm@87 266 (defmethod joint-dispatch :cone
rlm@87 267 [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87 268 (let [limit-xz (:limit-xz constraints)
rlm@87 269 limit-xy (:limit-xy constraints)
rlm@87 270 twist (:twist constraints)]
rlm@87 271
rlm@87 272 (println-repl "creating CONE joint")
rlm@87 273 (println-repl rotation)
rlm@87 274 (println-repl
rlm@87 275 "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0)))
rlm@87 276 (println-repl
rlm@87 277 "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0)))
rlm@87 278 (println-repl
rlm@87 279 "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1)))
rlm@87 280 (doto
rlm@87 281 (ConeJoint.
rlm@87 282 control-a
rlm@87 283 control-b
rlm@87 284 pivot-a
rlm@87 285 pivot-b
rlm@87 286 rotation
rlm@87 287 rotation)
rlm@87 288 (.setLimit (float limit-xz)
rlm@87 289 (float limit-xy)
rlm@87 290 (float twist)))))
rlm@87 291
rlm@88 292 (defn connect
rlm@87 293 "here are some examples:
rlm@87 294 {:type :point}
rlm@87 295 {:type :hinge :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}
rlm@87 296 (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
rlm@87 297
rlm@89 298 {:type :cone :limit-xz 0]
rlm@89 299 :limit-xy 0]
rlm@89 300 :twist 0]} (use XZY rotation mode in blender!)"
rlm@87 301 [#^Node obj-a #^Node obj-b #^Node joint]
rlm@87 302 (let [control-a (.getControl obj-a RigidBodyControl)
rlm@87 303 control-b (.getControl obj-b RigidBodyControl)
rlm@87 304 joint-center (.getWorldTranslation joint)
rlm@87 305 joint-rotation (.toRotationMatrix (.getWorldRotation joint))
rlm@87 306 pivot-a (world-to-local obj-a joint-center)
rlm@87 307 pivot-b (world-to-local obj-b joint-center)]
rlm@89 308
rlm@87 309 (if-let [constraints
rlm@87 310 (map-vals
rlm@87 311 eval
rlm@87 312 (read-string
rlm@87 313 (meta-data joint "joint")))]
rlm@89 314 ;; A side-effect of creating a joint registers
rlm@89 315 ;; it with both physics objects which in turn
rlm@89 316 ;; will register the joint with the physics system
rlm@89 317 ;; when the simulation is started.
rlm@87 318 (do
rlm@87 319 (println-repl "creating joint between"
rlm@87 320 (.getName obj-a) "and" (.getName obj-b))
rlm@87 321 (joint-dispatch constraints
rlm@87 322 control-a control-b
rlm@87 323 pivot-a pivot-b
rlm@87 324 joint-rotation))
rlm@87 325 (println-repl "could not find joint meta-data!"))))
rlm@87 326
rlm@130 327
rlm@130 328
rlm@130 329
rlm@78 330 (defn assemble-creature [#^Node pieces joints]
rlm@78 331 (dorun
rlm@78 332 (map
rlm@78 333 (fn [geom]
rlm@78 334 (let [physics-control
rlm@78 335 (RigidBodyControl.
rlm@78 336 (HullCollisionShape.
rlm@78 337 (.getMesh geom))
rlm@78 338 (if-let [mass (meta-data geom "mass")]
rlm@78 339 (do
rlm@78 340 (println-repl
rlm@78 341 "setting" (.getName geom) "mass to" (float mass))
rlm@78 342 (float mass))
rlm@78 343 (float 1)))]
rlm@78 344
rlm@78 345 (.addControl geom physics-control)))
rlm@78 346 (filter #(isa? (class %) Geometry )
rlm@78 347 (node-seq pieces))))
rlm@78 348 (dorun
rlm@78 349 (map
rlm@78 350 (fn [joint]
rlm@133 351 (let [[obj-a obj-b] (joint-targets pieces joint)]
rlm@88 352 (connect obj-a obj-b joint)))
rlm@78 353 joints))
rlm@78 354 pieces)
rlm@74 355
rlm@116 356 (declare blender-creature)
rlm@74 357
rlm@78 358 (def hand "Models/creature1/one.blend")
rlm@74 359
rlm@78 360 (def worm "Models/creature1/try-again.blend")
rlm@78 361
rlm@90 362 (def touch "Models/creature1/touch.blend")
rlm@90 363
rlm@90 364 (defn worm-model [] (load-blender-model worm))
rlm@90 365
rlm@80 366 (defn x-ray [#^ColorRGBA color]
rlm@80 367 (doto (Material. (asset-manager)
rlm@80 368 "Common/MatDefs/Misc/Unshaded.j3md")
rlm@80 369 (.setColor "Color" color)
rlm@80 370 (-> (.getAdditionalRenderState)
rlm@80 371 (.setDepthTest false))))
rlm@80 372
rlm@91 373 (defn colorful []
rlm@91 374 (.getChild (worm-model) "worm-21"))
rlm@90 375
rlm@90 376 (import jme3tools.converters.ImageToAwt)
rlm@90 377
rlm@90 378 (import ij.ImagePlus)
rlm@90 379
rlm@108 380 ;; Every Mesh has many triangles, each with its own index.
rlm@108 381 ;; Every vertex has its own index as well.
rlm@90 382
rlm@108 383 (defn tactile-sensor-image
rlm@110 384 "Return the touch-sensor distribution image in BufferedImage format,
rlm@110 385 or nil if it does not exist."
rlm@91 386 [#^Geometry obj]
rlm@110 387 (if-let [image-path (meta-data obj "touch")]
rlm@110 388 (ImageToAwt/convert
rlm@110 389 (.getImage
rlm@110 390 (.loadTexture
rlm@110 391 (asset-manager)
rlm@110 392 image-path))
rlm@110 393 false false 0)))
rlm@110 394
rlm@91 395
rlm@108 396
rlm@108 397 (defn triangle
rlm@112 398 "Get the triangle specified by triangle-index from the mesh within
rlm@112 399 bounds."
rlm@108 400 [#^Mesh mesh triangle-index]
rlm@108 401 (let [scratch (Triangle.)]
rlm@108 402 (.getTriangle mesh triangle-index scratch)
rlm@108 403 scratch))
rlm@108 404
rlm@108 405 (defn triangle-vertex-indices
rlm@108 406 "Get the triangle vertex indices of a given triangle from a given
rlm@108 407 mesh."
rlm@108 408 [#^Mesh mesh triangle-index]
rlm@108 409 (let [indices (int-array 3)]
rlm@108 410 (.getTriangle mesh triangle-index indices)
rlm@108 411 (vec indices)))
rlm@108 412
rlm@108 413 (defn vertex-UV-coord
rlm@108 414 "Get the uv-coordinates of the vertex named by vertex-index"
rlm@108 415 [#^Mesh mesh vertex-index]
rlm@108 416 (let [UV-buffer
rlm@108 417 (.getData
rlm@108 418 (.getBuffer
rlm@108 419 mesh
rlm@108 420 VertexBuffer$Type/TexCoord))]
rlm@108 421 [(.get UV-buffer (* vertex-index 2))
rlm@108 422 (.get UV-buffer (+ 1 (* vertex-index 2)))]))
rlm@108 423
rlm@108 424 (defn triangle-UV-coord
rlm@108 425 "Get the uv-cooridnates of the triangle's verticies."
rlm@108 426 [#^Mesh mesh width height triangle-index]
rlm@108 427 (map (fn [[u v]] (vector (* width u) (* height v)))
rlm@108 428 (map (partial vertex-UV-coord mesh)
rlm@108 429 (triangle-vertex-indices mesh triangle-index))))
rlm@91 430
rlm@102 431 (defn same-side?
rlm@102 432 "Given the points p1 and p2 and the reference point ref, is point p
rlm@102 433 on the same side of the line that goes through p1 and p2 as ref is?"
rlm@102 434 [p1 p2 ref p]
rlm@91 435 (<=
rlm@91 436 0
rlm@91 437 (.dot
rlm@91 438 (.cross (.subtract p2 p1) (.subtract p p1))
rlm@91 439 (.cross (.subtract p2 p1) (.subtract ref p1)))))
rlm@91 440
rlm@108 441 (defn triangle-seq [#^Triangle tri]
rlm@108 442 [(.get1 tri) (.get2 tri) (.get3 tri)])
rlm@108 443
rlm@108 444 (defn vector3f-seq [#^Vector3f v]
rlm@108 445 [(.getX v) (.getY v) (.getZ v)])
rlm@108 446
rlm@108 447 (defn inside-triangle?
rlm@108 448 "Is the point inside the triangle?"
rlm@108 449 {:author "Dylan Holmes"}
rlm@108 450 [#^Triangle tri #^Vector3f p]
rlm@108 451 (let [[vert-1 vert-2 vert-3] (triangle-seq tri)]
rlm@108 452 (and
rlm@108 453 (same-side? vert-1 vert-2 vert-3 p)
rlm@108 454 (same-side? vert-2 vert-3 vert-1 p)
rlm@108 455 (same-side? vert-3 vert-1 vert-2 p))))
rlm@108 456
rlm@94 457 (defn triangle->matrix4f
rlm@108 458 "Converts the triangle into a 4x4 matrix: The first three columns
rlm@108 459 contain the vertices of the triangle; the last contains the unit
rlm@108 460 normal of the triangle. The bottom row is filled with 1s."
rlm@94 461 [#^Triangle t]
rlm@94 462 (let [mat (Matrix4f.)
rlm@94 463 [vert-1 vert-2 vert-3]
rlm@94 464 ((comp vec map) #(.get t %) (range 3))
rlm@94 465 unit-normal (do (.calculateNormal t)(.getNormal t))
rlm@94 466 vertices [vert-1 vert-2 vert-3 unit-normal]]
rlm@94 467 (dorun
rlm@94 468 (for [row (range 4) col (range 3)]
rlm@94 469 (do
rlm@94 470 (.set mat col row (.get (vertices row)col))
rlm@94 471 (.set mat 3 row 1))))
rlm@94 472 mat))
rlm@94 473
rlm@94 474 (defn triangle-transformation
rlm@94 475 "Returns the affine transformation that converts each vertex in the
rlm@94 476 first triangle into the corresponding vertex in the second
rlm@94 477 triangle."
rlm@94 478 [#^Triangle tri-1 #^Triangle tri-2]
rlm@94 479 (.mult
rlm@94 480 (triangle->matrix4f tri-2)
rlm@94 481 (.invert (triangle->matrix4f tri-1))))
rlm@94 482
rlm@108 483 (defn point->vector2f [[u v]]
rlm@108 484 (Vector2f. u v))
rlm@94 485
rlm@94 486 (defn vector2f->vector3f [v]
rlm@94 487 (Vector3f. (.getX v) (.getY v) 0))
rlm@94 488
rlm@94 489 (defn map-triangle [f #^Triangle tri]
rlm@94 490 (Triangle.
rlm@94 491 (f 0 (.get1 tri))
rlm@94 492 (f 1 (.get2 tri))
rlm@94 493 (f 2 (.get3 tri))))
rlm@94 494
rlm@108 495 (defn points->triangle
rlm@108 496 "Convert a list of points into a triangle."
rlm@108 497 [points]
rlm@108 498 (apply #(Triangle. %1 %2 %3)
rlm@108 499 (map (fn [point]
rlm@108 500 (let [point (vec point)]
rlm@108 501 (Vector3f. (get point 0 0)
rlm@108 502 (get point 1 0)
rlm@108 503 (get point 2 0))))
rlm@108 504 (take 3 points))))
rlm@94 505
rlm@108 506 (defn convex-bounds
rlm@128 507 ;;dylan
rlm@128 508 "Returns the smallest square containing the given
rlm@128 509 vertices, as a vector of integers [left top width height]."
rlm@128 510 ;; "Dimensions of the smallest integer bounding square of the list of
rlm@128 511 ;; 2D verticies in the form: [x y width height]."
rlm@108 512 [uv-verts]
rlm@108 513 (let [xs (map first uv-verts)
rlm@108 514 ys (map second uv-verts)
rlm@108 515 x0 (Math/floor (apply min xs))
rlm@108 516 y0 (Math/floor (apply min ys))
rlm@108 517 x1 (Math/ceil (apply max xs))
rlm@108 518 y1 (Math/ceil (apply max ys))]
rlm@108 519 [x0 y0 (- x1 x0) (- y1 y0)]))
rlm@93 520
rlm@106 521 (defn sensors-in-triangle
rlm@128 522 ;;dylan
rlm@128 523 "Locate the touch sensors in the triangle, returning a map of their UV and geometry-relative coordinates."
rlm@128 524 ;;"Find the locations of the touch sensors within a triangle in both
rlm@128 525 ;; UV and gemoetry relative coordinates."
rlm@107 526 [image mesh tri-index]
rlm@107 527 (let [width (.getWidth image)
rlm@108 528 height (.getHeight image)
rlm@108 529 UV-vertex-coords (triangle-UV-coord mesh width height tri-index)
rlm@108 530 bounds (convex-bounds UV-vertex-coords)
rlm@108 531
rlm@108 532 cutout-triangle (points->triangle UV-vertex-coords)
rlm@108 533 UV-sensor-coords
rlm@108 534 (filter (comp (partial inside-triangle? cutout-triangle)
rlm@108 535 (fn [[u v]] (Vector3f. u v 0)))
rlm@108 536 (white-coordinates image bounds))
rlm@108 537 UV->geometry (triangle-transformation
rlm@108 538 cutout-triangle
rlm@108 539 (triangle mesh tri-index))
rlm@108 540 geometry-sensor-coords
rlm@108 541 (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0)))
rlm@108 542 UV-sensor-coords)]
rlm@108 543 {:UV UV-sensor-coords :geometry geometry-sensor-coords}))
rlm@107 544
rlm@108 545 (defn-memo locate-feelers
rlm@94 546 "Search the geometry's tactile UV image for touch sensors, returning
rlm@94 547 their positions in geometry-relative coordinates."
rlm@94 548 [#^Geometry geo]
rlm@108 549 (let [mesh (.getMesh geo)
rlm@108 550 num-triangles (.getTriangleCount mesh)]
rlm@108 551 (if-let [image (tactile-sensor-image geo)]
rlm@108 552 (map
rlm@108 553 (partial sensors-in-triangle image mesh)
rlm@108 554 (range num-triangles))
rlm@108 555 (repeat (.getTriangleCount mesh) {:UV nil :geometry nil}))))
rlm@102 556
rlm@102 557 (use 'clojure.contrib.def)
rlm@102 558
rlm@102 559 (defn-memo touch-topology [#^Gemoetry geo]
rlm@108 560 (vec (collapse (reduce concat (map :UV (locate-feelers geo))))))
rlm@108 561
rlm@108 562 (defn-memo feeler-coordinates [#^Geometry geo]
rlm@108 563 (vec (map :geometry (locate-feelers geo))))
rlm@102 564
rlm@97 565 (defn enable-touch [#^Geometry geo]
rlm@108 566 (let [feeler-coords (feeler-coordinates geo)
rlm@96 567 tris (triangles geo)
rlm@109 568 limit 0.1
rlm@109 569 ;;results (CollisionResults.)
rlm@109 570 ]
rlm@111 571 (if (empty? (touch-topology geo))
rlm@111 572 nil
rlm@111 573 (fn [node]
rlm@111 574 (let [sensor-origins
rlm@111 575 (map
rlm@111 576 #(map (partial local-to-world geo) %)
rlm@111 577 feeler-coords)
rlm@111 578 triangle-normals
rlm@111 579 (map (partial get-ray-direction geo)
rlm@111 580 tris)
rlm@111 581 rays
rlm@111 582 (flatten
rlm@111 583 (map (fn [origins norm]
rlm@111 584 (map #(doto (Ray. % norm)
rlm@97 585 (.setLimit limit)) origins))
rlm@111 586 sensor-origins triangle-normals))]
rlm@111 587 (vector
rlm@111 588 (touch-topology geo)
rlm@111 589 (vec
rlm@111 590 (for [ray rays]
rlm@111 591 (do
rlm@111 592 (let [results (CollisionResults.)]
rlm@111 593 (.collideWith node ray results)
rlm@111 594 (let [touch-objects
rlm@126 595 (filter #(not (= geo (.getGeometry %)))
rlm@126 596 results)]
rlm@126 597 (- 255
rlm@126 598 (if (empty? touch-objects) 255
rlm@126 599 (rem
rlm@126 600 (int
rlm@126 601 (* 255 (/ (.getDistance
rlm@126 602 (first touch-objects)) limit)))
rlm@126 603 256))))))))))))))
rlm@126 604
rlm@111 605
rlm@111 606 (defn touch [#^Node pieces]
rlm@111 607 (filter (comp not nil?)
rlm@111 608 (map enable-touch
rlm@111 609 (filter #(isa? (class %) Geometry)
rlm@111 610 (node-seq pieces)))))
rlm@94 611
rlm@109 612
rlm@111 613 (defn test-eye []
rlm@117 614 (.getChild
rlm@117 615 (.getChild (worm-model) "eyes")
rlm@117 616 "eye"))
rlm@111 617
rlm@111 618
rlm@123 619
rlm@123 620 ;; Ears work the same way as vision.
rlm@123 621
rlm@123 622 ;; (hearing creature) will return [init-functions
rlm@123 623 ;; sensor-functions]. The init functions each take the world and
rlm@123 624 ;; register a SoundProcessor that does foureier transforms on the
rlm@123 625 ;; incommong sound data, making it available to each sensor function.
rlm@123 626
rlm@123 627 (defn creature-ears
rlm@128 628 "Return the children of the creature's \"ears\" node."
rlm@128 629 ;;dylan
rlm@128 630 ;;"The ear nodes which are children of the \"ears\" node in the
rlm@128 631 ;;creature."
rlm@123 632 [#^Node creature]
rlm@123 633 (if-let [ear-node (.getChild creature "ears")]
rlm@123 634 (seq (.getChildren ear-node))
rlm@123 635 (do (println-repl "could not find ears node") [])))
rlm@123 636
rlm@116 637
rlm@128 638 ;;dylan (defn follow-sense, adjoin-sense, attach-stimuli,
rlm@128 639 ;;anchor-qualia, augment-organ, with-organ
rlm@117 640
rlm@117 641
rlm@123 642 (defn update-listener-velocity
rlm@123 643 "Update the listener's velocity every update loop."
rlm@123 644 [#^Spatial obj #^Listener lis]
rlm@123 645 (let [old-position (atom (.getLocation lis))]
rlm@123 646 (.addControl
rlm@123 647 obj
rlm@123 648 (proxy [AbstractControl] []
rlm@123 649 (controlUpdate [tpf]
rlm@123 650 (let [new-position (.getLocation lis)]
rlm@123 651 (.setVelocity
rlm@123 652 lis
rlm@123 653 (.mult (.subtract new-position @old-position)
rlm@123 654 (float (/ tpf))))
rlm@123 655 (reset! old-position new-position)))
rlm@123 656 (controlRender [_ _])))))
rlm@123 657
rlm@123 658 (import com.aurellem.capture.audio.AudioSendRenderer)
rlm@123 659
rlm@123 660 (defn attach-ear
rlm@123 661 [#^Application world #^Node creature #^Spatial ear continuation]
rlm@123 662 (let [target (closest-node creature ear)
rlm@123 663 lis (Listener.)
rlm@123 664 audio-renderer (.getAudioRenderer world)
rlm@123 665 sp (sound-processor continuation)]
rlm@123 666 (.setLocation lis (.getWorldTranslation ear))
rlm@123 667 (.setRotation lis (.getWorldRotation ear))
rlm@123 668 (bind-sense target lis)
rlm@123 669 (update-listener-velocity target lis)
rlm@123 670 (.addListener audio-renderer lis)
rlm@123 671 (.registerSoundProcessor audio-renderer lis sp)))
rlm@123 672
rlm@123 673 (defn enable-hearing
rlm@123 674 [#^Node creature #^Spatial ear]
rlm@123 675 (let [hearing-data (atom [])]
rlm@123 676 [(fn [world]
rlm@123 677 (attach-ear world creature ear
rlm@123 678 (fn [data]
rlm@123 679 (reset! hearing-data (vec data)))))
rlm@123 680 [(fn []
rlm@123 681 (let [data @hearing-data
rlm@123 682 topology
rlm@123 683 (vec (map #(vector % 0) (range 0 (count data))))
rlm@123 684 scaled-data
rlm@123 685 (vec
rlm@123 686 (map
rlm@123 687 #(rem (int (* 255 (/ (+ 1 %) 2))) 256)
rlm@123 688 data))]
rlm@123 689 [topology scaled-data]))
rlm@123 690 ]]))
rlm@123 691
rlm@123 692 (defn hearing
rlm@123 693 [#^Node creature]
rlm@123 694 (reduce
rlm@123 695 (fn [[init-a senses-a]
rlm@123 696 [init-b senses-b]]
rlm@123 697 [(conj init-a init-b)
rlm@123 698 (into senses-a senses-b)])
rlm@123 699 [[][]]
rlm@123 700 (for [ear (creature-ears creature)]
rlm@123 701 (enable-hearing creature ear))))
rlm@123 702
rlm@128 703
rlm@128 704
rlm@128 705
rlm@128 706
rlm@128 707
rlm@128 708 ;; lower level --- nodes
rlm@128 709 ;; closest-node "parse/compile-x" -> makes organ, which is spatial, fn pair
rlm@128 710
rlm@128 711 ;; higher level -- organs
rlm@128 712 ;;
rlm@128 713
rlm@128 714 ;; higher level --- sense/effector
rlm@128 715 ;; these are the functions that provide world i/o, chinese-room style
rlm@128 716
rlm@128 717
rlm@134 718
rlm@116 719
rlm@116 720 (defn blender-creature
rlm@116 721 "Return a creature with all joints in place."
rlm@116 722 [blender-path]
rlm@116 723 (let [model (load-blender-model blender-path)
rlm@134 724 joints (creature-joints model)]
rlm@134 725 (assemble-creature model joints)))
rlm@116 726
rlm@126 727 (defn gray-scale [num]
rlm@126 728 (+ num
rlm@126 729 (bit-shift-left num 8)
rlm@126 730 (bit-shift-left num 16)))
rlm@126 731
rlm@130 732 (defn debug-touch-window
rlm@103 733 "creates function that offers a debug view of sensor data"
rlm@103 734 []
rlm@103 735 (let [vi (view-image)]
rlm@103 736 (fn
rlm@103 737 [[coords sensor-data]]
rlm@103 738 (let [image (points->image coords)]
rlm@103 739 (dorun
rlm@103 740 (for [i (range (count coords))]
rlm@103 741 (.setRGB image ((coords i) 0) ((coords i) 1)
rlm@126 742 (gray-scale (sensor-data i)))))
rlm@126 743
rlm@126 744
rlm@103 745 (vi image)))))
rlm@103 746
rlm@118 747 (defn debug-vision-window
rlm@118 748 "creates function that offers a debug view of sensor data"
rlm@118 749 []
rlm@118 750 (let [vi (view-image)]
rlm@118 751 (fn
rlm@118 752 [[coords sensor-data]]
rlm@118 753 (let [image (points->image coords)]
rlm@118 754 (dorun
rlm@118 755 (for [i (range (count coords))]
rlm@118 756 (.setRGB image ((coords i) 0) ((coords i) 1)
rlm@118 757 (sensor-data i))))
rlm@118 758 (vi image)))))
rlm@118 759
rlm@123 760 (defn debug-hearing-window
rlm@123 761 "view audio data"
rlm@123 762 [height]
rlm@123 763 (let [vi (view-image)]
rlm@123 764 (fn [[coords sensor-data]]
rlm@123 765 (let [image (BufferedImage. (count coords) height
rlm@123 766 BufferedImage/TYPE_INT_RGB)]
rlm@123 767 (dorun
rlm@123 768 (for [x (range (count coords))]
rlm@123 769 (dorun
rlm@123 770 (for [y (range height)]
rlm@123 771 (let [raw-sensor (sensor-data x)]
rlm@126 772 (.setRGB image x y (gray-scale raw-sensor)))))))
rlm@126 773
rlm@123 774 (vi image)))))
rlm@123 775
rlm@123 776
rlm@123 777
rlm@106 778 ;;(defn test-touch [world creature]
rlm@83 779
rlm@78 780
rlm@123 781
rlm@123 782
rlm@130 783 ;; here's how motor-control/ proprioception will work: Each muscle is
rlm@130 784 ;; defined by a 1-D array of numbers (the "motor pool") each of which
rlm@130 785 ;; represent muscle fibers. A muscle also has a scalar :strength
rlm@130 786 ;; factor which determines how strong the muscle as a whole is.
rlm@130 787 ;; The effector function for a muscle takes a number < (count
rlm@130 788 ;; motor-pool) and that number is said to "activate" all the muscle
rlm@130 789 ;; fibers whose index is lower than the number. Each fiber will apply
rlm@130 790 ;; force in proportion to its value in the array. Lower values cause
rlm@130 791 ;; less force. The lower values can be put at the "beginning" of the
rlm@130 792 ;; 1-D array to simulate the layout of actual human muscles, which are
rlm@130 793 ;; capable of more percise movements when exerting less force.
rlm@129 794
rlm@130 795 ;; I don't know how to encode proprioception, so for now, just return
rlm@130 796 ;; a function for each joint that returns a triplet of floats which
rlm@130 797 ;; represent relative roll, pitch, and yaw. Write display code for
rlm@130 798 ;; this though.
rlm@130 799
rlm@147 800 (defn muscle-fiber-values
rlm@147 801 "get motor pool strengths"
rlm@130 802 [#^BufferedImage image]
rlm@147 803 (vec
rlm@147 804 (let [width (.getWidth image)]
rlm@147 805 (for [x (range width)]
rlm@147 806 (- 255
rlm@147 807 (bit-and
rlm@147 808 0x0000FF
rlm@147 809 (.getRGB image x 0)))))))
rlm@147 810
rlm@147 811
rlm@147 812 (defn creature-muscles
rlm@147 813 "Return the children of the creature's \"muscles\" node."
rlm@147 814 [#^Node creature]
rlm@147 815 (if-let [muscle-node (.getChild creature "muscles")]
rlm@147 816 (seq (.getChildren muscle-node))
rlm@147 817 (do (println-repl "could not find muscles node") [])))
rlm@147 818
rlm@147 819 (defn single-muscle [#^Node parts #^Node muscle]
rlm@147 820 (let [target (closest-node parts muscle)
rlm@147 821 axis
rlm@147 822 (.mult (.getWorldRotation muscle) Vector3f/UNIT_Y)
rlm@147 823 strength (meta-data muscle "strength")
rlm@147 824 image-name (read-string (meta-data muscle "muscle"))
rlm@147 825 image
rlm@147 826 (ImageToAwt/convert
rlm@147 827 (.getImage (.loadTexture (asset-manager) image-name))
rlm@147 828 false false 0)
rlm@147 829 fibers (muscle-fiber-values image)
rlm@147 830 fiber-integral (reductions + fibers)
rlm@147 831 force-index (vec
rlm@147 832 (map
rlm@147 833 #(float (* strength (/ % (last
rlm@147 834 fiber-integral))))
rlm@147 835 fiber-integral))
rlm@147 836 control (.getControl target RigidBodyControl)]
rlm@147 837 (fn [n]
rlm@147 838 (let [pool-index (min n (count fibers))]
rlm@148 839 (.applyTorque control (.mult axis (force-index n)))))))
rlm@147 840
rlm@147 841
rlm@147 842 (defn enable-muscles
rlm@147 843 "Must be called on a creature after RigidBodyControls have been
rlm@147 844 created."
rlm@147 845 [#^Node creature]
rlm@147 846 (let [muscles (creature-muscles creature)]
rlm@147 847 (for [muscle muscles]
rlm@147 848 (single-muscle creature muscle))))
rlm@130 849
rlm@106 850 (defn test-creature [thing]
rlm@106 851 (let [x-axis
rlm@106 852 (box 1 0.01 0.01 :physical? false :color ColorRGBA/Red)
rlm@106 853 y-axis
rlm@106 854 (box 0.01 1 0.01 :physical? false :color ColorRGBA/Green)
rlm@106 855 z-axis
rlm@106 856 (box 0.01 0.01 1 :physical? false :color ColorRGBA/Blue)
rlm@106 857 creature (blender-creature thing)
rlm@106 858 touch-nerves (touch creature)
rlm@130 859 touch-debug-windows (map (fn [_] (debug-touch-window)) touch-nerves)
rlm@121 860 [init-vision-fns vision-data] (vision creature)
rlm@121 861 vision-debug (map (fn [_] (debug-vision-window)) vision-data)
rlm@118 862 me (sphere 0.5 :color ColorRGBA/Blue :physical? false)
rlm@123 863 [init-hearing-fns hearing-senses] (hearing creature)
rlm@123 864 hearing-windows (map (fn [_] (debug-hearing-window 50))
rlm@123 865 hearing-senses)
rlm@124 866 bell (AudioNode. (asset-manager)
rlm@128 867 "Sounds/pure.wav" false)
rlm@130 868 prop (proprioception creature)
rlm@135 869 prop-debug (proprioception-debug-window)
rlm@148 870
rlm@148 871 muscle-fns (enable-muscles creature)
rlm@123 872 ;; dream
rlm@123 873
rlm@106 874 ]
rlm@143 875
rlm@143 876
rlm@143 877 (apply
rlm@143 878 world
rlm@143 879 (with-movement
rlm@143 880 (.getChild creature "worm-21")
rlm@143 881 ["key-r" "key-t"
rlm@143 882 "key-f" "key-g"
rlm@143 883 "key-v" "key-b"]
rlm@143 884 [10 10 10 10 1 1]
rlm@143 885 [(nodify [creature
rlm@143 886 (box 10 2 10 :position (Vector3f. 0 -9 0)
rlm@143 887 :color ColorRGBA/Gray :mass 0)
rlm@143 888 x-axis y-axis z-axis
rlm@143 889 me
rlm@143 890 ])
rlm@143 891 (merge standard-debug-controls
rlm@143 892 {"key-return"
rlm@143 893 (fn [_ value]
rlm@143 894 (if value
rlm@143 895 (do
rlm@143 896 (println-repl "play-sound")
rlm@148 897 (.play bell))))
rlm@148 898 "key-h"
rlm@148 899 (fn [_ value]
rlm@148 900 (if value
rlm@148 901 (do
rlm@148 902 (println-repl "muscle activating!")
rlm@148 903 ((first muscle-fns) 199))))
rlm@148 904
rlm@148 905 })
rlm@143 906 (fn [world]
rlm@143 907 (light-up-everything world)
rlm@143 908 (enable-debug world)
rlm@143 909 (dorun (map #(% world) init-vision-fns))
rlm@143 910 (dorun (map #(% world) init-hearing-fns))
rlm@143 911
rlm@143 912 (add-eye world
rlm@143 913 (attach-eye creature (test-eye))
rlm@143 914 (comp (view-image) BufferedImage!))
rlm@143 915
rlm@143 916 (add-eye world (.getCamera world) no-op)
rlm@145 917 ;;(set-gravity world (Vector3f. 0 0 0))
rlm@143 918 ;;(com.aurellem.capture.Capture/captureVideo
rlm@143 919 ;; world (file-str "/home/r/proj/ai-videos/hand"))
rlm@143 920 ;;(.setTimer world (RatchetTimer. 60))
rlm@143 921 (speed-up world)
rlm@148 922 (set-gravity world (Vector3f. 0 0 0))
rlm@143 923 )
rlm@143 924 (fn [world tpf]
rlm@143 925 ;;(dorun
rlm@143 926 ;; (map #(%1 %2) touch-nerves (repeat (.getRootNode world))))
rlm@143 927
rlm@143 928 (prop-debug (prop))
rlm@143 929
rlm@143 930 (dorun
rlm@143 931 (map #(%1 (%2 (.getRootNode world)))
rlm@143 932 touch-debug-windows touch-nerves))
rlm@143 933
rlm@143 934 (dorun
rlm@143 935 (map #(%1 (%2))
rlm@143 936 vision-debug vision-data))
rlm@143 937 (dorun
rlm@143 938 (map #(%1 (%2)) hearing-windows hearing-senses))
rlm@143 939
rlm@143 940
rlm@143 941 ;;(println-repl (vision-data))
rlm@143 942 (.setLocalTranslation me (.getLocation (.getCamera world)))
rlm@143 943
rlm@143 944
rlm@143 945 )]
rlm@106 946 ;;(let [timer (atom 0)]
rlm@106 947 ;; (fn [_ _]
rlm@106 948 ;; (swap! timer inc)
rlm@106 949 ;; (if (= (rem @timer 60) 0)
rlm@106 950 ;; (println-repl (float (/ @timer 60))))))
rlm@143 951 ))))
rlm@83 952
rlm@109 953
rlm@109 954
rlm@109 955
rlm@109 956
rlm@109 957
rlm@109 958
rlm@109 959
rlm@109 960
rlm@109 961 ;;; experiments in collisions
rlm@109 962
rlm@109 963
rlm@109 964
rlm@109 965 (defn collision-test []
rlm@110 966 (let [b-radius 1
rlm@110 967 b-position (Vector3f. 0 0 0)
rlm@109 968 obj-b (box 1 1 1 :color ColorRGBA/Blue
rlm@109 969 :position b-position
rlm@110 970 :mass 0)
rlm@110 971 node (nodify [obj-b])
rlm@110 972 bounds-b
rlm@110 973 (doto (Picture.)
rlm@110 974 (.setHeight 50)
rlm@110 975 (.setWidth 50)
rlm@110 976 (.setImage (asset-manager)
rlm@110 977 "Models/creature1/hand.png"
rlm@110 978 false
rlm@110 979 ))
rlm@110 980
rlm@110 981 ;;(Ray. (Vector3f. 0 -5 0) (.normalize (Vector3f. 0 1 0)))
rlm@110 982
rlm@110 983 collisions
rlm@110 984 (let [cr (CollisionResults.)]
rlm@110 985 (.collideWith node bounds-b cr)
rlm@110 986 (println (map #(.getContactPoint %) cr))
rlm@110 987 cr)
rlm@110 988
rlm@110 989 ;;collision-points
rlm@110 990 ;;(map #(sphere 0.1 :position (.getContactPoint %))
rlm@110 991 ;; collisions)
rlm@110 992
rlm@110 993 ;;node (nodify (conj collision-points obj-b))
rlm@110 994
rlm@109 995 sim
rlm@109 996 (world node
rlm@110 997 {"key-space"
rlm@130 998 (fn [_ value]
rlm@110 999 (if value
rlm@110 1000 (let [cr (CollisionResults.)]
rlm@110 1001 (.collideWith node bounds-b cr)
rlm@110 1002 (println-repl (map #(.getContactPoint %) cr))
rlm@110 1003 cr)))}
rlm@109 1004 no-op
rlm@109 1005 no-op)
rlm@109 1006
rlm@109 1007 ]
rlm@110 1008 sim
rlm@109 1009
rlm@109 1010 ))
rlm@109 1011
rlm@116 1012
rlm@116 1013 ;; the camera will stay in its initial position/rotation with relation
rlm@116 1014 ;; to the spatial.
rlm@116 1015
rlm@116 1016
rlm@117 1017 (defn follow-test
rlm@117 1018 "show a camera that stays in the same relative position to a blue cube."
rlm@117 1019 []
rlm@116 1020 (let [camera-pos (Vector3f. 0 30 0)
rlm@116 1021 rock (box 1 1 1 :color ColorRGBA/Blue
rlm@116 1022 :position (Vector3f. 0 10 0)
rlm@116 1023 :mass 30
rlm@116 1024 )
rlm@118 1025 rot (.getWorldRotation rock)
rlm@116 1026
rlm@116 1027 table (box 3 1 10 :color ColorRGBA/Gray :mass 0
rlm@116 1028 :position (Vector3f. 0 -3 0))]
rlm@116 1029
rlm@116 1030 (world
rlm@116 1031 (nodify [rock table])
rlm@116 1032 standard-debug-controls
rlm@116 1033 (fn [world]
rlm@116 1034 (let
rlm@116 1035 [cam (doto (.clone (.getCamera world))
rlm@116 1036 (.setLocation camera-pos)
rlm@116 1037 (.lookAt Vector3f/ZERO
rlm@116 1038 Vector3f/UNIT_X))]
rlm@123 1039 (bind-sense rock cam)
rlm@116 1040
rlm@116 1041 (.setTimer world (RatchetTimer. 60))
rlm@116 1042 (add-eye world cam (comp (view-image) BufferedImage!))
rlm@116 1043 (add-eye world (.getCamera world) no-op))
rlm@116 1044 )
rlm@118 1045 (fn [_ _] (println-repl rot)))))
rlm@116 1046
rlm@118 1047
rlm@123 1048
rlm@87 1049 #+end_src
rlm@83 1050
rlm@87 1051 #+results: body-1
rlm@133 1052 : #'cortex.silly/follow-test
rlm@78 1053
rlm@78 1054
rlm@78 1055 * COMMENT purgatory
rlm@78 1056 #+begin_src clojure
rlm@77 1057 (defn bullet-trans []
rlm@77 1058 (let [obj-a (sphere 0.5 :color ColorRGBA/Red
rlm@77 1059 :position (Vector3f. -10 5 0))
rlm@77 1060 obj-b (sphere 0.5 :color ColorRGBA/Blue
rlm@77 1061 :position (Vector3f. -10 -5 0)
rlm@77 1062 :mass 0)
rlm@77 1063 control-a (.getControl obj-a RigidBodyControl)
rlm@77 1064 control-b (.getControl obj-b RigidBodyControl)
rlm@77 1065 swivel
rlm@77 1066 (.toRotationMatrix
rlm@77 1067 (doto (Quaternion.)
rlm@77 1068 (.fromAngleAxis (/ Math/PI 2)
rlm@77 1069 Vector3f/UNIT_X)))]
rlm@77 1070 (doto
rlm@77 1071 (ConeJoint.
rlm@77 1072 control-a control-b
rlm@77 1073 (Vector3f. 0 5 0)
rlm@77 1074 (Vector3f. 0 -5 0)
rlm@77 1075 swivel swivel)
rlm@77 1076 (.setLimit (* 0.6 (/ Math/PI 4))
rlm@77 1077 (/ Math/PI 4)
rlm@77 1078 (* Math/PI 0.8)))
rlm@77 1079 (world (nodify
rlm@77 1080 [obj-a obj-b])
rlm@77 1081 standard-debug-controls
rlm@77 1082 enable-debug
rlm@77 1083 no-op)))
rlm@74 1084
rlm@74 1085
rlm@77 1086 (defn bullet-trans* []
rlm@77 1087 (let [obj-a (box 1.5 0.5 0.5 :color ColorRGBA/Red
rlm@77 1088 :position (Vector3f. 5 0 0)
rlm@77 1089 :mass 90)
rlm@77 1090 obj-b (sphere 0.5 :color ColorRGBA/Blue
rlm@77 1091 :position (Vector3f. -5 0 0)
rlm@77 1092 :mass 0)
rlm@77 1093 control-a (.getControl obj-a RigidBodyControl)
rlm@77 1094 control-b (.getControl obj-b RigidBodyControl)
rlm@77 1095 move-up? (atom nil)
rlm@77 1096 move-down? (atom nil)
rlm@77 1097 move-left? (atom nil)
rlm@77 1098 move-right? (atom nil)
rlm@77 1099 roll-left? (atom nil)
rlm@77 1100 roll-right? (atom nil)
rlm@77 1101 force 100
rlm@77 1102 swivel
rlm@77 1103 (.toRotationMatrix
rlm@77 1104 (doto (Quaternion.)
rlm@77 1105 (.fromAngleAxis (/ Math/PI 2)
rlm@77 1106 Vector3f/UNIT_X)))
rlm@77 1107 x-move
rlm@77 1108 (doto (Matrix3f.)
rlm@77 1109 (.fromStartEndVectors Vector3f/UNIT_X
rlm@77 1110 (.normalize (Vector3f. 1 1 0))))
rlm@77 1111
rlm@77 1112 timer (atom 0)]
rlm@77 1113 (doto
rlm@77 1114 (ConeJoint.
rlm@77 1115 control-a control-b
rlm@77 1116 (Vector3f. -8 0 0)
rlm@77 1117 (Vector3f. 2 0 0)
rlm@77 1118 ;;swivel swivel
rlm@77 1119 ;;Matrix3f/IDENTITY Matrix3f/IDENTITY
rlm@77 1120 x-move Matrix3f/IDENTITY
rlm@77 1121 )
rlm@77 1122 (.setCollisionBetweenLinkedBodys false)
rlm@77 1123 (.setLimit (* 1 (/ Math/PI 4)) ;; twist
rlm@77 1124 (* 1 (/ Math/PI 4)) ;; swing span in X-Y plane
rlm@77 1125 (* 0 (/ Math/PI 4)))) ;; swing span in Y-Z plane
rlm@77 1126 (world (nodify
rlm@77 1127 [obj-a obj-b])
rlm@77 1128 (merge standard-debug-controls
rlm@77 1129 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
rlm@77 1130 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
rlm@77 1131 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
rlm@77 1132 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
rlm@77 1133 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
rlm@77 1134 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})
rlm@77 1135
rlm@77 1136 (fn [world]
rlm@77 1137 (enable-debug world)
rlm@77 1138 (set-gravity world Vector3f/ZERO)
rlm@77 1139 )
rlm@77 1140
rlm@77 1141 (fn [world _]
rlm@77 1142
rlm@77 1143 (if @move-up?
rlm@77 1144 (.applyForce control-a
rlm@77 1145 (Vector3f. force 0 0)
rlm@77 1146 (Vector3f. 0 0 0)))
rlm@77 1147 (if @move-down?
rlm@77 1148 (.applyForce control-a
rlm@77 1149 (Vector3f. (- force) 0 0)
rlm@77 1150 (Vector3f. 0 0 0)))
rlm@77 1151 (if @move-left?
rlm@77 1152 (.applyForce control-a
rlm@77 1153 (Vector3f. 0 force 0)
rlm@77 1154 (Vector3f. 0 0 0)))
rlm@77 1155 (if @move-right?
rlm@77 1156 (.applyForce control-a
rlm@77 1157 (Vector3f. 0 (- force) 0)
rlm@77 1158 (Vector3f. 0 0 0)))
rlm@77 1159
rlm@77 1160 (if @roll-left?
rlm@77 1161 (.applyForce control-a
rlm@77 1162 (Vector3f. 0 0 force)
rlm@77 1163 (Vector3f. 0 0 0)))
rlm@77 1164 (if @roll-right?
rlm@77 1165 (.applyForce control-a
rlm@77 1166 (Vector3f. 0 0 (- force))
rlm@77 1167 (Vector3f. 0 0 0)))
rlm@77 1168
rlm@77 1169 (if (zero? (rem (swap! timer inc) 100))
rlm@77 1170 (.attachChild
rlm@77 1171 (.getRootNode world)
rlm@77 1172 (sphere 0.05 :color ColorRGBA/Yellow
rlm@77 1173 :physical? false :position
rlm@77 1174 (.getWorldTranslation obj-a)))))
rlm@77 1175 )
rlm@77 1176 ))
rlm@77 1177
rlm@94 1178 (defn transform-trianglesdsd
rlm@94 1179 "Transform that converts each vertex in the first triangle
rlm@94 1180 into the corresponding vertex in the second triangle."
rlm@94 1181 [#^Triangle tri-1 #^Triangle tri-2]
rlm@94 1182 (let [in [(.get1 tri-1)
rlm@94 1183 (.get2 tri-1)
rlm@94 1184 (.get3 tri-1)]
rlm@94 1185 out [(.get1 tri-2)
rlm@94 1186 (.get2 tri-2)
rlm@94 1187 (.get3 tri-2)]]
rlm@94 1188 (let [translate (doto (Matrix4f.) (.setTranslation (.negate (in 0))))
rlm@94 1189 in* [(.mult translate (in 0))
rlm@94 1190 (.mult translate (in 1))
rlm@94 1191 (.mult translate (in 2))]
rlm@94 1192 final-translation
rlm@94 1193 (doto (Matrix4f.)
rlm@94 1194 (.setTranslation (out 1)))
rlm@94 1195
rlm@94 1196 rotate-1
rlm@94 1197 (doto (Matrix3f.)
rlm@94 1198 (.fromStartEndVectors
rlm@94 1199 (.normalize
rlm@94 1200 (.subtract
rlm@94 1201 (in* 1) (in* 0)))
rlm@94 1202 (.normalize
rlm@94 1203 (.subtract
rlm@94 1204 (out 1) (out 0)))))
rlm@94 1205 in** [(.mult rotate-1 (in* 0))
rlm@94 1206 (.mult rotate-1 (in* 1))
rlm@94 1207 (.mult rotate-1 (in* 2))]
rlm@94 1208 scale-factor-1
rlm@94 1209 (.mult
rlm@94 1210 (.normalize
rlm@94 1211 (.subtract
rlm@94 1212 (out 1)
rlm@94 1213 (out 0)))
rlm@94 1214 (/ (.length
rlm@94 1215 (.subtract (out 1)
rlm@94 1216 (out 0)))
rlm@94 1217 (.length
rlm@94 1218 (.subtract (in** 1)
rlm@94 1219 (in** 0)))))
rlm@94 1220 scale-1 (doto (Matrix4f.) (.setScale scale-factor-1))
rlm@94 1221 in*** [(.mult scale-1 (in** 0))
rlm@94 1222 (.mult scale-1 (in** 1))
rlm@94 1223 (.mult scale-1 (in** 2))]
rlm@94 1224
rlm@94 1225
rlm@94 1226
rlm@94 1227
rlm@94 1228
rlm@94 1229 ]
rlm@94 1230
rlm@94 1231 (dorun (map println in))
rlm@94 1232 (println)
rlm@94 1233 (dorun (map println in*))
rlm@94 1234 (println)
rlm@94 1235 (dorun (map println in**))
rlm@94 1236 (println)
rlm@94 1237 (dorun (map println in***))
rlm@94 1238 (println)
rlm@94 1239
rlm@99 1240 ))))
rlm@94 1241
rlm@94 1242
rlm@106 1243 (defn world-setup [joint]
rlm@106 1244 (let [joint-position (Vector3f. 0 0 0)
rlm@106 1245 joint-rotation
rlm@106 1246 (.toRotationMatrix
rlm@106 1247 (.mult
rlm@106 1248 (doto (Quaternion.)
rlm@106 1249 (.fromAngleAxis
rlm@106 1250 (* 1 (/ Math/PI 4))
rlm@106 1251 (Vector3f. -1 0 0)))
rlm@106 1252 (doto (Quaternion.)
rlm@106 1253 (.fromAngleAxis
rlm@106 1254 (* 1 (/ Math/PI 2))
rlm@106 1255 (Vector3f. 0 0 1)))))
rlm@106 1256 top-position (.mult joint-rotation (Vector3f. 8 0 0))
rlm@106 1257
rlm@106 1258 origin (doto
rlm@106 1259 (sphere 0.1 :physical? false :color ColorRGBA/Cyan
rlm@106 1260 :position top-position))
rlm@106 1261 top (doto
rlm@106 1262 (sphere 0.1 :physical? false :color ColorRGBA/Yellow
rlm@106 1263 :position top-position)
rlm@106 1264
rlm@106 1265 (.addControl
rlm@106 1266 (RigidBodyControl.
rlm@106 1267 (CapsuleCollisionShape. 0.5 1.5 1) (float 20))))
rlm@106 1268 bottom (doto
rlm@106 1269 (sphere 0.1 :physical? false :color ColorRGBA/DarkGray
rlm@106 1270 :position (Vector3f. 0 0 0))
rlm@106 1271 (.addControl
rlm@106 1272 (RigidBodyControl.
rlm@106 1273 (CapsuleCollisionShape. 0.5 1.5 1) (float 0))))
rlm@106 1274 table (box 10 2 10 :position (Vector3f. 0 -20 0)
rlm@106 1275 :color ColorRGBA/Gray :mass 0)
rlm@106 1276 a (.getControl top RigidBodyControl)
rlm@106 1277 b (.getControl bottom RigidBodyControl)]
rlm@106 1278
rlm@106 1279 (cond
rlm@106 1280 (= joint :cone)
rlm@106 1281
rlm@106 1282 (doto (ConeJoint.
rlm@106 1283 a b
rlm@106 1284 (world-to-local top joint-position)
rlm@106 1285 (world-to-local bottom joint-position)
rlm@106 1286 joint-rotation
rlm@106 1287 joint-rotation
rlm@106 1288 )
rlm@106 1289
rlm@106 1290
rlm@106 1291 (.setLimit (* (/ 10) Math/PI)
rlm@106 1292 (* (/ 4) Math/PI)
rlm@106 1293 0)))
rlm@106 1294 [origin top bottom table]))
rlm@106 1295
rlm@106 1296 (defn test-joint [joint]
rlm@106 1297 (let [[origin top bottom floor] (world-setup joint)
rlm@106 1298 control (.getControl top RigidBodyControl)
rlm@106 1299 move-up? (atom false)
rlm@106 1300 move-down? (atom false)
rlm@106 1301 move-left? (atom false)
rlm@106 1302 move-right? (atom false)
rlm@106 1303 roll-left? (atom false)
rlm@106 1304 roll-right? (atom false)
rlm@106 1305 timer (atom 0)]
rlm@106 1306
rlm@106 1307 (world
rlm@106 1308 (nodify [top bottom floor origin])
rlm@106 1309 (merge standard-debug-controls
rlm@106 1310 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
rlm@106 1311 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
rlm@106 1312 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
rlm@106 1313 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
rlm@106 1314 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
rlm@106 1315 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})
rlm@106 1316
rlm@106 1317 (fn [world]
rlm@106 1318 (light-up-everything world)
rlm@106 1319 (enable-debug world)
rlm@106 1320 (set-gravity world (Vector3f. 0 0 0))
rlm@106 1321 )
rlm@106 1322
rlm@106 1323 (fn [world _]
rlm@106 1324 (if (zero? (rem (swap! timer inc) 100))
rlm@106 1325 (do
rlm@106 1326 ;; (println-repl @timer)
rlm@106 1327 (.attachChild (.getRootNode world)
rlm@106 1328 (sphere 0.05 :color ColorRGBA/Yellow
rlm@106 1329 :position (.getWorldTranslation top)
rlm@106 1330 :physical? false))
rlm@106 1331 (.attachChild (.getRootNode world)
rlm@106 1332 (sphere 0.05 :color ColorRGBA/LightGray
rlm@106 1333 :position (.getWorldTranslation bottom)
rlm@106 1334 :physical? false))))
rlm@106 1335
rlm@106 1336 (if @move-up?
rlm@106 1337 (.applyTorque control
rlm@106 1338 (.mult (.getPhysicsRotation control)
rlm@106 1339 (Vector3f. 0 0 10))))
rlm@106 1340 (if @move-down?
rlm@106 1341 (.applyTorque control
rlm@106 1342 (.mult (.getPhysicsRotation control)
rlm@106 1343 (Vector3f. 0 0 -10))))
rlm@106 1344 (if @move-left?
rlm@106 1345 (.applyTorque control
rlm@106 1346 (.mult (.getPhysicsRotation control)
rlm@106 1347 (Vector3f. 0 10 0))))
rlm@106 1348 (if @move-right?
rlm@106 1349 (.applyTorque control
rlm@106 1350 (.mult (.getPhysicsRotation control)
rlm@106 1351 (Vector3f. 0 -10 0))))
rlm@106 1352 (if @roll-left?
rlm@106 1353 (.applyTorque control
rlm@106 1354 (.mult (.getPhysicsRotation control)
rlm@106 1355 (Vector3f. -1 0 0))))
rlm@106 1356 (if @roll-right?
rlm@106 1357 (.applyTorque control
rlm@106 1358 (.mult (.getPhysicsRotation control)
rlm@106 1359 (Vector3f. 1 0 0))))))))
rlm@106 1360
rlm@99 1361
rlm@99 1362
rlm@107 1363 (defprotocol Frame
rlm@107 1364 (frame [this]))
rlm@107 1365
rlm@107 1366 (extend-type BufferedImage
rlm@107 1367 Frame
rlm@107 1368 (frame [image]
rlm@107 1369 (merge
rlm@107 1370 (apply
rlm@107 1371 hash-map
rlm@107 1372 (interleave
rlm@107 1373 (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]
rlm@107 1374 (vector x y)))
rlm@107 1375 (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]
rlm@107 1376 (let [data (.getRGB image x y)]
rlm@107 1377 (hash-map :r (bit-shift-right (bit-and 0xff0000 data) 16)
rlm@107 1378 :g (bit-shift-right (bit-and 0x00ff00 data) 8)
rlm@107 1379 :b (bit-and 0x0000ff data)))))))
rlm@107 1380 {:width (.getWidth image) :height (.getHeight image)})))
rlm@107 1381
rlm@107 1382
rlm@107 1383 (extend-type ImagePlus
rlm@107 1384 Frame
rlm@107 1385 (frame [image+]
rlm@107 1386 (frame (.getBufferedImage image+))))
rlm@107 1387
rlm@107 1388
rlm@99 1389 #+end_src
rlm@99 1390
rlm@99 1391
rlm@99 1392 * COMMENT generate source
rlm@99 1393 #+begin_src clojure :tangle ../src/cortex/silly.clj
rlm@99 1394 <<body-1>>
rlm@99 1395 #+end_src
rlm@99 1396
rlm@99 1397
rlm@94 1398
rlm@94 1399