Mercurial > cortex

annotate org/test-creature.org @ 133:2ed7e60d3821

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
FINALLY got proprioception working
author Robert McIntyre <rlm@mit.edu>
date Wed, 01 Feb 2012 02:27:18 -0700
parents b26017d1fe9a
children ac350a0ac6b0
rev   line source
rlm@73 1 #+title: First attempt at a creature!
rlm@73 2 #+author: Robert McIntyre
rlm@73 3 #+email: rlm@mit.edu
rlm@73 4 #+description:
rlm@73 5 #+keywords: simulation, jMonkeyEngine3, clojure
rlm@73 6 #+SETUPFILE: ../../aurellem/org/setup.org
rlm@73 7 #+INCLUDE: ../../aurellem/org/level-0.org
rlm@73 8
rlm@129 9
rlm@129 10
rlm@129 11 * Brainstorming different sensors and effectors.
rlm@129 12
rlm@129 13 Every sense that we have should have an effector that changes what
rlm@129 14 that sense (or others who have that sense) experiences.
rlm@129 15
rlm@129 16 ** Classic Senses
rlm@129 17 | Sense | Effector |
rlm@129 18 |------------------------------+---------------------------------|
rlm@129 19 | Vision | Variable Coloration |
rlm@129 20 | Hearing | Speech |
rlm@129 21 | Proprioception | Movement |
rlm@129 22 | Smell/Taste (Chemoreception) | Pheremones |
rlm@129 23 | Touch | Movement / Controllable Texture |
rlm@129 24 | Acceleration | Movement |
rlm@129 25 | Balance (sense gravity) | Movement |
rlm@129 26 | | |
rlm@129 27
rlm@129 28 - New Senses/Effectors
rlm@129 29 - Levitation
rlm@129 30 - Telekenesis
rlm@129 31
rlm@129 32 - Symbol Sense
rlm@129 33 Where objects in the world can be queried for description /
rlm@129 34 symbols.
rlm@129 35
rlm@129 36 - Symbol Marking
rlm@129 37 The ability to mark objects in the world with your own descriptions
rlm@129 38 and symbols.
rlm@129 39
rlm@129 40 - Vision
rlm@129 41 Distinguish the polarization of light
rlm@129 42 Color
rlm@129 43 Movement
rlm@129 44
rlm@129 45 * project ideas
rlm@129 46 - HACKER for writing muscle-control programs : Presented with
rlm@129 47 low-level muscle control/ sense API, generate higher level programs
rlm@129 48 for accomplishing various stated goals. Example goals might be
rlm@129 49 "extend all your fingers" or "move your hand into the area with
rlm@129 50 blue light" or "decrease the angle of this joint". It would be
rlm@129 51 like Sussman's HACKER, except it would operate with much more data
rlm@129 52 in a more realistic world. Start off with "calestanthics" to
rlm@129 53 develop subrouitines over the motor control API. This would be the
rlm@129 54 "spinal chord" of a more intelligent creature. The low level
rlm@129 55 programming code might be a turning machine that could develop
rlm@129 56 programs to iterate over a "tape" where each entry in the tape
rlm@129 57 could control recruitment of the fibers in a muscle.
rlm@129 58 - Make a virtual computer in the virtual world which with which the
rlm@129 59 creature interacts using its fingers to press keys on a virtual
rlm@129 60 keyboard. The creature can access the internet, watch videos, take
rlm@129 61 over the world, anything it wants.
rlm@129 62 - Make virtual insturments like pianos, drumbs, etc that it learns to
rlm@129 63 play.
rlm@129 64 - make a joint that figures out what type of joint it is (range of
rlm@129 65 motion)
rlm@129 66
rlm@129 67
rlm@129 68
rlm@129 69
rlm@129 70
rlm@129 71 * goals
rlm@125 72
rlm@125 73 ** have to get done before winston
rlm@126 74 - [ ] write an explination for why greyscale bitmaps for senses is
rlm@126 75 appropiate -- 1/2 day
rlm@126 76 - [ ] muscle control -- day
rlm@126 77 - [ ] proprioception sensor map in the style of the other senses -- day
rlm@125 78 - [ ] refactor integration code to distribute to each of the senses
rlm@126 79 -- day
rlm@126 80 - [ ] create video showing all the senses for Winston -- 2 days
rlm@126 81 - [ ] write summary of project for Winston \
rlm@126 82 - [ ] project proposals for Winston \
rlm@126 83 - [ ] additional senses to be implemented for Winston | -- 2 days
rlm@126 84 - [ ] send Winston package /
rlm@125 85
rlm@125 86 ** would be cool to get done before winston
rlm@126 87 - [X] enable greyscale bitmaps for touch -- 2 hours
rlm@126 88 - [X] use sawfish to auto-tile sense windows -- 6 hours
rlm@126 89 - [X] sawfish keybinding to automatically delete all sense windows
rlm@126 90 - [ ] directly change the UV-pixels to show sensor activation -- 2
rlm@126 91 days
rlm@126 92 - [ ] proof of concept C sense manipulation -- 2 days
rlm@126 93 - [ ] proof of concept GPU sense manipulation -- week
rlm@126 94 - [ ] fourier view of sound -- 2 or 3 days
rlm@127 95 - [ ] dancing music generator -- 1 day, depends on fourier
rlm@125 96
rlm@125 97 ** don't have to get done before winston
rlm@126 98 - [ ] write tests for integration -- 3 days
rlm@126 99 - [ ] usertime/gametime clock HUD display -- day
rlm@126 100 - [ ] find papers for each of the senses justifying my own
rlm@126 101 representation -- week
rlm@126 102 - [ ] show sensor maps in HUD display? -- 4 days
rlm@126 103 - [ ] show sensor maps in AWT display? -- 2 days
rlm@124 104
rlm@99 105
rlm@73 106 * Intro
rlm@73 107 So far, I've made the following senses --
rlm@73 108 - Vision
rlm@73 109 - Hearing
rlm@73 110 - Touch
rlm@73 111 - Proprioception
rlm@73 112
rlm@73 113 And one effector:
rlm@73 114 - Movement
rlm@73 115
rlm@73 116 However, the code so far has only enabled these senses, but has not
rlm@73 117 actually implemented them. For example, there is still a lot of work
rlm@73 118 to be done for vision. I need to be able to create an /eyeball/ in
rlm@73 119 simulation that can be moved around and see the world from different
rlm@73 120 angles. I also need to determine weather to use log-polar or cartesian
rlm@73 121 for the visual input, and I need to determine how/wether to
rlm@73 122 disceritise the visual input.
rlm@73 123
rlm@73 124 I also want to be able to visualize both the sensors and the
rlm@104 125 effectors in pretty pictures. This semi-retarted creature will be my
rlm@73 126 first attempt at bringing everything together.
rlm@73 127
rlm@73 128 * The creature's body
rlm@73 129
rlm@73 130 Still going to do an eve-like body in blender, but due to problems
rlm@104 131 importing the joints, etc into jMonkeyEngine3, I'm going to do all
rlm@73 132 the connecting here in clojure code, using the names of the individual
rlm@73 133 components and trial and error. Later, I'll maybe make some sort of
rlm@73 134 creature-building modifications to blender that support whatever
rlm@73 135 discreitized senses I'm going to make.
rlm@73 136
rlm@73 137 #+name: body-1
rlm@73 138 #+begin_src clojure
rlm@73 139 (ns cortex.silly
rlm@73 140 "let's play!"
rlm@73 141 {:author "Robert McIntyre"})
rlm@73 142
rlm@73 143 ;; TODO remove this!
rlm@73 144 (require 'cortex.import)
rlm@73 145 (cortex.import/mega-import-jme3)
rlm@73 146 (use '(cortex world util body hearing touch vision))
rlm@73 147
rlm@73 148 (rlm.rlm-commands/help)
rlm@99 149 (import java.awt.image.BufferedImage)
rlm@99 150 (import javax.swing.JPanel)
rlm@99 151 (import javax.swing.SwingUtilities)
rlm@99 152 (import java.awt.Dimension)
rlm@99 153 (import javax.swing.JFrame)
rlm@99 154 (import java.awt.Dimension)
rlm@106 155 (import com.aurellem.capture.RatchetTimer)
rlm@99 156 (declare joint-create)
rlm@108 157 (use 'clojure.contrib.def)
rlm@73 158
rlm@100 159 (defn points->image
rlm@100 160 "Take a sparse collection of points and visuliaze it as a
rlm@100 161 BufferedImage."
rlm@102 162
rlm@102 163 ;; TODO maybe parallelize this since it's easy
rlm@102 164
rlm@100 165 [points]
rlm@106 166 (if (empty? points)
rlm@106 167 (BufferedImage. 1 1 BufferedImage/TYPE_BYTE_BINARY)
rlm@106 168 (let [xs (vec (map first points))
rlm@106 169 ys (vec (map second points))
rlm@106 170 x0 (apply min xs)
rlm@106 171 y0 (apply min ys)
rlm@106 172 width (- (apply max xs) x0)
rlm@106 173 height (- (apply max ys) y0)
rlm@106 174 image (BufferedImage. (inc width) (inc height)
rlm@119 175 BufferedImage/TYPE_INT_RGB)]
rlm@118 176 (dorun
rlm@118 177 (for [x (range (.getWidth image))
rlm@118 178 y (range (.getHeight image))]
rlm@119 179 (.setRGB image x y 0xFF0000)))
rlm@106 180 (dorun
rlm@106 181 (for [index (range (count points))]
rlm@106 182 (.setRGB image (- (xs index) x0) (- (ys index) y0) -1)))
rlm@106 183
rlm@106 184 image)))
rlm@100 185
rlm@101 186 (defn average [coll]
rlm@101 187 (/ (reduce + coll) (count coll)))
rlm@101 188
rlm@101 189 (defn collapse-1d
rlm@101 190 "One dimensional analogue of collapse"
rlm@101 191 [center line]
rlm@101 192 (let [length (count line)
rlm@101 193 num-above (count (filter (partial < center) line))
rlm@101 194 num-below (- length num-above)]
rlm@101 195 (range (- center num-below)
rlm@115 196 (+ center num-above))))
rlm@99 197
rlm@99 198 (defn collapse
rlm@99 199 "Take a set of pairs of integers and collapse them into a
rlm@99 200 contigous bitmap."
rlm@99 201 [points]
rlm@108 202 (if (empty? points) []
rlm@108 203 (let
rlm@108 204 [num-points (count points)
rlm@108 205 center (vector
rlm@108 206 (int (average (map first points)))
rlm@108 207 (int (average (map first points))))
rlm@108 208 flattened
rlm@108 209 (reduce
rlm@108 210 concat
rlm@108 211 (map
rlm@108 212 (fn [column]
rlm@108 213 (map vector
rlm@108 214 (map first column)
rlm@108 215 (collapse-1d (second center)
rlm@108 216 (map second column))))
rlm@108 217 (partition-by first (sort-by first points))))
rlm@108 218 squeezed
rlm@108 219 (reduce
rlm@108 220 concat
rlm@108 221 (map
rlm@108 222 (fn [row]
rlm@108 223 (map vector
rlm@108 224 (collapse-1d (first center)
rlm@108 225 (map first row))
rlm@108 226 (map second row)))
rlm@108 227 (partition-by second (sort-by second flattened))))
rlm@108 228 relocate
rlm@108 229 (let [min-x (apply min (map first squeezed))
rlm@108 230 min-y (apply min (map second squeezed))]
rlm@108 231 (map (fn [[x y]]
rlm@108 232 [(- x min-x)
rlm@108 233 (- y min-y)])
rlm@108 234 squeezed))]
rlm@115 235 relocate)))
rlm@83 236
rlm@83 237 (defn load-bullet []
rlm@84 238 (let [sim (world (Node.) {} no-op no-op)]
rlm@102 239 (doto sim
rlm@102 240 (.enqueue
rlm@102 241 (fn []
rlm@102 242 (.stop sim)))
rlm@102 243 (.start))))
rlm@83 244
rlm@73 245 (defn load-blender-model
rlm@73 246 "Load a .blend file using an asset folder relative path."
rlm@73 247 [^String model]
rlm@73 248 (.loadModel
rlm@73 249 (doto (asset-manager)
rlm@73 250 (.registerLoader BlenderModelLoader (into-array String ["blend"])))
rlm@73 251 model))
rlm@73 252
rlm@74 253 (defn meta-data [blender-node key]
rlm@74 254 (if-let [data (.getUserData blender-node "properties")]
rlm@74 255 (.findValue data key)
rlm@74 256 nil))
rlm@73 257
rlm@78 258 (defn blender-to-jme
rlm@78 259 "Convert from Blender coordinates to JME coordinates"
rlm@78 260 [#^Vector3f in]
rlm@78 261 (Vector3f. (.getX in)
rlm@78 262 (.getZ in)
rlm@78 263 (- (.getY in))))
rlm@74 264
rlm@79 265 (defn jme-to-blender
rlm@79 266 "Convert from JME coordinates to Blender coordinates"
rlm@79 267 [#^Vector3f in]
rlm@79 268 (Vector3f. (.getX in)
rlm@79 269 (- (.getZ in))
rlm@79 270 (.getY in)))
rlm@79 271
rlm@78 272 (defn joint-targets
rlm@78 273 "Return the two closest two objects to the joint object, ordered
rlm@78 274 from bottom to top according to the joint's rotation."
rlm@78 275 [#^Node parts #^Node joint]
rlm@78 276 (loop [radius (float 0.01)]
rlm@78 277 (let [results (CollisionResults.)]
rlm@78 278 (.collideWith
rlm@78 279 parts
rlm@78 280 (BoundingBox. (.getWorldTranslation joint)
rlm@78 281 radius radius radius)
rlm@78 282 results)
rlm@78 283 (let [targets
rlm@78 284 (distinct
rlm@78 285 (map #(.getGeometry %) results))]
rlm@78 286 (if (>= (count targets) 2)
rlm@78 287 (sort-by
rlm@79 288 #(let [v
rlm@79 289 (jme-to-blender
rlm@79 290 (.mult
rlm@79 291 (.inverse (.getWorldRotation joint))
rlm@79 292 (.subtract (.getWorldTranslation %)
rlm@79 293 (.getWorldTranslation joint))))]
rlm@79 294 (println-repl (.getName %) ":" v)
rlm@79 295 (.dot (Vector3f. 1 1 1)
rlm@79 296 v))
rlm@78 297 (take 2 targets))
rlm@78 298 (recur (float (* radius 2))))))))
rlm@74 299
rlm@133 300
rlm@133 301 (defn proprio-joint [#^Node parts #^Node joint]
rlm@133 302 (let [[obj-a obj-b] (joint-targets parts joint)
rlm@133 303 joint-rot (.getWorldRotation joint)
rlm@133 304 x (.mult joint-rot Vector3f/UNIT_X)
rlm@133 305 y (.mult joint-rot Vector3f/UNIT_Y)
rlm@133 306 z (.mult joint-rot Vector3f/UNIT_Z)]
rlm@133 307 ;; this function will report proprioceptive information for the
rlm@133 308 ;; joint
rlm@133 309 (fn []
rlm@133 310 ;; x is the "twist" axis, y and z are the "bend" axes
rlm@133 311 (let [rot-a (.getWorldRotation obj-a)
rlm@133 312 rot-b (.getWorldRotation obj-b)
rlm@133 313 relative (.mult (.inverse rot-a) rot-b)
rlm@133 314 basis (doto (Matrix3f.)
rlm@133 315 (.setColumn 0 y)
rlm@133 316 (.setColumn 1 z)
rlm@133 317 (.setColumn 2 x))
rlm@133 318 rotation-about-joint
rlm@133 319 (doto (Quaternion.)
rlm@133 320 (.fromRotationMatrix
rlm@133 321 (.mult (.inverse basis)
rlm@133 322 (.toRotationMatrix relative))))
rlm@133 323
rlm@133 324 confirm-axes
rlm@133 325 (let [temp-axes (make-array Vector3f 3)]
rlm@133 326 (.toAxes rotation-about-joint temp-axes)
rlm@133 327 (seq temp-axes))
rlm@133 328 euler-angles
rlm@133 329 (seq (.toAngles rotation-about-joint nil))]
rlm@133 330 ;;return euler angles of the quaternion around the new basis
rlm@133 331 euler-angles))))
rlm@133 332
rlm@133 333
rlm@133 334
rlm@87 335 (defn world-to-local
rlm@87 336 "Convert the world coordinates into coordinates relative to the
rlm@87 337 object (i.e. local coordinates), taking into account the rotation
rlm@87 338 of object."
rlm@87 339 [#^Spatial object world-coordinate]
rlm@87 340 (let [out (Vector3f.)]
rlm@88 341 (.worldToLocal object world-coordinate out) out))
rlm@87 342
rlm@96 343 (defn local-to-world
rlm@96 344 "Convert the local coordinates into coordinates into world relative
rlm@96 345 coordinates"
rlm@96 346 [#^Spatial object local-coordinate]
rlm@96 347 (let [world-coordinate (Vector3f.)]
rlm@96 348 (.localToWorld object local-coordinate world-coordinate)
rlm@96 349 world-coordinate))
rlm@96 350
rlm@87 351 (defmulti joint-dispatch
rlm@87 352 "Translate blender pseudo-joints into real JME joints."
rlm@88 353 (fn [constraints & _]
rlm@87 354 (:type constraints)))
rlm@87 355
rlm@87 356 (defmethod joint-dispatch :point
rlm@87 357 [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87 358 (println-repl "creating POINT2POINT joint")
rlm@130 359 ;; bullet's point2point joints are BROKEN, so we must use the
rlm@130 360 ;; generic 6DOF joint instead of an actual Point2Point joint!
rlm@130 361
rlm@130 362 ;; should be able to do this:
rlm@130 363 (comment
rlm@130 364 (Point2PointJoint.
rlm@130 365 control-a
rlm@130 366 control-b
rlm@130 367 pivot-a
rlm@130 368 pivot-b))
rlm@130 369
rlm@130 370 ;; but instead we must do this:
rlm@130 371 (println-repl "substuting 6DOF joint for POINT2POINT joint!")
rlm@130 372 (doto
rlm@130 373 (SixDofJoint.
rlm@130 374 control-a
rlm@130 375 control-b
rlm@130 376 pivot-a
rlm@130 377 pivot-b
rlm@130 378 false)
rlm@130 379 (.setLinearLowerLimit Vector3f/ZERO)
rlm@130 380 (.setLinearUpperLimit Vector3f/ZERO)
rlm@130 381 ;;(.setAngularLowerLimit (Vector3f. 1 1 1))
rlm@130 382 ;;(.setAngularUpperLimit (Vector3f. 0 0 0))
rlm@130 383
rlm@130 384 ))
rlm@130 385
rlm@87 386
rlm@87 387 (defmethod joint-dispatch :hinge
rlm@87 388 [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87 389 (println-repl "creating HINGE joint")
rlm@87 390 (let [axis
rlm@87 391 (if-let
rlm@87 392 [axis (:axis constraints)]
rlm@87 393 axis
rlm@87 394 Vector3f/UNIT_X)
rlm@87 395 [limit-1 limit-2] (:limit constraints)
rlm@87 396 hinge-axis
rlm@87 397 (.mult
rlm@87 398 rotation
rlm@87 399 (blender-to-jme axis))]
rlm@87 400 (doto
rlm@87 401 (HingeJoint.
rlm@87 402 control-a
rlm@87 403 control-b
rlm@87 404 pivot-a
rlm@87 405 pivot-b
rlm@87 406 hinge-axis
rlm@87 407 hinge-axis)
rlm@87 408 (.setLimit limit-1 limit-2))))
rlm@87 409
rlm@87 410 (defmethod joint-dispatch :cone
rlm@87 411 [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87 412 (let [limit-xz (:limit-xz constraints)
rlm@87 413 limit-xy (:limit-xy constraints)
rlm@87 414 twist (:twist constraints)]
rlm@87 415
rlm@87 416 (println-repl "creating CONE joint")
rlm@87 417 (println-repl rotation)
rlm@87 418 (println-repl
rlm@87 419 "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0)))
rlm@87 420 (println-repl
rlm@87 421 "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0)))
rlm@87 422 (println-repl
rlm@87 423 "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1)))
rlm@87 424 (doto
rlm@87 425 (ConeJoint.
rlm@87 426 control-a
rlm@87 427 control-b
rlm@87 428 pivot-a
rlm@87 429 pivot-b
rlm@87 430 rotation
rlm@87 431 rotation)
rlm@87 432 (.setLimit (float limit-xz)
rlm@87 433 (float limit-xy)
rlm@87 434 (float twist)))))
rlm@87 435
rlm@88 436 (defn connect
rlm@87 437 "here are some examples:
rlm@87 438 {:type :point}
rlm@87 439 {:type :hinge :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}
rlm@87 440 (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
rlm@87 441
rlm@89 442 {:type :cone :limit-xz 0]
rlm@89 443 :limit-xy 0]
rlm@89 444 :twist 0]} (use XZY rotation mode in blender!)"
rlm@87 445 [#^Node obj-a #^Node obj-b #^Node joint]
rlm@87 446 (let [control-a (.getControl obj-a RigidBodyControl)
rlm@87 447 control-b (.getControl obj-b RigidBodyControl)
rlm@87 448 joint-center (.getWorldTranslation joint)
rlm@87 449 joint-rotation (.toRotationMatrix (.getWorldRotation joint))
rlm@87 450 pivot-a (world-to-local obj-a joint-center)
rlm@87 451 pivot-b (world-to-local obj-b joint-center)]
rlm@89 452
rlm@87 453 (if-let [constraints
rlm@87 454 (map-vals
rlm@87 455 eval
rlm@87 456 (read-string
rlm@87 457 (meta-data joint "joint")))]
rlm@89 458 ;; A side-effect of creating a joint registers
rlm@89 459 ;; it with both physics objects which in turn
rlm@89 460 ;; will register the joint with the physics system
rlm@89 461 ;; when the simulation is started.
rlm@87 462 (do
rlm@87 463 (println-repl "creating joint between"
rlm@87 464 (.getName obj-a) "and" (.getName obj-b))
rlm@87 465 (joint-dispatch constraints
rlm@87 466 control-a control-b
rlm@87 467 pivot-a pivot-b
rlm@87 468 joint-rotation))
rlm@87 469 (println-repl "could not find joint meta-data!"))))
rlm@87 470
rlm@130 471
rlm@130 472
rlm@130 473
rlm@78 474 (defn assemble-creature [#^Node pieces joints]
rlm@78 475 (dorun
rlm@78 476 (map
rlm@78 477 (fn [geom]
rlm@78 478 (let [physics-control
rlm@78 479 (RigidBodyControl.
rlm@78 480 (HullCollisionShape.
rlm@78 481 (.getMesh geom))
rlm@78 482 (if-let [mass (meta-data geom "mass")]
rlm@78 483 (do
rlm@78 484 (println-repl
rlm@78 485 "setting" (.getName geom) "mass to" (float mass))
rlm@78 486 (float mass))
rlm@78 487 (float 1)))]
rlm@78 488
rlm@78 489 (.addControl geom physics-control)))
rlm@78 490 (filter #(isa? (class %) Geometry )
rlm@78 491 (node-seq pieces))))
rlm@78 492 (dorun
rlm@78 493 (map
rlm@78 494 (fn [joint]
rlm@133 495 (let [[obj-a obj-b] (joint-targets pieces joint)]
rlm@88 496 (connect obj-a obj-b joint)))
rlm@78 497 joints))
rlm@78 498 pieces)
rlm@74 499
rlm@116 500 (declare blender-creature)
rlm@74 501
rlm@78 502 (def hand "Models/creature1/one.blend")
rlm@74 503
rlm@78 504 (def worm "Models/creature1/try-again.blend")
rlm@78 505
rlm@90 506 (def touch "Models/creature1/touch.blend")
rlm@90 507
rlm@90 508 (defn worm-model [] (load-blender-model worm))
rlm@90 509
rlm@80 510 (defn x-ray [#^ColorRGBA color]
rlm@80 511 (doto (Material. (asset-manager)
rlm@80 512 "Common/MatDefs/Misc/Unshaded.j3md")
rlm@80 513 (.setColor "Color" color)
rlm@80 514 (-> (.getAdditionalRenderState)
rlm@80 515 (.setDepthTest false))))
rlm@80 516
rlm@91 517 (defn colorful []
rlm@91 518 (.getChild (worm-model) "worm-21"))
rlm@90 519
rlm@90 520 (import jme3tools.converters.ImageToAwt)
rlm@90 521
rlm@90 522 (import ij.ImagePlus)
rlm@90 523
rlm@108 524 ;; Every Mesh has many triangles, each with its own index.
rlm@108 525 ;; Every vertex has its own index as well.
rlm@90 526
rlm@108 527 (defn tactile-sensor-image
rlm@110 528 "Return the touch-sensor distribution image in BufferedImage format,
rlm@110 529 or nil if it does not exist."
rlm@91 530 [#^Geometry obj]
rlm@110 531 (if-let [image-path (meta-data obj "touch")]
rlm@110 532 (ImageToAwt/convert
rlm@110 533 (.getImage
rlm@110 534 (.loadTexture
rlm@110 535 (asset-manager)
rlm@110 536 image-path))
rlm@110 537 false false 0)))
rlm@110 538
rlm@91 539 (import ij.process.ImageProcessor)
rlm@91 540 (import java.awt.image.BufferedImage)
rlm@91 541
rlm@92 542 (def white -1)
rlm@94 543
rlm@91 544 (defn filter-pixels
rlm@108 545 "List the coordinates of all pixels matching pred, within the bounds
rlm@108 546 provided. Bounds -> [x0 y0 width height]"
rlm@92 547 {:author "Dylan Holmes"}
rlm@108 548 ([pred #^BufferedImage image]
rlm@108 549 (filter-pixels pred image [0 0 (.getWidth image) (.getHeight image)]))
rlm@108 550 ([pred #^BufferedImage image [x0 y0 width height]]
rlm@108 551 ((fn accumulate [x y matches]
rlm@108 552 (cond
rlm@108 553 (>= y (+ height y0)) matches
rlm@108 554 (>= x (+ width x0)) (recur 0 (inc y) matches)
rlm@108 555 (pred (.getRGB image x y))
rlm@108 556 (recur (inc x) y (conj matches [x y]))
rlm@108 557 :else (recur (inc x) y matches)))
rlm@108 558 x0 y0 [])))
rlm@91 559
rlm@91 560 (defn white-coordinates
rlm@108 561 "Coordinates of all the white pixels in a subset of the image."
rlm@112 562 ([#^BufferedImage image bounds]
rlm@112 563 (filter-pixels #(= % white) image bounds))
rlm@112 564 ([#^BufferedImage image]
rlm@112 565 (filter-pixels #(= % white) image)))
rlm@108 566
rlm@108 567 (defn triangle
rlm@112 568 "Get the triangle specified by triangle-index from the mesh within
rlm@112 569 bounds."
rlm@108 570 [#^Mesh mesh triangle-index]
rlm@108 571 (let [scratch (Triangle.)]
rlm@108 572 (.getTriangle mesh triangle-index scratch)
rlm@108 573 scratch))
rlm@108 574
rlm@108 575 (defn triangle-vertex-indices
rlm@108 576 "Get the triangle vertex indices of a given triangle from a given
rlm@108 577 mesh."
rlm@108 578 [#^Mesh mesh triangle-index]
rlm@108 579 (let [indices (int-array 3)]
rlm@108 580 (.getTriangle mesh triangle-index indices)
rlm@108 581 (vec indices)))
rlm@108 582
rlm@108 583 (defn vertex-UV-coord
rlm@108 584 "Get the uv-coordinates of the vertex named by vertex-index"
rlm@108 585 [#^Mesh mesh vertex-index]
rlm@108 586 (let [UV-buffer
rlm@108 587 (.getData
rlm@108 588 (.getBuffer
rlm@108 589 mesh
rlm@108 590 VertexBuffer$Type/TexCoord))]
rlm@108 591 [(.get UV-buffer (* vertex-index 2))
rlm@108 592 (.get UV-buffer (+ 1 (* vertex-index 2)))]))
rlm@108 593
rlm@108 594 (defn triangle-UV-coord
rlm@108 595 "Get the uv-cooridnates of the triangle's verticies."
rlm@108 596 [#^Mesh mesh width height triangle-index]
rlm@108 597 (map (fn [[u v]] (vector (* width u) (* height v)))
rlm@108 598 (map (partial vertex-UV-coord mesh)
rlm@108 599 (triangle-vertex-indices mesh triangle-index))))
rlm@91 600
rlm@102 601 (defn same-side?
rlm@102 602 "Given the points p1 and p2 and the reference point ref, is point p
rlm@102 603 on the same side of the line that goes through p1 and p2 as ref is?"
rlm@102 604 [p1 p2 ref p]
rlm@91 605 (<=
rlm@91 606 0
rlm@91 607 (.dot
rlm@91 608 (.cross (.subtract p2 p1) (.subtract p p1))
rlm@91 609 (.cross (.subtract p2 p1) (.subtract ref p1)))))
rlm@91 610
rlm@108 611 (defn triangle-seq [#^Triangle tri]
rlm@108 612 [(.get1 tri) (.get2 tri) (.get3 tri)])
rlm@108 613
rlm@108 614 (defn vector3f-seq [#^Vector3f v]
rlm@108 615 [(.getX v) (.getY v) (.getZ v)])
rlm@108 616
rlm@108 617 (defn inside-triangle?
rlm@108 618 "Is the point inside the triangle?"
rlm@108 619 {:author "Dylan Holmes"}
rlm@108 620 [#^Triangle tri #^Vector3f p]
rlm@108 621 (let [[vert-1 vert-2 vert-3] (triangle-seq tri)]
rlm@108 622 (and
rlm@108 623 (same-side? vert-1 vert-2 vert-3 p)
rlm@108 624 (same-side? vert-2 vert-3 vert-1 p)
rlm@108 625 (same-side? vert-3 vert-1 vert-2 p))))
rlm@108 626
rlm@94 627 (defn triangle->matrix4f
rlm@108 628 "Converts the triangle into a 4x4 matrix: The first three columns
rlm@108 629 contain the vertices of the triangle; the last contains the unit
rlm@108 630 normal of the triangle. The bottom row is filled with 1s."
rlm@94 631 [#^Triangle t]
rlm@94 632 (let [mat (Matrix4f.)
rlm@94 633 [vert-1 vert-2 vert-3]
rlm@94 634 ((comp vec map) #(.get t %) (range 3))
rlm@94 635 unit-normal (do (.calculateNormal t)(.getNormal t))
rlm@94 636 vertices [vert-1 vert-2 vert-3 unit-normal]]
rlm@94 637 (dorun
rlm@94 638 (for [row (range 4) col (range 3)]
rlm@94 639 (do
rlm@94 640 (.set mat col row (.get (vertices row)col))
rlm@94 641 (.set mat 3 row 1))))
rlm@94 642 mat))
rlm@94 643
rlm@94 644 (defn triangle-transformation
rlm@94 645 "Returns the affine transformation that converts each vertex in the
rlm@94 646 first triangle into the corresponding vertex in the second
rlm@94 647 triangle."
rlm@94 648 [#^Triangle tri-1 #^Triangle tri-2]
rlm@94 649 (.mult
rlm@94 650 (triangle->matrix4f tri-2)
rlm@94 651 (.invert (triangle->matrix4f tri-1))))
rlm@94 652
rlm@108 653 (defn point->vector2f [[u v]]
rlm@108 654 (Vector2f. u v))
rlm@94 655
rlm@94 656 (defn vector2f->vector3f [v]
rlm@94 657 (Vector3f. (.getX v) (.getY v) 0))
rlm@94 658
rlm@94 659 (defn map-triangle [f #^Triangle tri]
rlm@94 660 (Triangle.
rlm@94 661 (f 0 (.get1 tri))
rlm@94 662 (f 1 (.get2 tri))
rlm@94 663 (f 2 (.get3 tri))))
rlm@94 664
rlm@108 665 (defn points->triangle
rlm@108 666 "Convert a list of points into a triangle."
rlm@108 667 [points]
rlm@108 668 (apply #(Triangle. %1 %2 %3)
rlm@108 669 (map (fn [point]
rlm@108 670 (let [point (vec point)]
rlm@108 671 (Vector3f. (get point 0 0)
rlm@108 672 (get point 1 0)
rlm@108 673 (get point 2 0))))
rlm@108 674 (take 3 points))))
rlm@94 675
rlm@108 676 (defn convex-bounds
rlm@128 677 ;;dylan
rlm@128 678 "Returns the smallest square containing the given
rlm@128 679 vertices, as a vector of integers [left top width height]."
rlm@128 680 ;; "Dimensions of the smallest integer bounding square of the list of
rlm@128 681 ;; 2D verticies in the form: [x y width height]."
rlm@108 682 [uv-verts]
rlm@108 683 (let [xs (map first uv-verts)
rlm@108 684 ys (map second uv-verts)
rlm@108 685 x0 (Math/floor (apply min xs))
rlm@108 686 y0 (Math/floor (apply min ys))
rlm@108 687 x1 (Math/ceil (apply max xs))
rlm@108 688 y1 (Math/ceil (apply max ys))]
rlm@108 689 [x0 y0 (- x1 x0) (- y1 y0)]))
rlm@93 690
rlm@106 691 (defn sensors-in-triangle
rlm@128 692 ;;dylan
rlm@128 693 "Locate the touch sensors in the triangle, returning a map of their UV and geometry-relative coordinates."
rlm@128 694 ;;"Find the locations of the touch sensors within a triangle in both
rlm@128 695 ;; UV and gemoetry relative coordinates."
rlm@107 696 [image mesh tri-index]
rlm@107 697 (let [width (.getWidth image)
rlm@108 698 height (.getHeight image)
rlm@108 699 UV-vertex-coords (triangle-UV-coord mesh width height tri-index)
rlm@108 700 bounds (convex-bounds UV-vertex-coords)
rlm@108 701
rlm@108 702 cutout-triangle (points->triangle UV-vertex-coords)
rlm@108 703 UV-sensor-coords
rlm@108 704 (filter (comp (partial inside-triangle? cutout-triangle)
rlm@108 705 (fn [[u v]] (Vector3f. u v 0)))
rlm@108 706 (white-coordinates image bounds))
rlm@108 707 UV->geometry (triangle-transformation
rlm@108 708 cutout-triangle
rlm@108 709 (triangle mesh tri-index))
rlm@108 710 geometry-sensor-coords
rlm@108 711 (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0)))
rlm@108 712 UV-sensor-coords)]
rlm@108 713 {:UV UV-sensor-coords :geometry geometry-sensor-coords}))
rlm@107 714
rlm@108 715 (defn-memo locate-feelers
rlm@94 716 "Search the geometry's tactile UV image for touch sensors, returning
rlm@94 717 their positions in geometry-relative coordinates."
rlm@94 718 [#^Geometry geo]
rlm@108 719 (let [mesh (.getMesh geo)
rlm@108 720 num-triangles (.getTriangleCount mesh)]
rlm@108 721 (if-let [image (tactile-sensor-image geo)]
rlm@108 722 (map
rlm@108 723 (partial sensors-in-triangle image mesh)
rlm@108 724 (range num-triangles))
rlm@108 725 (repeat (.getTriangleCount mesh) {:UV nil :geometry nil}))))
rlm@102 726
rlm@102 727 (use 'clojure.contrib.def)
rlm@102 728
rlm@102 729 (defn-memo touch-topology [#^Gemoetry geo]
rlm@108 730 (vec (collapse (reduce concat (map :UV (locate-feelers geo))))))
rlm@108 731
rlm@108 732 (defn-memo feeler-coordinates [#^Geometry geo]
rlm@108 733 (vec (map :geometry (locate-feelers geo))))
rlm@102 734
rlm@97 735 (defn enable-touch [#^Geometry geo]
rlm@108 736 (let [feeler-coords (feeler-coordinates geo)
rlm@96 737 tris (triangles geo)
rlm@109 738 limit 0.1
rlm@109 739 ;;results (CollisionResults.)
rlm@109 740 ]
rlm@111 741 (if (empty? (touch-topology geo))
rlm@111 742 nil
rlm@111 743 (fn [node]
rlm@111 744 (let [sensor-origins
rlm@111 745 (map
rlm@111 746 #(map (partial local-to-world geo) %)
rlm@111 747 feeler-coords)
rlm@111 748 triangle-normals
rlm@111 749 (map (partial get-ray-direction geo)
rlm@111 750 tris)
rlm@111 751 rays
rlm@111 752 (flatten
rlm@111 753 (map (fn [origins norm]
rlm@111 754 (map #(doto (Ray. % norm)
rlm@97 755 (.setLimit limit)) origins))
rlm@111 756 sensor-origins triangle-normals))]
rlm@111 757 (vector
rlm@111 758 (touch-topology geo)
rlm@111 759 (vec
rlm@111 760 (for [ray rays]
rlm@111 761 (do
rlm@111 762 (let [results (CollisionResults.)]
rlm@111 763 (.collideWith node ray results)
rlm@111 764 (let [touch-objects
rlm@126 765 (filter #(not (= geo (.getGeometry %)))
rlm@126 766 results)]
rlm@126 767 (- 255
rlm@126 768 (if (empty? touch-objects) 255
rlm@126 769 (rem
rlm@126 770 (int
rlm@126 771 (* 255 (/ (.getDistance
rlm@126 772 (first touch-objects)) limit)))
rlm@126 773 256))))))))))))))
rlm@126 774
rlm@111 775
rlm@111 776 (defn touch [#^Node pieces]
rlm@111 777 (filter (comp not nil?)
rlm@111 778 (map enable-touch
rlm@111 779 (filter #(isa? (class %) Geometry)
rlm@111 780 (node-seq pieces)))))
rlm@94 781
rlm@109 782
rlm@111 783 ;; human eye transmits 62kb/s to brain Bandwidth is 8.75 Mb/s
rlm@111 784 ;; http://en.wikipedia.org/wiki/Retina
rlm@109 785
rlm@111 786 (defn test-eye []
rlm@117 787 (.getChild
rlm@117 788 (.getChild (worm-model) "eyes")
rlm@117 789 "eye"))
rlm@111 790
rlm@111 791
rlm@111 792 (defn retina-sensor-image
rlm@111 793 "Return a map of pixel selection functions to BufferedImages
rlm@111 794 describing the distribution of light-sensitive components on this
rlm@111 795 geometry's surface. Each function creates an integer from the rgb
rlm@111 796 values found in the pixel. :red, :green, :blue, :gray are already
rlm@111 797 defined as extracting the red green blue and average components
rlm@111 798 respectively."
rlm@117 799 [#^Spatial eye]
rlm@111 800 (if-let [eye-map (meta-data eye "eye")]
rlm@111 801 (map-vals
rlm@111 802 #(ImageToAwt/convert
rlm@111 803 (.getImage (.loadTexture (asset-manager) %))
rlm@111 804 false false 0)
rlm@120 805 (eval (read-string eye-map)))))
rlm@111 806
rlm@117 807 (defn eye-dimensions
rlm@117 808 "returns the width and height specified in the metadata of the eye"
rlm@117 809 [#^Spatial eye]
rlm@117 810 (let [dimensions
rlm@117 811 (map #(vector (.getWidth %) (.getHeight %))
rlm@117 812 (vals (retina-sensor-image eye)))]
rlm@117 813 [(apply max (map first dimensions))
rlm@117 814 (apply max (map second dimensions))]))
rlm@117 815
rlm@116 816 (defn creature-eyes
rlm@128 817 ;;dylan
rlm@128 818 "Return the children of the creature's \"eyes\" node."
rlm@128 819 ;;"The eye nodes which are children of the \"eyes\" node in the
rlm@128 820 ;;creature."
rlm@116 821 [#^Node creature]
rlm@116 822 (if-let [eye-node (.getChild creature "eyes")]
rlm@116 823 (seq (.getChildren eye-node))
rlm@116 824 (do (println-repl "could not find eyes node") [])))
rlm@111 825
rlm@123 826 ;; Here's how vision will work.
rlm@112 827
rlm@123 828 ;; Make the continuation in scene-processor take FrameBuffer,
rlm@123 829 ;; byte-buffer, BufferedImage already sized to the correct
rlm@123 830 ;; dimensions. the continuation will decide wether to "mix" them
rlm@123 831 ;; into the BufferedImage, lazily ignore them, or mix them halfway
rlm@123 832 ;; and call c/graphics card routines.
rlm@112 833
rlm@123 834 ;; (vision creature) will take an optional :skip argument which will
rlm@123 835 ;; inform the continuations in scene processor to skip the given
rlm@123 836 ;; number of cycles; 0 means that no cycles will be skipped.
rlm@112 837
rlm@123 838 ;; (vision creature) will return [init-functions sensor-functions].
rlm@123 839 ;; The init-functions are each single-arg functions that take the
rlm@123 840 ;; world and register the cameras and must each be called before the
rlm@123 841 ;; corresponding sensor-functions. Each init-function returns the
rlm@123 842 ;; viewport for that eye which can be manipulated, saved, etc. Each
rlm@123 843 ;; sensor-function is a thunk and will return data in the same
rlm@123 844 ;; format as the tactile-sensor functions; the structure is
rlm@123 845 ;; [topology, sensor-data]. Internally, these sensor-functions
rlm@123 846 ;; maintain a reference to sensor-data which is periodically updated
rlm@123 847 ;; by the continuation function established by its init-function.
rlm@123 848 ;; They can be queried every cycle, but their information may not
rlm@123 849 ;; necessairly be different every cycle.
rlm@112 850
rlm@123 851 ;; Each eye in the creature in blender will work the same way as
rlm@123 852 ;; joints -- a zero dimensional object with no geometry whose local
rlm@123 853 ;; coordinate system determines the orientation of the resulting
rlm@123 854 ;; eye. All eyes will have a parent named "eyes" just as all joints
rlm@123 855 ;; have a parent named "joints". The resulting camera will be a
rlm@123 856 ;; ChaseCamera or a CameraNode bound to the geo that is closest to
rlm@123 857 ;; the eye marker. The eye marker will contain the metadata for the
rlm@123 858 ;; eye, and will be moved by it's bound geometry. The dimensions of
rlm@123 859 ;; the eye's camera are equal to the dimensions of the eye's "UV"
rlm@123 860 ;; map.
rlm@116 861
rlm@123 862
rlm@123 863 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
rlm@123 864
rlm@123 865 ;; Ears work the same way as vision.
rlm@123 866
rlm@123 867 ;; (hearing creature) will return [init-functions
rlm@123 868 ;; sensor-functions]. The init functions each take the world and
rlm@123 869 ;; register a SoundProcessor that does foureier transforms on the
rlm@123 870 ;; incommong sound data, making it available to each sensor function.
rlm@123 871
rlm@123 872 (defn creature-ears
rlm@128 873 "Return the children of the creature's \"ears\" node."
rlm@128 874 ;;dylan
rlm@128 875 ;;"The ear nodes which are children of the \"ears\" node in the
rlm@128 876 ;;creature."
rlm@123 877 [#^Node creature]
rlm@123 878 (if-let [ear-node (.getChild creature "ears")]
rlm@123 879 (seq (.getChildren ear-node))
rlm@123 880 (do (println-repl "could not find ears node") [])))
rlm@123 881
rlm@123 882 (defn closest-node
rlm@128 883 "Return the object in creature which is closest to the given node."
rlm@128 884 ;;dylan"The closest object in creature to the given node."
rlm@116 885 [#^Node creature #^Node eye]
rlm@116 886 (loop [radius (float 0.01)]
rlm@116 887 (let [results (CollisionResults.)]
rlm@116 888 (.collideWith
rlm@116 889 creature
rlm@116 890 (BoundingBox. (.getWorldTranslation eye)
rlm@116 891 radius radius radius)
rlm@116 892 results)
rlm@116 893 (if-let [target (first results)]
rlm@116 894 (.getGeometry target)
rlm@116 895 (recur (float (* 2 radius)))))))
rlm@116 896
rlm@128 897 ;;dylan (defn follow-sense, adjoin-sense, attach-stimuli,
rlm@128 898 ;;anchor-qualia, augment-organ, with-organ
rlm@123 899 (defn bind-sense
rlm@123 900 "Bind the sense to the Spatial such that it will maintain its
rlm@117 901 current position relative to the Spatial no matter how the spatial
rlm@123 902 moves. 'sense can be either a Camera or Listener object."
rlm@123 903 [#^Spatial obj sense]
rlm@123 904 (let [sense-offset (.subtract (.getLocation sense)
rlm@123 905 (.getWorldTranslation obj))
rlm@123 906 initial-sense-rotation (Quaternion. (.getRotation sense))
rlm@117 907 base-anti-rotation (.inverse (.getWorldRotation obj))]
rlm@117 908 (.addControl
rlm@117 909 obj
rlm@117 910 (proxy [AbstractControl] []
rlm@117 911 (controlUpdate [tpf]
rlm@117 912 (let [total-rotation
rlm@117 913 (.mult base-anti-rotation (.getWorldRotation obj))]
rlm@123 914 (.setLocation sense
rlm@117 915 (.add
rlm@123 916 (.mult total-rotation sense-offset)
rlm@117 917 (.getWorldTranslation obj)))
rlm@123 918 (.setRotation sense
rlm@123 919 (.mult total-rotation initial-sense-rotation))))
rlm@117 920 (controlRender [_ _])))))
rlm@117 921
rlm@117 922
rlm@123 923 (defn update-listener-velocity
rlm@123 924 "Update the listener's velocity every update loop."
rlm@123 925 [#^Spatial obj #^Listener lis]
rlm@123 926 (let [old-position (atom (.getLocation lis))]
rlm@123 927 (.addControl
rlm@123 928 obj
rlm@123 929 (proxy [AbstractControl] []
rlm@123 930 (controlUpdate [tpf]
rlm@123 931 (let [new-position (.getLocation lis)]
rlm@123 932 (.setVelocity
rlm@123 933 lis
rlm@123 934 (.mult (.subtract new-position @old-position)
rlm@123 935 (float (/ tpf))))
rlm@123 936 (reset! old-position new-position)))
rlm@123 937 (controlRender [_ _])))))
rlm@123 938
rlm@123 939 (import com.aurellem.capture.audio.AudioSendRenderer)
rlm@123 940
rlm@123 941 (defn attach-ear
rlm@123 942 [#^Application world #^Node creature #^Spatial ear continuation]
rlm@123 943 (let [target (closest-node creature ear)
rlm@123 944 lis (Listener.)
rlm@123 945 audio-renderer (.getAudioRenderer world)
rlm@123 946 sp (sound-processor continuation)]
rlm@123 947 (.setLocation lis (.getWorldTranslation ear))
rlm@123 948 (.setRotation lis (.getWorldRotation ear))
rlm@123 949 (bind-sense target lis)
rlm@123 950 (update-listener-velocity target lis)
rlm@123 951 (.addListener audio-renderer lis)
rlm@123 952 (.registerSoundProcessor audio-renderer lis sp)))
rlm@123 953
rlm@123 954 (defn enable-hearing
rlm@123 955 [#^Node creature #^Spatial ear]
rlm@123 956 (let [hearing-data (atom [])]
rlm@123 957 [(fn [world]
rlm@123 958 (attach-ear world creature ear
rlm@123 959 (fn [data]
rlm@123 960 (reset! hearing-data (vec data)))))
rlm@123 961 [(fn []
rlm@123 962 (let [data @hearing-data
rlm@123 963 topology
rlm@123 964 (vec (map #(vector % 0) (range 0 (count data))))
rlm@123 965 scaled-data
rlm@123 966 (vec
rlm@123 967 (map
rlm@123 968 #(rem (int (* 255 (/ (+ 1 %) 2))) 256)
rlm@123 969 data))]
rlm@123 970 [topology scaled-data]))
rlm@123 971 ]]))
rlm@123 972
rlm@123 973 (defn hearing
rlm@123 974 [#^Node creature]
rlm@123 975 (reduce
rlm@123 976 (fn [[init-a senses-a]
rlm@123 977 [init-b senses-b]]
rlm@123 978 [(conj init-a init-b)
rlm@123 979 (into senses-a senses-b)])
rlm@123 980 [[][]]
rlm@123 981 (for [ear (creature-ears creature)]
rlm@123 982 (enable-hearing creature ear))))
rlm@123 983
rlm@118 984 (defn attach-eye
rlm@118 985 "Attach a Camera to the appropiate area and return the Camera."
rlm@118 986 [#^Node creature #^Spatial eye]
rlm@123 987 (let [target (closest-node creature eye)
rlm@118 988 [cam-width cam-height] (eye-dimensions eye)
rlm@118 989 cam (Camera. cam-width cam-height)]
rlm@118 990 (.setLocation cam (.getWorldTranslation eye))
rlm@118 991 (.setRotation cam (.getWorldRotation eye))
rlm@119 992 (.setFrustumPerspective
rlm@119 993 cam 45 (/ (.getWidth cam) (.getHeight cam))
rlm@119 994 1 1000)
rlm@123 995 (bind-sense target cam)
rlm@118 996 cam))
rlm@118 997
rlm@118 998 (def presets
rlm@121 999 {:all 0xFFFFFF
rlm@119 1000 :red 0xFF0000
rlm@119 1001 :blue 0x0000FF
rlm@119 1002 :green 0x00FF00})
rlm@119 1003
rlm@118 1004 (defn enable-vision
rlm@118 1005 "return [init-function sensor-functions] for a particular eye"
rlm@118 1006 [#^Node creature #^Spatial eye & {skip :skip :or {skip 0}}]
rlm@118 1007 (let [retinal-map (retina-sensor-image eye)
rlm@123 1008 camera (attach-eye creature eye)
rlm@123 1009 vision-image
rlm@123 1010 (atom
rlm@123 1011 (BufferedImage. (.getWidth camera)
rlm@123 1012 (.getHeight camera)
rlm@123 1013 BufferedImage/TYPE_BYTE_BINARY))]
rlm@123 1014 [(fn [world]
rlm@123 1015 (add-eye
rlm@123 1016 world camera
rlm@123 1017 (let [counter (atom 0)]
rlm@123 1018 (fn [r fb bb bi]
rlm@123 1019 (if (zero? (rem (swap! counter inc) (inc skip)))
rlm@123 1020 (reset! vision-image (BufferedImage! r fb bb bi)))))))
rlm@123 1021 (vec
rlm@123 1022 (map
rlm@123 1023 (fn [[key image]]
rlm@123 1024 (let [whites (white-coordinates image)
rlm@123 1025 topology (vec (collapse whites))
rlm@123 1026 mask (presets key)]
rlm@123 1027 (fn []
rlm@123 1028 (vector
rlm@123 1029 topology
rlm@123 1030 (vec
rlm@123 1031 (for [[x y] whites]
rlm@123 1032 (bit-and
rlm@123 1033 mask (.getRGB @vision-image x y))))))))
rlm@123 1034 retinal-map))]))
rlm@118 1035
rlm@116 1036 (defn vision
rlm@121 1037 [#^Node creature & {skip :skip :or {skip 0}}]
rlm@121 1038 (reduce
rlm@121 1039 (fn [[init-a senses-a]
rlm@121 1040 [init-b senses-b]]
rlm@121 1041 [(conj init-a init-b)
rlm@121 1042 (into senses-a senses-b)])
rlm@121 1043 [[][]]
rlm@121 1044 (for [eye (creature-eyes creature)]
rlm@121 1045 (enable-vision creature eye))))
rlm@128 1046
rlm@128 1047
rlm@128 1048
rlm@128 1049
rlm@128 1050
rlm@128 1051 ;; lower level --- nodes
rlm@128 1052 ;; closest-node "parse/compile-x" -> makes organ, which is spatial, fn pair
rlm@128 1053
rlm@128 1054 ;; higher level -- organs
rlm@128 1055 ;;
rlm@128 1056
rlm@128 1057 ;; higher level --- sense/effector
rlm@128 1058 ;; these are the functions that provide world i/o, chinese-room style
rlm@128 1059
rlm@128 1060
rlm@116 1061
rlm@116 1062 (defn blender-creature
rlm@116 1063 "Return a creature with all joints in place."
rlm@116 1064 [blender-path]
rlm@116 1065 (let [model (load-blender-model blender-path)
rlm@116 1066 joints
rlm@116 1067 (if-let [joint-node (.getChild model "joints")]
rlm@116 1068 (seq (.getChildren joint-node))
rlm@116 1069 (do (println-repl "could not find joints node") []))]
rlm@116 1070 (assemble-creature model joints)))
rlm@116 1071
rlm@126 1072 (defn gray-scale [num]
rlm@126 1073 (+ num
rlm@126 1074 (bit-shift-left num 8)
rlm@126 1075 (bit-shift-left num 16)))
rlm@126 1076
rlm@130 1077 (defn debug-touch-window
rlm@103 1078 "creates function that offers a debug view of sensor data"
rlm@103 1079 []
rlm@103 1080 (let [vi (view-image)]
rlm@103 1081 (fn
rlm@103 1082 [[coords sensor-data]]
rlm@103 1083 (let [image (points->image coords)]
rlm@103 1084 (dorun
rlm@103 1085 (for [i (range (count coords))]
rlm@103 1086 (.setRGB image ((coords i) 0) ((coords i) 1)
rlm@126 1087 (gray-scale (sensor-data i)))))
rlm@126 1088
rlm@126 1089
rlm@103 1090 (vi image)))))
rlm@103 1091
rlm@118 1092 (defn debug-vision-window
rlm@118 1093 "creates function that offers a debug view of sensor data"
rlm@118 1094 []
rlm@118 1095 (let [vi (view-image)]
rlm@118 1096 (fn
rlm@118 1097 [[coords sensor-data]]
rlm@118 1098 (let [image (points->image coords)]
rlm@118 1099 (dorun
rlm@118 1100 (for [i (range (count coords))]
rlm@118 1101 (.setRGB image ((coords i) 0) ((coords i) 1)
rlm@118 1102 (sensor-data i))))
rlm@118 1103 (vi image)))))
rlm@118 1104
rlm@123 1105 (defn debug-hearing-window
rlm@123 1106 "view audio data"
rlm@123 1107 [height]
rlm@123 1108 (let [vi (view-image)]
rlm@123 1109 (fn [[coords sensor-data]]
rlm@123 1110 (let [image (BufferedImage. (count coords) height
rlm@123 1111 BufferedImage/TYPE_INT_RGB)]
rlm@123 1112 (dorun
rlm@123 1113 (for [x (range (count coords))]
rlm@123 1114 (dorun
rlm@123 1115 (for [y (range height)]
rlm@123 1116 (let [raw-sensor (sensor-data x)]
rlm@126 1117 (.setRGB image x y (gray-scale raw-sensor)))))))
rlm@126 1118
rlm@123 1119 (vi image)))))
rlm@123 1120
rlm@123 1121
rlm@123 1122
rlm@106 1123 ;;(defn test-touch [world creature]
rlm@83 1124
rlm@78 1125
rlm@123 1126
rlm@123 1127
rlm@130 1128 ;; here's how motor-control/ proprioception will work: Each muscle is
rlm@130 1129 ;; defined by a 1-D array of numbers (the "motor pool") each of which
rlm@130 1130 ;; represent muscle fibers. A muscle also has a scalar :strength
rlm@130 1131 ;; factor which determines how strong the muscle as a whole is.
rlm@130 1132 ;; The effector function for a muscle takes a number < (count
rlm@130 1133 ;; motor-pool) and that number is said to "activate" all the muscle
rlm@130 1134 ;; fibers whose index is lower than the number. Each fiber will apply
rlm@130 1135 ;; force in proportion to its value in the array. Lower values cause
rlm@130 1136 ;; less force. The lower values can be put at the "beginning" of the
rlm@130 1137 ;; 1-D array to simulate the layout of actual human muscles, which are
rlm@130 1138 ;; capable of more percise movements when exerting less force.
rlm@129 1139
rlm@130 1140 ;; I don't know how to encode proprioception, so for now, just return
rlm@130 1141 ;; a function for each joint that returns a triplet of floats which
rlm@130 1142 ;; represent relative roll, pitch, and yaw. Write display code for
rlm@130 1143 ;; this though.
rlm@130 1144
rlm@130 1145 (defn muscle-fibre-values
rlm@130 1146 "Take the first row of the image and return the low-order bytes."
rlm@130 1147 [#^BufferedImage image]
rlm@130 1148 (let [width (.getWidth image)]
rlm@130 1149 (for [x (range width)]
rlm@130 1150 (bit-and
rlm@130 1151 0xFF
rlm@130 1152 (.getRGB image x 0)))))
rlm@130 1153
rlm@130 1154
rlm@130 1155 (defn rad->deg [rad]
rlm@130 1156 (* 180 (/ Math/PI) rad))
rlm@130 1157
rlm@130 1158
rlm@130 1159 (defn debug-prop-window
rlm@130 1160 "create a debug view for proprioception"
rlm@130 1161 []
rlm@130 1162 (let [vi (view-image)]
rlm@130 1163 (fn [sensor-data]
rlm@130 1164 (println-repl
rlm@130 1165 (map
rlm@130 1166 (fn [[yaw pitch roll]]
rlm@130 1167 [(rad->deg yaw)
rlm@130 1168 (rad->deg pitch)
rlm@130 1169 (rad->deg roll)])
rlm@130 1170 sensor-data)))))
rlm@129 1171
rlm@129 1172
rlm@129 1173
rlm@129 1174
rlm@123 1175
rlm@123 1176
rlm@106 1177 (defn test-creature [thing]
rlm@106 1178 (let [x-axis
rlm@106 1179 (box 1 0.01 0.01 :physical? false :color ColorRGBA/Red)
rlm@106 1180 y-axis
rlm@106 1181 (box 0.01 1 0.01 :physical? false :color ColorRGBA/Green)
rlm@106 1182 z-axis
rlm@106 1183 (box 0.01 0.01 1 :physical? false :color ColorRGBA/Blue)
rlm@106 1184 creature (blender-creature thing)
rlm@106 1185 touch-nerves (touch creature)
rlm@130 1186 touch-debug-windows (map (fn [_] (debug-touch-window)) touch-nerves)
rlm@121 1187 [init-vision-fns vision-data] (vision creature)
rlm@121 1188 vision-debug (map (fn [_] (debug-vision-window)) vision-data)
rlm@118 1189 me (sphere 0.5 :color ColorRGBA/Blue :physical? false)
rlm@123 1190 [init-hearing-fns hearing-senses] (hearing creature)
rlm@123 1191 hearing-windows (map (fn [_] (debug-hearing-window 50))
rlm@123 1192 hearing-senses)
rlm@124 1193 bell (AudioNode. (asset-manager)
rlm@128 1194 "Sounds/pure.wav" false)
rlm@130 1195 prop (proprioception creature)
rlm@130 1196 prop-debug (debug-prop-window)
rlm@123 1197 ;; dream
rlm@123 1198
rlm@106 1199 ]
rlm@106 1200 (world
rlm@106 1201 (nodify [creature
rlm@106 1202 (box 10 2 10 :position (Vector3f. 0 -9 0)
rlm@106 1203 :color ColorRGBA/Gray :mass 0)
rlm@106 1204 x-axis y-axis z-axis
rlm@118 1205 me
rlm@106 1206 ])
rlm@123 1207 (merge standard-debug-controls
rlm@123 1208 {"key-return"
rlm@123 1209 (fn [_ value]
rlm@123 1210 (if value
rlm@123 1211 (do
rlm@123 1212 (println-repl "play-sound")
rlm@124 1213 (.play bell))))})
rlm@106 1214 (fn [world]
rlm@106 1215 (light-up-everything world)
rlm@106 1216 (enable-debug world)
rlm@122 1217 (dorun (map #(% world) init-vision-fns))
rlm@123 1218 (dorun (map #(% world) init-hearing-fns))
rlm@118 1219
rlm@118 1220 (add-eye world
rlm@118 1221 (attach-eye creature (test-eye))
rlm@118 1222 (comp (view-image) BufferedImage!))
rlm@118 1223
rlm@118 1224 (add-eye world (.getCamera world) no-op)
rlm@133 1225 ;;(set-gravity world (Vector3f. 0 0 0))
rlm@106 1226 ;;(com.aurellem.capture.Capture/captureVideo
rlm@106 1227 ;; world (file-str "/home/r/proj/ai-videos/hand"))
rlm@110 1228 ;;(.setTimer world (RatchetTimer. 60))
rlm@119 1229 (speed-up world)
rlm@106 1230 ;;(set-gravity world (Vector3f. 0 0 0))
rlm@106 1231 )
rlm@106 1232 (fn [world tpf]
rlm@109 1233 ;;(dorun
rlm@109 1234 ;; (map #(%1 %2) touch-nerves (repeat (.getRootNode world))))
rlm@123 1235
rlm@130 1236 (prop-debug (prop))
rlm@123 1237
rlm@106 1238 (dorun
rlm@109 1239 (map #(%1 (%2 (.getRootNode world)))
rlm@121 1240 touch-debug-windows touch-nerves))
rlm@123 1241
rlm@121 1242 (dorun
rlm@121 1243 (map #(%1 (%2))
rlm@121 1244 vision-debug vision-data))
rlm@123 1245 (dorun
rlm@123 1246 (map #(%1 (%2)) hearing-windows hearing-senses))
rlm@123 1247
rlm@123 1248
rlm@118 1249 ;;(println-repl (vision-data))
rlm@118 1250 (.setLocalTranslation me (.getLocation (.getCamera world)))
rlm@118 1251
rlm@121 1252
rlm@106 1253 )
rlm@106 1254 ;;(let [timer (atom 0)]
rlm@106 1255 ;; (fn [_ _]
rlm@106 1256 ;; (swap! timer inc)
rlm@106 1257 ;; (if (= (rem @timer 60) 0)
rlm@106 1258 ;; (println-repl (float (/ @timer 60))))))
rlm@106 1259 )))
rlm@83 1260
rlm@109 1261
rlm@109 1262
rlm@109 1263
rlm@109 1264
rlm@109 1265
rlm@109 1266
rlm@109 1267
rlm@109 1268
rlm@109 1269 ;;; experiments in collisions
rlm@109 1270
rlm@109 1271
rlm@109 1272
rlm@109 1273 (defn collision-test []
rlm@110 1274 (let [b-radius 1
rlm@110 1275 b-position (Vector3f. 0 0 0)
rlm@109 1276 obj-b (box 1 1 1 :color ColorRGBA/Blue
rlm@109 1277 :position b-position
rlm@110 1278 :mass 0)
rlm@110 1279 node (nodify [obj-b])
rlm@110 1280 bounds-b
rlm@110 1281 (doto (Picture.)
rlm@110 1282 (.setHeight 50)
rlm@110 1283 (.setWidth 50)
rlm@110 1284 (.setImage (asset-manager)
rlm@110 1285 "Models/creature1/hand.png"
rlm@110 1286 false
rlm@110 1287 ))
rlm@110 1288
rlm@110 1289 ;;(Ray. (Vector3f. 0 -5 0) (.normalize (Vector3f. 0 1 0)))
rlm@110 1290
rlm@110 1291 collisions
rlm@110 1292 (let [cr (CollisionResults.)]
rlm@110 1293 (.collideWith node bounds-b cr)
rlm@110 1294 (println (map #(.getContactPoint %) cr))
rlm@110 1295 cr)
rlm@110 1296
rlm@110 1297 ;;collision-points
rlm@110 1298 ;;(map #(sphere 0.1 :position (.getContactPoint %))
rlm@110 1299 ;; collisions)
rlm@110 1300
rlm@110 1301 ;;node (nodify (conj collision-points obj-b))
rlm@110 1302
rlm@109 1303 sim
rlm@109 1304 (world node
rlm@110 1305 {"key-space"
rlm@130 1306 (fn [_ value]
rlm@110 1307 (if value
rlm@110 1308 (let [cr (CollisionResults.)]
rlm@110 1309 (.collideWith node bounds-b cr)
rlm@110 1310 (println-repl (map #(.getContactPoint %) cr))
rlm@110 1311 cr)))}
rlm@109 1312 no-op
rlm@109 1313 no-op)
rlm@109 1314
rlm@109 1315 ]
rlm@110 1316 sim
rlm@109 1317
rlm@109 1318 ))
rlm@109 1319
rlm@116 1320
rlm@116 1321 ;; the camera will stay in its initial position/rotation with relation
rlm@116 1322 ;; to the spatial.
rlm@116 1323
rlm@116 1324
rlm@117 1325 (defn follow-test
rlm@117 1326 "show a camera that stays in the same relative position to a blue cube."
rlm@117 1327 []
rlm@116 1328 (let [camera-pos (Vector3f. 0 30 0)
rlm@116 1329 rock (box 1 1 1 :color ColorRGBA/Blue
rlm@116 1330 :position (Vector3f. 0 10 0)
rlm@116 1331 :mass 30
rlm@116 1332 )
rlm@118 1333 rot (.getWorldRotation rock)
rlm@116 1334
rlm@116 1335 table (box 3 1 10 :color ColorRGBA/Gray :mass 0
rlm@116 1336 :position (Vector3f. 0 -3 0))]
rlm@116 1337
rlm@116 1338 (world
rlm@116 1339 (nodify [rock table])
rlm@116 1340 standard-debug-controls
rlm@116 1341 (fn [world]
rlm@116 1342 (let
rlm@116 1343 [cam (doto (.clone (.getCamera world))
rlm@116 1344 (.setLocation camera-pos)
rlm@116 1345 (.lookAt Vector3f/ZERO
rlm@116 1346 Vector3f/UNIT_X))]
rlm@123 1347 (bind-sense rock cam)
rlm@116 1348
rlm@116 1349 (.setTimer world (RatchetTimer. 60))
rlm@116 1350 (add-eye world cam (comp (view-image) BufferedImage!))
rlm@116 1351 (add-eye world (.getCamera world) no-op))
rlm@116 1352 )
rlm@118 1353 (fn [_ _] (println-repl rot)))))
rlm@116 1354
rlm@118 1355
rlm@123 1356
rlm@87 1357 #+end_src
rlm@83 1358
rlm@87 1359 #+results: body-1
rlm@133 1360 : #'cortex.silly/follow-test
rlm@78 1361
rlm@78 1362
rlm@78 1363 * COMMENT purgatory
rlm@78 1364 #+begin_src clojure
rlm@77 1365 (defn bullet-trans []
rlm@77 1366 (let [obj-a (sphere 0.5 :color ColorRGBA/Red
rlm@77 1367 :position (Vector3f. -10 5 0))
rlm@77 1368 obj-b (sphere 0.5 :color ColorRGBA/Blue
rlm@77 1369 :position (Vector3f. -10 -5 0)
rlm@77 1370 :mass 0)
rlm@77 1371 control-a (.getControl obj-a RigidBodyControl)
rlm@77 1372 control-b (.getControl obj-b RigidBodyControl)
rlm@77 1373 swivel
rlm@77 1374 (.toRotationMatrix
rlm@77 1375 (doto (Quaternion.)
rlm@77 1376 (.fromAngleAxis (/ Math/PI 2)
rlm@77 1377 Vector3f/UNIT_X)))]
rlm@77 1378 (doto
rlm@77 1379 (ConeJoint.
rlm@77 1380 control-a control-b
rlm@77 1381 (Vector3f. 0 5 0)
rlm@77 1382 (Vector3f. 0 -5 0)
rlm@77 1383 swivel swivel)
rlm@77 1384 (.setLimit (* 0.6 (/ Math/PI 4))
rlm@77 1385 (/ Math/PI 4)
rlm@77 1386 (* Math/PI 0.8)))
rlm@77 1387 (world (nodify
rlm@77 1388 [obj-a obj-b])
rlm@77 1389 standard-debug-controls
rlm@77 1390 enable-debug
rlm@77 1391 no-op)))
rlm@74 1392
rlm@74 1393
rlm@77 1394 (defn bullet-trans* []
rlm@77 1395 (let [obj-a (box 1.5 0.5 0.5 :color ColorRGBA/Red
rlm@77 1396 :position (Vector3f. 5 0 0)
rlm@77 1397 :mass 90)
rlm@77 1398 obj-b (sphere 0.5 :color ColorRGBA/Blue
rlm@77 1399 :position (Vector3f. -5 0 0)
rlm@77 1400 :mass 0)
rlm@77 1401 control-a (.getControl obj-a RigidBodyControl)
rlm@77 1402 control-b (.getControl obj-b RigidBodyControl)
rlm@77 1403 move-up? (atom nil)
rlm@77 1404 move-down? (atom nil)
rlm@77 1405 move-left? (atom nil)
rlm@77 1406 move-right? (atom nil)
rlm@77 1407 roll-left? (atom nil)
rlm@77 1408 roll-right? (atom nil)
rlm@77 1409 force 100
rlm@77 1410 swivel
rlm@77 1411 (.toRotationMatrix
rlm@77 1412 (doto (Quaternion.)
rlm@77 1413 (.fromAngleAxis (/ Math/PI 2)
rlm@77 1414 Vector3f/UNIT_X)))
rlm@77 1415 x-move
rlm@77 1416 (doto (Matrix3f.)
rlm@77 1417 (.fromStartEndVectors Vector3f/UNIT_X
rlm@77 1418 (.normalize (Vector3f. 1 1 0))))
rlm@77 1419
rlm@77 1420 timer (atom 0)]
rlm@77 1421 (doto
rlm@77 1422 (ConeJoint.
rlm@77 1423 control-a control-b
rlm@77 1424 (Vector3f. -8 0 0)
rlm@77 1425 (Vector3f. 2 0 0)
rlm@77 1426 ;;swivel swivel
rlm@77 1427 ;;Matrix3f/IDENTITY Matrix3f/IDENTITY
rlm@77 1428 x-move Matrix3f/IDENTITY
rlm@77 1429 )
rlm@77 1430 (.setCollisionBetweenLinkedBodys false)
rlm@77 1431 (.setLimit (* 1 (/ Math/PI 4)) ;; twist
rlm@77 1432 (* 1 (/ Math/PI 4)) ;; swing span in X-Y plane
rlm@77 1433 (* 0 (/ Math/PI 4)))) ;; swing span in Y-Z plane
rlm@77 1434 (world (nodify
rlm@77 1435 [obj-a obj-b])
rlm@77 1436 (merge standard-debug-controls
rlm@77 1437 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
rlm@77 1438 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
rlm@77 1439 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
rlm@77 1440 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
rlm@77 1441 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
rlm@77 1442 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})
rlm@77 1443
rlm@77 1444 (fn [world]
rlm@77 1445 (enable-debug world)
rlm@77 1446 (set-gravity world Vector3f/ZERO)
rlm@77 1447 )
rlm@77 1448
rlm@77 1449 (fn [world _]
rlm@77 1450
rlm@77 1451 (if @move-up?
rlm@77 1452 (.applyForce control-a
rlm@77 1453 (Vector3f. force 0 0)
rlm@77 1454 (Vector3f. 0 0 0)))
rlm@77 1455 (if @move-down?
rlm@77 1456 (.applyForce control-a
rlm@77 1457 (Vector3f. (- force) 0 0)
rlm@77 1458 (Vector3f. 0 0 0)))
rlm@77 1459 (if @move-left?
rlm@77 1460 (.applyForce control-a
rlm@77 1461 (Vector3f. 0 force 0)
rlm@77 1462 (Vector3f. 0 0 0)))
rlm@77 1463 (if @move-right?
rlm@77 1464 (.applyForce control-a
rlm@77 1465 (Vector3f. 0 (- force) 0)
rlm@77 1466 (Vector3f. 0 0 0)))
rlm@77 1467
rlm@77 1468 (if @roll-left?
rlm@77 1469 (.applyForce control-a
rlm@77 1470 (Vector3f. 0 0 force)
rlm@77 1471 (Vector3f. 0 0 0)))
rlm@77 1472 (if @roll-right?
rlm@77 1473 (.applyForce control-a
rlm@77 1474 (Vector3f. 0 0 (- force))
rlm@77 1475 (Vector3f. 0 0 0)))
rlm@77 1476
rlm@77 1477 (if (zero? (rem (swap! timer inc) 100))
rlm@77 1478 (.attachChild
rlm@77 1479 (.getRootNode world)
rlm@77 1480 (sphere 0.05 :color ColorRGBA/Yellow
rlm@77 1481 :physical? false :position
rlm@77 1482 (.getWorldTranslation obj-a)))))
rlm@77 1483 )
rlm@77 1484 ))
rlm@77 1485
rlm@94 1486 (defn transform-trianglesdsd
rlm@94 1487 "Transform that converts each vertex in the first triangle
rlm@94 1488 into the corresponding vertex in the second triangle."
rlm@94 1489 [#^Triangle tri-1 #^Triangle tri-2]
rlm@94 1490 (let [in [(.get1 tri-1)
rlm@94 1491 (.get2 tri-1)
rlm@94 1492 (.get3 tri-1)]
rlm@94 1493 out [(.get1 tri-2)
rlm@94 1494 (.get2 tri-2)
rlm@94 1495 (.get3 tri-2)]]
rlm@94 1496 (let [translate (doto (Matrix4f.) (.setTranslation (.negate (in 0))))
rlm@94 1497 in* [(.mult translate (in 0))
rlm@94 1498 (.mult translate (in 1))
rlm@94 1499 (.mult translate (in 2))]
rlm@94 1500 final-translation
rlm@94 1501 (doto (Matrix4f.)
rlm@94 1502 (.setTranslation (out 1)))
rlm@94 1503
rlm@94 1504 rotate-1
rlm@94 1505 (doto (Matrix3f.)
rlm@94 1506 (.fromStartEndVectors
rlm@94 1507 (.normalize
rlm@94 1508 (.subtract
rlm@94 1509 (in* 1) (in* 0)))
rlm@94 1510 (.normalize
rlm@94 1511 (.subtract
rlm@94 1512 (out 1) (out 0)))))
rlm@94 1513 in** [(.mult rotate-1 (in* 0))
rlm@94 1514 (.mult rotate-1 (in* 1))
rlm@94 1515 (.mult rotate-1 (in* 2))]
rlm@94 1516 scale-factor-1
rlm@94 1517 (.mult
rlm@94 1518 (.normalize
rlm@94 1519 (.subtract
rlm@94 1520 (out 1)
rlm@94 1521 (out 0)))
rlm@94 1522 (/ (.length
rlm@94 1523 (.subtract (out 1)
rlm@94 1524 (out 0)))
rlm@94 1525 (.length
rlm@94 1526 (.subtract (in** 1)
rlm@94 1527 (in** 0)))))
rlm@94 1528 scale-1 (doto (Matrix4f.) (.setScale scale-factor-1))
rlm@94 1529 in*** [(.mult scale-1 (in** 0))
rlm@94 1530 (.mult scale-1 (in** 1))
rlm@94 1531 (.mult scale-1 (in** 2))]
rlm@94 1532
rlm@94 1533
rlm@94 1534
rlm@94 1535
rlm@94 1536
rlm@94 1537 ]
rlm@94 1538
rlm@94 1539 (dorun (map println in))
rlm@94 1540 (println)
rlm@94 1541 (dorun (map println in*))
rlm@94 1542 (println)
rlm@94 1543 (dorun (map println in**))
rlm@94 1544 (println)
rlm@94 1545 (dorun (map println in***))
rlm@94 1546 (println)
rlm@94 1547
rlm@99 1548 ))))
rlm@94 1549
rlm@94 1550
rlm@106 1551 (defn world-setup [joint]
rlm@106 1552 (let [joint-position (Vector3f. 0 0 0)
rlm@106 1553 joint-rotation
rlm@106 1554 (.toRotationMatrix
rlm@106 1555 (.mult
rlm@106 1556 (doto (Quaternion.)
rlm@106 1557 (.fromAngleAxis
rlm@106 1558 (* 1 (/ Math/PI 4))
rlm@106 1559 (Vector3f. -1 0 0)))
rlm@106 1560 (doto (Quaternion.)
rlm@106 1561 (.fromAngleAxis
rlm@106 1562 (* 1 (/ Math/PI 2))
rlm@106 1563 (Vector3f. 0 0 1)))))
rlm@106 1564 top-position (.mult joint-rotation (Vector3f. 8 0 0))
rlm@106 1565
rlm@106 1566 origin (doto
rlm@106 1567 (sphere 0.1 :physical? false :color ColorRGBA/Cyan
rlm@106 1568 :position top-position))
rlm@106 1569 top (doto
rlm@106 1570 (sphere 0.1 :physical? false :color ColorRGBA/Yellow
rlm@106 1571 :position top-position)
rlm@106 1572
rlm@106 1573 (.addControl
rlm@106 1574 (RigidBodyControl.
rlm@106 1575 (CapsuleCollisionShape. 0.5 1.5 1) (float 20))))
rlm@106 1576 bottom (doto
rlm@106 1577 (sphere 0.1 :physical? false :color ColorRGBA/DarkGray
rlm@106 1578 :position (Vector3f. 0 0 0))
rlm@106 1579 (.addControl
rlm@106 1580 (RigidBodyControl.
rlm@106 1581 (CapsuleCollisionShape. 0.5 1.5 1) (float 0))))
rlm@106 1582 table (box 10 2 10 :position (Vector3f. 0 -20 0)
rlm@106 1583 :color ColorRGBA/Gray :mass 0)
rlm@106 1584 a (.getControl top RigidBodyControl)
rlm@106 1585 b (.getControl bottom RigidBodyControl)]
rlm@106 1586
rlm@106 1587 (cond
rlm@106 1588 (= joint :cone)
rlm@106 1589
rlm@106 1590 (doto (ConeJoint.
rlm@106 1591 a b
rlm@106 1592 (world-to-local top joint-position)
rlm@106 1593 (world-to-local bottom joint-position)
rlm@106 1594 joint-rotation
rlm@106 1595 joint-rotation
rlm@106 1596 )
rlm@106 1597
rlm@106 1598
rlm@106 1599 (.setLimit (* (/ 10) Math/PI)
rlm@106 1600 (* (/ 4) Math/PI)
rlm@106 1601 0)))
rlm@106 1602 [origin top bottom table]))
rlm@106 1603
rlm@106 1604 (defn test-joint [joint]
rlm@106 1605 (let [[origin top bottom floor] (world-setup joint)
rlm@106 1606 control (.getControl top RigidBodyControl)
rlm@106 1607 move-up? (atom false)
rlm@106 1608 move-down? (atom false)
rlm@106 1609 move-left? (atom false)
rlm@106 1610 move-right? (atom false)
rlm@106 1611 roll-left? (atom false)
rlm@106 1612 roll-right? (atom false)
rlm@106 1613 timer (atom 0)]
rlm@106 1614
rlm@106 1615 (world
rlm@106 1616 (nodify [top bottom floor origin])
rlm@106 1617 (merge standard-debug-controls
rlm@106 1618 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
rlm@106 1619 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
rlm@106 1620 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
rlm@106 1621 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
rlm@106 1622 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
rlm@106 1623 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})
rlm@106 1624
rlm@106 1625 (fn [world]
rlm@106 1626 (light-up-everything world)
rlm@106 1627 (enable-debug world)
rlm@106 1628 (set-gravity world (Vector3f. 0 0 0))
rlm@106 1629 )
rlm@106 1630
rlm@106 1631 (fn [world _]
rlm@106 1632 (if (zero? (rem (swap! timer inc) 100))
rlm@106 1633 (do
rlm@106 1634 ;; (println-repl @timer)
rlm@106 1635 (.attachChild (.getRootNode world)
rlm@106 1636 (sphere 0.05 :color ColorRGBA/Yellow
rlm@106 1637 :position (.getWorldTranslation top)
rlm@106 1638 :physical? false))
rlm@106 1639 (.attachChild (.getRootNode world)
rlm@106 1640 (sphere 0.05 :color ColorRGBA/LightGray
rlm@106 1641 :position (.getWorldTranslation bottom)
rlm@106 1642 :physical? false))))
rlm@106 1643
rlm@106 1644 (if @move-up?
rlm@106 1645 (.applyTorque control
rlm@106 1646 (.mult (.getPhysicsRotation control)
rlm@106 1647 (Vector3f. 0 0 10))))
rlm@106 1648 (if @move-down?
rlm@106 1649 (.applyTorque control
rlm@106 1650 (.mult (.getPhysicsRotation control)
rlm@106 1651 (Vector3f. 0 0 -10))))
rlm@106 1652 (if @move-left?
rlm@106 1653 (.applyTorque control
rlm@106 1654 (.mult (.getPhysicsRotation control)
rlm@106 1655 (Vector3f. 0 10 0))))
rlm@106 1656 (if @move-right?
rlm@106 1657 (.applyTorque control
rlm@106 1658 (.mult (.getPhysicsRotation control)
rlm@106 1659 (Vector3f. 0 -10 0))))
rlm@106 1660 (if @roll-left?
rlm@106 1661 (.applyTorque control
rlm@106 1662 (.mult (.getPhysicsRotation control)
rlm@106 1663 (Vector3f. -1 0 0))))
rlm@106 1664 (if @roll-right?
rlm@106 1665 (.applyTorque control
rlm@106 1666 (.mult (.getPhysicsRotation control)
rlm@106 1667 (Vector3f. 1 0 0))))))))
rlm@106 1668
rlm@99 1669
rlm@99 1670
rlm@107 1671 (defprotocol Frame
rlm@107 1672 (frame [this]))
rlm@107 1673
rlm@107 1674 (extend-type BufferedImage
rlm@107 1675 Frame
rlm@107 1676 (frame [image]
rlm@107 1677 (merge
rlm@107 1678 (apply
rlm@107 1679 hash-map
rlm@107 1680 (interleave
rlm@107 1681 (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]
rlm@107 1682 (vector x y)))
rlm@107 1683 (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]
rlm@107 1684 (let [data (.getRGB image x y)]
rlm@107 1685 (hash-map :r (bit-shift-right (bit-and 0xff0000 data) 16)
rlm@107 1686 :g (bit-shift-right (bit-and 0x00ff00 data) 8)
rlm@107 1687 :b (bit-and 0x0000ff data)))))))
rlm@107 1688 {:width (.getWidth image) :height (.getHeight image)})))
rlm@107 1689
rlm@107 1690
rlm@107 1691 (extend-type ImagePlus
rlm@107 1692 Frame
rlm@107 1693 (frame [image+]
rlm@107 1694 (frame (.getBufferedImage image+))))
rlm@107 1695
rlm@107 1696
rlm@99 1697 #+end_src
rlm@99 1698
rlm@99 1699
rlm@99 1700 * COMMENT generate source
rlm@99 1701 #+begin_src clojure :tangle ../src/cortex/silly.clj
rlm@99 1702 <<body-1>>
rlm@99 1703 #+end_src
rlm@99 1704
rlm@99 1705
rlm@94 1706
rlm@94 1707