view org/test-creature.org @ 154:bb235258f835

deleted code from purgatory
author Robert McIntyre <rlm@mit.edu>
date Fri, 03 Feb 2012 06:01:22 -0700
parents c95179907951
children 95bf55614211
line wrap: on
line source
1 #+title: First attempt at a creature!
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+description:
5 #+keywords: simulation, jMonkeyEngine3, clojure
6 #+SETUPFILE: ../../aurellem/org/setup.org
7 #+INCLUDE: ../../aurellem/org/level-0.org
11 * Brainstorming different sensors and effectors.
13 Every sense that we have should have an effector that changes what
14 that sense (or others who have that sense) experiences.
16 ** Classic Senses
17 | Sense | Effector |
18 |------------------------------+---------------------------------|
19 | Vision | Variable Coloration |
20 | Hearing | Speech |
21 | Proprioception | Movement |
22 | Smell/Taste (Chemoreception) | Pheremones |
23 | Touch | Movement / Controllable Texture |
24 | Acceleration | Movement |
25 | Balance (sense gravity) | Movement |
26 | | |
28 - New Senses/Effectors
29 - Levitation
30 - Telekenesis
31 - control of gravity within a certain radius
32 - speed up/slow time
33 - object creation/destruction
35 - Symbol Sense
36 Where objects in the world can be queried for description /
37 symbols.
39 - Symbol Marking
40 The ability to mark objects in the world with your own descriptions
41 and symbols.
43 - Vision
44 Distinguish the polarization of light
45 Color
46 Movement
48 * project ideas
49 - HACKER for writing muscle-control programs : Presented with
50 low-level muscle control/ sense API, generate higher level programs
51 for accomplishing various stated goals. Example goals might be
52 "extend all your fingers" or "move your hand into the area with
53 blue light" or "decrease the angle of this joint". It would be
54 like Sussman's HACKER, except it would operate with much more data
55 in a more realistic world. Start off with "calestanthics" to
56 develop subrouitines over the motor control API. This would be the
57 "spinal chord" of a more intelligent creature. The low level
58 programming code might be a turning machine that could develop
59 programs to iterate over a "tape" where each entry in the tape
60 could control recruitment of the fibers in a muscle.
61 - Make a virtual computer in the virtual world which with which the
62 creature interacts using its fingers to press keys on a virtual
63 keyboard. The creature can access the internet, watch videos, take
64 over the world, anything it wants.
65 - Make virtual insturments like pianos, drumbs, etc that it learns to
66 play.
67 - make a joint that figures out what type of joint it is (range of
68 motion)
74 * goals
76 ** have to get done before winston
77 - [ ] write an explination for why greyscale bitmaps for senses is
78 appropiate -- 1/2 day
79 - [X] muscle control -- day
80 - [X] proprioception sensor map in the style of the other senses -- day
81 - [ ] refactor integration code to distribute to each of the senses
82 -- day
83 - [ ] create video showing all the senses for Winston -- 2 days
84 - [ ] send package to friends for critiques -- 2 days
85 - [ ] write summary of project for Winston \
86 - [ ] project proposals for Winston \
87 - [ ] additional senses to be implemented for Winston | -- 2 days
88 - [ ] send Winston package /
90 ** would be cool to get done before winston
91 - [X] enable greyscale bitmaps for touch -- 2 hours
92 - [X] use sawfish to auto-tile sense windows -- 6 hours
93 - [X] sawfish keybinding to automatically delete all sense windows
94 - [ ] directly change the UV-pixels to show sensor activation -- 2
95 days
96 - [ ] proof of concept C sense manipulation -- 2 days
97 - [ ] proof of concept GPU sense manipulation -- week
98 - [ ] fourier view of sound -- 2 or 3 days
99 - [ ] dancing music listener -- 1 day, depends on fourier
101 ** don't have to get done before winston
102 - [ ] write tests for integration -- 3 days
103 - [ ] usertime/gametime clock HUD display -- day
104 - [ ] find papers for each of the senses justifying my own
105 representation -- week
106 - [ ] show sensor maps in HUD display? -- 4 days
107 - [ ] show sensor maps in AWT display? -- 2 days
110 * Intro
111 So far, I've made the following senses --
112 - Vision
113 - Hearing
114 - Touch
115 - Proprioception
117 And one effector:
118 - Movement
120 However, the code so far has only enabled these senses, but has not
121 actually implemented them. For example, there is still a lot of work
122 to be done for vision. I need to be able to create an /eyeball/ in
123 simulation that can be moved around and see the world from different
124 angles. I also need to determine weather to use log-polar or cartesian
125 for the visual input, and I need to determine how/wether to
126 disceritise the visual input.
128 I also want to be able to visualize both the sensors and the
129 effectors in pretty pictures. This semi-retarted creature will be my
130 first attempt at bringing everything together.
132 * The creature's body
134 Still going to do an eve-like body in blender, but due to problems
135 importing the joints, etc into jMonkeyEngine3, I'm going to do all
136 the connecting here in clojure code, using the names of the individual
137 components and trial and error. Later, I'll maybe make some sort of
138 creature-building modifications to blender that support whatever
139 discreitized senses I'm going to make.
141 #+name: body-1
142 #+begin_src clojure
143 (ns cortex.silly
144 "let's play!"
145 {:author "Robert McIntyre"})
147 ;; TODO remove this!
148 (require 'cortex.import)
149 (cortex.import/mega-import-jme3)
150 (use '(cortex world util body hearing touch vision sense))
152 (rlm.rlm-commands/help)
153 (import java.awt.image.BufferedImage)
154 (import javax.swing.JPanel)
155 (import javax.swing.SwingUtilities)
156 (import java.awt.Dimension)
157 (import javax.swing.JFrame)
158 (import java.awt.Dimension)
159 (import com.aurellem.capture.RatchetTimer)
160 (declare joint-create)
161 (use 'clojure.contrib.def)
163 (defn load-blender-model
164 "Load a .blend file using an asset folder relative path."
165 [^String model]
166 (.loadModel
167 (doto (asset-manager)
168 (.registerLoader BlenderModelLoader (into-array String ["blend"])))
169 model))
171 (defn blender-to-jme
172 "Convert from Blender coordinates to JME coordinates"
173 [#^Vector3f in]
174 (Vector3f. (.getX in)
175 (.getZ in)
176 (- (.getY in))))
178 (defn world-to-local
179 "Convert the world coordinates into coordinates relative to the
180 object (i.e. local coordinates), taking into account the rotation
181 of object."
182 [#^Spatial object world-coordinate]
183 (.worldToLocal object world-coordinate nil))
185 (defn local-to-world
186 "Convert the local coordinates into coordinates into world relative
187 coordinates"
188 [#^Spatial object local-coordinate]
189 (.localToWorld object local-coordinate nil))
191 (defmulti joint-dispatch
192 "Translate blender pseudo-joints into real JME joints."
193 (fn [constraints & _]
194 (:type constraints)))
196 (defmethod joint-dispatch :point
197 [constraints control-a control-b pivot-a pivot-b rotation]
198 (println-repl "creating POINT2POINT joint")
199 ;; bullet's point2point joints are BROKEN, so we must use the
200 ;; generic 6DOF joint instead of an actual Point2Point joint!
202 ;; should be able to do this:
203 (comment
204 (Point2PointJoint.
205 control-a
206 control-b
207 pivot-a
208 pivot-b))
210 ;; but instead we must do this:
211 (println-repl "substuting 6DOF joint for POINT2POINT joint!")
212 (doto
213 (SixDofJoint.
214 control-a
215 control-b
216 pivot-a
217 pivot-b
218 false)
219 (.setLinearLowerLimit Vector3f/ZERO)
220 (.setLinearUpperLimit Vector3f/ZERO)
221 ;;(.setAngularLowerLimit (Vector3f. 1 1 1))
222 ;;(.setAngularUpperLimit (Vector3f. 0 0 0))
224 ))
227 (defmethod joint-dispatch :hinge
228 [constraints control-a control-b pivot-a pivot-b rotation]
229 (println-repl "creating HINGE joint")
230 (let [axis
231 (if-let
232 [axis (:axis constraints)]
233 axis
234 Vector3f/UNIT_X)
235 [limit-1 limit-2] (:limit constraints)
236 hinge-axis
237 (.mult
238 rotation
239 (blender-to-jme axis))]
240 (doto
241 (HingeJoint.
242 control-a
243 control-b
244 pivot-a
245 pivot-b
246 hinge-axis
247 hinge-axis)
248 (.setLimit limit-1 limit-2))))
250 (defmethod joint-dispatch :cone
251 [constraints control-a control-b pivot-a pivot-b rotation]
252 (let [limit-xz (:limit-xz constraints)
253 limit-xy (:limit-xy constraints)
254 twist (:twist constraints)]
256 (println-repl "creating CONE joint")
257 (println-repl rotation)
258 (println-repl
259 "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0)))
260 (println-repl
261 "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0)))
262 (println-repl
263 "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1)))
264 (doto
265 (ConeJoint.
266 control-a
267 control-b
268 pivot-a
269 pivot-b
270 rotation
271 rotation)
272 (.setLimit (float limit-xz)
273 (float limit-xy)
274 (float twist)))))
276 (defn connect
277 "here are some examples:
278 {:type :point}
279 {:type :hinge :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}
280 (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
282 {:type :cone :limit-xz 0]
283 :limit-xy 0]
284 :twist 0]} (use XZY rotation mode in blender!)"
285 [#^Node obj-a #^Node obj-b #^Node joint]
286 (let [control-a (.getControl obj-a RigidBodyControl)
287 control-b (.getControl obj-b RigidBodyControl)
288 joint-center (.getWorldTranslation joint)
289 joint-rotation (.toRotationMatrix (.getWorldRotation joint))
290 pivot-a (world-to-local obj-a joint-center)
291 pivot-b (world-to-local obj-b joint-center)]
293 (if-let [constraints
294 (map-vals
295 eval
296 (read-string
297 (meta-data joint "joint")))]
298 ;; A side-effect of creating a joint registers
299 ;; it with both physics objects which in turn
300 ;; will register the joint with the physics system
301 ;; when the simulation is started.
302 (do
303 (println-repl "creating joint between"
304 (.getName obj-a) "and" (.getName obj-b))
305 (joint-dispatch constraints
306 control-a control-b
307 pivot-a pivot-b
308 joint-rotation))
309 (println-repl "could not find joint meta-data!"))))
314 (defn assemble-creature [#^Node pieces joints]
315 (dorun
316 (map
317 (fn [geom]
318 (let [physics-control
319 (RigidBodyControl.
320 (HullCollisionShape.
321 (.getMesh geom))
322 (if-let [mass (meta-data geom "mass")]
323 (do
324 (println-repl
325 "setting" (.getName geom) "mass to" (float mass))
326 (float mass))
327 (float 1)))]
329 (.addControl geom physics-control)))
330 (filter #(isa? (class %) Geometry )
331 (node-seq pieces))))
332 (dorun
333 (map
334 (fn [joint]
335 (let [[obj-a obj-b] (joint-targets pieces joint)]
336 (connect obj-a obj-b joint)))
337 joints))
338 pieces)
340 (declare blender-creature)
342 (def hand "Models/creature1/one.blend")
344 (def worm "Models/creature1/try-again.blend")
346 (def touch "Models/creature1/touch.blend")
348 (defn worm-model [] (load-blender-model worm))
350 (defn x-ray [#^ColorRGBA color]
351 (doto (Material. (asset-manager)
352 "Common/MatDefs/Misc/Unshaded.j3md")
353 (.setColor "Color" color)
354 (-> (.getAdditionalRenderState)
355 (.setDepthTest false))))
357 (defn colorful []
358 (.getChild (worm-model) "worm-21"))
360 (import jme3tools.converters.ImageToAwt)
362 (import ij.ImagePlus)
364 ;; Every Mesh has many triangles, each with its own index.
365 ;; Every vertex has its own index as well.
367 (defn tactile-sensor-image
368 "Return the touch-sensor distribution image in BufferedImage format,
369 or nil if it does not exist."
370 [#^Geometry obj]
371 (if-let [image-path (meta-data obj "touch")]
372 (ImageToAwt/convert
373 (.getImage
374 (.loadTexture
375 (asset-manager)
376 image-path))
377 false false 0)))
381 (defn triangle
382 "Get the triangle specified by triangle-index from the mesh within
383 bounds."
384 [#^Mesh mesh triangle-index]
385 (let [scratch (Triangle.)]
386 (.getTriangle mesh triangle-index scratch)
387 scratch))
389 (defn triangle-vertex-indices
390 "Get the triangle vertex indices of a given triangle from a given
391 mesh."
392 [#^Mesh mesh triangle-index]
393 (let [indices (int-array 3)]
394 (.getTriangle mesh triangle-index indices)
395 (vec indices)))
397 (defn vertex-UV-coord
398 "Get the uv-coordinates of the vertex named by vertex-index"
399 [#^Mesh mesh vertex-index]
400 (let [UV-buffer
401 (.getData
402 (.getBuffer
403 mesh
404 VertexBuffer$Type/TexCoord))]
405 [(.get UV-buffer (* vertex-index 2))
406 (.get UV-buffer (+ 1 (* vertex-index 2)))]))
408 (defn triangle-UV-coord
409 "Get the uv-cooridnates of the triangle's verticies."
410 [#^Mesh mesh width height triangle-index]
411 (map (fn [[u v]] (vector (* width u) (* height v)))
412 (map (partial vertex-UV-coord mesh)
413 (triangle-vertex-indices mesh triangle-index))))
415 (defn same-side?
416 "Given the points p1 and p2 and the reference point ref, is point p
417 on the same side of the line that goes through p1 and p2 as ref is?"
418 [p1 p2 ref p]
419 (<=
420 0
421 (.dot
422 (.cross (.subtract p2 p1) (.subtract p p1))
423 (.cross (.subtract p2 p1) (.subtract ref p1)))))
425 (defn triangle-seq [#^Triangle tri]
426 [(.get1 tri) (.get2 tri) (.get3 tri)])
428 (defn vector3f-seq [#^Vector3f v]
429 [(.getX v) (.getY v) (.getZ v)])
431 (defn inside-triangle?
432 "Is the point inside the triangle?"
433 {:author "Dylan Holmes"}
434 [#^Triangle tri #^Vector3f p]
435 (let [[vert-1 vert-2 vert-3] (triangle-seq tri)]
436 (and
437 (same-side? vert-1 vert-2 vert-3 p)
438 (same-side? vert-2 vert-3 vert-1 p)
439 (same-side? vert-3 vert-1 vert-2 p))))
441 (defn triangle->matrix4f
442 "Converts the triangle into a 4x4 matrix: The first three columns
443 contain the vertices of the triangle; the last contains the unit
444 normal of the triangle. The bottom row is filled with 1s."
445 [#^Triangle t]
446 (let [mat (Matrix4f.)
447 [vert-1 vert-2 vert-3]
448 ((comp vec map) #(.get t %) (range 3))
449 unit-normal (do (.calculateNormal t)(.getNormal t))
450 vertices [vert-1 vert-2 vert-3 unit-normal]]
451 (dorun
452 (for [row (range 4) col (range 3)]
453 (do
454 (.set mat col row (.get (vertices row)col))
455 (.set mat 3 row 1))))
456 mat))
458 (defn triangle-transformation
459 "Returns the affine transformation that converts each vertex in the
460 first triangle into the corresponding vertex in the second
461 triangle."
462 [#^Triangle tri-1 #^Triangle tri-2]
463 (.mult
464 (triangle->matrix4f tri-2)
465 (.invert (triangle->matrix4f tri-1))))
467 (defn point->vector2f [[u v]]
468 (Vector2f. u v))
470 (defn vector2f->vector3f [v]
471 (Vector3f. (.getX v) (.getY v) 0))
473 (defn map-triangle [f #^Triangle tri]
474 (Triangle.
475 (f 0 (.get1 tri))
476 (f 1 (.get2 tri))
477 (f 2 (.get3 tri))))
479 (defn points->triangle
480 "Convert a list of points into a triangle."
481 [points]
482 (apply #(Triangle. %1 %2 %3)
483 (map (fn [point]
484 (let [point (vec point)]
485 (Vector3f. (get point 0 0)
486 (get point 1 0)
487 (get point 2 0))))
488 (take 3 points))))
490 (defn convex-bounds
491 ;;dylan
492 "Returns the smallest square containing the given
493 vertices, as a vector of integers [left top width height]."
494 ;; "Dimensions of the smallest integer bounding square of the list of
495 ;; 2D verticies in the form: [x y width height]."
496 [uv-verts]
497 (let [xs (map first uv-verts)
498 ys (map second uv-verts)
499 x0 (Math/floor (apply min xs))
500 y0 (Math/floor (apply min ys))
501 x1 (Math/ceil (apply max xs))
502 y1 (Math/ceil (apply max ys))]
503 [x0 y0 (- x1 x0) (- y1 y0)]))
505 (defn sensors-in-triangle
506 ;;dylan
507 "Locate the touch sensors in the triangle, returning a map of their UV and geometry-relative coordinates."
508 ;;"Find the locations of the touch sensors within a triangle in both
509 ;; UV and gemoetry relative coordinates."
510 [image mesh tri-index]
511 (let [width (.getWidth image)
512 height (.getHeight image)
513 UV-vertex-coords (triangle-UV-coord mesh width height tri-index)
514 bounds (convex-bounds UV-vertex-coords)
516 cutout-triangle (points->triangle UV-vertex-coords)
517 UV-sensor-coords
518 (filter (comp (partial inside-triangle? cutout-triangle)
519 (fn [[u v]] (Vector3f. u v 0)))
520 (white-coordinates image bounds))
521 UV->geometry (triangle-transformation
522 cutout-triangle
523 (triangle mesh tri-index))
524 geometry-sensor-coords
525 (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0)))
526 UV-sensor-coords)]
527 {:UV UV-sensor-coords :geometry geometry-sensor-coords}))
529 (defn-memo locate-feelers
530 "Search the geometry's tactile UV image for touch sensors, returning
531 their positions in geometry-relative coordinates."
532 [#^Geometry geo]
533 (let [mesh (.getMesh geo)
534 num-triangles (.getTriangleCount mesh)]
535 (if-let [image (tactile-sensor-image geo)]
536 (map
537 (partial sensors-in-triangle image mesh)
538 (range num-triangles))
539 (repeat (.getTriangleCount mesh) {:UV nil :geometry nil}))))
541 (use 'clojure.contrib.def)
543 (defn-memo touch-topology [#^Gemoetry geo]
544 (vec (collapse (reduce concat (map :UV (locate-feelers geo))))))
546 (defn-memo feeler-coordinates [#^Geometry geo]
547 (vec (map :geometry (locate-feelers geo))))
549 (defn enable-touch [#^Geometry geo]
550 (let [feeler-coords (feeler-coordinates geo)
551 tris (triangles geo)
552 limit 0.1
553 ;;results (CollisionResults.)
554 ]
555 (if (empty? (touch-topology geo))
556 nil
557 (fn [node]
558 (let [sensor-origins
559 (map
560 #(map (partial local-to-world geo) %)
561 feeler-coords)
562 triangle-normals
563 (map (partial get-ray-direction geo)
564 tris)
565 rays
566 (flatten
567 (map (fn [origins norm]
568 (map #(doto (Ray. % norm)
569 (.setLimit limit)) origins))
570 sensor-origins triangle-normals))]
571 (vector
572 (touch-topology geo)
573 (vec
574 (for [ray rays]
575 (do
576 (let [results (CollisionResults.)]
577 (.collideWith node ray results)
578 (let [touch-objects
579 (filter #(not (= geo (.getGeometry %)))
580 results)]
581 (- 255
582 (if (empty? touch-objects) 255
583 (rem
584 (int
585 (* 255 (/ (.getDistance
586 (first touch-objects)) limit)))
587 256))))))))))))))
590 (defn touch [#^Node pieces]
591 (filter (comp not nil?)
592 (map enable-touch
593 (filter #(isa? (class %) Geometry)
594 (node-seq pieces)))))
597 (defn test-eye []
598 (.getChild
599 (.getChild (worm-model) "eyes")
600 "eye"))
604 ;; Ears work the same way as vision.
606 ;; (hearing creature) will return [init-functions
607 ;; sensor-functions]. The init functions each take the world and
608 ;; register a SoundProcessor that does foureier transforms on the
609 ;; incommong sound data, making it available to each sensor function.
611 (defn creature-ears
612 "Return the children of the creature's \"ears\" node."
613 ;;dylan
614 ;;"The ear nodes which are children of the \"ears\" node in the
615 ;;creature."
616 [#^Node creature]
617 (if-let [ear-node (.getChild creature "ears")]
618 (seq (.getChildren ear-node))
619 (do (println-repl "could not find ears node") [])))
622 ;;dylan (defn follow-sense, adjoin-sense, attach-stimuli,
623 ;;anchor-qualia, augment-organ, with-organ
626 (defn update-listener-velocity
627 "Update the listener's velocity every update loop."
628 [#^Spatial obj #^Listener lis]
629 (let [old-position (atom (.getLocation lis))]
630 (.addControl
631 obj
632 (proxy [AbstractControl] []
633 (controlUpdate [tpf]
634 (let [new-position (.getLocation lis)]
635 (.setVelocity
636 lis
637 (.mult (.subtract new-position @old-position)
638 (float (/ tpf))))
639 (reset! old-position new-position)))
640 (controlRender [_ _])))))
642 (import com.aurellem.capture.audio.AudioSendRenderer)
644 (defn attach-ear
645 [#^Application world #^Node creature #^Spatial ear continuation]
646 (let [target (closest-node creature ear)
647 lis (Listener.)
648 audio-renderer (.getAudioRenderer world)
649 sp (sound-processor continuation)]
650 (.setLocation lis (.getWorldTranslation ear))
651 (.setRotation lis (.getWorldRotation ear))
652 (bind-sense target lis)
653 (update-listener-velocity target lis)
654 (.addListener audio-renderer lis)
655 (.registerSoundProcessor audio-renderer lis sp)))
657 (defn enable-hearing
658 [#^Node creature #^Spatial ear]
659 (let [hearing-data (atom [])]
660 [(fn [world]
661 (attach-ear world creature ear
662 (fn [data]
663 (reset! hearing-data (vec data)))))
664 [(fn []
665 (let [data @hearing-data
666 topology
667 (vec (map #(vector % 0) (range 0 (count data))))
668 scaled-data
669 (vec
670 (map
671 #(rem (int (* 255 (/ (+ 1 %) 2))) 256)
672 data))]
673 [topology scaled-data]))
674 ]]))
676 (defn hearing
677 [#^Node creature]
678 (reduce
679 (fn [[init-a senses-a]
680 [init-b senses-b]]
681 [(conj init-a init-b)
682 (into senses-a senses-b)])
683 [[][]]
684 (for [ear (creature-ears creature)]
685 (enable-hearing creature ear))))
692 ;; lower level --- nodes
693 ;; closest-node "parse/compile-x" -> makes organ, which is spatial, fn pair
695 ;; higher level -- organs
696 ;;
698 ;; higher level --- sense/effector
699 ;; these are the functions that provide world i/o, chinese-room style
704 (defn blender-creature
705 "Return a creature with all joints in place."
706 [blender-path]
707 (let [model (load-blender-model blender-path)
708 joints (creature-joints model)]
709 (assemble-creature model joints)))
711 (defn gray-scale [num]
712 (+ num
713 (bit-shift-left num 8)
714 (bit-shift-left num 16)))
716 (defn debug-touch-window
717 "creates function that offers a debug view of sensor data"
718 []
719 (let [vi (view-image)]
720 (fn
721 [[coords sensor-data]]
722 (let [image (points->image coords)]
723 (dorun
724 (for [i (range (count coords))]
725 (.setRGB image ((coords i) 0) ((coords i) 1)
726 (gray-scale (sensor-data i)))))
729 (vi image)))))
731 (defn debug-vision-window
732 "creates function that offers a debug view of sensor data"
733 []
734 (let [vi (view-image)]
735 (fn
736 [[coords sensor-data]]
737 (let [image (points->image coords)]
738 (dorun
739 (for [i (range (count coords))]
740 (.setRGB image ((coords i) 0) ((coords i) 1)
741 (sensor-data i))))
742 (vi image)))))
744 (defn debug-hearing-window
745 "view audio data"
746 [height]
747 (let [vi (view-image)]
748 (fn [[coords sensor-data]]
749 (let [image (BufferedImage. (count coords) height
750 BufferedImage/TYPE_INT_RGB)]
751 (dorun
752 (for [x (range (count coords))]
753 (dorun
754 (for [y (range height)]
755 (let [raw-sensor (sensor-data x)]
756 (.setRGB image x y (gray-scale raw-sensor)))))))
758 (vi image)))))
762 ;;(defn test-touch [world creature]
767 ;; here's how motor-control/ proprioception will work: Each muscle is
768 ;; defined by a 1-D array of numbers (the "motor pool") each of which
769 ;; represent muscle fibers. A muscle also has a scalar :strength
770 ;; factor which determines how strong the muscle as a whole is.
771 ;; The effector function for a muscle takes a number < (count
772 ;; motor-pool) and that number is said to "activate" all the muscle
773 ;; fibers whose index is lower than the number. Each fiber will apply
774 ;; force in proportion to its value in the array. Lower values cause
775 ;; less force. The lower values can be put at the "beginning" of the
776 ;; 1-D array to simulate the layout of actual human muscles, which are
777 ;; capable of more percise movements when exerting less force.
779 ;; I don't know how to encode proprioception, so for now, just return
780 ;; a function for each joint that returns a triplet of floats which
781 ;; represent relative roll, pitch, and yaw. Write display code for
782 ;; this though.
784 (defn muscle-fiber-values
785 "get motor pool strengths"
786 [#^BufferedImage image]
787 (vec
788 (let [width (.getWidth image)]
789 (for [x (range width)]
790 (- 255
791 (bit-and
792 0x0000FF
793 (.getRGB image x 0)))))))
796 (defn creature-muscles
797 "Return the children of the creature's \"muscles\" node."
798 [#^Node creature]
799 (if-let [muscle-node (.getChild creature "muscles")]
800 (seq (.getChildren muscle-node))
801 (do (println-repl "could not find muscles node") [])))
803 (defn single-muscle [#^Node parts #^Node muscle]
804 (let [target (closest-node parts muscle)
805 axis
806 (.mult (.getWorldRotation muscle) Vector3f/UNIT_Y)
807 strength (meta-data muscle "strength")
808 image-name (read-string (meta-data muscle "muscle"))
809 image
810 (ImageToAwt/convert
811 (.getImage (.loadTexture (asset-manager) image-name))
812 false false 0)
813 fibers (muscle-fiber-values image)
814 fiber-integral (reductions + fibers)
815 force-index (vec
816 (map
817 #(float (* strength (/ % (last
818 fiber-integral))))
819 fiber-integral))
820 control (.getControl target RigidBodyControl)]
821 (fn [n]
822 (let [pool-index (min n (count fibers))]
823 (.applyTorque control (.mult axis (force-index n)))))))
826 (defn enable-muscles
827 "Must be called on a creature after RigidBodyControls have been
828 created."
829 [#^Node creature]
830 (let [muscles (creature-muscles creature)]
831 (for [muscle muscles]
832 (single-muscle creature muscle))))
834 (defn test-creature [thing]
835 (let [x-axis
836 (box 1 0.01 0.01 :physical? false :color ColorRGBA/Red)
837 y-axis
838 (box 0.01 1 0.01 :physical? false :color ColorRGBA/Green)
839 z-axis
840 (box 0.01 0.01 1 :physical? false :color ColorRGBA/Blue)
841 creature (blender-creature thing)
842 touch-nerves (touch creature)
843 touch-debug-windows (map (fn [_] (debug-touch-window)) touch-nerves)
844 [init-vision-fns vision-data] (vision creature)
845 vision-debug (map (fn [_] (debug-vision-window)) vision-data)
846 me (sphere 0.5 :color ColorRGBA/Blue :physical? false)
847 [init-hearing-fns hearing-senses] (hearing creature)
848 hearing-windows (map (fn [_] (debug-hearing-window 50))
849 hearing-senses)
850 bell (AudioNode. (asset-manager)
851 "Sounds/pure.wav" false)
852 prop (proprioception creature)
853 prop-debug (proprioception-debug-window)
855 muscle-fns (enable-muscles creature)
856 ;; dream
858 ]
861 (apply
862 world
863 (with-movement
864 (.getChild creature "worm-21")
865 ["key-r" "key-t"
866 "key-f" "key-g"
867 "key-v" "key-b"]
868 [10 10 10 10 1 1]
869 [(nodify [creature
870 (box 10 2 10 :position (Vector3f. 0 -9 0)
871 :color ColorRGBA/Gray :mass 0)
872 x-axis y-axis z-axis
873 me
874 ])
875 (merge standard-debug-controls
876 {"key-return"
877 (fn [_ value]
878 (if value
879 (do
880 (println-repl "play-sound")
881 (.play bell))))
882 "key-h"
883 (fn [_ value]
884 (if value
885 (do
886 (println-repl "muscle activating!")
887 ((first muscle-fns) 199))))
889 })
890 (fn [world]
891 (light-up-everything world)
892 (enable-debug world)
893 (dorun (map #(% world) init-vision-fns))
894 (dorun (map #(% world) init-hearing-fns))
896 (add-eye world
897 (attach-eye creature (test-eye))
898 (comp (view-image) BufferedImage!))
900 (add-eye world (.getCamera world) no-op)
901 ;;(set-gravity world (Vector3f. 0 0 0))
902 ;;(com.aurellem.capture.Capture/captureVideo
903 ;; world (file-str "/home/r/proj/ai-videos/hand"))
904 ;;(.setTimer world (RatchetTimer. 60))
905 (speed-up world)
906 (set-gravity world (Vector3f. 0 0 0))
907 )
908 (fn [world tpf]
909 ;;(dorun
910 ;; (map #(%1 %2) touch-nerves (repeat (.getRootNode world))))
912 (prop-debug (prop))
914 (dorun
915 (map #(%1 (%2 (.getRootNode world)))
916 touch-debug-windows touch-nerves))
918 (dorun
919 (map #(%1 (%2))
920 vision-debug vision-data))
921 (dorun
922 (map #(%1 (%2)) hearing-windows hearing-senses))
925 ;;(println-repl (vision-data))
926 (.setLocalTranslation me (.getLocation (.getCamera world)))
929 )]
930 ;;(let [timer (atom 0)]
931 ;; (fn [_ _]
932 ;; (swap! timer inc)
933 ;; (if (= (rem @timer 60) 0)
934 ;; (println-repl (float (/ @timer 60))))))
935 ))))
945 ;;; experiments in collisions
949 (defn collision-test []
950 (let [b-radius 1
951 b-position (Vector3f. 0 0 0)
952 obj-b (box 1 1 1 :color ColorRGBA/Blue
953 :position b-position
954 :mass 0)
955 node (nodify [obj-b])
956 bounds-b
957 (doto (Picture.)
958 (.setHeight 50)
959 (.setWidth 50)
960 (.setImage (asset-manager)
961 "Models/creature1/hand.png"
962 false
963 ))
965 ;;(Ray. (Vector3f. 0 -5 0) (.normalize (Vector3f. 0 1 0)))
967 collisions
968 (let [cr (CollisionResults.)]
969 (.collideWith node bounds-b cr)
970 (println (map #(.getContactPoint %) cr))
971 cr)
973 ;;collision-points
974 ;;(map #(sphere 0.1 :position (.getContactPoint %))
975 ;; collisions)
977 ;;node (nodify (conj collision-points obj-b))
979 sim
980 (world node
981 {"key-space"
982 (fn [_ value]
983 (if value
984 (let [cr (CollisionResults.)]
985 (.collideWith node bounds-b cr)
986 (println-repl (map #(.getContactPoint %) cr))
987 cr)))}
988 no-op
989 no-op)
991 ]
992 sim
994 ))
997 ;; the camera will stay in its initial position/rotation with relation
998 ;; to the spatial.
1001 (defn follow-test
1002 "show a camera that stays in the same relative position to a blue cube."
1003 []
1004 (let [camera-pos (Vector3f. 0 30 0)
1005 rock (box 1 1 1 :color ColorRGBA/Blue
1006 :position (Vector3f. 0 10 0)
1007 :mass 30
1009 rot (.getWorldRotation rock)
1011 table (box 3 1 10 :color ColorRGBA/Gray :mass 0
1012 :position (Vector3f. 0 -3 0))]
1014 (world
1015 (nodify [rock table])
1016 standard-debug-controls
1017 (fn [world]
1018 (let
1019 [cam (doto (.clone (.getCamera world))
1020 (.setLocation camera-pos)
1021 (.lookAt Vector3f/ZERO
1022 Vector3f/UNIT_X))]
1023 (bind-sense rock cam)
1025 (.setTimer world (RatchetTimer. 60))
1026 (add-eye world cam (comp (view-image) BufferedImage!))
1027 (add-eye world (.getCamera world) no-op))
1029 (fn [_ _] (println-repl rot)))))
1033 #+end_src
1035 #+results: body-1
1036 : #'cortex.silly/follow-test
1039 * COMMENT purgatory
1040 #+begin_src clojure
1042 (defn bullet-trans* []
1043 (let [obj-a (box 1.5 0.5 0.5 :color ColorRGBA/Red
1044 :position (Vector3f. 5 0 0)
1045 :mass 90)
1046 obj-b (sphere 0.5 :color ColorRGBA/Blue
1047 :position (Vector3f. -5 0 0)
1048 :mass 0)
1049 control-a (.getControl obj-a RigidBodyControl)
1050 control-b (.getControl obj-b RigidBodyControl)
1051 move-up? (atom nil)
1052 move-down? (atom nil)
1053 move-left? (atom nil)
1054 move-right? (atom nil)
1055 roll-left? (atom nil)
1056 roll-right? (atom nil)
1057 force 100
1058 swivel
1059 (.toRotationMatrix
1060 (doto (Quaternion.)
1061 (.fromAngleAxis (/ Math/PI 2)
1062 Vector3f/UNIT_X)))
1063 x-move
1064 (doto (Matrix3f.)
1065 (.fromStartEndVectors Vector3f/UNIT_X
1066 (.normalize (Vector3f. 1 1 0))))
1068 timer (atom 0)]
1069 (doto
1070 (ConeJoint.
1071 control-a control-b
1072 (Vector3f. -8 0 0)
1073 (Vector3f. 2 0 0)
1074 ;;swivel swivel
1075 ;;Matrix3f/IDENTITY Matrix3f/IDENTITY
1076 x-move Matrix3f/IDENTITY
1078 (.setCollisionBetweenLinkedBodys false)
1079 (.setLimit (* 1 (/ Math/PI 4)) ;; twist
1080 (* 1 (/ Math/PI 4)) ;; swing span in X-Y plane
1081 (* 0 (/ Math/PI 4)))) ;; swing span in Y-Z plane
1082 (world (nodify
1083 [obj-a obj-b])
1084 (merge standard-debug-controls
1085 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
1086 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
1087 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
1088 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
1089 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
1090 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})
1092 (fn [world]
1093 (enable-debug world)
1094 (set-gravity world Vector3f/ZERO)
1097 (fn [world _]
1099 (if @move-up?
1100 (.applyForce control-a
1101 (Vector3f. force 0 0)
1102 (Vector3f. 0 0 0)))
1103 (if @move-down?
1104 (.applyForce control-a
1105 (Vector3f. (- force) 0 0)
1106 (Vector3f. 0 0 0)))
1107 (if @move-left?
1108 (.applyForce control-a
1109 (Vector3f. 0 force 0)
1110 (Vector3f. 0 0 0)))
1111 (if @move-right?
1112 (.applyForce control-a
1113 (Vector3f. 0 (- force) 0)
1114 (Vector3f. 0 0 0)))
1116 (if @roll-left?
1117 (.applyForce control-a
1118 (Vector3f. 0 0 force)
1119 (Vector3f. 0 0 0)))
1120 (if @roll-right?
1121 (.applyForce control-a
1122 (Vector3f. 0 0 (- force))
1123 (Vector3f. 0 0 0)))
1125 (if (zero? (rem (swap! timer inc) 100))
1126 (.attachChild
1127 (.getRootNode world)
1128 (sphere 0.05 :color ColorRGBA/Yellow
1129 :physical? false :position
1130 (.getWorldTranslation obj-a)))))
1132 ))
1134 (defn test-joint [joint]
1135 (let [[origin top bottom floor] (world-setup joint)
1136 control (.getControl top RigidBodyControl)
1137 move-up? (atom false)
1138 move-down? (atom false)
1139 move-left? (atom false)
1140 move-right? (atom false)
1141 roll-left? (atom false)
1142 roll-right? (atom false)
1143 timer (atom 0)]
1145 (world
1146 (nodify [top bottom floor origin])
1147 (merge standard-debug-controls
1148 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
1149 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
1150 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
1151 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
1152 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
1153 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})
1155 (fn [world]
1156 (light-up-everything world)
1157 (enable-debug world)
1158 (set-gravity world (Vector3f. 0 0 0))
1161 (fn [world _]
1162 (if (zero? (rem (swap! timer inc) 100))
1163 (do
1164 ;; (println-repl @timer)
1165 (.attachChild (.getRootNode world)
1166 (sphere 0.05 :color ColorRGBA/Yellow
1167 :position (.getWorldTranslation top)
1168 :physical? false))
1169 (.attachChild (.getRootNode world)
1170 (sphere 0.05 :color ColorRGBA/LightGray
1171 :position (.getWorldTranslation bottom)
1172 :physical? false))))
1174 (if @move-up?
1175 (.applyTorque control
1176 (.mult (.getPhysicsRotation control)
1177 (Vector3f. 0 0 10))))
1178 (if @move-down?
1179 (.applyTorque control
1180 (.mult (.getPhysicsRotation control)
1181 (Vector3f. 0 0 -10))))
1182 (if @move-left?
1183 (.applyTorque control
1184 (.mult (.getPhysicsRotation control)
1185 (Vector3f. 0 10 0))))
1186 (if @move-right?
1187 (.applyTorque control
1188 (.mult (.getPhysicsRotation control)
1189 (Vector3f. 0 -10 0))))
1190 (if @roll-left?
1191 (.applyTorque control
1192 (.mult (.getPhysicsRotation control)
1193 (Vector3f. -1 0 0))))
1194 (if @roll-right?
1195 (.applyTorque control
1196 (.mult (.getPhysicsRotation control)
1197 (Vector3f. 1 0 0))))))))
1198 #+end_src
1201 * COMMENT generate source
1202 #+begin_src clojure :tangle ../src/cortex/silly.clj
1203 <<body-1>>
1204 #+end_src