rlm@73: #+title: First attempt at a creature!
rlm@73: #+author: Robert McIntyre
rlm@73: #+email: rlm@mit.edu
rlm@73: #+description: 
rlm@73: #+keywords: simulation, jMonkeyEngine3, clojure
rlm@73: #+SETUPFILE: ../../aurellem/org/setup.org
rlm@73: #+INCLUDE: ../../aurellem/org/level-0.org
rlm@73: 
rlm@129: 
rlm@129: 
rlm@129: * Brainstorming different sensors and effectors.
rlm@129: 
rlm@129: Every sense that we have should have an effector that changes what
rlm@129: that sense (or others who have that sense) experiences.
rlm@129: 
rlm@129: ** Classic Senses
rlm@129: | Sense                        | Effector                        |
rlm@129: |------------------------------+---------------------------------|
rlm@129: | Vision                       | Variable Coloration             |
rlm@129: | Hearing                      | Speech                          |
rlm@129: | Proprioception               | Movement                        |
rlm@129: | Smell/Taste (Chemoreception) | Pheremones                      |
rlm@129: | Touch                        | Movement / Controllable Texture |
rlm@129: | Acceleration                 | Movement                        |
rlm@129: | Balance (sense gravity)      | Movement                        |
rlm@129: |                              |                                 |
rlm@129: 
rlm@129: - New Senses/Effectors
rlm@129: - Levitation
rlm@129: - Telekenesis
rlm@129: 
rlm@129: - Symbol Sense
rlm@129:    Where objects in the world can be queried for description /
rlm@129:    symbols.
rlm@129: 
rlm@129: - Symbol Marking
rlm@129:    The ability to mark objects in the world with your own descriptions
rlm@129:    and symbols.
rlm@129: 
rlm@129: - Vision 
rlm@129:   Distinguish the polarization of light
rlm@129:   Color
rlm@129:   Movement
rlm@129: 
rlm@129: * project ideas
rlm@129:  - HACKER for writing muscle-control programs : Presented with
rlm@129:    low-level muscle control/ sense API, generate higher level programs
rlm@129:    for accomplishing various stated goals. Example goals might be
rlm@129:    "extend all your fingers" or "move your hand into the area with
rlm@129:    blue light" or "decrease the angle of this joint".  It would be
rlm@129:    like Sussman's HACKER, except it would operate with much more data
rlm@129:    in a more realistic world.  Start off with "calestanthics" to
rlm@129:    develop subrouitines over the motor control API.  This would be the
rlm@129:    "spinal chord" of a more intelligent creature.  The low level
rlm@129:    programming code might be a turning machine that could develop
rlm@129:    programs to iterate over a "tape" where each entry in the tape
rlm@129:    could control recruitment of the fibers in a muscle.
rlm@129:  - Make a virtual computer in the virtual world which with which the
rlm@129:    creature interacts using its fingers to press keys on a virtual
rlm@129:    keyboard.  The creature can access the internet, watch videos, take
rlm@129:    over the world, anything it wants.
rlm@129:  - Make virtual insturments like pianos, drumbs, etc that it learns to
rlm@129:    play.
rlm@129:  - make a joint that figures out what type of joint it is (range of
rlm@129:    motion)
rlm@129: 
rlm@129:    
rlm@129: 
rlm@129: 
rlm@129: 
rlm@129: * goals
rlm@125: 
rlm@125: ** have to get done before winston
rlm@126:  - [ ] write an explination for why greyscale bitmaps for senses is
rlm@126:        appropiate -- 1/2 day
rlm@126:  - [ ] muscle control -- day
rlm@126:  - [ ] proprioception sensor map in the style of the other senses -- day
rlm@125:  - [ ] refactor integration code to distribute to each of the senses
rlm@126:        -- day
rlm@126:  - [ ] create video showing all the senses for Winston -- 2 days 
rlm@126:  - [ ] write summary of project for Winston            \
rlm@126:  - [ ] project proposals for Winston                    \
rlm@126:  - [ ] additional senses to be implemented for Winston   |  -- 2 days 
rlm@126:  - [ ] send Winston package                             /
rlm@125: 
rlm@125: ** would be cool to get done before winston
rlm@126:  - [X] enable greyscale bitmaps for touch -- 2 hours 
rlm@126:  - [X] use sawfish to auto-tile sense windows -- 6 hours
rlm@126:  - [X] sawfish keybinding to automatically delete all sense windows
rlm@126:  - [ ] directly change the UV-pixels to show sensor activation -- 2
rlm@126:        days
rlm@126:  - [ ] proof of concept C sense manipulation  -- 2 days 
rlm@126:  - [ ] proof of concept GPU sense manipulation -- week
rlm@126:  - [ ] fourier view of sound -- 2 or 3 days 
rlm@127:  - [ ] dancing music generator -- 1 day, depends on fourier
rlm@125: 
rlm@125: ** don't have to get done before winston
rlm@126:  - [ ] write tests for integration -- 3 days
rlm@126:  - [ ] usertime/gametime clock HUD display -- day 
rlm@126:  - [ ] find papers for each of the senses justifying my own
rlm@126:        representation -- week
rlm@126:  - [ ] show sensor maps in HUD display? -- 4 days
rlm@126:  - [ ] show sensor maps in AWT display? -- 2 days
rlm@124: 
rlm@99: 
rlm@73: * Intro 
rlm@73: So far, I've made the following senses -- 
rlm@73:  - Vision
rlm@73:  - Hearing
rlm@73:  - Touch
rlm@73:  - Proprioception
rlm@73: 
rlm@73: And one effector: 
rlm@73:  - Movement
rlm@73: 
rlm@73: However, the code so far has only enabled these senses, but has not
rlm@73: actually implemented them.  For example, there is still a lot of work
rlm@73: to be done for vision. I need to be able to create an /eyeball/ in
rlm@73: simulation that can be moved around and see the world from different
rlm@73: angles. I also need to determine weather to use log-polar or cartesian
rlm@73: for the visual input, and I need to determine how/wether to
rlm@73: disceritise the visual input.
rlm@73: 
rlm@73: I also want to be able to visualize both the sensors and the
rlm@104: effectors in pretty pictures. This semi-retarted creature will be my
rlm@73: first attempt at bringing everything together.
rlm@73: 
rlm@73: * The creature's body
rlm@73: 
rlm@73: Still going to do an eve-like body in blender, but due to problems
rlm@104: importing the joints, etc into jMonkeyEngine3, I'm going to do all
rlm@73: the connecting here in clojure code, using the names of the individual
rlm@73: components and trial and error. Later, I'll maybe make some sort of
rlm@73: creature-building modifications to blender that support whatever
rlm@73: discreitized senses I'm going to make.
rlm@73: 
rlm@73: #+name: body-1
rlm@73: #+begin_src clojure 
rlm@73: (ns cortex.silly
rlm@73:   "let's play!"
rlm@73:   {:author "Robert McIntyre"})
rlm@73: 
rlm@73: ;; TODO remove this!
rlm@73: (require 'cortex.import)
rlm@73: (cortex.import/mega-import-jme3)
rlm@73: (use '(cortex world util body hearing touch vision))
rlm@73: 
rlm@73: (rlm.rlm-commands/help)
rlm@99: (import java.awt.image.BufferedImage)
rlm@99: (import javax.swing.JPanel)
rlm@99: (import javax.swing.SwingUtilities)
rlm@99: (import java.awt.Dimension)
rlm@99: (import javax.swing.JFrame)
rlm@99: (import java.awt.Dimension)
rlm@106: (import com.aurellem.capture.RatchetTimer)
rlm@99: (declare joint-create)
rlm@108: (use 'clojure.contrib.def)
rlm@73: 
rlm@100: (defn points->image
rlm@100:   "Take a sparse collection of points and visuliaze it as a
rlm@100:    BufferedImage."
rlm@102: 
rlm@102:   ;; TODO maybe parallelize this since it's easy
rlm@102: 
rlm@100:   [points]
rlm@106:   (if (empty? points)
rlm@106:     (BufferedImage. 1 1 BufferedImage/TYPE_BYTE_BINARY)
rlm@106:     (let [xs (vec (map first points))
rlm@106:           ys (vec (map second points))
rlm@106:           x0 (apply min xs)
rlm@106:           y0 (apply min ys)
rlm@106:           width (- (apply max xs) x0)
rlm@106:           height (- (apply max ys) y0)
rlm@106:           image (BufferedImage. (inc width) (inc height)
rlm@119:                                 BufferedImage/TYPE_INT_RGB)]
rlm@118:       (dorun
rlm@118:        (for [x (range (.getWidth image))
rlm@118:              y (range (.getHeight image))]
rlm@119:          (.setRGB image x y 0xFF0000)))
rlm@106:       (dorun 
rlm@106:        (for [index (range (count points))]
rlm@106:          (.setRGB image (- (xs index) x0) (- (ys index) y0) -1)))
rlm@106:       
rlm@106:       image)))
rlm@100: 
rlm@101: (defn average [coll]
rlm@101:   (/ (reduce + coll) (count coll)))
rlm@101: 
rlm@101: (defn collapse-1d
rlm@101:   "One dimensional analogue of collapse"
rlm@101:   [center line]
rlm@101:   (let [length (count line)
rlm@101:         num-above (count (filter (partial < center) line))
rlm@101:         num-below (- length num-above)]
rlm@101:     (range (- center num-below)
rlm@115:            (+ center num-above))))
rlm@99: 
rlm@99: (defn collapse
rlm@99:   "Take a set of pairs of integers and collapse them into a
rlm@99:    contigous bitmap."
rlm@99:   [points]
rlm@108:   (if (empty? points) []
rlm@108:       (let
rlm@108:           [num-points (count points)
rlm@108:            center (vector
rlm@108:                    (int (average (map first points)))
rlm@108:                    (int (average (map first points))))
rlm@108:            flattened
rlm@108:            (reduce
rlm@108:             concat 
rlm@108:             (map
rlm@108:              (fn [column]
rlm@108:                (map vector
rlm@108:                     (map first column)
rlm@108:                     (collapse-1d (second center)
rlm@108:                                  (map second column))))
rlm@108:              (partition-by first (sort-by first points))))
rlm@108:            squeezed
rlm@108:            (reduce
rlm@108:             concat 
rlm@108:             (map
rlm@108:              (fn [row]
rlm@108:                (map vector
rlm@108:                     (collapse-1d (first center)
rlm@108:                                  (map first row))
rlm@108:                     (map second row)))
rlm@108:              (partition-by second (sort-by second flattened))))
rlm@108:            relocate
rlm@108:            (let [min-x (apply min (map first squeezed))
rlm@108:                  min-y (apply min (map second squeezed))]
rlm@108:              (map (fn [[x y]]
rlm@108:                     [(- x min-x)
rlm@108:                      (- y min-y)])
rlm@108:                   squeezed))]
rlm@115:         relocate)))
rlm@83: 
rlm@83: (defn load-bullet []
rlm@84:   (let [sim (world (Node.) {} no-op no-op)]
rlm@102:     (doto sim
rlm@102:       (.enqueue
rlm@102:        (fn []
rlm@102:          (.stop sim)))
rlm@102:       (.start))))
rlm@83: 
rlm@73: (defn load-blender-model
rlm@73:   "Load a .blend file using an asset folder relative path."
rlm@73:   [^String model]
rlm@73:   (.loadModel
rlm@73:    (doto (asset-manager)
rlm@73:      (.registerLoader BlenderModelLoader (into-array String ["blend"])))
rlm@73:    model))
rlm@73: 
rlm@74: (defn meta-data [blender-node key]
rlm@74:   (if-let [data (.getUserData blender-node "properties")]
rlm@74:     (.findValue data key)
rlm@74:     nil))
rlm@73: 
rlm@78: (defn blender-to-jme
rlm@78:   "Convert from Blender coordinates to JME coordinates"
rlm@78:   [#^Vector3f in]
rlm@78:   (Vector3f. (.getX in)
rlm@78:              (.getZ in)
rlm@78:              (- (.getY in))))
rlm@74: 
rlm@79: (defn jme-to-blender
rlm@79:   "Convert from JME coordinates to Blender coordinates"
rlm@79:   [#^Vector3f in]
rlm@79:   (Vector3f. (.getX in)
rlm@79:              (- (.getZ in))
rlm@79:              (.getY in)))
rlm@79: 
rlm@78: (defn joint-targets
rlm@78:   "Return the two closest two objects to the joint object, ordered
rlm@78:   from bottom to top according to the joint's rotation."
rlm@78:   [#^Node parts #^Node joint]
rlm@78:   (loop [radius (float 0.01)]
rlm@78:     (let [results (CollisionResults.)]
rlm@78:       (.collideWith
rlm@78:        parts
rlm@78:        (BoundingBox. (.getWorldTranslation joint)
rlm@78:                      radius radius radius)
rlm@78:        results)
rlm@78:       (let [targets
rlm@78:             (distinct
rlm@78:              (map  #(.getGeometry %) results))]
rlm@78:         (if (>= (count targets) 2)
rlm@78:           (sort-by
rlm@79:            #(let [v
rlm@79:                   (jme-to-blender
rlm@79:                    (.mult
rlm@79:                     (.inverse (.getWorldRotation joint))
rlm@79:                     (.subtract (.getWorldTranslation %)
rlm@79:                                (.getWorldTranslation joint))))]
rlm@79:               (println-repl (.getName %) ":" v)
rlm@79:               (.dot (Vector3f. 1 1 1)
rlm@79:                     v))                  
rlm@78:            (take 2 targets))
rlm@78:           (recur (float (* radius 2))))))))
rlm@74: 
rlm@87: (defn world-to-local
rlm@87:    "Convert the world coordinates into coordinates relative to the 
rlm@87:    object (i.e. local coordinates), taking into account the rotation
rlm@87:    of object."
rlm@87:    [#^Spatial object world-coordinate]
rlm@87:    (let [out (Vector3f.)]
rlm@88:      (.worldToLocal object world-coordinate out) out))
rlm@87: 
rlm@96: (defn local-to-world
rlm@96:    "Convert the local coordinates into coordinates into world relative
rlm@96:     coordinates" 
rlm@96:    [#^Spatial object local-coordinate]
rlm@96:    (let [world-coordinate (Vector3f.)]
rlm@96:      (.localToWorld object local-coordinate world-coordinate)
rlm@96:      world-coordinate))
rlm@96: 
rlm@87: (defmulti joint-dispatch
rlm@87:   "Translate blender pseudo-joints into real JME joints."
rlm@88:   (fn [constraints & _] 
rlm@87:     (:type constraints)))
rlm@87: 
rlm@87: (defmethod joint-dispatch :point
rlm@87:   [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87:   (println-repl "creating POINT2POINT joint")
rlm@87:   (Point2PointJoint.
rlm@87:    control-a
rlm@87:    control-b
rlm@87:    pivot-a
rlm@87:    pivot-b))
rlm@87: 
rlm@87: (defmethod joint-dispatch :hinge
rlm@87:   [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87:   (println-repl "creating HINGE joint")
rlm@87:   (let [axis
rlm@87:         (if-let
rlm@87:             [axis (:axis constraints)]
rlm@87:           axis
rlm@87:           Vector3f/UNIT_X)
rlm@87:         [limit-1 limit-2] (:limit constraints)
rlm@87:         hinge-axis
rlm@87:         (.mult
rlm@87:          rotation
rlm@87:          (blender-to-jme axis))]
rlm@87:     (doto
rlm@87:         (HingeJoint.
rlm@87:          control-a
rlm@87:          control-b
rlm@87:          pivot-a
rlm@87:          pivot-b
rlm@87:          hinge-axis
rlm@87:          hinge-axis)
rlm@87:       (.setLimit limit-1 limit-2))))
rlm@87: 
rlm@87: (defmethod joint-dispatch :cone
rlm@87:   [constraints control-a control-b pivot-a pivot-b rotation]
rlm@87:   (let [limit-xz (:limit-xz constraints)
rlm@87:         limit-xy (:limit-xy constraints)
rlm@87:         twist    (:twist constraints)]
rlm@87:     
rlm@87:     (println-repl "creating CONE joint")
rlm@87:     (println-repl rotation)
rlm@87:     (println-repl
rlm@87:      "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0)))
rlm@87:     (println-repl
rlm@87:      "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0)))
rlm@87:     (println-repl
rlm@87:      "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1)))
rlm@87:     (doto
rlm@87:         (ConeJoint.
rlm@87:          control-a
rlm@87:          control-b
rlm@87:          pivot-a
rlm@87:          pivot-b
rlm@87:          rotation
rlm@87:          rotation)
rlm@87:       (.setLimit (float limit-xz)
rlm@87:                  (float limit-xy)
rlm@87:                  (float twist)))))
rlm@87: 
rlm@88: (defn connect
rlm@87:   "here are some examples:
rlm@87:    {:type :point}
rlm@87:    {:type :hinge  :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}
rlm@87:    (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
rlm@87: 
rlm@89:    {:type :cone :limit-xz 0]
rlm@89:                 :limit-xy 0]
rlm@89:                 :twist 0]}   (use XZY rotation mode in blender!)"
rlm@87:   [#^Node obj-a #^Node obj-b #^Node joint]
rlm@87:   (let [control-a (.getControl obj-a RigidBodyControl)
rlm@87:         control-b (.getControl obj-b RigidBodyControl)
rlm@87:         joint-center (.getWorldTranslation joint)
rlm@87:         joint-rotation (.toRotationMatrix (.getWorldRotation joint))
rlm@87:         pivot-a (world-to-local obj-a joint-center)
rlm@87:         pivot-b (world-to-local obj-b joint-center)]
rlm@89:    
rlm@87:     (if-let [constraints
rlm@87:              (map-vals
rlm@87:               eval
rlm@87:               (read-string
rlm@87:                (meta-data joint "joint")))]
rlm@89:       ;; A side-effect of creating a joint registers
rlm@89:       ;; it with both physics objects which in turn
rlm@89:       ;; will register the joint with the physics system
rlm@89:       ;; when the simulation is started.
rlm@87:       (do
rlm@87:         (println-repl "creating joint between"
rlm@87:                       (.getName obj-a) "and" (.getName obj-b))
rlm@87:         (joint-dispatch constraints
rlm@87:                         control-a control-b
rlm@87:                         pivot-a pivot-b
rlm@87:                         joint-rotation))
rlm@87:       (println-repl "could not find joint meta-data!"))))
rlm@87: 
rlm@78: (defn assemble-creature [#^Node pieces joints]
rlm@78:   (dorun
rlm@78:    (map
rlm@78:     (fn [geom]
rlm@78:       (let [physics-control
rlm@78:             (RigidBodyControl.
rlm@78:              (HullCollisionShape.
rlm@78:               (.getMesh geom))
rlm@78:              (if-let [mass (meta-data geom "mass")]
rlm@78:                (do
rlm@78:                  (println-repl
rlm@78:                   "setting" (.getName geom) "mass to" (float mass))
rlm@78:                  (float mass))
rlm@78:                (float 1)))]
rlm@78:                
rlm@78:         (.addControl geom physics-control)))
rlm@78:     (filter #(isa? (class %) Geometry )
rlm@78:             (node-seq pieces))))
rlm@78:   (dorun
rlm@78:    (map
rlm@78:     (fn [joint]
rlm@78:       (let [[obj-a obj-b]
rlm@78:             (joint-targets pieces joint)]
rlm@88:         (connect obj-a obj-b joint)))
rlm@78:     joints))
rlm@78:   pieces)
rlm@74: 
rlm@116: (declare blender-creature)
rlm@74: 
rlm@78: (def hand "Models/creature1/one.blend")
rlm@74: 
rlm@78: (def worm "Models/creature1/try-again.blend")
rlm@78: 
rlm@90: (def touch "Models/creature1/touch.blend")
rlm@90: 
rlm@90: (defn worm-model [] (load-blender-model worm))
rlm@90: 
rlm@80: (defn x-ray [#^ColorRGBA color]
rlm@80:   (doto (Material. (asset-manager)
rlm@80:                    "Common/MatDefs/Misc/Unshaded.j3md")
rlm@80:     (.setColor "Color" color)
rlm@80:     (-> (.getAdditionalRenderState)
rlm@80:         (.setDepthTest false))))
rlm@80: 
rlm@91: (defn colorful []
rlm@91:   (.getChild (worm-model) "worm-21"))
rlm@90: 
rlm@90: (import jme3tools.converters.ImageToAwt)
rlm@90: 
rlm@90: (import ij.ImagePlus)
rlm@90: 
rlm@108: ;; Every Mesh has many triangles, each with its own index.
rlm@108: ;; Every vertex has its own index as well.
rlm@90: 
rlm@108: (defn tactile-sensor-image
rlm@110:   "Return the touch-sensor distribution image in BufferedImage format,
rlm@110:    or nil if it does not exist."
rlm@91:   [#^Geometry obj]
rlm@110:   (if-let [image-path (meta-data obj "touch")]
rlm@110:     (ImageToAwt/convert
rlm@110:      (.getImage
rlm@110:       (.loadTexture
rlm@110:        (asset-manager)
rlm@110:        image-path))
rlm@110:     false false 0)))
rlm@110:      
rlm@91: (import ij.process.ImageProcessor)
rlm@91: (import java.awt.image.BufferedImage)
rlm@91: 
rlm@92: (def white -1)
rlm@94:   
rlm@91: (defn filter-pixels
rlm@108:   "List the coordinates of all pixels matching pred, within the bounds
rlm@108:    provided. Bounds -> [x0 y0 width height]"
rlm@92:   {:author "Dylan Holmes"}
rlm@108:   ([pred #^BufferedImage image]
rlm@108:      (filter-pixels pred image [0 0 (.getWidth image) (.getHeight image)]))
rlm@108:   ([pred #^BufferedImage image [x0 y0 width height]]
rlm@108:      ((fn accumulate [x y matches]
rlm@108:         (cond
rlm@108:          (>= y (+ height y0)) matches
rlm@108:          (>= x (+ width x0)) (recur 0 (inc y) matches)
rlm@108:          (pred (.getRGB image x y))
rlm@108:          (recur (inc x) y (conj matches [x y]))
rlm@108:          :else (recur (inc x) y matches)))
rlm@108:       x0 y0 [])))
rlm@91: 
rlm@91: (defn white-coordinates
rlm@108:   "Coordinates of all the white pixels in a subset of the image."
rlm@112:   ([#^BufferedImage image bounds]
rlm@112:      (filter-pixels #(= % white) image bounds))
rlm@112:   ([#^BufferedImage image]
rlm@112:      (filter-pixels #(= % white) image)))
rlm@108: 
rlm@108: (defn triangle
rlm@112:   "Get the triangle specified by triangle-index from the mesh within
rlm@112:   bounds."
rlm@108:   [#^Mesh mesh triangle-index]
rlm@108:   (let [scratch (Triangle.)]
rlm@108:     (.getTriangle mesh triangle-index scratch)
rlm@108:     scratch))
rlm@108: 
rlm@108: (defn triangle-vertex-indices
rlm@108:   "Get the triangle vertex indices of a given triangle from a given
rlm@108:    mesh."
rlm@108:   [#^Mesh mesh triangle-index]
rlm@108:   (let [indices (int-array 3)]
rlm@108:     (.getTriangle mesh triangle-index indices)
rlm@108:     (vec indices)))
rlm@108: 
rlm@108: (defn vertex-UV-coord
rlm@108:   "Get the uv-coordinates of the vertex named by vertex-index"
rlm@108:   [#^Mesh mesh vertex-index]
rlm@108:   (let [UV-buffer
rlm@108:         (.getData
rlm@108:          (.getBuffer
rlm@108:           mesh
rlm@108:           VertexBuffer$Type/TexCoord))]
rlm@108:     [(.get UV-buffer (* vertex-index 2))
rlm@108:      (.get UV-buffer (+ 1 (* vertex-index 2)))]))
rlm@108: 
rlm@108: (defn triangle-UV-coord
rlm@108:   "Get the uv-cooridnates of the triangle's verticies."
rlm@108:   [#^Mesh mesh width height triangle-index]
rlm@108:   (map (fn [[u v]] (vector (* width u) (* height v)))
rlm@108:        (map (partial vertex-UV-coord mesh)
rlm@108:             (triangle-vertex-indices mesh triangle-index))))
rlm@91:   
rlm@102: (defn same-side?
rlm@102:   "Given the points p1 and p2 and the reference point ref, is point p
rlm@102:   on the same side of the line that goes through p1 and p2 as ref is?" 
rlm@102:   [p1 p2 ref p]
rlm@91:   (<=
rlm@91:    0
rlm@91:    (.dot 
rlm@91:     (.cross (.subtract p2 p1) (.subtract p p1))
rlm@91:     (.cross (.subtract p2 p1) (.subtract ref p1)))))
rlm@91: 
rlm@108: (defn triangle-seq [#^Triangle tri]
rlm@108:   [(.get1 tri) (.get2 tri) (.get3 tri)])
rlm@108: 
rlm@108: (defn vector3f-seq [#^Vector3f v]
rlm@108:   [(.getX v) (.getY v) (.getZ v)])
rlm@108: 
rlm@108: (defn inside-triangle?
rlm@108:   "Is the point inside the triangle?"
rlm@108:   {:author "Dylan Holmes"}
rlm@108:   [#^Triangle tri #^Vector3f p]
rlm@108:   (let [[vert-1 vert-2 vert-3] (triangle-seq tri)]
rlm@108:     (and
rlm@108:      (same-side? vert-1 vert-2 vert-3 p)
rlm@108:      (same-side? vert-2 vert-3 vert-1 p)
rlm@108:      (same-side? vert-3 vert-1 vert-2 p))))
rlm@108: 
rlm@94: (defn triangle->matrix4f
rlm@108:   "Converts the triangle into a 4x4 matrix: The first three columns
rlm@108:    contain the vertices of the triangle; the last contains the unit
rlm@108:    normal of the triangle. The bottom row is filled with 1s."
rlm@94:   [#^Triangle t]
rlm@94:   (let [mat (Matrix4f.)
rlm@94:         [vert-1 vert-2 vert-3]
rlm@94:         ((comp vec map) #(.get t %) (range 3))
rlm@94:         unit-normal (do (.calculateNormal t)(.getNormal t))
rlm@94:         vertices [vert-1 vert-2 vert-3 unit-normal]]
rlm@94:     (dorun 
rlm@94:      (for [row (range 4) col (range 3)]
rlm@94:        (do
rlm@94:          (.set mat col row (.get (vertices row)col))
rlm@94:          (.set mat 3 row 1))))
rlm@94:     mat))
rlm@94: 
rlm@94: (defn triangle-transformation
rlm@94:   "Returns the affine transformation that converts each vertex in the
rlm@94:    first triangle into the corresponding vertex in the second
rlm@94:    triangle."
rlm@94:   [#^Triangle tri-1 #^Triangle tri-2]
rlm@94:   (.mult 
rlm@94:    (triangle->matrix4f tri-2)
rlm@94:    (.invert (triangle->matrix4f tri-1))))
rlm@94: 
rlm@108: (defn point->vector2f [[u v]]
rlm@108:   (Vector2f. u v))
rlm@94: 
rlm@94: (defn vector2f->vector3f [v]
rlm@94:   (Vector3f. (.getX v) (.getY v) 0))
rlm@94: 
rlm@94: (defn map-triangle [f #^Triangle tri]
rlm@94:   (Triangle.
rlm@94:    (f 0 (.get1 tri))
rlm@94:    (f 1 (.get2 tri))
rlm@94:    (f 2 (.get3 tri))))
rlm@94: 
rlm@108: (defn points->triangle
rlm@108:   "Convert a list of points into a triangle."
rlm@108:   [points]
rlm@108:   (apply #(Triangle. %1 %2 %3)
rlm@108:          (map (fn [point]
rlm@108:                 (let [point (vec point)]
rlm@108:                   (Vector3f. (get point 0 0)
rlm@108:                              (get point 1 0)
rlm@108:                              (get point 2 0))))
rlm@108:               (take 3 points))))
rlm@94: 
rlm@108: (defn convex-bounds
rlm@128:   ;;dylan
rlm@128:   "Returns the smallest square containing the given
rlm@128: vertices, as a vector of integers [left top width height]."
rlm@128:  ;; "Dimensions of the smallest integer bounding square of the list of
rlm@128:  ;;  2D verticies in the form: [x y width height]."
rlm@108:   [uv-verts]
rlm@108:   (let [xs (map first uv-verts)
rlm@108:         ys (map second uv-verts)
rlm@108:         x0 (Math/floor (apply min xs))
rlm@108:         y0 (Math/floor (apply min ys))
rlm@108:         x1 (Math/ceil (apply max xs))
rlm@108:         y1 (Math/ceil (apply max ys))]
rlm@108:     [x0 y0 (- x1 x0) (- y1 y0)]))
rlm@93: 
rlm@106: (defn sensors-in-triangle
rlm@128:   ;;dylan
rlm@128:   "Locate the touch sensors in the triangle, returning a map of their UV and geometry-relative coordinates."
rlm@128:   ;;"Find the locations of the touch sensors within a triangle in both
rlm@128:   ;; UV and gemoetry relative coordinates."
rlm@107:   [image mesh tri-index]
rlm@107:   (let [width (.getWidth image)
rlm@108:         height (.getHeight image)
rlm@108:         UV-vertex-coords (triangle-UV-coord mesh width height tri-index)
rlm@108:         bounds (convex-bounds UV-vertex-coords)
rlm@108:         
rlm@108:         cutout-triangle (points->triangle UV-vertex-coords)
rlm@108:         UV-sensor-coords
rlm@108:         (filter (comp (partial inside-triangle? cutout-triangle)
rlm@108:                       (fn [[u v]] (Vector3f. u v 0)))
rlm@108:                 (white-coordinates image bounds))
rlm@108:         UV->geometry (triangle-transformation
rlm@108:                       cutout-triangle
rlm@108:                       (triangle mesh tri-index))
rlm@108:         geometry-sensor-coords
rlm@108:         (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0)))
rlm@108:              UV-sensor-coords)]
rlm@108:   {:UV UV-sensor-coords :geometry geometry-sensor-coords}))
rlm@107: 
rlm@108: (defn-memo locate-feelers
rlm@94:   "Search the geometry's tactile UV image for touch sensors, returning
rlm@94:   their positions in geometry-relative coordinates."
rlm@94:   [#^Geometry geo]
rlm@108:   (let [mesh (.getMesh geo)
rlm@108:         num-triangles (.getTriangleCount mesh)]
rlm@108:     (if-let [image (tactile-sensor-image geo)]
rlm@108:       (map
rlm@108:        (partial sensors-in-triangle image mesh)
rlm@108:        (range num-triangles))
rlm@108:       (repeat (.getTriangleCount mesh) {:UV nil :geometry nil}))))
rlm@102: 
rlm@102: (use 'clojure.contrib.def)
rlm@102: 
rlm@102: (defn-memo touch-topology [#^Gemoetry geo]
rlm@108:   (vec (collapse (reduce concat (map :UV (locate-feelers geo))))))
rlm@108: 
rlm@108: (defn-memo feeler-coordinates [#^Geometry geo]
rlm@108:   (vec (map :geometry (locate-feelers geo))))
rlm@102: 
rlm@97: (defn enable-touch [#^Geometry geo]
rlm@108:   (let [feeler-coords (feeler-coordinates geo)
rlm@96:         tris (triangles geo)
rlm@109:         limit 0.1
rlm@109:         ;;results (CollisionResults.)
rlm@109:         ]
rlm@111:     (if (empty? (touch-topology geo))
rlm@111:       nil
rlm@111:       (fn [node]
rlm@111:         (let [sensor-origins 
rlm@111:               (map
rlm@111:                #(map (partial local-to-world geo) %)
rlm@111:                feeler-coords)
rlm@111:               triangle-normals 
rlm@111:               (map (partial get-ray-direction geo)
rlm@111:                    tris)
rlm@111:               rays
rlm@111:               (flatten
rlm@111:                (map (fn [origins norm]
rlm@111:                       (map #(doto (Ray. % norm)
rlm@97:                               (.setLimit limit)) origins))
rlm@111:                     sensor-origins triangle-normals))]
rlm@111:           (vector
rlm@111:            (touch-topology geo)
rlm@111:            (vec
rlm@111:             (for [ray rays]
rlm@111:               (do
rlm@111:                 (let [results (CollisionResults.)]
rlm@111:                   (.collideWith node ray results)
rlm@111:                   (let [touch-objects
rlm@126:                         (filter #(not (= geo (.getGeometry %)))
rlm@126:                                 results)]
rlm@126:                     (- 255
rlm@126:                        (if (empty? touch-objects) 255
rlm@126:                            (rem 
rlm@126:                             (int
rlm@126:                              (* 255 (/ (.getDistance
rlm@126:                                         (first touch-objects)) limit)))
rlm@126:                             256))))))))))))))
rlm@126:                          
rlm@111:   
rlm@111: (defn touch [#^Node pieces]
rlm@111:   (filter (comp not nil?)
rlm@111:           (map enable-touch
rlm@111:                (filter #(isa? (class %) Geometry)
rlm@111:                        (node-seq pieces)))))
rlm@94: 
rlm@109: 
rlm@111: ;; human eye transmits 62kb/s to brain Bandwidth is 8.75 Mb/s
rlm@111: ;; http://en.wikipedia.org/wiki/Retina
rlm@109: 
rlm@111: (defn test-eye  []
rlm@117:   (.getChild 
rlm@117:    (.getChild (worm-model) "eyes")
rlm@117:    "eye"))
rlm@111: 
rlm@111: 
rlm@111: (defn retina-sensor-image
rlm@111:   "Return a map of pixel selection functions to BufferedImages
rlm@111:    describing the distribution of light-sensitive components on this
rlm@111:    geometry's surface. Each function creates an integer from the rgb
rlm@111:    values found in the pixel. :red, :green, :blue, :gray are already
rlm@111:    defined as extracting the red green blue and average components
rlm@111:    respectively."
rlm@117:    [#^Spatial eye]
rlm@111:    (if-let [eye-map (meta-data eye "eye")]
rlm@111:      (map-vals
rlm@111:       #(ImageToAwt/convert
rlm@111:         (.getImage (.loadTexture (asset-manager) %))
rlm@111:         false false 0)
rlm@120:       (eval (read-string eye-map)))))
rlm@111: 
rlm@117: (defn eye-dimensions
rlm@117:   "returns the width and height specified in the metadata of the eye"
rlm@117:   [#^Spatial eye]
rlm@117:   (let [dimensions
rlm@117:           (map #(vector (.getWidth %) (.getHeight %))
rlm@117:                (vals (retina-sensor-image eye)))]
rlm@117:     [(apply max (map first dimensions))
rlm@117:      (apply max (map second dimensions))]))
rlm@117: 
rlm@116: (defn creature-eyes
rlm@128:   ;;dylan
rlm@128:   "Return the children of the creature's \"eyes\" node."
rlm@128:   ;;"The eye nodes which are children of the \"eyes\" node in the
rlm@128:   ;;creature."
rlm@116:   [#^Node creature]
rlm@116:   (if-let [eye-node (.getChild creature "eyes")]
rlm@116:     (seq (.getChildren eye-node))
rlm@116:     (do (println-repl "could not find eyes node") [])))
rlm@111: 
rlm@123: ;; Here's how vision will work.
rlm@112: 
rlm@123: ;; Make the continuation in scene-processor take FrameBuffer,
rlm@123: ;; byte-buffer, BufferedImage already sized to the correct
rlm@123: ;; dimensions. the continuation will decide wether to "mix" them
rlm@123: ;; into the BufferedImage, lazily ignore them, or mix them halfway
rlm@123: ;; and call c/graphics card routines.
rlm@112: 
rlm@123: ;; (vision creature) will take an optional :skip argument which will
rlm@123: ;; inform the continuations in scene processor to skip the given
rlm@123: ;; number of cycles; 0 means that no cycles will be skipped.
rlm@112: 
rlm@123: ;; (vision creature) will return [init-functions sensor-functions].
rlm@123: ;; The init-functions are each single-arg functions that take the
rlm@123: ;; world and register the cameras and must each be called before the
rlm@123: ;; corresponding sensor-functions.  Each init-function returns the
rlm@123: ;; viewport for that eye which can be manipulated, saved, etc. Each
rlm@123: ;; sensor-function is a thunk and will return data in the same
rlm@123: ;; format as the tactile-sensor functions; the structure is
rlm@123: ;; [topology, sensor-data]. Internally, these sensor-functions
rlm@123: ;; maintain a reference to sensor-data which is periodically updated
rlm@123: ;; by the continuation function established by its init-function.
rlm@123: ;; They can be queried every cycle, but their information may not
rlm@123: ;; necessairly be different every cycle.
rlm@112: 
rlm@123: ;; Each eye in the creature in blender will work the same way as
rlm@123: ;; joints -- a zero dimensional object with no geometry whose local
rlm@123: ;; coordinate system determines the orientation of the resulting
rlm@123: ;; eye. All eyes will have a parent named "eyes" just as all joints
rlm@123: ;; have a parent named "joints". The resulting camera will be a
rlm@123: ;; ChaseCamera or a CameraNode bound to the geo that is closest to
rlm@123: ;; the eye marker. The eye marker will contain the metadata for the
rlm@123: ;; eye, and will be moved by it's bound geometry. The dimensions of
rlm@123: ;; the eye's camera are equal to the dimensions of the eye's "UV"
rlm@123: ;; map.
rlm@116: 
rlm@123: 
rlm@123: ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
rlm@123: 
rlm@123: ;; Ears work the same way as vision.
rlm@123: 
rlm@123: ;; (hearing creature) will return [init-functions
rlm@123: ;; sensor-functions]. The init functions each take the world and
rlm@123: ;; register a SoundProcessor that does foureier transforms on the
rlm@123: ;; incommong sound data, making it available to each sensor function.
rlm@123: 
rlm@123: (defn creature-ears
rlm@128:   "Return the children of the creature's \"ears\" node."
rlm@128:   ;;dylan
rlm@128:   ;;"The ear nodes which are children of the \"ears\" node in the
rlm@128:   ;;creature."
rlm@123:   [#^Node creature]
rlm@123:   (if-let [ear-node (.getChild creature "ears")]
rlm@123:     (seq (.getChildren ear-node))
rlm@123:     (do (println-repl "could not find ears node") [])))
rlm@123: 
rlm@123: (defn closest-node
rlm@128:   "Return the object in creature which is closest to the given node."
rlm@128:   ;;dylan"The closest object in creature to the given node."
rlm@116:   [#^Node creature #^Node eye]
rlm@116:   (loop [radius (float 0.01)]
rlm@116:     (let [results (CollisionResults.)]
rlm@116:       (.collideWith
rlm@116:        creature
rlm@116:        (BoundingBox. (.getWorldTranslation eye)
rlm@116:                      radius radius radius)
rlm@116:        results)
rlm@116:       (if-let [target (first results)]
rlm@116:         (.getGeometry target)
rlm@116:         (recur (float (* 2 radius)))))))
rlm@116: 
rlm@128: ;;dylan (defn follow-sense, adjoin-sense, attach-stimuli,
rlm@128: ;;anchor-qualia, augment-organ, with-organ
rlm@123: (defn bind-sense
rlm@123:   "Bind the sense to the Spatial such that it will maintain its
rlm@117:    current position relative to the Spatial no matter how the spatial
rlm@123:    moves. 'sense can be either a Camera or Listener object."
rlm@123:   [#^Spatial obj sense]
rlm@123:   (let [sense-offset (.subtract (.getLocation sense)
rlm@123:                                 (.getWorldTranslation obj))
rlm@123:         initial-sense-rotation (Quaternion. (.getRotation sense))
rlm@117:         base-anti-rotation (.inverse (.getWorldRotation obj))]
rlm@117:     (.addControl
rlm@117:      obj
rlm@117:      (proxy [AbstractControl] []
rlm@117:        (controlUpdate [tpf]
rlm@117:          (let [total-rotation
rlm@117:                (.mult base-anti-rotation (.getWorldRotation obj))]
rlm@123:            (.setLocation sense
rlm@117:                          (.add
rlm@123:                           (.mult total-rotation sense-offset)
rlm@117:                           (.getWorldTranslation obj)))
rlm@123:            (.setRotation sense
rlm@123:                          (.mult total-rotation initial-sense-rotation))))
rlm@117:        (controlRender [_ _])))))
rlm@117: 
rlm@117: 
rlm@123: (defn update-listener-velocity
rlm@123:   "Update the listener's velocity every update loop."
rlm@123:   [#^Spatial obj #^Listener lis]
rlm@123:   (let [old-position (atom (.getLocation lis))]
rlm@123:     (.addControl
rlm@123:      obj
rlm@123:      (proxy [AbstractControl] []
rlm@123:        (controlUpdate [tpf]
rlm@123:          (let [new-position (.getLocation lis)]
rlm@123:            (.setVelocity
rlm@123:             lis 
rlm@123:             (.mult (.subtract new-position @old-position)
rlm@123:                    (float (/ tpf))))
rlm@123:            (reset! old-position new-position)))
rlm@123:        (controlRender [_ _])))))
rlm@123: 
rlm@123: (import com.aurellem.capture.audio.AudioSendRenderer)
rlm@123: 
rlm@123: (defn attach-ear
rlm@123:   [#^Application world #^Node creature #^Spatial ear continuation]
rlm@123:   (let [target (closest-node creature ear)
rlm@123:         lis (Listener.)
rlm@123:         audio-renderer (.getAudioRenderer world)
rlm@123:         sp (sound-processor continuation)]
rlm@123:     (.setLocation lis (.getWorldTranslation ear))
rlm@123:     (.setRotation lis (.getWorldRotation ear))
rlm@123:     (bind-sense target lis)
rlm@123:     (update-listener-velocity target lis)
rlm@123:     (.addListener audio-renderer lis)
rlm@123:     (.registerSoundProcessor audio-renderer lis sp)))
rlm@123: 
rlm@123: (defn enable-hearing
rlm@123:   [#^Node creature #^Spatial ear]
rlm@123:   (let [hearing-data (atom [])]
rlm@123:     [(fn [world]
rlm@123:        (attach-ear world creature ear
rlm@123:                    (fn [data]
rlm@123:                      (reset! hearing-data (vec data)))))
rlm@123:      [(fn []
rlm@123:         (let [data @hearing-data
rlm@123:               topology              
rlm@123:               (vec (map #(vector % 0) (range 0 (count data))))
rlm@123:               scaled-data
rlm@123:               (vec
rlm@123:                (map
rlm@123:                 #(rem (int (* 255 (/ (+ 1 %) 2)))  256)
rlm@123:                    data))]
rlm@123:           [topology scaled-data]))
rlm@123:         ]]))
rlm@123: 
rlm@123: (defn hearing
rlm@123:   [#^Node creature]
rlm@123:   (reduce
rlm@123:    (fn [[init-a senses-a]
rlm@123:         [init-b senses-b]]
rlm@123:      [(conj init-a init-b)
rlm@123:       (into senses-a senses-b)])
rlm@123:    [[][]]      
rlm@123:    (for [ear (creature-ears creature)]
rlm@123:      (enable-hearing creature ear))))
rlm@123: 
rlm@118: (defn attach-eye
rlm@118:   "Attach a Camera to the appropiate area and return the Camera."
rlm@118:   [#^Node creature #^Spatial eye]
rlm@123:   (let [target (closest-node creature eye)
rlm@118:         [cam-width cam-height] (eye-dimensions eye)
rlm@118:         cam (Camera. cam-width cam-height)]
rlm@118:     (.setLocation cam (.getWorldTranslation eye))
rlm@118:     (.setRotation cam (.getWorldRotation eye))
rlm@119:     (.setFrustumPerspective
rlm@119:      cam 45 (/ (.getWidth cam) (.getHeight cam))
rlm@119:      1 1000)
rlm@123:     (bind-sense target cam)
rlm@118:     cam))
rlm@118: 
rlm@118: (def presets
rlm@121:   {:all    0xFFFFFF
rlm@119:    :red    0xFF0000
rlm@119:    :blue   0x0000FF
rlm@119:    :green  0x00FF00})
rlm@119: 
rlm@118: (defn enable-vision
rlm@118:   "return [init-function sensor-functions] for a particular eye"
rlm@118:   [#^Node creature #^Spatial eye & {skip :skip :or {skip 0}}]
rlm@118:   (let [retinal-map (retina-sensor-image eye)
rlm@123:         camera (attach-eye creature eye)
rlm@123:         vision-image
rlm@123:         (atom
rlm@123:          (BufferedImage. (.getWidth camera)
rlm@123:                          (.getHeight camera)
rlm@123:                          BufferedImage/TYPE_BYTE_BINARY))]
rlm@123:     [(fn [world]
rlm@123:        (add-eye
rlm@123:         world camera
rlm@123:         (let [counter  (atom 0)]
rlm@123:           (fn [r fb bb bi]
rlm@123:             (if (zero? (rem (swap! counter inc) (inc skip)))
rlm@123:               (reset! vision-image (BufferedImage! r fb bb bi)))))))
rlm@123:      (vec
rlm@123:       (map
rlm@123:        (fn [[key image]]
rlm@123:          (let [whites (white-coordinates image)
rlm@123:                topology (vec (collapse whites))
rlm@123:                mask (presets key)]
rlm@123:            (fn []
rlm@123:              (vector
rlm@123:               topology
rlm@123:               (vec 
rlm@123:                (for [[x y] whites]
rlm@123:                  (bit-and
rlm@123:                   mask (.getRGB @vision-image x y))))))))
rlm@123:        retinal-map))]))
rlm@118: 
rlm@116: (defn vision
rlm@121:   [#^Node creature & {skip :skip :or {skip 0}}]
rlm@121:   (reduce
rlm@121:    (fn [[init-a senses-a]
rlm@121:         [init-b senses-b]]
rlm@121:      [(conj init-a init-b)
rlm@121:       (into senses-a senses-b)])
rlm@121:    [[][]]      
rlm@121:    (for [eye (creature-eyes creature)]
rlm@121:      (enable-vision creature eye))))
rlm@128: 
rlm@128: 
rlm@128: 
rlm@128: 
rlm@128: 
rlm@128: ;; lower level --- nodes
rlm@128: ;; closest-node "parse/compile-x" -> makes organ, which is spatial, fn pair
rlm@128: 
rlm@128: ;; higher level -- organs
rlm@128: ;; 
rlm@128: 
rlm@128: ;; higher level --- sense/effector
rlm@128: ;; these are the functions that provide world i/o, chinese-room style
rlm@128: 
rlm@128: 
rlm@116: 
rlm@116: (defn blender-creature
rlm@116:   "Return a creature with all joints in place."
rlm@116:   [blender-path]
rlm@116:   (let [model (load-blender-model blender-path)
rlm@116:         joints
rlm@116:         (if-let [joint-node (.getChild model "joints")]
rlm@116:           (seq (.getChildren joint-node))
rlm@116:           (do (println-repl "could not find joints node") []))]
rlm@116:   (assemble-creature model joints)))
rlm@116: 
rlm@126: (defn gray-scale [num]
rlm@126:   (+ num
rlm@126:      (bit-shift-left num 8)
rlm@126:      (bit-shift-left num 16)))
rlm@126: 
rlm@103: (defn debug-window
rlm@103:   "creates function that offers a debug view of sensor data"
rlm@103:   []
rlm@103:   (let [vi (view-image)]
rlm@103:     (fn 
rlm@103:       [[coords sensor-data]]
rlm@103:       (let [image (points->image coords)]
rlm@103:         (dorun
rlm@103:          (for [i (range (count coords))]
rlm@103:            (.setRGB image ((coords i) 0) ((coords i) 1)
rlm@126:                     (gray-scale (sensor-data i)))))
rlm@126: 
rlm@126:                    
rlm@103:         (vi image)))))
rlm@103: 
rlm@118: (defn debug-vision-window
rlm@118:   "creates function that offers a debug view of sensor data"
rlm@118:   []
rlm@118:   (let [vi (view-image)]
rlm@118:     (fn 
rlm@118:       [[coords sensor-data]]
rlm@118:       (let [image (points->image coords)]
rlm@118:         (dorun
rlm@118:          (for [i (range (count coords))]
rlm@118:            (.setRGB image ((coords i) 0) ((coords i) 1)
rlm@118:                    (sensor-data i))))
rlm@118:         (vi image)))))
rlm@118: 
rlm@123: (defn debug-hearing-window
rlm@123:   "view audio data"
rlm@123:   [height]
rlm@123:   (let [vi (view-image)]
rlm@123:     (fn [[coords sensor-data]]
rlm@123:       (let [image (BufferedImage. (count coords) height
rlm@123:                                   BufferedImage/TYPE_INT_RGB)]
rlm@123:         (dorun
rlm@123:          (for [x (range (count coords))]
rlm@123:            (dorun
rlm@123:             (for [y (range height)]
rlm@123:               (let [raw-sensor (sensor-data x)]
rlm@126:                 (.setRGB image x y (gray-scale raw-sensor)))))))
rlm@126: 
rlm@123:         (vi image)))))
rlm@123:         
rlm@123:          
rlm@123: 
rlm@106: ;;(defn test-touch [world creature]
rlm@83: 
rlm@78: 
rlm@123: 
rlm@123: 
rlm@129: ;; here's how motor-control/ proprioception will work:
rlm@129: 
rlm@129: 
rlm@129: 
rlm@129: 
rlm@129: 
rlm@123: 
rlm@123: 
rlm@106: (defn test-creature [thing]
rlm@106:   (let [x-axis
rlm@106:         (box 1 0.01 0.01 :physical? false :color ColorRGBA/Red)
rlm@106:         y-axis
rlm@106:         (box 0.01 1 0.01 :physical? false :color ColorRGBA/Green)
rlm@106:         z-axis
rlm@106:         (box 0.01 0.01 1 :physical? false :color ColorRGBA/Blue)
rlm@106:         creature (blender-creature thing)
rlm@106:         touch-nerves (touch creature)
rlm@106:         touch-debug-windows (map (fn [_] (debug-window)) touch-nerves)
rlm@121:         [init-vision-fns vision-data] (vision creature)
rlm@121:         vision-debug (map (fn [_] (debug-vision-window)) vision-data)
rlm@118:         me (sphere 0.5 :color ColorRGBA/Blue :physical? false)
rlm@123:         [init-hearing-fns hearing-senses] (hearing creature)
rlm@123:         hearing-windows (map (fn [_] (debug-hearing-window 50))
rlm@123:                              hearing-senses)
rlm@124:         bell (AudioNode. (asset-manager)
rlm@128:                          "Sounds/pure.wav" false)
rlm@123:         ;; dream
rlm@123: 
rlm@106:         ]
rlm@106:   (world
rlm@106:    (nodify [creature
rlm@106:             (box 10 2 10 :position (Vector3f. 0 -9 0)
rlm@106:                  :color ColorRGBA/Gray :mass 0)
rlm@106:             x-axis y-axis z-axis
rlm@118:             me
rlm@106:             ])
rlm@123:    (merge standard-debug-controls
rlm@123:           {"key-return"
rlm@123:            (fn [_ value]
rlm@123:              (if value
rlm@123:                (do
rlm@123:                  (println-repl "play-sound")
rlm@124:                  (.play bell))))})
rlm@106:    (fn [world]
rlm@106:      (light-up-everything world)
rlm@106:      (enable-debug world)
rlm@122:      (dorun (map #(% world) init-vision-fns))
rlm@123:      (dorun (map #(% world) init-hearing-fns))
rlm@118:      
rlm@118:      (add-eye world
rlm@118:               (attach-eye creature (test-eye))
rlm@118:               (comp (view-image) BufferedImage!))
rlm@118:      
rlm@118:      (add-eye world (.getCamera world) no-op)
rlm@118:      
rlm@106:      ;;(com.aurellem.capture.Capture/captureVideo
rlm@106:      ;; world (file-str "/home/r/proj/ai-videos/hand"))
rlm@110:      ;;(.setTimer world (RatchetTimer. 60))
rlm@119:      (speed-up world)
rlm@106:      ;;(set-gravity world (Vector3f. 0 0 0))
rlm@106:      )
rlm@106:    (fn [world tpf]
rlm@109:      ;;(dorun 
rlm@109:      ;; (map #(%1 %2) touch-nerves (repeat (.getRootNode world))))
rlm@123:      
rlm@123: 
rlm@123:      
rlm@106:      (dorun
rlm@109:       (map #(%1 (%2 (.getRootNode world)))
rlm@121:            touch-debug-windows touch-nerves))
rlm@123:      
rlm@121:      (dorun
rlm@121:       (map #(%1 (%2))
rlm@121:            vision-debug vision-data))
rlm@123:      (dorun
rlm@123:       (map #(%1 (%2)) hearing-windows hearing-senses))
rlm@123:      
rlm@123:      
rlm@118:      ;;(println-repl (vision-data))
rlm@118:      (.setLocalTranslation me (.getLocation (.getCamera world)))
rlm@118:      
rlm@121: 
rlm@106:      )
rlm@106:    ;;(let [timer (atom 0)]
rlm@106:    ;;  (fn [_ _]
rlm@106:    ;;    (swap! timer inc)
rlm@106:    ;;    (if (= (rem @timer 60) 0)
rlm@106:    ;;      (println-repl (float (/ @timer 60))))))
rlm@106:    )))
rlm@83: 
rlm@109: 
rlm@109: 
rlm@109: 
rlm@109: 
rlm@109: 
rlm@109: 
rlm@109: 
rlm@109: 
rlm@109: ;;; experiments in collisions
rlm@109: 
rlm@109: 
rlm@109: 
rlm@109: (defn collision-test []
rlm@110:   (let [b-radius 1
rlm@110:         b-position (Vector3f. 0 0 0)
rlm@109:         obj-b (box 1 1 1 :color ColorRGBA/Blue
rlm@109:                       :position b-position
rlm@110:                       :mass 0)
rlm@110:         node (nodify [obj-b])
rlm@110:         bounds-b
rlm@110:         (doto (Picture.)
rlm@110:           (.setHeight 50)
rlm@110:           (.setWidth 50)
rlm@110:           (.setImage (asset-manager)
rlm@110:                      "Models/creature1/hand.png"
rlm@110:                      false
rlm@110:           ))
rlm@110:         
rlm@110:         ;;(Ray. (Vector3f. 0 -5 0) (.normalize (Vector3f. 0 1 0)))
rlm@110: 
rlm@110:         collisions
rlm@110:         (let [cr (CollisionResults.)]
rlm@110:           (.collideWith node bounds-b cr)
rlm@110:           (println (map #(.getContactPoint %) cr))
rlm@110:           cr)
rlm@110: 
rlm@110:         ;;collision-points
rlm@110:         ;;(map #(sphere 0.1 :position (.getContactPoint %))
rlm@110:         ;;     collisions)
rlm@110: 
rlm@110:         ;;node (nodify (conj collision-points obj-b))
rlm@110:         
rlm@109:         sim
rlm@109:         (world node
rlm@110:                {"key-space"
rlm@129: vvvvvv                (fn [_ value]
rlm@110:                   (if value
rlm@110:                     (let [cr (CollisionResults.)]
rlm@110:                       (.collideWith node bounds-b cr)
rlm@110:                       (println-repl (map #(.getContactPoint %) cr))
rlm@110:                       cr)))}
rlm@109:                no-op
rlm@109:                no-op)
rlm@109: 
rlm@109:         ]
rlm@110:     sim
rlm@109:     
rlm@109: ))
rlm@109:    
rlm@116: 
rlm@116: ;; the camera will stay in its initial position/rotation with relation
rlm@116: ;; to the spatial.
rlm@116: 
rlm@116: 
rlm@117: (defn follow-test
rlm@117:   "show a camera that stays in the same relative position to a blue cube."
rlm@117:   []
rlm@116:   (let [camera-pos (Vector3f. 0 30 0)
rlm@116:         rock (box 1 1 1 :color ColorRGBA/Blue
rlm@116:                      :position (Vector3f. 0 10 0)
rlm@116:                      :mass 30
rlm@116:                      )
rlm@118:         rot (.getWorldRotation rock)
rlm@116:         
rlm@116:         table (box 3 1 10 :color ColorRGBA/Gray :mass 0
rlm@116:                    :position (Vector3f. 0 -3 0))]
rlm@116: 
rlm@116:     (world
rlm@116:      (nodify [rock table])
rlm@116:              standard-debug-controls
rlm@116:              (fn [world]
rlm@116:                (let 
rlm@116:                    [cam (doto (.clone (.getCamera world))
rlm@116:                           (.setLocation camera-pos)
rlm@116:                           (.lookAt Vector3f/ZERO
rlm@116:                                    Vector3f/UNIT_X))]
rlm@123:                  (bind-sense rock cam)
rlm@116:                  
rlm@116:                  (.setTimer world (RatchetTimer. 60))
rlm@116:                  (add-eye world cam (comp (view-image) BufferedImage!))
rlm@116:                  (add-eye world (.getCamera world) no-op))
rlm@116:                )
rlm@118:              (fn [_ _] (println-repl rot)))))
rlm@116:        
rlm@118: 
rlm@123: 
rlm@87: #+end_src
rlm@83: 
rlm@87: #+results: body-1
rlm@109: : #'cortex.silly/test-creature
rlm@78: 
rlm@78: 
rlm@78: * COMMENT purgatory
rlm@78: #+begin_src clojure
rlm@77: (defn bullet-trans []
rlm@77:   (let [obj-a (sphere 0.5 :color ColorRGBA/Red
rlm@77:                       :position (Vector3f. -10 5 0))
rlm@77:         obj-b (sphere 0.5 :color ColorRGBA/Blue
rlm@77:                       :position (Vector3f. -10 -5 0)
rlm@77:                       :mass 0)
rlm@77:         control-a (.getControl obj-a RigidBodyControl)
rlm@77:         control-b (.getControl obj-b RigidBodyControl)
rlm@77:         swivel
rlm@77:         (.toRotationMatrix
rlm@77:          (doto (Quaternion.)
rlm@77:            (.fromAngleAxis (/ Math/PI 2)
rlm@77:                            Vector3f/UNIT_X)))]
rlm@77:     (doto 
rlm@77:         (ConeJoint.
rlm@77:          control-a control-b
rlm@77:          (Vector3f. 0 5 0)
rlm@77:          (Vector3f. 0 -5 0)
rlm@77:          swivel swivel)
rlm@77:       (.setLimit (* 0.6 (/ Math/PI 4))
rlm@77:                  (/ Math/PI 4)
rlm@77:                  (* Math/PI 0.8)))
rlm@77:     (world (nodify
rlm@77:             [obj-a obj-b])
rlm@77:             standard-debug-controls
rlm@77:             enable-debug
rlm@77:             no-op)))
rlm@74: 
rlm@74: 
rlm@77: (defn bullet-trans* []
rlm@77:   (let [obj-a (box 1.5 0.5 0.5 :color ColorRGBA/Red
rlm@77:                    :position (Vector3f. 5 0 0)
rlm@77:                    :mass 90)
rlm@77:         obj-b (sphere 0.5 :color ColorRGBA/Blue
rlm@77:                       :position (Vector3f. -5 0 0)
rlm@77:                       :mass 0)
rlm@77:         control-a (.getControl obj-a RigidBodyControl)
rlm@77:         control-b (.getControl obj-b RigidBodyControl)
rlm@77:         move-up? (atom nil)
rlm@77:         move-down? (atom nil)
rlm@77:         move-left? (atom nil)
rlm@77:         move-right? (atom nil)
rlm@77:         roll-left? (atom nil)
rlm@77:         roll-right? (atom nil)
rlm@77:         force 100
rlm@77:         swivel
rlm@77:         (.toRotationMatrix
rlm@77:          (doto (Quaternion.)
rlm@77:            (.fromAngleAxis (/ Math/PI 2)
rlm@77:                            Vector3f/UNIT_X)))
rlm@77:         x-move
rlm@77:         (doto (Matrix3f.)
rlm@77:           (.fromStartEndVectors Vector3f/UNIT_X
rlm@77:                                 (.normalize (Vector3f. 1 1 0))))
rlm@77:         
rlm@77:         timer (atom 0)]
rlm@77:     (doto 
rlm@77:         (ConeJoint.
rlm@77:          control-a control-b
rlm@77:          (Vector3f. -8 0 0)
rlm@77:          (Vector3f. 2 0 0)
rlm@77:          ;;swivel swivel
rlm@77:          ;;Matrix3f/IDENTITY Matrix3f/IDENTITY
rlm@77:          x-move Matrix3f/IDENTITY 
rlm@77:          )
rlm@77:       (.setCollisionBetweenLinkedBodys false)
rlm@77:       (.setLimit (* 1 (/ Math/PI 4))    ;; twist 
rlm@77:                  (* 1 (/ Math/PI 4))  ;; swing span in X-Y plane
rlm@77:                  (* 0 (/ Math/PI 4))))  ;; swing span in Y-Z plane
rlm@77:     (world (nodify
rlm@77:             [obj-a obj-b])
rlm@77:            (merge standard-debug-controls
rlm@77:                   {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
rlm@77:                    "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
rlm@77:                    "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
rlm@77:                    "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
rlm@77:                    "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
rlm@77:                    "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})     
rlm@77:            
rlm@77:            (fn [world]
rlm@77:              (enable-debug world)
rlm@77:              (set-gravity world Vector3f/ZERO)
rlm@77:              )
rlm@77:                            
rlm@77:            (fn [world _]
rlm@77: 
rlm@77:              (if @move-up?
rlm@77:                (.applyForce control-a
rlm@77:                               (Vector3f. force 0 0)
rlm@77:                               (Vector3f. 0 0 0)))
rlm@77:              (if @move-down?
rlm@77:                (.applyForce control-a
rlm@77:                              (Vector3f. (- force) 0 0)
rlm@77:                              (Vector3f. 0 0 0)))
rlm@77:              (if @move-left?
rlm@77:                (.applyForce control-a
rlm@77:                              (Vector3f. 0 force 0)
rlm@77:                              (Vector3f. 0 0 0)))
rlm@77:              (if @move-right?
rlm@77:                (.applyForce control-a
rlm@77:                              (Vector3f. 0 (- force) 0)
rlm@77:                              (Vector3f. 0 0 0)))
rlm@77: 
rlm@77:              (if @roll-left?
rlm@77:                (.applyForce control-a
rlm@77:                              (Vector3f. 0 0 force)
rlm@77:                              (Vector3f. 0 0 0)))
rlm@77:              (if @roll-right?
rlm@77:                (.applyForce control-a
rlm@77:                              (Vector3f. 0 0 (- force))
rlm@77:                              (Vector3f. 0 0 0)))
rlm@77: 
rlm@77:              (if (zero? (rem (swap! timer inc) 100))
rlm@77:                (.attachChild
rlm@77:                 (.getRootNode world)
rlm@77:                 (sphere 0.05 :color ColorRGBA/Yellow
rlm@77:                         :physical? false :position
rlm@77:                         (.getWorldTranslation obj-a)))))
rlm@77:            )
rlm@77:     ))
rlm@77: 
rlm@94: (defn transform-trianglesdsd
rlm@94:   "Transform that converts each vertex in the first triangle
rlm@94:   into the corresponding vertex in the second triangle."
rlm@94:   [#^Triangle tri-1 #^Triangle tri-2]
rlm@94:   (let [in  [(.get1 tri-1)
rlm@94:              (.get2 tri-1)
rlm@94:              (.get3 tri-1)]
rlm@94:         out [(.get1 tri-2)
rlm@94:              (.get2 tri-2)
rlm@94:              (.get3 tri-2)]]
rlm@94:     (let [translate (doto (Matrix4f.) (.setTranslation (.negate (in 0))))
rlm@94:           in* [(.mult translate (in 0))
rlm@94:                (.mult translate (in 1))
rlm@94:                (.mult translate (in 2))]
rlm@94:           final-translation
rlm@94:           (doto (Matrix4f.)
rlm@94:             (.setTranslation (out 1)))
rlm@94:           
rlm@94:           rotate-1
rlm@94:           (doto (Matrix3f.)
rlm@94:             (.fromStartEndVectors
rlm@94:              (.normalize
rlm@94:               (.subtract
rlm@94:                (in* 1) (in* 0)))
rlm@94:              (.normalize
rlm@94:               (.subtract
rlm@94:                (out 1) (out 0)))))
rlm@94:           in** [(.mult rotate-1 (in* 0))
rlm@94:                 (.mult rotate-1 (in* 1))
rlm@94:                 (.mult rotate-1 (in* 2))]
rlm@94:           scale-factor-1
rlm@94:           (.mult
rlm@94:            (.normalize
rlm@94:             (.subtract
rlm@94:              (out 1)
rlm@94:              (out 0)))
rlm@94:            (/ (.length
rlm@94:                (.subtract (out 1)
rlm@94:                           (out 0)))
rlm@94:               (.length
rlm@94:                (.subtract (in** 1)
rlm@94:                           (in** 0)))))
rlm@94:           scale-1 (doto (Matrix4f.) (.setScale scale-factor-1))
rlm@94:           in*** [(.mult scale-1 (in** 0))
rlm@94:                  (.mult scale-1 (in** 1))
rlm@94:                  (.mult scale-1 (in** 2))]
rlm@94:           
rlm@94:           
rlm@94:           
rlm@94:           
rlm@94:           
rlm@94:           ]
rlm@94:       
rlm@94:       (dorun (map println in))
rlm@94:       (println)
rlm@94:       (dorun (map println in*))
rlm@94:       (println)
rlm@94:       (dorun (map println in**))
rlm@94:       (println)
rlm@94:       (dorun (map println in***))
rlm@94:       (println)
rlm@94:       
rlm@99:     ))))
rlm@94: 
rlm@94: 
rlm@106: (defn world-setup [joint]
rlm@106:   (let [joint-position (Vector3f. 0 0 0)
rlm@106:         joint-rotation
rlm@106:         (.toRotationMatrix
rlm@106:          (.mult
rlm@106:           (doto (Quaternion.)
rlm@106:             (.fromAngleAxis
rlm@106:              (* 1 (/ Math/PI 4))
rlm@106:              (Vector3f. -1 0 0)))
rlm@106:           (doto (Quaternion.)
rlm@106:             (.fromAngleAxis
rlm@106:              (* 1 (/ Math/PI 2))
rlm@106:              (Vector3f. 0 0 1)))))
rlm@106:         top-position (.mult joint-rotation (Vector3f. 8 0 0))
rlm@106:         
rlm@106:         origin (doto
rlm@106:                    (sphere 0.1 :physical? false :color ColorRGBA/Cyan
rlm@106:                            :position top-position))
rlm@106:         top (doto
rlm@106:                 (sphere 0.1 :physical? false :color ColorRGBA/Yellow
rlm@106:                         :position top-position)
rlm@106: 
rlm@106:               (.addControl
rlm@106:                (RigidBodyControl.
rlm@106:                 (CapsuleCollisionShape. 0.5 1.5 1) (float 20))))
rlm@106:         bottom (doto
rlm@106:                    (sphere 0.1 :physical? false :color ColorRGBA/DarkGray
rlm@106:                            :position (Vector3f. 0 0 0))
rlm@106:                  (.addControl
rlm@106:                (RigidBodyControl.
rlm@106:                 (CapsuleCollisionShape. 0.5 1.5 1) (float 0))))
rlm@106:         table (box 10 2 10 :position (Vector3f. 0 -20 0)
rlm@106:                    :color ColorRGBA/Gray :mass 0)
rlm@106:         a (.getControl top RigidBodyControl)
rlm@106:         b (.getControl bottom RigidBodyControl)]
rlm@106: 
rlm@106:     (cond
rlm@106:      (= joint :cone)
rlm@106:      
rlm@106:        (doto (ConeJoint.
rlm@106:               a b
rlm@106:               (world-to-local top joint-position)
rlm@106:               (world-to-local bottom joint-position)
rlm@106:               joint-rotation
rlm@106:               joint-rotation
rlm@106:               )
rlm@106: 
rlm@106:               
rlm@106:          (.setLimit (* (/ 10) Math/PI)
rlm@106:                     (* (/ 4)  Math/PI)
rlm@106:                     0)))
rlm@106:   [origin top bottom table]))
rlm@106:             
rlm@106: (defn test-joint [joint]
rlm@106:   (let [[origin top bottom floor] (world-setup joint)
rlm@106:         control (.getControl top RigidBodyControl)
rlm@106:         move-up? (atom false)
rlm@106:         move-down? (atom false)
rlm@106:         move-left? (atom false)
rlm@106:         move-right? (atom false)
rlm@106:         roll-left? (atom false)
rlm@106:         roll-right? (atom false)
rlm@106:         timer (atom 0)]
rlm@106: 
rlm@106:     (world
rlm@106:      (nodify [top bottom floor origin])
rlm@106:      (merge standard-debug-controls
rlm@106:             {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))
rlm@106:              "key-t" (fn [_ pressed?] (reset! move-down? pressed?))
rlm@106:              "key-f" (fn [_ pressed?] (reset! move-left? pressed?))
rlm@106:              "key-g" (fn [_ pressed?] (reset! move-right? pressed?))
rlm@106:              "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))
rlm@106:              "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})     
rlm@106:      
rlm@106:      (fn [world]
rlm@106:        (light-up-everything world)
rlm@106:        (enable-debug world)
rlm@106:        (set-gravity world (Vector3f. 0 0 0))
rlm@106:        )
rlm@106:      
rlm@106:      (fn [world _]
rlm@106:        (if (zero? (rem (swap! timer inc) 100))
rlm@106:          (do
rlm@106:          ;;  (println-repl @timer)
rlm@106:            (.attachChild (.getRootNode world)
rlm@106:                        (sphere 0.05 :color ColorRGBA/Yellow
rlm@106:                                :position (.getWorldTranslation top)
rlm@106:                                :physical? false))
rlm@106:            (.attachChild (.getRootNode world)
rlm@106:                          (sphere 0.05 :color ColorRGBA/LightGray
rlm@106:                                  :position (.getWorldTranslation bottom)
rlm@106:                                  :physical? false))))
rlm@106: 
rlm@106:        (if @move-up?
rlm@106:          (.applyTorque control
rlm@106:                        (.mult (.getPhysicsRotation control)
rlm@106:                               (Vector3f. 0 0 10))))
rlm@106:        (if @move-down?
rlm@106:          (.applyTorque control
rlm@106:                        (.mult (.getPhysicsRotation control)
rlm@106:                               (Vector3f. 0 0 -10))))
rlm@106:        (if @move-left?
rlm@106:          (.applyTorque control
rlm@106:                        (.mult (.getPhysicsRotation control)
rlm@106:                               (Vector3f. 0 10 0))))
rlm@106:        (if @move-right?
rlm@106:          (.applyTorque control
rlm@106:                        (.mult (.getPhysicsRotation control)
rlm@106:                               (Vector3f. 0 -10 0))))
rlm@106:        (if @roll-left?
rlm@106:          (.applyTorque control
rlm@106:                        (.mult (.getPhysicsRotation control)
rlm@106:                               (Vector3f. -1 0 0))))
rlm@106:        (if @roll-right?
rlm@106:          (.applyTorque control
rlm@106:                        (.mult (.getPhysicsRotation control)
rlm@106:                               (Vector3f. 1 0 0))))))))
rlm@106: 
rlm@99: 
rlm@99: 
rlm@107: (defprotocol Frame
rlm@107:   (frame [this]))
rlm@107: 
rlm@107: (extend-type BufferedImage
rlm@107:   Frame
rlm@107:   (frame [image]
rlm@107:     (merge  
rlm@107:      (apply
rlm@107:       hash-map 
rlm@107:       (interleave 
rlm@107:        (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]
rlm@107:                 (vector x y)))
rlm@107:        (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]
rlm@107:                 (let [data (.getRGB image x y)] 
rlm@107:                   (hash-map :r (bit-shift-right (bit-and 0xff0000 data) 16)
rlm@107:                             :g (bit-shift-right (bit-and 0x00ff00 data) 8)
rlm@107:                             :b (bit-and 0x0000ff data)))))))
rlm@107:      {:width  (.getWidth image) :height (.getHeight image)})))
rlm@107: 
rlm@107: 
rlm@107: (extend-type ImagePlus
rlm@107:   Frame
rlm@107:   (frame [image+]
rlm@107:     (frame (.getBufferedImage image+))))
rlm@107: 
rlm@107: 
rlm@99: #+end_src
rlm@99: 
rlm@99: 
rlm@99: * COMMENT generate source
rlm@99: #+begin_src clojure :tangle ../src/cortex/silly.clj
rlm@99: <<body-1>>
rlm@99: #+end_src
rlm@99: 
rlm@99: 
rlm@94:   
rlm@94: