rlm@73: #+title: First attempt at a creature! rlm@73: #+author: Robert McIntyre rlm@73: #+email: rlm@mit.edu rlm@73: #+description: rlm@73: #+keywords: simulation, jMonkeyEngine3, clojure rlm@73: #+SETUPFILE: ../../aurellem/org/setup.org rlm@73: #+INCLUDE: ../../aurellem/org/level-0.org rlm@73: rlm@99: rlm@99: * objectives rlm@99: - [ ] get an overall bitmap-like image for touch rlm@99: - [ ] write code to visuliaze this bitmap rlm@99: - [ ] directly change the UV-pixels to show touch sensor activation rlm@99: - [ ] write an explination for why b&w bitmaps for senses is appropiate rlm@99: - [ ] clean up touch code and write visulazation test rlm@99: - [ ] do the same for eyes rlm@99: rlm@73: * Intro rlm@73: So far, I've made the following senses -- rlm@73: - Vision rlm@73: - Hearing rlm@73: - Touch rlm@73: - Proprioception rlm@73: rlm@73: And one effector: rlm@73: - Movement rlm@73: rlm@73: However, the code so far has only enabled these senses, but has not rlm@73: actually implemented them. For example, there is still a lot of work rlm@73: to be done for vision. I need to be able to create an /eyeball/ in rlm@73: simulation that can be moved around and see the world from different rlm@73: angles. I also need to determine weather to use log-polar or cartesian rlm@73: for the visual input, and I need to determine how/wether to rlm@73: disceritise the visual input. rlm@73: rlm@73: I also want to be able to visualize both the sensors and the rlm@73: effectors in pretty pictures. This semi-retarted creature will by my rlm@73: first attempt at bringing everything together. rlm@73: rlm@73: * The creature's body rlm@73: rlm@73: Still going to do an eve-like body in blender, but due to problems rlm@73: importing the joints, etc into jMonkeyEngine3, I',m going to do all rlm@73: the connecting here in clojure code, using the names of the individual rlm@73: components and trial and error. Later, I'll maybe make some sort of rlm@73: creature-building modifications to blender that support whatever rlm@73: discreitized senses I'm going to make. rlm@73: rlm@73: #+name: body-1 rlm@73: #+begin_src clojure rlm@73: (ns cortex.silly rlm@73: "let's play!" rlm@73: {:author "Robert McIntyre"}) rlm@73: rlm@73: ;; TODO remove this! rlm@73: (require 'cortex.import) rlm@73: (cortex.import/mega-import-jme3) rlm@73: (use '(cortex world util body hearing touch vision)) rlm@73: rlm@73: (rlm.rlm-commands/help) rlm@99: (import java.awt.image.BufferedImage) rlm@99: (import javax.swing.JPanel) rlm@99: (import javax.swing.SwingUtilities) rlm@99: (import java.awt.Dimension) rlm@99: (import javax.swing.JFrame) rlm@99: (import java.awt.Dimension) rlm@99: (declare joint-create) rlm@73: rlm@99: (defn view-image rlm@99: "Initailizes a JPanel on which you may draw a BufferedImage. rlm@99: Returns a function that accepts a BufferedImage and draws it to the rlm@99: JPanel." rlm@99: [] rlm@99: (let [image rlm@99: (atom rlm@99: (BufferedImage. 1 1 BufferedImage/TYPE_4BYTE_ABGR)) rlm@99: panel rlm@99: (proxy [JPanel] [] rlm@99: (paint rlm@99: [graphics] rlm@99: (proxy-super paintComponent graphics) rlm@99: (.drawImage graphics @image 0 0 nil))) rlm@99: frame (JFrame. "Display Image")] rlm@99: (SwingUtilities/invokeLater rlm@99: (fn [] rlm@99: (doto frame rlm@99: (-> (.getContentPane) (.add panel)) rlm@99: (.pack) rlm@99: (.setLocationRelativeTo nil) rlm@99: (.setResizable true) rlm@99: (.setVisible true)))) rlm@99: (fn [#^BufferedImage i] rlm@99: (reset! image i) rlm@99: (.setSize frame (+ 8 (.getWidth i)) (+ 28 (.getHeight i))) rlm@99: (.repaint panel 0 0 (.getWidth i) (.getHeight i))))) rlm@99: rlm@99: rlm@99: (defn visualize [points] rlm@99: rlm@99: rlm@99: rlm@99: (defn collapse rlm@99: "Take a set of pairs of integers and collapse them into a rlm@99: contigous bitmap." rlm@99: [points] rlm@99: (let [center [0 0]] rlm@99: rlm@99: ) rlm@99: rlm@99: rlm@99: rlm@99: rlm@99: rlm@99: rlm@99: rlm@99: rlm@99: rlm@83: rlm@83: (defn load-bullet [] rlm@84: (let [sim (world (Node.) {} no-op no-op)] rlm@84: (.enqueue rlm@84: sim rlm@84: (fn [] rlm@84: (.stop sim))) rlm@84: (.start sim))) rlm@83: rlm@73: (defn load-blender-model rlm@73: "Load a .blend file using an asset folder relative path." rlm@73: [^String model] rlm@73: (.loadModel rlm@73: (doto (asset-manager) rlm@73: (.registerLoader BlenderModelLoader (into-array String ["blend"]))) rlm@73: model)) rlm@73: rlm@74: (defn meta-data [blender-node key] rlm@74: (if-let [data (.getUserData blender-node "properties")] rlm@74: (.findValue data key) rlm@74: nil)) rlm@73: rlm@78: (defn blender-to-jme rlm@78: "Convert from Blender coordinates to JME coordinates" rlm@78: [#^Vector3f in] rlm@78: (Vector3f. (.getX in) rlm@78: (.getZ in) rlm@78: (- (.getY in)))) rlm@74: rlm@79: (defn jme-to-blender rlm@79: "Convert from JME coordinates to Blender coordinates" rlm@79: [#^Vector3f in] rlm@79: (Vector3f. (.getX in) rlm@79: (- (.getZ in)) rlm@79: (.getY in))) rlm@79: rlm@78: (defn joint-targets rlm@78: "Return the two closest two objects to the joint object, ordered rlm@78: from bottom to top according to the joint's rotation." rlm@78: [#^Node parts #^Node joint] rlm@78: ;;(println (meta-data joint "joint")) rlm@78: (.getWorldRotation joint) rlm@78: (loop [radius (float 0.01)] rlm@78: (let [results (CollisionResults.)] rlm@78: (.collideWith rlm@78: parts rlm@78: (BoundingBox. (.getWorldTranslation joint) rlm@78: radius radius radius) rlm@78: results) rlm@78: (let [targets rlm@78: (distinct rlm@78: (map #(.getGeometry %) results))] rlm@78: (if (>= (count targets) 2) rlm@78: (sort-by rlm@79: #(let [v rlm@79: (jme-to-blender rlm@79: (.mult rlm@79: (.inverse (.getWorldRotation joint)) rlm@79: (.subtract (.getWorldTranslation %) rlm@79: (.getWorldTranslation joint))))] rlm@79: (println-repl (.getName %) ":" v) rlm@79: (.dot (Vector3f. 1 1 1) rlm@79: v)) rlm@78: (take 2 targets)) rlm@78: (recur (float (* radius 2)))))))) rlm@74: rlm@87: (defn world-to-local rlm@87: "Convert the world coordinates into coordinates relative to the rlm@87: object (i.e. local coordinates), taking into account the rotation rlm@87: of object." rlm@87: [#^Spatial object world-coordinate] rlm@87: (let [out (Vector3f.)] rlm@88: (.worldToLocal object world-coordinate out) out)) rlm@87: rlm@96: (defn local-to-world rlm@96: "Convert the local coordinates into coordinates into world relative rlm@96: coordinates" rlm@96: [#^Spatial object local-coordinate] rlm@96: (let [world-coordinate (Vector3f.)] rlm@96: (.localToWorld object local-coordinate world-coordinate) rlm@96: world-coordinate)) rlm@96: rlm@96: rlm@87: (defmulti joint-dispatch rlm@87: "Translate blender pseudo-joints into real JME joints." rlm@88: (fn [constraints & _] rlm@87: (:type constraints))) rlm@87: rlm@87: (defmethod joint-dispatch :point rlm@87: [constraints control-a control-b pivot-a pivot-b rotation] rlm@87: (println-repl "creating POINT2POINT joint") rlm@87: (Point2PointJoint. rlm@87: control-a rlm@87: control-b rlm@87: pivot-a rlm@87: pivot-b)) rlm@87: rlm@87: (defmethod joint-dispatch :hinge rlm@87: [constraints control-a control-b pivot-a pivot-b rotation] rlm@87: (println-repl "creating HINGE joint") rlm@87: (let [axis rlm@87: (if-let rlm@87: [axis (:axis constraints)] rlm@87: axis rlm@87: Vector3f/UNIT_X) rlm@87: [limit-1 limit-2] (:limit constraints) rlm@87: hinge-axis rlm@87: (.mult rlm@87: rotation rlm@87: (blender-to-jme axis))] rlm@87: (doto rlm@87: (HingeJoint. rlm@87: control-a rlm@87: control-b rlm@87: pivot-a rlm@87: pivot-b rlm@87: hinge-axis rlm@87: hinge-axis) rlm@87: (.setLimit limit-1 limit-2)))) rlm@87: rlm@87: (defmethod joint-dispatch :cone rlm@87: [constraints control-a control-b pivot-a pivot-b rotation] rlm@87: (let [limit-xz (:limit-xz constraints) rlm@87: limit-xy (:limit-xy constraints) rlm@87: twist (:twist constraints)] rlm@87: rlm@87: (println-repl "creating CONE joint") rlm@87: (println-repl rotation) rlm@87: (println-repl rlm@87: "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0))) rlm@87: (println-repl rlm@87: "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0))) rlm@87: (println-repl rlm@87: "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1))) rlm@87: (doto rlm@87: (ConeJoint. rlm@87: control-a rlm@87: control-b rlm@87: pivot-a rlm@87: pivot-b rlm@87: rotation rlm@87: rotation) rlm@87: (.setLimit (float limit-xz) rlm@87: (float limit-xy) rlm@87: (float twist))))) rlm@87: rlm@88: (defn connect rlm@87: "here are some examples: rlm@87: {:type :point} rlm@87: {:type :hinge :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)} rlm@87: (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints) rlm@87: rlm@89: {:type :cone :limit-xz 0] rlm@89: :limit-xy 0] rlm@89: :twist 0]} (use XZY rotation mode in blender!)" rlm@87: [#^Node obj-a #^Node obj-b #^Node joint] rlm@87: (let [control-a (.getControl obj-a RigidBodyControl) rlm@87: control-b (.getControl obj-b RigidBodyControl) rlm@87: joint-center (.getWorldTranslation joint) rlm@87: joint-rotation (.toRotationMatrix (.getWorldRotation joint)) rlm@87: pivot-a (world-to-local obj-a joint-center) rlm@87: pivot-b (world-to-local obj-b joint-center)] rlm@89: rlm@87: (if-let [constraints rlm@87: (map-vals rlm@87: eval rlm@87: (read-string rlm@87: (meta-data joint "joint")))] rlm@89: ;; A side-effect of creating a joint registers rlm@89: ;; it with both physics objects which in turn rlm@89: ;; will register the joint with the physics system rlm@89: ;; when the simulation is started. rlm@87: (do rlm@87: (println-repl "creating joint between" rlm@87: (.getName obj-a) "and" (.getName obj-b)) rlm@87: (joint-dispatch constraints rlm@87: control-a control-b rlm@87: pivot-a pivot-b rlm@87: joint-rotation)) rlm@87: (println-repl "could not find joint meta-data!")))) rlm@87: rlm@78: (defn assemble-creature [#^Node pieces joints] rlm@78: (dorun rlm@78: (map rlm@78: (fn [geom] rlm@78: (let [physics-control rlm@78: (RigidBodyControl. rlm@78: (HullCollisionShape. rlm@78: (.getMesh geom)) rlm@78: (if-let [mass (meta-data geom "mass")] rlm@78: (do rlm@78: (println-repl rlm@78: "setting" (.getName geom) "mass to" (float mass)) rlm@78: (float mass)) rlm@78: (float 1)))] rlm@78: rlm@78: (.addControl geom physics-control))) rlm@78: (filter #(isa? (class %) Geometry ) rlm@78: (node-seq pieces)))) rlm@78: (dorun rlm@78: (map rlm@78: (fn [joint] rlm@78: (let [[obj-a obj-b] rlm@78: (joint-targets pieces joint)] rlm@88: (connect obj-a obj-b joint))) rlm@78: joints)) rlm@78: pieces) rlm@74: rlm@78: (defn blender-creature [blender-path] rlm@78: (let [model (load-blender-model blender-path) rlm@78: joints rlm@78: (if-let [joint-node (.getChild model "joints")] rlm@78: (seq (.getChildren joint-node)) rlm@78: (do (println-repl "could not find joints node") rlm@78: []))] rlm@78: (assemble-creature model joints))) rlm@74: rlm@78: (def hand "Models/creature1/one.blend") rlm@74: rlm@78: (def worm "Models/creature1/try-again.blend") rlm@78: rlm@90: (def touch "Models/creature1/touch.blend") rlm@90: rlm@90: (defn worm-model [] (load-blender-model worm)) rlm@90: rlm@80: (defn x-ray [#^ColorRGBA color] rlm@80: (doto (Material. (asset-manager) rlm@80: "Common/MatDefs/Misc/Unshaded.j3md") rlm@80: (.setColor "Color" color) rlm@80: (-> (.getAdditionalRenderState) rlm@80: (.setDepthTest false)))) rlm@80: rlm@78: (defn test-creature [thing] rlm@80: (let [x-axis rlm@80: (box 1 0.01 0.01 :physical? false :color ColorRGBA/Red) rlm@80: y-axis rlm@80: (box 0.01 1 0.01 :physical? false :color ColorRGBA/Green) rlm@80: z-axis rlm@80: (box 0.01 0.01 1 :physical? false :color ColorRGBA/Blue)] rlm@78: (world rlm@78: (nodify [(blender-creature thing) rlm@81: (box 10 2 10 :position (Vector3f. 0 -9 0) rlm@80: :color ColorRGBA/Gray :mass 0) rlm@80: x-axis y-axis z-axis rlm@80: ]) rlm@78: standard-debug-controls rlm@90: (fn [world] rlm@90: (light-up-everything world) rlm@90: (enable-debug world) rlm@90: ;;(com.aurellem.capture.Capture/captureVideo rlm@90: ;; world (file-str "/home/r/proj/ai-videos/hand")) rlm@90: (.setTimer world (NanoTimer.)) rlm@93: (set-gravity world (Vector3f. 0 0 0)) rlm@90: (speed-up world) rlm@90: ) rlm@90: no-op rlm@90: ;;(let [timer (atom 0)] rlm@90: ;; (fn [_ _] rlm@90: ;; (swap! timer inc) rlm@90: ;; (if (= (rem @timer 60) 0) rlm@90: ;; (println-repl (float (/ @timer 60)))))) rlm@90: ))) rlm@90: rlm@91: (defn colorful [] rlm@91: (.getChild (worm-model) "worm-21")) rlm@90: rlm@90: (import jme3tools.converters.ImageToAwt) rlm@90: rlm@90: (import ij.ImagePlus) rlm@90: rlm@90: (defn triangle-indices rlm@90: "Get the triangle vertex indices of a given triangle from a given rlm@90: mesh." rlm@90: [#^Mesh mesh triangle-index] rlm@90: (let [indices (int-array 3)] rlm@90: (.getTriangle mesh triangle-index indices) rlm@90: (vec indices))) rlm@90: rlm@90: (defn uv-coord rlm@90: "Get the uv-coordinates of the vertex named by vertex-index" rlm@90: [#^Mesh mesh vertex-index] rlm@90: (let [UV-buffer rlm@90: (.getData rlm@90: (.getBuffer rlm@90: mesh rlm@90: VertexBuffer$Type/TexCoord))] rlm@91: (Vector2f. rlm@91: (.get UV-buffer (* vertex-index 2)) rlm@91: (.get UV-buffer (+ 1 (* vertex-index 2)))))) rlm@90: rlm@93: (defn tri-uv-coord rlm@93: "Get the uv-cooridnates of the triangle's verticies." rlm@93: [#^Mesh mesh #^Triangle triangle] rlm@93: (map (partial uv-coord mesh) rlm@93: (triangle-indices mesh (.getIndex triangle)))) rlm@93: rlm@91: (defn touch-receptor-image rlm@97: "Return the touch-sensor distribution image in ImagePlus format, or rlm@97: nil if it does not exist." rlm@91: [#^Geometry obj] rlm@97: (let [mat (.getMaterial obj)] rlm@97: (if-let [texture-param rlm@97: (.getTextureParam rlm@97: mat rlm@97: MaterialHelper/TEXTURE_TYPE_DIFFUSE)] rlm@97: (let rlm@97: [texture rlm@97: (.getTextureValue texture-param) rlm@97: im (.getImage texture)] rlm@97: (ImagePlus. rlm@97: "UV-map" rlm@97: (ImageToAwt/convert im false false 0)))))) rlm@91: rlm@91: (import ij.process.ImageProcessor) rlm@91: (import java.awt.image.BufferedImage) rlm@91: rlm@91: (defprotocol Frame rlm@91: (frame [this])) rlm@91: rlm@91: (extend-type BufferedImage rlm@91: Frame rlm@91: (frame [image] rlm@91: (merge rlm@91: (apply rlm@91: hash-map rlm@91: (interleave rlm@91: (doall (for [x (range (.getWidth image)) y (range (.getHeight image))] rlm@91: (vector x y))) rlm@91: (doall (for [x (range (.getWidth image)) y (range (.getHeight image))] rlm@91: (let [data (.getRGB image x y)] rlm@91: (hash-map :r (bit-shift-right (bit-and 0xff0000 data) 16) rlm@91: :g (bit-shift-right (bit-and 0x00ff00 data) 8) rlm@91: :b (bit-and 0x0000ff data))))))) rlm@91: {:width (.getWidth image) :height (.getHeight image)}))) rlm@91: rlm@91: rlm@91: (extend-type ImagePlus rlm@91: Frame rlm@91: (frame [image+] rlm@91: (frame (.getBufferedImage image+)))) rlm@91: rlm@94: rlm@92: (def white -1) rlm@94: rlm@91: (defn filter-pixels rlm@91: "List the coordinates of all pixels matching pred." rlm@92: {:author "Dylan Holmes"} rlm@91: [pred #^ImageProcessor ip] rlm@91: (let rlm@91: [width (.getWidth ip) rlm@91: height (.getHeight ip)] rlm@91: ((fn accumulate [x y matches] rlm@91: (cond rlm@91: (>= y height) matches rlm@91: (>= x width) (recur 0 (inc y) matches) rlm@91: (pred (.getPixel ip x y)) rlm@91: (recur (inc x) y (conj matches (Vector2f. x y))) rlm@91: :else (recur (inc x) y matches))) rlm@91: 0 0 []))) rlm@91: rlm@91: (defn white-coordinates rlm@91: "List the coordinates of all the white pixels in an image." rlm@91: [#^ImageProcessor ip] rlm@92: (filter-pixels #(= % white) ip)) rlm@91: rlm@91: (defn same-side? [p1 p2 ref p] rlm@91: (<= rlm@91: 0 rlm@91: (.dot rlm@91: (.cross (.subtract p2 p1) (.subtract p p1)) rlm@91: (.cross (.subtract p2 p1) (.subtract ref p1))))) rlm@91: rlm@94: rlm@94: (defn triangle->matrix4f rlm@94: "Converts the triangle into a 4x4 matrix of vertices: The first rlm@94: three columns contain the vertices of the triangle; the last rlm@94: contains the unit normal of the triangle. The bottom row is filled rlm@94: with 1s." rlm@94: [#^Triangle t] rlm@94: (let [mat (Matrix4f.) rlm@94: [vert-1 vert-2 vert-3] rlm@94: ((comp vec map) #(.get t %) (range 3)) rlm@94: unit-normal (do (.calculateNormal t)(.getNormal t)) rlm@94: vertices [vert-1 vert-2 vert-3 unit-normal]] rlm@94: rlm@94: (dorun rlm@94: (for [row (range 4) col (range 3)] rlm@94: (do rlm@94: (.set mat col row (.get (vertices row)col)) rlm@94: (.set mat 3 row 1)))) rlm@94: mat)) rlm@94: rlm@94: (defn triangle-transformation rlm@94: "Returns the affine transformation that converts each vertex in the rlm@94: first triangle into the corresponding vertex in the second rlm@94: triangle." rlm@94: [#^Triangle tri-1 #^Triangle tri-2] rlm@94: (.mult rlm@94: (triangle->matrix4f tri-2) rlm@94: (.invert (triangle->matrix4f tri-1)))) rlm@94: rlm@94: (def death (Triangle. rlm@94: (Vector3f. 1 1 1) rlm@94: (Vector3f. 1 2 3) rlm@94: (Vector3f. 5 6 7))) rlm@94: rlm@94: (def death-2 (Triangle. rlm@94: (Vector3f. 2 2 2) rlm@94: (Vector3f. 1 1 1) rlm@94: (Vector3f. 0 1 0))) rlm@94: rlm@94: (defn vector2f->vector3f [v] rlm@94: (Vector3f. (.getX v) (.getY v) 0)) rlm@94: rlm@94: rlm@94: (extend-type Triangle rlm@94: Textual rlm@94: (text [t] rlm@94: (println "Triangle: " \newline (.get1 t) \newline rlm@94: (.get2 t) \newline (.get3 t)))) rlm@94: rlm@94: rlm@94: (defn map-triangle [f #^Triangle tri] rlm@94: (Triangle. rlm@94: (f 0 (.get1 tri)) rlm@94: (f 1 (.get2 tri)) rlm@94: (f 2 (.get3 tri)))) rlm@94: rlm@94: (defn triangle-seq [#^Triangle tri] rlm@94: [(.get1 tri) (.get2 tri) (.get3 tri)]) rlm@94: rlm@94: (defn vector3f-seq [#^Vector3f v] rlm@94: [(.getX v) (.getY v) (.getZ v)]) rlm@94: rlm@91: (defn inside-triangle? rlm@94: "Is the point inside the triangle? Now what do we do? rlm@94: You might want to hold on there" rlm@95: {:author "Dylan Holmes"} rlm@94: [tri p] rlm@94: (let [[vert-1 vert-2 vert-3] (triangle-seq tri)] rlm@94: (and rlm@94: (same-side? vert-1 vert-2 vert-3 p) rlm@94: (same-side? vert-2 vert-3 vert-1 p) rlm@94: (same-side? vert-3 vert-1 vert-2 p)))) rlm@91: rlm@94: (defn uv-triangle rlm@94: "Convert the mesh triangle into the cooresponding triangle in rlm@94: UV-space. Z-component of these triangles is always zero." rlm@94: [#^Mesh mesh #^Triangle tri] rlm@94: (apply #(Triangle. %1 %2 %3) rlm@94: (map vector2f->vector3f rlm@94: (tri-uv-coord mesh tri)))) rlm@91: rlm@94: (defn pixel-triangle rlm@94: "Convert the mesh triange into the corresponding triangle in rlm@94: UV-pixel-space. Z compenent will be zero." rlm@94: [#^Mesh mesh #^Triangle tri width height] rlm@94: (map-triangle (fn [_ v] rlm@94: (Vector3f. (* width (.getX v)) rlm@94: (* height (.getY v)) rlm@94: 0)) rlm@94: (uv-triangle mesh tri))) rlm@93: rlm@96: (def rasterize pixel-triangle) rlm@96: rlm@96: rlm@94: (defn triangle-bounds rlm@94: "Dimensions of the bounding square of the triangle in the form rlm@94: [x y width height]. rlm@94: Assumes that the triangle lies in the XY plane." rlm@94: [#^Triangle tri] rlm@94: (let [verts (map vector3f-seq (triangle-seq tri)) rlm@94: x (apply min (map first verts)) rlm@94: y (apply min (map second verts))] rlm@93: rlm@94: [x y rlm@94: (- (apply max (map first verts)) x) rlm@94: (- (apply max (map second verts)) y) rlm@94: ])) rlm@93: rlm@93: rlm@97: (defn locate-feelers rlm@94: "Search the geometry's tactile UV image for touch sensors, returning rlm@94: their positions in geometry-relative coordinates." rlm@94: [#^Geometry geo] rlm@97: (if-let [image (touch-receptor-image geo)] rlm@97: (let [mesh (.getMesh geo) rlm@97: tris (triangles geo) rlm@97: rlm@97: rlm@97: width (.getWidth image) rlm@97: height (.getHeight image) rlm@97: rlm@97: ;; for each triangle rlm@97: sensor-coords rlm@97: (fn [tri] rlm@97: ;; translate triangle to uv-pixel-space rlm@97: (let [uv-tri rlm@97: (pixel-triangle mesh tri width height) rlm@97: bounds (vec (triangle-bounds uv-tri))] rlm@97: rlm@97: ;; get that part of the picture rlm@97: rlm@97: (apply #(.setRoi image %1 %2 %3 %4) bounds) rlm@97: (let [cutout (.crop (.getProcessor image)) rlm@97: ;; extract white pixels inside triangle rlm@97: cutout-tri rlm@97: (map-triangle rlm@97: (fn [_ v] rlm@97: (.subtract rlm@97: v rlm@97: (Vector3f. (bounds 0) (bounds 1) (float 0)))) rlm@97: uv-tri) rlm@97: whites (filter (partial inside-triangle? cutout-tri) rlm@97: (map vector2f->vector3f rlm@97: (white-coordinates cutout))) rlm@97: ;; translate pixel coordinates to world-space rlm@97: transform (triangle-transformation cutout-tri tri)] rlm@97: (map #(.mult transform %) whites))))] rlm@97: (vec (map sensor-coords tris))) rlm@97: (repeat (count (triangles geo)) []))) rlm@97: rlm@97: (defn enable-touch [#^Geometry geo] rlm@97: (let [feeler-coords (locate-feelers geo) rlm@96: tris (triangles geo) rlm@97: limit 0.1] rlm@97: (fn [node] rlm@97: (let [sensor-origins rlm@97: (map rlm@97: #(map (partial local-to-world geo) %) rlm@97: feeler-coords) rlm@97: triangle-normals rlm@97: (map (partial get-ray-direction geo) rlm@97: tris) rlm@97: rays rlm@97: (flatten rlm@97: (map (fn [origins norm] rlm@97: (map #(doto (Ray. % norm) rlm@97: (.setLimit limit)) origins)) rlm@97: sensor-origins triangle-normals))] rlm@97: (for [ray rays] rlm@97: (do rlm@97: (let [results (CollisionResults.)] rlm@97: (.collideWith node ray results) rlm@97: (let [touch-objects rlm@97: (set rlm@97: (filter #(not (= geo %)) rlm@97: (map #(.getGeometry %) results)))] rlm@99: (if (> (count touch-objects) 0) rlm@99: 1 0))))))))) rlm@94: rlm@97: (defn touch [#^Node pieces] rlm@97: (let [touch-components rlm@97: (map enable-touch rlm@97: (filter #(isa? (class %) Geometry) rlm@97: (node-seq pieces)))] rlm@97: (fn [node] rlm@97: (reduce into [] (map #(% node) touch-components))))) rlm@96: rlm@91: (defn all-names [] rlm@91: (concat rlm@91: (re-split #"\n" (slurp (file-str rlm@91: "/home/r/proj/names/dist.female.first"))) rlm@91: (re-split #"\n" (slurp (file-str rlm@91: "/home/r/proj/names/dist.male.first"))) rlm@91: (re-split #"\n" (slurp (file-str rlm@91: "/home/r/proj/names/dist.all.last"))))) rlm@90: rlm@90: rlm@90: rlm@90: rlm@90: rlm@90: rlm@90: rlm@90: rlm@90: rlm@90: (defrecord LulzLoader []) rlm@90: (defprotocol Lulzable (load-lulz [this])) rlm@90: (extend-type LulzLoader rlm@90: Lulzable rlm@90: (load-lulz [this] (println "the lulz have arrived!"))) rlm@90: rlm@78: rlm@78: (defn world-setup [joint] rlm@90: (let [joint-position (Vector3f. 0 0 0) rlm@83: joint-rotation rlm@83: (.toRotationMatrix rlm@83: (.mult rlm@83: (doto (Quaternion.) rlm@83: (.fromAngleAxis rlm@83: (* 1 (/ Math/PI 4)) rlm@85: (Vector3f. -1 0 0))) rlm@85: (doto (Quaternion.) rlm@85: (.fromAngleAxis rlm@85: (* 1 (/ Math/PI 2)) rlm@85: (Vector3f. 0 0 1))))) rlm@84: top-position (.mult joint-rotation (Vector3f. 8 0 0)) rlm@83: rlm@83: origin (doto rlm@83: (sphere 0.1 :physical? false :color ColorRGBA/Cyan rlm@84: :position top-position)) rlm@83: top (doto rlm@78: (sphere 0.1 :physical? false :color ColorRGBA/Yellow rlm@84: :position top-position) rlm@83: rlm@78: (.addControl rlm@78: (RigidBodyControl. rlm@83: (CapsuleCollisionShape. 0.5 1.5 1) (float 20)))) rlm@78: bottom (doto rlm@78: (sphere 0.1 :physical? false :color ColorRGBA/DarkGray rlm@83: :position (Vector3f. 0 0 0)) rlm@83: (.addControl rlm@78: (RigidBodyControl. rlm@78: (CapsuleCollisionShape. 0.5 1.5 1) (float 0)))) rlm@83: table (box 10 2 10 :position (Vector3f. 0 -20 0) rlm@78: :color ColorRGBA/Gray :mass 0) rlm@78: a (.getControl top RigidBodyControl) rlm@78: b (.getControl bottom RigidBodyControl)] rlm@83: rlm@78: (cond rlm@83: (= joint :cone) rlm@83: rlm@83: (doto (ConeJoint. rlm@83: a b rlm@87: (world-to-local top joint-position) rlm@87: (world-to-local bottom joint-position) rlm@83: joint-rotation rlm@83: joint-rotation rlm@83: ) rlm@78: rlm@83: rlm@83: (.setLimit (* (/ 10) Math/PI) rlm@83: (* (/ 4) Math/PI) rlm@83: 0))) rlm@83: [origin top bottom table])) rlm@78: rlm@78: (defn test-joint [joint] rlm@83: (let [[origin top bottom floor] (world-setup joint) rlm@78: control (.getControl top RigidBodyControl) rlm@78: move-up? (atom false) rlm@78: move-down? (atom false) rlm@78: move-left? (atom false) rlm@78: move-right? (atom false) rlm@78: roll-left? (atom false) rlm@78: roll-right? (atom false) rlm@78: timer (atom 0)] rlm@78: rlm@78: (world rlm@83: (nodify [top bottom floor origin]) rlm@78: (merge standard-debug-controls rlm@78: {"key-r" (fn [_ pressed?] (reset! move-up? pressed?)) rlm@78: "key-t" (fn [_ pressed?] (reset! move-down? pressed?)) rlm@78: "key-f" (fn [_ pressed?] (reset! move-left? pressed?)) rlm@78: "key-g" (fn [_ pressed?] (reset! move-right? pressed?)) rlm@78: "key-v" (fn [_ pressed?] (reset! roll-left? pressed?)) rlm@78: "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))}) rlm@78: rlm@78: (fn [world] rlm@78: (light-up-everything world) rlm@78: (enable-debug world) rlm@78: (set-gravity world (Vector3f. 0 0 0)) rlm@78: ) rlm@78: rlm@78: (fn [world _] rlm@78: (if (zero? (rem (swap! timer inc) 100)) rlm@78: (do rlm@78: ;; (println-repl @timer) rlm@78: (.attachChild (.getRootNode world) rlm@78: (sphere 0.05 :color ColorRGBA/Yellow rlm@78: :position (.getWorldTranslation top) rlm@83: :physical? false)) rlm@83: (.attachChild (.getRootNode world) rlm@83: (sphere 0.05 :color ColorRGBA/LightGray rlm@83: :position (.getWorldTranslation bottom) rlm@83: :physical? false)))) rlm@83: rlm@78: (if @move-up? rlm@78: (.applyTorque control rlm@78: (.mult (.getPhysicsRotation control) rlm@78: (Vector3f. 0 0 10)))) rlm@78: (if @move-down? rlm@78: (.applyTorque control rlm@78: (.mult (.getPhysicsRotation control) rlm@78: (Vector3f. 0 0 -10)))) rlm@78: (if @move-left? rlm@78: (.applyTorque control rlm@78: (.mult (.getPhysicsRotation control) rlm@78: (Vector3f. 0 10 0)))) rlm@78: (if @move-right? rlm@78: (.applyTorque control rlm@78: (.mult (.getPhysicsRotation control) rlm@78: (Vector3f. 0 -10 0)))) rlm@78: (if @roll-left? rlm@78: (.applyTorque control rlm@78: (.mult (.getPhysicsRotation control) rlm@78: (Vector3f. -1 0 0)))) rlm@78: (if @roll-right? rlm@78: (.applyTorque control rlm@78: (.mult (.getPhysicsRotation control) rlm@78: (Vector3f. 1 0 0)))))))) rlm@83: rlm@83: rlm@97: (defn locate-feelers* rlm@97: "Search the geometry's tactile UV image for touch sensors, returning rlm@97: their positions in geometry-relative coordinates." rlm@97: [#^Geometry geo] rlm@97: (let [uv-image (touch-receptor-image geo) rlm@97: width (.getWidth uv-image) rlm@97: height (.getHeight uv-image) rlm@97: rlm@97: mesh (.getMesh geo) rlm@97: mesh-tris (triangles geo) rlm@97: rlm@97: ;; for each triangle rlm@97: sensor-coords rlm@97: (fn [tri] rlm@97: ;; translate triangle to uv-pixel-space rlm@97: (let [uv-tri rlm@97: (rasterize mesh tri width height) rlm@97: bounds (vec (triangle-bounds uv-tri))] rlm@97: rlm@97: ;; get that part of the picture rlm@97: rlm@97: (apply (partial (memfn setRoi) uv-image) bounds) rlm@97: (let [cutout (.crop (.getProcessor uv-image)) rlm@97: ;; extract white pixels inside triangle rlm@97: cutout-tri rlm@97: (map-triangle rlm@97: (fn [_ v] rlm@97: (.subtract rlm@97: v rlm@97: (Vector3f. (bounds 0) (bounds 1) (float 0)))) rlm@97: uv-tri) rlm@97: whites (filter (partial inside-triangle? cutout-tri) rlm@97: (map vector2f->vector3f rlm@97: (white-coordinates cutout))) rlm@97: ;; translate pixel coordinates to world-space rlm@97: transform (triangle-transformation cutout-tri tri)] rlm@97: (map #(.mult transform %) whites))))] rlm@97: rlm@97: rlm@97: rlm@97: (for [mesh-tri mesh-tris] rlm@97: rlm@97: (let [uv-tri (rasterize mesh mesh-tri width height) rlm@97: bounding-box (vec (triangle-bounds uv-tri))] rlm@97: (apply (partial (memfn setRoi) uv-image) bounding-box) rlm@97: )) rlm@97: (vec (map sensor-coords mesh-tris)))) rlm@97: rlm@97: rlm@97: (defn tactile-coords [#^Geometry obj] rlm@97: (let [mesh (.getMesh obj) rlm@97: num-triangles (.getTriangleCount mesh) rlm@97: num-verticies (.getVertexCount mesh) rlm@97: uv-coord (partial uv-coord mesh) rlm@97: triangle-indices (partial triangle-indices mesh) rlm@97: receptors (touch-receptor-image obj) rlm@97: tris (triangles obj) rlm@97: ] rlm@97: (map rlm@97: (fn [[tri-1 tri-2 tri-3]] rlm@97: (let [width (.getWidth receptors) rlm@97: height (.getHeight receptors) rlm@97: uv-1 (uv-coord tri-1) rlm@97: uv-2 (uv-coord tri-2) rlm@97: uv-3 (uv-coord tri-3) rlm@97: x-coords (map #(.getX %) [uv-1 uv-2 uv-3]) rlm@97: y-coords (map #(.getY %) [uv-1 uv-2 uv-3]) rlm@97: max-x (Math/ceil (* width (apply max x-coords))) rlm@97: min-x (Math/floor (* width (apply min x-coords))) rlm@97: max-y (Math/ceil (* height (apply max y-coords))) rlm@97: min-y (Math/floor (* height (apply min y-coords))) rlm@97: rlm@97: image-1 (Vector2f. (* width (.getX uv-1)) rlm@97: (* height (.getY uv-1))) rlm@97: image-2 (Vector2f. (* width (.getX uv-2)) rlm@97: (* height (.getY uv-2))) rlm@97: image-3 (Vector2f. (* width (.getX uv-3)) rlm@97: (* height (.getY uv-3))) rlm@97: left-corner rlm@97: (Vector2f. min-x min-y) rlm@97: ] rlm@97: rlm@97: (.setRoi receptors min-x min-y (- max-x min-x) (- max-y min-y)) rlm@97: (let [processor (.crop (.getProcessor receptors))] rlm@97: (map rlm@97: #(.add left-corner %) rlm@97: rlm@97: (filter rlm@97: (partial rlm@97: inside-triangle? rlm@97: (.subtract image-1 left-corner) rlm@97: (.subtract image-2 left-corner) rlm@97: (.subtract image-3 left-corner)) rlm@97: (white-coordinates processor)))) rlm@97: )) (map triangle-indices (range num-triangles))))) rlm@83: rlm@87: #+end_src rlm@83: rlm@87: #+results: body-1 rlm@87: : #'cortex.silly/test-joint rlm@78: rlm@78: rlm@78: * COMMENT purgatory rlm@78: #+begin_src clojure rlm@77: (defn bullet-trans [] rlm@77: (let [obj-a (sphere 0.5 :color ColorRGBA/Red rlm@77: :position (Vector3f. -10 5 0)) rlm@77: obj-b (sphere 0.5 :color ColorRGBA/Blue rlm@77: :position (Vector3f. -10 -5 0) rlm@77: :mass 0) rlm@77: control-a (.getControl obj-a RigidBodyControl) rlm@77: control-b (.getControl obj-b RigidBodyControl) rlm@77: swivel rlm@77: (.toRotationMatrix rlm@77: (doto (Quaternion.) rlm@77: (.fromAngleAxis (/ Math/PI 2) rlm@77: Vector3f/UNIT_X)))] rlm@77: (doto rlm@77: (ConeJoint. rlm@77: control-a control-b rlm@77: (Vector3f. 0 5 0) rlm@77: (Vector3f. 0 -5 0) rlm@77: swivel swivel) rlm@77: (.setLimit (* 0.6 (/ Math/PI 4)) rlm@77: (/ Math/PI 4) rlm@77: (* Math/PI 0.8))) rlm@77: (world (nodify rlm@77: [obj-a obj-b]) rlm@77: standard-debug-controls rlm@77: enable-debug rlm@77: no-op))) rlm@74: rlm@74: rlm@77: (defn bullet-trans* [] rlm@77: (let [obj-a (box 1.5 0.5 0.5 :color ColorRGBA/Red rlm@77: :position (Vector3f. 5 0 0) rlm@77: :mass 90) rlm@77: obj-b (sphere 0.5 :color ColorRGBA/Blue rlm@77: :position (Vector3f. -5 0 0) rlm@77: :mass 0) rlm@77: control-a (.getControl obj-a RigidBodyControl) rlm@77: control-b (.getControl obj-b RigidBodyControl) rlm@77: move-up? (atom nil) rlm@77: move-down? (atom nil) rlm@77: move-left? (atom nil) rlm@77: move-right? (atom nil) rlm@77: roll-left? (atom nil) rlm@77: roll-right? (atom nil) rlm@77: force 100 rlm@77: swivel rlm@77: (.toRotationMatrix rlm@77: (doto (Quaternion.) rlm@77: (.fromAngleAxis (/ Math/PI 2) rlm@77: Vector3f/UNIT_X))) rlm@77: x-move rlm@77: (doto (Matrix3f.) rlm@77: (.fromStartEndVectors Vector3f/UNIT_X rlm@77: (.normalize (Vector3f. 1 1 0)))) rlm@77: rlm@77: timer (atom 0)] rlm@77: (doto rlm@77: (ConeJoint. rlm@77: control-a control-b rlm@77: (Vector3f. -8 0 0) rlm@77: (Vector3f. 2 0 0) rlm@77: ;;swivel swivel rlm@77: ;;Matrix3f/IDENTITY Matrix3f/IDENTITY rlm@77: x-move Matrix3f/IDENTITY rlm@77: ) rlm@77: (.setCollisionBetweenLinkedBodys false) rlm@77: (.setLimit (* 1 (/ Math/PI 4)) ;; twist rlm@77: (* 1 (/ Math/PI 4)) ;; swing span in X-Y plane rlm@77: (* 0 (/ Math/PI 4)))) ;; swing span in Y-Z plane rlm@77: (world (nodify rlm@77: [obj-a obj-b]) rlm@77: (merge standard-debug-controls rlm@77: {"key-r" (fn [_ pressed?] (reset! move-up? pressed?)) rlm@77: "key-t" (fn [_ pressed?] (reset! move-down? pressed?)) rlm@77: "key-f" (fn [_ pressed?] (reset! move-left? pressed?)) rlm@77: "key-g" (fn [_ pressed?] (reset! move-right? pressed?)) rlm@77: "key-v" (fn [_ pressed?] (reset! roll-left? pressed?)) rlm@77: "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))}) rlm@77: rlm@77: (fn [world] rlm@77: (enable-debug world) rlm@77: (set-gravity world Vector3f/ZERO) rlm@77: ) rlm@77: rlm@77: (fn [world _] rlm@77: rlm@77: (if @move-up? rlm@77: (.applyForce control-a rlm@77: (Vector3f. force 0 0) rlm@77: (Vector3f. 0 0 0))) rlm@77: (if @move-down? rlm@77: (.applyForce control-a rlm@77: (Vector3f. (- force) 0 0) rlm@77: (Vector3f. 0 0 0))) rlm@77: (if @move-left? rlm@77: (.applyForce control-a rlm@77: (Vector3f. 0 force 0) rlm@77: (Vector3f. 0 0 0))) rlm@77: (if @move-right? rlm@77: (.applyForce control-a rlm@77: (Vector3f. 0 (- force) 0) rlm@77: (Vector3f. 0 0 0))) rlm@77: rlm@77: (if @roll-left? rlm@77: (.applyForce control-a rlm@77: (Vector3f. 0 0 force) rlm@77: (Vector3f. 0 0 0))) rlm@77: (if @roll-right? rlm@77: (.applyForce control-a rlm@77: (Vector3f. 0 0 (- force)) rlm@77: (Vector3f. 0 0 0))) rlm@77: rlm@77: (if (zero? (rem (swap! timer inc) 100)) rlm@77: (.attachChild rlm@77: (.getRootNode world) rlm@77: (sphere 0.05 :color ColorRGBA/Yellow rlm@77: :physical? false :position rlm@77: (.getWorldTranslation obj-a))))) rlm@77: ) rlm@77: )) rlm@77: rlm@94: (defn transform-trianglesdsd rlm@94: "Transform that converts each vertex in the first triangle rlm@94: into the corresponding vertex in the second triangle." rlm@94: [#^Triangle tri-1 #^Triangle tri-2] rlm@94: (let [in [(.get1 tri-1) rlm@94: (.get2 tri-1) rlm@94: (.get3 tri-1)] rlm@94: out [(.get1 tri-2) rlm@94: (.get2 tri-2) rlm@94: (.get3 tri-2)]] rlm@94: (let [translate (doto (Matrix4f.) (.setTranslation (.negate (in 0)))) rlm@94: in* [(.mult translate (in 0)) rlm@94: (.mult translate (in 1)) rlm@94: (.mult translate (in 2))] rlm@94: final-translation rlm@94: (doto (Matrix4f.) rlm@94: (.setTranslation (out 1))) rlm@94: rlm@94: rotate-1 rlm@94: (doto (Matrix3f.) rlm@94: (.fromStartEndVectors rlm@94: (.normalize rlm@94: (.subtract rlm@94: (in* 1) (in* 0))) rlm@94: (.normalize rlm@94: (.subtract rlm@94: (out 1) (out 0))))) rlm@94: in** [(.mult rotate-1 (in* 0)) rlm@94: (.mult rotate-1 (in* 1)) rlm@94: (.mult rotate-1 (in* 2))] rlm@94: scale-factor-1 rlm@94: (.mult rlm@94: (.normalize rlm@94: (.subtract rlm@94: (out 1) rlm@94: (out 0))) rlm@94: (/ (.length rlm@94: (.subtract (out 1) rlm@94: (out 0))) rlm@94: (.length rlm@94: (.subtract (in** 1) rlm@94: (in** 0))))) rlm@94: scale-1 (doto (Matrix4f.) (.setScale scale-factor-1)) rlm@94: in*** [(.mult scale-1 (in** 0)) rlm@94: (.mult scale-1 (in** 1)) rlm@94: (.mult scale-1 (in** 2))] rlm@94: rlm@94: rlm@94: rlm@94: rlm@94: rlm@94: ] rlm@94: rlm@94: (dorun (map println in)) rlm@94: (println) rlm@94: (dorun (map println in*)) rlm@94: (println) rlm@94: (dorun (map println in**)) rlm@94: (println) rlm@94: (dorun (map println in***)) rlm@94: (println) rlm@94: rlm@99: )))) rlm@94: rlm@94: rlm@99: rlm@99: rlm@99: #+end_src rlm@99: rlm@99: rlm@99: * COMMENT generate source rlm@99: #+begin_src clojure :tangle ../src/cortex/silly.clj rlm@99: <> rlm@99: #+end_src rlm@99: rlm@99: rlm@94: rlm@94: