Mercurial > cortex
view org/test-creature.org @ 129:bab47091534e
add some ideas
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Mon, 30 Jan 2012 00:25:11 -0700 |
parents | 4b38355ad6e3 |
children | b26017d1fe9a |
line wrap: on
line source
1 #+title: First attempt at a creature!2 #+author: Robert McIntyre3 #+email: rlm@mit.edu4 #+description:5 #+keywords: simulation, jMonkeyEngine3, clojure6 #+SETUPFILE: ../../aurellem/org/setup.org7 #+INCLUDE: ../../aurellem/org/level-0.org11 * Brainstorming different sensors and effectors.13 Every sense that we have should have an effector that changes what14 that sense (or others who have that sense) experiences.16 ** Classic Senses17 | Sense | Effector |18 |------------------------------+---------------------------------|19 | Vision | Variable Coloration |20 | Hearing | Speech |21 | Proprioception | Movement |22 | Smell/Taste (Chemoreception) | Pheremones |23 | Touch | Movement / Controllable Texture |24 | Acceleration | Movement |25 | Balance (sense gravity) | Movement |26 | | |28 - New Senses/Effectors29 - Levitation30 - Telekenesis32 - Symbol Sense33 Where objects in the world can be queried for description /34 symbols.36 - Symbol Marking37 The ability to mark objects in the world with your own descriptions38 and symbols.40 - Vision41 Distinguish the polarization of light42 Color43 Movement45 * project ideas46 - HACKER for writing muscle-control programs : Presented with47 low-level muscle control/ sense API, generate higher level programs48 for accomplishing various stated goals. Example goals might be49 "extend all your fingers" or "move your hand into the area with50 blue light" or "decrease the angle of this joint". It would be51 like Sussman's HACKER, except it would operate with much more data52 in a more realistic world. Start off with "calestanthics" to53 develop subrouitines over the motor control API. This would be the54 "spinal chord" of a more intelligent creature. The low level55 programming code might be a turning machine that could develop56 programs to iterate over a "tape" where each entry in the tape57 could control recruitment of the fibers in a muscle.58 - Make a virtual computer in the virtual world which with which the59 creature interacts using its fingers to press keys on a virtual60 keyboard. The creature can access the internet, watch videos, take61 over the world, anything it wants.62 - Make virtual insturments like pianos, drumbs, etc that it learns to63 play.64 - make a joint that figures out what type of joint it is (range of65 motion)71 * goals73 ** have to get done before winston74 - [ ] write an explination for why greyscale bitmaps for senses is75 appropiate -- 1/2 day76 - [ ] muscle control -- day77 - [ ] proprioception sensor map in the style of the other senses -- day78 - [ ] refactor integration code to distribute to each of the senses79 -- day80 - [ ] create video showing all the senses for Winston -- 2 days81 - [ ] write summary of project for Winston \82 - [ ] project proposals for Winston \83 - [ ] additional senses to be implemented for Winston | -- 2 days84 - [ ] send Winston package /86 ** would be cool to get done before winston87 - [X] enable greyscale bitmaps for touch -- 2 hours88 - [X] use sawfish to auto-tile sense windows -- 6 hours89 - [X] sawfish keybinding to automatically delete all sense windows90 - [ ] directly change the UV-pixels to show sensor activation -- 291 days92 - [ ] proof of concept C sense manipulation -- 2 days93 - [ ] proof of concept GPU sense manipulation -- week94 - [ ] fourier view of sound -- 2 or 3 days95 - [ ] dancing music generator -- 1 day, depends on fourier97 ** don't have to get done before winston98 - [ ] write tests for integration -- 3 days99 - [ ] usertime/gametime clock HUD display -- day100 - [ ] find papers for each of the senses justifying my own101 representation -- week102 - [ ] show sensor maps in HUD display? -- 4 days103 - [ ] show sensor maps in AWT display? -- 2 days106 * Intro107 So far, I've made the following senses --108 - Vision109 - Hearing110 - Touch111 - Proprioception113 And one effector:114 - Movement116 However, the code so far has only enabled these senses, but has not117 actually implemented them. For example, there is still a lot of work118 to be done for vision. I need to be able to create an /eyeball/ in119 simulation that can be moved around and see the world from different120 angles. I also need to determine weather to use log-polar or cartesian121 for the visual input, and I need to determine how/wether to122 disceritise the visual input.124 I also want to be able to visualize both the sensors and the125 effectors in pretty pictures. This semi-retarted creature will be my126 first attempt at bringing everything together.128 * The creature's body130 Still going to do an eve-like body in blender, but due to problems131 importing the joints, etc into jMonkeyEngine3, I'm going to do all132 the connecting here in clojure code, using the names of the individual133 components and trial and error. Later, I'll maybe make some sort of134 creature-building modifications to blender that support whatever135 discreitized senses I'm going to make.137 #+name: body-1138 #+begin_src clojure139 (ns cortex.silly140 "let's play!"141 {:author "Robert McIntyre"})143 ;; TODO remove this!144 (require 'cortex.import)145 (cortex.import/mega-import-jme3)146 (use '(cortex world util body hearing touch vision))148 (rlm.rlm-commands/help)149 (import java.awt.image.BufferedImage)150 (import javax.swing.JPanel)151 (import javax.swing.SwingUtilities)152 (import java.awt.Dimension)153 (import javax.swing.JFrame)154 (import java.awt.Dimension)155 (import com.aurellem.capture.RatchetTimer)156 (declare joint-create)157 (use 'clojure.contrib.def)159 (defn points->image160 "Take a sparse collection of points and visuliaze it as a161 BufferedImage."163 ;; TODO maybe parallelize this since it's easy165 [points]166 (if (empty? points)167 (BufferedImage. 1 1 BufferedImage/TYPE_BYTE_BINARY)168 (let [xs (vec (map first points))169 ys (vec (map second points))170 x0 (apply min xs)171 y0 (apply min ys)172 width (- (apply max xs) x0)173 height (- (apply max ys) y0)174 image (BufferedImage. (inc width) (inc height)175 BufferedImage/TYPE_INT_RGB)]176 (dorun177 (for [x (range (.getWidth image))178 y (range (.getHeight image))]179 (.setRGB image x y 0xFF0000)))180 (dorun181 (for [index (range (count points))]182 (.setRGB image (- (xs index) x0) (- (ys index) y0) -1)))184 image)))186 (defn average [coll]187 (/ (reduce + coll) (count coll)))189 (defn collapse-1d190 "One dimensional analogue of collapse"191 [center line]192 (let [length (count line)193 num-above (count (filter (partial < center) line))194 num-below (- length num-above)]195 (range (- center num-below)196 (+ center num-above))))198 (defn collapse199 "Take a set of pairs of integers and collapse them into a200 contigous bitmap."201 [points]202 (if (empty? points) []203 (let204 [num-points (count points)205 center (vector206 (int (average (map first points)))207 (int (average (map first points))))208 flattened209 (reduce210 concat211 (map212 (fn [column]213 (map vector214 (map first column)215 (collapse-1d (second center)216 (map second column))))217 (partition-by first (sort-by first points))))218 squeezed219 (reduce220 concat221 (map222 (fn [row]223 (map vector224 (collapse-1d (first center)225 (map first row))226 (map second row)))227 (partition-by second (sort-by second flattened))))228 relocate229 (let [min-x (apply min (map first squeezed))230 min-y (apply min (map second squeezed))]231 (map (fn [[x y]]232 [(- x min-x)233 (- y min-y)])234 squeezed))]235 relocate)))237 (defn load-bullet []238 (let [sim (world (Node.) {} no-op no-op)]239 (doto sim240 (.enqueue241 (fn []242 (.stop sim)))243 (.start))))245 (defn load-blender-model246 "Load a .blend file using an asset folder relative path."247 [^String model]248 (.loadModel249 (doto (asset-manager)250 (.registerLoader BlenderModelLoader (into-array String ["blend"])))251 model))253 (defn meta-data [blender-node key]254 (if-let [data (.getUserData blender-node "properties")]255 (.findValue data key)256 nil))258 (defn blender-to-jme259 "Convert from Blender coordinates to JME coordinates"260 [#^Vector3f in]261 (Vector3f. (.getX in)262 (.getZ in)263 (- (.getY in))))265 (defn jme-to-blender266 "Convert from JME coordinates to Blender coordinates"267 [#^Vector3f in]268 (Vector3f. (.getX in)269 (- (.getZ in))270 (.getY in)))272 (defn joint-targets273 "Return the two closest two objects to the joint object, ordered274 from bottom to top according to the joint's rotation."275 [#^Node parts #^Node joint]276 (loop [radius (float 0.01)]277 (let [results (CollisionResults.)]278 (.collideWith279 parts280 (BoundingBox. (.getWorldTranslation joint)281 radius radius radius)282 results)283 (let [targets284 (distinct285 (map #(.getGeometry %) results))]286 (if (>= (count targets) 2)287 (sort-by288 #(let [v289 (jme-to-blender290 (.mult291 (.inverse (.getWorldRotation joint))292 (.subtract (.getWorldTranslation %)293 (.getWorldTranslation joint))))]294 (println-repl (.getName %) ":" v)295 (.dot (Vector3f. 1 1 1)296 v))297 (take 2 targets))298 (recur (float (* radius 2))))))))300 (defn world-to-local301 "Convert the world coordinates into coordinates relative to the302 object (i.e. local coordinates), taking into account the rotation303 of object."304 [#^Spatial object world-coordinate]305 (let [out (Vector3f.)]306 (.worldToLocal object world-coordinate out) out))308 (defn local-to-world309 "Convert the local coordinates into coordinates into world relative310 coordinates"311 [#^Spatial object local-coordinate]312 (let [world-coordinate (Vector3f.)]313 (.localToWorld object local-coordinate world-coordinate)314 world-coordinate))316 (defmulti joint-dispatch317 "Translate blender pseudo-joints into real JME joints."318 (fn [constraints & _]319 (:type constraints)))321 (defmethod joint-dispatch :point322 [constraints control-a control-b pivot-a pivot-b rotation]323 (println-repl "creating POINT2POINT joint")324 (Point2PointJoint.325 control-a326 control-b327 pivot-a328 pivot-b))330 (defmethod joint-dispatch :hinge331 [constraints control-a control-b pivot-a pivot-b rotation]332 (println-repl "creating HINGE joint")333 (let [axis334 (if-let335 [axis (:axis constraints)]336 axis337 Vector3f/UNIT_X)338 [limit-1 limit-2] (:limit constraints)339 hinge-axis340 (.mult341 rotation342 (blender-to-jme axis))]343 (doto344 (HingeJoint.345 control-a346 control-b347 pivot-a348 pivot-b349 hinge-axis350 hinge-axis)351 (.setLimit limit-1 limit-2))))353 (defmethod joint-dispatch :cone354 [constraints control-a control-b pivot-a pivot-b rotation]355 (let [limit-xz (:limit-xz constraints)356 limit-xy (:limit-xy constraints)357 twist (:twist constraints)]359 (println-repl "creating CONE joint")360 (println-repl rotation)361 (println-repl362 "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0)))363 (println-repl364 "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0)))365 (println-repl366 "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1)))367 (doto368 (ConeJoint.369 control-a370 control-b371 pivot-a372 pivot-b373 rotation374 rotation)375 (.setLimit (float limit-xz)376 (float limit-xy)377 (float twist)))))379 (defn connect380 "here are some examples:381 {:type :point}382 {:type :hinge :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}383 (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)385 {:type :cone :limit-xz 0]386 :limit-xy 0]387 :twist 0]} (use XZY rotation mode in blender!)"388 [#^Node obj-a #^Node obj-b #^Node joint]389 (let [control-a (.getControl obj-a RigidBodyControl)390 control-b (.getControl obj-b RigidBodyControl)391 joint-center (.getWorldTranslation joint)392 joint-rotation (.toRotationMatrix (.getWorldRotation joint))393 pivot-a (world-to-local obj-a joint-center)394 pivot-b (world-to-local obj-b joint-center)]396 (if-let [constraints397 (map-vals398 eval399 (read-string400 (meta-data joint "joint")))]401 ;; A side-effect of creating a joint registers402 ;; it with both physics objects which in turn403 ;; will register the joint with the physics system404 ;; when the simulation is started.405 (do406 (println-repl "creating joint between"407 (.getName obj-a) "and" (.getName obj-b))408 (joint-dispatch constraints409 control-a control-b410 pivot-a pivot-b411 joint-rotation))412 (println-repl "could not find joint meta-data!"))))414 (defn assemble-creature [#^Node pieces joints]415 (dorun416 (map417 (fn [geom]418 (let [physics-control419 (RigidBodyControl.420 (HullCollisionShape.421 (.getMesh geom))422 (if-let [mass (meta-data geom "mass")]423 (do424 (println-repl425 "setting" (.getName geom) "mass to" (float mass))426 (float mass))427 (float 1)))]429 (.addControl geom physics-control)))430 (filter #(isa? (class %) Geometry )431 (node-seq pieces))))432 (dorun433 (map434 (fn [joint]435 (let [[obj-a obj-b]436 (joint-targets pieces joint)]437 (connect obj-a obj-b joint)))438 joints))439 pieces)441 (declare blender-creature)443 (def hand "Models/creature1/one.blend")445 (def worm "Models/creature1/try-again.blend")447 (def touch "Models/creature1/touch.blend")449 (defn worm-model [] (load-blender-model worm))451 (defn x-ray [#^ColorRGBA color]452 (doto (Material. (asset-manager)453 "Common/MatDefs/Misc/Unshaded.j3md")454 (.setColor "Color" color)455 (-> (.getAdditionalRenderState)456 (.setDepthTest false))))458 (defn colorful []459 (.getChild (worm-model) "worm-21"))461 (import jme3tools.converters.ImageToAwt)463 (import ij.ImagePlus)465 ;; Every Mesh has many triangles, each with its own index.466 ;; Every vertex has its own index as well.468 (defn tactile-sensor-image469 "Return the touch-sensor distribution image in BufferedImage format,470 or nil if it does not exist."471 [#^Geometry obj]472 (if-let [image-path (meta-data obj "touch")]473 (ImageToAwt/convert474 (.getImage475 (.loadTexture476 (asset-manager)477 image-path))478 false false 0)))480 (import ij.process.ImageProcessor)481 (import java.awt.image.BufferedImage)483 (def white -1)485 (defn filter-pixels486 "List the coordinates of all pixels matching pred, within the bounds487 provided. Bounds -> [x0 y0 width height]"488 {:author "Dylan Holmes"}489 ([pred #^BufferedImage image]490 (filter-pixels pred image [0 0 (.getWidth image) (.getHeight image)]))491 ([pred #^BufferedImage image [x0 y0 width height]]492 ((fn accumulate [x y matches]493 (cond494 (>= y (+ height y0)) matches495 (>= x (+ width x0)) (recur 0 (inc y) matches)496 (pred (.getRGB image x y))497 (recur (inc x) y (conj matches [x y]))498 :else (recur (inc x) y matches)))499 x0 y0 [])))501 (defn white-coordinates502 "Coordinates of all the white pixels in a subset of the image."503 ([#^BufferedImage image bounds]504 (filter-pixels #(= % white) image bounds))505 ([#^BufferedImage image]506 (filter-pixels #(= % white) image)))508 (defn triangle509 "Get the triangle specified by triangle-index from the mesh within510 bounds."511 [#^Mesh mesh triangle-index]512 (let [scratch (Triangle.)]513 (.getTriangle mesh triangle-index scratch)514 scratch))516 (defn triangle-vertex-indices517 "Get the triangle vertex indices of a given triangle from a given518 mesh."519 [#^Mesh mesh triangle-index]520 (let [indices (int-array 3)]521 (.getTriangle mesh triangle-index indices)522 (vec indices)))524 (defn vertex-UV-coord525 "Get the uv-coordinates of the vertex named by vertex-index"526 [#^Mesh mesh vertex-index]527 (let [UV-buffer528 (.getData529 (.getBuffer530 mesh531 VertexBuffer$Type/TexCoord))]532 [(.get UV-buffer (* vertex-index 2))533 (.get UV-buffer (+ 1 (* vertex-index 2)))]))535 (defn triangle-UV-coord536 "Get the uv-cooridnates of the triangle's verticies."537 [#^Mesh mesh width height triangle-index]538 (map (fn [[u v]] (vector (* width u) (* height v)))539 (map (partial vertex-UV-coord mesh)540 (triangle-vertex-indices mesh triangle-index))))542 (defn same-side?543 "Given the points p1 and p2 and the reference point ref, is point p544 on the same side of the line that goes through p1 and p2 as ref is?"545 [p1 p2 ref p]546 (<=547 0548 (.dot549 (.cross (.subtract p2 p1) (.subtract p p1))550 (.cross (.subtract p2 p1) (.subtract ref p1)))))552 (defn triangle-seq [#^Triangle tri]553 [(.get1 tri) (.get2 tri) (.get3 tri)])555 (defn vector3f-seq [#^Vector3f v]556 [(.getX v) (.getY v) (.getZ v)])558 (defn inside-triangle?559 "Is the point inside the triangle?"560 {:author "Dylan Holmes"}561 [#^Triangle tri #^Vector3f p]562 (let [[vert-1 vert-2 vert-3] (triangle-seq tri)]563 (and564 (same-side? vert-1 vert-2 vert-3 p)565 (same-side? vert-2 vert-3 vert-1 p)566 (same-side? vert-3 vert-1 vert-2 p))))568 (defn triangle->matrix4f569 "Converts the triangle into a 4x4 matrix: The first three columns570 contain the vertices of the triangle; the last contains the unit571 normal of the triangle. The bottom row is filled with 1s."572 [#^Triangle t]573 (let [mat (Matrix4f.)574 [vert-1 vert-2 vert-3]575 ((comp vec map) #(.get t %) (range 3))576 unit-normal (do (.calculateNormal t)(.getNormal t))577 vertices [vert-1 vert-2 vert-3 unit-normal]]578 (dorun579 (for [row (range 4) col (range 3)]580 (do581 (.set mat col row (.get (vertices row)col))582 (.set mat 3 row 1))))583 mat))585 (defn triangle-transformation586 "Returns the affine transformation that converts each vertex in the587 first triangle into the corresponding vertex in the second588 triangle."589 [#^Triangle tri-1 #^Triangle tri-2]590 (.mult591 (triangle->matrix4f tri-2)592 (.invert (triangle->matrix4f tri-1))))594 (defn point->vector2f [[u v]]595 (Vector2f. u v))597 (defn vector2f->vector3f [v]598 (Vector3f. (.getX v) (.getY v) 0))600 (defn map-triangle [f #^Triangle tri]601 (Triangle.602 (f 0 (.get1 tri))603 (f 1 (.get2 tri))604 (f 2 (.get3 tri))))606 (defn points->triangle607 "Convert a list of points into a triangle."608 [points]609 (apply #(Triangle. %1 %2 %3)610 (map (fn [point]611 (let [point (vec point)]612 (Vector3f. (get point 0 0)613 (get point 1 0)614 (get point 2 0))))615 (take 3 points))))617 (defn convex-bounds618 ;;dylan619 "Returns the smallest square containing the given620 vertices, as a vector of integers [left top width height]."621 ;; "Dimensions of the smallest integer bounding square of the list of622 ;; 2D verticies in the form: [x y width height]."623 [uv-verts]624 (let [xs (map first uv-verts)625 ys (map second uv-verts)626 x0 (Math/floor (apply min xs))627 y0 (Math/floor (apply min ys))628 x1 (Math/ceil (apply max xs))629 y1 (Math/ceil (apply max ys))]630 [x0 y0 (- x1 x0) (- y1 y0)]))632 (defn sensors-in-triangle633 ;;dylan634 "Locate the touch sensors in the triangle, returning a map of their UV and geometry-relative coordinates."635 ;;"Find the locations of the touch sensors within a triangle in both636 ;; UV and gemoetry relative coordinates."637 [image mesh tri-index]638 (let [width (.getWidth image)639 height (.getHeight image)640 UV-vertex-coords (triangle-UV-coord mesh width height tri-index)641 bounds (convex-bounds UV-vertex-coords)643 cutout-triangle (points->triangle UV-vertex-coords)644 UV-sensor-coords645 (filter (comp (partial inside-triangle? cutout-triangle)646 (fn [[u v]] (Vector3f. u v 0)))647 (white-coordinates image bounds))648 UV->geometry (triangle-transformation649 cutout-triangle650 (triangle mesh tri-index))651 geometry-sensor-coords652 (map (fn [[u v]] (.mult UV->geometry (Vector3f. u v 0)))653 UV-sensor-coords)]654 {:UV UV-sensor-coords :geometry geometry-sensor-coords}))656 (defn-memo locate-feelers657 "Search the geometry's tactile UV image for touch sensors, returning658 their positions in geometry-relative coordinates."659 [#^Geometry geo]660 (let [mesh (.getMesh geo)661 num-triangles (.getTriangleCount mesh)]662 (if-let [image (tactile-sensor-image geo)]663 (map664 (partial sensors-in-triangle image mesh)665 (range num-triangles))666 (repeat (.getTriangleCount mesh) {:UV nil :geometry nil}))))668 (use 'clojure.contrib.def)670 (defn-memo touch-topology [#^Gemoetry geo]671 (vec (collapse (reduce concat (map :UV (locate-feelers geo))))))673 (defn-memo feeler-coordinates [#^Geometry geo]674 (vec (map :geometry (locate-feelers geo))))676 (defn enable-touch [#^Geometry geo]677 (let [feeler-coords (feeler-coordinates geo)678 tris (triangles geo)679 limit 0.1680 ;;results (CollisionResults.)681 ]682 (if (empty? (touch-topology geo))683 nil684 (fn [node]685 (let [sensor-origins686 (map687 #(map (partial local-to-world geo) %)688 feeler-coords)689 triangle-normals690 (map (partial get-ray-direction geo)691 tris)692 rays693 (flatten694 (map (fn [origins norm]695 (map #(doto (Ray. % norm)696 (.setLimit limit)) origins))697 sensor-origins triangle-normals))]698 (vector699 (touch-topology geo)700 (vec701 (for [ray rays]702 (do703 (let [results (CollisionResults.)]704 (.collideWith node ray results)705 (let [touch-objects706 (filter #(not (= geo (.getGeometry %)))707 results)]708 (- 255709 (if (empty? touch-objects) 255710 (rem711 (int712 (* 255 (/ (.getDistance713 (first touch-objects)) limit)))714 256))))))))))))))717 (defn touch [#^Node pieces]718 (filter (comp not nil?)719 (map enable-touch720 (filter #(isa? (class %) Geometry)721 (node-seq pieces)))))724 ;; human eye transmits 62kb/s to brain Bandwidth is 8.75 Mb/s725 ;; http://en.wikipedia.org/wiki/Retina727 (defn test-eye []728 (.getChild729 (.getChild (worm-model) "eyes")730 "eye"))733 (defn retina-sensor-image734 "Return a map of pixel selection functions to BufferedImages735 describing the distribution of light-sensitive components on this736 geometry's surface. Each function creates an integer from the rgb737 values found in the pixel. :red, :green, :blue, :gray are already738 defined as extracting the red green blue and average components739 respectively."740 [#^Spatial eye]741 (if-let [eye-map (meta-data eye "eye")]742 (map-vals743 #(ImageToAwt/convert744 (.getImage (.loadTexture (asset-manager) %))745 false false 0)746 (eval (read-string eye-map)))))748 (defn eye-dimensions749 "returns the width and height specified in the metadata of the eye"750 [#^Spatial eye]751 (let [dimensions752 (map #(vector (.getWidth %) (.getHeight %))753 (vals (retina-sensor-image eye)))]754 [(apply max (map first dimensions))755 (apply max (map second dimensions))]))757 (defn creature-eyes758 ;;dylan759 "Return the children of the creature's \"eyes\" node."760 ;;"The eye nodes which are children of the \"eyes\" node in the761 ;;creature."762 [#^Node creature]763 (if-let [eye-node (.getChild creature "eyes")]764 (seq (.getChildren eye-node))765 (do (println-repl "could not find eyes node") [])))767 ;; Here's how vision will work.769 ;; Make the continuation in scene-processor take FrameBuffer,770 ;; byte-buffer, BufferedImage already sized to the correct771 ;; dimensions. the continuation will decide wether to "mix" them772 ;; into the BufferedImage, lazily ignore them, or mix them halfway773 ;; and call c/graphics card routines.775 ;; (vision creature) will take an optional :skip argument which will776 ;; inform the continuations in scene processor to skip the given777 ;; number of cycles; 0 means that no cycles will be skipped.779 ;; (vision creature) will return [init-functions sensor-functions].780 ;; The init-functions are each single-arg functions that take the781 ;; world and register the cameras and must each be called before the782 ;; corresponding sensor-functions. Each init-function returns the783 ;; viewport for that eye which can be manipulated, saved, etc. Each784 ;; sensor-function is a thunk and will return data in the same785 ;; format as the tactile-sensor functions; the structure is786 ;; [topology, sensor-data]. Internally, these sensor-functions787 ;; maintain a reference to sensor-data which is periodically updated788 ;; by the continuation function established by its init-function.789 ;; They can be queried every cycle, but their information may not790 ;; necessairly be different every cycle.792 ;; Each eye in the creature in blender will work the same way as793 ;; joints -- a zero dimensional object with no geometry whose local794 ;; coordinate system determines the orientation of the resulting795 ;; eye. All eyes will have a parent named "eyes" just as all joints796 ;; have a parent named "joints". The resulting camera will be a797 ;; ChaseCamera or a CameraNode bound to the geo that is closest to798 ;; the eye marker. The eye marker will contain the metadata for the799 ;; eye, and will be moved by it's bound geometry. The dimensions of800 ;; the eye's camera are equal to the dimensions of the eye's "UV"801 ;; map.804 ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;806 ;; Ears work the same way as vision.808 ;; (hearing creature) will return [init-functions809 ;; sensor-functions]. The init functions each take the world and810 ;; register a SoundProcessor that does foureier transforms on the811 ;; incommong sound data, making it available to each sensor function.813 (defn creature-ears814 "Return the children of the creature's \"ears\" node."815 ;;dylan816 ;;"The ear nodes which are children of the \"ears\" node in the817 ;;creature."818 [#^Node creature]819 (if-let [ear-node (.getChild creature "ears")]820 (seq (.getChildren ear-node))821 (do (println-repl "could not find ears node") [])))823 (defn closest-node824 "Return the object in creature which is closest to the given node."825 ;;dylan"The closest object in creature to the given node."826 [#^Node creature #^Node eye]827 (loop [radius (float 0.01)]828 (let [results (CollisionResults.)]829 (.collideWith830 creature831 (BoundingBox. (.getWorldTranslation eye)832 radius radius radius)833 results)834 (if-let [target (first results)]835 (.getGeometry target)836 (recur (float (* 2 radius)))))))838 ;;dylan (defn follow-sense, adjoin-sense, attach-stimuli,839 ;;anchor-qualia, augment-organ, with-organ840 (defn bind-sense841 "Bind the sense to the Spatial such that it will maintain its842 current position relative to the Spatial no matter how the spatial843 moves. 'sense can be either a Camera or Listener object."844 [#^Spatial obj sense]845 (let [sense-offset (.subtract (.getLocation sense)846 (.getWorldTranslation obj))847 initial-sense-rotation (Quaternion. (.getRotation sense))848 base-anti-rotation (.inverse (.getWorldRotation obj))]849 (.addControl850 obj851 (proxy [AbstractControl] []852 (controlUpdate [tpf]853 (let [total-rotation854 (.mult base-anti-rotation (.getWorldRotation obj))]855 (.setLocation sense856 (.add857 (.mult total-rotation sense-offset)858 (.getWorldTranslation obj)))859 (.setRotation sense860 (.mult total-rotation initial-sense-rotation))))861 (controlRender [_ _])))))864 (defn update-listener-velocity865 "Update the listener's velocity every update loop."866 [#^Spatial obj #^Listener lis]867 (let [old-position (atom (.getLocation lis))]868 (.addControl869 obj870 (proxy [AbstractControl] []871 (controlUpdate [tpf]872 (let [new-position (.getLocation lis)]873 (.setVelocity874 lis875 (.mult (.subtract new-position @old-position)876 (float (/ tpf))))877 (reset! old-position new-position)))878 (controlRender [_ _])))))880 (import com.aurellem.capture.audio.AudioSendRenderer)882 (defn attach-ear883 [#^Application world #^Node creature #^Spatial ear continuation]884 (let [target (closest-node creature ear)885 lis (Listener.)886 audio-renderer (.getAudioRenderer world)887 sp (sound-processor continuation)]888 (.setLocation lis (.getWorldTranslation ear))889 (.setRotation lis (.getWorldRotation ear))890 (bind-sense target lis)891 (update-listener-velocity target lis)892 (.addListener audio-renderer lis)893 (.registerSoundProcessor audio-renderer lis sp)))895 (defn enable-hearing896 [#^Node creature #^Spatial ear]897 (let [hearing-data (atom [])]898 [(fn [world]899 (attach-ear world creature ear900 (fn [data]901 (reset! hearing-data (vec data)))))902 [(fn []903 (let [data @hearing-data904 topology905 (vec (map #(vector % 0) (range 0 (count data))))906 scaled-data907 (vec908 (map909 #(rem (int (* 255 (/ (+ 1 %) 2))) 256)910 data))]911 [topology scaled-data]))912 ]]))914 (defn hearing915 [#^Node creature]916 (reduce917 (fn [[init-a senses-a]918 [init-b senses-b]]919 [(conj init-a init-b)920 (into senses-a senses-b)])921 [[][]]922 (for [ear (creature-ears creature)]923 (enable-hearing creature ear))))925 (defn attach-eye926 "Attach a Camera to the appropiate area and return the Camera."927 [#^Node creature #^Spatial eye]928 (let [target (closest-node creature eye)929 [cam-width cam-height] (eye-dimensions eye)930 cam (Camera. cam-width cam-height)]931 (.setLocation cam (.getWorldTranslation eye))932 (.setRotation cam (.getWorldRotation eye))933 (.setFrustumPerspective934 cam 45 (/ (.getWidth cam) (.getHeight cam))935 1 1000)936 (bind-sense target cam)937 cam))939 (def presets940 {:all 0xFFFFFF941 :red 0xFF0000942 :blue 0x0000FF943 :green 0x00FF00})945 (defn enable-vision946 "return [init-function sensor-functions] for a particular eye"947 [#^Node creature #^Spatial eye & {skip :skip :or {skip 0}}]948 (let [retinal-map (retina-sensor-image eye)949 camera (attach-eye creature eye)950 vision-image951 (atom952 (BufferedImage. (.getWidth camera)953 (.getHeight camera)954 BufferedImage/TYPE_BYTE_BINARY))]955 [(fn [world]956 (add-eye957 world camera958 (let [counter (atom 0)]959 (fn [r fb bb bi]960 (if (zero? (rem (swap! counter inc) (inc skip)))961 (reset! vision-image (BufferedImage! r fb bb bi)))))))962 (vec963 (map964 (fn [[key image]]965 (let [whites (white-coordinates image)966 topology (vec (collapse whites))967 mask (presets key)]968 (fn []969 (vector970 topology971 (vec972 (for [[x y] whites]973 (bit-and974 mask (.getRGB @vision-image x y))))))))975 retinal-map))]))977 (defn vision978 [#^Node creature & {skip :skip :or {skip 0}}]979 (reduce980 (fn [[init-a senses-a]981 [init-b senses-b]]982 [(conj init-a init-b)983 (into senses-a senses-b)])984 [[][]]985 (for [eye (creature-eyes creature)]986 (enable-vision creature eye))))992 ;; lower level --- nodes993 ;; closest-node "parse/compile-x" -> makes organ, which is spatial, fn pair995 ;; higher level -- organs996 ;;998 ;; higher level --- sense/effector999 ;; these are the functions that provide world i/o, chinese-room style1003 (defn blender-creature1004 "Return a creature with all joints in place."1005 [blender-path]1006 (let [model (load-blender-model blender-path)1007 joints1008 (if-let [joint-node (.getChild model "joints")]1009 (seq (.getChildren joint-node))1010 (do (println-repl "could not find joints node") []))]1011 (assemble-creature model joints)))1013 (defn gray-scale [num]1014 (+ num1015 (bit-shift-left num 8)1016 (bit-shift-left num 16)))1018 (defn debug-window1019 "creates function that offers a debug view of sensor data"1020 []1021 (let [vi (view-image)]1022 (fn1023 [[coords sensor-data]]1024 (let [image (points->image coords)]1025 (dorun1026 (for [i (range (count coords))]1027 (.setRGB image ((coords i) 0) ((coords i) 1)1028 (gray-scale (sensor-data i)))))1031 (vi image)))))1033 (defn debug-vision-window1034 "creates function that offers a debug view of sensor data"1035 []1036 (let [vi (view-image)]1037 (fn1038 [[coords sensor-data]]1039 (let [image (points->image coords)]1040 (dorun1041 (for [i (range (count coords))]1042 (.setRGB image ((coords i) 0) ((coords i) 1)1043 (sensor-data i))))1044 (vi image)))))1046 (defn debug-hearing-window1047 "view audio data"1048 [height]1049 (let [vi (view-image)]1050 (fn [[coords sensor-data]]1051 (let [image (BufferedImage. (count coords) height1052 BufferedImage/TYPE_INT_RGB)]1053 (dorun1054 (for [x (range (count coords))]1055 (dorun1056 (for [y (range height)]1057 (let [raw-sensor (sensor-data x)]1058 (.setRGB image x y (gray-scale raw-sensor)))))))1060 (vi image)))))1064 ;;(defn test-touch [world creature]1069 ;; here's how motor-control/ proprioception will work:1077 (defn test-creature [thing]1078 (let [x-axis1079 (box 1 0.01 0.01 :physical? false :color ColorRGBA/Red)1080 y-axis1081 (box 0.01 1 0.01 :physical? false :color ColorRGBA/Green)1082 z-axis1083 (box 0.01 0.01 1 :physical? false :color ColorRGBA/Blue)1084 creature (blender-creature thing)1085 touch-nerves (touch creature)1086 touch-debug-windows (map (fn [_] (debug-window)) touch-nerves)1087 [init-vision-fns vision-data] (vision creature)1088 vision-debug (map (fn [_] (debug-vision-window)) vision-data)1089 me (sphere 0.5 :color ColorRGBA/Blue :physical? false)1090 [init-hearing-fns hearing-senses] (hearing creature)1091 hearing-windows (map (fn [_] (debug-hearing-window 50))1092 hearing-senses)1093 bell (AudioNode. (asset-manager)1094 "Sounds/pure.wav" false)1095 ;; dream1097 ]1098 (world1099 (nodify [creature1100 (box 10 2 10 :position (Vector3f. 0 -9 0)1101 :color ColorRGBA/Gray :mass 0)1102 x-axis y-axis z-axis1103 me1104 ])1105 (merge standard-debug-controls1106 {"key-return"1107 (fn [_ value]1108 (if value1109 (do1110 (println-repl "play-sound")1111 (.play bell))))})1112 (fn [world]1113 (light-up-everything world)1114 (enable-debug world)1115 (dorun (map #(% world) init-vision-fns))1116 (dorun (map #(% world) init-hearing-fns))1118 (add-eye world1119 (attach-eye creature (test-eye))1120 (comp (view-image) BufferedImage!))1122 (add-eye world (.getCamera world) no-op)1124 ;;(com.aurellem.capture.Capture/captureVideo1125 ;; world (file-str "/home/r/proj/ai-videos/hand"))1126 ;;(.setTimer world (RatchetTimer. 60))1127 (speed-up world)1128 ;;(set-gravity world (Vector3f. 0 0 0))1129 )1130 (fn [world tpf]1131 ;;(dorun1132 ;; (map #(%1 %2) touch-nerves (repeat (.getRootNode world))))1136 (dorun1137 (map #(%1 (%2 (.getRootNode world)))1138 touch-debug-windows touch-nerves))1140 (dorun1141 (map #(%1 (%2))1142 vision-debug vision-data))1143 (dorun1144 (map #(%1 (%2)) hearing-windows hearing-senses))1147 ;;(println-repl (vision-data))1148 (.setLocalTranslation me (.getLocation (.getCamera world)))1151 )1152 ;;(let [timer (atom 0)]1153 ;; (fn [_ _]1154 ;; (swap! timer inc)1155 ;; (if (= (rem @timer 60) 0)1156 ;; (println-repl (float (/ @timer 60))))))1157 )))1167 ;;; experiments in collisions1171 (defn collision-test []1172 (let [b-radius 11173 b-position (Vector3f. 0 0 0)1174 obj-b (box 1 1 1 :color ColorRGBA/Blue1175 :position b-position1176 :mass 0)1177 node (nodify [obj-b])1178 bounds-b1179 (doto (Picture.)1180 (.setHeight 50)1181 (.setWidth 50)1182 (.setImage (asset-manager)1183 "Models/creature1/hand.png"1184 false1185 ))1187 ;;(Ray. (Vector3f. 0 -5 0) (.normalize (Vector3f. 0 1 0)))1189 collisions1190 (let [cr (CollisionResults.)]1191 (.collideWith node bounds-b cr)1192 (println (map #(.getContactPoint %) cr))1193 cr)1195 ;;collision-points1196 ;;(map #(sphere 0.1 :position (.getContactPoint %))1197 ;; collisions)1199 ;;node (nodify (conj collision-points obj-b))1201 sim1202 (world node1203 {"key-space"1204 vvvvvv (fn [_ value]1205 (if value1206 (let [cr (CollisionResults.)]1207 (.collideWith node bounds-b cr)1208 (println-repl (map #(.getContactPoint %) cr))1209 cr)))}1210 no-op1211 no-op)1213 ]1214 sim1216 ))1219 ;; the camera will stay in its initial position/rotation with relation1220 ;; to the spatial.1223 (defn follow-test1224 "show a camera that stays in the same relative position to a blue cube."1225 []1226 (let [camera-pos (Vector3f. 0 30 0)1227 rock (box 1 1 1 :color ColorRGBA/Blue1228 :position (Vector3f. 0 10 0)1229 :mass 301230 )1231 rot (.getWorldRotation rock)1233 table (box 3 1 10 :color ColorRGBA/Gray :mass 01234 :position (Vector3f. 0 -3 0))]1236 (world1237 (nodify [rock table])1238 standard-debug-controls1239 (fn [world]1240 (let1241 [cam (doto (.clone (.getCamera world))1242 (.setLocation camera-pos)1243 (.lookAt Vector3f/ZERO1244 Vector3f/UNIT_X))]1245 (bind-sense rock cam)1247 (.setTimer world (RatchetTimer. 60))1248 (add-eye world cam (comp (view-image) BufferedImage!))1249 (add-eye world (.getCamera world) no-op))1250 )1251 (fn [_ _] (println-repl rot)))))1255 #+end_src1257 #+results: body-11258 : #'cortex.silly/test-creature1261 * COMMENT purgatory1262 #+begin_src clojure1263 (defn bullet-trans []1264 (let [obj-a (sphere 0.5 :color ColorRGBA/Red1265 :position (Vector3f. -10 5 0))1266 obj-b (sphere 0.5 :color ColorRGBA/Blue1267 :position (Vector3f. -10 -5 0)1268 :mass 0)1269 control-a (.getControl obj-a RigidBodyControl)1270 control-b (.getControl obj-b RigidBodyControl)1271 swivel1272 (.toRotationMatrix1273 (doto (Quaternion.)1274 (.fromAngleAxis (/ Math/PI 2)1275 Vector3f/UNIT_X)))]1276 (doto1277 (ConeJoint.1278 control-a control-b1279 (Vector3f. 0 5 0)1280 (Vector3f. 0 -5 0)1281 swivel swivel)1282 (.setLimit (* 0.6 (/ Math/PI 4))1283 (/ Math/PI 4)1284 (* Math/PI 0.8)))1285 (world (nodify1286 [obj-a obj-b])1287 standard-debug-controls1288 enable-debug1289 no-op)))1292 (defn bullet-trans* []1293 (let [obj-a (box 1.5 0.5 0.5 :color ColorRGBA/Red1294 :position (Vector3f. 5 0 0)1295 :mass 90)1296 obj-b (sphere 0.5 :color ColorRGBA/Blue1297 :position (Vector3f. -5 0 0)1298 :mass 0)1299 control-a (.getControl obj-a RigidBodyControl)1300 control-b (.getControl obj-b RigidBodyControl)1301 move-up? (atom nil)1302 move-down? (atom nil)1303 move-left? (atom nil)1304 move-right? (atom nil)1305 roll-left? (atom nil)1306 roll-right? (atom nil)1307 force 1001308 swivel1309 (.toRotationMatrix1310 (doto (Quaternion.)1311 (.fromAngleAxis (/ Math/PI 2)1312 Vector3f/UNIT_X)))1313 x-move1314 (doto (Matrix3f.)1315 (.fromStartEndVectors Vector3f/UNIT_X1316 (.normalize (Vector3f. 1 1 0))))1318 timer (atom 0)]1319 (doto1320 (ConeJoint.1321 control-a control-b1322 (Vector3f. -8 0 0)1323 (Vector3f. 2 0 0)1324 ;;swivel swivel1325 ;;Matrix3f/IDENTITY Matrix3f/IDENTITY1326 x-move Matrix3f/IDENTITY1327 )1328 (.setCollisionBetweenLinkedBodys false)1329 (.setLimit (* 1 (/ Math/PI 4)) ;; twist1330 (* 1 (/ Math/PI 4)) ;; swing span in X-Y plane1331 (* 0 (/ Math/PI 4)))) ;; swing span in Y-Z plane1332 (world (nodify1333 [obj-a obj-b])1334 (merge standard-debug-controls1335 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))1336 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))1337 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))1338 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))1339 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))1340 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})1342 (fn [world]1343 (enable-debug world)1344 (set-gravity world Vector3f/ZERO)1345 )1347 (fn [world _]1349 (if @move-up?1350 (.applyForce control-a1351 (Vector3f. force 0 0)1352 (Vector3f. 0 0 0)))1353 (if @move-down?1354 (.applyForce control-a1355 (Vector3f. (- force) 0 0)1356 (Vector3f. 0 0 0)))1357 (if @move-left?1358 (.applyForce control-a1359 (Vector3f. 0 force 0)1360 (Vector3f. 0 0 0)))1361 (if @move-right?1362 (.applyForce control-a1363 (Vector3f. 0 (- force) 0)1364 (Vector3f. 0 0 0)))1366 (if @roll-left?1367 (.applyForce control-a1368 (Vector3f. 0 0 force)1369 (Vector3f. 0 0 0)))1370 (if @roll-right?1371 (.applyForce control-a1372 (Vector3f. 0 0 (- force))1373 (Vector3f. 0 0 0)))1375 (if (zero? (rem (swap! timer inc) 100))1376 (.attachChild1377 (.getRootNode world)1378 (sphere 0.05 :color ColorRGBA/Yellow1379 :physical? false :position1380 (.getWorldTranslation obj-a)))))1381 )1382 ))1384 (defn transform-trianglesdsd1385 "Transform that converts each vertex in the first triangle1386 into the corresponding vertex in the second triangle."1387 [#^Triangle tri-1 #^Triangle tri-2]1388 (let [in [(.get1 tri-1)1389 (.get2 tri-1)1390 (.get3 tri-1)]1391 out [(.get1 tri-2)1392 (.get2 tri-2)1393 (.get3 tri-2)]]1394 (let [translate (doto (Matrix4f.) (.setTranslation (.negate (in 0))))1395 in* [(.mult translate (in 0))1396 (.mult translate (in 1))1397 (.mult translate (in 2))]1398 final-translation1399 (doto (Matrix4f.)1400 (.setTranslation (out 1)))1402 rotate-11403 (doto (Matrix3f.)1404 (.fromStartEndVectors1405 (.normalize1406 (.subtract1407 (in* 1) (in* 0)))1408 (.normalize1409 (.subtract1410 (out 1) (out 0)))))1411 in** [(.mult rotate-1 (in* 0))1412 (.mult rotate-1 (in* 1))1413 (.mult rotate-1 (in* 2))]1414 scale-factor-11415 (.mult1416 (.normalize1417 (.subtract1418 (out 1)1419 (out 0)))1420 (/ (.length1421 (.subtract (out 1)1422 (out 0)))1423 (.length1424 (.subtract (in** 1)1425 (in** 0)))))1426 scale-1 (doto (Matrix4f.) (.setScale scale-factor-1))1427 in*** [(.mult scale-1 (in** 0))1428 (.mult scale-1 (in** 1))1429 (.mult scale-1 (in** 2))]1435 ]1437 (dorun (map println in))1438 (println)1439 (dorun (map println in*))1440 (println)1441 (dorun (map println in**))1442 (println)1443 (dorun (map println in***))1444 (println)1446 ))))1449 (defn world-setup [joint]1450 (let [joint-position (Vector3f. 0 0 0)1451 joint-rotation1452 (.toRotationMatrix1453 (.mult1454 (doto (Quaternion.)1455 (.fromAngleAxis1456 (* 1 (/ Math/PI 4))1457 (Vector3f. -1 0 0)))1458 (doto (Quaternion.)1459 (.fromAngleAxis1460 (* 1 (/ Math/PI 2))1461 (Vector3f. 0 0 1)))))1462 top-position (.mult joint-rotation (Vector3f. 8 0 0))1464 origin (doto1465 (sphere 0.1 :physical? false :color ColorRGBA/Cyan1466 :position top-position))1467 top (doto1468 (sphere 0.1 :physical? false :color ColorRGBA/Yellow1469 :position top-position)1471 (.addControl1472 (RigidBodyControl.1473 (CapsuleCollisionShape. 0.5 1.5 1) (float 20))))1474 bottom (doto1475 (sphere 0.1 :physical? false :color ColorRGBA/DarkGray1476 :position (Vector3f. 0 0 0))1477 (.addControl1478 (RigidBodyControl.1479 (CapsuleCollisionShape. 0.5 1.5 1) (float 0))))1480 table (box 10 2 10 :position (Vector3f. 0 -20 0)1481 :color ColorRGBA/Gray :mass 0)1482 a (.getControl top RigidBodyControl)1483 b (.getControl bottom RigidBodyControl)]1485 (cond1486 (= joint :cone)1488 (doto (ConeJoint.1489 a b1490 (world-to-local top joint-position)1491 (world-to-local bottom joint-position)1492 joint-rotation1493 joint-rotation1494 )1497 (.setLimit (* (/ 10) Math/PI)1498 (* (/ 4) Math/PI)1499 0)))1500 [origin top bottom table]))1502 (defn test-joint [joint]1503 (let [[origin top bottom floor] (world-setup joint)1504 control (.getControl top RigidBodyControl)1505 move-up? (atom false)1506 move-down? (atom false)1507 move-left? (atom false)1508 move-right? (atom false)1509 roll-left? (atom false)1510 roll-right? (atom false)1511 timer (atom 0)]1513 (world1514 (nodify [top bottom floor origin])1515 (merge standard-debug-controls1516 {"key-r" (fn [_ pressed?] (reset! move-up? pressed?))1517 "key-t" (fn [_ pressed?] (reset! move-down? pressed?))1518 "key-f" (fn [_ pressed?] (reset! move-left? pressed?))1519 "key-g" (fn [_ pressed?] (reset! move-right? pressed?))1520 "key-v" (fn [_ pressed?] (reset! roll-left? pressed?))1521 "key-b" (fn [_ pressed?] (reset! roll-right? pressed?))})1523 (fn [world]1524 (light-up-everything world)1525 (enable-debug world)1526 (set-gravity world (Vector3f. 0 0 0))1527 )1529 (fn [world _]1530 (if (zero? (rem (swap! timer inc) 100))1531 (do1532 ;; (println-repl @timer)1533 (.attachChild (.getRootNode world)1534 (sphere 0.05 :color ColorRGBA/Yellow1535 :position (.getWorldTranslation top)1536 :physical? false))1537 (.attachChild (.getRootNode world)1538 (sphere 0.05 :color ColorRGBA/LightGray1539 :position (.getWorldTranslation bottom)1540 :physical? false))))1542 (if @move-up?1543 (.applyTorque control1544 (.mult (.getPhysicsRotation control)1545 (Vector3f. 0 0 10))))1546 (if @move-down?1547 (.applyTorque control1548 (.mult (.getPhysicsRotation control)1549 (Vector3f. 0 0 -10))))1550 (if @move-left?1551 (.applyTorque control1552 (.mult (.getPhysicsRotation control)1553 (Vector3f. 0 10 0))))1554 (if @move-right?1555 (.applyTorque control1556 (.mult (.getPhysicsRotation control)1557 (Vector3f. 0 -10 0))))1558 (if @roll-left?1559 (.applyTorque control1560 (.mult (.getPhysicsRotation control)1561 (Vector3f. -1 0 0))))1562 (if @roll-right?1563 (.applyTorque control1564 (.mult (.getPhysicsRotation control)1565 (Vector3f. 1 0 0))))))))1569 (defprotocol Frame1570 (frame [this]))1572 (extend-type BufferedImage1573 Frame1574 (frame [image]1575 (merge1576 (apply1577 hash-map1578 (interleave1579 (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]1580 (vector x y)))1581 (doall (for [x (range (.getWidth image)) y (range (.getHeight image))]1582 (let [data (.getRGB image x y)]1583 (hash-map :r (bit-shift-right (bit-and 0xff0000 data) 16)1584 :g (bit-shift-right (bit-and 0x00ff00 data) 8)1585 :b (bit-and 0x0000ff data)))))))1586 {:width (.getWidth image) :height (.getHeight image)})))1589 (extend-type ImagePlus1590 Frame1591 (frame [image+]1592 (frame (.getBufferedImage image+))))1595 #+end_src1598 * COMMENT generate source1599 #+begin_src clojure :tangle ../src/cortex/silly.clj1600 <<body-1>>1601 #+end_src