# HG changeset patch # User Robert McIntyre # Date 1328705415 25200 # Node ID d5c597a7aed47ef78b2e0f52061a35e50fcb1117 # Parent 1c915cc1118b863a00fd281ac616ee89d0ee8c42 writing prose for body.org diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/basic-muscle.png Binary file assets/Models/creature1/basic-muscle.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/hand.png Binary file assets/Models/creature1/hand.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/one.bk.blend Binary file assets/Models/creature1/one.bk.blend has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/one.blend Binary file assets/Models/creature1/one.blend has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/retina-UV-small.png Binary file assets/Models/creature1/retina-UV-small.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/retina-small.png Binary file assets/Models/creature1/retina-small.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/retina-small.xcf Binary file assets/Models/creature1/retina-small.xcf has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/retina-uv.png Binary file assets/Models/creature1/retina-uv.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/retina.png Binary file assets/Models/creature1/retina.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/retina.xcf Binary file assets/Models/creature1/retina.xcf has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/tip.png Binary file assets/Models/creature1/tip.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/tip.xcf Binary file assets/Models/creature1/tip.xcf has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/touch.blend Binary file assets/Models/creature1/touch.blend has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/try-again.blend Binary file assets/Models/creature1/try-again.blend has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/worm-2-UV-layout.png Binary file assets/Models/creature1/worm-2-UV-layout.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/creature1/worm-2-UV-layout.svg --- a/assets/Models/creature1/worm-2-UV-layout.svg Tue Feb 07 07:13:45 2012 -0700 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,8 +0,0 @@ - - - -'try-again.blend', worm-2, (Blender 2.59 (sub 4)) - - diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/basic-muscle.png Binary file assets/Models/test-creature/basic-muscle.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/hand.blend Binary file assets/Models/test-creature/hand.blend has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/hand.png Binary file assets/Models/test-creature/hand.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/palm-UV.png Binary file assets/Models/test-creature/palm-UV.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/palm.png Binary file assets/Models/test-creature/palm.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/palm.xcf Binary file assets/Models/test-creature/palm.xcf has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/retina-UV-small.png Binary file assets/Models/test-creature/retina-UV-small.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/retina-small.png Binary file assets/Models/test-creature/retina-small.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/retina-small.xcf Binary file assets/Models/test-creature/retina-small.xcf has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/retina-uv.png Binary file assets/Models/test-creature/retina-uv.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/retina.png Binary file assets/Models/test-creature/retina.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/retina.xcf Binary file assets/Models/test-creature/retina.xcf has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/tip.png Binary file assets/Models/test-creature/tip.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/tip.xcf Binary file assets/Models/test-creature/tip.xcf has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/worm-2-UV-layout.png Binary file assets/Models/test-creature/worm-2-UV-layout.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/worm-2-UV-layout.svg --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/assets/Models/test-creature/worm-2-UV-layout.svg Wed Feb 08 05:50:15 2012 -0700 @@ -0,0 +1,8 @@ + + + +'try-again.blend', worm-2, (Blender 2.59 (sub 4)) + + diff -r 1c915cc1118b -r d5c597a7aed4 assets/Models/test-creature/worm.blend Binary file assets/Models/test-creature/worm.blend has changed diff -r 1c915cc1118b -r d5c597a7aed4 images/Eve.jpg Binary file images/Eve.jpg has changed diff -r 1c915cc1118b -r d5c597a7aed4 images/hand-screenshot.png Binary file images/hand-screenshot.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 images/hand-screenshot0.png Binary file images/hand-screenshot0.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 images/hand-screenshot1.png Binary file images/hand-screenshot1.png has changed diff -r 1c915cc1118b -r d5c597a7aed4 org/body.org --- a/org/body.org Tue Feb 07 07:13:45 2012 -0700 +++ b/org/body.org Wed Feb 08 05:50:15 2012 -0700 @@ -1,31 +1,200 @@ -#+title: The BODY!!! +#+title: Building a Body #+author: Robert McIntyre #+email: rlm@mit.edu #+description: Simulating a body (movement, touch, propioception) in jMonkeyEngine3. #+SETUPFILE: ../../aurellem/org/setup.org #+INCLUDE: ../../aurellem/org/level-0.org -* Making a solid, connected body. -#+name: joints + +* Design Constraints + +I use [[www.blender.org/][blender]] to design bodies. The design of the bodies is +determined by the requirements of the AI that will use them. The +bodies must be easy for an AI to sense and control, and they must be +relatively simple for jMonkeyEngine to compute. + +** Bag of Bones + +How to create such a body? One option I ultimately rejected is to use +blender's [[http://wiki.blender.org/index.php/Doc:2.6/Manual/Rigging/Armatures][armature]] system. The idea would have been to define a mesh +which describes the creature's entire body. To this you add an +(skeleton) which deforms this mesh. This technique is used extensively +to model humans and create realistic animations. It is hard to use for +my purposes because it is difficult to update the creature's Physics +Collision Mesh in tandem with its Geometric Mesh under the influence +of the armature. Withouth this the creature will not be able to grab +things in its environment, and it won't be able to tell where its +physical body is by using its eyes. Also, armatures do not specify +any rotational limits for a joint, making it hard to model elbows, +shoulders, etc. + +** EVE + +Instead of using the human-like "deformable bag of bones" approach, I +decided to base my body plans on the robot EVE from the movie wall-E. + +#+caption: EVE from the movie WALL-E. This body plan turns out to be much better suited to my purposes than a more human-like one. +[[../images/Eve.jpg]] + +The main reason that I use eve-style bodies is so that there will be +correspondence between the AI's vision and the physical presence of +its body. + +* Solidifying the Body + +Here is a hand designed eve-style in blender. + + +#+attr_html: width="500" +[[../images/hand-screenshot0.png]] + +If we load it directly into jMonkeyEngine, we get this: + +#+name: test-0 +#+begin_src clojure +(ns cortex.test.body + (:use (cortex world util body)) + (:import (com.aurellem.capture Capture RatchetTimer) + (com.jme3.math Quaternion Vector3f) + java.io.File)) + +(def hand-path "Models/test-creature/hand.blend") + +(defn hand [] (load-blender-model hand-path)) + +(defn setup [world] + (let [cam (.getCamera world)] + (println-repl cam) + (.setLocation + cam (Vector3f. + -6.9015837, 8.644911, 5.6043186)) + (.setRotation + cam + (Quaternion. + 0.14046453, 0.85894054, -0.34301838, 0.3533118))) + (light-up-everything world) + (.setTimer world (RatchetTimer. 60)) + world) + +(defn test-one [] + (world (hand) + standard-debug-controls + (comp + #(Capture/captureVideo + % (File. "/home/r/proj/cortex/render/body/1")) + setup) + no-op)) +#+end_src + + +#+begin_src clojure :results silent +(.start (cortex.test.body/test-one)) +#+end_src + +#+begin_html + +#+end_html + +You will notice that the hand has no physical presence -- it's a +hologram through witch everything passes. Therefore, the first thing +to do is to make it solid. Blender has physics simulation on par with +jMonkeyEngine (they both use bullet as their physics backend), but it +can be difficult to translate between the two systems, so for now I +specify the mass of each object in blender and construct the physics +shape based on the mesh in jMonkeyEngine. + +#+name: joints-1 +#+begin_src clojure +(defn physical! + "Iterate through the nodes in creature and make them real physical + objects in the simulation." + [#^Node creature] + (dorun + (map + (fn [geom] + (let [physics-control + (RigidBodyControl. + (HullCollisionShape. + (.getMesh geom)) + (if-let [mass (meta-data geom "mass")] + (do + (println-repl + "setting" (.getName geom) "mass to" (float mass)) + (float mass)) + (float 1)))] + (.addControl geom physics-control))) + (filter #(isa? (class %) Geometry ) + (node-seq creature))))) +#+end_src + +=(physical!)= iterates through a creature's node structure, creating +CollisionShapes for each geometry with the mass specified in that +geometry's meta-data. + #+begin_src clojure -(ns cortex.body - "Assemble a physical creature using the definitions found in a - specially prepared blender file. Creates rigid bodies and joints so - that a creature can have a physical presense in the simulation." - {:author "Robert McIntyre"} - (:use (cortex world util sense)) - (:use clojure.contrib.def) - (:import - (com.jme3.math Vector3f Quaternion Vector2f Matrix3f) - (com.jme3.bullet.joints - SixDofJoint Point2PointJoint HingeJoint ConeJoint) - com.jme3.bullet.control.RigidBodyControl - com.jme3.collision.CollisionResults - com.jme3.bounding.BoundingBox - com.jme3.scene.Node - com.jme3.scene.Geometry - com.jme3.bullet.collision.shapes.HullCollisionShape)) +(in-ns 'cortex.test.body) +(def normal-gravity + {"key-g" (fn [world _] + (set-gravity world (Vector3f. 0 -9.81 0)))}) + +(defn floor [] + (box 10 3 10 :position (Vector3f. 0 -10 0) + :color ColorRGBA/Gray :mass 0)) + +(defn test-two [] + (world (nodify + [(doto (hand) + (physical!)) + (floor)]) + (merge standard-debug-controls normal-gravity) + (comp + #(Capture/captureVideo + % (File. "/home/r/proj/cortex/render/body/2")) + #(do (set-gravity % Vector3f/ZERO) %) + setup) + no-op)) +#+end_src + +#+begin_html + +#+end_html + +Now that's some progress. + + +* Joints + +Obviously, an AI is not going to be doing much just lying in pieces on +the floor. So, the next step to making a proper body is to connect +those pieces together with joints. jMonkeyEngine has a large array of +joints available via bullet, such as Point2Point, Cone, Hinge, and a +generic Six Degree of Freedom joint, with or without spring +restitution. + +Although it should be possible to specify the joints using blender's +physics system, and then automatically import them with jMonkeyEngine, +the support isn't there yet, and there are a few problems with bullet +itself that need to be solved before it can happen. + +So, I will use the same system for specifying joints as I will do for +some senses. Each joint is specified by an empty node whose parent +has the name "joints". Their orientation and meta-data determine what +joint is created. + +[[../images/hand-screenshot1.png]] + + + + +#+name: joints-2 +#+begin_src clojure (defn joint-targets "Return the two closest two objects to the joint object, ordered from bottom to top according to the joint's rotation." @@ -183,28 +352,6 @@ (sense-nodes "joints") "Return the children of the creature's \"joints\" node.") -(defn physical! - "Iterate through the nodes in creature and make them real physical - objects in the simulation." - [#^Node creature] - (dorun - (map - (fn [geom] - (let [physics-control - (RigidBodyControl. - (HullCollisionShape. - (.getMesh geom)) - (if-let [mass (meta-data geom "mass")] - (do - (println-repl - "setting" (.getName geom) "mass to" (float mass)) - (float mass)) - (float 1)))] - - (.addControl geom physics-control))) - (filter #(isa? (class %) Geometry ) - (node-seq creature))))) - (defn joints! "Connect the solid parts of the creature with physical joints. The joints are taken from the \"joints\" node in the creature." @@ -225,13 +372,32 @@ (joints! creature)) #+end_src -#+results: joints -: #'cortex.body/body! +* Bookkeeping -#+results: proprioception -: #'cortex.body/proprioception-debug-window +#+name: body-0 +#+begin_src clojure +(ns cortex.body + "Assemble a physical creature using the definitions found in a + specially prepared blender file. Creates rigid bodies and joints so + that a creature can have a physical presense in the simulation." + {:author "Robert McIntyre"} + (:use (cortex world util sense)) + (:use clojure.contrib.def) + (:import + (com.jme3.math Vector3f Quaternion Vector2f Matrix3f) + (com.jme3.bullet.joints + SixDofJoint Point2PointJoint HingeJoint ConeJoint) + com.jme3.bullet.control.RigidBodyControl + com.jme3.collision.CollisionResults + com.jme3.bounding.BoundingBox + com.jme3.scene.Node + com.jme3.scene.Geometry + com.jme3.bullet.collision.shapes.HullCollisionShape)) +#+end_src -* Examples +* Source + +* COMMENT Examples #+name: test-body #+begin_src clojure @@ -967,7 +1133,7 @@ #+end_src #+begin_src clojure :tangle ../src/cortex/test/body.clj -<> +<> #+end_src diff -r 1c915cc1118b -r d5c597a7aed4 org/sense.org --- a/org/sense.org Tue Feb 07 07:13:45 2012 -0700 +++ b/org/sense.org Wed Feb 08 05:50:15 2012 -0700 @@ -1,4 +1,4 @@ -#+title: +#+title: Helper Functions / Motivations #+author: Robert McIntyre #+email: rlm@mit.edu #+description: sensory utilities @@ -461,7 +461,6 @@ - #+end_html With this, eyes are easy --- you just bind the camera closer to the