1.1 --- a/thesis/cortex.org	Fri Mar 28 11:08:32 2014 -0400
     1.2 +++ b/thesis/cortex.org	Fri Mar 28 13:16:37 2014 -0400
     1.3 @@ -21,7 +21,7 @@
     1.4    #+caption: 
     1.5    #+name: name
     1.6    #+ATTR_LaTeX: :width 10cm
     1.7 -  [[./images/Eve.jpg]]
     1.8 +  [[./images/aurellem-gray.png]]
     1.9  
    1.10  
    1.11  
    1.12 @@ -530,16 +530,18 @@
    1.13     While trying to find a good compromise for body-design, one option
    1.14     I ultimately rejected is to use blender's [[http://wiki.blender.org/index.php/Doc:2.6/Manual/Rigging/Armatures][armature]] system. The idea
    1.15     would have been to define a mesh which describes the creature's
    1.16 -   entire body. To this you add an skeleton which deforms this
    1.17 -   mesh. This technique is used extensively to model humans and create
    1.18 -   realistic animations. It is hard to use for my purposes because it
    1.19 -   is difficult to update the creature's Physics Collision Mesh in
    1.20 -   tandem with its Geometric Mesh under the influence of the
    1.21 -   armature. Without this the creature will not be able to grab things
    1.22 -   in its environment, and it won't be able to tell where its physical
    1.23 -   body is by using its eyes. Also, armatures do not specify any
    1.24 -   rotational limits for a joint, making it hard to model elbows,
    1.25 -   shoulders, etc.
    1.26 +   entire body. To this you add a skeleton which deforms this mesh
    1.27 +   (called rigging). This technique is used extensively to model
    1.28 +   humans and create realistic animations. It is not a good technique
    1.29 +   for physical simulation, because deformable surfaces are hard to
    1.30 +   model. Humans work like a squishy bag with some hard bones to give
    1.31 +   it shape. The bones are easy to simulate physically, but they
    1.32 +   interact with thr world though the skin, which is contiguous, but
    1.33 +   does not have a constant shape. In order to simulate skin you need
    1.34 +   some way to continuously update the physical model of the skin
    1.35 +   along with the movement of the bones. Given that bullet is
    1.36 +   optimized for rigid, solid objects, this leads to unmanagable
    1.37 +   computation and incorrect simulation. 
    1.38  
    1.39     Instead of using the human-like ``deformable bag of bones''
    1.40     approach, I decided to base my body plans on multiple solid objects
    1.41 @@ -586,14 +588,248 @@
    1.42     #+caption: node (highlighted in yellow) and its children which each
    1.43     #+caption: represent a joint in the hand. Each joint node has metadata
    1.44     #+caption: specifying what sort of joint it is.
    1.45 +   #+name: blender-hand
    1.46     #+ATTR_LaTeX: :width 10cm
    1.47     [[./images/hand-screenshot1.png]]
    1.48  
    1.49     
    1.50 +   =CORTEX= creates a creature in two steps: first, it traverses the
    1.51 +   nodes in the blender file and creates physical representations for
    1.52 +   any of them that have mass defined.
    1.53 +
    1.54 +   #+caption: Program for iterating through the nodes in a blender file
    1.55 +   #+caption: and generating physical jMonkeyEngine3 objects with mass
    1.56 +   #+caption: and a matching physics shape.
    1.57 +   #+name: name
    1.58 +   #+begin_listing clojure
    1.59 +   #+begin_src clojure
    1.60 +(defn physical!
    1.61 +  "Iterate through the nodes in creature and make them real physical
    1.62 +   objects in the simulation."
    1.63 +  [#^Node creature]
    1.64 +  (dorun
    1.65 +   (map
    1.66 +    (fn [geom]
    1.67 +      (let [physics-control
    1.68 +            (RigidBodyControl.
    1.69 +             (HullCollisionShape.
    1.70 +              (.getMesh geom))
    1.71 +             (if-let [mass (meta-data geom "mass")]
    1.72 +               (float mass) (float 1)))]
    1.73 +        (.addControl geom physics-control)))
    1.74 +    (filter #(isa? (class %) Geometry )
    1.75 +            (node-seq creature)))))
    1.76 +   #+end_src
    1.77 +   #+end_listing
    1.78     
    1.79 +   The next step to making a proper body is to connect those pieces
    1.80 +   together with joints. jMonkeyEngine has a large array of joints
    1.81 +   available via =bullet=, such as Point2Point, Cone, Hinge, and a
    1.82 +   generic Six Degree of Freedom joint, with or without spring
    1.83 +   restitution. =CORTEX='s procedure for binding the creature together
    1.84 +   with joints is as follows:
    1.85  
    1.86 +   - Find the children of the "joints" node.
    1.87 +   - Determine the two spatials the joint is meant to connect.
    1.88 +   - Create the joint based on the meta-data of the empty node.
    1.89 +
    1.90 +   The higher order function =sense-nodes= from =cortex.sense=
    1.91 +   simplifies finding the joints based on their parent ``joints''
    1.92 +   node.
    1.93 +
    1.94 +   #+caption: Retrieving the children empty nodes from a single 
    1.95 +   #+caption: named empty node is a common pattern in =CORTEX=
    1.96 +   #+caption: further instances of this technique for the senses 
    1.97 +   #+caption: will be omitted
    1.98 +   #+name: get-empty-nodes
    1.99 +   #+begin_listing clojure
   1.100 +   #+begin_src clojure
   1.101 +(defn sense-nodes
   1.102 +  "For some senses there is a special empty blender node whose
   1.103 +   children are considered markers for an instance of that sense. This
   1.104 +   function generates functions to find those children, given the name
   1.105 +   of the special parent node."
   1.106 +  [parent-name]
   1.107 +  (fn [#^Node creature]
   1.108 +    (if-let [sense-node (.getChild creature parent-name)]
   1.109 +      (seq (.getChildren sense-node)) [])))
   1.110 +
   1.111 +(def
   1.112 +  ^{:doc "Return the children of the creature's \"joints\" node."
   1.113 +    :arglists '([creature])}
   1.114 +  joints
   1.115 +  (sense-nodes "joints"))
   1.116 +   #+end_src
   1.117 +   #+end_listing
   1.118 +
   1.119 +   To find a joint's targets targets, =CORTEX= creates a small cube,
   1.120 +   centered around the empty-node, and grows the cube exponentially
   1.121 +   until it intersects two /physical/ objects. The objects are ordered
   1.122 +   according to the joint's rotation, with the first one being the
   1.123 +   object that has more negative coordinates in the joint's reference
   1.124 +   frame. Since the objects must be physical, the empty-node itself
   1.125 +   escapes detection. Because the objects must be physical,
   1.126 +   =joint-targets= must be called /after/ =physical!= is called.
   1.127     
   1.128 +   #+caption: Program to find the targets of a joint node by 
   1.129 +   #+caption: exponentiallly growth of a search cube.
   1.130 +   #+name: joint-targets
   1.131 +   #+begin_listing clojure
   1.132 +   #+begin_src clojure
   1.133 +(defn joint-targets
   1.134 +  "Return the two closest two objects to the joint object, ordered
   1.135 +  from bottom to top according to the joint's rotation."
   1.136 +  [#^Node parts #^Node joint]
   1.137 +  (loop [radius (float 0.01)]
   1.138 +    (let [results (CollisionResults.)]
   1.139 +      (.collideWith
   1.140 +       parts
   1.141 +       (BoundingBox. (.getWorldTranslation joint)
   1.142 +                     radius radius radius) results)
   1.143 +      (let [targets
   1.144 +            (distinct
   1.145 +             (map  #(.getGeometry %) results))]
   1.146 +        (if (>= (count targets) 2)
   1.147 +          (sort-by
   1.148 +           #(let [joint-ref-frame-position
   1.149 +                  (jme-to-blender
   1.150 +                   (.mult
   1.151 +                    (.inverse (.getWorldRotation joint))
   1.152 +                    (.subtract (.getWorldTranslation %)
   1.153 +                               (.getWorldTranslation joint))))]
   1.154 +              (.dot (Vector3f. 1 1 1) joint-ref-frame-position))                  
   1.155 +           (take 2 targets))
   1.156 +          (recur (float (* radius 2))))))))
   1.157 +   #+end_src
   1.158 +   #+end_listing
   1.159     
   1.160 +   Once =CORTEX= finds all joints and targets, it creates them using a
   1.161 +   simple dispatch on the metadata of the joint node. 
   1.162 +
   1.163 +   #+caption: Program to dispatch on blender metadata and create joints
   1.164 +   #+caption: sutiable for physical simulation.
   1.165 +   #+name: joint-dispatch
   1.166 +   #+begin_listing clojure
   1.167 +   #+begin_src clojure
   1.168 +(defmulti joint-dispatch
   1.169 +  "Translate blender pseudo-joints into real JME joints."
   1.170 +  (fn [constraints & _] 
   1.171 +    (:type constraints)))
   1.172 +
   1.173 +(defmethod joint-dispatch :point
   1.174 +  [constraints control-a control-b pivot-a pivot-b rotation]
   1.175 +  (doto (SixDofJoint. control-a control-b pivot-a pivot-b false)
   1.176 +    (.setLinearLowerLimit Vector3f/ZERO)
   1.177 +    (.setLinearUpperLimit Vector3f/ZERO)))
   1.178 +
   1.179 +(defmethod joint-dispatch :hinge
   1.180 +  [constraints control-a control-b pivot-a pivot-b rotation]
   1.181 +  (let [axis (if-let [axis (:axis constraints)] axis Vector3f/UNIT_X)
   1.182 +        [limit-1 limit-2] (:limit constraints)
   1.183 +        hinge-axis (.mult rotation (blender-to-jme axis))]
   1.184 +    (doto (HingeJoint. control-a control-b pivot-a pivot-b 
   1.185 +                       hinge-axis hinge-axis)
   1.186 +      (.setLimit limit-1 limit-2))))
   1.187 +
   1.188 +(defmethod joint-dispatch :cone
   1.189 +  [constraints control-a control-b pivot-a pivot-b rotation]
   1.190 +  (let [limit-xz (:limit-xz constraints)
   1.191 +        limit-xy (:limit-xy constraints)
   1.192 +        twist    (:twist constraints)]
   1.193 +    (doto (ConeJoint. control-a control-b pivot-a pivot-b
   1.194 +                      rotation rotation)
   1.195 +      (.setLimit (float limit-xz) (float limit-xy)
   1.196 +                 (float twist)))))
   1.197 +   #+end_src
   1.198 +   #+end_listing
   1.199 +
   1.200 +   All that is left for joints it to combine the above pieces into a
   1.201 +   something that can operate on the collection of nodes that a
   1.202 +   blender file represents.
   1.203 +
   1.204 +   #+caption: Program to completely create a joint given information 
   1.205 +   #+caption: from a blender file.
   1.206 +   #+name: connect
   1.207 +   #+begin_listing clojure
   1.208 +   #+begin_src clojure
   1.209 +(defn connect
   1.210 +  "Create a joint between 'obj-a and 'obj-b at the location of
   1.211 +  'joint. The type of joint is determined by the metadata on 'joint.
   1.212 +
   1.213 +   Here are some examples:
   1.214 +   {:type :point}
   1.215 +   {:type :hinge  :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}
   1.216 +   (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
   1.217 +
   1.218 +   {:type :cone :limit-xz 0]
   1.219 +                :limit-xy 0]
   1.220 +                :twist 0]}   (use XZY rotation mode in blender!)"
   1.221 +  [#^Node obj-a #^Node obj-b #^Node joint]
   1.222 +  (let [control-a (.getControl obj-a RigidBodyControl)
   1.223 +        control-b (.getControl obj-b RigidBodyControl)
   1.224 +        joint-center (.getWorldTranslation joint)
   1.225 +        joint-rotation (.toRotationMatrix (.getWorldRotation joint))
   1.226 +        pivot-a (world-to-local obj-a joint-center)
   1.227 +        pivot-b (world-to-local obj-b joint-center)]
   1.228 +    (if-let
   1.229 +        [constraints (map-vals eval (read-string (meta-data joint "joint")))]
   1.230 +      ;; A side-effect of creating a joint registers
   1.231 +      ;; it with both physics objects which in turn
   1.232 +      ;; will register the joint with the physics system
   1.233 +      ;; when the simulation is started.
   1.234 +        (joint-dispatch constraints
   1.235 +                        control-a control-b
   1.236 +                        pivot-a pivot-b
   1.237 +                        joint-rotation))))
   1.238 +   #+end_src
   1.239 +   #+end_listing
   1.240 +
   1.241 +   In general, whenever =CORTEX= exposes a sense (or in this case
   1.242 +   physicality), it provides a function of the type =sense!=, which
   1.243 +   takes in a collection of nodes and augments it to support that
   1.244 +   sense. The function returns any controlls necessary to use that
   1.245 +   sense. In this case =body!= cerates a physical body and returns no
   1.246 +   control functions.
   1.247 +
   1.248 +   #+caption: Program to give joints to a creature.
   1.249 +   #+name: name
   1.250 +   #+begin_listing clojure
   1.251 +   #+begin_src clojure
   1.252 +(defn joints!
   1.253 +  "Connect the solid parts of the creature with physical joints. The
   1.254 +   joints are taken from the \"joints\" node in the creature."
   1.255 +  [#^Node creature]
   1.256 +  (dorun
   1.257 +   (map
   1.258 +    (fn [joint]
   1.259 +      (let [[obj-a obj-b] (joint-targets creature joint)]
   1.260 +        (connect obj-a obj-b joint)))
   1.261 +    (joints creature))))
   1.262 +(defn body!
   1.263 +  "Endow the creature with a physical body connected with joints.  The
   1.264 +   particulars of the joints and the masses of each body part are
   1.265 +   determined in blender."
   1.266 +  [#^Node creature]
   1.267 +  (physical! creature)
   1.268 +  (joints! creature))
   1.269 +   #+end_src
   1.270 +   #+end_listing
   1.271 +
   1.272 +   All of the code you have just seen amounts to only 130 lines, yet
   1.273 +   because it builds on top of Blender and jMonkeyEngine3, those few
   1.274 +   lines pack quite a punch!
   1.275 +
   1.276 +   The hand from figure \ref{blender-hand}, which was modeled after my
   1.277 +   own right hand, can now be given joints and simulated as a
   1.278 +   creature.
   1.279 +   
   1.280 +   #+caption: With the ability to create physical creatures from blender,
   1.281 +   #+caption: =CORTEX= gets one step closer to a full creature simulation
   1.282 +   #+caption: environment.
   1.283 +   #+name: name
   1.284 +   #+ATTR_LaTeX: :width 15cm
   1.285 +   [[./images/physical-hand.png]]
   1.286 +
   1.287  
   1.288  ** Eyes reuse standard video game components
   1.289