1.1 --- a/org/movement.org	Mon Feb 13 21:53:28 2012 -0600
     1.2 +++ b/org/movement.org	Mon Feb 13 21:53:54 2012 -0600
     1.3 @@ -1,4 +1,4 @@
     1.4 -#+title: Movement!
     1.5 +#+title: Simulated Muscles
     1.6  #+author: Robert McIntyre
     1.7  #+email: rlm@mit.edu
     1.8  #+description: muscles for a simulated creature
     1.9 @@ -7,48 +7,61 @@
    1.10  #+INCLUDE: ../../aurellem/org/level-0.org
    1.11  
    1.12  
    1.13 +* Muscles
    1.14 +
    1.15  Surprisingly enough, terristerial creatures only move by using torque
    1.16  applied about their joints.  There's not a single straight line of
    1.17  force in the human body at all!  (A straight line of force would
    1.18 -correspond to some sort of jet or rocket propulseion.)
    1.19 +correspond to some sort of jet or rocket propulsion.)
    1.20  
    1.21 +*(next paragraph is from memory and needs to be checked!)*
    1.22  
    1.23 -Here's how motor-control/ proprioception will work: Each muscle is
    1.24 -defined by a 1-D array of numbers (the "motor pool") each of which
    1.25 -represent muscle fibers. A muscle also has a scalar :strength factor
    1.26 -which determines how strong the muscle as a whole is.  The effector
    1.27 -function for a muscle takes a number < (count motor-pool) and that
    1.28 -number is said to "activate" all the muscle fibers whose index is
    1.29 -lower than the number.  Each fiber will apply force in proportion to
    1.30 -its value in the array.  Lower values cause less force.  The lower
    1.31 -values can be put at the "beginning" of the 1-D array to simulate the
    1.32 -layout of actual human muscles, which are capable of more percise
    1.33 -movements when exerting less force.
    1.34 +In humans, muscles are composed of millions of sarcomeres, which can
    1.35 +contract to exert force. A single motor neuron might control 100-1,000
    1.36 +sarcomeres. When the motor neuron is engaged by the brain, it
    1.37 +activates all of the sarcomeres to which it is attached. Some motor
    1.38 +neurons command many sarcomeres, and some command only a few. The
    1.39 +spinal cord generally engages the motor neurons which control few
    1.40 +sarcomeres before the motor neurons which control many sarcomeres.
    1.41 +This recruitment stragety allows for percise movements at low
    1.42 +strength. The collection of all motor neurons that control a muscle is
    1.43 +called the motor pool. The brain essentially says "activate 30% of the
    1.44 +motor pool" and the spinal cord recruits motor neurons untill 30% are
    1.45 +activated. Since the distribution of power among motor neurons is
    1.46 +unequal and recruitment goes from weakest to strongest, 30% of the
    1.47 +motor pool might be 5% of the strength of the muscle.
    1.48  
    1.49 -#+name: movement
    1.50 +My simulated muscles follow a similiar design: Each muscle is defined
    1.51 +by a 1-D array of numbers (the "motor pool"). Each number represents a
    1.52 +motor neuron which controlls a number of sarcomeres equal to the
    1.53 +number. A muscle also has a scalar :strength factor which determines
    1.54 +the total force the muscle can exert when all motor neurons are
    1.55 +activated.  The effector function for a muscle takes a number to index
    1.56 +into the motor pool, and that number "activates" all the motor neurons
    1.57 +whose index is lower or equal to the number.  Each motor-neuron will
    1.58 +apply force in proportion to its value in the array.  Lower values
    1.59 +cause less force.  The lower values can be put at the "beginning" of
    1.60 +the 1-D array to simulate the layout of actual human muscles, which
    1.61 +are capable of more percise movements when exerting less force. Or,
    1.62 +the motor pool can simulate more exoitic recruitment strageties which
    1.63 +do not correspond to human muscles. 
    1.64 +
    1.65 +This 1D array is defined in an image file for ease of
    1.66 +creation/visualization. Here is an example muscle profile image.
    1.67 +
    1.68 +#+caption: A muscle profile image that describes the strengths of each motor neuron in a muscle. White is weakest and dark red is strongest. This particular pattern has weaker motor neurons at the beginning, just like human muscle.
    1.69 +[[../images/basic-muscle.png]]
    1.70 +
    1.71 +* Blender Meta-data
    1.72 +
    1.73 +In blender, each muscle is an empty node whose top level parent is
    1.74 +named "muscles", just like eyes, ears, and joints.
    1.75 +
    1.76 +These functions define the expected meta-data for a muscle node.
    1.77 +
    1.78 +#+name: muscle-meta-data
    1.79  #+begin_src clojure 
    1.80 -(ns cortex.movement
    1.81 -  "Give simulated creatures defined in special blender files the power
    1.82 -   to move around in a simulated environment."
    1.83 -  {:author "Robert McIntyre"}
    1.84 -  (:use (cortex world util sense body))
    1.85 -  (:use clojure.contrib.def)
    1.86 -  (:import java.awt.image.BufferedImage)
    1.87 -  (:import com.jme3.scene.Node)
    1.88 -  (:import com.jme3.math.Vector3f)
    1.89 -  (:import com.jme3.bullet.control.RigidBodyControl))
    1.90 -
    1.91 -(defn muscle-profile
    1.92 -  "Return a vector where each entry is the strength of the \"motor
    1.93 -  pool\" at that part in the muscle."
    1.94 -  [#^BufferedImage profile]
    1.95 -  (vec
    1.96 -   (let [width (.getWidth profile)]
    1.97 -     (for [x (range width)]
    1.98 -       (- 255
    1.99 -          (bit-and
   1.100 -           0x0000FF
   1.101 -           (.getRGB profile x 0)))))))
   1.102 +(in-ns 'cortex.movement)
   1.103  
   1.104  (defvar 
   1.105    ^{:arglists '([creature])}
   1.106 @@ -56,30 +69,64 @@
   1.107    (sense-nodes "muscles")
   1.108    "Return the children of the creature's \"muscles\" node.")
   1.109  
   1.110 -(defn movement-fn
   1.111 +(defn muscle-profile-image
   1.112 +  "Get the muscle-profile image from the node's blender meta-data."
   1.113 +  [#^Node muscle]
   1.114 +  (if-let [image (meta-data muscle "muscle")]
   1.115 +    (load-image image)))
   1.116 +
   1.117 +(defn muscle-strength
   1.118 +  "Return the strength of this muscle, or 1 if it is not defined."
   1.119 +  [#^Node muscle]
   1.120 +  (if-let [strength (meta-data muscle "strength")]
   1.121 +    strength 1))
   1.122 +
   1.123 +(defn motor-pool
   1.124 +  "Return a vector where each entry is the strength of the \"motor
   1.125 +   neuron\" at that part in the muscle."
   1.126 +  [#^Node muscle]
   1.127 +  (let [profile (muscle-profile-image muscle)]
   1.128 +    (vec
   1.129 +     (let [width (.getWidth profile)]
   1.130 +       (for [x (range width)]
   1.131 +       (- 255
   1.132 +          (bit-and
   1.133 +           0x0000FF
   1.134 +           (.getRGB profile x 0))))))))
   1.135 +#+end_src
   1.136 +
   1.137 +Of note here is =(motor-pool)= which interprets the muscle-profile
   1.138 +image in a way that allows me to use gradients between white and red,
   1.139 +instead of shades of gray as I've been using for all the other
   1.140 +senses. This is purely an aesthetic touch.
   1.141 +
   1.142 +* Creating Muscles
   1.143 +#+name: muscle-kernel
   1.144 +#+begin_src clojure
   1.145 +(in-ns 'cortex.movement)
   1.146 +
   1.147 +(defn movement-kernel
   1.148    "Returns a function which when called with a integer value inside a
   1.149     running simulation will cause movement in the creature according
   1.150     to the muscle's position and strength profile. Each function
   1.151     returns the amount of force applied / max force."
   1.152 -  [#^Node parts #^Node muscle]
   1.153 -  (let [target (closest-node parts muscle)
   1.154 +  [#^Node creature #^Node muscle]
   1.155 +  (let [target (closest-node creature muscle)
   1.156          axis
   1.157          (.mult (.getWorldRotation muscle) Vector3f/UNIT_Y)
   1.158 -        strength (meta-data muscle "strength")
   1.159 -        image-name (read-string (meta-data muscle "muscle"))
   1.160 -        image (load-image image-name)
   1.161 -        fibers (muscle-profile image)
   1.162 -        fiber-integral (reductions + fibers)
   1.163 +        strength (muscle-strength muscle)
   1.164 +        
   1.165 +        pool (motor-pool muscle)
   1.166 +        pool-integral (reductions + pool)
   1.167          force-index
   1.168 -        (vec (map  #(float (* strength (/ % (last fiber-integral))))
   1.169 -              fiber-integral))
   1.170 +        (vec (map  #(float (* strength (/ % (last pool-integral))))
   1.171 +              pool-integral))
   1.172          control (.getControl target RigidBodyControl)]
   1.173      (fn [n]
   1.174 -      (let [pool-index (max 0 (min n (dec (count fibers))))
   1.175 +      (let [pool-index (max 0 (min n (dec (count pool))))
   1.176              force (force-index pool-index)]
   1.177          (.applyTorque control (.mult axis force))
   1.178          (float (/ force strength))))))
   1.179 -        
   1.180  
   1.181  (defn movement!
   1.182    "Endow the creature with the power of movement. Returns a sequence
   1.183 @@ -87,8 +134,24 @@
   1.184     activate their corresponding muscle."
   1.185    [#^Node creature]
   1.186      (for [muscle (muscles creature)]
   1.187 -      (movement-fn creature muscle)))
   1.188 +      (movement-kernel creature muscle)))
   1.189 +#+end_src
   1.190  
   1.191 +=(movement-kernel)= creates a function that will move the nearest
   1.192 +physical object to the muscle node. The muscle exerts a rotational
   1.193 +force dependant on it's orientation to the object in the blender
   1.194 +file. The function returned by =(movement-kernel)= is also a sense
   1.195 +function: it returns the percent of the total muscle strength that is
   1.196 +currently being employed. This is analogous to muscle tension in
   1.197 +humans and completes the sense of proprioception begun in the last
   1.198 +post. 
   1.199 +
   1.200 +* Visualizing Muscle Tension
   1.201 +Muscle exertion is a percent of a total, so the visulazation is just a
   1.202 +simple percent bar.
   1.203 +
   1.204 +#+name: visualization
   1.205 +#+begin_src clojure
   1.206  (defn movement-display-kernel
   1.207    "Display muscle exertion data as a bar filling up with red."
   1.208    [exertion]
   1.209 @@ -108,14 +171,166 @@
   1.210    displays each element of the list to the screen."
   1.211    []
   1.212    (view-sense movement-display-kernel))
   1.213 -
   1.214  #+end_src
   1.215  
   1.216 +* Adding Touch to the Worm
   1.217  
   1.218 +#+begin_src clojure
   1.219 +(defn test-movement
   1.220 +  ([] (test-movement false))
   1.221 +  ([record?]
   1.222 +     (let [creature (doto (worm) (body!))
   1.223  
   1.224 +           muscle-exertion (atom 0)
   1.225 +           muscles (movement! creature)
   1.226 +           muscle-display (view-movement)]
   1.227 +       (.setMass
   1.228 +        (.getControl (.getChild creature "worm-11") RigidBodyControl)
   1.229 +        (float 0))
   1.230 +       (world
   1.231 +        (nodify [creature (floor)])
   1.232 +        (merge standard-debug-controls
   1.233 +               {"key-h"
   1.234 +                (fn [_ value]
   1.235 +                  (if value
   1.236 +                    (swap!  muscle-exertion (partial + 20))))
   1.237 +                "key-n"
   1.238 +                (fn [_ value]
   1.239 +                  (if value
   1.240 +                    (swap! muscle-exertion (fn [v] (- v 20)))))})
   1.241 +        (fn [world]
   1.242 +          (if record?
   1.243 +            (Capture/captureVideo
   1.244 +             world
   1.245 +             (File. "/home/r/proj/cortex/render/worm-muscles/main-view")))
   1.246 +          (light-up-everything world)
   1.247 +          (enable-debug world)
   1.248 +          (.setTimer world (RatchetTimer. 60))
   1.249 +          (set-gravity world (Vector3f. 0 0 0))
   1.250 +          (.setLocation (.getCamera world)
   1.251 +                        (Vector3f. -4.912815, 2.004171, 0.15710819))
   1.252 +          (.setRotation (.getCamera world)
   1.253 +                        (Quaternion. 0.13828252, 0.65516764, 
   1.254 +                                     -0.12370994, 0.7323449))
   1.255  
   1.256 +          (comment 
   1.257 +            (com.aurellem.capture.Capture/captureVideo
   1.258 +             world (file-str "/home/r/proj/ai-videos/hand"))))
   1.259 +        (fn [world tpf]
   1.260 +          (muscle-display
   1.261 +           (map #(% @muscle-exertion) muscles)
   1.262 +           (if record?
   1.263 +             (File. "/home/r/proj/cortex/render/worm-muscles/muscles"))))))))
   1.264 +#+end_src
   1.265 +
   1.266 +* Video Demonstration 
   1.267 +
   1.268 +#+begin_html
   1.269 +<div class="figure">
   1.270 +<center>
   1.271 +<video controls="controls" width="550">
   1.272 +  <source src="../video/worm-muscles.ogg" type="video/ogg"
   1.273 +	  preload="none" poster="../images/aurellem-1280x480.png" />
   1.274 +</video>
   1.275 +</center>
   1.276 +<p>The worm is now able to move. The bar in the lower right displays
   1.277 +  the power output of the muscle . Each jump causes 20 more motor neurons to
   1.278 +  be recruited.  Notice that the power output increases non-linearly
   1.279 +  with motror neuron recruitement, similiar to a human muscle.</p>
   1.280 +</div>
   1.281 +#+end_html
   1.282 +
   1.283 +
   1.284 +** Making the Worm Muscles Video
   1.285 +#+name: magick7
   1.286 +#+begin_src clojure
   1.287 +(ns cortex.video.magick7
   1.288 +  (:import java.io.File)
   1.289 +  (:use clojure.contrib.shell-out))
   1.290 +
   1.291 +(defn images [path]
   1.292 +  (sort (rest (file-seq (File. path)))))
   1.293 +
   1.294 +(def base "/home/r/proj/cortex/render/worm-muscles/")
   1.295 +
   1.296 +(defn pics [file]
   1.297 +  (images (str base file)))
   1.298 +
   1.299 +(defn combine-images []
   1.300 +  (let [main-view (pics "main-view")
   1.301 +        muscles (pics "muscles/0")
   1.302 +        targets (map
   1.303 +                 #(File. (str base "out/" (format "%07d.png" %)))
   1.304 +                 (range 0 (count main-view)))]
   1.305 +    (dorun
   1.306 +     (pmap
   1.307 +      (comp
   1.308 +       (fn [[ main-view muscles target]]
   1.309 +         (println target)
   1.310 +         (sh "convert"
   1.311 +             main-view 
   1.312 +             muscles "-geometry" "+320+440" "-composite"
   1.313 +             target))
   1.314 +       (fn [& args] (map #(.getCanonicalPath %) args)))
   1.315 +       main-view muscles targets))))
   1.316 +#+end_src
   1.317 +
   1.318 +#+begin_src sh :results silent
   1.319 +cd ~/proj/cortex/render/worm-muscles
   1.320 +ffmpeg -r 60 -i out/%07d.png -b:v 9000k -c:v libtheora worm-muscles.ogg
   1.321 +#+end_src
   1.322 +
   1.323 +* Headers
   1.324 +#+name: muscle-header
   1.325 +#+begin_src clojure
   1.326 +(ns cortex.movement
   1.327 +  "Give simulated creatures defined in special blender files the power
   1.328 +   to move around in a simulated environment."
   1.329 +  {:author "Robert McIntyre"}
   1.330 +  (:use (cortex world util sense body))
   1.331 +  (:use clojure.contrib.def)
   1.332 +  (:import java.awt.image.BufferedImage)
   1.333 +  (:import com.jme3.scene.Node)
   1.334 +  (:import com.jme3.math.Vector3f)
   1.335 +  (:import com.jme3.bullet.control.RigidBodyControl))
   1.336 +#+end_src
   1.337 +
   1.338 +#+name: test-header
   1.339 +#+begin_src clojure
   1.340 +(ns cortex.test.movement
   1.341 +  (:use (cortex world util sense body movement))
   1.342 +  (:use cortex.test.body)
   1.343 +  (:use clojure.contrib.def)
   1.344 +  (:import java.io.File)
   1.345 +  (:import java.awt.image.BufferedImage)
   1.346 +  (:import com.jme3.scene.Node)
   1.347 +  (:import com.jme3.math.Vector3f)
   1.348 +  (:import (com.aurellem.capture Capture RatchetTimer))
   1.349 +  (:import com.jme3.bullet.control.RigidBodyControl))
   1.350 +
   1.351 +(cortex.import/mega-import-jme3)
   1.352 +#+end_src
   1.353 +
   1.354 +* Source Listing
   1.355 +  - [[../src/cortex/movement.clj][cortex.movement]]
   1.356 +  - [[../src/cortex/test/movement.clj][cortex.test.movement]]
   1.357 +  - [[../src/cortex/video/magick7.clj][cortex.video.magick7]]
   1.358 +#+html: <ul> <li> <a href="../org/movement.org">This org file</a> </li> </ul>
   1.359 +  - [[http://hg.bortreb.com ][source-repository]]
   1.360  
   1.361  * COMMENT code generation
   1.362  #+begin_src clojure :tangle ../src/cortex/movement.clj
   1.363 -<<movement>>
   1.364 +<<muscle-header>>
   1.365 +<<muscle-meta-data>>
   1.366 +<<muscle-kernel>>
   1.367 +<<visualization>>
   1.368  #+end_src
   1.369 +
   1.370 +#+begin_src clojure :tangle ../src/cortex/test/movement.clj
   1.371 +<<test-header>>
   1.372 +<<test-movement>>
   1.373 +#+end_src
   1.374 +
   1.375 +#+begin_src clojure :tangle ../src/cortex/video/magick7.clj
   1.376 +<<magick7>>
   1.377 +#+end_src