Mercurial > cortex
changeset 266:bee5145ce463
Merged Robert's proprioception with Dylan's vision.
author | Dylan Holmes <ocsenave@gmail.com> |
---|---|
date | Mon, 13 Feb 2012 21:53:54 -0600 |
parents | e57d8c52f12f (current diff) 2fdcbe8185b1 (diff) |
children | 6446e964810f |
files | |
diffstat | 10 files changed, 642 insertions(+), 192 deletions(-) [+] |
line wrap: on
line diff
1.1 Binary file assets/Models/test-creature/basic-muscle.png has changed
2.1 Binary file assets/Models/test-creature/worm.blend has changed
3.1 Binary file assets/Models/test-touch/touch-cube.blend has changed
4.1 Binary file images/basic-muscle.png has changed
5.1 Binary file images/worm-with-muscle.png has changed
6.1 --- a/org/hearing.org Mon Feb 13 21:53:28 2012 -0600 6.2 +++ b/org/hearing.org Mon Feb 13 21:53:54 2012 -0600 6.3 @@ -950,7 +950,6 @@ 6.4 #+end_src 6.5 6.6 * Testing Hearing 6.7 - 6.8 ** Advanced Java Example 6.9 6.10 I wrote a test case in Java that demonstrates the use of the Java 6.11 @@ -1083,7 +1082,6 @@ 6.12 sound it hears beomes fainter. This shows the 3D localization of 6.13 sound in this world.</p> 6.14 </div> 6.15 - 6.16 #+end_html 6.17 6.18 *** Creating the Ear Video 6.19 @@ -1169,7 +1167,6 @@ 6.20 #+html: <ul> <li> <a href="../org/hearing.org">This org file</a> </li> </ul> 6.21 - [[http://hg.bortreb.com ][source-repository]] 6.22 6.23 - 6.24 * Next 6.25 The worm can see and hear, but it can't feel the world or 6.26 itself. Next post, I'll give the worm a [[./touch.org][sense of touch]].
7.1 --- a/org/ideas.org Mon Feb 13 21:53:28 2012 -0600 7.2 +++ b/org/ideas.org Mon Feb 13 21:53:54 2012 -0600 7.3 @@ -100,6 +100,7 @@ 7.4 - [ ] show sensor maps in HUD display? -- 4 days 7.5 - [ ] show sensor maps in AWT display? -- 2 days 7.6 - [ ] upgrade to clojure 1.3, replace all defvars with new def 7.7 + - [ ] common video creation code. 7.8 7.9 7.10
8.1 --- a/org/movement.org Mon Feb 13 21:53:28 2012 -0600 8.2 +++ b/org/movement.org Mon Feb 13 21:53:54 2012 -0600 8.3 @@ -1,4 +1,4 @@ 8.4 -#+title: Movement! 8.5 +#+title: Simulated Muscles 8.6 #+author: Robert McIntyre 8.7 #+email: rlm@mit.edu 8.8 #+description: muscles for a simulated creature 8.9 @@ -7,48 +7,61 @@ 8.10 #+INCLUDE: ../../aurellem/org/level-0.org 8.11 8.12 8.13 +* Muscles 8.14 + 8.15 Surprisingly enough, terristerial creatures only move by using torque 8.16 applied about their joints. There's not a single straight line of 8.17 force in the human body at all! (A straight line of force would 8.18 -correspond to some sort of jet or rocket propulseion.) 8.19 +correspond to some sort of jet or rocket propulsion.) 8.20 8.21 +*(next paragraph is from memory and needs to be checked!)* 8.22 8.23 -Here's how motor-control/ proprioception will work: Each muscle is 8.24 -defined by a 1-D array of numbers (the "motor pool") each of which 8.25 -represent muscle fibers. A muscle also has a scalar :strength factor 8.26 -which determines how strong the muscle as a whole is. The effector 8.27 -function for a muscle takes a number < (count motor-pool) and that 8.28 -number is said to "activate" all the muscle fibers whose index is 8.29 -lower than the number. Each fiber will apply force in proportion to 8.30 -its value in the array. Lower values cause less force. The lower 8.31 -values can be put at the "beginning" of the 1-D array to simulate the 8.32 -layout of actual human muscles, which are capable of more percise 8.33 -movements when exerting less force. 8.34 +In humans, muscles are composed of millions of sarcomeres, which can 8.35 +contract to exert force. A single motor neuron might control 100-1,000 8.36 +sarcomeres. When the motor neuron is engaged by the brain, it 8.37 +activates all of the sarcomeres to which it is attached. Some motor 8.38 +neurons command many sarcomeres, and some command only a few. The 8.39 +spinal cord generally engages the motor neurons which control few 8.40 +sarcomeres before the motor neurons which control many sarcomeres. 8.41 +This recruitment stragety allows for percise movements at low 8.42 +strength. The collection of all motor neurons that control a muscle is 8.43 +called the motor pool. The brain essentially says "activate 30% of the 8.44 +motor pool" and the spinal cord recruits motor neurons untill 30% are 8.45 +activated. Since the distribution of power among motor neurons is 8.46 +unequal and recruitment goes from weakest to strongest, 30% of the 8.47 +motor pool might be 5% of the strength of the muscle. 8.48 8.49 -#+name: movement 8.50 +My simulated muscles follow a similiar design: Each muscle is defined 8.51 +by a 1-D array of numbers (the "motor pool"). Each number represents a 8.52 +motor neuron which controlls a number of sarcomeres equal to the 8.53 +number. A muscle also has a scalar :strength factor which determines 8.54 +the total force the muscle can exert when all motor neurons are 8.55 +activated. The effector function for a muscle takes a number to index 8.56 +into the motor pool, and that number "activates" all the motor neurons 8.57 +whose index is lower or equal to the number. Each motor-neuron will 8.58 +apply force in proportion to its value in the array. Lower values 8.59 +cause less force. The lower values can be put at the "beginning" of 8.60 +the 1-D array to simulate the layout of actual human muscles, which 8.61 +are capable of more percise movements when exerting less force. Or, 8.62 +the motor pool can simulate more exoitic recruitment strageties which 8.63 +do not correspond to human muscles. 8.64 + 8.65 +This 1D array is defined in an image file for ease of 8.66 +creation/visualization. Here is an example muscle profile image. 8.67 + 8.68 +#+caption: A muscle profile image that describes the strengths of each motor neuron in a muscle. White is weakest and dark red is strongest. This particular pattern has weaker motor neurons at the beginning, just like human muscle. 8.69 +[[../images/basic-muscle.png]] 8.70 + 8.71 +* Blender Meta-data 8.72 + 8.73 +In blender, each muscle is an empty node whose top level parent is 8.74 +named "muscles", just like eyes, ears, and joints. 8.75 + 8.76 +These functions define the expected meta-data for a muscle node. 8.77 + 8.78 +#+name: muscle-meta-data 8.79 #+begin_src clojure 8.80 -(ns cortex.movement 8.81 - "Give simulated creatures defined in special blender files the power 8.82 - to move around in a simulated environment." 8.83 - {:author "Robert McIntyre"} 8.84 - (:use (cortex world util sense body)) 8.85 - (:use clojure.contrib.def) 8.86 - (:import java.awt.image.BufferedImage) 8.87 - (:import com.jme3.scene.Node) 8.88 - (:import com.jme3.math.Vector3f) 8.89 - (:import com.jme3.bullet.control.RigidBodyControl)) 8.90 - 8.91 -(defn muscle-profile 8.92 - "Return a vector where each entry is the strength of the \"motor 8.93 - pool\" at that part in the muscle." 8.94 - [#^BufferedImage profile] 8.95 - (vec 8.96 - (let [width (.getWidth profile)] 8.97 - (for [x (range width)] 8.98 - (- 255 8.99 - (bit-and 8.100 - 0x0000FF 8.101 - (.getRGB profile x 0))))))) 8.102 +(in-ns 'cortex.movement) 8.103 8.104 (defvar 8.105 ^{:arglists '([creature])} 8.106 @@ -56,30 +69,64 @@ 8.107 (sense-nodes "muscles") 8.108 "Return the children of the creature's \"muscles\" node.") 8.109 8.110 -(defn movement-fn 8.111 +(defn muscle-profile-image 8.112 + "Get the muscle-profile image from the node's blender meta-data." 8.113 + [#^Node muscle] 8.114 + (if-let [image (meta-data muscle "muscle")] 8.115 + (load-image image))) 8.116 + 8.117 +(defn muscle-strength 8.118 + "Return the strength of this muscle, or 1 if it is not defined." 8.119 + [#^Node muscle] 8.120 + (if-let [strength (meta-data muscle "strength")] 8.121 + strength 1)) 8.122 + 8.123 +(defn motor-pool 8.124 + "Return a vector where each entry is the strength of the \"motor 8.125 + neuron\" at that part in the muscle." 8.126 + [#^Node muscle] 8.127 + (let [profile (muscle-profile-image muscle)] 8.128 + (vec 8.129 + (let [width (.getWidth profile)] 8.130 + (for [x (range width)] 8.131 + (- 255 8.132 + (bit-and 8.133 + 0x0000FF 8.134 + (.getRGB profile x 0)))))))) 8.135 +#+end_src 8.136 + 8.137 +Of note here is =(motor-pool)= which interprets the muscle-profile 8.138 +image in a way that allows me to use gradients between white and red, 8.139 +instead of shades of gray as I've been using for all the other 8.140 +senses. This is purely an aesthetic touch. 8.141 + 8.142 +* Creating Muscles 8.143 +#+name: muscle-kernel 8.144 +#+begin_src clojure 8.145 +(in-ns 'cortex.movement) 8.146 + 8.147 +(defn movement-kernel 8.148 "Returns a function which when called with a integer value inside a 8.149 running simulation will cause movement in the creature according 8.150 to the muscle's position and strength profile. Each function 8.151 returns the amount of force applied / max force." 8.152 - [#^Node parts #^Node muscle] 8.153 - (let [target (closest-node parts muscle) 8.154 + [#^Node creature #^Node muscle] 8.155 + (let [target (closest-node creature muscle) 8.156 axis 8.157 (.mult (.getWorldRotation muscle) Vector3f/UNIT_Y) 8.158 - strength (meta-data muscle "strength") 8.159 - image-name (read-string (meta-data muscle "muscle")) 8.160 - image (load-image image-name) 8.161 - fibers (muscle-profile image) 8.162 - fiber-integral (reductions + fibers) 8.163 + strength (muscle-strength muscle) 8.164 + 8.165 + pool (motor-pool muscle) 8.166 + pool-integral (reductions + pool) 8.167 force-index 8.168 - (vec (map #(float (* strength (/ % (last fiber-integral)))) 8.169 - fiber-integral)) 8.170 + (vec (map #(float (* strength (/ % (last pool-integral)))) 8.171 + pool-integral)) 8.172 control (.getControl target RigidBodyControl)] 8.173 (fn [n] 8.174 - (let [pool-index (max 0 (min n (dec (count fibers)))) 8.175 + (let [pool-index (max 0 (min n (dec (count pool)))) 8.176 force (force-index pool-index)] 8.177 (.applyTorque control (.mult axis force)) 8.178 (float (/ force strength)))))) 8.179 - 8.180 8.181 (defn movement! 8.182 "Endow the creature with the power of movement. Returns a sequence 8.183 @@ -87,8 +134,24 @@ 8.184 activate their corresponding muscle." 8.185 [#^Node creature] 8.186 (for [muscle (muscles creature)] 8.187 - (movement-fn creature muscle))) 8.188 + (movement-kernel creature muscle))) 8.189 +#+end_src 8.190 8.191 +=(movement-kernel)= creates a function that will move the nearest 8.192 +physical object to the muscle node. The muscle exerts a rotational 8.193 +force dependant on it's orientation to the object in the blender 8.194 +file. The function returned by =(movement-kernel)= is also a sense 8.195 +function: it returns the percent of the total muscle strength that is 8.196 +currently being employed. This is analogous to muscle tension in 8.197 +humans and completes the sense of proprioception begun in the last 8.198 +post. 8.199 + 8.200 +* Visualizing Muscle Tension 8.201 +Muscle exertion is a percent of a total, so the visulazation is just a 8.202 +simple percent bar. 8.203 + 8.204 +#+name: visualization 8.205 +#+begin_src clojure 8.206 (defn movement-display-kernel 8.207 "Display muscle exertion data as a bar filling up with red." 8.208 [exertion] 8.209 @@ -108,14 +171,166 @@ 8.210 displays each element of the list to the screen." 8.211 [] 8.212 (view-sense movement-display-kernel)) 8.213 - 8.214 #+end_src 8.215 8.216 +* Adding Touch to the Worm 8.217 8.218 +#+begin_src clojure 8.219 +(defn test-movement 8.220 + ([] (test-movement false)) 8.221 + ([record?] 8.222 + (let [creature (doto (worm) (body!)) 8.223 8.224 + muscle-exertion (atom 0) 8.225 + muscles (movement! creature) 8.226 + muscle-display (view-movement)] 8.227 + (.setMass 8.228 + (.getControl (.getChild creature "worm-11") RigidBodyControl) 8.229 + (float 0)) 8.230 + (world 8.231 + (nodify [creature (floor)]) 8.232 + (merge standard-debug-controls 8.233 + {"key-h" 8.234 + (fn [_ value] 8.235 + (if value 8.236 + (swap! muscle-exertion (partial + 20)))) 8.237 + "key-n" 8.238 + (fn [_ value] 8.239 + (if value 8.240 + (swap! muscle-exertion (fn [v] (- v 20)))))}) 8.241 + (fn [world] 8.242 + (if record? 8.243 + (Capture/captureVideo 8.244 + world 8.245 + (File. "/home/r/proj/cortex/render/worm-muscles/main-view"))) 8.246 + (light-up-everything world) 8.247 + (enable-debug world) 8.248 + (.setTimer world (RatchetTimer. 60)) 8.249 + (set-gravity world (Vector3f. 0 0 0)) 8.250 + (.setLocation (.getCamera world) 8.251 + (Vector3f. -4.912815, 2.004171, 0.15710819)) 8.252 + (.setRotation (.getCamera world) 8.253 + (Quaternion. 0.13828252, 0.65516764, 8.254 + -0.12370994, 0.7323449)) 8.255 8.256 + (comment 8.257 + (com.aurellem.capture.Capture/captureVideo 8.258 + world (file-str "/home/r/proj/ai-videos/hand")))) 8.259 + (fn [world tpf] 8.260 + (muscle-display 8.261 + (map #(% @muscle-exertion) muscles) 8.262 + (if record? 8.263 + (File. "/home/r/proj/cortex/render/worm-muscles/muscles")))))))) 8.264 +#+end_src 8.265 + 8.266 +* Video Demonstration 8.267 + 8.268 +#+begin_html 8.269 +<div class="figure"> 8.270 +<center> 8.271 +<video controls="controls" width="550"> 8.272 + <source src="../video/worm-muscles.ogg" type="video/ogg" 8.273 + preload="none" poster="../images/aurellem-1280x480.png" /> 8.274 +</video> 8.275 +</center> 8.276 +<p>The worm is now able to move. The bar in the lower right displays 8.277 + the power output of the muscle . Each jump causes 20 more motor neurons to 8.278 + be recruited. Notice that the power output increases non-linearly 8.279 + with motror neuron recruitement, similiar to a human muscle.</p> 8.280 +</div> 8.281 +#+end_html 8.282 + 8.283 + 8.284 +** Making the Worm Muscles Video 8.285 +#+name: magick7 8.286 +#+begin_src clojure 8.287 +(ns cortex.video.magick7 8.288 + (:import java.io.File) 8.289 + (:use clojure.contrib.shell-out)) 8.290 + 8.291 +(defn images [path] 8.292 + (sort (rest (file-seq (File. path))))) 8.293 + 8.294 +(def base "/home/r/proj/cortex/render/worm-muscles/") 8.295 + 8.296 +(defn pics [file] 8.297 + (images (str base file))) 8.298 + 8.299 +(defn combine-images [] 8.300 + (let [main-view (pics "main-view") 8.301 + muscles (pics "muscles/0") 8.302 + targets (map 8.303 + #(File. (str base "out/" (format "%07d.png" %))) 8.304 + (range 0 (count main-view)))] 8.305 + (dorun 8.306 + (pmap 8.307 + (comp 8.308 + (fn [[ main-view muscles target]] 8.309 + (println target) 8.310 + (sh "convert" 8.311 + main-view 8.312 + muscles "-geometry" "+320+440" "-composite" 8.313 + target)) 8.314 + (fn [& args] (map #(.getCanonicalPath %) args))) 8.315 + main-view muscles targets)))) 8.316 +#+end_src 8.317 + 8.318 +#+begin_src sh :results silent 8.319 +cd ~/proj/cortex/render/worm-muscles 8.320 +ffmpeg -r 60 -i out/%07d.png -b:v 9000k -c:v libtheora worm-muscles.ogg 8.321 +#+end_src 8.322 + 8.323 +* Headers 8.324 +#+name: muscle-header 8.325 +#+begin_src clojure 8.326 +(ns cortex.movement 8.327 + "Give simulated creatures defined in special blender files the power 8.328 + to move around in a simulated environment." 8.329 + {:author "Robert McIntyre"} 8.330 + (:use (cortex world util sense body)) 8.331 + (:use clojure.contrib.def) 8.332 + (:import java.awt.image.BufferedImage) 8.333 + (:import com.jme3.scene.Node) 8.334 + (:import com.jme3.math.Vector3f) 8.335 + (:import com.jme3.bullet.control.RigidBodyControl)) 8.336 +#+end_src 8.337 + 8.338 +#+name: test-header 8.339 +#+begin_src clojure 8.340 +(ns cortex.test.movement 8.341 + (:use (cortex world util sense body movement)) 8.342 + (:use cortex.test.body) 8.343 + (:use clojure.contrib.def) 8.344 + (:import java.io.File) 8.345 + (:import java.awt.image.BufferedImage) 8.346 + (:import com.jme3.scene.Node) 8.347 + (:import com.jme3.math.Vector3f) 8.348 + (:import (com.aurellem.capture Capture RatchetTimer)) 8.349 + (:import com.jme3.bullet.control.RigidBodyControl)) 8.350 + 8.351 +(cortex.import/mega-import-jme3) 8.352 +#+end_src 8.353 + 8.354 +* Source Listing 8.355 + - [[../src/cortex/movement.clj][cortex.movement]] 8.356 + - [[../src/cortex/test/movement.clj][cortex.test.movement]] 8.357 + - [[../src/cortex/video/magick7.clj][cortex.video.magick7]] 8.358 +#+html: <ul> <li> <a href="../org/movement.org">This org file</a> </li> </ul> 8.359 + - [[http://hg.bortreb.com ][source-repository]] 8.360 8.361 * COMMENT code generation 8.362 #+begin_src clojure :tangle ../src/cortex/movement.clj 8.363 -<<movement>> 8.364 +<<muscle-header>> 8.365 +<<muscle-meta-data>> 8.366 +<<muscle-kernel>> 8.367 +<<visualization>> 8.368 #+end_src 8.369 + 8.370 +#+begin_src clojure :tangle ../src/cortex/test/movement.clj 8.371 +<<test-header>> 8.372 +<<test-movement>> 8.373 +#+end_src 8.374 + 8.375 +#+begin_src clojure :tangle ../src/cortex/video/magick7.clj 8.376 +<<magick7>> 8.377 +#+end_src
9.1 --- a/org/proprioception.org Mon Feb 13 21:53:28 2012 -0600 9.2 +++ b/org/proprioception.org Mon Feb 13 21:53:54 2012 -0600 9.3 @@ -6,36 +6,89 @@ 9.4 #+SETUPFILE: ../../aurellem/org/setup.org 9.5 #+INCLUDE: ../../aurellem/org/level-0.org 9.6 9.7 -#+name: proprioception 9.8 +* Proprioception 9.9 + 9.10 +Close your eyes, and touch your nose with your right index finger. How 9.11 +did you do it? You could not see your hand, and neither your hand nor 9.12 +your nose could use the sense of touch to guide the path of your hand. 9.13 +There are no sound cues, and Taste and Smell certainly don't provide 9.14 +any help. You know where your hand is without your other senses 9.15 +because of Proprioception. 9.16 + 9.17 +Humans can sometimes loose this sense through viral infections or 9.18 +damage to the spinal cord or brain, and when they do, they loose the 9.19 +ability to control their own bodies without looking directly at the 9.20 +parts they want to move. In [[http://en.wikipedia.org/wiki/The_Man_Who_Mistook_His_Wife_for_a_Hat][The Man Who Mistook His Wife for a Hat]], 9.21 +a woman named Christina looses this sense and has to learn how to move 9.22 +by carefully watching her arms and legs. She describes proprioception 9.23 +as the "eyes of the body, the way the body sees itself". 9.24 + 9.25 +Proprioception in humans is mediated by [[http://en.wikipedia.org/wiki/Articular_capsule][joint capsules]], [[http://en.wikipedia.org/wiki/Muscle_spindle][muscle 9.26 +spindles]], and the [[http://en.wikipedia.org/wiki/Golgi_tendon_organ][Golgi tendon organs]]. These measure the relative 9.27 +positions of each pody part by monitoring muscle strain and length. 9.28 + 9.29 +It's clear that this is a vital sense for fulid, graceful 9.30 +movement. It's also particurally easy to implement in jMonkeyEngine. 9.31 + 9.32 +My simulated proprioception calculates the relative angles of each 9.33 +joint from the rest position defined in the blender file. This 9.34 +simulates the muscle-spindles and joint capsules. I will deal with 9.35 +Golgi tendon organs, which calculate muscle strain, in the [[./movement.org][next post]]. 9.36 + 9.37 +* Helper Functions 9.38 + 9.39 +=(absolute-angle)= calculates the angle between two vectors, relative to a 9.40 +third axis vector. This angle is the number of radians you have to 9.41 +move counterclockwise around the axis vector to get from the first to 9.42 +the second vector. It is not commutative like a normal dot-product 9.43 +angle is. 9.44 + 9.45 +#+name: helpers 9.46 #+begin_src clojure 9.47 -(ns cortex.proprioception 9.48 - "Simulate the sense of proprioception (ability to detect the 9.49 - relative positions of body parts with repsect to other body parts) 9.50 - in jMonkeyEngine3. Reads specially prepared blender files to 9.51 - automatically generate proprioceptive senses." 9.52 - (:use (cortex world util sense body)) 9.53 - (:use clojure.contrib.def) 9.54 - (:import com.jme3.scene.Node) 9.55 - (:import java.awt.image.BufferedImage) 9.56 - (:import (com.jme3.math Vector3f Quaternion))) 9.57 +(in-ns 'cortex.proprioception) 9.58 9.59 (defn right-handed? 9.60 "true iff the three vectors form a right handed coordinate 9.61 - system. The three vectors do not have to be normalized or 9.62 - orthogonal." 9.63 + system. The three vectors do not have to be normalized or 9.64 + orthogonal." 9.65 [vec1 vec2 vec3] 9.66 (< 0 (.dot (.cross vec1 vec2) vec3))) 9.67 9.68 (defn absolute-angle 9.69 - "The angle between 'vec1 and 'vec2. Positive if the angle to get 9.70 - from 'vec1 to 'vec2 is counterclockwise around 'axis, and negative 9.71 - otherwise." 9.72 + "The angle between 'vec1 and 'vec2 around 'axis. In the range 9.73 + [0 (* 2 Math/PI)]." 9.74 [vec1 vec2 axis] 9.75 (let [angle (.angleBetween vec1 vec2)] 9.76 (if (right-handed? vec1 vec2 axis) 9.77 angle (- (* 2 Math/PI) angle)))) 9.78 +#+end_src 9.79 9.80 -(defn proprioception-fn 9.81 +#+begin_src clojure :exports both 9.82 +(in-ns 'cortex.proprioception) 9.83 +(absolute-angle Vector3f/UNIT_X Vector3f/UNIT_Y Vector3f/UNIT_Z) 9.84 +#+end_src 9.85 + 9.86 +#+results: 9.87 +: 1.5707964 9.88 + 9.89 +#+begin_src clojure :exports both 9.90 +(in-ns 'cortex.proprioception) 9.91 +(absolute-angle 9.92 + Vector3f/UNIT_X (.mult Vector3f/UNIT_Y (float -1)) Vector3f/UNIT_Z) 9.93 +#+end_src 9.94 + 9.95 +#+results: 9.96 +: 4.7123889366733 9.97 + 9.98 +* Proprioception Kernel 9.99 + 9.100 +Given a joint, =(proprioception-kernel)= produces a function that 9.101 +calculates the euler angles between the the objects the joint 9.102 +connects. 9.103 + 9.104 +#+name: proprioception 9.105 +#+begin_src clojure 9.106 +(defn proprioception-kernel 9.107 "Returns a function which returns proprioceptive sensory data when 9.108 called inside a running simulation." 9.109 [#^Node parts #^Node joint] 9.110 @@ -44,12 +97,6 @@ 9.111 x0 (.mult joint-rot Vector3f/UNIT_X) 9.112 y0 (.mult joint-rot Vector3f/UNIT_Y) 9.113 z0 (.mult joint-rot Vector3f/UNIT_Z)] 9.114 - (println-repl "x:" x0) 9.115 - (println-repl "y:" y0) 9.116 - (println-repl "z:" z0) 9.117 - (println-repl "init-a:" (.getWorldRotation obj-a)) 9.118 - (println-repl "init-b:" (.getWorldRotation obj-b)) 9.119 - 9.120 (fn [] 9.121 (let [rot-a (.clone (.getWorldRotation obj-a)) 9.122 rot-b (.clone (.getWorldRotation obj-b)) 9.123 @@ -80,18 +127,33 @@ 9.124 that joint." 9.125 [#^Node creature] 9.126 ;; extract the body's joints 9.127 - (let [senses (map (partial proprioception-fn creature) 9.128 + (let [senses (map (partial proprioception-kernel creature) 9.129 (joints creature))] 9.130 (fn [] 9.131 (map #(%) senses)))) 9.132 +#+end_src 9.133 9.134 9.135 +=(proprioception!)= maps =(proprioception-kernel)= across all the 9.136 +joints of the creature. It uses the same list of joints that 9.137 +=(cortex.body/joints)= uses. 9.138 + 9.139 +* Visualizing Proprioception 9.140 + 9.141 +Proprioception has the lowest bandwidth of all the senses so far, and 9.142 +it doesn't lend itself as readily to visual representation like 9.143 +vision, hearing, or touch. This visualization code creates a "guage" 9.144 +to view each of the three relative angles along a circle. 9.145 + 9.146 +#+name: visualize 9.147 +#+begin_src clojure 9.148 +(in-ns 'cortex.proprioception) 9.149 + 9.150 (defn draw-sprite [image sprite x y color ] 9.151 (dorun 9.152 (for [[u v] sprite] 9.153 (.setRGB image (+ u x) (+ v y) color)))) 9.154 9.155 - 9.156 (defn view-angle 9.157 "create a debug view of an angle" 9.158 [color] 9.159 @@ -109,7 +171,6 @@ 9.160 (reset! previous position)) 9.161 image)))) 9.162 9.163 - 9.164 (defn proprioception-display-kernel 9.165 "Display proprioception angles in a BufferedImage" 9.166 [[h p r]] 9.167 @@ -143,14 +204,16 @@ 9.168 display each element of the list to the screen as an image." 9.169 [] 9.170 (view-sense proprioception-display-kernel)) 9.171 - 9.172 - 9.173 - 9.174 #+end_src 9.175 9.176 -#+name: test-body 9.177 +* Proprioception Test 9.178 +This test does not use the worm, but instead uses two bars, bound 9.179 +together by a point2point joint. One bar is fixed, and I control the 9.180 +other bar from the keyboard. 9.181 + 9.182 +#+name: test-proprioception 9.183 #+begin_src clojure 9.184 - 9.185 +(in-ns 'cortex.test.proprioception) 9.186 9.187 (defn test-proprioception 9.188 "Testing proprioception: 9.189 @@ -159,92 +222,166 @@ 9.190 change only the value of pitch. key-f/key-g moves it side to side 9.191 and changes yaw. key-v/key-b will spin the blue segment clockwise 9.192 and counterclockwise, and only affect roll." 9.193 - [] 9.194 - (let [hand (box 0.2 1 0.2 :position (Vector3f. 0 0 0) 9.195 - :mass 0 :color ColorRGBA/Green :name "hand") 9.196 - finger (box 0.2 1 0.2 :position (Vector3f. 0 2.4 0) 9.197 - :mass 1 :color ColorRGBA/Red :name "finger") 9.198 - joint-node (box 0.1 0.05 0.05 :color ColorRGBA/Yellow 9.199 - :position (Vector3f. 0 1.2 0) 9.200 - :rotation (doto (Quaternion.) 9.201 - (.fromAngleAxis 9.202 - (/ Math/PI 2) 9.203 - (Vector3f. 0 0 1))) 9.204 - :physical? false) 9.205 - joint (join-at-point hand finger (Vector3f. 0 1.2 0 )) 9.206 - creature (nodify [hand finger joint-node]) 9.207 - finger-control (.getControl finger RigidBodyControl) 9.208 - hand-control (.getControl hand RigidBodyControl)] 9.209 - 9.210 + ([] (test-proprioception false)) 9.211 + ([record?] 9.212 + (let [hand (box 0.2 1 0.2 :position (Vector3f. 0 0 0) 9.213 + :mass 0 :color ColorRGBA/Gray :name "hand") 9.214 + finger (box 0.2 1 0.2 :position (Vector3f. 0 2.4 0) 9.215 + :mass 1 9.216 + :color 9.217 + (ColorRGBA. (/ 184 255) (/ 127 255) (/ 201 255) 1) 9.218 + :name "finger") 9.219 + joint-node (box 0.1 0.05 0.05 :color ColorRGBA/Yellow 9.220 + :position (Vector3f. 0 1.2 0) 9.221 + :rotation (doto (Quaternion.) 9.222 + (.fromAngleAxis 9.223 + (/ Math/PI 2) 9.224 + (Vector3f. 0 0 1))) 9.225 + :physical? false) 9.226 + creature (nodify [hand finger joint-node]) 9.227 + finger-control (.getControl finger RigidBodyControl) 9.228 + hand-control (.getControl hand RigidBodyControl) 9.229 + joint (joint-dispatch {:type :point} hand-control finger-control 9.230 + (Vector3f. 0 1.2 0) 9.231 + (Vector3f. 0 -1.2 0) nil) 9.232 9.233 - (let 9.234 - ;; ******************************************* 9.235 - 9.236 - [floor (box 10 10 10 :position (Vector3f. 0 -15 0) 9.237 - :mass 0 :color ColorRGBA/Gray) 9.238 - 9.239 - root (nodify [creature floor]) 9.240 - prop (joint-proprioception creature joint-node) 9.241 - prop-view (proprioception-debug-window) 9.242 - 9.243 - controls 9.244 - (merge standard-debug-controls 9.245 - {"key-o" 9.246 - (fn [_ _] (.setEnabled finger-control true)) 9.247 - "key-p" 9.248 - (fn [_ _] (.setEnabled finger-control false)) 9.249 - "key-k" 9.250 - (fn [_ _] (.setEnabled hand-control true)) 9.251 - "key-l" 9.252 - (fn [_ _] (.setEnabled hand-control false)) 9.253 - "key-i" 9.254 - (fn [world _] (set-gravity world (Vector3f. 0 0 0))) 9.255 - "key-period" 9.256 - (fn [world _] 9.257 - (.setEnabled finger-control false) 9.258 - (.setEnabled hand-control false) 9.259 - (.rotate creature (doto (Quaternion.) 9.260 - (.fromAngleAxis 9.261 - (float (/ Math/PI 15)) 9.262 - (Vector3f. 0 0 -1)))) 9.263 - 9.264 - (.setEnabled finger-control true) 9.265 - (.setEnabled hand-control true) 9.266 - (set-gravity world (Vector3f. 0 0 0)) 9.267 - ) 9.268 - 9.269 - 9.270 - } 9.271 - ) 9.272 + root (nodify [creature]) 9.273 + prop (proprioception-kernel creature joint-node) 9.274 + prop-view (view-proprioception)] 9.275 + (.setCollisionGroup 9.276 + (.getControl hand RigidBodyControl) 9.277 + PhysicsCollisionObject/COLLISION_GROUP_NONE) 9.278 + (apply 9.279 + world 9.280 + (with-movement 9.281 + finger 9.282 + ["key-r" "key-t" "key-f" "key-g" "key-v" "key-b"] 9.283 + [1 1 10 10 10 10] 9.284 + [root 9.285 + standard-debug-controls 9.286 + (fn [world] 9.287 + (if record? 9.288 + (Capture/captureVideo 9.289 + world 9.290 + (File. "/home/r/proj/cortex/render/proprio/main-view"))) 9.291 + (.setTimer world (com.aurellem.capture.RatchetTimer. 60)) 9.292 + (set-gravity world (Vector3f. 0 0 0)) 9.293 + (enable-debug world) 9.294 + (light-up-everything world)) 9.295 + (fn [_ _] 9.296 + (prop-view 9.297 + (list (prop)) 9.298 + (if record? 9.299 + (File. "/home/r/proj/cortex/render/proprio/proprio"))))]))))) 9.300 +#+end_src 9.301 9.302 - ] 9.303 - (comment 9.304 - (.setCollisionGroup 9.305 - (.getControl hand RigidBodyControl) 9.306 - PhysicsCollisionObject/COLLISION_GROUP_NONE) 9.307 - ) 9.308 - (apply 9.309 - world 9.310 - (with-movement 9.311 - hand 9.312 - ["key-y" "key-u" "key-h" "key-j" "key-n" "key-m"] 9.313 - [10 10 10 10 1 1] 9.314 - (with-movement 9.315 - finger 9.316 - ["key-r" "key-t" "key-f" "key-g" "key-v" "key-b"] 9.317 - [1 1 10 10 10 10] 9.318 - [root 9.319 - controls 9.320 - (fn [world] 9.321 - (.setTimer world (com.aurellem.capture.RatchetTimer. 60)) 9.322 - (set-gravity world (Vector3f. 0 0 0)) 9.323 - (light-up-everything world)) 9.324 - (fn [_ _] (prop-view (list (prop))))])))))) 9.325 +#+results: test-proprioception 9.326 +: #'cortex.test.proprioception/test-proprioception 9.327 9.328 +* Video of Proprioception 9.329 + 9.330 +#+begin_html 9.331 +<div class="figure"> 9.332 +<center> 9.333 +<video controls="controls" width="550"> 9.334 + <source src="../video/test-proprioception.ogg" type="video/ogg" 9.335 + preload="none" poster="../images/aurellem-1280x480.png" /> 9.336 +</video> 9.337 +</center> 9.338 +<p>Proprioception in a simple creature. The proprioceptive readout is 9.339 + in the upper left corner of the screen.</p> 9.340 +</div> 9.341 +#+end_html 9.342 + 9.343 +** Generating the Proprioception Video 9.344 +#+name: magick6 9.345 +#+begin_src clojure 9.346 +(ns cortex.video.magick6 9.347 + (:import java.io.File) 9.348 + (:use clojure.contrib.shell-out)) 9.349 + 9.350 +(defn images [path] 9.351 + (sort (rest (file-seq (File. path))))) 9.352 + 9.353 +(def base "/home/r/proj/cortex/render/proprio/") 9.354 + 9.355 +(defn pics [file] 9.356 + (images (str base file))) 9.357 + 9.358 +(defn combine-images [] 9.359 + (let [main-view (pics "main-view") 9.360 + proprioception (pics "proprio/0") 9.361 + targets (map 9.362 + #(File. (str base "out/" (format "%07d.png" %))) 9.363 + (range 0 (count main-view)))] 9.364 + (dorun 9.365 + (pmap 9.366 + (comp 9.367 + (fn [[ main-view proprioception target]] 9.368 + (println target) 9.369 + (sh "convert" 9.370 + main-view 9.371 + proprioception "-geometry" "+20+20" "-composite" 9.372 + target)) 9.373 + (fn [& args] (map #(.getCanonicalPath %) args))) 9.374 + main-view proprioception targets)))) 9.375 #+end_src 9.376 9.377 +#+begin_src sh :results silent 9.378 +cd ~/proj/cortex/render/proprio 9.379 +ffmpeg -r 60 -i out/%07d.png -b:v 9000k -c:v libtheora \ 9.380 + test-proprioception.ogg 9.381 +#+end_src 9.382 + 9.383 +* Headers 9.384 +#+name: proprioception-header 9.385 +#+begin_src clojure 9.386 +(ns cortex.proprioception 9.387 + "Simulate the sense of proprioception (ability to detect the 9.388 + relative positions of body parts with repsect to other body parts) 9.389 + in jMonkeyEngine3. Reads specially prepared blender files to 9.390 + automatically generate proprioceptive senses." 9.391 + (:use (cortex world util sense body)) 9.392 + (:use clojure.contrib.def) 9.393 + (:import com.jme3.scene.Node) 9.394 + (:import java.awt.image.BufferedImage) 9.395 + (:import (com.jme3.math Vector3f Quaternion))) 9.396 +#+end_src 9.397 + 9.398 +#+name: test-proprioception-header 9.399 +#+begin_src clojure 9.400 +(ns cortex.test.proprioception 9.401 +(:import (com.aurellem.capture Capture RatchetTimer)) 9.402 +(:use (cortex util world proprioception body)) 9.403 +(:import java.io.File)) 9.404 +(cortex.import/mega-import-jme3) 9.405 +#+end_src 9.406 + 9.407 +* Source Listing 9.408 + - [[../src/cortex/proprioception.clj][cortex.proprioception]] 9.409 + - [[../src/cortex/test/touch.clj][cortex.test.proprioception]] 9.410 + - [[../src/cortex/video/magick6.clj][cortex.video.magick6]] 9.411 +#+html: <ul> <li> <a href="../org/proprioception.org">This org file</a> </li> </ul> 9.412 + - [[http://hg.bortreb.com ][source-repository]] 9.413 + 9.414 +* Next 9.415 + 9.416 +Next time, I'll give the Worm the power to [[./movement.org][move on it's own]]. 9.417 + 9.418 9.419 * COMMENT generate source 9.420 #+begin_src clojure :tangle ../src/cortex/proprioception.clj 9.421 +<<proprioception-header>> 9.422 +<<helpers>> 9.423 <<proprioception>> 9.424 +<<visualize>> 9.425 #+end_src 9.426 + 9.427 +#+begin_src clojure :tangle ../src/cortex/test/proprioception.clj 9.428 +<<test-proprioception-header>> 9.429 +<<test-proprioception>> 9.430 +#+end_src 9.431 + 9.432 +#+begin_src clojure :tangle ../src/cortex/video/magick6.clj 9.433 +<<magick6>> 9.434 +#+end_src
10.1 --- a/org/touch.org Mon Feb 13 21:53:28 2012 -0600 10.2 +++ b/org/touch.org Mon Feb 13 21:53:54 2012 -0600 10.3 @@ -122,7 +122,7 @@ 10.4 #+end_src 10.5 10.6 It is convienent to treat a =Triangle= as a vector of vectors, and a 10.7 -=Vector2f= and =Vector3f= as vectors of floats. (->vector3f) and 10.8 +=Vector2f= or =Vector3f= as vectors of floats. (->vector3f) and 10.9 (->triangle) undo the operations of =(vector3f-seq)= and 10.10 =(triangle-seq)=. If these classes implemented =Iterable= then =(seq)= 10.11 would work on them automitacally. 10.12 @@ -497,6 +497,7 @@ 10.13 I'll use a new creature --- a simple cube which has touch sensors 10.14 evenly distributed along each of its sides. 10.15 10.16 +#+name: test-touch-0 10.17 #+begin_src clojure 10.18 (in-ns 'cortex.test.touch) 10.19 10.20 @@ -504,10 +505,7 @@ 10.21 (load-blender-model "Models/test-touch/touch-cube.blend")) 10.22 #+end_src 10.23 10.24 -#+begin_html 10.25 -<br> 10.26 -#+end_html 10.27 - 10.28 +** The Touch Cube 10.29 #+begin_html 10.30 <div class="figure"> 10.31 <center> 10.32 @@ -527,12 +525,10 @@ 10.33 #+caption: The distribution of feelers along the touch-cube. The colors of the faces are irrelevant; only the white pixels specify feelers. 10.34 [[../images/touch-profile.png]] 10.35 10.36 +#+name: test-touch-1 10.37 #+begin_src clojure 10.38 (in-ns 'cortex.test.touch) 10.39 10.40 -(import com.aurellem.capture.Capture) 10.41 -(import java.io.File) 10.42 - 10.43 (defn test-basic-touch 10.44 ([] (test-basic-touch false)) 10.45 ([record?] 10.46 @@ -576,6 +572,7 @@ 10.47 #+end_html 10.48 10.49 ** Generating the Basic Touch Video 10.50 +#+name: magick4 10.51 #+begin_src clojure 10.52 (ns cortex.video.magick4 10.53 (:import java.io.File) 10.54 @@ -615,31 +612,88 @@ 10.55 background main-view touch targets)))) 10.56 #+end_src 10.57 10.58 +#+begin_src sh :results silent 10.59 +cd /home/r/proj/cortex/render/touch-cube/ 10.60 +ffmpeg -r 60 -i out/%07d.png -b:v 9000k -c:v libtheora basic-touch.ogg 10.61 +#+end_src 10.62 10.63 * Adding Touch to the Worm 10.64 10.65 -#+name: test-touch 10.66 +#+name: test-touch-2 10.67 #+begin_src clojure 10.68 -(ns cortex.test.touch 10.69 - (:use (cortex world util sense body touch)) 10.70 - (:use cortex.test.body)) 10.71 +(in-ns 'cortex.test.touch) 10.72 10.73 -(cortex.import/mega-import-jme3) 10.74 +(defn test-touch 10.75 + ([] (test-touch false)) 10.76 + ([record?] 10.77 + (let [the-worm (doto (worm) (body!)) 10.78 + touch (touch! the-worm) 10.79 + touch-display (view-touch)] 10.80 + (world 10.81 + (nodify [the-worm (floor)]) 10.82 + standard-debug-controls 10.83 + 10.84 + (fn [world] 10.85 + (if record? 10.86 + (Capture/captureVideo 10.87 + world 10.88 + (File. "/home/r/proj/cortex/render/worm-touch/main-view/"))) 10.89 + (speed-up world) 10.90 + (light-up-everything world)) 10.91 10.92 -(defn test-touch [] 10.93 - (let [the-worm (doto (worm) (body!)) 10.94 - touch (touch! the-worm) 10.95 - touch-display (view-touch)] 10.96 - (world (nodify [the-worm (floor)]) 10.97 - standard-debug-controls 10.98 - 10.99 - (fn [world] 10.100 - (speed-up world) 10.101 - (light-up-everything world)) 10.102 + (fn [world tpf] 10.103 + (touch-display 10.104 + (map #(% (.getRootNode world)) touch) 10.105 + (if record? 10.106 + (File. "/home/r/proj/cortex/render/worm-touch/touch/")))))))) 10.107 +#+end_src 10.108 10.109 - (fn [world tpf] 10.110 - (touch-display 10.111 - (map #(% (.getRootNode world)) touch)))))) 10.112 +** Worm Touch Demonstration 10.113 +#+begin_html 10.114 +<div class="figure"> 10.115 +<center> 10.116 +<video controls="controls" width="550"> 10.117 + <source src="../video/worm-touch.ogg" type="video/ogg" 10.118 + preload="none" poster="../images/aurellem-1280x480.png" /> 10.119 +</video> 10.120 +</center> 10.121 +<p>The worm responds to touch.</p> 10.122 +</div> 10.123 +#+end_html 10.124 + 10.125 + 10.126 +** Generating the Worm Touch Video 10.127 +#+name: magick5 10.128 +#+begin_src clojure 10.129 +(ns cortex.video.magick5 10.130 + (:import java.io.File) 10.131 + (:use clojure.contrib.shell-out)) 10.132 + 10.133 +(defn images [path] 10.134 + (sort (rest (file-seq (File. path))))) 10.135 + 10.136 +(def base "/home/r/proj/cortex/render/worm-touch/") 10.137 + 10.138 +(defn pics [file] 10.139 + (images (str base file))) 10.140 + 10.141 +(defn combine-images [] 10.142 + (let [main-view (pics "main-view") 10.143 + touch (pics "touch/0") 10.144 + targets (map 10.145 + #(File. (str base "out/" (format "%07d.png" %))) 10.146 + (range 0 (count main-view)))] 10.147 + (dorun 10.148 + (pmap 10.149 + (comp 10.150 + (fn [[ main-view touch target]] 10.151 + (println target) 10.152 + (sh "convert" 10.153 + main-view 10.154 + touch "-geometry" "+0+0" "-composite" 10.155 + target)) 10.156 + (fn [& args] (map #(.getCanonicalPath %) args))) 10.157 + main-view touch targets)))) 10.158 #+end_src 10.159 10.160 * Headers 10.161 @@ -661,9 +715,44 @@ 10.162 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f))) 10.163 #+end_src 10.164 10.165 +#+name: test-touch-header 10.166 +#+begin_src clojure 10.167 +(ns cortex.test.touch 10.168 + (:use (cortex world util sense body touch)) 10.169 + (:use cortex.test.body) 10.170 + (:import com.aurellem.capture.Capture) 10.171 + (:import java.io.File) 10.172 + (:import (com.jme3.math Vector3f ColorRGBA))) 10.173 +#+end_src 10.174 + 10.175 * Source Listing 10.176 + - [[../src/cortex/touch.clj][cortex.touch]] 10.177 + - [[../src/cortex/test/touch.clj][cortex.test.touch]] 10.178 + - [[../src/cortex/video/magick4.clj][cortex.video.magick4]] 10.179 + - [[../src/cortex/video/magick5.clj][cortex.video.magick5]] 10.180 +#+html: <ul> <li> <a href="../org/touch.org">This org file</a> </li> </ul> 10.181 + - [[http://hg.bortreb.com ][source-repository]] 10.182 + 10.183 * Next 10.184 +So far I've implemented simulated Vision, Hearing, and Touch, the most 10.185 +obvious and promiment senses that humans have. Smell and Taste shall 10.186 +remain unimplemented for now. This accounts for the "five senses" that 10.187 +feature so prominently in our lives. But humans have far more than the 10.188 +five main senses. There are internal chemical senses, pain (which is 10.189 +*not* the same as touch), heat sensitivity, and our sense of balance, 10.190 +among others. One extra sense is so important that I must implement it 10.191 +to have a hope of making creatures that can gracefully control their 10.192 +own bodies. It is Proprioception, which is the sense of the location 10.193 +of each body part in relation to the other body parts. 10.194 10.195 +Close your eyes, and touch your nose with your right index finger. How 10.196 +did you do it? You could not see your hand, and neither your hand nor 10.197 +your nose could use the sense of touch to guide the path of your hand. 10.198 +There are no sound cues, and Taste and Smell certainly don't provide 10.199 +any help. You know where your hand is without your other senses 10.200 +because of Proprioception. 10.201 + 10.202 +Onward to [[./proprioception.org][proprioception]]! 10.203 10.204 * COMMENT Code Generation 10.205 #+begin_src clojure :tangle ../src/cortex/touch.clj 10.206 @@ -678,11 +767,22 @@ 10.207 <<visualization>> 10.208 #+end_src 10.209 10.210 +#+begin_src clojure :tangle ../src/cortex/test/touch.clj 10.211 +<<test-touch-header>> 10.212 +<<test-touch-0>> 10.213 +<<test-touch-1>> 10.214 +<<test-touch-2>> 10.215 +#+end_src 10.216 10.217 -#+begin_src clojure :tangle ../src/cortex/test/touch.clj 10.218 -<<test-touch>> 10.219 +#+begin_src clojure :tangle ../src/cortex/video/magick4.clj 10.220 +<<magick4>> 10.221 #+end_src 10.222 10.223 +#+begin_src clojure :tangle ../src/cortex/video/magick5.clj 10.224 +<<magick5>> 10.225 +#+end_src 10.226 + 10.227 + 10.228 10.229 10.230