Mercurial > cortex
diff org/integration.org @ 304:2dfebf71053c
Merged Winston cover letter
author | Dylan Holmes <ocsenave@gmail.com> |
---|---|
date | Sat, 18 Feb 2012 02:07:40 -0600 |
parents | 7e3938f40c52 |
children | 7e7f8d6d9ec5 |
line wrap: on
line diff
1.1 --- a/org/integration.org Sat Feb 18 02:06:06 2012 -0600 1.2 +++ b/org/integration.org Sat Feb 18 02:07:40 2012 -0600 1.3 @@ -1,4 +1,4 @@ 1.4 -#+title: First attempt at a creature! 1.5 +#+title: 1.6 #+author: Robert McIntyre 1.7 #+email: rlm@mit.edu 1.8 #+description: 1.9 @@ -6,72 +6,235 @@ 1.10 #+SETUPFILE: ../../aurellem/org/setup.org 1.11 #+INCLUDE: ../../aurellem/org/level-0.org 1.12 1.13 +* Integration 1.14 1.15 +This is the ultimate test which features all of the senses that I've 1.16 +made so far. The blender file for the creature serves as an example of 1.17 +a fully equipped creature in terms of senses. You can find it [[../assets/Models/test-creature/hand.blend][here]]. 1.18 1.19 1.20 -* Intro 1.21 -So far, I've made the following senses -- 1.22 - - Vision 1.23 - - Hearing 1.24 - - Touch 1.25 - - Proprioception 1.26 +#+begin_html 1.27 +<div class="figure"> 1.28 +<video controls="controls" width="755"> 1.29 + <source src="../video/hand.ogg" type="video/ogg" 1.30 + preload="none" poster="../images/aurellem-1280x480.png" /> 1.31 +</video> 1.32 +<p>Simulated Senses in a Virtual Environment</p> 1.33 +</div> 1.34 +#+end_html 1.35 1.36 -And one effector: 1.37 - - Movement 1.38 - 1.39 -However, the code so far has only enabled these senses, but has not 1.40 -actually implemented them. For example, there is still a lot of work 1.41 -to be done for vision. I need to be able to create an /eyeball/ in 1.42 -simulation that can be moved around and see the world from different 1.43 -angles. I also need to determine weather to use log-polar or cartesian 1.44 -for the visual input, and I need to determine how/wether to 1.45 -disceritise the visual input. 1.46 - 1.47 -I also want to be able to visualize both the sensors and the 1.48 -effectors in pretty pictures. This semi-retarted creature will be my 1.49 -first attempt at bringing everything together. 1.50 - 1.51 -* The creature's body 1.52 - 1.53 -Still going to do an eve-like body in blender, but due to problems 1.54 -importing the joints, etc into jMonkeyEngine3, I'm going to do all 1.55 -the connecting here in clojure code, using the names of the individual 1.56 -components and trial and error. Later, I'll maybe make some sort of 1.57 -creature-building modifications to blender that support whatever 1.58 -discritized senses I'm going to make. 1.59 +* Generating the Video 1.60 1.61 #+name: integration 1.62 #+begin_src clojure 1.63 -(ns cortex.integration 1.64 +(ns cortex.test.integration 1.65 "let's play!" 1.66 {:author "Robert McIntyre"} 1.67 - (:use (cortex world util body 1.68 - hearing touch vision sense proprioception movement)) 1.69 + (:use (cortex world util body sense 1.70 + hearing touch vision proprioception movement)) 1.71 (:import (com.jme3.math ColorRGBA Vector3f)) 1.72 + (:import java.io.File) 1.73 (:import com.jme3.audio.AudioNode) 1.74 (:import com.aurellem.capture.RatchetTimer)) 1.75 1.76 -(def hand "Models/creature1/one.blend") 1.77 +(dorun (cortex.import/mega-import-jme3)) 1.78 +(rlm.rlm-commands/help) 1.79 1.80 -(def worm "Models/creature1/try-again.blend") 1.81 +(def hand "Models/test-creature/hand.blend") 1.82 1.83 -(defn test-creature [thing] 1.84 - (let [x-axis 1.85 - (box 1 0.01 0.01 :physical? false :color ColorRGBA/Red) 1.86 - y-axis 1.87 - (box 0.01 1 0.01 :physical? false :color ColorRGBA/Green) 1.88 - z-axis 1.89 - (box 0.01 0.01 1 :physical? false :color ColorRGBA/Blue) 1.90 +(def output-base (File. "/home/r/proj/cortex/render/hand")) 1.91 +#+end_src 1.92 1.93 - me (sphere 0.5 :color ColorRGBA/Blue :physical? false) 1.94 - bell (AudioNode. (asset-manager) 1.95 - "Sounds/pure.wav" false) 1.96 +For this demonstration I have to manually drive the muscles of the 1.97 +hand. I do this by creating a little mini-language to describe 1.98 +simulated muscle contraction. 1.99 1.100 - fix-display 1.101 - (runonce (fn [world] 1.102 - (add-camera! world (.getCamera world) no-op))) 1.103 - creature (doto (load-blender-model thing) (body!)) 1.104 +#+name: integration-2 1.105 +#+begin_src clojure 1.106 +(defn motor-control-program 1.107 + "Create a function which will execute the motor script" 1.108 + [muscle-positions 1.109 + script] 1.110 + (let [current-frame (atom -1) 1.111 + keyed-script (group-by first script) 1.112 + current-forces (atom {}) ] 1.113 + (fn [effectors] 1.114 + (let [indexed-effectors (vec effectors)] 1.115 + (dorun 1.116 + (for [[_ part force] (keyed-script (swap! current-frame inc))] 1.117 + (swap! current-forces (fn [m] (assoc m part force))))) 1.118 + (doall (map (fn [effector power] 1.119 + (effector (int power))) 1.120 + effectors 1.121 + (map #(@current-forces % 0) muscle-positions))))))) 1.122 1.123 +(def muscle-positions 1.124 + [:pointer-2-e 1.125 + :pointer-2-f 1.126 + :thumb-1 1.127 + :thumb-1 1.128 + :pointer-1-e 1.129 + :pointer-1-f 1.130 + :thumb-2-e 1.131 + :thumb-2-f 1.132 + :middle-1-e 1.133 + :middle-1-f 1.134 + :pointer-3-f 1.135 + :pointer-3-e 1.136 + :middle-2-e 1.137 + :middle-2-f 1.138 + :middle-3-f 1.139 + :middle-3-e 1.140 + :pinky-2-e 1.141 + :pinky-2-f 1.142 + :pinky-3-f 1.143 + :pinky-3-e 1.144 + :ring-3-e 1.145 + :ring-3-f 1.146 + :ring-2-f 1.147 + :ring-2-e 1.148 + :ring-1-e 1.149 + :ring-1-f 1.150 + :thumb-1-e 1.151 + :thumb-1-f 1.152 + :pinky-1-f 1.153 + :pinky-1-e]) 1.154 + 1.155 +(def full 9001) 1.156 + 1.157 + 1.158 +;; Coreography: 1.159 + 1.160 +;; Let the hand fall palm-up 1.161 + 1.162 +;; it curls its phalanges, starting with the pinky. 1.163 + 1.164 +;; it lets its phalanges fall back down. 1.165 + 1.166 +;; block falls down onto the hand, accompanied by a sound. The block 1.167 +;; can be seen by the hand's eye. 1.168 + 1.169 +;; hand FORCEFULLY catapults the block so that it hits the camera. 1.170 + 1.171 + 1.172 +;; the systax here is [keyframe body-part force] 1.173 +(def move-fingers 1.174 + [[300 :pinky-3-f 50] 1.175 + [320 :pinky-2-f 80] 1.176 + [340 :pinky-1-f 100] 1.177 + 1.178 + [310 :ring-3-f 100] 1.179 + [330 :ring-2-f 120] 1.180 + [350 :ring-1-f 140] 1.181 + 1.182 + [330 :middle-3-f 120] 1.183 + [340 :middle-2-f 120] 1.184 + [360 :middle-1-f 30] 1.185 + 1.186 + [350 :pointer-3-f 120] 1.187 + [360 :pointer-2-f 120] 1.188 + [380 :pointer-1-f 30] 1.189 + 1.190 + [800 :pinky-3-f 0] 1.191 + [800 :pinky-2-f 0] 1.192 + [800 :pinky-1-f 0] 1.193 + 1.194 + [800 :ring-3-f 0] 1.195 + [800 :ring-2-f 0] 1.196 + [800 :ring-1-f 0] 1.197 + 1.198 + [800 :middle-3-f 0] 1.199 + [800 :middle-2-f 0] 1.200 + [800 :middle-1-f 0] 1.201 + 1.202 + [800 :pointer-3-f 0] 1.203 + [800 :pointer-2-f 0] 1.204 + [800 :pointer-1-f 0] 1.205 + 1.206 + 1.207 + [800 :pinky-3-e 50] 1.208 + [800 :pinky-2-e 80] 1.209 + [800 :pinky-1-e 100] 1.210 + 1.211 + [800 :ring-3-e 100] 1.212 + [800 :ring-2-e 120] 1.213 + [800 :ring-1-e 140] 1.214 + 1.215 + [800 :middle-3-e 120] 1.216 + [800 :middle-2-e 120] 1.217 + [800 :middle-1-e 30] 1.218 + 1.219 + [800 :pointer-3-e 120] 1.220 + [800 :pointer-2-e 120] 1.221 + [800 :pointer-1-e 30] 1.222 + 1.223 + [870 :pinky-3-e 0] 1.224 + [870 :pinky-2-e 0] 1.225 + [870 :pinky-1-e 0] 1.226 + 1.227 + [870 :ring-3-e 0] 1.228 + [870 :ring-2-e 0] 1.229 + [870 :ring-1-e 0] 1.230 + 1.231 + [870 :middle-3-e 0] 1.232 + [870 :middle-2-e 0] 1.233 + [870 :middle-1-e 0] 1.234 + 1.235 + [870 :pointer-3-e 0] 1.236 + [870 :pointer-2-e 0] 1.237 + [870 :pointer-1-e 0] 1.238 + 1.239 + [1500 :pointer-1-f full] 1.240 + [1500 :pointer-2-f full] 1.241 + [1500 :pointer-3-f full] 1.242 + 1.243 + [1500 :middle-1-f full] 1.244 + [1500 :middle-2-f full] 1.245 + [1500 :middle-3-f full] 1.246 + 1.247 + [1510 :pointer-1-f 0] 1.248 + [1510 :pointer-2-f 0] 1.249 + [1510 :pointer-3-f 0] 1.250 + 1.251 + [1510 :middle-1-f 0] 1.252 + [1510 :middle-2-f 0] 1.253 + [1510 :middle-3-f 0] 1.254 + ]) 1.255 + 1.256 +(defn gen-summon-ball [debug?] 1.257 + (let [wait (atom 1100)] 1.258 + (fn [world] 1.259 + (if (= 0 (swap! wait dec)) 1.260 + (let [brick 1.261 + (box 0.8 0.8 0.8 :mass 0.05 1.262 + :position (Vector3f. -0.5 0 0.5) 1.263 + :color (ColorRGBA/Red)) 1.264 + bell (AudioNode. (asset-manager) 1.265 + "Sounds/pure.wav" false)] 1.266 + (.play bell) 1.267 + (if debug? 1.268 + (.addControl 1.269 + brick 1.270 + (proxy [AbstractControl] [] 1.271 + (controlUpdate [tpf] 1.272 + (println-repl (.getWorldTranslation brick))) 1.273 + (controlRender [_ _])))) 1.274 + (add-element world brick)))))) 1.275 + 1.276 +(import com.aurellem.capture.Capture) 1.277 + 1.278 +(defn test-everything! 1.279 + ([] (test-everything! false)) 1.280 + ([record?] 1.281 + (let [me (sphere 0.5 :color ColorRGBA/Blue :physical? false) 1.282 + 1.283 + base (File. "/home/r/proj/cortex/render/hand") 1.284 + 1.285 + 1.286 + creature (doto (load-blender-model hand) 1.287 + (body!)) 1.288 + 1.289 + summon-ball (gen-summon-ball false) 1.290 ;;;;;;;;;;;; Sensors/Effectors ;;;;;;;;;;;;;;;;;;;;;;;;;;;; 1.291 touch (touch! creature) 1.292 touch-display (view-touch) 1.293 @@ -85,69 +248,466 @@ 1.294 prop (proprioception! creature) 1.295 prop-display (view-proprioception) 1.296 1.297 - muscle-exertion (atom 0) 1.298 + control-script (motor-control-program 1.299 + muscle-positions move-fingers) 1.300 muscles (movement! creature) 1.301 - muscle-display (view-movement)] 1.302 + muscle-display (view-movement) 1.303 + ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 1.304 1.305 - (apply 1.306 - world 1.307 - (with-movement 1.308 - (.getChild creature "worm-21") 1.309 - ["key-r" "key-t" 1.310 - "key-f" "key-g" 1.311 - "key-v" "key-b"] 1.312 - [10 10 10 10 1 1] 1.313 - [(nodify [creature 1.314 - (box 10 2 10 :position (Vector3f. 0 -9 0) 1.315 - :color ColorRGBA/Gray :mass 0) 1.316 - x-axis y-axis z-axis 1.317 - me]) 1.318 - (merge standard-debug-controls 1.319 - {"key-return" 1.320 - (fn [_ value] 1.321 - (if value 1.322 - (do 1.323 - (println-repl "play-sound") 1.324 - (.play bell)))) 1.325 - "key-h" 1.326 - (fn [_ value] 1.327 - (if value 1.328 - (swap! muscle-exertion (partial + 20)))) 1.329 - "key-n" 1.330 - (fn [_ value] 1.331 - (if value 1.332 - (swap! muscle-exertion (fn [v] (- v 20)))))}) 1.333 - (fn [world] 1.334 - (light-up-everything world) 1.335 - (enable-debug world) 1.336 - (add-camera! world 1.337 - (add-eye! creature 1.338 - (.getChild 1.339 - (.getChild creature "eyes") "eye")) 1.340 - (comp (view-image) BufferedImage!)) 1.341 - (.setTimer world (RatchetTimer. 60)) 1.342 - (speed-up world) 1.343 - (set-gravity world (Vector3f. 0 0 0)) 1.344 - (comment 1.345 - (com.aurellem.capture.Capture/captureVideo 1.346 - world (file-str "/home/r/proj/ai-videos/hand")))) 1.347 - (fn [world tpf] 1.348 - (prop-display (prop)) 1.349 - (touch-display (map #(% (.getRootNode world)) touch)) 1.350 - (vision-display (map #(% world) vision)) 1.351 - (hearing-display (map #(% world) hearing)) 1.352 - (muscle-display (map #(% @muscle-exertion) muscles)) 1.353 - (.setLocalTranslation me (.getLocation (.getCamera world))) 1.354 - (fix-display world))])))) 1.355 + fix-display (gen-fix-display)] 1.356 + (world 1.357 + (nodify [creature 1.358 + (box 10 2 10 :position (Vector3f. 0 -9 0) 1.359 + :color ColorRGBA/Gray :mass 0) 1.360 + me]) 1.361 + standard-debug-controls 1.362 + 1.363 + (fn [world] 1.364 + (.setTimer world (RatchetTimer. 60)) 1.365 + (position-camera 1.366 + world (Vector3f. -0.13217318, 5.816415, -5.3089414) 1.367 + (Quaternion. 0.55685693, 0.0042774677, -0.0028673497, 0.83059245)) 1.368 + 1.369 + (light-up-everything world) 1.370 + (enable-debug world) 1.371 + (add-camera! world 1.372 + (add-eye! creature 1.373 + (.getChild 1.374 + (.getChild creature "eyes") "eye")) 1.375 + (comp (view-image) BufferedImage!)) 1.376 + 1.377 + (if record? 1.378 + (Capture/captureVideo 1.379 + world (File. base "main"))) 1.380 + (if record? 1.381 + (Capture/captureAudio 1.382 + world (File. base "main.wav")))) 1.383 + (fn [world tpf] 1.384 + (prop-display 1.385 + (prop) 1.386 + (if record? (File. base "proprio"))) 1.387 + (touch-display 1.388 + (map #(% (.getRootNode world)) touch) 1.389 + (if record? (File. base "touch"))) 1.390 + (vision-display 1.391 + (map #(% world) vision) 1.392 + (if record? (File. base "vision"))) 1.393 + (hearing-display 1.394 + (map #(% world) hearing) 1.395 + (if record? (File. base "hearing"))) 1.396 + (muscle-display 1.397 + (control-script muscles) 1.398 + (if record? (File. base "muscle"))) 1.399 + 1.400 + (summon-ball world) 1.401 + 1.402 + (.setLocalTranslation me (.getLocation (.getCamera world))) 1.403 + (fix-display world)))))) 1.404 #+end_src 1.405 1.406 -#+results: body-1 1.407 -: #'cortex.silly/follow-test 1.408 +** ImageMagick / ffmpeg 1.409 + 1.410 +Just a bunch of calls to imagemagick to arrange the data that 1.411 +=test-everything!= produces. 1.412 + 1.413 +#+name: magick-8 1.414 +#+begin_src clojure 1.415 +(ns cortex.video.magick8 1.416 + (:import java.io.File) 1.417 + (:use clojure.contrib.shell-out)) 1.418 + 1.419 +(comment 1.420 +;; list of touch targets 1.421 +0 middle-11 1.422 +1 middle-21 1.423 +2 middle-31 1.424 +3 pinky-11 1.425 +4 pinky-21 1.426 +5 pinky-31 1.427 +6 pointer-11 1.428 +7 pointer-21 1.429 +8 pointer-31 1.430 +9 ring-11 1.431 +10 ring-21 1.432 +11 ring-31 1.433 +12 thumb-11 1.434 +13 thumb-2.0011 1.435 + 1.436 + 1.437 +;; list of vision targets 1.438 +0 :all 1.439 +1 :green 1.440 +2 :blue 1.441 +3 :red 1.442 + 1.443 +;; list of proprio targets 1.444 +0 middle-11 -> middle-21 1.445 +1 middle-21 -> middle-31 1.446 +2 thumb-11 -> thumb-2.0011 1.447 +3 pointer-11 -> pointer-21 1.448 +4 pointer-21 -> pointer-31 1.449 +5 ring-21 -> ring-31 1.450 +6 ring-11 -> ring-21 1.451 +7 pinky-21 -> pinky-31 1.452 +8 pinky-11 -> pinky-21 1.453 +9 middle-11 -> palm1 1.454 +10 pinky-11 -> palm1 1.455 +11 palm1 -> pointer-11 1.456 +12 palm1 -> ring-11 1.457 +13 palm1 -> thumb-11 1.458 + 1.459 + 1.460 +;; list of muscle targets 1.461 +0 :pointer-2-e 1.462 +1 :pointer-2-f 1.463 +2 :thumb-1 1.464 +3 :thumb-1 1.465 +4 :pointer-1-e 1.466 +5 :pointer-1-f 1.467 +6 :thumb-2-e 1.468 +7 :thumb-2-f 1.469 +8 :middle-1-e 1.470 +9 :middle-1-f 1.471 +10 :pointer-3-f 1.472 +11 :pointer-3-e 1.473 +12 :middle-2-e 1.474 +13 :middle-2-f 1.475 +14 :middle-3-f 1.476 +15 :middle-3-e 1.477 +16 :pinky-2-e 1.478 +17 :pinky-2-f 1.479 +18 :pinky-3-f 1.480 +19 :pinky-3-e 1.481 +20 :ring-3-e 1.482 +21 :ring-3-f 1.483 +22 :ring-2-f 1.484 +23 :ring-2-e 1.485 +24 :ring-1-e 1.486 +25 :ring-1-f 1.487 +26 :thumb-1-e 1.488 +27 :thumb-1-f 1.489 +28 :pinky-1-f 1.490 +29 :pinky-1-e 1.491 +) 1.492 + 1.493 +(def base (File. "/home/r/proj/cortex/render/hand")) 1.494 + 1.495 +(defn prepare-muscle [muscle] 1.496 + ["(" muscle "-rotate" "90" "-scale" "15x60!" ")"]) 1.497 + 1.498 +(defn prepare-touch [touch] 1.499 + ["(" touch "-rotate" "180" ")"]) 1.500 + 1.501 +(defn generate-top-finger [tip-flexor tip-extensor tip 1.502 + joint-2-3 1.503 + mid-flexor mid-extensor mid 1.504 + joint-1-2] 1.505 + ["(" 1.506 + "-size" "113x357" "xc:transparent" 1.507 + (prepare-muscle tip-flexor) "-geometry" "+0+7" "-composite" 1.508 + (prepare-muscle tip-extensor) "-geometry" "+98+7" "-composite" 1.509 + (prepare-touch tip) "-geometry" "+18+0" "-composite" 1.510 + 1.511 + joint-2-3 "-geometry" "+32+79" "-composite" 1.512 + 1.513 + (prepare-muscle mid-flexor) "-geometry" "+19+131" "-composite" 1.514 + (prepare-muscle mid-extensor) "-geometry" "+80+131" "-composite" 1.515 + (prepare-touch mid) "-geometry" "+39+133" "-composite" 1.516 + 1.517 + joint-1-2 "-geometry" "+32+193" "-composite" 1.518 + ")"]) 1.519 + 1.520 +(defn file-names [#^File dir] 1.521 + (map #(.getCanonicalPath %) (next (sort (file-seq dir))))) 1.522 + 1.523 +(defn file-groups [& paths] 1.524 + (apply (partial map list ) 1.525 + (map (comp file-names #(File. base %)) 1.526 + paths))) 1.527 + 1.528 +(defn pinky [] 1.529 + (file-groups 1.530 + "muscle/18" 1.531 + "muscle/19" 1.532 + "touch/5" 1.533 + "proprio/7" 1.534 + 1.535 + "muscle/17" 1.536 + "muscle/16" 1.537 + "touch/4" 1.538 + "proprio/8" 1.539 + 1.540 + "muscle/28" 1.541 + "muscle/29" 1.542 + "touch/3" 1.543 + "proprio/10")) 1.544 + 1.545 +(defn ring [] 1.546 + (file-groups 1.547 + "muscle/21" 1.548 + "muscle/20" 1.549 + "touch/11" 1.550 + "proprio/5" 1.551 + 1.552 + "muscle/22" 1.553 + "muscle/23" 1.554 + "touch/10" 1.555 + "proprio/6" 1.556 + 1.557 + "muscle/25" 1.558 + "muscle/24" 1.559 + "touch/9" 1.560 + "proprio/12")) 1.561 + 1.562 +(defn middle [] 1.563 + (file-groups 1.564 + "muscle/14" 1.565 + "muscle/15" 1.566 + "touch/2" 1.567 + "proprio/1" 1.568 + 1.569 + "muscle/13" 1.570 + "muscle/12" 1.571 + "touch/1" 1.572 + "proprio/0" 1.573 + 1.574 + "muscle/9" 1.575 + "muscle/8" 1.576 + "touch/0" 1.577 + "proprio/9")) 1.578 + 1.579 +(defn pointer [] 1.580 + (file-groups 1.581 + "muscle/10" 1.582 + "muscle/11" 1.583 + "touch/8" 1.584 + "proprio/4" 1.585 + 1.586 + "muscle/1" 1.587 + "muscle/0" 1.588 + "touch/7" 1.589 + "proprio/3" 1.590 + 1.591 + "muscle/5" 1.592 + "muscle/4" 1.593 + "touch/6" 1.594 + "proprio/11")) 1.595 + 1.596 +(defn thumb [] 1.597 + (file-groups 1.598 + "muscle/7" 1.599 + "muscle/6" 1.600 + "touch/13" 1.601 + "proprio/2" 1.602 + 1.603 + "muscle/27" 1.604 + "muscle/26" 1.605 + "muscle/3" 1.606 + "muscle/2" 1.607 + "touch/12" 1.608 + "proprio/13")) 1.609 + 1.610 +(defn generate-finger 1.611 + [tip-flexor tip-extensor tip 1.612 + joint-2-3 1.613 + mid-flexor mid-extensor mid 1.614 + joint-1-2 1.615 + base-flexor base-extensor base 1.616 + joint-palm-1] 1.617 + ["(" 1.618 + "-size" "113x357" "xc:transparent" 1.619 + (generate-top-finger 1.620 + tip-flexor tip-extensor tip 1.621 + joint-2-3 1.622 + mid-flexor mid-extensor mid 1.623 + joint-1-2) "-geometry" "+0+0" "-composite" 1.624 + (prepare-muscle base-flexor) "-geometry" "+19+245" "-composite" 1.625 + (prepare-muscle base-extensor) "-geometry" "+80+245" "-composite" 1.626 + (prepare-touch base) "-geometry" "+39+247" "-composite" 1.627 + joint-palm-1 "-geometry" "+32+307" "-composite" 1.628 + ")"]) 1.629 + 1.630 +(defn generate-thumb 1.631 + [tip-flexor tip-extensor tip 1.632 + joint-1-2 1.633 + mid-flexor mid-extensor mid-flexor-2 mid-extensor-2 mid 1.634 + joint-palm-1] 1.635 + ["(" 1.636 + "-size" "113x357" "xc:transparent" 1.637 + (generate-top-finger 1.638 + tip-flexor tip-extensor tip 1.639 + joint-1-2 1.640 + mid-flexor mid-extensor mid 1.641 + joint-palm-1) "-geometry" "+0+0" "-composite" 1.642 + (prepare-muscle mid-flexor-2) "-geometry" "+2+131" "-composite" 1.643 + (prepare-muscle mid-extensor-2) "-geometry" "+100+131" "-composite" 1.644 + ")"]) 1.645 + 1.646 +(defn generate-hand 1.647 + [pinky-pieces 1.648 + ring-pieces 1.649 + middle-pieces 1.650 + pointer-pieces 1.651 + thumb-pieces] 1.652 + ["(" 1.653 + "-size" "688x769" "xc:transparent" 1.654 + (apply generate-finger pinky-pieces) 1.655 + "-geometry" "+0+195" "-composite" 1.656 + (apply generate-finger ring-pieces) 1.657 + "-geometry" "+111+100" "-composite" 1.658 + (apply generate-finger middle-pieces) 1.659 + "-geometry" "+228+0" "-composite" 1.660 + "(" (apply generate-thumb thumb-pieces) "-background" "#00000000" 1.661 + "-rotate" "45" ")" 1.662 + "-geometry" "+300+420" "-composite" 1.663 + (apply generate-finger pointer-pieces) 1.664 + "-geometry" "+350+96" "-composite" 1.665 + ")"]) 1.666 + 1.667 +(defn generate-vision 1.668 + [all green blue red] 1.669 + ["(" 1.670 + "-size" "204x192" "xc:transparent" 1.671 + all "-geometry" "+0+0" "-composite" 1.672 + green "-geometry" "+113+0" "-composite" 1.673 + blue "-geometry" "+0+105" "-composite" 1.674 + red "-geometry" "+113+105" "-composite" 1.675 + ")"]) 1.676 + 1.677 +(def test-muscle (File. base "muscle/0/0000000.png")) 1.678 +(def test-proprio (File. base "proprio/0/0000000.png")) 1.679 +(def test-tip (File. base "touch/2/0000000.png")) 1.680 +(def test-mid (File. base "touch/0/0000000.png")) 1.681 +(def test-vision (File. base "vision/0/0000000.png")) 1.682 +(def test-hearing (File. base "hearing/0/0000000.png")) 1.683 +(def test-main (File. base "main/0000000.png")) 1.684 + 1.685 +(def test-target (File. base "output.png")) 1.686 + 1.687 +(def background (File. base "background.png")) 1.688 + 1.689 +(use 'clojure.contrib.shell-out) 1.690 + 1.691 +(defn vision [] 1.692 + (file-groups 1.693 + "vision/0" 1.694 + "vision/1" 1.695 + "vision/2" 1.696 + "vision/3")) 1.697 + 1.698 +(defn hearing [] 1.699 + (file-names (File. base "hearing/0"))) 1.700 + 1.701 +(defn main [] 1.702 + (file-names (File. base "main"))) 1.703 + 1.704 +(defn targets [dest max] 1.705 + (map 1.706 + (comp #(.getCanonicalPath %) 1.707 + #(File. (str base dest (format "%07d.png" %)))) 1.708 + (range max))) 1.709 + 1.710 + 1.711 +(defn final-image [main [all red green blue] hearing 1.712 + pinky ring middle pointer thumb target] 1.713 + (println target) 1.714 + (apply 1.715 + sh 1.716 + (flatten 1.717 + ["convert" 1.718 + (.getCanonicalPath background) 1.719 + 1.720 + (generate-hand pinky ring middle pointer thumb) 1.721 + "-geometry" "+809+22" "-composite" 1.722 + 1.723 + (generate-vision all red green blue) 1.724 + "-geometry" "+974+599" "-composite" 1.725 + 1.726 + hearing 1.727 + "-geometry" "+784+819" "-composite" 1.728 + 1.729 + main 1.730 + "-geometry" "+78+202" "-composite" 1.731 + 1.732 + target]))) 1.733 + 1.734 +(defn combine-files [] 1.735 + (dorun 1.736 + (pmap final-image 1.737 + (main) 1.738 + (vision) 1.739 + (hearing) 1.740 + (pinky) 1.741 + (ring) 1.742 + (middle) 1.743 + (pointer) 1.744 + (thumb) 1.745 + (targets "/out/" (count (main)))))) 1.746 + 1.747 +(defn subtitles [] 1.748 + (file-names (File. base "subs"))) 1.749 + 1.750 +(defn outs [] 1.751 + (file-names (File. base "out"))) 1.752 + 1.753 + 1.754 +(defn mix-subtitles [] 1.755 + (let [subs (subtitles) 1.756 + targets (targets "/out-subs/" (count subs)) 1.757 + overlay (.getCanonicalPath (File. base "output.png"))] 1.758 + (dorun 1.759 + (pmap 1.760 + (fn [sub target] 1.761 + (sh 1.762 + "convert" 1.763 + overlay 1.764 + sub "-geometry" "+0+0" "-composite" 1.765 + target)) 1.766 + subs targets)))) 1.767 + 1.768 +(defn out-subtitles [] 1.769 + (file-names (File. base "out-subs"))) 1.770 + 1.771 + 1.772 +(defn insert-subtitles [] 1.773 + (let [subtitles (out-subtitles) 1.774 + outs (outs) 1.775 + targets (targets 1.776 + "/final/" 1.777 + (+ (count outs) (count subtitles)))] 1.778 + (dorun 1.779 + (pmap 1.780 + #(sh "cp" %1 %2) 1.781 + (concat subtitles outs) targets)))) 1.782 + 1.783 +(defn generate-final [] 1.784 + (combine-files) 1.785 + (mix-subtitles) 1.786 + (insert-subtitles)) 1.787 +#+end_src 1.788 + 1.789 +#+begin_src sh :results silent 1.790 +cd /home/r/proj/cortex/render/hand 1.791 + 1.792 +sox --ignore-length main.wav main-delayed.wav delay 24 1.793 + 1.794 +mogrify -resize 755x final/* 1.795 + 1.796 +ffmpeg -r 60 -i final/%07d.png -i main-delayed.wav -b:a 128k \ 1.797 + -b:v 9000k -c:a libvorbis -c:v libtheora hand.ogg 1.798 +#+end_src 1.799 + 1.800 + 1.801 +* Source Listing 1.802 + - [[../src/cortex/test/integration.clj][cortex.test.integration]] 1.803 + - [[../src/cortex/video/magick8.clj][cortex.video.magick8]] 1.804 + - [[../assets/Models/subtitles/hand.blend][subtitles/hand.blend]] 1.805 +#+html: <ul> <li> <a href="../org/integration.org">This org file</a> </li> </ul> 1.806 + - [[http://hg.bortreb.com ][source-repository]] 1.807 1.808 1.809 * COMMENT purgatory 1.810 #+begin_src clojure 1.811 - 1.812 (defn bullet-trans* [] 1.813 (let [obj-a (box 1.5 0.5 0.5 :color ColorRGBA/Red 1.814 :position (Vector3f. 5 0 0) 1.815 @@ -308,10 +868,11 @@ 1.816 1.817 1.818 * COMMENT generate source 1.819 -#+begin_src clojure :tangle ../src/cortex/integration.clj 1.820 +#+begin_src clojure :tangle ../src/cortex/test/integration.clj 1.821 <<integration>> 1.822 +<<integration-2>> 1.823 #+end_src 1.824 1.825 - 1.826 - 1.827 - 1.828 +#+begin_src clojure :tangle ../src/cortex/video/magick8.clj 1.829 +<<magick-8>> 1.830 +#+end_src