comparison thesis/cortex.org @ 509:81c845a91378

completed initial render at 81 pages, but images are unacceptably scattered.
author Robert McIntyre <rlm@mit.edu>
date Sun, 30 Mar 2014 01:22:23 -0400
parents c11d3fc3e6f0
children f639e2139ce2
comparison
equal deleted inserted replaced
508:c11d3fc3e6f0 509:81c845a91378
245 #+caption: Body-centerd actions are best expressed in a body-centered 245 #+caption: Body-centerd actions are best expressed in a body-centered
246 #+caption: language. This code detects when the worm has curled into a 246 #+caption: language. This code detects when the worm has curled into a
247 #+caption: full circle. Imagine how you would replicate this functionality 247 #+caption: full circle. Imagine how you would replicate this functionality
248 #+caption: using low-level pixel features such as HOG filters! 248 #+caption: using low-level pixel features such as HOG filters!
249 #+name: grand-circle-intro 249 #+name: grand-circle-intro
250 #+attr_latex: [htpb] 250 #+begin_listing clojure
251 #+begin_listing clojure
252 #+begin_src clojure 251 #+begin_src clojure
253 (defn grand-circle? 252 (defn grand-circle?
254 "Does the worm form a majestic circle (one end touching the other)?" 253 "Does the worm form a majestic circle (one end touching the other)?"
255 [experiences] 254 [experiences]
256 (and (curled? experiences) 255 (and (curled? experiences)
1376 another. 1375 another.
1377 1376
1378 #+caption: Program for extending =OpenAL= to support multiple 1377 #+caption: Program for extending =OpenAL= to support multiple
1379 #+caption: listeners via context copying/switching. 1378 #+caption: listeners via context copying/switching.
1380 #+name: sync-openal-sources 1379 #+name: sync-openal-sources
1381 #+begin_listing C 1380 #+begin_listing c
1382 #+BEGIN_SRC C 1381 #+BEGIN_SRC c
1383 void syncSources(ALsource *masterSource, ALsource *slaveSource, 1382 void syncSources(ALsource *masterSource, ALsource *slaveSource,
1384 ALCcontext *masterCtx, ALCcontext *slaveCtx){ 1383 ALCcontext *masterCtx, ALCcontext *slaveCtx){
1385 ALuint master = masterSource->source; 1384 ALuint master = masterSource->source;
1386 ALuint slave = slaveSource->source; 1385 ALuint slave = slaveSource->source;
1387 ALCcontext *current = alcGetCurrentContext(); 1386 ALCcontext *current = alcGetCurrentContext();
1558 #+caption: the ball is constantally emiting a pure tone of 1557 #+caption: the ball is constantally emiting a pure tone of
1559 #+caption: constant volume. As it approaches the cubes, they each 1558 #+caption: constant volume. As it approaches the cubes, they each
1560 #+caption: change color in response to the sound. 1559 #+caption: change color in response to the sound.
1561 #+name: sound-cubes. 1560 #+name: sound-cubes.
1562 #+ATTR_LaTeX: :width 10cm 1561 #+ATTR_LaTeX: :width 10cm
1563 [[./images/aurellem-gray.png]] 1562 [[./images/java-hearing-test.png]]
1564 1563
1565 This system of hearing has also been co-opted by the 1564 This system of hearing has also been co-opted by the
1566 jMonkeyEngine3 community and is used to record audio for demo 1565 jMonkeyEngine3 community and is used to record audio for demo
1567 videos. 1566 videos.
1568 1567
2439 - Inverse kinematics :: experiments in sense guided motor control 2438 - Inverse kinematics :: experiments in sense guided motor control
2440 are easy given =CORTEX='s support -- you can get right to the 2439 are easy given =CORTEX='s support -- you can get right to the
2441 hard control problems without worrying about physics or 2440 hard control problems without worrying about physics or
2442 senses. 2441 senses.
2443 2442
2444 * COMMENT Empathy in a simulated worm 2443 * Empathy in a simulated worm
2445 2444
2446 Here I develop a computational model of empathy, using =CORTEX= as a 2445 Here I develop a computational model of empathy, using =CORTEX= as a
2447 base. Empathy in this context is the ability to observe another 2446 base. Empathy in this context is the ability to observe another
2448 creature and infer what sorts of sensations that creature is 2447 creature and infer what sorts of sensations that creature is
2449 feeling. My empathy algorithm involves multiple phases. First is 2448 feeling. My empathy algorithm involves multiple phases. First is
2523 #+caption: of sensory experience, and only uses proprioceptive data. Even 2522 #+caption: of sensory experience, and only uses proprioceptive data. Even
2524 #+caption: this simple predicate, however, is automatically frame 2523 #+caption: this simple predicate, however, is automatically frame
2525 #+caption: independent and ignores vermopomorphic differences such as 2524 #+caption: independent and ignores vermopomorphic differences such as
2526 #+caption: worm textures and colors. 2525 #+caption: worm textures and colors.
2527 #+name: curled 2526 #+name: curled
2528 #+attr_latex: [htpb] 2527 #+begin_listing clojure
2529 #+begin_listing clojure
2530 #+begin_src clojure 2528 #+begin_src clojure
2531 (defn curled? 2529 (defn curled?
2532 "Is the worm curled up?" 2530 "Is the worm curled up?"
2533 [experiences] 2531 [experiences]
2534 (every? 2532 (every?
2539 #+end_listing 2537 #+end_listing
2540 2538
2541 #+caption: Program for summarizing the touch information in a patch 2539 #+caption: Program for summarizing the touch information in a patch
2542 #+caption: of skin. 2540 #+caption: of skin.
2543 #+name: touch-summary 2541 #+name: touch-summary
2544 #+attr_latex: [htpb] 2542 #+begin_listing clojure
2545
2546 #+begin_listing clojure
2547 #+begin_src clojure 2543 #+begin_src clojure
2548 (defn contact 2544 (defn contact
2549 "Determine how much contact a particular worm segment has with 2545 "Determine how much contact a particular worm segment has with
2550 other objects. Returns a value between 0 and 1, where 1 is full 2546 other objects. Returns a value between 0 and 1, where 1 is full
2551 contact and 0 is no contact." 2547 contact and 0 is no contact."
2566 #+caption: uses a summary of the tactile information from the underbelly 2562 #+caption: uses a summary of the tactile information from the underbelly
2567 #+caption: of the worm, and is only true if every segment is touching the 2563 #+caption: of the worm, and is only true if every segment is touching the
2568 #+caption: floor. Note that this function contains no references to 2564 #+caption: floor. Note that this function contains no references to
2569 #+caption: proprioction at all. 2565 #+caption: proprioction at all.
2570 #+name: resting 2566 #+name: resting
2571 #+attr_latex: [htpb]
2572 #+begin_listing clojure 2567 #+begin_listing clojure
2573 #+begin_src clojure 2568 #+begin_src clojure
2574 (def worm-segment-bottom (rect-region [8 15] [14 22])) 2569 (def worm-segment-bottom (rect-region [8 15] [14 22]))
2575 2570
2576 (defn resting? 2571 (defn resting?
2586 #+caption: Program for detecting whether the worm is curled up into a 2581 #+caption: Program for detecting whether the worm is curled up into a
2587 #+caption: full circle. Here the embodied approach begins to shine, as 2582 #+caption: full circle. Here the embodied approach begins to shine, as
2588 #+caption: I am able to both use a previous action predicate (=curled?=) 2583 #+caption: I am able to both use a previous action predicate (=curled?=)
2589 #+caption: as well as the direct tactile experience of the head and tail. 2584 #+caption: as well as the direct tactile experience of the head and tail.
2590 #+name: grand-circle 2585 #+name: grand-circle
2591 #+attr_latex: [htpb]
2592 #+begin_listing clojure 2586 #+begin_listing clojure
2593 #+begin_src clojure 2587 #+begin_src clojure
2594 (def worm-segment-bottom-tip (rect-region [15 15] [22 22])) 2588 (def worm-segment-bottom-tip (rect-region [15 15] [22 22]))
2595 2589
2596 (def worm-segment-top-tip (rect-region [0 15] [7 22])) 2590 (def worm-segment-top-tip (rect-region [0 15] [7 22]))
2617 #+caption: wiggling this way also gives the worm an opportunity to learn 2611 #+caption: wiggling this way also gives the worm an opportunity to learn
2618 #+caption: and recognize ``frustrated wiggling'', where the worm tries to 2612 #+caption: and recognize ``frustrated wiggling'', where the worm tries to
2619 #+caption: wiggle but can't. Frustrated wiggling is very visually different 2613 #+caption: wiggle but can't. Frustrated wiggling is very visually different
2620 #+caption: from actual wiggling, but this definition gives it to us for free. 2614 #+caption: from actual wiggling, but this definition gives it to us for free.
2621 #+name: wiggling 2615 #+name: wiggling
2622 #+attr_latex: [htpb]
2623 #+begin_listing clojure 2616 #+begin_listing clojure
2624 #+begin_src clojure 2617 #+begin_src clojure
2625 (defn fft [nums] 2618 (defn fft [nums]
2626 (map 2619 (map
2627 #(.getReal %) 2620 #(.getReal %)
2658 all the worm's senses. 2651 all the worm's senses.
2659 2652
2660 #+caption: Use the action predicates defined earlier to report on 2653 #+caption: Use the action predicates defined earlier to report on
2661 #+caption: what the worm is doing while in simulation. 2654 #+caption: what the worm is doing while in simulation.
2662 #+name: report-worm-activity 2655 #+name: report-worm-activity
2663 #+attr_latex: [htpb]
2664 #+begin_listing clojure 2656 #+begin_listing clojure
2665 #+begin_src clojure 2657 #+begin_src clojure
2666 (defn debug-experience 2658 (defn debug-experience
2667 [experiences text] 2659 [experiences text]
2668 (cond 2660 (cond
2777 mistake. 2769 mistake.
2778 2770
2779 #+caption: Program to convert an experience vector into a 2771 #+caption: Program to convert an experience vector into a
2780 #+caption: proprioceptively binned lookup function. 2772 #+caption: proprioceptively binned lookup function.
2781 #+name: bin 2773 #+name: bin
2782 #+attr_latex: [htpb]
2783 #+begin_listing clojure 2774 #+begin_listing clojure
2784 #+begin_src clojure 2775 #+begin_src clojure
2785 (defn bin [digits] 2776 (defn bin [digits]
2786 (fn [angles] 2777 (fn [angles]
2787 (->> angles 2778 (->> angles
2836 2827
2837 #+caption: Program to calculate empathy by tracing though \Phi-space 2828 #+caption: Program to calculate empathy by tracing though \Phi-space
2838 #+caption: and finding the longest (ie. most coherent) interpretation 2829 #+caption: and finding the longest (ie. most coherent) interpretation
2839 #+caption: of the data. 2830 #+caption: of the data.
2840 #+name: longest-thread 2831 #+name: longest-thread
2841 #+attr_latex: [htpb]
2842 #+begin_listing clojure 2832 #+begin_listing clojure
2843 #+begin_src clojure 2833 #+begin_src clojure
2844 (defn longest-thread 2834 (defn longest-thread
2845 "Find the longest thread from phi-index-sets. The index sets should 2835 "Find the longest thread from phi-index-sets. The index sets should
2846 be ordered from most recent to least recent." 2836 be ordered from most recent to least recent."
2879 fill in the gaps. 2869 fill in the gaps.
2880 2870
2881 #+caption: Fill in blanks in sensory experience by replicating the most 2871 #+caption: Fill in blanks in sensory experience by replicating the most
2882 #+caption: recent experience. 2872 #+caption: recent experience.
2883 #+name: infer-nils 2873 #+name: infer-nils
2884 #+attr_latex: [htpb]
2885 #+begin_listing clojure 2874 #+begin_listing clojure
2886 #+begin_src clojure 2875 #+begin_src clojure
2887 (defn infer-nils 2876 (defn infer-nils
2888 "Replace nils with the next available non-nil element in the 2877 "Replace nils with the next available non-nil element in the
2889 sequence, or barring that, 0." 2878 sequence, or barring that, 0."
2914 #+caption: Program to gather the worm's experiences into a vector for 2903 #+caption: Program to gather the worm's experiences into a vector for
2915 #+caption: further processing. The =motor-control-program= line uses 2904 #+caption: further processing. The =motor-control-program= line uses
2916 #+caption: a motor control script that causes the worm to execute a series 2905 #+caption: a motor control script that causes the worm to execute a series
2917 #+caption: of ``exercices'' that include all the action predicates. 2906 #+caption: of ``exercices'' that include all the action predicates.
2918 #+name: generate-phi-space 2907 #+name: generate-phi-space
2919 #+attr_latex: [htpb]
2920 #+begin_listing clojure 2908 #+begin_listing clojure
2921 #+begin_src clojure 2909 #+begin_src clojure
2922 (def do-all-the-things 2910 (def do-all-the-things
2923 (concat 2911 (concat
2924 curl-script 2912 curl-script
2942 #+end_listing 2930 #+end_listing
2943 2931
2944 #+caption: Use longest thread and a phi-space generated from a short 2932 #+caption: Use longest thread and a phi-space generated from a short
2945 #+caption: exercise routine to interpret actions during free play. 2933 #+caption: exercise routine to interpret actions during free play.
2946 #+name: empathy-debug 2934 #+name: empathy-debug
2947 #+attr_latex: [htpb]
2948 #+begin_listing clojure 2935 #+begin_listing clojure
2949 #+begin_src clojure 2936 #+begin_src clojure
2950 (defn init [] 2937 (defn init []
2951 (def phi-space (generate-phi-space)) 2938 (def phi-space (generate-phi-space))
2952 (def phi-scan (gen-phi-scan phi-space))) 2939 (def phi-scan (gen-phi-scan phi-space)))
2991 action predicates as the real sensory experience. 2978 action predicates as the real sensory experience.
2992 2979
2993 #+caption: Determine how closely empathy approximates actual 2980 #+caption: Determine how closely empathy approximates actual
2994 #+caption: sensory data. 2981 #+caption: sensory data.
2995 #+name: test-empathy-accuracy 2982 #+name: test-empathy-accuracy
2996 #+attr_latex: [htpb]
2997 #+begin_listing clojure 2983 #+begin_listing clojure
2998 #+begin_src clojure 2984 #+begin_src clojure
2999 (def worm-action-label 2985 (def worm-action-label
3000 (juxt grand-circle? curled? wiggling?)) 2986 (juxt grand-circle? curled? wiggling?))
3001 2987
3032 73%. This is based on very limited worm experience. By training the 3018 73%. This is based on very limited worm experience. By training the
3033 worm for longer, the accuracy dramatically improves. 3019 worm for longer, the accuracy dramatically improves.
3034 3020
3035 #+caption: Program to generate \Phi-space using manual training. 3021 #+caption: Program to generate \Phi-space using manual training.
3036 #+name: manual-phi-space 3022 #+name: manual-phi-space
3037 #+attr_latex: [htpb]
3038 #+begin_listing clojure 3023 #+begin_listing clojure
3039 #+begin_src clojure 3024 #+begin_src clojure
3040 (defn init-interactive [] 3025 (defn init-interactive []
3041 (def phi-space 3026 (def phi-space
3042 (let [experiences (atom [])] 3027 (let [experiences (atom [])]
3219 deduce that the worm has six sides. Note that =learn-touch-regions= 3204 deduce that the worm has six sides. Note that =learn-touch-regions=
3220 would work just as well even if the worm's touch sense data were 3205 would work just as well even if the worm's touch sense data were
3221 completely scrambled. The cross shape is just for convienence. This 3206 completely scrambled. The cross shape is just for convienence. This
3222 example justifies the use of pre-defined touch regions in =EMPATH=. 3207 example justifies the use of pre-defined touch regions in =EMPATH=.
3223 3208
3224 * COMMENT Contributions 3209 * Contributions
3225 3210
3226 In this thesis you have seen the =CORTEX= system, a complete 3211 In this thesis you have seen the =CORTEX= system, a complete
3227 environment for creating simulated creatures. You have seen how to 3212 environment for creating simulated creatures. You have seen how to
3228 implement five senses including touch, proprioception, hearing, 3213 implement five senses including touch, proprioception, hearing,
3229 vision, and muscle tension. You have seen how to create new creatues 3214 vision, and muscle tension. You have seen how to create new creatues
3258 # - Sensory fleshing-out 3243 # - Sensory fleshing-out
3259 # - Classification 3244 # - Classification
3260 #+BEGIN_LaTeX 3245 #+BEGIN_LaTeX
3261 \appendix 3246 \appendix
3262 #+END_LaTeX 3247 #+END_LaTeX
3263 * COMMENT Appendix: =CORTEX= User Guide 3248 * Appendix: =CORTEX= User Guide
3264 3249
3265 Those who write a thesis should endeavor to make their code not only 3250 Those who write a thesis should endeavor to make their code not only
3266 accessable, but actually useable, as a way to pay back the community 3251 accessable, but actually useable, as a way to pay back the community
3267 that made the thesis possible in the first place. This thesis would 3252 that made the thesis possible in the first place. This thesis would
3268 not be possible without Free Software such as jMonkeyEngine3, 3253 not be possible without Free Software such as jMonkeyEngine3,
3545 3530
3546 - =(load-bullet)= :: unpack native libraries and initialize 3531 - =(load-bullet)= :: unpack native libraries and initialize
3547 blender. This function is required before other world building 3532 blender. This function is required before other world building
3548 functions are called. 3533 functions are called.
3549 3534
3550
3551 *** Creature Manipulation / Import 3535 *** Creature Manipulation / Import
3552 3536
3553 - =(body! creature)= :: give the creature a physical body. 3537 - =(body! creature)= :: give the creature a physical body.
3554 3538
3555 - =(vision! creature)= :: give the creature a sense of vision. 3539 - =(vision! creature)= :: give the creature a sense of vision.