Mercurial > cortex
comparison thesis/cortex.org @ 509:81c845a91378
completed initial render at 81 pages, but images are unacceptably scattered.
author | Robert McIntyre <rlm@mit.edu> |
---|---|
date | Sun, 30 Mar 2014 01:22:23 -0400 |
parents | c11d3fc3e6f0 |
children | f639e2139ce2 |
comparison
equal
deleted
inserted
replaced
508:c11d3fc3e6f0 | 509:81c845a91378 |
---|---|
245 #+caption: Body-centerd actions are best expressed in a body-centered | 245 #+caption: Body-centerd actions are best expressed in a body-centered |
246 #+caption: language. This code detects when the worm has curled into a | 246 #+caption: language. This code detects when the worm has curled into a |
247 #+caption: full circle. Imagine how you would replicate this functionality | 247 #+caption: full circle. Imagine how you would replicate this functionality |
248 #+caption: using low-level pixel features such as HOG filters! | 248 #+caption: using low-level pixel features such as HOG filters! |
249 #+name: grand-circle-intro | 249 #+name: grand-circle-intro |
250 #+attr_latex: [htpb] | 250 #+begin_listing clojure |
251 #+begin_listing clojure | |
252 #+begin_src clojure | 251 #+begin_src clojure |
253 (defn grand-circle? | 252 (defn grand-circle? |
254 "Does the worm form a majestic circle (one end touching the other)?" | 253 "Does the worm form a majestic circle (one end touching the other)?" |
255 [experiences] | 254 [experiences] |
256 (and (curled? experiences) | 255 (and (curled? experiences) |
1376 another. | 1375 another. |
1377 | 1376 |
1378 #+caption: Program for extending =OpenAL= to support multiple | 1377 #+caption: Program for extending =OpenAL= to support multiple |
1379 #+caption: listeners via context copying/switching. | 1378 #+caption: listeners via context copying/switching. |
1380 #+name: sync-openal-sources | 1379 #+name: sync-openal-sources |
1381 #+begin_listing C | 1380 #+begin_listing c |
1382 #+BEGIN_SRC C | 1381 #+BEGIN_SRC c |
1383 void syncSources(ALsource *masterSource, ALsource *slaveSource, | 1382 void syncSources(ALsource *masterSource, ALsource *slaveSource, |
1384 ALCcontext *masterCtx, ALCcontext *slaveCtx){ | 1383 ALCcontext *masterCtx, ALCcontext *slaveCtx){ |
1385 ALuint master = masterSource->source; | 1384 ALuint master = masterSource->source; |
1386 ALuint slave = slaveSource->source; | 1385 ALuint slave = slaveSource->source; |
1387 ALCcontext *current = alcGetCurrentContext(); | 1386 ALCcontext *current = alcGetCurrentContext(); |
1558 #+caption: the ball is constantally emiting a pure tone of | 1557 #+caption: the ball is constantally emiting a pure tone of |
1559 #+caption: constant volume. As it approaches the cubes, they each | 1558 #+caption: constant volume. As it approaches the cubes, they each |
1560 #+caption: change color in response to the sound. | 1559 #+caption: change color in response to the sound. |
1561 #+name: sound-cubes. | 1560 #+name: sound-cubes. |
1562 #+ATTR_LaTeX: :width 10cm | 1561 #+ATTR_LaTeX: :width 10cm |
1563 [[./images/aurellem-gray.png]] | 1562 [[./images/java-hearing-test.png]] |
1564 | 1563 |
1565 This system of hearing has also been co-opted by the | 1564 This system of hearing has also been co-opted by the |
1566 jMonkeyEngine3 community and is used to record audio for demo | 1565 jMonkeyEngine3 community and is used to record audio for demo |
1567 videos. | 1566 videos. |
1568 | 1567 |
2439 - Inverse kinematics :: experiments in sense guided motor control | 2438 - Inverse kinematics :: experiments in sense guided motor control |
2440 are easy given =CORTEX='s support -- you can get right to the | 2439 are easy given =CORTEX='s support -- you can get right to the |
2441 hard control problems without worrying about physics or | 2440 hard control problems without worrying about physics or |
2442 senses. | 2441 senses. |
2443 | 2442 |
2444 * COMMENT Empathy in a simulated worm | 2443 * Empathy in a simulated worm |
2445 | 2444 |
2446 Here I develop a computational model of empathy, using =CORTEX= as a | 2445 Here I develop a computational model of empathy, using =CORTEX= as a |
2447 base. Empathy in this context is the ability to observe another | 2446 base. Empathy in this context is the ability to observe another |
2448 creature and infer what sorts of sensations that creature is | 2447 creature and infer what sorts of sensations that creature is |
2449 feeling. My empathy algorithm involves multiple phases. First is | 2448 feeling. My empathy algorithm involves multiple phases. First is |
2523 #+caption: of sensory experience, and only uses proprioceptive data. Even | 2522 #+caption: of sensory experience, and only uses proprioceptive data. Even |
2524 #+caption: this simple predicate, however, is automatically frame | 2523 #+caption: this simple predicate, however, is automatically frame |
2525 #+caption: independent and ignores vermopomorphic differences such as | 2524 #+caption: independent and ignores vermopomorphic differences such as |
2526 #+caption: worm textures and colors. | 2525 #+caption: worm textures and colors. |
2527 #+name: curled | 2526 #+name: curled |
2528 #+attr_latex: [htpb] | 2527 #+begin_listing clojure |
2529 #+begin_listing clojure | |
2530 #+begin_src clojure | 2528 #+begin_src clojure |
2531 (defn curled? | 2529 (defn curled? |
2532 "Is the worm curled up?" | 2530 "Is the worm curled up?" |
2533 [experiences] | 2531 [experiences] |
2534 (every? | 2532 (every? |
2539 #+end_listing | 2537 #+end_listing |
2540 | 2538 |
2541 #+caption: Program for summarizing the touch information in a patch | 2539 #+caption: Program for summarizing the touch information in a patch |
2542 #+caption: of skin. | 2540 #+caption: of skin. |
2543 #+name: touch-summary | 2541 #+name: touch-summary |
2544 #+attr_latex: [htpb] | 2542 #+begin_listing clojure |
2545 | |
2546 #+begin_listing clojure | |
2547 #+begin_src clojure | 2543 #+begin_src clojure |
2548 (defn contact | 2544 (defn contact |
2549 "Determine how much contact a particular worm segment has with | 2545 "Determine how much contact a particular worm segment has with |
2550 other objects. Returns a value between 0 and 1, where 1 is full | 2546 other objects. Returns a value between 0 and 1, where 1 is full |
2551 contact and 0 is no contact." | 2547 contact and 0 is no contact." |
2566 #+caption: uses a summary of the tactile information from the underbelly | 2562 #+caption: uses a summary of the tactile information from the underbelly |
2567 #+caption: of the worm, and is only true if every segment is touching the | 2563 #+caption: of the worm, and is only true if every segment is touching the |
2568 #+caption: floor. Note that this function contains no references to | 2564 #+caption: floor. Note that this function contains no references to |
2569 #+caption: proprioction at all. | 2565 #+caption: proprioction at all. |
2570 #+name: resting | 2566 #+name: resting |
2571 #+attr_latex: [htpb] | |
2572 #+begin_listing clojure | 2567 #+begin_listing clojure |
2573 #+begin_src clojure | 2568 #+begin_src clojure |
2574 (def worm-segment-bottom (rect-region [8 15] [14 22])) | 2569 (def worm-segment-bottom (rect-region [8 15] [14 22])) |
2575 | 2570 |
2576 (defn resting? | 2571 (defn resting? |
2586 #+caption: Program for detecting whether the worm is curled up into a | 2581 #+caption: Program for detecting whether the worm is curled up into a |
2587 #+caption: full circle. Here the embodied approach begins to shine, as | 2582 #+caption: full circle. Here the embodied approach begins to shine, as |
2588 #+caption: I am able to both use a previous action predicate (=curled?=) | 2583 #+caption: I am able to both use a previous action predicate (=curled?=) |
2589 #+caption: as well as the direct tactile experience of the head and tail. | 2584 #+caption: as well as the direct tactile experience of the head and tail. |
2590 #+name: grand-circle | 2585 #+name: grand-circle |
2591 #+attr_latex: [htpb] | |
2592 #+begin_listing clojure | 2586 #+begin_listing clojure |
2593 #+begin_src clojure | 2587 #+begin_src clojure |
2594 (def worm-segment-bottom-tip (rect-region [15 15] [22 22])) | 2588 (def worm-segment-bottom-tip (rect-region [15 15] [22 22])) |
2595 | 2589 |
2596 (def worm-segment-top-tip (rect-region [0 15] [7 22])) | 2590 (def worm-segment-top-tip (rect-region [0 15] [7 22])) |
2617 #+caption: wiggling this way also gives the worm an opportunity to learn | 2611 #+caption: wiggling this way also gives the worm an opportunity to learn |
2618 #+caption: and recognize ``frustrated wiggling'', where the worm tries to | 2612 #+caption: and recognize ``frustrated wiggling'', where the worm tries to |
2619 #+caption: wiggle but can't. Frustrated wiggling is very visually different | 2613 #+caption: wiggle but can't. Frustrated wiggling is very visually different |
2620 #+caption: from actual wiggling, but this definition gives it to us for free. | 2614 #+caption: from actual wiggling, but this definition gives it to us for free. |
2621 #+name: wiggling | 2615 #+name: wiggling |
2622 #+attr_latex: [htpb] | |
2623 #+begin_listing clojure | 2616 #+begin_listing clojure |
2624 #+begin_src clojure | 2617 #+begin_src clojure |
2625 (defn fft [nums] | 2618 (defn fft [nums] |
2626 (map | 2619 (map |
2627 #(.getReal %) | 2620 #(.getReal %) |
2658 all the worm's senses. | 2651 all the worm's senses. |
2659 | 2652 |
2660 #+caption: Use the action predicates defined earlier to report on | 2653 #+caption: Use the action predicates defined earlier to report on |
2661 #+caption: what the worm is doing while in simulation. | 2654 #+caption: what the worm is doing while in simulation. |
2662 #+name: report-worm-activity | 2655 #+name: report-worm-activity |
2663 #+attr_latex: [htpb] | |
2664 #+begin_listing clojure | 2656 #+begin_listing clojure |
2665 #+begin_src clojure | 2657 #+begin_src clojure |
2666 (defn debug-experience | 2658 (defn debug-experience |
2667 [experiences text] | 2659 [experiences text] |
2668 (cond | 2660 (cond |
2777 mistake. | 2769 mistake. |
2778 | 2770 |
2779 #+caption: Program to convert an experience vector into a | 2771 #+caption: Program to convert an experience vector into a |
2780 #+caption: proprioceptively binned lookup function. | 2772 #+caption: proprioceptively binned lookup function. |
2781 #+name: bin | 2773 #+name: bin |
2782 #+attr_latex: [htpb] | |
2783 #+begin_listing clojure | 2774 #+begin_listing clojure |
2784 #+begin_src clojure | 2775 #+begin_src clojure |
2785 (defn bin [digits] | 2776 (defn bin [digits] |
2786 (fn [angles] | 2777 (fn [angles] |
2787 (->> angles | 2778 (->> angles |
2836 | 2827 |
2837 #+caption: Program to calculate empathy by tracing though \Phi-space | 2828 #+caption: Program to calculate empathy by tracing though \Phi-space |
2838 #+caption: and finding the longest (ie. most coherent) interpretation | 2829 #+caption: and finding the longest (ie. most coherent) interpretation |
2839 #+caption: of the data. | 2830 #+caption: of the data. |
2840 #+name: longest-thread | 2831 #+name: longest-thread |
2841 #+attr_latex: [htpb] | |
2842 #+begin_listing clojure | 2832 #+begin_listing clojure |
2843 #+begin_src clojure | 2833 #+begin_src clojure |
2844 (defn longest-thread | 2834 (defn longest-thread |
2845 "Find the longest thread from phi-index-sets. The index sets should | 2835 "Find the longest thread from phi-index-sets. The index sets should |
2846 be ordered from most recent to least recent." | 2836 be ordered from most recent to least recent." |
2879 fill in the gaps. | 2869 fill in the gaps. |
2880 | 2870 |
2881 #+caption: Fill in blanks in sensory experience by replicating the most | 2871 #+caption: Fill in blanks in sensory experience by replicating the most |
2882 #+caption: recent experience. | 2872 #+caption: recent experience. |
2883 #+name: infer-nils | 2873 #+name: infer-nils |
2884 #+attr_latex: [htpb] | |
2885 #+begin_listing clojure | 2874 #+begin_listing clojure |
2886 #+begin_src clojure | 2875 #+begin_src clojure |
2887 (defn infer-nils | 2876 (defn infer-nils |
2888 "Replace nils with the next available non-nil element in the | 2877 "Replace nils with the next available non-nil element in the |
2889 sequence, or barring that, 0." | 2878 sequence, or barring that, 0." |
2914 #+caption: Program to gather the worm's experiences into a vector for | 2903 #+caption: Program to gather the worm's experiences into a vector for |
2915 #+caption: further processing. The =motor-control-program= line uses | 2904 #+caption: further processing. The =motor-control-program= line uses |
2916 #+caption: a motor control script that causes the worm to execute a series | 2905 #+caption: a motor control script that causes the worm to execute a series |
2917 #+caption: of ``exercices'' that include all the action predicates. | 2906 #+caption: of ``exercices'' that include all the action predicates. |
2918 #+name: generate-phi-space | 2907 #+name: generate-phi-space |
2919 #+attr_latex: [htpb] | |
2920 #+begin_listing clojure | 2908 #+begin_listing clojure |
2921 #+begin_src clojure | 2909 #+begin_src clojure |
2922 (def do-all-the-things | 2910 (def do-all-the-things |
2923 (concat | 2911 (concat |
2924 curl-script | 2912 curl-script |
2942 #+end_listing | 2930 #+end_listing |
2943 | 2931 |
2944 #+caption: Use longest thread and a phi-space generated from a short | 2932 #+caption: Use longest thread and a phi-space generated from a short |
2945 #+caption: exercise routine to interpret actions during free play. | 2933 #+caption: exercise routine to interpret actions during free play. |
2946 #+name: empathy-debug | 2934 #+name: empathy-debug |
2947 #+attr_latex: [htpb] | |
2948 #+begin_listing clojure | 2935 #+begin_listing clojure |
2949 #+begin_src clojure | 2936 #+begin_src clojure |
2950 (defn init [] | 2937 (defn init [] |
2951 (def phi-space (generate-phi-space)) | 2938 (def phi-space (generate-phi-space)) |
2952 (def phi-scan (gen-phi-scan phi-space))) | 2939 (def phi-scan (gen-phi-scan phi-space))) |
2991 action predicates as the real sensory experience. | 2978 action predicates as the real sensory experience. |
2992 | 2979 |
2993 #+caption: Determine how closely empathy approximates actual | 2980 #+caption: Determine how closely empathy approximates actual |
2994 #+caption: sensory data. | 2981 #+caption: sensory data. |
2995 #+name: test-empathy-accuracy | 2982 #+name: test-empathy-accuracy |
2996 #+attr_latex: [htpb] | |
2997 #+begin_listing clojure | 2983 #+begin_listing clojure |
2998 #+begin_src clojure | 2984 #+begin_src clojure |
2999 (def worm-action-label | 2985 (def worm-action-label |
3000 (juxt grand-circle? curled? wiggling?)) | 2986 (juxt grand-circle? curled? wiggling?)) |
3001 | 2987 |
3032 73%. This is based on very limited worm experience. By training the | 3018 73%. This is based on very limited worm experience. By training the |
3033 worm for longer, the accuracy dramatically improves. | 3019 worm for longer, the accuracy dramatically improves. |
3034 | 3020 |
3035 #+caption: Program to generate \Phi-space using manual training. | 3021 #+caption: Program to generate \Phi-space using manual training. |
3036 #+name: manual-phi-space | 3022 #+name: manual-phi-space |
3037 #+attr_latex: [htpb] | |
3038 #+begin_listing clojure | 3023 #+begin_listing clojure |
3039 #+begin_src clojure | 3024 #+begin_src clojure |
3040 (defn init-interactive [] | 3025 (defn init-interactive [] |
3041 (def phi-space | 3026 (def phi-space |
3042 (let [experiences (atom [])] | 3027 (let [experiences (atom [])] |
3219 deduce that the worm has six sides. Note that =learn-touch-regions= | 3204 deduce that the worm has six sides. Note that =learn-touch-regions= |
3220 would work just as well even if the worm's touch sense data were | 3205 would work just as well even if the worm's touch sense data were |
3221 completely scrambled. The cross shape is just for convienence. This | 3206 completely scrambled. The cross shape is just for convienence. This |
3222 example justifies the use of pre-defined touch regions in =EMPATH=. | 3207 example justifies the use of pre-defined touch regions in =EMPATH=. |
3223 | 3208 |
3224 * COMMENT Contributions | 3209 * Contributions |
3225 | 3210 |
3226 In this thesis you have seen the =CORTEX= system, a complete | 3211 In this thesis you have seen the =CORTEX= system, a complete |
3227 environment for creating simulated creatures. You have seen how to | 3212 environment for creating simulated creatures. You have seen how to |
3228 implement five senses including touch, proprioception, hearing, | 3213 implement five senses including touch, proprioception, hearing, |
3229 vision, and muscle tension. You have seen how to create new creatues | 3214 vision, and muscle tension. You have seen how to create new creatues |
3258 # - Sensory fleshing-out | 3243 # - Sensory fleshing-out |
3259 # - Classification | 3244 # - Classification |
3260 #+BEGIN_LaTeX | 3245 #+BEGIN_LaTeX |
3261 \appendix | 3246 \appendix |
3262 #+END_LaTeX | 3247 #+END_LaTeX |
3263 * COMMENT Appendix: =CORTEX= User Guide | 3248 * Appendix: =CORTEX= User Guide |
3264 | 3249 |
3265 Those who write a thesis should endeavor to make their code not only | 3250 Those who write a thesis should endeavor to make their code not only |
3266 accessable, but actually useable, as a way to pay back the community | 3251 accessable, but actually useable, as a way to pay back the community |
3267 that made the thesis possible in the first place. This thesis would | 3252 that made the thesis possible in the first place. This thesis would |
3268 not be possible without Free Software such as jMonkeyEngine3, | 3253 not be possible without Free Software such as jMonkeyEngine3, |
3545 | 3530 |
3546 - =(load-bullet)= :: unpack native libraries and initialize | 3531 - =(load-bullet)= :: unpack native libraries and initialize |
3547 blender. This function is required before other world building | 3532 blender. This function is required before other world building |
3548 functions are called. | 3533 functions are called. |
3549 | 3534 |
3550 | |
3551 *** Creature Manipulation / Import | 3535 *** Creature Manipulation / Import |
3552 | 3536 |
3553 - =(body! creature)= :: give the creature a physical body. | 3537 - =(body! creature)= :: give the creature a physical body. |
3554 | 3538 |
3555 - =(vision! creature)= :: give the creature a sense of vision. | 3539 - =(vision! creature)= :: give the creature a sense of vision. |