# HG changeset patch # User Robert McIntyre # Date 1395816121 14400 # Node ID 09b7c8dd436562d5a817fec546b44e861353c3fe # Parent af13fc73e851e57981d9d9c2f840fb4994101cfd first chapter done, half of last chapter done. diff -r af13fc73e851 -r 09b7c8dd4365 org/worm_learn.clj --- a/org/worm_learn.clj Tue Mar 25 22:54:41 2014 -0400 +++ b/org/worm_learn.clj Wed Mar 26 02:42:01 2014 -0400 @@ -27,6 +27,13 @@ (defn worm-model [] (load-blender-model "Models/worm/worm.blend")) +(defn worm [] + (let [model (load-blender-model "Models/worm/worm.blend")] + {:body (doto model (body!)) + :touch (touch! model) + :proprioception (proprioception! model) + :muscles (movement! model)})) + (def output-base (File. "/home/r/proj/cortex/render/worm-learn/curl")) @@ -220,15 +227,8 @@ (< 0.55 (contact worm-segment-top-tip head-touch)))))) -(declare phi-space phi-scan) +(declare phi-space phi-scan debug-experience) -(defn debug-experience - [experiences text] - (cond - (grand-circle? experiences) (.setText text "Grand Circle") - (curled? experiences) (.setText text "Curled") - (wiggling? experiences) (.setText text "Wiggling") - (resting? experiences) (.setText text "Resting"))) (def standard-world-view @@ -277,16 +277,14 @@ (.setFilterMode PssmShadowRenderer$FilterMode/Bilinear))] (.addProcessor (.getViewPort world) pssm))) +(defn debug-experience + [experiences text] + (cond + (grand-circle? experiences) (.setText text "Grand Circle") + (curled? experiences) (.setText text "Curled") + (wiggling? experiences) (.setText text "Wiggling") + (resting? experiences) (.setText text "Resting"))) -(defn display-text [[x y :as location]] - (let [] - (.setLocalTranslation text 300 (.getLineHeight text) 0) - (fn [world] - - - - - (fn [new-text] (defn worm-world [& {:keys [record motor-control keybindings view experiences @@ -294,14 +292,11 @@ (let [{:keys [record motor-control keybindings view experiences worm-model end-frame experience-watch]} (merge (worm-world-defaults) settings) - worm (doto (worm-model) (body!)) - touch (touch! worm) - prop (proprioception! worm) - muscles (movement! worm) - + touch-display (view-touch) prop-display (view-proprioception) muscle-display (view-movement) + {:keys [proprioception touch muscles body]} (worm) floor (box 5 1 5 :position (Vector3f. 0 -10 0) @@ -316,7 +311,7 @@ (.setColor (ColorRGBA/Black)))] (world - (nodify [worm floor]) + (nodify [body floor]) (merge standard-debug-controls keybindings) (fn [world] (.setLocalTranslation @@ -324,7 +319,7 @@ (.attachChild (.getGuiNode world) worm-action) (enable-good-shadows world) - (.setShadowMode worm RenderQueue$ShadowMode/CastAndReceive) + (.setShadowMode body RenderQueue$ShadowMode/CastAndReceive) (.setShadowMode floor RenderQueue$ShadowMode/Receive) (.setBackgroundColor (.getViewPort world) (ColorRGBA/White)) @@ -332,7 +327,7 @@ (.setDisplayFps world false) (position-camera world view) (.setTimer world timer) - (display-dilated-time world timer) + ;;(display-dilated-time world timer) (when record (dir! record) (Capture/captureVideo @@ -345,7 +340,7 @@ (if (and end-frame (> (.getTime timer) end-frame)) (.stop world)) (let [muscle-data (vec (motor-control muscles)) - proprioception-data (prop) + proprioception-data (proprioception) touch-data (mapv #(% (.getRootNode world)) touch)] (when experiences (record-experience! diff -r af13fc73e851 -r 09b7c8dd4365 thesis/cortex.org --- a/thesis/cortex.org Tue Mar 25 22:54:41 2014 -0400 +++ b/thesis/cortex.org Wed Mar 26 02:42:01 2014 -0400 @@ -226,7 +226,7 @@ #+end_listing -** =CORTEX= is a toolkit for building sensate creatures +** =CORTEX= is a toolkit for building sensate creatures I built =CORTEX= to be a general AI research platform for doing experiments involving multiple rich senses and a wide variety and @@ -269,14 +269,16 @@ engine designed to create cross-platform 3D desktop games. =CORTEX= is mainly written in clojure, a dialect of =LISP= that runs on the java virtual machine (JVM). The API for creating and simulating - creatures is entirely expressed in clojure. Hearing is implemented - as a layer of clojure code on top of a layer of java code on top of - a layer of =C++= code which implements a modified version of - =OpenAL= to support multiple listeners. =CORTEX= is the only - simulation environment that I know of that can support multiple - entities that can each hear the world from their own perspective. - Other senses also require a small layer of Java code. =CORTEX= also - uses =bullet=, a physics simulator written in =C=. + creatures and senses is entirely expressed in clojure, though many + senses are implemented at the layer of jMonkeyEngine or below. For + example, for the sense of hearing I use a layer of clojure code on + top of a layer of java JNI bindings that drive a layer of =C++= + code which implements a modified version of =OpenAL= to support + multiple listeners. =CORTEX= is the only simulation environment + that I know of that can support multiple entities that can each + hear the world from their own perspective. Other senses also + require a small layer of Java code. =CORTEX= also uses =bullet=, a + physics simulator written in =C=. #+caption: Here is the worm from above modeled in Blender, a free #+caption: 3D-modeling program. Senses and joints are described @@ -285,26 +287,46 @@ #+ATTR_LaTeX: :width 12cm [[./images/blender-worm.png]] + Here are some thing I anticipate that =CORTEX= might be used for: + + - exploring new ideas about sensory integration + - distributed communication among swarm creatures + - self-learning using free exploration, + - evolutionary algorithms involving creature construction + - exploration of exoitic senses and effectors that are not possible + in the real world (such as telekenisis or a semantic sense) + - imagination using subworlds + During one test with =CORTEX=, I created 3,000 entities each with their own independent senses and ran them all at only 1/80 real time. In another test, I created a detailed model of my own hand, equipped with a realistic distribution of touch (more sensitive at the fingertips), as well as eyes and ears, and it ran at around 1/4 - real time. + real time. - #+caption: Here is the worm from above modeled in Blender, a free - #+caption: 3D-modeling program. Senses and joints are described - #+caption: using special nodes in Blender. - #+name: worm-recognition-intro - #+ATTR_LaTeX: :width 15cm - [[./images/full-hand.png]] - - - + #+BEGIN_LaTeX + \begin{sidewaysfigure} + \includegraphics[width=9.5in]{images/full-hand.png} + \caption{Here is the worm from above modeled in Blender, + a free 3D-modeling program. Senses and joints are described + using special nodes in Blender. The senses are displayed on + the right, and the simulation is displayed on the left. Notice + that the hand is curling its fingers, that it can see its own + finger from the eye in its palm, and thta it can feel its own + thumb touching its palm.} + \end{sidewaysfigure} + #+END_LaTeX - ** Contributions + I built =CORTEX=, a comprehensive platform for embodied AI + experiments. =CORTEX= many new features lacking in other systems, + such as sound. It is easy to create new creatures using Blender, a + free 3D modeling program. + + I built =EMPATH=, which uses =CORTEX= to identify the actions of a + worm-like creature using a computational model of empathy. + * Building =CORTEX= ** To explore embodiment, we need a world, body, and senses @@ -331,52 +353,409 @@ * Empathy in a simulated worm + Here I develop a computational model of empathy, using =CORTEX= as a + base. Empathy in this context is the ability to observe another + creature and infer what sorts of sensations that creature is + feeling. My empathy algorithm involves multiple phases. First is + free-play, where the creature moves around and gains sensory + experience. From this experience I construct a representation of the + creature's sensory state space, which I call \Phi-space. Using + \Phi-space, I construct an efficient function which takes the + limited data that comes from observing another creature and enriches + it full compliment of imagined sensory data. I can then use the + imagined sensory data to recognize what the observed creature is + doing and feeling, using straightforward embodied action predicates. + This is all demonstrated with using a simple worm-like creature, and + recognizing worm-actions based on limited data. + + #+caption: Here is the worm with which we will be working. + #+caption: It is composed of 5 segments. Each segment has a + #+caption: pair of extensor and flexor muscles. Each of the + #+caption: worm's four joints is a hinge joint which allows + #+caption: 30 degrees of rotation to either side. Each segment + #+caption: of the worm is touch-capable and has a uniform + #+caption: distribution of touch sensors on each of its faces. + #+caption: Each joint has a proprioceptive sense to detect + #+caption: relative positions. The worm segments are all the + #+caption: same except for the first one, which has a much + #+caption: higher weight than the others to allow for easy + #+caption: manual motor control. + #+name: basic-worm-view + #+ATTR_LaTeX: :width 10cm + [[./images/basic-worm-view.png]] + + #+caption: Program for reading a worm from a blender file and + #+caption: outfitting it with the senses of proprioception, + #+caption: touch, and the ability to move, as specified in the + #+caption: blender file. + #+name: get-worm + #+begin_listing clojure + #+begin_src clojure +(defn worm [] + (let [model (load-blender-model "Models/worm/worm.blend")] + {:body (doto model (body!)) + :touch (touch! model) + :proprioception (proprioception! model) + :muscles (movement! model)})) + #+end_src + #+end_listing + ** Embodiment factors action recognition into managable parts + Using empathy, I divide the problem of action recognition into a + recognition process expressed in the language of a full compliment + of senses, and an imaganitive process that generates full sensory + data from partial sensory data. Splitting the action recognition + problem in this manner greatly reduces the total amount of work to + recognize actions: The imaganitive process is mostly just matching + previous experience, and the recognition process gets to use all + the senses to directly describe any action. + ** Action recognition is easy with a full gamut of senses -** Digression: bootstrapping touch using free exploration + Embodied representations using multiple senses such as touch, + proprioception, and muscle tension turns out be be exceedingly + efficient at describing body-centered actions. It is the ``right + language for the job''. For example, it takes only around 5 lines + of LISP code to describe the action of ``curling'' using embodied + primitives. It takes about 8 lines to describe the seemingly + complicated action of wiggling. + + The following action predicates each take a stream of sensory + experience, observe however much of it they desire, and decide + whether the worm is doing the action they describe. =curled?= + relies on proprioception, =resting?= relies on touch, =wiggling?= + relies on a fourier analysis of muscle contraction, and + =grand-circle?= relies on touch and reuses =curled?= as a gaurd. + + #+caption: Program for detecting whether the worm is curled. This is the + #+caption: simplest action predicate, because it only uses the last frame + #+caption: of sensory experience, and only uses proprioceptive data. Even + #+caption: this simple predicate, however, is automatically frame + #+caption: independent and ignores vermopomorphic differences such as + #+caption: worm textures and colors. + #+name: curled + #+begin_listing clojure + #+begin_src clojure +(defn curled? + "Is the worm curled up?" + [experiences] + (every? + (fn [[_ _ bend]] + (> (Math/sin bend) 0.64)) + (:proprioception (peek experiences)))) + #+end_src + #+end_listing + + #+caption: Program for summarizing the touch information in a patch + #+caption: of skin. + #+name: touch-summary + #+begin_listing clojure + #+begin_src clojure +(defn contact + "Determine how much contact a particular worm segment has with + other objects. Returns a value between 0 and 1, where 1 is full + contact and 0 is no contact." + [touch-region [coords contact :as touch]] + (-> (zipmap coords contact) + (select-keys touch-region) + (vals) + (#(map first %)) + (average) + (* 10) + (- 1) + (Math/abs))) + #+end_src + #+end_listing + + + #+caption: Program for detecting whether the worm is at rest. This program + #+caption: uses a summary of the tactile information from the underbelly + #+caption: of the worm, and is only true if every segment is touching the + #+caption: floor. Note that this function contains no references to + #+caption: proprioction at all. + #+name: resting + #+begin_listing clojure + #+begin_src clojure +(def worm-segment-bottom (rect-region [8 15] [14 22])) + +(defn resting? + "Is the worm resting on the ground?" + [experiences] + (every? + (fn [touch-data] + (< 0.9 (contact worm-segment-bottom touch-data))) + (:touch (peek experiences)))) + #+end_src + #+end_listing + + #+caption: Program for detecting whether the worm is curled up into a + #+caption: full circle. Here the embodied approach begins to shine, as + #+caption: I am able to both use a previous action predicate (=curled?=) + #+caption: as well as the direct tactile experience of the head and tail. + #+name: grand-circle + #+begin_listing clojure + #+begin_src clojure +(def worm-segment-bottom-tip (rect-region [15 15] [22 22])) + +(def worm-segment-top-tip (rect-region [0 15] [7 22])) + +(defn grand-circle? + "Does the worm form a majestic circle (one end touching the other)?" + [experiences] + (and (curled? experiences) + (let [worm-touch (:touch (peek experiences)) + tail-touch (worm-touch 0) + head-touch (worm-touch 4)] + (and (< 0.55 (contact worm-segment-bottom-tip tail-touch)) + (< 0.55 (contact worm-segment-top-tip head-touch)))))) + #+end_src + #+end_listing + + + #+caption: Program for detecting whether the worm has been wiggling for + #+caption: the last few frames. It uses a fourier analysis of the muscle + #+caption: contractions of the worm's tail to determine wiggling. This is + #+caption: signigicant because there is no particular frame that clearly + #+caption: indicates that the worm is wiggling --- only when multiple frames + #+caption: are analyzed together is the wiggling revealed. Defining + #+caption: wiggling this way also gives the worm an opportunity to learn + #+caption: and recognize ``frustrated wiggling'', where the worm tries to + #+caption: wiggle but can't. Frustrated wiggling is very visually different + #+caption: from actual wiggling, but this definition gives it to us for free. + #+name: wiggling + #+begin_listing clojure + #+begin_src clojure +(defn fft [nums] + (map + #(.getReal %) + (.transform + (FastFourierTransformer. DftNormalization/STANDARD) + (double-array nums) TransformType/FORWARD))) + +(def indexed (partial map-indexed vector)) + +(defn max-indexed [s] + (first (sort-by (comp - second) (indexed s)))) + +(defn wiggling? + "Is the worm wiggling?" + [experiences] + (let [analysis-interval 0x40] + (when (> (count experiences) analysis-interval) + (let [a-flex 3 + a-ex 2 + muscle-activity + (map :muscle (vector:last-n experiences analysis-interval)) + base-activity + (map #(- (% a-flex) (% a-ex)) muscle-activity)] + (= 2 + (first + (max-indexed + (map #(Math/abs %) + (take 20 (fft base-activity)))))))))) + #+end_src + #+end_listing + + With these action predicates, I can now recognize the actions of + the worm while it is moving under my control and I have access to + all the worm's senses. + + #+caption: Use the action predicates defined earlier to report on + #+caption: what the worm is doing while in simulation. + #+name: report-worm-activity + #+begin_listing clojure + #+begin_src clojure +(defn debug-experience + [experiences text] + (cond + (grand-circle? experiences) (.setText text "Grand Circle") + (curled? experiences) (.setText text "Curled") + (wiggling? experiences) (.setText text "Wiggling") + (resting? experiences) (.setText text "Resting"))) + #+end_src + #+end_listing + + #+caption: Using =debug-experience=, the body-centered predicates + #+caption: work together to classify the behaviour of the worm. + #+caption: while under manual motor control. + #+name: basic-worm-view + #+ATTR_LaTeX: :width 10cm + [[./images/worm-identify-init.png]] + + These action predicates satisfy the recognition requirement of an + empathic recognition system. There is a lot of power in the + simplicity of the action predicates. They describe their actions + without getting confused in visual details of the worm. Each one is + frame independent, but more than that, they are each indepent of + irrelevant visual details of the worm and the environment. They + will work regardless of whether the worm is a different color or + hevaily textured, or of the environment has strange lighting. + + The trick now is to make the action predicates work even when the + sensory data on which they depend is absent. If I can do that, then + I will have gained much, ** \Phi-space describes the worm's experiences + + As a first step towards building empathy, I need to gather all of + the worm's experiences during free play. I use a simple vector to + store all the experiences. + + #+caption: Program to gather the worm's experiences into a vector for + #+caption: further processing. The =motor-control-program= line uses + #+caption: a motor control script that causes the worm to execute a series + #+caption: of ``exercices'' that include all the action predicates. + #+name: generate-phi-space + #+begin_listing clojure + #+begin_src clojure +(defn generate-phi-space [] + (let [experiences (atom [])] + (run-world + (apply-map + worm-world + (merge + (worm-world-defaults) + {:end-frame 700 + :motor-control + (motor-control-program worm-muscle-labels do-all-the-things) + :experiences experiences}))) + @experiences)) + #+end_src + #+end_listing + + Each element of the experience vector exists in the vast space of + all possible worm-experiences. Most of this vast space is actually + unreachable due to physical constraints of the worm's body. For + example, the worm's segments are connected by hinge joints that put + a practical limit on the worm's degrees of freedom. Also, the worm + can not be bent into a circle so that its ends are touching and at + the same time not also experience the sensation of touching itself. + + As the worm moves around during free play and the vector grows + larger, the vector begins to define a subspace which is all the + practical experiences the worm can experience during normal + operation, which I call \Phi-space, short for physical-space. The + vector defines a path through \Phi-space. This path has interesting + properties that all derive from embodiment. The proprioceptive + components are completely smooth, because in order for the worm to + move from one position to another, it must pass through the + intermediate positions. The path invariably forms loops as actions + are repeated. Finally and most importantly, proprioception actually + gives very strong inference about the other senses. For example, + when the worm is flat, you can infer that it is touching the ground + and that its muscles are not active, because if the muscles were + active, the worm would be moving and would not be perfectly flat. + In order to stay flat, the worm has to be touching the ground, or + it would again be moving out of the flat position due to gravity. + If the worm is positioned in such a way that it interacts with + itself, then it is very likely to be feeling the same tactile + feelings as the last time it was in that position, because it has + the same body as then. If you observe multiple frames of + proprioceptive data, then you can become increasingly confident + about the exact activations of the worm's muscles, because it + generally takes a unique combination of muscle contractions to + transform the worm's body along a specific path through \Phi-space. + + There is a simple way of taking \Phi-space and the total ordering + provided by an experience vector and reliably infering the rest of + the senses. ** Empathy is the process of tracing though \Phi-space + + + +(defn bin [digits] + (fn [angles] + (->> angles + (flatten) + (map (juxt #(Math/sin %) #(Math/cos %))) + (flatten) + (mapv #(Math/round (* % (Math/pow 10 (dec digits)))))))) + +(defn gen-phi-scan +"Nearest-neighbors with spatial binning. Only returns a result if + the propriceptive data is within 10% of a previously recorded + result in all dimensions." + +[phi-space] + (let [bin-keys (map bin [3 2 1]) + bin-maps + (map (fn [bin-key] + (group-by + (comp bin-key :proprioception phi-space) + (range (count phi-space)))) bin-keys) + lookups (map (fn [bin-key bin-map] + (fn [proprio] (bin-map (bin-key proprio)))) + bin-keys bin-maps)] + (fn lookup [proprio-data] + (set (some #(% proprio-data) lookups))))) + + +(defn longest-thread + "Find the longest thread from phi-index-sets. The index sets should + be ordered from most recent to least recent." + [phi-index-sets] + (loop [result '() + [thread-bases & remaining :as phi-index-sets] phi-index-sets] + (if (empty? phi-index-sets) + (vec result) + (let [threads + (for [thread-base thread-bases] + (loop [thread (list thread-base) + remaining remaining] + (let [next-index (dec (first thread))] + (cond (empty? remaining) thread + (contains? (first remaining) next-index) + (recur + (cons next-index thread) (rest remaining)) + :else thread)))) + longest-thread + (reduce (fn [thread-a thread-b] + (if (> (count thread-a) (count thread-b)) + thread-a thread-b)) + '(nil) + threads)] + (recur (concat longest-thread result) + (drop (count longest-thread) phi-index-sets)))))) + +There is one final piece, which is to replace missing sensory data +with a best-guess estimate. While I could fill in missing data by +using a gradient over the closest known sensory data points, averages +can be misleading. It is certainly possible to create an impossible +sensory state by averaging two possible sensory states. Therefore, I +simply replicate the most recent sensory experience to fill in the +gaps. + + #+caption: Fill in blanks in sensory experience by replicating the most + #+caption: recent experience. + #+name: infer-nils + #+begin_listing clojure + #+begin_src clojure +(defn infer-nils + "Replace nils with the next available non-nil element in the + sequence, or barring that, 0." + [s] + (loop [i (dec (count s)) + v (transient s)] + (if (zero? i) (persistent! v) + (if-let [cur (v i)] + (if (get v (dec i) 0) + (recur (dec i) v) + (recur (dec i) (assoc! v (dec i) cur))) + (recur i (assoc! v i 0)))))) + #+end_src + #+end_listing + + + + ** Efficient action recognition with =EMPATH= +** Digression: bootstrapping touch using free exploration + * Contributions - - Built =CORTEX=, a comprehensive platform for embodied AI - experiments. Has many new features lacking in other systems, such - as sound. Easy to model/create new creatures. - - created a novel concept for action recognition by using artificial - imagination. - -In the second half of the thesis I develop a computational model of -empathy, using =CORTEX= as a base. Empathy in this context is the -ability to observe another creature and infer what sorts of sensations -that creature is feeling. My empathy algorithm involves multiple -phases. First is free-play, where the creature moves around and gains -sensory experience. From this experience I construct a representation -of the creature's sensory state space, which I call \Phi-space. Using -\Phi-space, I construct an efficient function for enriching the -limited data that comes from observing another creature with a full -compliment of imagined sensory data based on previous experience. I -can then use the imagined sensory data to recognize what the observed -creature is doing and feeling, using straightforward embodied action -predicates. This is all demonstrated with using a simple worm-like -creature, and recognizing worm-actions based on limited data. - -Embodied representation using multiple senses such as touch, -proprioception, and muscle tension turns out be be exceedingly -efficient at describing body-centered actions. It is the ``right -language for the job''. For example, it takes only around 5 lines of -LISP code to describe the action of ``curling'' using embodied -primitives. It takes about 8 lines to describe the seemingly -complicated action of wiggling. - - - -* COMMENT names for cortex - - bioland diff -r af13fc73e851 -r 09b7c8dd4365 thesis/images/basic-worm-view.png Binary file thesis/images/basic-worm-view.png has changed diff -r af13fc73e851 -r 09b7c8dd4365 thesis/images/full-hand.png Binary file thesis/images/full-hand.png has changed diff -r af13fc73e851 -r 09b7c8dd4365 thesis/images/worm-identify-init.png Binary file thesis/images/worm-identify-init.png has changed diff -r af13fc73e851 -r 09b7c8dd4365 thesis/rlm-cortex-meng.tex --- a/thesis/rlm-cortex-meng.tex Tue Mar 25 22:54:41 2014 -0400 +++ b/thesis/rlm-cortex-meng.tex Wed Mar 26 02:42:01 2014 -0400 @@ -43,7 +43,7 @@ \usepackage{hyperref} \usepackage{libertine} \usepackage{inconsolata} - +\usepackage{rotating} \usepackage[backend=bibtex,style=alphabetic]{biblatex} \addbibresource{cortex.bib}