#+title: =CORTEX=

 #+author: Robert McIntyre

 #+email: rlm@mit.edu

 #+description: Using embodied AI to facilitate Artificial Imagination.

 #+keywords: AI, clojure, embodiment

 

 * Embodiment is a critical component of Intelligence

 

 ** Recognizing actions in video is extremely difficult

    cat drinking, mimes, leaning, common sense

 

 ** Embodiment is the the right language for the job

 

    a new possibility for the question ``what is a chair?'' -- it's the

    feeling of your butt on something and your knees bent, with your

    back muscles and legs relaxed.

 

 ** =CORTEX= is a system for exploring embodiment

 

    Hand integration demo

 

 ** =CORTEX= solves recognition problems using empathy

    

    worm empathy demo

 

 ** Overview

 

 * Building =CORTEX=

 

 ** To explore embodiment, we need a world, body, and senses

 

 ** Because of Time, simulation is perferable to reality

 

 ** Video game engines are a great starting point

 

 ** Bodies are composed of segments connected by joints

 

 ** Eyes reuse standard video game components

 

 ** Hearing is hard; =CORTEX= does it right

 

 ** Touch uses hundreds of hair-like elements

 

 ** Proprioception is the force that makes everything ``real''

 

 ** Muscles are both effectors and sensors

 

 ** =CORTEX= brings complex creatures to life!

 

 ** =CORTEX= enables many possiblities for further research

 

 * Empathy in a simulated worm

 

 ** Embodiment factors action recognition into managable parts

 

 ** Action recognition is easy with a full gamut of senses

 

 ** Digression: bootstrapping with multiple senses

 

 ** \Phi-space describes the worm's experiences

 

 ** Empathy is the process of tracing though \Phi-space 

   

 ** Efficient action recognition via empathy

 

 * Contributions

   - Built =CORTEX=, a comprehensive platform for embodied AI

     experiments. Has many new features lacking in other systems, such

     as sound. Easy to model/create new creatures.

   - created a novel concept for action recognition by using artificial

     imagination. 

 

 * =CORTEX= User Guide

 

 

 

 In the second half of the thesis I develop a computational model of

 empathy, using =CORTEX= as a base. Empathy in this context is the

 ability to observe another creature and infer what sorts of sensations

 that creature is feeling. My empathy algorithm involves multiple

 phases. First is free-play, where the creature moves around and gains

 sensory experience. From this experience I construct a representation

 of the creature's sensory state space, which I call \phi-space. Using

 \phi-space, I construct an efficient function for enriching the

 limited data that comes from observing another creature with a full

 compliment of imagined sensory data based on previous experience. I

 can then use the imagined sensory data to recognize what the observed

 creature is doing and feeling, using straightforward embodied action

 predicates. This is all demonstrated with using a simple worm-like

 creature, and recognizing worm-actions based on limited data.

 

 Embodied representation using multiple senses such as touch,

 proprioception, and muscle tension turns out be be exceedingly

 efficient at describing body-centered actions. It is the ``right

 language for the job''. For example, it takes only around 5 lines of

 LISP code to describe the action of ``curling'' using embodied

 primitives. It takes about 8 lines to describe the seemingly

 complicated action of wiggling.