# HG changeset patch # User Robert McIntyre # Date 1395627644 14400 # Node ID c1e6b7221b2fb3daaa4085045be597609d48b424 # Parent 853377051f1e61fc282d55232d31c0fb56fbe5b6 progress on intro. diff -r 853377051f1e -r c1e6b7221b2f thesis/Makefile --- a/thesis/Makefile Sun Mar 23 19:09:14 2014 -0400 +++ b/thesis/Makefile Sun Mar 23 22:20:44 2014 -0400 @@ -7,7 +7,7 @@ rsync -avz --delete /home/r/proj/cortex/thesis "r@aurellem.org:~" ssh r@aurellem.org cd "~/thesis; $(INVOKE_LATEX)" scp "r@aurellem.org:/home/r/thesis/$(THESIS_NAME).pdf" . - rm cortex.tex abstract.tex + rm cortex.tex abstract.tex user-guide.tex diff -r 853377051f1e -r c1e6b7221b2f thesis/abstract.org --- a/thesis/abstract.org Sun Mar 23 19:09:14 2014 -0400 +++ b/thesis/abstract.org Sun Mar 23 22:20:44 2014 -0400 @@ -1,12 +1,12 @@ Here I demonstrate the power of using embodied artificial intelligence to attack the /action recognition/ problem, which is the challenge of recognizing actions performed by a creature given limited data about -the creature's actions, such as a video recording. I solve this problem -in the case of a worm-like creature performing actions such as curling -and wiggling. +the creature's actions, such as a video recording. I solve this +problem in the case of a worm-like creature performing actions such as +curling and wiggling. To attack the action recognition problem, I developed a computational -model of empathy which allows me to use simple, embodied +model of empathy (=EMPATH=) which allows me to use simple, embodied representations of actions (which require rich sensory data), even when that sensory data is not actually available. The missing sense data is ``imagined'' by the system by combining previous experiences diff -r 853377051f1e -r c1e6b7221b2f thesis/cortex.org --- a/thesis/cortex.org Sun Mar 23 19:09:14 2014 -0400 +++ b/thesis/cortex.org Sun Mar 23 22:20:44 2014 -0400 @@ -4,26 +4,102 @@ #+description: Using embodied AI to facilitate Artificial Imagination. #+keywords: AI, clojure, embodiment -* Embodiment is a critical component of Intelligence + +* Empathy and Embodiment as a problem solving strategy + + By the end of this thesis, you will have seen a novel approach to + interpreting video using embodiment and empathy. You will have also + seen one way to efficiently implement empathy for embodied + creatures. + + The core vision of this thesis is that one of the important ways in + which we understand others is by imagining ourselves in their + posistion and empathicaly feeling experiences based on our own past + experiences and imagination. + + By understanding events in terms of our own previous corperal + experience, we greatly constrain the possibilities of what would + otherwise be an unweidly exponential search. This extra constraint + can be the difference between easily understanding what is happening + in a video and being completely lost in a sea of incomprehensible + color and movement. ** Recognizing actions in video is extremely difficult + + Consider for example the problem of determining what is happening in + a video of which this is one frame: + + #+caption: A cat drinking some water. Identifying this action is beyond the state of the art for computers. + #+ATTR_LaTeX: :width 7cm + [[./images/cat-drinking.jpg]] + + It is currently impossible for any computer program to reliably + label such an video as "drinking". And rightly so -- it is a very + hard problem! What features can you describe in terms of low level + functions of pixels that can even begin to describe what is + happening here? + + Or suppose that you are building a program that recognizes + chairs. How could you ``see'' the chair in the following picture? + + #+caption: When you look at this, do you think ``chair''? I certainly do. + #+ATTR_LaTeX: :width 10cm + [[./images/invisible-chair.png]] + + #+caption: The chair in this image is quite obvious to humans, but I doubt any computer program can find it. + #+ATTR_LaTeX: :width 10cm + [[./images/fat-person-sitting-at-desk.jpg]] + + + I think humans are able to label + such video as "drinking" because they imagine /themselves/ as the + cat, and imagine putting their face up against a stream of water and + sticking out their tongue. In that imagined world, they can feel the + cool water hitting their tongue, and feel the water entering their + body, and are able to recognize that /feeling/ as drinking. So, the + label of the action is not really in the pixels of the image, but is + found clearly in a simulation inspired by those pixels. An + imaginative system, having been trained on drinking and non-drinking + examples and learning that the most important component of drinking + is the feeling of water sliding down one's throat, would analyze a + video of a cat drinking in the following manner: + + - Create a physical model of the video by putting a "fuzzy" model + of its own body in place of the cat. Also, create a simulation of + the stream of water. + + - Play out this simulated scene and generate imagined sensory + experience. This will include relevant muscle contractions, a + close up view of the stream from the cat's perspective, and most + importantly, the imagined feeling of water entering the mouth. + + - The action is now easily identified as drinking by the sense of + taste alone. The other senses (such as the tongue moving in and + out) help to give plausibility to the simulated action. Note that + the sense of vision, while critical in creating the simulation, + is not critical for identifying the action from the simulation. + + + + + + + cat drinking, mimes, leaning, common sense -** Embodiment is the the right language for the job +** =EMPATH= neatly solves recognition problems + + factorization , right language, etc a new possibility for the question ``what is a chair?'' -- it's the feeling of your butt on something and your knees bent, with your back muscles and legs relaxed. -** =CORTEX= is a system for exploring embodiment +** =CORTEX= is a toolkit for building sensate creatures Hand integration demo -** =CORTEX= solves recognition problems using empathy - - worm empathy demo - -** Overview +** Contributions * Building =CORTEX= @@ -55,7 +131,7 @@ ** Action recognition is easy with a full gamut of senses -** Digression: bootstrapping with multiple senses +** Digression: bootstrapping touch using free exploration ** \Phi-space describes the worm's experiences @@ -70,10 +146,6 @@ - created a novel concept for action recognition by using artificial imagination. -* =CORTEX= User Guide - - - In the second half of the thesis I develop a computational model of empathy, using =CORTEX= as a base. Empathy in this context is the ability to observe another creature and infer what sorts of sensations @@ -97,3 +169,7 @@ primitives. It takes about 8 lines to describe the seemingly complicated action of wiggling. + + +* COMMENT names for cortex + - bioland \ No newline at end of file diff -r 853377051f1e -r c1e6b7221b2f thesis/cover.tex --- a/thesis/cover.tex Sun Mar 23 19:09:14 2014 -0400 +++ b/thesis/cover.tex Sun Mar 23 22:20:44 2014 -0400 @@ -45,7 +45,7 @@ % however the specifications can change. We recommend that you verify the % layout of your title page with your thesis advisor and/or the MIT % Libraries before printing your final copy. -\title{CORTEX : A Virtual World for Sensate AI} +\title{Solving Problems using Embodiment \& Empathy.} \author{Robert Louis M\raisebox{\depth}{\small \underline{\underline{c}}}Intyre} %\author{Robert McIntyre} diff -r 853377051f1e -r c1e6b7221b2f thesis/images/fat-person-sitting-at-desk.jpg Binary file thesis/images/fat-person-sitting-at-desk.jpg has changed diff -r 853377051f1e -r c1e6b7221b2f thesis/images/invisible-chair.png Binary file thesis/images/invisible-chair.png has changed diff -r 853377051f1e -r c1e6b7221b2f thesis/rlm-cortex-meng.tex --- a/thesis/rlm-cortex-meng.tex Sun Mar 23 19:09:14 2014 -0400 +++ b/thesis/rlm-cortex-meng.tex Sun Mar 23 22:20:44 2014 -0400 @@ -100,7 +100,7 @@ %\bibliographystyle{agsm} %\bibliographystyle{apa} %\bibliographystyle{plainnat} - +\include{user-guide} \printbibliography \end{singlespace} \end{document} diff -r 853377051f1e -r c1e6b7221b2f thesis/user-guide.org --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/thesis/user-guide.org Sun Mar 23 22:20:44 2014 -0400 @@ -0,0 +1,6 @@ +* Appendix: =CORTEX= User Guide + + For future students who whould like to use =CORTEX= in their own + projects. + + diff -r 853377051f1e -r c1e6b7221b2f thesis/weave-thesis.sh --- a/thesis/weave-thesis.sh Sun Mar 23 19:09:14 2014 -0400 +++ b/thesis/weave-thesis.sh Sun Mar 23 22:20:44 2014 -0400 @@ -9,6 +9,8 @@ (progn (find-file \"cortex.org\") (org-latex-export-to-latex nil nil nil t nil) \ + (find-file \"user-guide.org\") + (org-latex-export-to-latex nil nil nil t nil) \ (find-file \"abstract.org\") (org-latex-export-to-latex nil nil nil t nil))" \ \