1.1 --- a/thesis/cortex.tex	Sat Mar 22 23:31:07 2014 -0400
     1.2 +++ b/thesis/cortex.tex	Sun Mar 23 16:33:01 2014 -0400
     1.3 @@ -1,100 +1,56 @@
     1.4  
     1.5 -\section{Artificial Imagination}
     1.6 +\section{Vision}
     1.7  \label{sec-1}
     1.8  
     1.9 -Imagine watching a video of someone skateboarding. When you watch
    1.10 -the video, you can imagine yourself skateboarding, and your
    1.11 -knowledge of the human body and its dynamics guides your
    1.12 -interpretation of the scene. For example, even if the skateboarder
    1.13 -is partially occluded, you can infer the positions of his arms and
    1.14 -body from your own knowledge of how your body would be positioned if
    1.15 -you were skateboarding. If the skateboarder suffers an accident, you
    1.16 -wince in sympathy, imagining the pain your own body would experience
    1.17 -if it were in the same situation. This empathy with other people
    1.18 -guides our understanding of whatever they are doing because it is a
    1.19 -powerful constraint on what is probable and possible. In order to
    1.20 -make use of this powerful empathy constraint, I need a system that
    1.21 -can generate and make sense of sensory data from the many different
    1.22 -senses that humans possess. The two key proprieties of such a system
    1.23 -are \emph{embodiment} and \emph{imagination}.
    1.24 +System for understanding what the actors in a video are doing --
    1.25 +Action Recognition.
    1.26  
    1.27 -\subsection{What is imagination?}
    1.28 -\label{sec-1-1}
    1.29 -
    1.30 -One kind of imagination is \emph{sympathetic} imagination: you imagine
    1.31 -yourself in the position of something/someone you are
    1.32 -observing. This type of imagination comes into play when you follow
    1.33 -along visually when watching someone perform actions, or when you
    1.34 -sympathetically grimace when someone hurts themselves. This type of
    1.35 -imagination uses the constraints you have learned about your own
    1.36 -body to highly constrain the possibilities in whatever you are
    1.37 -seeing. It uses all your senses to including your senses of touch,
    1.38 -proprioception, etc. Humans are flexible when it comes to "putting
    1.39 -themselves in another's shoes," and can sympathetically understand
    1.40 -not only other humans, but entities ranging from animals to cartoon
    1.41 -characters to \href{http://www.youtube.com/watch?v=0jz4HcwTQmU}{single dots} on a screen!
    1.42 -
    1.43 -
    1.44 -\begin{figure}[htb]
    1.45 -\centering
    1.46 -\includegraphics[width=5cm]{./images/cat-drinking.jpg}
    1.47 -\caption{A cat drinking some water. Identifying this action is beyond the state of the art for computers.}
    1.48 -\end{figure}
    1.49 -
    1.50 -
    1.51 -\begin{listing}
    1.52 -\caption{This is a basic test for the vision system. It only tests the vision-pipeline and does not deal with loading eyes from a blender file. The code creates two videos of the same rotating cube from different angles.}
    1.53 -\begin{clojurecode}
    1.54 -(defn test-pipeline
    1.55 -  "Testing vision:
    1.56 -   Tests the vision system by creating two views of the same rotating
    1.57 -   object from different angles and displaying both of those views in
    1.58 -   JFrames.
    1.59 -
    1.60 -   You should see a rotating cube, and two windows,
    1.61 -   each displaying a different view of the cube."
    1.62 -  ([] (test-pipeline false))
    1.63 -  ([record?]
    1.64 -     (let [candy
    1.65 -           (box 1 1 1 :physical? false :color ColorRGBA/Blue)]
    1.66 -       (world
    1.67 -        (doto (Node.)
    1.68 -          (.attachChild candy))
    1.69 -        {}
    1.70 -        (fn [world]
    1.71 -          (let [cam (.clone (.getCamera world))
    1.72 -                width (.getWidth cam)
    1.73 -                height (.getHeight cam)]
    1.74 -            (add-camera! world cam 
    1.75 -                         (comp
    1.76 -                          (view-image
    1.77 -                           (if record?
    1.78 -                             (File. "/home/r/proj/cortex/render/vision/1")))
    1.79 -                          BufferedImage!))
    1.80 -            (add-camera! world
    1.81 -                         (doto (.clone cam)
    1.82 -                           (.setLocation (Vector3f. -10 0 0))
    1.83 -                           (.lookAt Vector3f/ZERO Vector3f/UNIT_Y))
    1.84 -                         (comp
    1.85 -                          (view-image
    1.86 -                           (if record?
    1.87 -                             (File. "/home/r/proj/cortex/render/vision/2")))
    1.88 -                          BufferedImage!))
    1.89 -            (let [timer (IsoTimer. 60)]
    1.90 -              (.setTimer world timer)
    1.91 -              (display-dilated-time world timer))
    1.92 -            ;; This is here to restore the main view
    1.93 -            ;; after the other views have completed processing
    1.94 -            (add-camera! world (.getCamera world) no-op)))
    1.95 -        (fn [world tpf]
    1.96 -          (.rotate candy (* tpf 0.2) 0 0))))))
    1.97 -\end{clojurecode}
    1.98 -\end{listing}
    1.99 +Separate action recognition into three components:
   1.100  
   1.101  \begin{itemize}
   1.102 -\item This is test1 \cite{Tappert77}.
   1.103 +\item free play
   1.104 +\item embodied action predicates
   1.105 +\item model alignment
   1.106 +\item sensory imagination
   1.107 +\end{itemize}
   1.108 +\section{Steps}
   1.109 +\label{sec-2}
   1.110 +
   1.111 +\begin{itemize}
   1.112 +\item Build cortex, a simulated environment for sensate AI
   1.113 +\begin{itemize}
   1.114 +\item solid bodies w/ joints
   1.115 +\item vision
   1.116 +\item touch
   1.117 +\item vision
   1.118 +\item hearing
   1.119 +\item proprioception
   1.120 +\item muscle contraction
   1.121  \end{itemize}
   1.122  
   1.123 -\cite{Tappert77}
   1.124 -lol
   1.125 -\cite{Tappert77}
   1.126 +\item Build experimental framework for worm-actions
   1.127 +\begin{itemize}
   1.128 +\item embodied stream predicates
   1.129 +\item \(\phi\)-space
   1.130 +\item \(\phi\)-scan
   1.131 +\end{itemize}
   1.132 +\end{itemize}
   1.133 +\section{News}
   1.134 +\label{sec-3}
   1.135 +
   1.136 +Experimental results:
   1.137 +
   1.138 +\begin{itemize}
   1.139 +\item \(\phi\)-space actually works very well for the worm!
   1.140 +\item self organizing touch map
   1.141 +\end{itemize}
   1.142 +
   1.143 +\section{Contributions}
   1.144 +\label{sec-4}
   1.145 +\begin{itemize}
   1.146 +\item Built \texttt{CORTEX}, a comprehensive platform for embodied AI
   1.147 +experiments. Has many new features lacking in other systems, such
   1.148 +as sound. Easy to model/create new creatures.
   1.149 +\item created a novel concept for action recognition by using artificial
   1.150 +imagination.
   1.151 +\end{itemize}