rlm@432: #+title: =CORTEX=
rlm@432: #+author: Robert McIntyre
rlm@432: #+email: rlm@mit.edu
rlm@432: #+description: Using embodied AI to facilitate Artificial Imagination.
rlm@432: #+keywords: AI, clojure, embodiment
rlm@432: 
rlm@432: * Artificial Imagination
rlm@432: 
rlm@432:   Imagine watching a video of someone skateboarding. When you watch
rlm@432:   the video, you can imagine yourself skateboarding, and your
rlm@432:   knowledge of the human body and its dynamics guides your
rlm@432:   interpretation of the scene. For example, even if the skateboarder
rlm@432:   is partially occluded, you can infer the positions of his arms and
rlm@432:   body from your own knowledge of how your body would be positioned if
rlm@432:   you were skateboarding. If the skateboarder suffers an accident, you
rlm@432:   wince in sympathy, imagining the pain your own body would experience
rlm@432:   if it were in the same situation. This empathy with other people
rlm@432:   guides our understanding of whatever they are doing because it is a
rlm@432:   powerful constraint on what is probable and possible. In order to
rlm@432:   make use of this powerful empathy constraint, I need a system that
rlm@432:   can generate and make sense of sensory data from the many different
rlm@432:   senses that humans possess. The two key proprieties of such a system
rlm@432:   are /embodiment/ and /imagination/.
rlm@432: 
rlm@432: ** What is imagination?
rlm@432: 
rlm@432:    One kind of imagination is /sympathetic/ imagination: you imagine
rlm@432:    yourself in the position of something/someone you are
rlm@432:    observing. This type of imagination comes into play when you follow
rlm@432:    along visually when watching someone perform actions, or when you
rlm@432:    sympathetically grimace when someone hurts themselves. This type of
rlm@432:    imagination uses the constraints you have learned about your own
rlm@432:    body to highly constrain the possibilities in whatever you are
rlm@432:    seeing. It uses all your senses to including your senses of touch,
rlm@432:    proprioception, etc. Humans are flexible when it comes to "putting
rlm@432:    themselves in another's shoes," and can sympathetically understand
rlm@432:    not only other humans, but entities ranging from animals to cartoon
rlm@432:    characters to [[http://www.youtube.com/watch?v=0jz4HcwTQmU][single dots]] on a screen!
rlm@432: 
rlm@432: 
rlm@432:    #+caption: A cat drinking some water. Identifying this action is beyond the state of the art for computers.
rlm@432:    #+ATTR_LaTeX: :width 5cm
rlm@432:    [[./images/cat-drinking.jpg]]
rlm@432: 
rlm@432: 
rlm@432: #+begin_listing clojure
rlm@432: \caption{This is a basic test for the vision system. It only tests the vision-pipeline and does not deal with loading eyes from a blender file. The code creates two videos of the same rotating cube from different angles.}
rlm@432: #+name: test-1
rlm@432: #+begin_src clojure
rlm@432: (defn test-pipeline
rlm@432:   "Testing vision:
rlm@432:    Tests the vision system by creating two views of the same rotating
rlm@432:    object from different angles and displaying both of those views in
rlm@432:    JFrames.
rlm@432: 
rlm@432:    You should see a rotating cube, and two windows,
rlm@432:    each displaying a different view of the cube."
rlm@432:   ([] (test-pipeline false))
rlm@432:   ([record?]
rlm@432:      (let [candy
rlm@432:            (box 1 1 1 :physical? false :color ColorRGBA/Blue)]
rlm@432:        (world
rlm@432:         (doto (Node.)
rlm@432:           (.attachChild candy))
rlm@432:         {}
rlm@432:         (fn [world]
rlm@432:           (let [cam (.clone (.getCamera world))
rlm@432:                 width (.getWidth cam)
rlm@432:                 height (.getHeight cam)]
rlm@432:             (add-camera! world cam 
rlm@432:                          (comp
rlm@432:                           (view-image
rlm@432:                            (if record?
rlm@432:                              (File. "/home/r/proj/cortex/render/vision/1")))
rlm@432:                           BufferedImage!))
rlm@432:             (add-camera! world
rlm@432:                          (doto (.clone cam)
rlm@432:                            (.setLocation (Vector3f. -10 0 0))
rlm@432:                            (.lookAt Vector3f/ZERO Vector3f/UNIT_Y))
rlm@432:                          (comp
rlm@432:                           (view-image
rlm@432:                            (if record?
rlm@432:                              (File. "/home/r/proj/cortex/render/vision/2")))
rlm@432:                           BufferedImage!))
rlm@432:             (let [timer (IsoTimer. 60)]
rlm@432:               (.setTimer world timer)
rlm@432:               (display-dilated-time world timer))
rlm@432:             ;; This is here to restore the main view
rlm@432:             ;; after the other views have completed processing
rlm@432:             (add-camera! world (.getCamera world) no-op)))
rlm@432:         (fn [world tpf]
rlm@432:           (.rotate candy (* tpf 0.2) 0 0))))))
rlm@432: #+end_src
rlm@432: #+end_listing
rlm@432: 
rlm@432: - This is test1 \cite{Tappert77}.
rlm@432: 
rlm@432: \cite{Tappert77}
rlm@432: lol
rlm@432: \cite{Tappert77}