Mercurial > cortex

comparison thesis/cortex.org @ 425:efba8526a662

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
happy with code formatting.
author Robert McIntyre <rlm@mit.edu>
date Fri, 21 Mar 2014 02:48:23 -0400
parents 6b0f77df0e53
children 435b5e22d72a
comparison
equal deleted inserted replaced
424:d55197a40b47 425:efba8526a662
1 d 1 #+title: =CORTEX=
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+description: Using embodied AI to facilitate Artificial Imagination.
5 #+keywords: AI, clojure, embodiment
6
7 * Artificial Imagination
8
9 Imagine watching a video of someone skateboarding. When you watch
10 the video, you can imagine yourself skateboarding, and your
11 knowledge of the human body and its dynamics guides your
12 interpretation of the scene. For example, even if the skateboarder
13 is partially occluded, you can infer the positions of his arms and
14 body from your own knowledge of how your body would be positioned if
15 you were skateboarding. If the skateboarder suffers an accident, you
16 wince in sympathy, imagining the pain your own body would experience
17 if it were in the same situation. This empathy with other people
18 guides our understanding of whatever they are doing because it is a
19 powerful constraint on what is probable and possible. In order to
20 make use of this powerful empathy constraint, I need a system that
21 can generate and make sense of sensory data from the many different
22 senses that humans possess. The two key proprieties of such a system
23 are /embodiment/ and /imagination/.
24
25 ** What is imagination?
26
27 One kind of imagination is /sympathetic/ imagination: you imagine
28 yourself in the position of something/someone you are
29 observing. This type of imagination comes into play when you follow
30 along visually when watching someone perform actions, or when you
31 sympathetically grimace when someone hurts themselves. This type of
32 imagination uses the constraints you have learned about your own
33 body to highly constrain the possibilities in whatever you are
34 seeing. It uses all your senses to including your senses of touch,
35 proprioception, etc. Humans are flexible when it comes to "putting
36 themselves in another's shoes," and can sympathetically understand
37 not only other humans, but entities ranging from animals to cartoon
38 characters to [[http://www.youtube.com/watch?v=0jz4HcwTQmU][single dots]] on a screen!
2 39
3 40
4 lol whatevar 41 #+caption: A cat drinking some water. Identifying this action is beyond the state of the art for computers.
42 #+ATTR_LaTeX: :width 5cm
43 [[./images/cat-drinking.jpg]]
5 44
6 * lol
7 45
46 This is a basic test for the vision system. It only tests the
47 vision-pipeline and does not deal with loading eyes from a blender
48 file. The code creates two videos of the same rotating cube from
49 different angles.
50
51
52 #+name: test-1
53 #+begin_src clojure
54 (in-ns 'cortex.test.vision)
55
56 (defn test-pipeline
57 "Testing vision:
58 Tests the vision system by creating two views of the same rotating
59 object from different angles and displaying both of those views in
60 JFrames.
61
62 You should see a rotating cube, and two windows,
63 each displaying a different view of the cube."
64 ([] (test-pipeline false))
65 ([record?]
66 (let [candy
67 (box 1 1 1 :physical? false :color ColorRGBA/Blue)]
68 (world
69 (doto (Node.)
70 (.attachChild candy))
71 {}
72 (fn [world]
73 (let [cam (.clone (.getCamera world))
74 width (.getWidth cam)
75 height (.getHeight cam)]
76 (add-camera! world cam
77 (comp
78 (view-image
79 (if record?
80 (File. "/home/r/proj/cortex/render/vision/1")))
81 BufferedImage!))
82 (add-camera! world
83 (doto (.clone cam)
84 (.setLocation (Vector3f. -10 0 0))
85 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y))
86 (comp
87 (view-image
88 (if record?
89 (File. "/home/r/proj/cortex/render/vision/2")))
90 BufferedImage!))
91 (let [timer (IsoTimer. 60)]
92 (.setTimer world timer)
93 (display-dilated-time world timer))
94 ;; This is here to restore the main view
95 ;; after the other views have completed processing
96 (add-camera! world (.getCamera world) no-op)))
97 (fn [world tpf]
98 (.rotate candy (* tpf 0.2) 0 0))))))
99 #+end_src