Mercurial > cortex
changeset 34:183744c179e6
MASSIVE cleanup, especially in the vision code
| author | Robert McIntyre <rlm@mit.edu> |
|---|---|
| date | Thu, 03 Nov 2011 08:28:26 -0700 |
| parents | c377222528e6 |
| children | 5eb5c6f0590b |
| files | org/eyes.org org/games.org org/intro.org org/setup.org org/util.org org/world.org |
| diffstat | 6 files changed, 235 insertions(+), 371 deletions(-) [+] |
line wrap: on
line diff
1.1 --- a/org/eyes.org Wed Nov 02 11:03:12 2011 -0700 1.2 +++ b/org/eyes.org Thu Nov 03 08:28:26 2011 -0700 1.3 @@ -1,47 +1,37 @@ 1.4 -#+title: Eyes 1.5 +#+title: Simulated Sense of Sight 1.6 #+author: Robert McIntyre 1.7 #+email: rlm@mit.edu 1.8 -#+description: Simulating senses for AI research using JMonkeyEngine3 1.9 +#+description: Simulated sight for AI research using JMonkeyEngine3 1.10 +#+keywords: computer vision, jMonkeyEngine3, clojure 1.11 #+SETUPFILE: ../../aurellem/org/setup.org 1.12 #+INCLUDE: ../../aurellem/org/level-0.org 1.13 #+babel: :mkdirp yes :noweb yes :exports both 1.14 1.15 +* Vision 1.16 1.17 - 1.18 -** Eyes 1.19 - 1.20 -Ultimately I want to make creatures with eyes. Each eye can be 1.21 -independely moved and should see its own version of the world 1.22 -depending on where it is. 1.23 +I want to make creatures with eyes. Each eye can be independely moved 1.24 +and should see its own version of the world depending on where it is. 1.25 #+srcname: eyes 1.26 #+begin_src clojure 1.27 -(ns body.eye) 1.28 -(use 'cortex.world) 1.29 -(use 'cortex.import) 1.30 -(use 'clojure.contrib.def) 1.31 -(cortex.import/mega-import-jme3) 1.32 -(rlm.rlm-commands/help) 1.33 -(import java.nio.ByteBuffer) 1.34 -(import java.awt.image.BufferedImage) 1.35 -(import java.awt.Color) 1.36 -(import java.awt.Dimension) 1.37 -(import java.awt.Graphics) 1.38 -(import java.awt.Graphics2D) 1.39 -(import java.awt.event.WindowAdapter) 1.40 -(import java.awt.event.WindowEvent) 1.41 -(import java.awt.image.BufferedImage) 1.42 -(import java.nio.ByteBuffer) 1.43 -(import javax.swing.JFrame) 1.44 -(import javax.swing.JPanel) 1.45 -(import javax.swing.SwingUtilities) 1.46 -(import javax.swing.ImageIcon) 1.47 -(import javax.swing.JOptionPane) 1.48 -(import java.awt.image.ImageObserver) 1.49 +(ns cortex.vision 1.50 + "Simulate the sense of vision in jMonkeyEngine3. Enables multiple 1.51 + eyes from different positions to observe the same world, and pass 1.52 + the observed data to any arbitray function." 1.53 + {:author "Robert McIntyre"} 1.54 + (:use cortex.world) 1.55 + (:import com.jme3.post.SceneProcessor) 1.56 + (:import (com.jme3.util Screenshots BufferUtils)) 1.57 + (:import java.nio.ByteBuffer) 1.58 + (:import java.awt.image.BufferedImage) 1.59 + (:import com.jme3.renderer.ViewPort) 1.60 + (:import com.jme3.math.ColorRGBA)) 1.61 1.62 (defn scene-processor 1.63 - "deals with converting FrameBuffers to BufferedImages so 1.64 - that the continuation function can be defined only in terms 1.65 - of what it does with BufferedImages" 1.66 + "Create a SceneProcessor object which wraps a vision processing 1.67 + continuation function. The SceneProcessor will take care of 1.68 + converting the rendered frame to a BufferedImage and passing that 1.69 + BufferedImage to the continuation. The continuation should be a 1.70 + function that takes a BufferedImage." 1.71 [continuation] 1.72 (let [byte-buffer (atom nil) 1.73 renderer (atom nil) 1.74 @@ -56,8 +46,9 @@ 1.75 (reset! byte-buffer 1.76 (BufferUtils/createByteBuffer 1.77 (* width height 4))) 1.78 - (reset! image (BufferedImage. width height 1.79 - BufferedImage/TYPE_4BYTE_ABGR)))) 1.80 + (reset! image (BufferedImage. 1.81 + width height 1.82 + BufferedImage/TYPE_4BYTE_ABGR)))) 1.83 (isInitialized [] (not (nil? @byte-buffer))) 1.84 (reshape [_ _ _]) 1.85 (preFrame [_]) 1.86 @@ -71,179 +62,108 @@ 1.87 (cleanup [])))) 1.88 1.89 (defn add-eye 1.90 - "Add an eye to the world, and call continuation on 1.91 - every frame produced" 1.92 + "Add an eye to the world, calling continuation on every frame 1.93 + produced." 1.94 [world camera continuation] 1.95 (let [width (.getWidth camera) 1.96 height (.getHeight camera) 1.97 render-manager (.getRenderManager world) 1.98 viewport (.createMainView render-manager "eye-view" camera)] 1.99 (doto viewport 1.100 - (.setBackgroundColor ColorRGBA/Black) 1.101 (.setClearFlags true true true) 1.102 + (.setBackgroundColor ColorRGBA/Gray) 1.103 (.addProcessor (scene-processor continuation)) 1.104 (.attachScene (.getRootNode world))))) 1.105 1.106 -(defn make-display-frame [display width height] 1.107 - (SwingUtilities/invokeLater 1.108 - (fn [] 1.109 - (.setPreferredSize display (Dimension. width height)) 1.110 - (doto (JFrame. "Eye Camera!") 1.111 - (-> (.getContentPane) (.add display)) 1.112 - (.setDefaultCloseOperation JFrame/DISPOSE_ON_CLOSE) 1.113 - (.pack) 1.114 - (.setLocationRelativeTo nil) 1.115 - (.setResizable false) 1.116 - (.setVisible true))))) 1.117 - 1.118 -(defn image-monitor [#^BufferedImage image] 1.119 - (proxy [JPanel] [] 1.120 - (paintComponent 1.121 - [g] 1.122 - (proxy-super paintComponent g) 1.123 - (locking image 1.124 - (.drawImage g image 0 0 1.125 - (proxy [ImageObserver] 1.126 - [] 1.127 - (imageUpdate 1.128 - [] 1.129 - (proxy-super imageUpdate)))))))) 1.130 +#+end_src 1.131 1.132 -(defn movie-image [] 1.133 - (let [setup 1.134 - (runonce 1.135 - (fn [#^BufferedImage image] 1.136 - (let [width (.getWidth image) 1.137 - height (.getHeight image) 1.138 - display (image-monitor image) 1.139 - frame (make-display-frame display width height)] 1.140 - display)))] 1.141 - (fn [#^BufferedImage image] 1.142 - (.repaint (setup image))))) 1.143 - 1.144 +Note the use of continuation passing style for connecting the eye to a 1.145 +function to process the output. You can create any number of eyes, and 1.146 +each of them will see the world from their own =Camera=. Once every 1.147 +frame, the rendered image is copied to a =BufferedImage=, and that 1.148 +data is sent off to the continuation function. Moving the =Camera= 1.149 +which was used to create the eye will change what the eye sees. 1.150 1.151 -(defn observer 1.152 - "place thy eye!" 1.153 - [world camera] 1.154 - (let [eye camera 1.155 - width (.getWidth eye) 1.156 - height (.getHeight eye)] 1.157 - (no-exceptions 1.158 - (add-eye 1.159 - world 1.160 - eye 1.161 - (movie-image))))) 1.162 -#+end_src 1.163 +* Example 1.164 1.165 #+srcname: test-vision 1.166 #+begin_src clojure 1.167 +(ns test.vision 1.168 + (:use (cortex world util vision)) 1.169 + (:import java.awt.image.BufferedImage) 1.170 + (:import javax.swing.JPanel) 1.171 + (:import javax.swing.SwingUtilities) 1.172 + (:import java.awt.Dimension) 1.173 + (:import javax.swing.JFrame) 1.174 + (:import com.jme3.math.ColorRGBA) 1.175 + (:import com.jme3.scene.Node)) 1.176 1.177 -(ns test.vision) 1.178 -(use 'cortex.world) 1.179 -(use 'cortex.import) 1.180 -(use 'clojure.contrib.def) 1.181 -(use 'body.eye) 1.182 -(cortex.import/mega-import-jme3) 1.183 -(rlm.rlm-commands/help) 1.184 -(import java.nio.ByteBuffer) 1.185 -(import java.awt.image.BufferedImage) 1.186 -(import java.awt.Color) 1.187 -(import java.awt.Dimension) 1.188 -(import java.awt.Graphics) 1.189 -(import java.awt.Graphics2D) 1.190 -(import java.awt.event.WindowAdapter) 1.191 -(import java.awt.event.WindowEvent) 1.192 -(import java.awt.image.BufferedImage) 1.193 -(import java.nio.ByteBuffer) 1.194 -(import javax.swing.JFrame) 1.195 -(import javax.swing.JPanel) 1.196 -(import javax.swing.SwingUtilities) 1.197 -(import javax.swing.ImageIcon) 1.198 -(import javax.swing.JOptionPane) 1.199 -(import java.awt.image.ImageObserver) 1.200 +(defn view-image 1.201 + "Initailizes a JPanel on which you may draw a BufferedImage of the 1.202 + given width and height. Returns a function that accepts a 1.203 + BufferedImage and draws it to the JPanel." 1.204 + [width height] 1.205 + (let [image 1.206 + (atom 1.207 + (BufferedImage. width height BufferedImage/TYPE_4BYTE_ABGR)) 1.208 + panel 1.209 + (proxy [JPanel] [] 1.210 + (paint 1.211 + [graphics] 1.212 + (proxy-super paintComponent graphics) 1.213 + (.drawImage graphics @image 0 0 nil) 1.214 + ))] 1.215 + 1.216 + (SwingUtilities/invokeLater 1.217 + (fn [] 1.218 + (.setPreferredSize panel (Dimension. width height)) 1.219 + (doto (JFrame. "Eye Camera!") 1.220 + (-> (.getContentPane) (.add panel)) 1.221 + (.pack) 1.222 + (.setLocationRelativeTo nil) 1.223 + (.setResizable false) 1.224 + (.setVisible true)))) 1.225 + (fn [#^BufferedImage i] 1.226 + (reset! image i) 1.227 + (.repaint panel)))) 1.228 1.229 - 1.230 -(def width 200) 1.231 -(def height 200) 1.232 - 1.233 -(defn camera [] 1.234 - (doto (Camera. width height) 1.235 - (.setFrustumPerspective 45 1 1 1000) 1.236 - (.setLocation (Vector3f. -3 0 -5)) 1.237 - (.lookAt Vector3f/ZERO Vector3f/UNIT_Y))) 1.238 - 1.239 -(defn camera2 [] 1.240 - (doto (Camera. width height) 1.241 - (.setFrustumPerspective 45 1 1 1000) 1.242 - (.setLocation (Vector3f. 3 0 -5)) 1.243 - (.lookAt Vector3f/ZERO Vector3f/UNIT_Y))) 1.244 - 1.245 -(defn setup-fn [world] 1.246 - (let [eye (camera) 1.247 - width (.getWidth eye) 1.248 - height (.getHeight eye)] 1.249 - (no-exceptions 1.250 - (add-eye 1.251 - world 1.252 - eye 1.253 - (runonce visual)) 1.254 - (add-eye 1.255 - world 1.256 - (camera2) 1.257 - (runonce visual))))) 1.258 - 1.259 -(defn spider-eye [position] 1.260 - (doto (Camera. 200 200 ) 1.261 - (.setFrustumPerspective 45 1 1 1000) 1.262 - (.setLocation position) 1.263 - (.lookAt Vector3f/ZERO Vector3f/UNIT_Y))) 1.264 - 1.265 -(defn setup-fn* [world] 1.266 - (let [eye (camera) 1.267 - width (.getWidth eye) 1.268 - height (.getHeight eye)] 1.269 - ;;(.setClearFlags (.getViewPort world) true true true) 1.270 - (observer world (.getCamera world)) 1.271 - (observer world (spider-eye (Vector3f. 3 0 -5))) 1.272 - ;;(observer world (spider-eye (Vector3f. 0 0 -5))) 1.273 - ;; (observer world (spider-eye (Vector3f. -3 0 -5))) 1.274 - ;; (observer world (spider-eye (Vector3f. 0 3 -5))) 1.275 - ;; (observer world (spider-eye (Vector3f. 0 -3 -5))) 1.276 - ;; (observer world (spider-eye (Vector3f. 3 3 -5))) 1.277 - ;; (observer world (spider-eye (Vector3f. -3 3 -5))) 1.278 - ;; (observer world (spider-eye (Vector3f. 3 -3 -5))) 1.279 - ;; (observer world (spider-eye (Vector3f. -3 -3 -5))) 1.280 - 1.281 - ) 1.282 - world) 1.283 - 1.284 -(defn test-world [] 1.285 - (let [thing (box 1 1 1 :physical? false)] 1.286 - (world 1.287 - (doto (Node.) 1.288 - (.attachChild thing)) 1.289 - {} 1.290 - setup-fn 1.291 - (fn [world tpf] 1.292 - (.rotate thing (* tpf 0.2) 0 0) 1.293 - )))) 1.294 - 1.295 - 1.296 +(defn test-vision 1.297 + "Tests the vision system by creating two views of the same rotating 1.298 + object from different angles and displaying both of those views in 1.299 + JFrames." [] 1.300 + (.start 1.301 + (let [candy 1.302 + (box 1 1 1 :physical? false :color ColorRGBA/Blue)] 1.303 + (world (doto (Node.) 1.304 + (.attachChild candy)) 1.305 + {} 1.306 + (fn [world] 1.307 + (let [cam (.clone (.getCamera world)) 1.308 + width (.getWidth cam) 1.309 + height (.getHeight cam)] 1.310 + (add-eye world 1.311 + (doto (.clone cam) 1.312 + (.setLocation (Vector3f. -10 0 0)) 1.313 + (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)) 1.314 + (view-image width height)) 1.315 + (add-eye world cam 1.316 + (view-image width height)) 1.317 + ;; this is here to restore the main view 1.318 + ;; after the other views have completed processing 1.319 + (add-eye world (.getCamera world) no-op) 1.320 + )) 1.321 + (fn [world tpf] 1.322 + (.rotate candy (* tpf 0.2) 0 0)))))) 1.323 #+end_src 1.324 1.325 - 1.326 -#+results: eyes 1.327 -: #'body.eye/test-world 1.328 - 1.329 -Note the use of continuation passing style for connecting the eye to a 1.330 -function to process the output. The example code will create two 1.331 -videos of the same rotating cube from different angles, sutiable for 1.332 -stereoscopic vision. 1.333 - 1.334 +The example code will create two videos of the same rotating object 1.335 +from different angles. It can be used both for stereoscopic vision 1.336 +simulation or for simulating multiple creatures, each with their own 1.337 +sense of vision. 1.338 1.339 1.340 * COMMENT code generation 1.341 -#+begin_src clojure :tangle ../src/body/eye.clj 1.342 +#+begin_src clojure :tangle ../src/cortex/vision.clj 1.343 <<eyes>> 1.344 #+end_src 1.345
2.1 --- a/org/games.org Wed Nov 02 11:03:12 2011 -0700 2.2 +++ b/org/games.org Thu Nov 03 08:28:26 2011 -0700 2.3 @@ -1,16 +1,18 @@ 2.4 -#+title: Games! Games! Games! <3 2.5 +#+title: jMonkeyEngine3 from Clojure 2.6 #+author: Robert McIntyre 2.7 #+email: rlm@mit.edu 2.8 -#+description: Simulating senses for AI research using JMonkeyEngine3 2.9 +#+description: Using jMonkeyEngine3 from clojure 2.10 +#+keywords: clojure, jMonkeyEngine3, tutorial, examples 2.11 #+SETUPFILE: ../../aurellem/org/setup.org 2.12 #+INCLUDE: ../../aurellem/org/level-0.org 2.13 #+babel: :mkdirp yes :noweb yes :exports both 2.14 2.15 +[TABLE-OF-CONTENTS] 2.16 2.17 -* Games! 2.18 2.19 Here are the jMonkeyEngine "Hello" programs translated to clojure. 2.20 -** Hello Simple App 2.21 + 2.22 +* Hello Simple App 2.23 Here is the hello world example for jme3 in clojure. It's a more or 2.24 less direct translation from the java source [[http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_simpleapplication][here]]. 2.25 2.26 @@ -20,13 +22,14 @@ 2.27 2.28 #+srcname: hello-simple-app 2.29 #+begin_src clojure :results silent 2.30 -(ns hello.hello-simple-app) 2.31 -(require 'cortex.import) 2.32 -(use 'clojure.contrib.def) 2.33 -(rlm.rlm-commands/help) 2.34 -(cortex.import/mega-import-jme3) 2.35 -(use 'cortex.world) 2.36 - 2.37 +(ns hello.hello-simple-app 2.38 + (:use cortex.world) 2.39 + (:import com.jme3.math.Vector3f) 2.40 + (:import com.jme3.material.Material) 2.41 + (:import com.jme3.scene.Geometry) 2.42 + (:import com.jme3.math.ColorRGBA) 2.43 + (:import com.jme3.app.SimpleApplication) 2.44 + (:import com.jme3.scene.shape.Box)) 2.45 2.46 (def cube (Box. Vector3f/ZERO 1 1 1)) 2.47 2.48 @@ -62,8 +65,31 @@ 2.49 #+caption: the simplest JME game. 2.50 [[../images/simple-app.jpg]] 2.51 2.52 +* Simpler HelloSimpleApp 2.53 2.54 -** Hello Physics 2.55 +#+srcname: hello-simpler-app 2.56 +#+begin_src clojure 2.57 +(ns hello.hello-simpler-app 2.58 + (:use cortex.world) 2.59 + (:use cortex.util) 2.60 + (:import com.jme3.math.ColorRGBA) 2.61 + (:import com.jme3.scene.Node)) 2.62 + 2.63 +(defn simpler-world 2.64 + "The jMonkeyEngine3 Hello World program. Displays a blue 3D cube in 2.65 + a basic 3D world." 2.66 + [] 2.67 + (world (doto (Node.) 2.68 + (.attachChild 2.69 + (box 1 1 1 2.70 + :color ColorRGBA/Blue :physical? false))) 2.71 + {} no-op no-op)) 2.72 +#+end_src 2.73 + 2.74 +More information about the jMonkeyEngine3 hello world program can be 2.75 +found [[http://jmonkeyengine.org/wiki/doku.php/starter:hello_world][here]]. 2.76 + 2.77 +* COMMENT Hello Physics 2.78 From http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_physics 2.79 2.80 #+srcname: brick-wall-header 2.81 @@ -90,7 +116,7 @@ 2.82 (def brick-length 0.48) 2.83 (def brick-width 0.24) 2.84 (def brick-height 0.12) 2.85 - 2.86 +(def gravity (Vector3f. 0 -9.81 0)) 2.87 2.88 (defn brick* [position] 2.89 (doto (box brick-length brick-height brick-width 2.90 @@ -126,8 +152,8 @@ 2.91 (fn [game value] 2.92 (println-repl "set gravity to " new-value) 2.93 (if value 2.94 - (set-gravity* game new-value) 2.95 - (set-gravity* game gravity)))) 2.96 + (set-gravity game new-value) 2.97 + (set-gravity game gravity)))) 2.98 2.99 (defn fire-cannon-ball 2.100 ([node] 2.101 @@ -216,10 +242,10 @@ 2.102 #+caption: the brick wall after it has been knocked over by a "pok\eacute{}ball" 2.103 [[../images/brick-wall-knocked-down.jpg]] 2.104 2.105 -** Other Brick Games 2.106 +* COMMENT Other Brick Games 2.107 #+srcname: other-games 2.108 #+begin_src clojure :results silent 2.109 -(ns cortex.other-games 2.110 +(ns hello.other-games 2.111 {:author "Dylan Holmes"}) 2.112 (use 'cortex.world) 2.113 (use 'hello.brick-wall) 2.114 @@ -364,15 +390,15 @@ 2.115 #+caption: floating dominos 2.116 [[../images/dominos.jpg]] 2.117 2.118 -** Hello Loop 2.119 +* Hello Loop 2.120 #+srcname: hello-loop 2.121 #+begin_src clojure :results silent 2.122 -(ns hello.loop) 2.123 -(use 'cortex.world) 2.124 -(use 'cortex.import) 2.125 -(cortex.import/mega-import-jme3) 2.126 -(rlm.rlm-commands/help) 2.127 - 2.128 +(ns hello.loop 2.129 + (:use cortex.world) 2.130 + (:use cortex.util) 2.131 + (:import com.jme3.math.ColorRGBA) 2.132 + (:import com.jme3.scene.Node)) 2.133 + 2.134 (defn blue-cube [] 2.135 (box 1 1 1 2.136 :color ColorRGBA/Blue 2.137 @@ -391,7 +417,7 @@ 2.138 (.rotate cube 0.0 (* 2 tpf) 0.0))))) 2.139 #+end_src 2.140 2.141 -** Hello Collision 2.142 +* COMMENT Hello Collision 2.143 2.144 #+srcname: hello-collision 2.145 #+begin_src clojure :results silent 2.146 @@ -513,7 +539,7 @@ 2.147 update-fn))) 2.148 #+end_src 2.149 2.150 -** Hello Terrain 2.151 +* COMMENT Hello Terrain 2.152 #+srcname: hello-terrain 2.153 #+begin_src clojure :results silent 2.154 (ns hello.terrain) 2.155 @@ -663,12 +689,13 @@ 2.156 no-op)))) 2.157 #+end_src 2.158 2.159 -** Hello Materials 2.160 +* COMMENT Hello Materials 2.161 #+srcname: material 2.162 #+begin_src clojure :results silent 2.163 (ns hello.material) 2.164 (use 'cortex.world) 2.165 (use 'cortex.import) 2.166 +(use 'cortex.util) 2.167 (use 'clojure.contrib.def) 2.168 (cortex.import/mega-import-jme3) 2.169 (rlm.rlm-commands/help) 2.170 @@ -717,7 +744,7 @@ 2.171 (doto 2.172 (.setTexture "DiffuseMap" 2.173 (.loadTexture (asset-manager) 2.174 - "Textures/Terrain/Pond/Pond.png")) 2.175 + "Textures/Terrain/Pond/Pond.jpg")) 2.176 (.setTexture "NormalMap" 2.177 (.loadTexture (asset-manager) 2.178 "Textures/Terrain/Pond/Pond_normal.png")) 2.179 @@ -755,7 +782,12 @@ 2.180 <<hello-simple-app>> 2.181 #+end_src 2.182 2.183 -#+begin_src clojure :tangle ../src/cortex/other_games.clj 2.184 +#+begin_src clojure :tangle ../src/hello/hello_simpler_app.clj 2.185 +<<hello-simpler-app>> 2.186 +#+end_src 2.187 + 2.188 + 2.189 +#+begin_src clojure :tangle ../src/hello/other_games.clj 2.190 <<other-games>> 2.191 #+end_src 2.192
3.1 --- a/org/intro.org Wed Nov 02 11:03:12 2011 -0700 3.2 +++ b/org/intro.org Thu Nov 03 08:28:26 2011 -0700 3.3 @@ -2,6 +2,7 @@ 3.4 #+author: Robert McIntyre 3.5 #+email: rlm@mit.edu 3.6 #+description: Simulating senses for AI research using JMonkeyEngine3 3.7 +#+keywords: Alan Turing, AI, sinulated senses, jMonkeyEngine3, virtual world 3.8 #+SETUPFILE: ../../aurellem/org/setup.org 3.9 #+INCLUDE: ../../aurellem/org/level-0.org 3.10 #+babel: :mkdirp yes :noweb yes 3.11 @@ -14,7 +15,7 @@ 3.12 already. What idea could be missing? 3.13 3.14 When Turing first proposed his famous "Turing Test" in the 3.15 -groundbreaking paper [[./sources/turing.pdf][/Computing Machines and Intelligence/]], he gave 3.16 +groundbreaking paper [[../sources/turing.pdf][/Computing Machines and Intelligence/]], he gave 3.17 little importance to how a computer program might interact with the 3.18 world: 3.19 3.20 @@ -42,25 +43,24 @@ 3.21 your life from birth, you would never learn anything, and could never 3.22 become intelligent. Actual humans placed in sensory deprivation 3.23 chambers experience hallucinations and can begin to loose their sense 3.24 -of reality in as little as 15 minutes[sensory-deprivation]. Most of 3.25 -the time, the programs we write are in exactly this situation. They do 3.26 -not interface with cameras and microphones, and they do not control a 3.27 -real or simulated body or interact with any sort of world. 3.28 +of reality. Most of the time, the programs we write are in exactly 3.29 +this situation. They do not interface with cameras and microphones, 3.30 +and they do not control a real or simulated body or interact with any 3.31 +sort of world. 3.32 3.33 * Simulation vs. Reality 3.34 I want demonstrate that multiple senses are what enable 3.35 intelligence. There are two ways of playing around with senses and 3.36 computer programs: 3.37 3.38 + 3.39 +** Simulation 3.40 The first is to go entirely with simulation: virtual world, virtual 3.41 character, virtual senses. The advantages are that when everything is 3.42 a simulation, experiments in that simulation are absolutely 3.43 reproducible. It's also easier to change the character and world to 3.44 explore new situations and different sensory combinations. 3.45 3.46 - 3.47 -** Issues with Simulation 3.48 - 3.49 If the world is to be simulated on a computer, then not only do you 3.50 have to worry about whether the character's senses are rich enough to 3.51 learn from the world, but whether the world itself is rendered with 3.52 @@ -75,7 +75,7 @@ 3.53 dust. Maybe a simulated world with today's limitations doesn't provide 3.54 enough richness for real intelligence to evolve. 3.55 3.56 -** Issues with Reality 3.57 +** Reality 3.58 3.59 The other approach for playing with senses is to hook your software up 3.60 to real cameras, microphones, robots, etc., and let it loose in the 3.61 @@ -107,16 +107,15 @@ 3.62 * Choose a Simulation Engine 3.63 3.64 Mainly because of issues with controlling the flow of time, I chose to 3.65 -simulate both the world and the character. I set out to make a minimal 3.66 -world in which I could embed a character with multiple senses. My main 3.67 -goal is to make an environment where I can perform further experiments 3.68 -in simulated senses. 3.69 +simulate both the world and the character. I set out to make a world 3.70 +in which I could embed a character with multiple senses. My main goal 3.71 +is to make an environment where I can perform further experiments in 3.72 +simulated senses. 3.73 3.74 -As Carl Sagan once said, "If you wish to make an apple pie from 3.75 -scratch, you must first invent the universe." I examined many 3.76 -different 3D environments to try and find something I would use as the 3.77 -base for my simulation; eventually the choice came down to three 3.78 -engines: the Quake II engine, the Source Engine, and jMonkeyEngine. 3.79 +I examined many different 3D environments to try and find something I 3.80 +would use as the base for my simulation; eventually the choice came 3.81 +down to three engines: the Quake II engine, the Source Engine, and 3.82 +jMonkeyEngine. 3.83 3.84 ** [[http://www.idsoftware.com][Quake II]]/[[http://www.bytonic.de/html/jake2.html][Jake2]] 3.85 3.86 @@ -172,7 +171,7 @@ 3.87 others for about 2 months I settled on jMonkeyEngine. I chose it 3.88 because it had the most features out of all the open projects I looked 3.89 at, and because I could then write my code in Clojure, an 3.90 -implementation of LISP that runs on the JVM... 3.91 +implementation of LISP that runs on the JVM. 3.92 3.93 3.94 3.95 @@ -206,4 +205,3 @@ 3.96 3.97 3.98 3.99 -
4.1 --- a/org/setup.org Wed Nov 02 11:03:12 2011 -0700 4.2 +++ b/org/setup.org Thu Nov 03 08:28:26 2011 -0700 4.3 @@ -2,6 +2,7 @@ 4.4 #+author: Robert McIntyre 4.5 #+email: rlm@mit.edu 4.6 #+description: Simulating senses for AI research using JMonkeyEngine3 4.7 +#+keywords: JMonkeyEngine3, clojure, java, setup 4.8 #+SETUPFILE: ../../aurellem/org/setup.org 4.9 #+INCLUDE: ../../aurellem/org/level-0.org 4.10 #+babel: :mkdirp yes :noweb yes :exports both 4.11 @@ -12,7 +13,8 @@ 4.12 4.13 #+srcname: checkout 4.14 #+begin_src sh :results verbatim 4.15 -svn checkout http://jmonkeyengine.googlecode.com/svn/trunk/engine jme3 4.16 +svn checkout http://jmonkeyengine.googlecode.com/svn/trunk/engine \ 4.17 + /home/r/proj/jMonkeyEngine3 4.18 #+end_src 4.19 4.20 #+results: checkout 4.21 @@ -23,114 +25,34 @@ 4.22 4.23 #+srcname: build 4.24 #+begin_src sh :results verbatim 4.25 -cd jme3 4.26 +cd /home/r/proj/jMonkeyEngine3 4.27 ant jar | tail -n 2 4.28 #+end_src 4.29 4.30 #+results: build 4.31 : BUILD SUCCESSFUL 4.32 -: Total time: 15 seconds 4.33 +: Total time: 5 seconds 4.34 4.35 4.36 Also build the javadoc: 4.37 4.38 #+srcname: javadoc 4.39 #+begin_src sh :results verbatim 4.40 -cd jme3 4.41 +cd /home/r/proj/jMonkeyEngine3 4.42 ant javadoc | tail -n 2 4.43 #+end_src 4.44 4.45 #+results: javadoc 4.46 : BUILD SUCCESSFUL 4.47 -: Total time: 12 seconds 4.48 +: Total time: 10 seconds 4.49 4.50 -Now, move the jars from the compilation into the project's lib folder. 4.51 4.52 -#+srcname: move-jars 4.53 -#+begin_src sh :results verbatim 4.54 -mkdir -p lib 4.55 -mkdir -p src 4.56 -cp jme3/dist/jMonkeyEngine3.jar lib/ 4.57 -cp jme3/dist/lib/* lib/ 4.58 -ls lib 4.59 -#+end_src 4.60 - 4.61 -#+results: move-jars 4.62 -#+begin_example 4.63 -eventbus-1.4.jar 4.64 -jbullet.jar 4.65 -jheora-jst-debug-0.6.0.jar 4.66 -jinput.jar 4.67 -jME3-jbullet.jar 4.68 -jME3-lwjgl-natives.jar 4.69 -jME3-testdata.jar 4.70 -jME3-test.jar 4.71 -jMonkeyEngine3.jar 4.72 -j-ogg-oggd.jar 4.73 -j-ogg-vorbisd.jar 4.74 -lwjgl.jar 4.75 -nifty-1.3.jar 4.76 -nifty-default-controls-1.3.jar 4.77 -nifty-examples-1.3.jar 4.78 -nifty-lwjgl-renderer-1.3.jar 4.79 -nifty-openal-soundsystem-1.0.jar 4.80 -nifty-style-black-1.3.jar 4.81 -nifty-style-grey-1.0.jar 4.82 -noise-0.0.1-SNAPSHOT.jar 4.83 -stack-alloc.jar 4.84 -vecmath.jar 4.85 -xmlpull-xpp3-1.1.4c.jar 4.86 -#+end_example 4.87 - 4.88 -It's good to create a =assets= directory in the style that the 4.89 -=AssetManager= will like. 4.90 - 4.91 -#+srcname: create-assets 4.92 -#+begin_src sh :results verbatim 4.93 -mkdir -p assets 4.94 -mkdir -p assets/Interface 4.95 -mkdir -p assets/Materials 4.96 -mkdir -p assets/MatDefs 4.97 -mkdir -p assets/Models 4.98 -mkdir -p assets/Scenes 4.99 -mkdir -p assets/Shaders 4.100 -mkdir -p assets/Sounds 4.101 -mkdir -p assets/Textures 4.102 -tree -L 1 assets 4.103 -#+end_src 4.104 - 4.105 -#+results: create-assets 4.106 -#+begin_example 4.107 -assets 4.108 -|-- Interface 4.109 -|-- MatDefs 4.110 -|-- Materials 4.111 -|-- Models 4.112 -|-- Scenes 4.113 -|-- Shaders 4.114 -|-- Sounds 4.115 -`-- Textures 4.116 - 4.117 -8 directories, 0 files 4.118 -#+end_example 4.119 - 4.120 - 4.121 -The java classpath should have all the jars contained in the =lib= 4.122 -directory as well as the src directory. 4.123 +The java classpath should have all the jars from the jMonkeyEngine 4.124 +directory. 4.125 4.126 For example, here is the file I use to run my REPL for clojure. 4.127 4.128 #+include: "/home/r/bin/swank-all" src sh :exports code 4.129 4.130 -The important thing here is that =cortex/lib/*=, =cortex/src=, and 4.131 -=cortex/assets= appear on the classpath. (=cortex= is the base 4.132 -directory of this project.) 4.133 4.134 -#+srcname: pwd 4.135 -#+begin_src sh 4.136 -pwd 4.137 -#+end_src 4.138 4.139 -#+results: pwd 4.140 -: /home/r/proj/cortex 4.141 -
5.1 --- a/org/util.org Wed Nov 02 11:03:12 2011 -0700 5.2 +++ b/org/util.org Thu Nov 03 08:28:26 2011 -0700 5.3 @@ -6,12 +6,12 @@ 5.4 #+SETUPFILE: ../../aurellem/org/setup.org 5.5 #+INCLUDE: ../../aurellem/org/level-0.org 5.6 5.7 -* Utilities 5.8 +[TABLE-OF-CONTENTS] 5.9 5.10 These are a collection of functions to make programming jMonkeyEngine 5.11 in clojure easier. 5.12 5.13 -** Imports 5.14 +* Imports 5.15 5.16 #+srcname: import 5.17 #+begin_src clojure :results silent 5.18 @@ -53,7 +53,7 @@ 5.19 importing only the classes it actually needs. 5.20 5.21 The =mega-import-jme3= is quite usefull for debugging purposes since 5.22 -it allows completion for almost all of JME's classes. 5.23 +it allows completion for almost all of JME's classes from the REPL. 5.24 5.25 Out of curiousity, let's see just how many classes =mega-import-jme3= 5.26 imports: 5.27 @@ -66,7 +66,7 @@ 5.28 : 955 classes 5.29 5.30 5.31 -** Utilities 5.32 +* Utilities 5.33 5.34 The utilities here come in three main groups: 5.35 - Changing settings in a running =Application= 5.36 @@ -79,8 +79,8 @@ 5.37 #+srcname: util 5.38 #+begin_src clojure 5.39 (ns cortex.util 5.40 - "Utility functions for making jMonkeyEngine easier to program from 5.41 - clojure" 5.42 + "Utility functions for making jMonkeyEngine3 easier to program from 5.43 + clojure." 5.44 {:author "Robert McIntyre"} 5.45 (:use cortex.world) 5.46 (:use clojure.contrib.def) 5.47 @@ -105,16 +105,17 @@ 5.48 will always output to the REPL") 5.49 5.50 (defn position-camera 5.51 - ([game position direction up] 5.52 - (doto (.getCamera game) 5.53 + "Change the position of the in-world camera." 5.54 + ([world position direction up] 5.55 + (doto (.getCamera world) 5.56 (.setLocation ) 5.57 (.lookAt direction up))) 5.58 - ([game position direction] 5.59 + ([world position direction] 5.60 (position-camera 5.61 - game position direction Vector3f/UNIT_Y))) 5.62 + world position direction Vector3f/UNIT_Y))) 5.63 5.64 (defn enable-debug 5.65 - "Turn on the debug wireframes for every object in this simulation" 5.66 + "Turn on debug wireframes for every object in this simulation." 5.67 [world] 5.68 (.enableDebug 5.69 (.getPhysicsSpace 5.70 @@ -123,11 +124,11 @@ 5.71 BulletAppState)) 5.72 (asset-manager))) 5.73 5.74 -(defn set-gravity 5.75 +(defn set-gravity 5.76 "In order to change the gravity of a scene, it is not only necessary 5.77 to set the gravity variable, but to \"tap\" every physics object in 5.78 the scene to reactivate physics calculations." 5.79 - [game gravity] 5.80 + [world gravity] 5.81 (traverse 5.82 (fn [geom] 5.83 (if-let 5.84 @@ -137,20 +138,20 @@ 5.85 (.setGravity control gravity) 5.86 ;; tappsies! 5.87 (.applyImpulse control Vector3f/ZERO Vector3f/ZERO)))) 5.88 - (.getRootNode game))) 5.89 + (.getRootNode world))) 5.90 5.91 (defn add-element 5.92 - "Add the Spatial to the game's environment" 5.93 - ([game element node] 5.94 + "Add the Spatial to the world's environment" 5.95 + ([world element node] 5.96 (.addAll 5.97 (.getPhysicsSpace 5.98 (.getState 5.99 - (.getStateManager game) 5.100 + (.getStateManager world) 5.101 BulletAppState)) 5.102 element) 5.103 (.attachChild node element)) 5.104 - ([game element] 5.105 - (add-element game element (.getRootNode game)))) 5.106 + ([world element] 5.107 + (add-element world element (.getRootNode world)))) 5.108 5.109 (defn apply-map 5.110 "Like apply, but works for maps and functions that expect an 5.111 @@ -186,7 +187,7 @@ 5.112 GImpact? 5.113 ]) 5.114 5.115 -(def base-shape 5.116 +(defvar base-shape 5.117 (shape-description. 5.118 "default-shape" 5.119 false 5.120 @@ -201,7 +202,8 @@ 5.121 Quaternion/IDENTITY 5.122 (Box. Vector3f/ZERO 0.5 0.5 0.5) 5.123 true 5.124 - false)) 5.125 + false) 5.126 + "Basic settings for shapes.") 5.127 5.128 (defn make-shape 5.129 [#^shape-description d] 5.130 @@ -210,8 +212,9 @@ 5.131 geom (Geometry. (:name d) (:shape d))] 5.132 (if (:texture d) 5.133 (let [key (TextureKey. (:texture d))] 5.134 - (.setGenerateMips key true) 5.135 - (.setTexture mat "ColorMap" (.loadTexture asset-manager key)))) 5.136 + ;;(.setGenerateMips key true) 5.137 + ;;(.setTexture mat "ColorMap" (.loadTexture asset-manager key)) 5.138 + )) 5.139 (if (:color d) (.setColor mat "Color" (:color d))) 5.140 (.setMaterial geom mat) 5.141 (if-let [rotation (:rotation d)] (.rotate geom rotation)) 5.142 @@ -248,7 +251,6 @@ 5.143 ([] (sphere 0.5))) 5.144 #+end_src 5.145 5.146 - 5.147 *** Viewing Objects 5.148 5.149 #+srcname: world-view 5.150 @@ -279,12 +281,13 @@ 5.151 (.setDirection 5.152 (.normalizeLocal (Vector3f. 1 0 -2))) 5.153 (.setColor ColorRGBA/White))] 5.154 + ;; lights are required to view some objects. 5.155 (.addLight (.getRootNode world) sun))) 5.156 no-op)))) 5.157 #+end_src 5.158 5.159 Here I make the =Viewable= protocol and extend it to JME's types. Now 5.160 -hello-world can be written as easily as: 5.161 +JME3's =hello-world= can be written as easily as: 5.162 5.163 #+begin_src clojure :results silent 5.164 (cortex.util/view (cortex.util/box))
6.1 --- a/org/world.org Wed Nov 02 11:03:12 2011 -0700 6.2 +++ b/org/world.org Thu Nov 03 08:28:26 2011 -0700 6.3 @@ -1,4 +1,4 @@ 6.4 -#+title: A World for the Creatures 6.5 +#+title: A Virtual World for Sensate Creatures 6.6 #+author: Robert McIntyre 6.7 #+email: rlm@mit.edu 6.8 #+description: Creating a Virtual World for AI constructs using clojure and JME3 6.9 @@ -31,12 +31,12 @@ 6.10 - Use a map from keys->functions to specify key-bindings. 6.11 - Use functions to create objects separately from any particular 6.12 application. 6.13 - - Use an REPL -- this means that there's only ever one JVM, and 6.14 + - Use a REPL -- this means that there's only ever one JVM, and 6.15 Applications come and go. 6.16 6.17 Since most development work using jMonkeyEngine is done in Java, jme3 6.18 supports "the Java way" quite well out of the box. To work "the 6.19 -clojure way", it necessary to wrap the jme3 elements that deal with 6.20 +clojure way", it necessary to wrap the JME3 elements that deal with 6.21 the Application life-cycle with a REPL driven interface. 6.22 6.23 The most important modifications are: 6.24 @@ -57,11 +57,12 @@ 6.25 (:use [pokemon [lpsolve :only [constant-map]]]) 6.26 (:use [clojure.contrib [str-utils :only [re-gsub]]]) 6.27 6.28 + (:import com.aurellem.capture.IsoTimer) 6.29 + 6.30 (:import com.jme3.math.Vector3f) 6.31 (:import com.jme3.scene.Node) 6.32 (:import com.jme3.system.AppSettings) 6.33 (:import com.jme3.system.JmeSystem) 6.34 - (:import com.jme3.system.IsoTimer) 6.35 (:import com.jme3.input.KeyInput) 6.36 (:import com.jme3.input.controls.KeyTrigger) 6.37 (:import com.jme3.input.controls.MouseButtonTrigger) 6.38 @@ -83,9 +84,8 @@ 6.39 (doto (AppSettings. true) 6.40 (.setFullscreen false) 6.41 (.setTitle "Aurellem.") 6.42 - ;; disable 32 bit stuff for now 6.43 - ;;(.setAudioRenderer "Send") 6.44 - ) 6.45 + ;; The "Send" AudioRenderer supports sumulated hearing. 6.46 + (.setAudioRenderer "Send")) 6.47 "These settings control how the game is displayed on the screen for 6.48 debugging purposes. Use binding forms to change this if desired. 6.49 Full-screen mode does not work on some computers.") 6.50 @@ -102,7 +102,8 @@ 6.51 =Application= whenever they extend that class. However, 6.52 =AssetManagers= are useful on their own to create objects/ materials, 6.53 independent from any particular application. =(asset-manager)= makes 6.54 -object creation less tightly bound to Application initialization. 6.55 +object creation less tightly bound to a particular Application 6.56 +Instance. 6.57 6.58 6.59 ** Exception Protection 6.60 @@ -170,18 +171,18 @@ 6.61 (map (fn [name] 6.62 (.addListener 6.63 ^InputManager input-manager game 6.64 - (into-array String [name]))) (keys key-map)))) 6.65 + (into-array String [name]))) (keys key-map)))) 6.66 6.67 #+end_src 6.68 6.69 These functions are for controlling the world through the keyboard and 6.70 mouse. 6.71 6.72 -I reuse =constant-map= from [[../../pokemon-types/html/lpsolve.html#sec-3-3-4][=pokemon.lpsolve=]] to get the numerical 6.73 +I reuse =constant-map= from [[../../pokemon-types/html/lpsolve.html#sec-3-2-4][=pokemon.lpsolve=]] to get the numerical 6.74 values for all the keys defined in the =KeyInput= class. The 6.75 documentation for =constant-map= is: 6.76 6.77 -#+begin_src clojure :results output 6.78 +#+begin_src clojure :results output :exports both 6.79 (doc pokemon.lpsolve/constant-map) 6.80 #+end_src 6.81 6.82 @@ -220,7 +221,6 @@ 6.83 : ["key-o" #<KeyTrigger com.jme3.input.controls.KeyTrigger@685c53ff>] 6.84 : ["key-at" #<KeyTrigger com.jme3.input.controls.KeyTrigger@4c3e2e5f>]) 6.85 6.86 - 6.87 ** World Creation 6.88 #+srcname: world 6.89 #+begin_src clojure :results silent 6.90 @@ -271,9 +271,7 @@ 6.91 value always being the same. 6.92 " 6.93 [root-node key-map setup-fn update-fn] 6.94 - (let [physics-manager (BulletAppState.) 6.95 - shadow-renderer (BasicShadowRenderer. 6.96 - (asset-manager) (int 256))] 6.97 + (let [physics-manager (BulletAppState.)] 6.98 (doto 6.99 (proxy [SimpleApplication ActionListener] [] 6.100 (simpleInitApp 6.101 @@ -302,15 +300,6 @@ 6.102 (.add (.getPhysicsSpace physics-manager) 6.103 (.getControl geom n)))))) 6.104 (.getRootNode this)) 6.105 - ;;(.addAll (.getPhysicsSpace physics-manager) (.getRootNode this)) 6.106 - 6.107 - ;; set some basic defaults for the shadow renderer. 6.108 - ;; these can be undone in the setup function 6.109 - (.setDirection shadow-renderer 6.110 - (.normalizeLocal (Vector3f. -1 -1 -1))) 6.111 - (.addProcessor (.getViewPort this) shadow-renderer) 6.112 - (.setShadowMode (.getRootNode this) 6.113 - RenderQueue$ShadowMode/Off) 6.114 ;; call the supplied setup-fn 6.115 (if setup-fn 6.116 (setup-fn this))))