#+title: Simulated Senses

 #+author: Robert McIntyre

 #+email: rlm@mit.edu

 #+MATHJAX: align:"left" mathml:t path:"../aurellem/src/MathJax/MathJax.js"

 #+STYLE: <link rel="stylesheet" type="text/css" href="../aurellem/src/css/aurellem.css"/>

 #+BABEL: :exports both :noweb yes :cache no :mkdirp yes

 #+INCLUDE: ../aurellem/src/templates/level-0.org

 #+description: Simulating senses for AI research using JMonkeyEngine3

 

 * Background

 Artificial Intelligence has tried and failed for more than half a

 century to produce programs as flexible, creative, and “intelligent”

 as the human mind itself. Clearly, we are still missing some important

 ideas concerning intelligent programs or we would have strong AI

 already. What idea could be missing?

 

 When Turing first proposed his famous “Turing Test” in the

 groundbreaking paper [[./sources/turing.pdf][/Computing Machines and Intelligence/]], he gave

 little importance to how a computer program might interact with the

 world:

 

 #+BEGIN_QUOTE

 \ldquo{}We need not be too concerned about the legs, eyes, etc. The example of

 Miss Helen Keller shows that education can take place provided that

 communication in both directions between teacher and pupil can take

 place by some means or other.\rdquo{}

 #+END_QUOTE

 

 And from the example of Hellen Keller he went on to assume that the

 only thing a fledgling AI program could need by way of communication

 is a teletypewriter. But Hellen Keller did possess vision and hearing

 for the first few months of her life, and her tactile sense was far

 more rich than any text-stream could hope to achieve. She possessed a

 body she could move freely, and had continual access to the real world

 to learn from her actions.

 

 I believe that our programs are suffering from too little sensory

 input to become really intelligent. Imagine for a moment that you

 lived in a world completely cut off form all sensory stimulation. You

 have no eyes to see, no ears to hear, no mouth to speak. No body, no

 taste, no feeling whatsoever. The only sense you get at all is a

 single point of light, flickering on and off in the void. If this was

 your life from birth, you would never learn anything, and could never

 become intelligent. Actual humans placed in sensory deprivation

 chambers experience hallucinations and can begin to loose their sense

 of reality in as little as 15 minutes[sensory-deprivation]. Most of

 the time, the programs we write are in exactly this situation. They do

 not interface with cameras and microphones, and they do not control a

 real or simulated body or interact with any sort of world.

 

 

 * Simulation vs. Reality

 I want demonstrate that multiple senses are what enable

 intelligence. There are two ways of playing around with senses and

 computer programs:

 

 The first is to go entirely with simulation: virtual world, virtual

 character, virtual senses. The advantages are that when everything is

 a simulation, experiments in that simulation are absolutely

 reproducible. It's also easier to change the character and world to

 explore new situations and different sensory combinations.

 

 

 ** Issues with Simulation

 

 If the world is to be simulated on a computer, then not only do you

 have to worry about whether the character's senses are rich enough to

 learn from the world, but whether the world itself is rendered with

 enough detail and realism to give enough working material to the

 character's senses. To name just a few difficulties facing modern

 physics simulators: destructibility of the environment, simulation of

 water/other fluids, large areas, nonrigid bodies, lots of objects,

 smoke. I don't know of any computer simulation that would allow a

 character to take a rock and grind it into fine dust, then use that

 dust to make a clay sculpture, at least not without spending years

 calculating the interactions of every single small grain of

 dust. Maybe a simulated world with today's limitations doesn't provide

 enough richness for real intelligence to evolve.

 

 ** Issues with Reality

 

 The other approach for playing with senses is to hook your software up

 to real cameras, microphones, robots, etc., and let it loose in the

 real world. This has the advantage of eliminating concerns about

 simulating the world at the expense of increasing the complexity of

 implementing the senses. Instead of just grabbing the current rendered

 frame for processing, you have to use an actual camera with real

 lenses and interact with photons to get an image. It is much harder to

 change the character, which is now partly a physical robot of some

 sort, since doing so involves changing things around in the real world

 instead of modifying lines of code. While the real world is very rich

 and definitely provides enough stimulation for intelligence to develop

 as evidenced by our own existence, it is also uncontrollable in the

 sense that a particular situation cannot be recreated perfectly or

 saved for later use. It is harder to conduct science because it is

 harder to repeat an experiment. The worst thing about using the real

 world instead of a simulation is the matter of time. Instead of

 simulated time you get the constant and unstoppable flow of real

 time. This severely limits the sorts of software you can use to

 program the AI because all sense inputs must be handled in real

 time. Complicated ideas may have to be implemented in hardware or may

 simply be impossible given the current speed of our

 processors. Contrast this with a simulation, in which the flow of time

 in the simulated world can be slowed down to accommodate the

 limitations of the character's programming. In terms of cost, doing

 everything in software is far cheaper than building custom real-time

 hardware. All you need is a laptop and some patience.

 

 * Choose a Simulation Engine

 

 Mainly because of issues with controlling the flow of time, I chose to

 simulate both the world and the character. I set out to make a minimal

 world in which I could embed a character with multiple senses. My main

 goal is to make an environment where I can perform further experiments

 in simulated senses.

 

 As Carl Sagan once said, "If you wish to make an apple pie from

 scratch, you must first invent the universe.” I examined many

 different 3D environments to try and find something I would use as the

 base for my simulation; eventually the choice came down to three

 engines: the Quake II engine, the Source Engine, and jMonkeyEngine.

 

 ** Quake II/Jake2

 

 I spent a bit more than a month working with the Quake II Engine from

 ID software to see if I could use it for my purposes. All the source

 code was released by ID software into the Public Domain several years

 ago, and as a result it has been ported and modified for many

 different reasons. This engine was famous for its advanced use of

 realistic shading and had decent and fast physics

 simulation. Researchers at Princeton [[http://www.nature.com/nature/journal/v461/n7266/pdf/nature08499.pdf][used this code]] to study spatial

 information encoding in the hippocampal cells of rats. Those

 researchers created a special Quake II level that simulated a maze,

 and added an interface where a mouse could run around inside a ball in

 various directions to move the character in the simulated maze. They

 measured hippocampal activity during this exercise to try and tease

 out the method in which spatial data was stored in that area of the

 brain. I find this promising because if a real living rat can interact

 with a computer simulation of a maze in the same way as it interacts

 with a real-world maze, then maybe that simulation is close enough to

 reality that a simulated sense of vision and motor control interacting

 with that simulation could reveal useful information about the real

 thing. It happens that there is a Java port of the original C source

 code called Jake2. The port demonstrates Java's OpenGL bindings and

 runs anywhere from 90% to 105% as fast as the C version. After

 reviewing much of the source of Jake2, I eventually rejected it

 because the engine is too tied to the concept of a first-person

 shooter game. One of the problems I had was that there does not seem

 to be any easy way to attach multiple cameras to a single

 character. There are also several physics clipping issues that are

 corrected in a way that only applies to the main character and does

 not apply to arbitrary objects. While there is a large community of

 level modders, I couldn't find a community to support using the engine

 to make new things.

 

 ** Source Engine

 

 The Source Engine evolved from the Quake II and Quake I engines and is

 used by Valve in the Half-Life series of games. The physics simulation

 in the Source Engine is quite accurate and probably the best out of

 all the engines I investigated. There is also an extensive community

 actively working with the engine. However, applications that use the

 Source Engine must be written in C++, the code is not open, it only

 runs on Windows, and the tools that come with the SDK to handle models

 and textures are complicated and awkward to use.

 

 ** jMonkeyEngine

 

 jMonkeyEngine is a new library for creating games in Java. It uses

 OpenGL to render to the screen and uses screengraphs to avoid drawing

 things that do not appear on the screen. It has an active community

 and several games in the pipeline. The engine was not built to serve

 any particular game but is instead meant to be used for any 3D

 game. After experimenting with each of these three engines and a few

 others for about 2 months I settled on jMonkeyEngine. I chose it

 because it had the most features out of all the open projects I looked

 at, and because I could then write my code in Clojure, an

 implementation of LISP that runs on the JVM.

 

 * Setup

 

 First, I checked out the source to jMonkeyEngine:

 

 #+srcname: checkout 

 #+begin_src sh :results verbatim

 svn checkout http://jmonkeyengine.googlecode.com/svn/trunk/engine jme3

 #+end_src

 

 #+results: checkout

 : Checked out revision 7975.

 

 

 Building jMonkeyEngine is easy enough:

 

 #+srcname: build

 #+begin_src sh :results verbatim

 cd jme3

 ant jar | tail -n 2

 #+end_src

 

 #+results: build

 : BUILD SUCCESSFUL

 : Total time: 15 seconds

 

 

 Also build the javadoc:

 

 #+srcname: javadoc

 #+begin_src sh :results verbatim

 cd jme3

 ant javadoc | tail -n 2

 #+end_src

 

 #+results: javadoc

 : BUILD SUCCESSFUL

 : Total time: 12 seconds

 

 Now, move the jars from the compilation into the project's lib folder.

 

 #+srcname: move-jars

 #+begin_src sh :results verbatim

 mkdir -p lib 

 mkdir -p src

 cp jme3/dist/jMonkeyEngine3.jar lib/

 cp jme3/dist/lib/* lib/

 ls lib

 #+end_src

 

 #+results: move-jars

 #+begin_example

 eventbus-1.4.jar

 jbullet.jar

 jheora-jst-debug-0.6.0.jar

 jinput.jar

 jME3-jbullet.jar

 jME3-lwjgl-natives.jar

 jME3-testdata.jar

 jME3-test.jar

 jMonkeyEngine3.jar

 j-ogg-oggd.jar

 j-ogg-vorbisd.jar

 lwjgl.jar

 nifty-1.3.jar

 nifty-default-controls-1.3.jar

 nifty-examples-1.3.jar

 nifty-lwjgl-renderer-1.3.jar

 nifty-openal-soundsystem-1.0.jar

 nifty-style-black-1.3.jar

 nifty-style-grey-1.0.jar

 noise-0.0.1-SNAPSHOT.jar

 stack-alloc.jar

 vecmath.jar

 xmlpull-xpp3-1.1.4c.jar

 #+end_example

 

 It's good to create a =assets= directory in the style that the

 =AssetManager= will like.

 

 #+srcname: create-assets

 #+begin_src sh :results verbatim

 mkdir -p assets

 mkdir -p assets/Interface

 mkdir -p assets/Materials

 mkdir -p assets/MatDefs

 mkdir -p assets/Models

 mkdir -p assets/Scenes

 mkdir -p assets/Shaders

 mkdir -p assets/Sounds

 mkdir -p assets/Textures

 tree -L 1 assets

 #+end_src

 

 #+results: create-assets

 #+begin_example

 assets

 |-- Interface

 |-- MatDefs

 |-- Materials

 |-- Models

 |-- Scenes

 |-- Shaders

 |-- Sounds

 `-- Textures

 

 8 directories, 0 files

 #+end_example

 

 

 The java classpath should have all the jars contained in the =lib=

 directory as well as the src directory.

 

 For example, here is the file I use to run my REPL for clojure.

 

 #+include: "~/swank-all" src sh :exports code

 

 The important thing here is that =cortex/lib/*=, =cortex/src=, and

 =cortex/assets= appear on the classpath. (=cortex= is the base

 directory of this project.)

 

 #+srcname: pwd

 #+begin_src sh 

 pwd

 #+end_src

 

 #+results: pwd

 : /home/r/cortex

 

 

 * Simulation Base

   

 ** Imports

 First, I'll import jme core classes. 

 #+srcname: import

 #+begin_src clojure :results silent

 (ns cortex.import

   (:require swank.util.class-browse))

 

 (defn import-jme3 []

   (import '[com.jme3.system AppSettings JmeSystem]) 

   (import '[com.jme3.app Application SimpleApplication])

   (import 'com.jme3.material.Material)

   (import '[com.jme3.math Vector3f ColorRGBA Quaternion Transform])

   (import '[com.jme3.scene Node  Geometry])

   (import '[com.jme3.scene.shape Box Sphere Sphere$TextureMode])

   (import 'com.jme3.font.BitmapText)

   (import '[com.jme3.input KeyInput InputManager])

   (import '[com.jme3.input.controls

 	    ActionListener AnalogListener KeyTrigger MouseButtonTrigger])

   (import '[com.jme3.asset AssetManager DesktopAssetManager] )

   (import '[com.jme3.asset.plugins HttpZipLocator ZipLocator])

   (import '[com.jme3.light PointLight DirectionalLight])

   (import '[com.jme3.animation AnimControl Skeleton Bone])

   (import '[com.jme3.bullet.collision.shapes

 	    MeshCollisionShape SphereCollisionShape BoxCollisionShape])

   (import 'com.jme3.renderer.queue.RenderQueue$ShadowMode)

   (import 'jme3test.TestChooser)

   (import '[com.jme3.bullet PhysicsTickListener PhysicsSpace])

   (import '[com.jme3.bullet.joints SixDofJoint HingeJoint

 	    SliderJoint Point2PointJoint ConeJoint]))

 

 

 (defmacro permissive-import* [class-symbol]

   `(try

        (import ~class-symbol)

        (catch Exception e#

 	 (println "can't import " ~class-symbol))))

 

 (defn permissive-import [class-symbol]

   (eval (list 'cortex.import/permissive-import* class-symbol)))

 

 (defn selection-import [selection-fn]

   (dorun

    (map (comp permissive-import symbol)

 	(filter selection-fn

 		(map :name

 		     swank.util.class-browse/available-classes)))))

   

 (defn mega-import-jme3

   "ALL the jme classes. For REPL use."

   []

   (selection-import

    #(and

      (.startsWith % "com.jme3.")

      ;; Don't import the Lwjgl stuff since it can throw exceptions

      ;; upon being loaded.

      (not (re-matches #".*Lwjgl.*" %)))))

 #+end_src  

 

 The =mega-import-jme3= is quite usefull for debugging purposes since

 it allows completion for almost all of JME's classes

 

 ** Simplification

 *** World

 

 It is comvienent to wrap the JME elements that deal with creating a

 world, creation of basic objects, and Keyboard input with a nicer

 interface (at least for my purposes).

 

 #+srcname: world-inputs

 #+begin_src clojure :results silent

 (ns cortex.world)

 (require 'cortex.import)

 (use 'clojure.contrib.def)

 (rlm.rlm-commands/help)

 (cortex.import/mega-import-jme3)

 

 (defvar *app-settings*

   (doto (AppSettings. true)

     (.setFullscreen false)

     (.setTitle "Aurellem.")

     ;; disable 32 bit stuff for now

     ;;(.setAudioRenderer "Send")

     )

   "These settings control how the game is displayed on the screen for

    debugging purposes.  Use binding forms to change this if desired.

    Full-screen mode does not work on some computers.")

 

 (defn asset-manager

   "returns a new, configured assetManager" []

   (JmeSystem/newAssetManager

    (.getResource

     (.getContextClassLoader (Thread/currentThread))

     "com/jme3/asset/Desktop.cfg")))

 

 (defmacro no-exceptions

   "Sweet relief like I never knew."

   [& forms]

   `(try ~@forms (catch Exception e# (.printStackTrace e#))))

 

 (defn thread-exception-removal []

   (println "removing exceptions from " (Thread/currentThread))

   (.setUncaughtExceptionHandler

    (Thread/currentThread)

    (proxy [Thread$UncaughtExceptionHandler] []

      (uncaughtException

       [thread thrown]

       (println "uncaught-exception thrown in " thread)

       (println (.getMessage thrown))))))

 

 (def println-repl (bound-fn [& args] (apply println args)))

 

 (use '[pokemon [lpsolve :only [constant-map]]])

 

 (defn no-op [& _])

 

 (defn all-keys

   "Construct a map of strings representing all the manual inputs from

    either the keyboard or mouse." 

   []

   (let [inputs (constant-map KeyInput)]

     (assoc

 	(zipmap (map (fn [field]

 		       (.toLowerCase (re-gsub #"_" "-" field))) (vals inputs))

 		(map (fn [val] (KeyTrigger. val)) (keys inputs)))

       ;;explicitly add mouse controls

       "mouse-left" (MouseButtonTrigger. 0)

       "mouse-middle" (MouseButtonTrigger. 2)

       "mouse-right" (MouseButtonTrigger. 1))))

 

 (defn initialize-inputs

   "more java-interop cruft to establish keybindings for a particular virtual world"

   [game  input-manager key-map]

   (doall (map (fn [[name trigger]]

 		(.addMapping ^InputManager input-manager

 			     name (into-array (class trigger) [trigger]))) key-map))

   (doall (map (fn [name] 

 		(.addListener ^InputManager input-manager game

 			      (into-array String  [name]))) (keys key-map))))

 

 #+end_src

 

 These functions are all for debug controlling of the world through

 keyboard and mouse. 

 

 We reuse  =constant-map= from =pokemon.lpsolve= to get the numerical

 values for all the keys defined in the =KeyInput= class. The

 documentation for =constant-map= is:

 

 #+begin_src clojure :results output

 (doc pokemon.lpsolve/constant-map)

 #+end_src

 

 #+results:

 : -------------------------

 : pokemon.lpsolve/constant-map

 : ([class])

 :   Takes a class and creates a map of the static constant integer

 :   fields with their names.  This helps with C wrappers where they have

 :   just defined a bunch of integer constants instead of enums

 

 

 Then, =all-keys= converts the constant names like =KEY_J= to the more

 clojure-like =key-j=, and returns a map from these keys to

 jMonkeyEngine KeyTrigger objects, the use of which will soon become

 apparent. =all-keys= also adds the three mouse button controls to the

 map.

 

 #+srcname: world

 #+begin_src clojure :results silent

 (in-ns 'cortex.world)

 

 (defn traverse

   "apply f to every non-node, deeply"

   [f node]

   (if (isa? (class node) Node)

     (dorun (map (partial traverse f) (.getChildren node)))

     (f node)))

 

 (def gravity (Vector3f. 0 -9.81 0))

 

 (defn world

   [root-node key-map setup-fn update-fn]

   (let [physics-manager (BulletAppState.)

 	shadow-renderer (BasicShadowRenderer. (asset-manager) (int 256))

 	;;maybe use a better shadow renderer someday!

 	;;shadow-renderer (PssmShadowRenderer. (asset-manager) 256 1)

 	]   	

   (doto

       (proxy [SimpleApplication ActionListener] []

 	(simpleInitApp

 	 []

 	 (no-exceptions

 	  (.setTimer this (IsoTimer. 60))

 	  ;; Create key-map.

 	  (.setFrustumFar (.getCamera this) 300)

 	  (initialize-inputs this (.getInputManager this) (all-keys))

 	  ;; Don't take control of the mouse

 	  (org.lwjgl.input.Mouse/setGrabbed false)

 	  ;; add all objects to the world

 	  (.attachChild (.getRootNode this) root-node)

 	  ;; enable physics

 	  ;; add a physics manager

 	  (.attach (.getStateManager this) physics-manager)

 	  (.setGravity (.getPhysicsSpace physics-manager) gravity)

 

 

 	  ;; go through every object and add it to the physics manager

 	  ;; if relavant.

 	   (traverse (fn [geom]

 		       (dorun

 			(for [n (range (.getNumControls geom))]

 			  (do

 			    (println-repl "adding control " (.getControl geom n))

 			    (.add (.getPhysicsSpace physics-manager)

 				  (.getControl geom n))))))

 		     (.getRootNode this))

 	  ;;(.addAll (.getPhysicsSpace physics-manager) (.getRootNode this))

 

 	  (setup-fn this)

 	  (.setDirection shadow-renderer

 			 (.normalizeLocal (Vector3f. -1 -1 -1)))

 	  (.addProcessor (.getViewPort this) shadow-renderer)

 	  (.setShadowMode (.getRootNode this) RenderQueue$ShadowMode/Off)

 	  ))

 	(simpleUpdate

 	 [tpf]

 	 (no-exceptions

 	  (update-fn this tpf))) 

 	(onAction

 	 [binding value tpf]

 	 ;; whenever a key is pressed, call the function returned from

 	 ;; key-map.

 	 (no-exceptions

 	  (if-let [react (key-map binding)]

 	    (react this value)))))

     ;; don't show a menu to change options.

     

     (.setShowSettings false)

     (.setPauseOnLostFocus false)

     (.setSettings *app-settings*))))

 

 (defn apply-map

   "Like apply, but works for maps and functions that expect an implicit map

    and nothing else as in (fn [& {}]).

 ------- Example -------

  (defn jjj [& {:keys [www] :or {www \"oph yeah\"} :as env}] (println www))

  (apply-map jjj {:www \"whatever\"})

   -->\"whatever\""

   [fn m]

   (apply fn (reduce #(into %1 %2) [] m)))

 

 #+end_src 

 

 

 =world= is the most important function here.  

 ***  TODO more documentation

 

 #+srcname: world-shapes

 #+begin_src clojure :results silent

 (in-ns 'cortex.world)

 (defrecord shape-description

   [name

    color

    mass

    friction

    texture

    material

    position

    rotation

    shape

    physical?])

 

 (def base-shape

      (shape-description.

       "default-shape"

       false

       ;;ColorRGBA/Blue

       1.0 ;; mass

       1.0 ;; friction

       ;; texture

       "Textures/Terrain/BrickWall/BrickWall.jpg"

       ;; material

       "Common/MatDefs/Misc/Unshaded.j3md"

       Vector3f/ZERO

       Quaternion/IDENTITY

       (Box. Vector3f/ZERO 0.5 0.5 0.5)

       true))

 

 (defn make-shape

   [#^shape-description d]

   (let [mat (Material. (asset-manager) (:material d))

 	geom (Geometry. (:name d) (:shape d))]

     (if (:texture d)

       (let [key (TextureKey. (:texture d))]

 	(.setGenerateMips key true)

 	(.setTexture mat "ColorMap" (.loadTexture (asset-manager) key))))

     (if (:color d) (.setColor mat "Color" (:color d)))

     (.setMaterial geom mat)

     (if-let [rotation (:rotation d)] (.rotate geom rotation))

     (.setLocalTranslation geom (:position d))

     (if (:physical? d)

       (let [impact-shape (doto (GImpactCollisionShape. (.getMesh geom)) (.setMargin 0))

 	    physics-control (RigidBodyControl.

 			     impact-shape 

 			     (float (:mass d)))]

 	(.createJmeMesh impact-shape)

 	(.addControl geom physics-control)

 	;;(.setSleepingThresholds physics-control (float 0) (float 0))

 	(.setFriction physics-control (:friction d))))

     ;;the default is to keep this node in the physics engine forever.

     ;;these commands must come after the control is added to the geometry.

     ;;

     geom))

   

 (defn box

   ([l w h & {:as options}]

      (let [options (merge base-shape options)]

        (make-shape (assoc options

 		     :shape (Box. l w h)))))

   ([] (box 0.5 0.5 0.5)))

 

 (defn sphere

   ([r & {:as options}]

      (let [options (merge base-shape options)]

        (make-shape (assoc options

 		     :shape (Sphere. 32 32 (float r)))))) 

   ([] (sphere 0.5)))

 

 (defn add-element [game node]

   (.addAll

    (.getPhysicsSpace

     (.getState

      (.getStateManager game)

      BulletAppState))

     node)

   (.attachChild (.getRootNode game) node))

 

 (defn set-gravity*

   [game gravity]

   (traverse

    (fn [geom]

      (if-let

 	 [control (.getControl geom RigidBodyControl)]

        (do

 	 (.setGravity control gravity)

 	 (.applyImpulse control Vector3f/ZERO Vector3f/ZERO)

 	 )))

    (.getRootNode game)))

 

 #+end_src

 

 These are convienence functions for creating JME objects and

 manipulating a world.

 

 #+srcname: world-view

 #+begin_src clojure :results silent

 (in-ns 'cortex.world)

 

 (defprotocol Viewable

   (view [something]))

 

 (extend-type com.jme3.scene.Geometry

   Viewable

   (view [geo]

 	(view (doto (Node.)(.attachChild geo)))))

 

 (extend-type com.jme3.scene.Node

   Viewable

   (view [node]

 	(.start

 	 (world node

 		{}

 		(fn [world]

 		  (.enableDebug 

 		   (.getPhysicsSpace

 		    (.getState

 		     (.getStateManager world)

 		     BulletAppState))

 		   (asset-manager))

 		  (set-gravity* world Vector3f/ZERO)

 ;;		  (set-gravity* world (Vector3f. 0 (float -0.4) 0))

 		  (let [sun (doto (DirectionalLight.)

 			      (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))

 			      (.setColor ColorRGBA/White))]

 		    (.addLight (.getRootNode world) sun)))

 		no-op))))

 

 (defn position-camera [game]

   (doto (.getCamera game)

     (.setLocation (Vector3f. 0 6 6))

     (.lookAt Vector3f/ZERO (Vector3f. 0 1 0))))

 

 #+end_src

 

 Here I make the =Viewable= protocol and extend it to JME's types.  Now

 hello-world can be written as easily as:

 

 #+begin_src clojure :results silent

 (cortex.world/view (cortex.world/box))

 #+end_src

 

 ** Hello

 Here are the jmonkeyengine "Hello" programs translated to clojure.

 *** Hello Simple App

 Here is the hello world example for jme3 in clojure.

 It's a more or less direct translation from the java source

 from

 http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_simpleapplication.

 

 Of note is the fact that since we don't have access to the

 =AssetManager= via extendig =SimpleApplication=, we have to build one

 ourselves. 

 

 #+srcname: hello-simple-app

 #+begin_src clojure :results silent

 (ns hello.hello-simple-app)

 (require 'cortex.import)

 (use 'clojure.contrib.def)

 (rlm.rlm-commands/help)

 (cortex.import/import-jme3)

 (use 'cortex.world)

 

 

 (def cube (Box. Vector3f/ZERO 1 1 1))

 

 (def geom (Geometry. "Box" cube))

 

 (def mat (Material. (asset-manager) "Common/MatDefs/Misc/Unshaded.j3md"))

 

 (.setColor mat "Color" ColorRGBA/Blue)

 

 (.setMaterial geom mat)

 

 (defn simple-app []

   (doto

       (proxy [SimpleApplication] []

 	(simpleInitApp

 	 []

 	 ;; Don't take control of the mouse

 	 (org.lwjgl.input.Mouse/setGrabbed false)

 	 (.attachChild (.getRootNode this) geom)))

     ;; don't show a menu to change options.

     (.setShowSettings false)

     (.setPauseOnLostFocus false)

     (.setSettings *app-settings*)))

 #+end_src

 

 Running this program will begin a new jMonkeyEngine game which

 displays a single blue cube.

 

 #+begin_src clojure :exports code :results silent

 (.start (hello.hello-simple-app/simple-app))

 #+end_src

 

 #+caption: the simplest JME game.

 [[./images/simple-app.jpg]]

 

 

 

 *** Hello Physics

 From http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_physics

 

 #+srcname: brick-wall-header

 #+begin_src clojure :results silent

 (ns hello.brick-wall)

 (require 'cortex.import)

 (use 'clojure.contrib.def)

 (rlm.rlm-commands/help)

 (cortex.import/mega-import-jme3)

 (use '[pokemon [lpsolve :only [constant-map]]])

 (use 'cortex.world)

 #+end_src

 

 #+srcname: brick-wall-body

 #+begin_src clojure :results silent

 (in-ns 'hello.brick-wall)

 

 (defn floor

   "make a sturdy, unmovable physical floor"

   []

   (box 20 1 20 :mass 0 :color false :position (Vector3f. 0 -2 0)))

 

 (def brick-length 0.48)

 (def brick-width 0.24)

 (def brick-height 0.12)

 

 

 (defn brick* [position]

   (doto (box brick-length brick-height brick-width

 	     :position position :name "brick"

 	     :material "Common/MatDefs/Misc/Unshaded.j3md"

 	     :texture "Textures/Terrain/BrickWall/BrickWall.jpg"

 	     :mass 36)

     (->

      (.getMesh)

      (.scaleTextureCoordinates (Vector2f. 1 0.5)))

     ;;(.setShadowMode RenderQueue$ShadowMode/CastAndReceive)

     )

   )

 

 (defn inception-brick-wall

   "construct a physical brick wall"

   []

   (let [node (Node. "brick-wall")]

     (dorun

      (map (comp #(.attachChild node %) brick*)

 	  (for

 	      [x (range 15)

 	       y (range 10)

 	       z (range 1)]

 	    (Vector3f.

 	     (* brick-length x 1.03)

 	     (* brick-width y y 10)

 	     (* brick-height z)))))

     node))

 

 (defn gravity-toggle

   [new-value]

   (fn [game value]

     (println-repl "set gravity to " new-value)

     (if value

       (set-gravity* game new-value)

       (set-gravity* game gravity))))

 

 (defn fire-cannon-ball []

   (fn [game value]

     (if (not value)

       (let [camera (.getCamera game)

 	    cannon-ball

 	    (sphere  0.7 

 		     :material "Common/MatDefs/Misc/Unshaded.j3md"

 		     :texture "Textures/PokeCopper.jpg"

 		     :position

 		     (.add (.getLocation camera)

 			   (.mult (.getDirection camera) (float 1)))

 		     :mass 3)] ;200 0.05

 	(.setShadowMode cannon-ball RenderQueue$ShadowMode/CastAndReceive)

 	(.setLinearVelocity

 	 (.getControl cannon-ball RigidBodyControl)

 	 (.mult (.getDirection camera) (float 50))) ;50

 	(add-element game cannon-ball)))))

 

 (defn floor* []

   (doto (box 10 0.1 5 :name "floor" ;10 0.1 5 ; 240 0.1 240

 	     :material "Common/MatDefs/Misc/Unshaded.j3md"

 	     :texture "Textures/Terrain/Pond/Pond.png"

 	     :position (Vector3f. 0 -0.1 0 )

 	     :mass 0)

     (->

      (.getMesh)

      (.scaleTextureCoordinates (Vector2f. 3 6)));64 64

     (->

      (.getMaterial)

      (.getTextureParam "ColorMap")

      (.getTextureValue)

      (.setWrap Texture$WrapMode/Repeat))

     (.setShadowMode RenderQueue$ShadowMode/Receive)

   ))

     

 (defn brick-wall* []

   (let [node (Node. "brick-wall")]

     (dorun

      (map

       (comp  #(.attachChild node %) brick*)

        (for [y (range 15)

 	     x (range 4)

 	     z (range 1)]

        	    (Vector3f.

        	     (+ (* 2 x brick-length)

 		(if (even? (+ y z)) 

 		  (/ brick-length 4) (/ brick-length -4)))

        	     (+ (* brick-height (inc (* 2 y))))

 	     (* 2 z brick-width) ))))

     (.setShadowMode node RenderQueue$ShadowMode/CastAndReceive)

     node))

 

 (defn brick-wall-game-run []

   (doto

       (world

        (doto (Node.) (.attachChild (floor*))

 	     (.attachChild (brick-wall*))

 	     )

        {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))

 	"key-m" (gravity-toggle (Vector3f. 0 0 9.81))

 	"key-l" (gravity-toggle (Vector3f. 9.81 0 0))

 	"key-j" (gravity-toggle (Vector3f. -9.81 0 0))

 	"key-k" (gravity-toggle Vector3f/ZERO)

 	"key-u" (gravity-toggle (Vector3f. 0 9.81 0))

 	"key-o" (gravity-toggle (Vector3f. 0 -9.81 0))

 	"key-f" (fn[game value] 

 		  (if (not value) (add-element game (brick-wall*))))

 	"key-return" (fire-cannon-ball)}

        position-camera

        (fn [& _]))     

     (.start)))

 #+end_src

 

 #+begin_src clojure :results silent

 (hello.brick-wall/brick-wall-game-run)

 #+end_src

 

 #+caption: the brick wall standing

 [[./images/brick-wall-standing.jpg]]

 

 #+caption: the brick wall after it has been knocked over by a "pok\eacute{}ball"

 [[./images/brick-wall-knocked-down.jpg]]

 

 *** Other Brick Games

 #+srcname: other-games

 #+begin_src clojure :results silent

 (ns cortex.other-games

    {:author "Dylan Holmes"})

 (use 'cortex.world)

 (use 'hello.brick-wall)

 (use 'cortex.import)

 (cortex.import/mega-import-jme3)

 

 (defn scad [position]

   (doto (box 0.1 0.1 0.1

 	     :position position :name "brick"

 	     :material "Common/MatDefs/Misc/Unshaded.j3md"

 	     :texture "Textures/Terrain/BrickWall/BrickWall.jpg"

 	     :mass 20)

     (->

      (.getMesh)

      (.scaleTextureCoordinates (Vector2f. 1 0.5))

      )

         (->	(.getControl RigidBodyControl)

 	(.setLinearVelocity (Vector3f. 0 100 0))

 	)

 	

     ;;(.setShadowMode RenderQueue$ShadowMode/Cast)

     ))

 

 

 (defn shrapnel []

   (let [node (Node. "explosion-day")]

     (dorun

      (map

       (comp  #(.attachChild node %) scad)

        (for [y (range 15)

 	     x (range 4)

 	     z (range 1)]

        	    (Vector3f.

        	     (+ (* 2 x brick-height)

 		(if (even? (+ y z)) (/ brick-height 4) (/ brick-height -4)))

        	     (+ (* brick-height (inc (* 2 y))))

 	     (* 2 z brick-height) ))))

     node))

 

 

 (def domino-height 0.48)

 (def domino-thickness 0.12)

 (def domino-width 0.24)

 

 (def domino-thickness 0.05)

 (def domino-width 0.5)

 (def domino-height 1)

 

 (defn domino

   ([position]

      (domino position (Quaternion/IDENTITY)))

   ([position rotation]

      (doto (box domino-width domino-height domino-thickness

 	     :position position :name "domino"

 	     :material "Common/MatDefs/Misc/Unshaded.j3md"

 	     :texture "Textures/Terrain/BrickWall/BrickWall.jpg"

 	     :mass 1

 	     :rotation rotation)

        (.setShadowMode RenderQueue$ShadowMode/CastAndReceive)

        )))

 

 

 (defn domino-row []

   (let [node (Node. "domino-row")]

     (dorun

      (map

       (comp  #(.attachChild node %) domino)

        (for [

 	     z (range 10)

 	     x (range 5)

 	     ]

        	    (Vector3f.

 	     (+ (* z domino-width) (* x 5 domino-width))

 	     (/ domino-height 1)

 	     (* -5.5 domino-thickness z) ))))

 

     node))

 

 (defn domino-cycle []

   (let [node (Node. "domino-cycle")]

     (dorun

      (map

       (comp  #(.attachChild node %) (partial apply domino) ) 

       (for [n (range 720)]

 	(let [space (* domino-height 5.5)

 	      r (fn[n] (* (+ n 3) domino-width 0.5)) 

 	      t (fn[n] (reduce

 			+

 			(map

 			 (fn dt[n] (/ space (* 2 (Math/PI) (r n))))

 			 (range n))))

 	      t (t n)

 	      r (r n)

 	      ct (Math/cos t)

 	      st (Math/sin t)

 	      ]

 	(list

        	    (Vector3f.

 	     (* -1 r st)

 	     (/ domino-height 1)

 	     (* r ct))

 	    (.fromAngleAxis (Quaternion.)

 			    (- (/ 3.1415926 2) t) (Vector3f. 0 1 0))

 	    )))

 	))

     node))

 

 

 (defn domino-game-run []

   (doto

       (world

        (doto (Node.) (.attachChild (floor*))

 	     )

        {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))

 	"key-m" (gravity-toggle (Vector3f. 0 0 9.81))

 	"key-l" (gravity-toggle (Vector3f. 9.81 0 0))

 	"key-j" (gravity-toggle (Vector3f. -9.81 0 0))

 	"key-k" (gravity-toggle (Vector3f. 0 9.81 0) )

 	"key-u" (fn[g v] ((gravity-toggle (Vector3f. 0 -0 0)) g true))

 	"key-o" (gravity-toggle (Vector3f. 0 -9.81 0))

 

 	"key-space"

 	(fn[game value]

 	  

 	  (if (not value)

 	    (let [d (domino (Vector3f. 0 (/ domino-height 0.25) 0)

 			    (.fromAngleAxis (Quaternion.)

 					    (/ Math/PI 2) (Vector3f. 0 1 0)))]

 	      (add-element game d))))

 	"key-f"

 	(fn[game value](if (not value) (add-element game (domino-cycle))))

 	"key-return" (fire-cannon-ball)}

        position-camera

        (fn [& _]))     

     (.start)))       

 #+end_src

 

 #+begin_src clojure :results silent

 (cortex.other-games/domino-game-run)

 #+end_src

 

 #+caption: floating dominos

 [[./images/dominos.jpg]]

 

 *** Hello Loop

 #+srcname: hello-loop

 #+begin_src clojure :results silent

 (ns hello.loop)

 (use 'cortex.world)

 (use 'cortex.import)

 (cortex.import/mega-import-jme3)

 (rlm.rlm-commands/help)

 

 (defn blue-cube []

   (box 1 1 1

        :color ColorRGBA/Blue

        :texture false

        :material "Common/MatDefs/Misc/Unshaded.j3md"

        :name "blue-cube"

        :physical? false))

 

 (defn blue-cube-game []

   (let [cube (blue-cube)

 	root (doto (Node.) (.attachChild cube))]

   (world root

 	 {}

 	 no-op

 	 (fn [game tpf]

 	   (.rotate cube 0.0 (* 2 tpf) 0.0)))))	 	 		  

 #+end_src

 

 *** Hello Collision

 

 #+srcname: hello-collision

 #+begin_src clojure :results silent

 (ns hello.collision)

 (use 'cortex.world)

 (use 'cortex.import)

 (use 'clojure.contrib.def)

 

 

 (cortex.import/mega-import-jme3)

 (rlm.rlm-commands/help)

 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])

 

 

 (defn environment []

      (let

 	 [scene-model

 	  (doto

 	      (.loadModel

 	       (doto (asset-manager)

 		 (.registerLocator

 		  "/home/r/cortex/assets/zips/town.zip" ZipLocator))

 	       "main.scene")

 	    (.setLocalScale (float 2.0)))

 	  collision-shape

 	  (CollisionShapeFactory/createMeshShape #^Node scene-model)

 	  landscape (RigidBodyControl. collision-shape 0)]

        (.setShadowMode scene-model RenderQueue$ShadowMode/CastAndReceive)

        (.addControl scene-model landscape)

        scene-model))

 	  

 (defn player-fn []

   (doto

       (CharacterControl.

        (CapsuleCollisionShape. (float 1.5) (float 6)(float 1))

        (float 0.05))

     (.setJumpSpeed 20)

     (.setFallSpeed 30)

     (.setGravity 30) ;30

     (.setPhysicsLocation (Vector3f. 0 10 0))))

 

 (defn lights []

   [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 1 1 1) (float 1))))

    (doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 0.7 0 1) (float 1))))

    (doto (DirectionalLight.)

      (.setColor (.mult ColorRGBA/White (float 0.9) ))

      (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])

 

 (defn night-lights []

   [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 0.275 0.467 0.784 1) (float 0.3))))

    (doto (DirectionalLight.)

      (.setColor (.mult ColorRGBA/White (float 0.2) ))

      (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])

 

 (def player (atom (player-fn)))

 

 (defn setup-fn [game]

   (dorun (map #(.addLight (.getRootNode game) %) (lights)))

   ;; set the color of the sky

   (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))

   ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)

   (doto (.getFlyByCamera game)

     (.setMoveSpeed (float 100))

     (.setRotationSpeed 3))

   (.add

    (.getPhysicsSpace

     (.getState (.getStateManager game) BulletAppState))

    @player)

 

    (doto (Node.) (.attachChild (.getRootNode game))

 	     (.attachChild (brick-wall*))

    )

 

 )

 

 

 (def walking-up? (atom false))

 (def walking-down? (atom false))

 (def walking-left? (atom false))

 (def walking-right? (atom false))

 

 (defn set-walk [walk-atom game value]

   ;;(println-repl "setting  stuff to " value)

   (reset! walk-atom value))

 

 (defn responses []

      {"key-w" (partial set-walk walking-up?)

       "key-d" (partial set-walk walking-right?)

       "key-s" (partial set-walk walking-down?)

       "key-a" (partial set-walk walking-left?)

       "key-return" (fire-cannon-ball)

       "key-space" (fn [game value] (.jump @player))

       })

      

 (defn update-fn

   [game tpf]

   (let [camera (.getCamera game)         

 	cam-dir (.multLocal

 		 (.clone

 		  (.getDirection camera)) (float 0.6))

 	cam-left (.multLocal

 		  (.clone

 		   (.getLeft camera)) (float 0.4))

 	walk-direction (Vector3f. 0 0 0)]

     

     (cond

      @walking-up?     (.addLocal walk-direction cam-dir)

      @walking-right?  (.addLocal walk-direction (.negate cam-left))

      @walking-down?   (.addLocal walk-direction (.negate cam-dir))

      @walking-left?   (.addLocal walk-direction cam-left))

     (.setWalkDirection @player walk-direction)

     (.setLocation camera (.getPhysicsLocation @player))))

     

 (defn run-game []

   (.start

    (world (environment)

 	  (responses)

 	  setup-fn

 	  update-fn)))

 #+end_src

 

 *** Hello Terrain

 #+srcname: hello-terrain

 #+begin_src clojure :results silent

 (ns hello.terrain)

 (use 'cortex.world)

 (use 'cortex.import)

 (use 'clojure.contrib.def)

 (import jme3tools.converters.ImageToAwt)

 

 

 (cortex.import/mega-import-jme3)

 (rlm.rlm-commands/help)

 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])

 

 

 (defn setup-fn [type game]

   (.setMoveSpeed (.getFlyByCamera game) 50)

   (.setFrustumFar (.getCamera game) 10000)

   (let [env (environment type)

 	cameras [(.getCamera game)]

 	control (TerrainLodControl. env cameras)]

     ;;(.addControl env control)

     (.attachChild (.getRootNode game) env)))                

 

 (defn environment [type]

   (let

       [mat_terrain

        (Material. (asset-manager) "Common/MatDefs/Terrain/Terrain.j3md")

        grass (.loadTexture (asset-manager) "Textures/Terrain/splat/grass.jpg")

        dirt (.loadTexture (asset-manager) "Textures/Terrain/splat/dirt.jpg")

        rock (.loadTexture (asset-manager) "Textures/Terrain/splat/road.jpg")

        heightmap-image (.loadTexture (asset-manager)

 				     ({:mountain "Textures/Terrain/splat/mountains512.png"

 				       :fortress "Textures/Terrain/splat/fortress512.png"

 				       }type))

        heightmap (ImageBasedHeightMap.

 		  (ImageToAwt/convert (.getImage heightmap-image) false true 0))

        terrain (do (.load heightmap)

 		   (TerrainQuad. "my terrain" 65 513 (.getHeightMap heightmap)))

        ]

     

     (dorun (map #(.setWrap % Texture$WrapMode/Repeat)

 		[grass dirt rock]))

 

     (doto mat_terrain

       (.setTexture "Tex1" grass)

       (.setFloat "Tex1Scale" (float 64))

 

       (.setTexture "Tex2" dirt)

       (.setFloat "Tex2Scale" (float 32))

 

       (.setTexture "Tex3" rock)

       (.setFloat "Tex3Scale" (float 128))

 

       (.setTexture "Alpha"

 		   (.loadTexture

 		    (asset-manager) 

 		    ({:mountain "Textures/Terrain/splat/alphamap.png"

 		      :fortress "Textures/Terrain/splat/alphamap2.png"} type))))

 

     (doto terrain

       (.setMaterial mat_terrain)

       (.setLocalTranslation 0 -100 0)

       (.setLocalScale 2 1 2))))

       

 

 

 (defn run-terrain-game [type]

   (.start

    (world

     (Node.)

     {}

     (partial setup-fn type)

     no-op)))

 #+end_src

 

 

 

 #+srcname: hello-animation

 #+begin_src clojure :results silent

 (ns hello.animation)

 (use 'cortex.world)

 (use 'cortex.import)

 (use 'clojure.contrib.def)

 (cortex.import/mega-import-jme3)

 (rlm.rlm-commands/help)

 (use '[hello [collision :only [lights]]])

 

 (defn stand

   [channel]

   (doto channel

     (.setAnim "stand" (float 0.5))

     (.setLoopMode LoopMode/DontLoop)

     (.setSpeed (float 1))))

 

 (defn anim-listener []

      (proxy [AnimEventListener] []

        (onAnimChange

 	[control channel animation-name]

 	(println-repl "RLM --- onAnimChange"))

        (onAnimCycleDone

 	[control channel animation-name]

 	(if (= animation-name "Walk")

 	  (stand channel)

 	  ))))

        

 (defn setup-fn [channel game]

   (dorun (map #(.addLight (.getRootNode game) %) (lights)))

   ;; set the color of the sky

   (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))

   ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)

   (.setAnim channel "stand")

   (doto (.getFlyByCamera game)

     (.setMoveSpeed (float 10))

     (.setRotationSpeed 1)))

 

 (defn walk [channel]

   (println-repl "zzz")

   (doto channel

     (.setAnim "Walk" (float 0.5))

     (.setLoopMode LoopMode/Loop)))

     

 

 (defn key-map [channel]

   {"key-space" (fn [game value]

 		 (if (not value)

 		   (walk channel)))})

 

 (defn player []

   (let [model (.loadModel (asset-manager) "Models/Oto/Oto.mesh.xml")

 	control (.getControl model AnimControl)]

     (.setLocalScale model (float 0.5))

     (.clearListeners control)

     (.addListener control (anim-control))

     model))

     

 

 

 (defn run-anim-game []

   (let [ninja (player)

 	control  (.getControl ninja AnimControl)

 	channel (.createChannel control)]

     (.start

      (world

       ninja

       (key-map channel)

       (partial setup-fn channel)

       no-op))))

 #+end_src

 

 *** Hello Materials

 #+srcname: material

 #+begin_src clojure :results silent

 (ns hello.material)

 (use 'cortex.world)

 (use 'cortex.import)

 (use 'clojure.contrib.def)

 (cortex.import/mega-import-jme3)

 (rlm.rlm-commands/help)

 

 (defn simple-cube []

   (box 1 1 1

        :position (Vector3f. -3 1.1 0)

        :material "Common/MatDefs/Misc/Unshaded.j3md"

        :texture "Interface/Logo/Monkey.jpg"

        :physical? false))

 

 (defn leaky-box []

   (box 1 1 1

        :position (Vector3f. 3 -1 0)

        :material "Common/MatDefs/Misc/ColoredTextured.j3md"

        :texture  "Textures/ColoredTex/Monkey.png"

        :color    (ColorRGBA. 1 0 1 1)

        :physical? false))

 

 (defn transparent-box []

   (doto

       (box 1 1 0.1

 	   :position Vector3f/ZERO

 	   :name "window frame"

 	   :material "Common/MatDefs/Misc/Unshaded.j3md"

 	   :texture "Textures/ColoredTex/Monkey.png"

 	   :physical? false)

     (-> (.getMaterial)

 	(.getAdditionalRenderState)

 	(.setBlendMode RenderState$BlendMode/Alpha))

     (.setQueueBucket RenderQueue$Bucket/Transparent)))

                   

 (defn bumpy-sphere []

   (doto 

       (sphere 2

 	      :position (Vector3f. 0 2 -2)

 	      :name "Shiny rock"

 	      :material "Common/MatDefs/Light/Lighting.j3md"

 	      :texture false

 	      :physical? false)

     (-> (.getMesh)

 	(doto 

 	  (.setTextureMode Sphere$TextureMode/Projected)

 	  (TangentBinormalGenerator/generate)))

     (-> (.getMaterial)

 	(doto

 	  (.setTexture "DiffuseMap" (.loadTexture (asset-manager)

 						  "Textures/Terrain/Pond/Pond.png"))

 	  (.setTexture "NormalMap"  (.loadTexture (asset-manager)

 						  "Textures/Terrain/Pond/Pond_normal.png"))

 	  (.setFloat   "Shininess"  (float 5))))

     (.rotate (float 1.6) 0 0)))

 

 

 (defn start-game []

   (.start

    (world

     (let [root  (Node.)]

       (dorun (map #(.attachChild root %)

 		  [(simple-cube) (leaky-box) (transparent-box) (bumpy-sphere)]))

       root)

     {}

     (fn [world]

       (let [sun (doto (DirectionalLight.)

 		  (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))

 		  (.setColor ColorRGBA/White))]

 	(.addLight (.getRootNode world) sun)))

     no-op 

     )))

 #+end_src

 

 

     

 * The Body

 ** Eyes

 

 Ultimately I want to make creatures with eyes. Each eye can be

 independely moved and should see its own version of the world

 depending on where it is.

 #+srcname: eyes

 #+begin_src clojure 

 (ns body.eye)

 (use 'cortex.world)

 (use 'cortex.import)

 (use 'clojure.contrib.def)

 (cortex.import/mega-import-jme3)

 (rlm.rlm-commands/help)

 (import java.nio.ByteBuffer)

 (import java.awt.image.BufferedImage)

 (import java.awt.Color)

 (import java.awt.Dimension)

 (import java.awt.Graphics)

 (import java.awt.Graphics2D)

 (import java.awt.event.WindowAdapter)

 (import java.awt.event.WindowEvent)

 (import java.awt.image.BufferedImage)

 (import java.nio.ByteBuffer)

 (import javax.swing.JFrame)

 (import javax.swing.JPanel)

 (import javax.swing.SwingUtilities)

 (import javax.swing.ImageIcon)

 (import javax.swing.JOptionPane)

 (import java.awt.image.ImageObserver)

 

 

 

 (defn scene-processor

   "deals with converting FrameBuffers to BufferedImages so

    that the continuation function can be defined only in terms

    of what it does with BufferedImages"

   [continuation]

   (let [byte-buffer (atom nil)

 	renderer (atom nil)

 	image (atom nil)]

   (proxy [SceneProcessor] []

     (initialize

      [renderManager viewPort]

      (let [cam (.getCamera viewPort)

 	   width (.getWidth cam)

 	   height (.getHeight cam)]

        (reset! renderer (.getRenderer renderManager))

        (reset! byte-buffer

 	     (BufferUtils/createByteBuffer

 	      (* width height 4)))

        (reset! image (BufferedImage. width height

 				     BufferedImage/TYPE_4BYTE_ABGR))))

     (isInitialized [] (not (nil? @byte-buffer)))

     (reshape [_ _ _])

     (preFrame [_])

     (postQueue [_])

     (postFrame

      [#^FrameBuffer fb]

      (.clear @byte-buffer)

      (.readFrameBuffer @renderer fb @byte-buffer)

      (Screenshots/convertScreenShot @byte-buffer @image)

      (continuation @image))

     (cleanup []))))

     

 (defn add-eye

   "Add an eye to the world, and call continuation on

    every frame produced"

   [world camera continuation]

   (let [width (.getWidth camera)

 	height (.getHeight camera)

 	render-manager (.getRenderManager world)

 	viewport (.createMainView render-manager "eye-view" camera)]

     (doto viewport

       (.setBackgroundColor ColorRGBA/Black)

       (.setClearFlags true true true)

       (.addProcessor (scene-processor continuation))

       (.attachScene (.getRootNode world)))))

 

 (defn make-display-frame [display width height]

   (SwingUtilities/invokeLater

    (fn []

      (.setPreferredSize display (Dimension. width height))

      (doto (JFrame. "Eye Camera!")

        (-> (.getContentPane) (.add display))

        (.setDefaultCloseOperation JFrame/DISPOSE_ON_CLOSE)

        (.pack)

        (.setLocationRelativeTo nil)

        (.setResizable false)

        (.setVisible true)))))

 	   

 (defn image-monitor [#^BufferedImage image]

   (proxy [JPanel] []

     (paintComponent 

      [g]

      (proxy-super paintComponent g)

      (locking image

        (.drawImage g image 0 0

 		   (proxy [ImageObserver]

 		       []

 		     (imageUpdate

 		      []

 		      (proxy-super imageUpdate))))))))

 

 (defn movie-image []

   (let [setup

 	(runonce

 	 (fn [#^BufferedImage image]

 	   (let [width (.getWidth image)

 		 height (.getHeight image)

 		 display (image-monitor image)

 		 frame (make-display-frame display width height)]

 	     display)))]

     (fn [#^BufferedImage image]

       (.repaint (setup image)))))

 	     

 

 (defn observer

   "place thy eye!"

   [world camera]

   (let [eye camera

 	width (.getWidth eye)

 	height (.getHeight eye)]

     (no-exceptions

      (add-eye

       world

       eye

       (movie-image)))))

 #+end_src

 

 #+srcname: test-vision

 #+begin_src clojure

 

 (ns test.vision)

 (use 'cortex.world)

 (use 'cortex.import)

 (use 'clojure.contrib.def)

 (use 'body.eye)

 (cortex.import/mega-import-jme3)

 (rlm.rlm-commands/help)

 (import java.nio.ByteBuffer)

 (import java.awt.image.BufferedImage)

 (import java.awt.Color)

 (import java.awt.Dimension)

 (import java.awt.Graphics)

 (import java.awt.Graphics2D)

 (import java.awt.event.WindowAdapter)

 (import java.awt.event.WindowEvent)

 (import java.awt.image.BufferedImage)

 (import java.nio.ByteBuffer)

 (import javax.swing.JFrame)

 (import javax.swing.JPanel)

 (import javax.swing.SwingUtilities)

 (import javax.swing.ImageIcon)

 (import javax.swing.JOptionPane)

 (import java.awt.image.ImageObserver)

 

 

 (def width 200)

 (def height 200)

 

 (defn camera []

   (doto (Camera. width height)

     (.setFrustumPerspective 45 1 1 1000)

     (.setLocation (Vector3f. -3 0 -5))

     (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))

 

 (defn camera2 []

   (doto (Camera. width height)

     (.setFrustumPerspective 45 1 1 1000)

     (.setLocation (Vector3f. 3 0 -5))

     (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))

 

 (defn setup-fn [world]

   (let [eye (camera)

 	width (.getWidth eye)

 	height (.getHeight eye)]

     (no-exceptions

      (add-eye

       world

       eye

       (runonce visual))

      (add-eye

       world

       (camera2)

       (runonce visual)))))

 

 (defn spider-eye [position]

   (doto (Camera. 200 200 )

     (.setFrustumPerspective 45 1 1 1000)

     (.setLocation position)

     (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))

   

 (defn setup-fn* [world]

   (let [eye (camera)

 	width (.getWidth eye)

 	height (.getHeight eye)]

     ;;(.setClearFlags (.getViewPort world) true true true)

     (observer world (.getCamera world))

     (observer world (spider-eye (Vector3f.  3  0 -5)))

     ;;(observer world (spider-eye (Vector3f.  0  0 -5)))

     ;; (observer world (spider-eye (Vector3f. -3  0 -5)))

     ;; (observer world (spider-eye (Vector3f.  0  3 -5)))

     ;; (observer world (spider-eye (Vector3f.  0 -3 -5)))

     ;; (observer world (spider-eye (Vector3f.  3  3 -5)))

     ;; (observer world (spider-eye (Vector3f. -3  3 -5)))

     ;; (observer world (spider-eye (Vector3f.  3 -3 -5)))

     ;; (observer world (spider-eye (Vector3f. -3 -3 -5)))

 

     )

   world)

 

 (defn test-world []

   (let [thing (box 1 1 1 :physical? false)]

     (world

      (doto (Node.)

        (.attachChild thing))

      {}

      setup-fn

      (fn [world tpf]

        (.rotate thing (* tpf 0.2) 0 0)

        ))))

 

 

 #+end_src

 

 

 #+results: eyes

 : #'body.eye/test-world

 

 Note the use of continuation passing style for connecting the eye to a

 function to process the output. The example code will create two

 videos of the same rotating cube from different angles, sutiable for

 stereoscopic vision.

 

 

 

 

 

 

 * COMMENT code generation

 #+begin_src clojure :tangle ../src/cortex/import.clj

 <<import>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/hello/brick_wall.clj

 <<brick-wall-header>>

 <<brick-wall-body>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/hello/hello_simple_app.clj

 <<hello-simple-app>>

 #+end_src

   

 #+begin_src clojure :tangle ../src/cortex/world.clj

 <<world-inputs>>

 <<world>>

 <<world-shapes>>

 <<world-view>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/cortex/other_games.clj

 <<other-games>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/hello/loop.clj

 <<hello-loop>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/hello/collision.clj

 <<hello-collision>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/hello/terrain.clj

 <<hello-terrain>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/hello/animation.clj

 <<hello-animation>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/hello/material.clj

 <<material>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/body/eye.clj

 <<eyes>>

 #+end_src

 

 #+begin_src clojure :tangle ../src/test/vision.clj

 <<test-vision>>

 #+end_src