view org/cortex.org @ 2:46b2d4c9522d

moved common options from hgignore to a central file
author Robert McIntyre <rlm@mit.edu>
date Sun, 16 Oct 2011 06:49:58 -0700
parents 92f8d83b5d0b
children 50c92af2018e
line wrap: on
line source
1 #+title: Simulated Senses
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+MATHJAX: align:"left" mathml:t path:"../aurellem/src/MathJax/MathJax.js"
5 #+STYLE: <link rel="stylesheet" type="text/css" href="../aurellem/src/css/aurellem.css"/>
6 #+BABEL: :exports both :noweb yes :cache no :mkdirp yes
7 #+INCLUDE: ../aurellem/src/templates/level-0.org
8 #+description: Simulating senses for AI research using JMonkeyEngine3
10 * Background
11 Artificial Intelligence has tried and failed for more than half a
12 century to produce programs as flexible, creative, and “intelligent”
13 as the human mind itself. Clearly, we are still missing some important
14 ideas concerning intelligent programs or we would have strong AI
15 already. What idea could be missing?
17 When Turing first proposed his famous “Turing Test” in the
18 groundbreaking paper [[./sources/turing.pdf][/Computing Machines and Intelligence/]], he gave
19 little importance to how a computer program might interact with the
20 world:
22 #+BEGIN_QUOTE
23 \ldquo{}We need not be too concerned about the legs, eyes, etc. The example of
24 Miss Helen Keller shows that education can take place provided that
25 communication in both directions between teacher and pupil can take
26 place by some means or other.\rdquo{}
27 #+END_QUOTE
29 And from the example of Hellen Keller he went on to assume that the
30 only thing a fledgling AI program could need by way of communication
31 is a teletypewriter. But Hellen Keller did possess vision and hearing
32 for the first few months of her life, and her tactile sense was far
33 more rich than any text-stream could hope to achieve. She possessed a
34 body she could move freely, and had continual access to the real world
35 to learn from her actions.
37 I believe that our programs are suffering from too little sensory
38 input to become really intelligent. Imagine for a moment that you
39 lived in a world completely cut off form all sensory stimulation. You
40 have no eyes to see, no ears to hear, no mouth to speak. No body, no
41 taste, no feeling whatsoever. The only sense you get at all is a
42 single point of light, flickering on and off in the void. If this was
43 your life from birth, you would never learn anything, and could never
44 become intelligent. Actual humans placed in sensory deprivation
45 chambers experience hallucinations and can begin to loose their sense
46 of reality in as little as 15 minutes[sensory-deprivation]. Most of
47 the time, the programs we write are in exactly this situation. They do
48 not interface with cameras and microphones, and they do not control a
49 real or simulated body or interact with any sort of world.
52 * Simulation vs. Reality
53 I want demonstrate that multiple senses are what enable
54 intelligence. There are two ways of playing around with senses and
55 computer programs:
57 The first is to go entirely with simulation: virtual world, virtual
58 character, virtual senses. The advantages are that when everything is
59 a simulation, experiments in that simulation are absolutely
60 reproducible. It's also easier to change the character and world to
61 explore new situations and different sensory combinations.
64 ** Issues with Simulation
66 If the world is to be simulated on a computer, then not only do you
67 have to worry about whether the character's senses are rich enough to
68 learn from the world, but whether the world itself is rendered with
69 enough detail and realism to give enough working material to the
70 character's senses. To name just a few difficulties facing modern
71 physics simulators: destructibility of the environment, simulation of
72 water/other fluids, large areas, nonrigid bodies, lots of objects,
73 smoke. I don't know of any computer simulation that would allow a
74 character to take a rock and grind it into fine dust, then use that
75 dust to make a clay sculpture, at least not without spending years
76 calculating the interactions of every single small grain of
77 dust. Maybe a simulated world with today's limitations doesn't provide
78 enough richness for real intelligence to evolve.
80 ** Issues with Reality
82 The other approach for playing with senses is to hook your software up
83 to real cameras, microphones, robots, etc., and let it loose in the
84 real world. This has the advantage of eliminating concerns about
85 simulating the world at the expense of increasing the complexity of
86 implementing the senses. Instead of just grabbing the current rendered
87 frame for processing, you have to use an actual camera with real
88 lenses and interact with photons to get an image. It is much harder to
89 change the character, which is now partly a physical robot of some
90 sort, since doing so involves changing things around in the real world
91 instead of modifying lines of code. While the real world is very rich
92 and definitely provides enough stimulation for intelligence to develop
93 as evidenced by our own existence, it is also uncontrollable in the
94 sense that a particular situation cannot be recreated perfectly or
95 saved for later use. It is harder to conduct science because it is
96 harder to repeat an experiment. The worst thing about using the real
97 world instead of a simulation is the matter of time. Instead of
98 simulated time you get the constant and unstoppable flow of real
99 time. This severely limits the sorts of software you can use to
100 program the AI because all sense inputs must be handled in real
101 time. Complicated ideas may have to be implemented in hardware or may
102 simply be impossible given the current speed of our
103 processors. Contrast this with a simulation, in which the flow of time
104 in the simulated world can be slowed down to accommodate the
105 limitations of the character's programming. In terms of cost, doing
106 everything in software is far cheaper than building custom real-time
107 hardware. All you need is a laptop and some patience.
109 * Choose a Simulation Engine
111 Mainly because of issues with controlling the flow of time, I chose to
112 simulate both the world and the character. I set out to make a minimal
113 world in which I could embed a character with multiple senses. My main
114 goal is to make an environment where I can perform further experiments
115 in simulated senses.
117 As Carl Sagan once said, "If you wish to make an apple pie from
118 scratch, you must first invent the universe.” I examined many
119 different 3D environments to try and find something I would use as the
120 base for my simulation; eventually the choice came down to three
121 engines: the Quake II engine, the Source Engine, and jMonkeyEngine.
123 ** Quake II/Jake2
125 I spent a bit more than a month working with the Quake II Engine from
126 ID software to see if I could use it for my purposes. All the source
127 code was released by ID software into the Public Domain several years
128 ago, and as a result it has been ported and modified for many
129 different reasons. This engine was famous for its advanced use of
130 realistic shading and had decent and fast physics
131 simulation. Researchers at Princeton [[http://www.nature.com/nature/journal/v461/n7266/pdf/nature08499.pdf][used this code]] to study spatial
132 information encoding in the hippocampal cells of rats. Those
133 researchers created a special Quake II level that simulated a maze,
134 and added an interface where a mouse could run around inside a ball in
135 various directions to move the character in the simulated maze. They
136 measured hippocampal activity during this exercise to try and tease
137 out the method in which spatial data was stored in that area of the
138 brain. I find this promising because if a real living rat can interact
139 with a computer simulation of a maze in the same way as it interacts
140 with a real-world maze, then maybe that simulation is close enough to
141 reality that a simulated sense of vision and motor control interacting
142 with that simulation could reveal useful information about the real
143 thing. It happens that there is a Java port of the original C source
144 code called Jake2. The port demonstrates Java's OpenGL bindings and
145 runs anywhere from 90% to 105% as fast as the C version. After
146 reviewing much of the source of Jake2, I eventually rejected it
147 because the engine is too tied to the concept of a first-person
148 shooter game. One of the problems I had was that there does not seem
149 to be any easy way to attach multiple cameras to a single
150 character. There are also several physics clipping issues that are
151 corrected in a way that only applies to the main character and does
152 not apply to arbitrary objects. While there is a large community of
153 level modders, I couldn't find a community to support using the engine
154 to make new things.
156 ** Source Engine
158 The Source Engine evolved from the Quake II and Quake I engines and is
159 used by Valve in the Half-Life series of games. The physics simulation
160 in the Source Engine is quite accurate and probably the best out of
161 all the engines I investigated. There is also an extensive community
162 actively working with the engine. However, applications that use the
163 Source Engine must be written in C++, the code is not open, it only
164 runs on Windows, and the tools that come with the SDK to handle models
165 and textures are complicated and awkward to use.
167 ** jMonkeyEngine
169 jMonkeyEngine is a new library for creating games in Java. It uses
170 OpenGL to render to the screen and uses screengraphs to avoid drawing
171 things that do not appear on the screen. It has an active community
172 and several games in the pipeline. The engine was not built to serve
173 any particular game but is instead meant to be used for any 3D
174 game. After experimenting with each of these three engines and a few
175 others for about 2 months I settled on jMonkeyEngine. I chose it
176 because it had the most features out of all the open projects I looked
177 at, and because I could then write my code in Clojure, an
178 implementation of LISP that runs on the JVM.
180 * Setup
182 First, I checked out the source to jMonkeyEngine:
184 #+srcname: checkout
185 #+begin_src sh :results verbatim
186 svn checkout http://jmonkeyengine.googlecode.com/svn/trunk/engine jme3
187 #+end_src
189 #+results: checkout
190 : Checked out revision 7975.
193 Building jMonkeyEngine is easy enough:
195 #+srcname: build
196 #+begin_src sh :results verbatim
197 cd jme3
198 ant jar | tail -n 2
199 #+end_src
201 #+results: build
202 : BUILD SUCCESSFUL
203 : Total time: 15 seconds
206 Also build the javadoc:
208 #+srcname: javadoc
209 #+begin_src sh :results verbatim
210 cd jme3
211 ant javadoc | tail -n 2
212 #+end_src
214 #+results: javadoc
215 : BUILD SUCCESSFUL
216 : Total time: 12 seconds
218 Now, move the jars from the compilation into the project's lib folder.
220 #+srcname: move-jars
221 #+begin_src sh :results verbatim
222 mkdir -p lib
223 mkdir -p src
224 cp jme3/dist/jMonkeyEngine3.jar lib/
225 cp jme3/dist/lib/* lib/
226 ls lib
227 #+end_src
229 #+results: move-jars
230 #+begin_example
231 eventbus-1.4.jar
232 jbullet.jar
233 jheora-jst-debug-0.6.0.jar
234 jinput.jar
235 jME3-jbullet.jar
236 jME3-lwjgl-natives.jar
237 jME3-testdata.jar
238 jME3-test.jar
239 jMonkeyEngine3.jar
240 j-ogg-oggd.jar
241 j-ogg-vorbisd.jar
242 lwjgl.jar
243 nifty-1.3.jar
244 nifty-default-controls-1.3.jar
245 nifty-examples-1.3.jar
246 nifty-lwjgl-renderer-1.3.jar
247 nifty-openal-soundsystem-1.0.jar
248 nifty-style-black-1.3.jar
249 nifty-style-grey-1.0.jar
250 noise-0.0.1-SNAPSHOT.jar
251 stack-alloc.jar
252 vecmath.jar
253 xmlpull-xpp3-1.1.4c.jar
254 #+end_example
256 It's good to create a =assets= directory in the style that the
257 =AssetManager= will like.
259 #+srcname: create-assets
260 #+begin_src sh :results verbatim
261 mkdir -p assets
262 mkdir -p assets/Interface
263 mkdir -p assets/Materials
264 mkdir -p assets/MatDefs
265 mkdir -p assets/Models
266 mkdir -p assets/Scenes
267 mkdir -p assets/Shaders
268 mkdir -p assets/Sounds
269 mkdir -p assets/Textures
270 tree -L 1 assets
271 #+end_src
273 #+results: create-assets
274 #+begin_example
275 assets
276 |-- Interface
277 |-- MatDefs
278 |-- Materials
279 |-- Models
280 |-- Scenes
281 |-- Shaders
282 |-- Sounds
283 `-- Textures
285 8 directories, 0 files
286 #+end_example
289 The java classpath should have all the jars contained in the =lib=
290 directory as well as the src directory.
292 For example, here is the file I use to run my REPL for clojure.
294 #+include: "~/swank-all" src sh :exports code
296 The important thing here is that =cortex/lib/*=, =cortex/src=, and
297 =cortex/assets= appear on the classpath. (=cortex= is the base
298 directory of this project.)
300 #+srcname: pwd
301 #+begin_src sh
302 pwd
303 #+end_src
305 #+results: pwd
306 : /home/r/cortex
309 * Simulation Base
311 ** Imports
312 First, I'll import jme core classes.
313 #+srcname: import
314 #+begin_src clojure :results silent
315 (ns cortex.import
316 (:require swank.util.class-browse))
318 (defn import-jme3 []
319 (import '[com.jme3.system AppSettings JmeSystem])
320 (import '[com.jme3.app Application SimpleApplication])
321 (import 'com.jme3.material.Material)
322 (import '[com.jme3.math Vector3f ColorRGBA Quaternion Transform])
323 (import '[com.jme3.scene Node Geometry])
324 (import '[com.jme3.scene.shape Box Sphere Sphere$TextureMode])
325 (import 'com.jme3.font.BitmapText)
326 (import '[com.jme3.input KeyInput InputManager])
327 (import '[com.jme3.input.controls
328 ActionListener AnalogListener KeyTrigger MouseButtonTrigger])
329 (import '[com.jme3.asset AssetManager DesktopAssetManager] )
330 (import '[com.jme3.asset.plugins HttpZipLocator ZipLocator])
331 (import '[com.jme3.light PointLight DirectionalLight])
332 (import '[com.jme3.animation AnimControl Skeleton Bone])
333 (import '[com.jme3.bullet.collision.shapes
334 MeshCollisionShape SphereCollisionShape BoxCollisionShape])
335 (import 'com.jme3.renderer.queue.RenderQueue$ShadowMode)
336 (import 'jme3test.TestChooser)
337 (import '[com.jme3.bullet PhysicsTickListener PhysicsSpace])
338 (import '[com.jme3.bullet.joints SixDofJoint HingeJoint
339 SliderJoint Point2PointJoint ConeJoint]))
342 (defmacro permissive-import* [class-symbol]
343 `(try
344 (import ~class-symbol)
345 (catch Exception e#
346 (println "can't import " ~class-symbol))))
348 (defn permissive-import [class-symbol]
349 (eval (list 'cortex.import/permissive-import* class-symbol)))
351 (defn selection-import [selection-fn]
352 (dorun
353 (map (comp permissive-import symbol)
354 (filter selection-fn
355 (map :name
356 swank.util.class-browse/available-classes)))))
358 (defn mega-import-jme3
359 "ALL the jme classes. For REPL use."
360 []
361 (selection-import
362 #(and
363 (.startsWith % "com.jme3.")
364 ;; Don't import the Lwjgl stuff since it can throw exceptions
365 ;; upon being loaded.
366 (not (re-matches #".*Lwjgl.*" %)))))
367 #+end_src
369 The =mega-import-jme3= is quite usefull for debugging purposes since
370 it allows completion for almost all of JME's classes
372 ** Simplification
373 *** World
375 It is comvienent to wrap the JME elements that deal with creating a
376 world, creation of basic objects, and Keyboard input with a nicer
377 interface (at least for my purposes).
379 #+srcname: world-inputs
380 #+begin_src clojure :results silent
381 (ns cortex.world)
382 (require 'cortex.import)
383 (use 'clojure.contrib.def)
384 (rlm.rlm-commands/help)
385 (cortex.import/mega-import-jme3)
387 (defvar *app-settings*
388 (doto (AppSettings. true)
389 (.setFullscreen false)
390 (.setTitle "Aurellem.")
391 ;; disable 32 bit stuff for now
392 ;;(.setAudioRenderer "Send")
393 )
394 "These settings control how the game is displayed on the screen for
395 debugging purposes. Use binding forms to change this if desired.
396 Full-screen mode does not work on some computers.")
398 (defn asset-manager
399 "returns a new, configured assetManager" []
400 (JmeSystem/newAssetManager
401 (.getResource
402 (.getContextClassLoader (Thread/currentThread))
403 "com/jme3/asset/Desktop.cfg")))
405 (defmacro no-exceptions
406 "Sweet relief like I never knew."
407 [& forms]
408 `(try ~@forms (catch Exception e# (.printStackTrace e#))))
410 (defn thread-exception-removal []
411 (println "removing exceptions from " (Thread/currentThread))
412 (.setUncaughtExceptionHandler
413 (Thread/currentThread)
414 (proxy [Thread$UncaughtExceptionHandler] []
415 (uncaughtException
416 [thread thrown]
417 (println "uncaught-exception thrown in " thread)
418 (println (.getMessage thrown))))))
420 (def println-repl (bound-fn [& args] (apply println args)))
422 (use '[pokemon [lpsolve :only [constant-map]]])
424 (defn no-op [& _])
426 (defn all-keys
427 "Construct a map of strings representing all the manual inputs from
428 either the keyboard or mouse."
429 []
430 (let [inputs (constant-map KeyInput)]
431 (assoc
432 (zipmap (map (fn [field]
433 (.toLowerCase (re-gsub #"_" "-" field))) (vals inputs))
434 (map (fn [val] (KeyTrigger. val)) (keys inputs)))
435 ;;explicitly add mouse controls
436 "mouse-left" (MouseButtonTrigger. 0)
437 "mouse-middle" (MouseButtonTrigger. 2)
438 "mouse-right" (MouseButtonTrigger. 1))))
440 (defn initialize-inputs
441 "more java-interop cruft to establish keybindings for a particular virtual world"
442 [game input-manager key-map]
443 (doall (map (fn [[name trigger]]
444 (.addMapping ^InputManager input-manager
445 name (into-array (class trigger) [trigger]))) key-map))
446 (doall (map (fn [name]
447 (.addListener ^InputManager input-manager game
448 (into-array String [name]))) (keys key-map))))
450 #+end_src
452 These functions are all for debug controlling of the world through
453 keyboard and mouse.
455 We reuse =constant-map= from =pokemon.lpsolve= to get the numerical
456 values for all the keys defined in the =KeyInput= class. The
457 documentation for =constant-map= is:
459 #+begin_src clojure :results output
460 (doc pokemon.lpsolve/constant-map)
461 #+end_src
463 #+results:
464 : -------------------------
465 : pokemon.lpsolve/constant-map
466 : ([class])
467 : Takes a class and creates a map of the static constant integer
468 : fields with their names. This helps with C wrappers where they have
469 : just defined a bunch of integer constants instead of enums
472 Then, =all-keys= converts the constant names like =KEY_J= to the more
473 clojure-like =key-j=, and returns a map from these keys to
474 jMonkeyEngine KeyTrigger objects, the use of which will soon become
475 apparent. =all-keys= also adds the three mouse button controls to the
476 map.
478 #+srcname: world
479 #+begin_src clojure :results silent
480 (in-ns 'cortex.world)
482 (defn traverse
483 "apply f to every non-node, deeply"
484 [f node]
485 (if (isa? (class node) Node)
486 (dorun (map (partial traverse f) (.getChildren node)))
487 (f node)))
489 (def gravity (Vector3f. 0 -9.81 0))
491 (defn world
492 [root-node key-map setup-fn update-fn]
493 (let [physics-manager (BulletAppState.)
494 shadow-renderer (BasicShadowRenderer. (asset-manager) (int 256))
495 ;;maybe use a better shadow renderer someday!
496 ;;shadow-renderer (PssmShadowRenderer. (asset-manager) 256 1)
497 ]
498 (doto
499 (proxy [SimpleApplication ActionListener] []
500 (simpleInitApp
501 []
502 (no-exceptions
503 (.setTimer this (IsoTimer. 60))
504 ;; Create key-map.
505 (.setFrustumFar (.getCamera this) 300)
506 (initialize-inputs this (.getInputManager this) (all-keys))
507 ;; Don't take control of the mouse
508 (org.lwjgl.input.Mouse/setGrabbed false)
509 ;; add all objects to the world
510 (.attachChild (.getRootNode this) root-node)
511 ;; enable physics
512 ;; add a physics manager
513 (.attach (.getStateManager this) physics-manager)
514 (.setGravity (.getPhysicsSpace physics-manager) gravity)
517 ;; go through every object and add it to the physics manager
518 ;; if relavant.
519 (traverse (fn [geom]
520 (dorun
521 (for [n (range (.getNumControls geom))]
522 (do
523 (println-repl "adding control " (.getControl geom n))
524 (.add (.getPhysicsSpace physics-manager)
525 (.getControl geom n))))))
526 (.getRootNode this))
527 ;;(.addAll (.getPhysicsSpace physics-manager) (.getRootNode this))
529 (setup-fn this)
530 (.setDirection shadow-renderer
531 (.normalizeLocal (Vector3f. -1 -1 -1)))
532 (.addProcessor (.getViewPort this) shadow-renderer)
533 (.setShadowMode (.getRootNode this) RenderQueue$ShadowMode/Off)
534 ))
535 (simpleUpdate
536 [tpf]
537 (no-exceptions
538 (update-fn this tpf)))
539 (onAction
540 [binding value tpf]
541 ;; whenever a key is pressed, call the function returned from
542 ;; key-map.
543 (no-exceptions
544 (if-let [react (key-map binding)]
545 (react this value)))))
546 ;; don't show a menu to change options.
548 (.setShowSettings false)
549 (.setPauseOnLostFocus false)
550 (.setSettings *app-settings*))))
552 (defn apply-map
553 "Like apply, but works for maps and functions that expect an implicit map
554 and nothing else as in (fn [& {}]).
555 ------- Example -------
556 (defn jjj [& {:keys [www] :or {www \"oph yeah\"} :as env}] (println www))
557 (apply-map jjj {:www \"whatever\"})
558 -->\"whatever\""
559 [fn m]
560 (apply fn (reduce #(into %1 %2) [] m)))
562 #+end_src
565 =world= is the most important function here.
566 *** TODO more documentation
568 #+srcname: world-shapes
569 #+begin_src clojure :results silent
570 (in-ns 'cortex.world)
571 (defrecord shape-description
572 [name
573 color
574 mass
575 friction
576 texture
577 material
578 position
579 rotation
580 shape
581 physical?])
583 (def base-shape
584 (shape-description.
585 "default-shape"
586 false
587 ;;ColorRGBA/Blue
588 1.0 ;; mass
589 1.0 ;; friction
590 ;; texture
591 "Textures/Terrain/BrickWall/BrickWall.jpg"
592 ;; material
593 "Common/MatDefs/Misc/Unshaded.j3md"
594 Vector3f/ZERO
595 Quaternion/IDENTITY
596 (Box. Vector3f/ZERO 0.5 0.5 0.5)
597 true))
599 (defn make-shape
600 [#^shape-description d]
601 (let [mat (Material. (asset-manager) (:material d))
602 geom (Geometry. (:name d) (:shape d))]
603 (if (:texture d)
604 (let [key (TextureKey. (:texture d))]
605 (.setGenerateMips key true)
606 (.setTexture mat "ColorMap" (.loadTexture (asset-manager) key))))
607 (if (:color d) (.setColor mat "Color" (:color d)))
608 (.setMaterial geom mat)
609 (if-let [rotation (:rotation d)] (.rotate geom rotation))
610 (.setLocalTranslation geom (:position d))
611 (if (:physical? d)
612 (let [impact-shape (doto (GImpactCollisionShape. (.getMesh geom)) (.setMargin 0))
613 physics-control (RigidBodyControl.
614 impact-shape
615 (float (:mass d)))]
616 (.createJmeMesh impact-shape)
617 (.addControl geom physics-control)
618 ;;(.setSleepingThresholds physics-control (float 0) (float 0))
619 (.setFriction physics-control (:friction d))))
620 ;;the default is to keep this node in the physics engine forever.
621 ;;these commands must come after the control is added to the geometry.
622 ;;
623 geom))
625 (defn box
626 ([l w h & {:as options}]
627 (let [options (merge base-shape options)]
628 (make-shape (assoc options
629 :shape (Box. l w h)))))
630 ([] (box 0.5 0.5 0.5)))
632 (defn sphere
633 ([r & {:as options}]
634 (let [options (merge base-shape options)]
635 (make-shape (assoc options
636 :shape (Sphere. 32 32 (float r))))))
637 ([] (sphere 0.5)))
639 (defn add-element [game node]
640 (.addAll
641 (.getPhysicsSpace
642 (.getState
643 (.getStateManager game)
644 BulletAppState))
645 node)
646 (.attachChild (.getRootNode game) node))
648 (defn set-gravity*
649 [game gravity]
650 (traverse
651 (fn [geom]
652 (if-let
653 [control (.getControl geom RigidBodyControl)]
654 (do
655 (.setGravity control gravity)
656 (.applyImpulse control Vector3f/ZERO Vector3f/ZERO)
657 )))
658 (.getRootNode game)))
660 #+end_src
662 These are convienence functions for creating JME objects and
663 manipulating a world.
665 #+srcname: world-view
666 #+begin_src clojure :results silent
667 (in-ns 'cortex.world)
669 (defprotocol Viewable
670 (view [something]))
672 (extend-type com.jme3.scene.Geometry
673 Viewable
674 (view [geo]
675 (view (doto (Node.)(.attachChild geo)))))
677 (extend-type com.jme3.scene.Node
678 Viewable
679 (view [node]
680 (.start
681 (world node
682 {}
683 (fn [world]
684 (.enableDebug
685 (.getPhysicsSpace
686 (.getState
687 (.getStateManager world)
688 BulletAppState))
689 (asset-manager))
690 (set-gravity* world Vector3f/ZERO)
691 ;; (set-gravity* world (Vector3f. 0 (float -0.4) 0))
692 (let [sun (doto (DirectionalLight.)
693 (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))
694 (.setColor ColorRGBA/White))]
695 (.addLight (.getRootNode world) sun)))
696 no-op))))
698 (defn position-camera [game]
699 (doto (.getCamera game)
700 (.setLocation (Vector3f. 0 6 6))
701 (.lookAt Vector3f/ZERO (Vector3f. 0 1 0))))
703 #+end_src
705 Here I make the =Viewable= protocol and extend it to JME's types. Now
706 hello-world can be written as easily as:
708 #+begin_src clojure :results silent
709 (cortex.world/view (cortex.world/box))
710 #+end_src
712 ** Hello
713 Here are the jmonkeyengine "Hello" programs translated to clojure.
714 *** Hello Simple App
715 Here is the hello world example for jme3 in clojure.
716 It's a more or less direct translation from the java source
717 from
718 http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_simpleapplication.
720 Of note is the fact that since we don't have access to the
721 =AssetManager= via extendig =SimpleApplication=, we have to build one
722 ourselves.
724 #+srcname: hello-simple-app
725 #+begin_src clojure :results silent
726 (ns hello.hello-simple-app)
727 (require 'cortex.import)
728 (use 'clojure.contrib.def)
729 (rlm.rlm-commands/help)
730 (cortex.import/import-jme3)
731 (use 'cortex.world)
734 (def cube (Box. Vector3f/ZERO 1 1 1))
736 (def geom (Geometry. "Box" cube))
738 (def mat (Material. (asset-manager) "Common/MatDefs/Misc/Unshaded.j3md"))
740 (.setColor mat "Color" ColorRGBA/Blue)
742 (.setMaterial geom mat)
744 (defn simple-app []
745 (doto
746 (proxy [SimpleApplication] []
747 (simpleInitApp
748 []
749 ;; Don't take control of the mouse
750 (org.lwjgl.input.Mouse/setGrabbed false)
751 (.attachChild (.getRootNode this) geom)))
752 ;; don't show a menu to change options.
753 (.setShowSettings false)
754 (.setPauseOnLostFocus false)
755 (.setSettings *app-settings*)))
756 #+end_src
758 Running this program will begin a new jMonkeyEngine game which
759 displays a single blue cube.
761 #+begin_src clojure :exports code :results silent
762 (.start (hello.hello-simple-app/simple-app))
763 #+end_src
765 #+caption: the simplest JME game.
766 [[./images/simple-app.jpg]]
770 *** Hello Physics
771 From http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_physics
773 #+srcname: brick-wall-header
774 #+begin_src clojure :results silent
775 (ns hello.brick-wall)
776 (require 'cortex.import)
777 (use 'clojure.contrib.def)
778 (rlm.rlm-commands/help)
779 (cortex.import/mega-import-jme3)
780 (use '[pokemon [lpsolve :only [constant-map]]])
781 (use 'cortex.world)
782 #+end_src
784 #+srcname: brick-wall-body
785 #+begin_src clojure :results silent
786 (in-ns 'hello.brick-wall)
788 (defn floor
789 "make a sturdy, unmovable physical floor"
790 []
791 (box 20 1 20 :mass 0 :color false :position (Vector3f. 0 -2 0)))
793 (def brick-length 0.48)
794 (def brick-width 0.24)
795 (def brick-height 0.12)
798 (defn brick* [position]
799 (doto (box brick-length brick-height brick-width
800 :position position :name "brick"
801 :material "Common/MatDefs/Misc/Unshaded.j3md"
802 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
803 :mass 36)
804 (->
805 (.getMesh)
806 (.scaleTextureCoordinates (Vector2f. 1 0.5)))
807 ;;(.setShadowMode RenderQueue$ShadowMode/CastAndReceive)
808 )
809 )
811 (defn inception-brick-wall
812 "construct a physical brick wall"
813 []
814 (let [node (Node. "brick-wall")]
815 (dorun
816 (map (comp #(.attachChild node %) brick*)
817 (for
818 [x (range 15)
819 y (range 10)
820 z (range 1)]
821 (Vector3f.
822 (* brick-length x 1.03)
823 (* brick-width y y 10)
824 (* brick-height z)))))
825 node))
827 (defn gravity-toggle
828 [new-value]
829 (fn [game value]
830 (println-repl "set gravity to " new-value)
831 (if value
832 (set-gravity* game new-value)
833 (set-gravity* game gravity))))
835 (defn fire-cannon-ball []
836 (fn [game value]
837 (if (not value)
838 (let [camera (.getCamera game)
839 cannon-ball
840 (sphere 0.7
841 :material "Common/MatDefs/Misc/Unshaded.j3md"
842 :texture "Textures/PokeCopper.jpg"
843 :position
844 (.add (.getLocation camera)
845 (.mult (.getDirection camera) (float 1)))
846 :mass 3)] ;200 0.05
847 (.setShadowMode cannon-ball RenderQueue$ShadowMode/CastAndReceive)
848 (.setLinearVelocity
849 (.getControl cannon-ball RigidBodyControl)
850 (.mult (.getDirection camera) (float 50))) ;50
851 (add-element game cannon-ball)))))
853 (defn floor* []
854 (doto (box 10 0.1 5 :name "floor" ;10 0.1 5 ; 240 0.1 240
855 :material "Common/MatDefs/Misc/Unshaded.j3md"
856 :texture "Textures/Terrain/Pond/Pond.png"
857 :position (Vector3f. 0 -0.1 0 )
858 :mass 0)
859 (->
860 (.getMesh)
861 (.scaleTextureCoordinates (Vector2f. 3 6)));64 64
862 (->
863 (.getMaterial)
864 (.getTextureParam "ColorMap")
865 (.getTextureValue)
866 (.setWrap Texture$WrapMode/Repeat))
867 (.setShadowMode RenderQueue$ShadowMode/Receive)
868 ))
870 (defn brick-wall* []
871 (let [node (Node. "brick-wall")]
872 (dorun
873 (map
874 (comp #(.attachChild node %) brick*)
875 (for [y (range 15)
876 x (range 4)
877 z (range 1)]
878 (Vector3f.
879 (+ (* 2 x brick-length)
880 (if (even? (+ y z))
881 (/ brick-length 4) (/ brick-length -4)))
882 (+ (* brick-height (inc (* 2 y))))
883 (* 2 z brick-width) ))))
884 (.setShadowMode node RenderQueue$ShadowMode/CastAndReceive)
885 node))
887 (defn brick-wall-game-run []
888 (doto
889 (world
890 (doto (Node.) (.attachChild (floor*))
891 (.attachChild (brick-wall*))
892 )
893 {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))
894 "key-m" (gravity-toggle (Vector3f. 0 0 9.81))
895 "key-l" (gravity-toggle (Vector3f. 9.81 0 0))
896 "key-j" (gravity-toggle (Vector3f. -9.81 0 0))
897 "key-k" (gravity-toggle Vector3f/ZERO)
898 "key-u" (gravity-toggle (Vector3f. 0 9.81 0))
899 "key-o" (gravity-toggle (Vector3f. 0 -9.81 0))
900 "key-f" (fn[game value]
901 (if (not value) (add-element game (brick-wall*))))
902 "key-return" (fire-cannon-ball)}
903 position-camera
904 (fn [& _]))
905 (.start)))
906 #+end_src
908 #+begin_src clojure :results silent
909 (hello.brick-wall/brick-wall-game-run)
910 #+end_src
912 #+caption: the brick wall standing
913 [[./images/brick-wall-standing.jpg]]
915 #+caption: the brick wall after it has been knocked over by a "pok\eacute{}ball"
916 [[./images/brick-wall-knocked-down.jpg]]
918 *** Other Brick Games
919 #+srcname: other-games
920 #+begin_src clojure :results silent
921 (ns cortex.other-games
922 {:author "Dylan Holmes"})
923 (use 'cortex.world)
924 (use 'hello.brick-wall)
925 (use 'cortex.import)
926 (cortex.import/mega-import-jme3)
928 (defn scad [position]
929 (doto (box 0.1 0.1 0.1
930 :position position :name "brick"
931 :material "Common/MatDefs/Misc/Unshaded.j3md"
932 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
933 :mass 20)
934 (->
935 (.getMesh)
936 (.scaleTextureCoordinates (Vector2f. 1 0.5))
937 )
938 (-> (.getControl RigidBodyControl)
939 (.setLinearVelocity (Vector3f. 0 100 0))
940 )
942 ;;(.setShadowMode RenderQueue$ShadowMode/Cast)
943 ))
946 (defn shrapnel []
947 (let [node (Node. "explosion-day")]
948 (dorun
949 (map
950 (comp #(.attachChild node %) scad)
951 (for [y (range 15)
952 x (range 4)
953 z (range 1)]
954 (Vector3f.
955 (+ (* 2 x brick-height)
956 (if (even? (+ y z)) (/ brick-height 4) (/ brick-height -4)))
957 (+ (* brick-height (inc (* 2 y))))
958 (* 2 z brick-height) ))))
959 node))
962 (def domino-height 0.48)
963 (def domino-thickness 0.12)
964 (def domino-width 0.24)
966 (def domino-thickness 0.05)
967 (def domino-width 0.5)
968 (def domino-height 1)
970 (defn domino
971 ([position]
972 (domino position (Quaternion/IDENTITY)))
973 ([position rotation]
974 (doto (box domino-width domino-height domino-thickness
975 :position position :name "domino"
976 :material "Common/MatDefs/Misc/Unshaded.j3md"
977 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
978 :mass 1
979 :rotation rotation)
980 (.setShadowMode RenderQueue$ShadowMode/CastAndReceive)
981 )))
984 (defn domino-row []
985 (let [node (Node. "domino-row")]
986 (dorun
987 (map
988 (comp #(.attachChild node %) domino)
989 (for [
990 z (range 10)
991 x (range 5)
992 ]
993 (Vector3f.
994 (+ (* z domino-width) (* x 5 domino-width))
995 (/ domino-height 1)
996 (* -5.5 domino-thickness z) ))))
998 node))
1000 (defn domino-cycle []
1001 (let [node (Node. "domino-cycle")]
1002 (dorun
1003 (map
1004 (comp #(.attachChild node %) (partial apply domino) )
1005 (for [n (range 720)]
1006 (let [space (* domino-height 5.5)
1007 r (fn[n] (* (+ n 3) domino-width 0.5))
1008 t (fn[n] (reduce
1010 (map
1011 (fn dt[n] (/ space (* 2 (Math/PI) (r n))))
1012 (range n))))
1013 t (t n)
1014 r (r n)
1015 ct (Math/cos t)
1016 st (Math/sin t)
1018 (list
1019 (Vector3f.
1020 (* -1 r st)
1021 (/ domino-height 1)
1022 (* r ct))
1023 (.fromAngleAxis (Quaternion.)
1024 (- (/ 3.1415926 2) t) (Vector3f. 0 1 0))
1025 )))
1026 ))
1027 node))
1030 (defn domino-game-run []
1031 (doto
1032 (world
1033 (doto (Node.) (.attachChild (floor*))
1035 {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))
1036 "key-m" (gravity-toggle (Vector3f. 0 0 9.81))
1037 "key-l" (gravity-toggle (Vector3f. 9.81 0 0))
1038 "key-j" (gravity-toggle (Vector3f. -9.81 0 0))
1039 "key-k" (gravity-toggle (Vector3f. 0 9.81 0) )
1040 "key-u" (fn[g v] ((gravity-toggle (Vector3f. 0 -0 0)) g true))
1041 "key-o" (gravity-toggle (Vector3f. 0 -9.81 0))
1043 "key-space"
1044 (fn[game value]
1046 (if (not value)
1047 (let [d (domino (Vector3f. 0 (/ domino-height 0.25) 0)
1048 (.fromAngleAxis (Quaternion.)
1049 (/ Math/PI 2) (Vector3f. 0 1 0)))]
1050 (add-element game d))))
1051 "key-f"
1052 (fn[game value](if (not value) (add-element game (domino-cycle))))
1053 "key-return" (fire-cannon-ball)}
1054 position-camera
1055 (fn [& _]))
1056 (.start)))
1057 #+end_src
1059 #+begin_src clojure :results silent
1060 (cortex.other-games/domino-game-run)
1061 #+end_src
1063 #+caption: floating dominos
1064 [[./images/dominos.jpg]]
1066 *** Hello Loop
1067 #+srcname: hello-loop
1068 #+begin_src clojure :results silent
1069 (ns hello.loop)
1070 (use 'cortex.world)
1071 (use 'cortex.import)
1072 (cortex.import/mega-import-jme3)
1073 (rlm.rlm-commands/help)
1075 (defn blue-cube []
1076 (box 1 1 1
1077 :color ColorRGBA/Blue
1078 :texture false
1079 :material "Common/MatDefs/Misc/Unshaded.j3md"
1080 :name "blue-cube"
1081 :physical? false))
1083 (defn blue-cube-game []
1084 (let [cube (blue-cube)
1085 root (doto (Node.) (.attachChild cube))]
1086 (world root
1087 {}
1088 no-op
1089 (fn [game tpf]
1090 (.rotate cube 0.0 (* 2 tpf) 0.0)))))
1091 #+end_src
1093 *** Hello Collision
1095 #+srcname: hello-collision
1096 #+begin_src clojure :results silent
1097 (ns hello.collision)
1098 (use 'cortex.world)
1099 (use 'cortex.import)
1100 (use 'clojure.contrib.def)
1103 (cortex.import/mega-import-jme3)
1104 (rlm.rlm-commands/help)
1105 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])
1108 (defn environment []
1109 (let
1110 [scene-model
1111 (doto
1112 (.loadModel
1113 (doto (asset-manager)
1114 (.registerLocator
1115 "/home/r/cortex/assets/zips/town.zip" ZipLocator))
1116 "main.scene")
1117 (.setLocalScale (float 2.0)))
1118 collision-shape
1119 (CollisionShapeFactory/createMeshShape #^Node scene-model)
1120 landscape (RigidBodyControl. collision-shape 0)]
1121 (.setShadowMode scene-model RenderQueue$ShadowMode/CastAndReceive)
1122 (.addControl scene-model landscape)
1123 scene-model))
1125 (defn player-fn []
1126 (doto
1127 (CharacterControl.
1128 (CapsuleCollisionShape. (float 1.5) (float 6)(float 1))
1129 (float 0.05))
1130 (.setJumpSpeed 20)
1131 (.setFallSpeed 30)
1132 (.setGravity 30) ;30
1133 (.setPhysicsLocation (Vector3f. 0 10 0))))
1135 (defn lights []
1136 [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 1 1 1) (float 1))))
1137 (doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 0.7 0 1) (float 1))))
1138 (doto (DirectionalLight.)
1139 (.setColor (.mult ColorRGBA/White (float 0.9) ))
1140 (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])
1142 (defn night-lights []
1143 [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 0.275 0.467 0.784 1) (float 0.3))))
1144 (doto (DirectionalLight.)
1145 (.setColor (.mult ColorRGBA/White (float 0.2) ))
1146 (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])
1148 (def player (atom (player-fn)))
1150 (defn setup-fn [game]
1151 (dorun (map #(.addLight (.getRootNode game) %) (lights)))
1152 ;; set the color of the sky
1153 (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))
1154 ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)
1155 (doto (.getFlyByCamera game)
1156 (.setMoveSpeed (float 100))
1157 (.setRotationSpeed 3))
1158 (.add
1159 (.getPhysicsSpace
1160 (.getState (.getStateManager game) BulletAppState))
1161 @player)
1163 (doto (Node.) (.attachChild (.getRootNode game))
1164 (.attachChild (brick-wall*))
1170 (def walking-up? (atom false))
1171 (def walking-down? (atom false))
1172 (def walking-left? (atom false))
1173 (def walking-right? (atom false))
1175 (defn set-walk [walk-atom game value]
1176 ;;(println-repl "setting stuff to " value)
1177 (reset! walk-atom value))
1179 (defn responses []
1180 {"key-w" (partial set-walk walking-up?)
1181 "key-d" (partial set-walk walking-right?)
1182 "key-s" (partial set-walk walking-down?)
1183 "key-a" (partial set-walk walking-left?)
1184 "key-return" (fire-cannon-ball)
1185 "key-space" (fn [game value] (.jump @player))
1186 })
1188 (defn update-fn
1189 [game tpf]
1190 (let [camera (.getCamera game)
1191 cam-dir (.multLocal
1192 (.clone
1193 (.getDirection camera)) (float 0.6))
1194 cam-left (.multLocal
1195 (.clone
1196 (.getLeft camera)) (float 0.4))
1197 walk-direction (Vector3f. 0 0 0)]
1199 (cond
1200 @walking-up? (.addLocal walk-direction cam-dir)
1201 @walking-right? (.addLocal walk-direction (.negate cam-left))
1202 @walking-down? (.addLocal walk-direction (.negate cam-dir))
1203 @walking-left? (.addLocal walk-direction cam-left))
1204 (.setWalkDirection @player walk-direction)
1205 (.setLocation camera (.getPhysicsLocation @player))))
1207 (defn run-game []
1208 (.start
1209 (world (environment)
1210 (responses)
1211 setup-fn
1212 update-fn)))
1213 #+end_src
1215 *** Hello Terrain
1216 #+srcname: hello-terrain
1217 #+begin_src clojure :results silent
1218 (ns hello.terrain)
1219 (use 'cortex.world)
1220 (use 'cortex.import)
1221 (use 'clojure.contrib.def)
1222 (import jme3tools.converters.ImageToAwt)
1225 (cortex.import/mega-import-jme3)
1226 (rlm.rlm-commands/help)
1227 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])
1230 (defn setup-fn [type game]
1231 (.setMoveSpeed (.getFlyByCamera game) 50)
1232 (.setFrustumFar (.getCamera game) 10000)
1233 (let [env (environment type)
1234 cameras [(.getCamera game)]
1235 control (TerrainLodControl. env cameras)]
1236 ;;(.addControl env control)
1237 (.attachChild (.getRootNode game) env)))
1239 (defn environment [type]
1240 (let
1241 [mat_terrain
1242 (Material. (asset-manager) "Common/MatDefs/Terrain/Terrain.j3md")
1243 grass (.loadTexture (asset-manager) "Textures/Terrain/splat/grass.jpg")
1244 dirt (.loadTexture (asset-manager) "Textures/Terrain/splat/dirt.jpg")
1245 rock (.loadTexture (asset-manager) "Textures/Terrain/splat/road.jpg")
1246 heightmap-image (.loadTexture (asset-manager)
1247 ({:mountain "Textures/Terrain/splat/mountains512.png"
1248 :fortress "Textures/Terrain/splat/fortress512.png"
1249 }type))
1250 heightmap (ImageBasedHeightMap.
1251 (ImageToAwt/convert (.getImage heightmap-image) false true 0))
1252 terrain (do (.load heightmap)
1253 (TerrainQuad. "my terrain" 65 513 (.getHeightMap heightmap)))
1256 (dorun (map #(.setWrap % Texture$WrapMode/Repeat)
1257 [grass dirt rock]))
1259 (doto mat_terrain
1260 (.setTexture "Tex1" grass)
1261 (.setFloat "Tex1Scale" (float 64))
1263 (.setTexture "Tex2" dirt)
1264 (.setFloat "Tex2Scale" (float 32))
1266 (.setTexture "Tex3" rock)
1267 (.setFloat "Tex3Scale" (float 128))
1269 (.setTexture "Alpha"
1270 (.loadTexture
1271 (asset-manager)
1272 ({:mountain "Textures/Terrain/splat/alphamap.png"
1273 :fortress "Textures/Terrain/splat/alphamap2.png"} type))))
1275 (doto terrain
1276 (.setMaterial mat_terrain)
1277 (.setLocalTranslation 0 -100 0)
1278 (.setLocalScale 2 1 2))))
1282 (defn run-terrain-game [type]
1283 (.start
1284 (world
1285 (Node.)
1286 {}
1287 (partial setup-fn type)
1288 no-op)))
1289 #+end_src
1293 #+srcname: hello-animation
1294 #+begin_src clojure :results silent
1295 (ns hello.animation)
1296 (use 'cortex.world)
1297 (use 'cortex.import)
1298 (use 'clojure.contrib.def)
1299 (cortex.import/mega-import-jme3)
1300 (rlm.rlm-commands/help)
1301 (use '[hello [collision :only [lights]]])
1303 (defn stand
1304 [channel]
1305 (doto channel
1306 (.setAnim "stand" (float 0.5))
1307 (.setLoopMode LoopMode/DontLoop)
1308 (.setSpeed (float 1))))
1310 (defn anim-listener []
1311 (proxy [AnimEventListener] []
1312 (onAnimChange
1313 [control channel animation-name]
1314 (println-repl "RLM --- onAnimChange"))
1315 (onAnimCycleDone
1316 [control channel animation-name]
1317 (if (= animation-name "Walk")
1318 (stand channel)
1319 ))))
1321 (defn setup-fn [channel game]
1322 (dorun (map #(.addLight (.getRootNode game) %) (lights)))
1323 ;; set the color of the sky
1324 (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))
1325 ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)
1326 (.setAnim channel "stand")
1327 (doto (.getFlyByCamera game)
1328 (.setMoveSpeed (float 10))
1329 (.setRotationSpeed 1)))
1331 (defn walk [channel]
1332 (println-repl "zzz")
1333 (doto channel
1334 (.setAnim "Walk" (float 0.5))
1335 (.setLoopMode LoopMode/Loop)))
1338 (defn key-map [channel]
1339 {"key-space" (fn [game value]
1340 (if (not value)
1341 (walk channel)))})
1343 (defn player []
1344 (let [model (.loadModel (asset-manager) "Models/Oto/Oto.mesh.xml")
1345 control (.getControl model AnimControl)]
1346 (.setLocalScale model (float 0.5))
1347 (.clearListeners control)
1348 (.addListener control (anim-control))
1349 model))
1353 (defn run-anim-game []
1354 (let [ninja (player)
1355 control (.getControl ninja AnimControl)
1356 channel (.createChannel control)]
1357 (.start
1358 (world
1359 ninja
1360 (key-map channel)
1361 (partial setup-fn channel)
1362 no-op))))
1363 #+end_src
1365 *** Hello Materials
1366 #+srcname: material
1367 #+begin_src clojure :results silent
1368 (ns hello.material)
1369 (use 'cortex.world)
1370 (use 'cortex.import)
1371 (use 'clojure.contrib.def)
1372 (cortex.import/mega-import-jme3)
1373 (rlm.rlm-commands/help)
1375 (defn simple-cube []
1376 (box 1 1 1
1377 :position (Vector3f. -3 1.1 0)
1378 :material "Common/MatDefs/Misc/Unshaded.j3md"
1379 :texture "Interface/Logo/Monkey.jpg"
1380 :physical? false))
1382 (defn leaky-box []
1383 (box 1 1 1
1384 :position (Vector3f. 3 -1 0)
1385 :material "Common/MatDefs/Misc/ColoredTextured.j3md"
1386 :texture "Textures/ColoredTex/Monkey.png"
1387 :color (ColorRGBA. 1 0 1 1)
1388 :physical? false))
1390 (defn transparent-box []
1391 (doto
1392 (box 1 1 0.1
1393 :position Vector3f/ZERO
1394 :name "window frame"
1395 :material "Common/MatDefs/Misc/Unshaded.j3md"
1396 :texture "Textures/ColoredTex/Monkey.png"
1397 :physical? false)
1398 (-> (.getMaterial)
1399 (.getAdditionalRenderState)
1400 (.setBlendMode RenderState$BlendMode/Alpha))
1401 (.setQueueBucket RenderQueue$Bucket/Transparent)))
1403 (defn bumpy-sphere []
1404 (doto
1405 (sphere 2
1406 :position (Vector3f. 0 2 -2)
1407 :name "Shiny rock"
1408 :material "Common/MatDefs/Light/Lighting.j3md"
1409 :texture false
1410 :physical? false)
1411 (-> (.getMesh)
1412 (doto
1413 (.setTextureMode Sphere$TextureMode/Projected)
1414 (TangentBinormalGenerator/generate)))
1415 (-> (.getMaterial)
1416 (doto
1417 (.setTexture "DiffuseMap" (.loadTexture (asset-manager)
1418 "Textures/Terrain/Pond/Pond.png"))
1419 (.setTexture "NormalMap" (.loadTexture (asset-manager)
1420 "Textures/Terrain/Pond/Pond_normal.png"))
1421 (.setFloat "Shininess" (float 5))))
1422 (.rotate (float 1.6) 0 0)))
1425 (defn start-game []
1426 (.start
1427 (world
1428 (let [root (Node.)]
1429 (dorun (map #(.attachChild root %)
1430 [(simple-cube) (leaky-box) (transparent-box) (bumpy-sphere)]))
1431 root)
1432 {}
1433 (fn [world]
1434 (let [sun (doto (DirectionalLight.)
1435 (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))
1436 (.setColor ColorRGBA/White))]
1437 (.addLight (.getRootNode world) sun)))
1438 no-op
1439 )))
1440 #+end_src
1444 * The Body
1445 ** Eyes
1447 Ultimately I want to make creatures with eyes. Each eye can be
1448 independely moved and should see its own version of the world
1449 depending on where it is.
1450 #+srcname: eyes
1451 #+begin_src clojure
1452 (ns body.eye)
1453 (use 'cortex.world)
1454 (use 'cortex.import)
1455 (use 'clojure.contrib.def)
1456 (cortex.import/mega-import-jme3)
1457 (rlm.rlm-commands/help)
1458 (import java.nio.ByteBuffer)
1459 (import java.awt.image.BufferedImage)
1460 (import java.awt.Color)
1461 (import java.awt.Dimension)
1462 (import java.awt.Graphics)
1463 (import java.awt.Graphics2D)
1464 (import java.awt.event.WindowAdapter)
1465 (import java.awt.event.WindowEvent)
1466 (import java.awt.image.BufferedImage)
1467 (import java.nio.ByteBuffer)
1468 (import javax.swing.JFrame)
1469 (import javax.swing.JPanel)
1470 (import javax.swing.SwingUtilities)
1471 (import javax.swing.ImageIcon)
1472 (import javax.swing.JOptionPane)
1473 (import java.awt.image.ImageObserver)
1477 (defn scene-processor
1478 "deals with converting FrameBuffers to BufferedImages so
1479 that the continuation function can be defined only in terms
1480 of what it does with BufferedImages"
1481 [continuation]
1482 (let [byte-buffer (atom nil)
1483 renderer (atom nil)
1484 image (atom nil)]
1485 (proxy [SceneProcessor] []
1486 (initialize
1487 [renderManager viewPort]
1488 (let [cam (.getCamera viewPort)
1489 width (.getWidth cam)
1490 height (.getHeight cam)]
1491 (reset! renderer (.getRenderer renderManager))
1492 (reset! byte-buffer
1493 (BufferUtils/createByteBuffer
1494 (* width height 4)))
1495 (reset! image (BufferedImage. width height
1496 BufferedImage/TYPE_4BYTE_ABGR))))
1497 (isInitialized [] (not (nil? @byte-buffer)))
1498 (reshape [_ _ _])
1499 (preFrame [_])
1500 (postQueue [_])
1501 (postFrame
1502 [#^FrameBuffer fb]
1503 (.clear @byte-buffer)
1504 (.readFrameBuffer @renderer fb @byte-buffer)
1505 (Screenshots/convertScreenShot @byte-buffer @image)
1506 (continuation @image))
1507 (cleanup []))))
1509 (defn add-eye
1510 "Add an eye to the world, and call continuation on
1511 every frame produced"
1512 [world camera continuation]
1513 (let [width (.getWidth camera)
1514 height (.getHeight camera)
1515 render-manager (.getRenderManager world)
1516 viewport (.createMainView render-manager "eye-view" camera)]
1517 (doto viewport
1518 (.setBackgroundColor ColorRGBA/Black)
1519 (.setClearFlags true true true)
1520 (.addProcessor (scene-processor continuation))
1521 (.attachScene (.getRootNode world)))))
1523 (defn make-display-frame [display width height]
1524 (SwingUtilities/invokeLater
1525 (fn []
1526 (.setPreferredSize display (Dimension. width height))
1527 (doto (JFrame. "Eye Camera!")
1528 (-> (.getContentPane) (.add display))
1529 (.setDefaultCloseOperation JFrame/DISPOSE_ON_CLOSE)
1530 (.pack)
1531 (.setLocationRelativeTo nil)
1532 (.setResizable false)
1533 (.setVisible true)))))
1535 (defn image-monitor [#^BufferedImage image]
1536 (proxy [JPanel] []
1537 (paintComponent
1538 [g]
1539 (proxy-super paintComponent g)
1540 (locking image
1541 (.drawImage g image 0 0
1542 (proxy [ImageObserver]
1543 []
1544 (imageUpdate
1545 []
1546 (proxy-super imageUpdate))))))))
1548 (defn movie-image []
1549 (let [setup
1550 (runonce
1551 (fn [#^BufferedImage image]
1552 (let [width (.getWidth image)
1553 height (.getHeight image)
1554 display (image-monitor image)
1555 frame (make-display-frame display width height)]
1556 display)))]
1557 (fn [#^BufferedImage image]
1558 (.repaint (setup image)))))
1561 (defn observer
1562 "place thy eye!"
1563 [world camera]
1564 (let [eye camera
1565 width (.getWidth eye)
1566 height (.getHeight eye)]
1567 (no-exceptions
1568 (add-eye
1569 world
1570 eye
1571 (movie-image)))))
1572 #+end_src
1574 #+srcname: test-vision
1575 #+begin_src clojure
1577 (ns test.vision)
1578 (use 'cortex.world)
1579 (use 'cortex.import)
1580 (use 'clojure.contrib.def)
1581 (use 'body.eye)
1582 (cortex.import/mega-import-jme3)
1583 (rlm.rlm-commands/help)
1584 (import java.nio.ByteBuffer)
1585 (import java.awt.image.BufferedImage)
1586 (import java.awt.Color)
1587 (import java.awt.Dimension)
1588 (import java.awt.Graphics)
1589 (import java.awt.Graphics2D)
1590 (import java.awt.event.WindowAdapter)
1591 (import java.awt.event.WindowEvent)
1592 (import java.awt.image.BufferedImage)
1593 (import java.nio.ByteBuffer)
1594 (import javax.swing.JFrame)
1595 (import javax.swing.JPanel)
1596 (import javax.swing.SwingUtilities)
1597 (import javax.swing.ImageIcon)
1598 (import javax.swing.JOptionPane)
1599 (import java.awt.image.ImageObserver)
1602 (def width 200)
1603 (def height 200)
1605 (defn camera []
1606 (doto (Camera. width height)
1607 (.setFrustumPerspective 45 1 1 1000)
1608 (.setLocation (Vector3f. -3 0 -5))
1609 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1611 (defn camera2 []
1612 (doto (Camera. width height)
1613 (.setFrustumPerspective 45 1 1 1000)
1614 (.setLocation (Vector3f. 3 0 -5))
1615 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1617 (defn setup-fn [world]
1618 (let [eye (camera)
1619 width (.getWidth eye)
1620 height (.getHeight eye)]
1621 (no-exceptions
1622 (add-eye
1623 world
1624 eye
1625 (runonce visual))
1626 (add-eye
1627 world
1628 (camera2)
1629 (runonce visual)))))
1631 (defn spider-eye [position]
1632 (doto (Camera. 200 200 )
1633 (.setFrustumPerspective 45 1 1 1000)
1634 (.setLocation position)
1635 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1637 (defn setup-fn* [world]
1638 (let [eye (camera)
1639 width (.getWidth eye)
1640 height (.getHeight eye)]
1641 ;;(.setClearFlags (.getViewPort world) true true true)
1642 (observer world (.getCamera world))
1643 (observer world (spider-eye (Vector3f. 3 0 -5)))
1644 ;;(observer world (spider-eye (Vector3f. 0 0 -5)))
1645 ;; (observer world (spider-eye (Vector3f. -3 0 -5)))
1646 ;; (observer world (spider-eye (Vector3f. 0 3 -5)))
1647 ;; (observer world (spider-eye (Vector3f. 0 -3 -5)))
1648 ;; (observer world (spider-eye (Vector3f. 3 3 -5)))
1649 ;; (observer world (spider-eye (Vector3f. -3 3 -5)))
1650 ;; (observer world (spider-eye (Vector3f. 3 -3 -5)))
1651 ;; (observer world (spider-eye (Vector3f. -3 -3 -5)))
1654 world)
1656 (defn test-world []
1657 (let [thing (box 1 1 1 :physical? false)]
1658 (world
1659 (doto (Node.)
1660 (.attachChild thing))
1661 {}
1662 setup-fn
1663 (fn [world tpf]
1664 (.rotate thing (* tpf 0.2) 0 0)
1665 ))))
1668 #+end_src
1671 #+results: eyes
1672 : #'body.eye/test-world
1674 Note the use of continuation passing style for connecting the eye to a
1675 function to process the output. The example code will create two
1676 videos of the same rotating cube from different angles, sutiable for
1677 stereoscopic vision.
1684 * COMMENT code generation
1685 #+begin_src clojure :tangle ../src/cortex/import.clj
1686 <<import>>
1687 #+end_src
1689 #+begin_src clojure :tangle ../src/hello/brick_wall.clj
1690 <<brick-wall-header>>
1691 <<brick-wall-body>>
1692 #+end_src
1694 #+begin_src clojure :tangle ../src/hello/hello_simple_app.clj
1695 <<hello-simple-app>>
1696 #+end_src
1698 #+begin_src clojure :tangle ../src/cortex/world.clj
1699 <<world-inputs>>
1700 <<world>>
1701 <<world-shapes>>
1702 <<world-view>>
1703 #+end_src
1705 #+begin_src clojure :tangle ../src/cortex/other_games.clj
1706 <<other-games>>
1707 #+end_src
1709 #+begin_src clojure :tangle ../src/hello/loop.clj
1710 <<hello-loop>>
1711 #+end_src
1713 #+begin_src clojure :tangle ../src/hello/collision.clj
1714 <<hello-collision>>
1715 #+end_src
1717 #+begin_src clojure :tangle ../src/hello/terrain.clj
1718 <<hello-terrain>>
1719 #+end_src
1721 #+begin_src clojure :tangle ../src/hello/animation.clj
1722 <<hello-animation>>
1723 #+end_src
1725 #+begin_src clojure :tangle ../src/hello/material.clj
1726 <<material>>
1727 #+end_src
1729 #+begin_src clojure :tangle ../src/body/eye.clj
1730 <<eyes>>
1731 #+end_src
1733 #+begin_src clojure :tangle ../src/test/vision.clj
1734 <<test-vision>>
1735 #+end_src