view org/cortex.org @ 8:522cf85fdb57

getting there!
author Robert McIntyre <rlm@mit.edu>
date Sun, 23 Oct 2011 00:59:22 -0700
parents e3c6d1c1cb00
children c32f3eb9fdeb
line wrap: on
line source
1 #+title: Simulated Senses
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+description: Simulating senses for AI research using JMonkeyEngine3
5 #+SETUPFILE: ../../aurellem/org/setup.org
6 #+INCLUDE: ../../aurellem/org/level-0.org
7 #+babel: :mkdirp yes :noweb yes
9 * Background
10 Artificial Intelligence has tried and failed for more than half a
11 century to produce programs as flexible, creative, and “intelligent”
12 as the human mind itself. Clearly, we are still missing some important
13 ideas concerning intelligent programs or we would have strong AI
14 already. What idea could be missing?
16 When Turing first proposed his famous “Turing Test” in the
17 groundbreaking paper [[./sources/turing.pdf][/Computing Machines and Intelligence/]], he gave
18 little importance to how a computer program might interact with the
19 world:
21 #+BEGIN_QUOTE
22 \ldquo{}We need not be too concerned about the legs, eyes, etc. The example of
23 Miss Helen Keller shows that education can take place provided that
24 communication in both directions between teacher and pupil can take
25 place by some means or other.\rdquo{}
26 #+END_QUOTE
28 And from the example of Hellen Keller he went on to assume that the
29 only thing a fledgling AI program could need by way of communication
30 is a teletypewriter. But Hellen Keller did possess vision and hearing
31 for the first few months of her life, and her tactile sense was far
32 more rich than any text-stream could hope to achieve. She possessed a
33 body she could move freely, and had continual access to the real world
34 to learn from her actions.
36 I believe that our programs are suffering from too little sensory
37 input to become really intelligent. Imagine for a moment that you
38 lived in a world completely cut off form all sensory stimulation. You
39 have no eyes to see, no ears to hear, no mouth to speak. No body, no
40 taste, no feeling whatsoever. The only sense you get at all is a
41 single point of light, flickering on and off in the void. If this was
42 your life from birth, you would never learn anything, and could never
43 become intelligent. Actual humans placed in sensory deprivation
44 chambers experience hallucinations and can begin to loose their sense
45 of reality in as little as 15 minutes[sensory-deprivation]. Most of
46 the time, the programs we write are in exactly this situation. They do
47 not interface with cameras and microphones, and they do not control a
48 real or simulated body or interact with any sort of world.
51 * Simulation vs. Reality
52 I want demonstrate that multiple senses are what enable
53 intelligence. There are two ways of playing around with senses and
54 computer programs:
56 The first is to go entirely with simulation: virtual world, virtual
57 character, virtual senses. The advantages are that when everything is
58 a simulation, experiments in that simulation are absolutely
59 reproducible. It's also easier to change the character and world to
60 explore new situations and different sensory combinations.
63 ** Issues with Simulation
65 If the world is to be simulated on a computer, then not only do you
66 have to worry about whether the character's senses are rich enough to
67 learn from the world, but whether the world itself is rendered with
68 enough detail and realism to give enough working material to the
69 character's senses. To name just a few difficulties facing modern
70 physics simulators: destructibility of the environment, simulation of
71 water/other fluids, large areas, nonrigid bodies, lots of objects,
72 smoke. I don't know of any computer simulation that would allow a
73 character to take a rock and grind it into fine dust, then use that
74 dust to make a clay sculpture, at least not without spending years
75 calculating the interactions of every single small grain of
76 dust. Maybe a simulated world with today's limitations doesn't provide
77 enough richness for real intelligence to evolve.
79 ** Issues with Reality
81 The other approach for playing with senses is to hook your software up
82 to real cameras, microphones, robots, etc., and let it loose in the
83 real world. This has the advantage of eliminating concerns about
84 simulating the world at the expense of increasing the complexity of
85 implementing the senses. Instead of just grabbing the current rendered
86 frame for processing, you have to use an actual camera with real
87 lenses and interact with photons to get an image. It is much harder to
88 change the character, which is now partly a physical robot of some
89 sort, since doing so involves changing things around in the real world
90 instead of modifying lines of code. While the real world is very rich
91 and definitely provides enough stimulation for intelligence to develop
92 as evidenced by our own existence, it is also uncontrollable in the
93 sense that a particular situation cannot be recreated perfectly or
94 saved for later use. It is harder to conduct science because it is
95 harder to repeat an experiment. The worst thing about using the real
96 world instead of a simulation is the matter of time. Instead of
97 simulated time you get the constant and unstoppable flow of real
98 time. This severely limits the sorts of software you can use to
99 program the AI because all sense inputs must be handled in real
100 time. Complicated ideas may have to be implemented in hardware or may
101 simply be impossible given the current speed of our
102 processors. Contrast this with a simulation, in which the flow of time
103 in the simulated world can be slowed down to accommodate the
104 limitations of the character's programming. In terms of cost, doing
105 everything in software is far cheaper than building custom real-time
106 hardware. All you need is a laptop and some patience.
108 * Choose a Simulation Engine
110 Mainly because of issues with controlling the flow of time, I chose to
111 simulate both the world and the character. I set out to make a minimal
112 world in which I could embed a character with multiple senses. My main
113 goal is to make an environment where I can perform further experiments
114 in simulated senses.
116 As Carl Sagan once said, "If you wish to make an apple pie from
117 scratch, you must first invent the universe.” I examined many
118 different 3D environments to try and find something I would use as the
119 base for my simulation; eventually the choice came down to three
120 engines: the Quake II engine, the Source Engine, and jMonkeyEngine.
122 ** Quake II/Jake2
124 I spent a bit more than a month working with the Quake II Engine from
125 ID software to see if I could use it for my purposes. All the source
126 code was released by ID software into the Public Domain several years
127 ago, and as a result it has been ported and modified for many
128 different reasons. This engine was famous for its advanced use of
129 realistic shading and had decent and fast physics
130 simulation. Researchers at Princeton [[http://www.nature.com/nature/journal/v461/n7266/pdf/nature08499.pdf][used this code]] to study spatial
131 information encoding in the hippocampal cells of rats. Those
132 researchers created a special Quake II level that simulated a maze,
133 and added an interface where a mouse could run around inside a ball in
134 various directions to move the character in the simulated maze. They
135 measured hippocampal activity during this exercise to try and tease
136 out the method in which spatial data was stored in that area of the
137 brain. I find this promising because if a real living rat can interact
138 with a computer simulation of a maze in the same way as it interacts
139 with a real-world maze, then maybe that simulation is close enough to
140 reality that a simulated sense of vision and motor control interacting
141 with that simulation could reveal useful information about the real
142 thing. It happens that there is a Java port of the original C source
143 code called Jake2. The port demonstrates Java's OpenGL bindings and
144 runs anywhere from 90% to 105% as fast as the C version. After
145 reviewing much of the source of Jake2, I eventually rejected it
146 because the engine is too tied to the concept of a first-person
147 shooter game. One of the problems I had was that there does not seem
148 to be any easy way to attach multiple cameras to a single
149 character. There are also several physics clipping issues that are
150 corrected in a way that only applies to the main character and does
151 not apply to arbitrary objects. While there is a large community of
152 level modders, I couldn't find a community to support using the engine
153 to make new things.
155 ** Source Engine
157 The Source Engine evolved from the Quake II and Quake I engines and is
158 used by Valve in the Half-Life series of games. The physics simulation
159 in the Source Engine is quite accurate and probably the best out of
160 all the engines I investigated. There is also an extensive community
161 actively working with the engine. However, applications that use the
162 Source Engine must be written in C++, the code is not open, it only
163 runs on Windows, and the tools that come with the SDK to handle models
164 and textures are complicated and awkward to use.
166 ** jMonkeyEngine
168 jMonkeyEngine is a new library for creating games in Java. It uses
169 OpenGL to render to the screen and uses screengraphs to avoid drawing
170 things that do not appear on the screen. It has an active community
171 and several games in the pipeline. The engine was not built to serve
172 any particular game but is instead meant to be used for any 3D
173 game. After experimenting with each of these three engines and a few
174 others for about 2 months I settled on jMonkeyEngine. I chose it
175 because it had the most features out of all the open projects I looked
176 at, and because I could then write my code in Clojure, an
177 implementation of LISP that runs on the JVM.
179 * Setup
181 First, I checked out the source to jMonkeyEngine:
183 #+srcname: checkout
184 #+begin_src sh :results verbatim
185 svn checkout http://jmonkeyengine.googlecode.com/svn/trunk/engine jme3
186 #+end_src
188 #+results: checkout
189 : Checked out revision 7975.
192 Building jMonkeyEngine is easy enough:
194 #+srcname: build
195 #+begin_src sh :results verbatim
196 cd jme3
197 ant jar | tail -n 2
198 #+end_src
200 #+results: build
201 : BUILD SUCCESSFUL
202 : Total time: 15 seconds
205 Also build the javadoc:
207 #+srcname: javadoc
208 #+begin_src sh :results verbatim
209 cd jme3
210 ant javadoc | tail -n 2
211 #+end_src
213 #+results: javadoc
214 : BUILD SUCCESSFUL
215 : Total time: 12 seconds
217 Now, move the jars from the compilation into the project's lib folder.
219 #+srcname: move-jars
220 #+begin_src sh :results verbatim
221 mkdir -p lib
222 mkdir -p src
223 cp jme3/dist/jMonkeyEngine3.jar lib/
224 cp jme3/dist/lib/* lib/
225 ls lib
226 #+end_src
228 #+results: move-jars
229 #+begin_example
230 eventbus-1.4.jar
231 jbullet.jar
232 jheora-jst-debug-0.6.0.jar
233 jinput.jar
234 jME3-jbullet.jar
235 jME3-lwjgl-natives.jar
236 jME3-testdata.jar
237 jME3-test.jar
238 jMonkeyEngine3.jar
239 j-ogg-oggd.jar
240 j-ogg-vorbisd.jar
241 lwjgl.jar
242 nifty-1.3.jar
243 nifty-default-controls-1.3.jar
244 nifty-examples-1.3.jar
245 nifty-lwjgl-renderer-1.3.jar
246 nifty-openal-soundsystem-1.0.jar
247 nifty-style-black-1.3.jar
248 nifty-style-grey-1.0.jar
249 noise-0.0.1-SNAPSHOT.jar
250 stack-alloc.jar
251 vecmath.jar
252 xmlpull-xpp3-1.1.4c.jar
253 #+end_example
255 It's good to create a =assets= directory in the style that the
256 =AssetManager= will like.
258 #+srcname: create-assets
259 #+begin_src sh :results verbatim
260 mkdir -p assets
261 mkdir -p assets/Interface
262 mkdir -p assets/Materials
263 mkdir -p assets/MatDefs
264 mkdir -p assets/Models
265 mkdir -p assets/Scenes
266 mkdir -p assets/Shaders
267 mkdir -p assets/Sounds
268 mkdir -p assets/Textures
269 tree -L 1 assets
270 #+end_src
272 #+results: create-assets
273 #+begin_example
274 assets
275 |-- Interface
276 |-- MatDefs
277 |-- Materials
278 |-- Models
279 |-- Scenes
280 |-- Shaders
281 |-- Sounds
282 `-- Textures
284 8 directories, 0 files
285 #+end_example
288 The java classpath should have all the jars contained in the =lib=
289 directory as well as the src directory.
291 For example, here is the file I use to run my REPL for clojure.
293 #+include: "~/swank-all" src sh :exports code
295 The important thing here is that =cortex/lib/*=, =cortex/src=, and
296 =cortex/assets= appear on the classpath. (=cortex= is the base
297 directory of this project.)
299 #+srcname: pwd
300 #+begin_src sh
301 pwd
302 #+end_src
304 #+results: pwd
305 : /home/r/cortex
308 * Simulation Base
310 ** Imports
311 First, I'll import jme core classes.
312 #+srcname: import
313 #+begin_src clojure :results silent
314 (ns cortex.import
315 (:require swank.util.class-browse))
317 (defn import-jme3 []
318 (import '[com.jme3.system AppSettings JmeSystem])
319 (import '[com.jme3.app Application SimpleApplication])
320 (import 'com.jme3.material.Material)
321 (import '[com.jme3.math Vector3f ColorRGBA Quaternion Transform])
322 (import '[com.jme3.scene Node Geometry])
323 (import '[com.jme3.scene.shape Box Sphere Sphere$TextureMode])
324 (import 'com.jme3.font.BitmapText)
325 (import '[com.jme3.input KeyInput InputManager])
326 (import '[com.jme3.input.controls
327 ActionListener AnalogListener KeyTrigger MouseButtonTrigger])
328 (import '[com.jme3.asset AssetManager DesktopAssetManager] )
329 (import '[com.jme3.asset.plugins HttpZipLocator ZipLocator])
330 (import '[com.jme3.light PointLight DirectionalLight])
331 (import '[com.jme3.animation AnimControl Skeleton Bone])
332 (import '[com.jme3.bullet.collision.shapes
333 MeshCollisionShape SphereCollisionShape BoxCollisionShape])
334 (import 'com.jme3.renderer.queue.RenderQueue$ShadowMode)
335 (import 'jme3test.TestChooser)
336 (import '[com.jme3.bullet PhysicsTickListener PhysicsSpace])
337 (import '[com.jme3.bullet.joints SixDofJoint HingeJoint
338 SliderJoint Point2PointJoint ConeJoint]))
341 (defmacro permissive-import* [class-symbol]
342 `(try
343 (import ~class-symbol)
344 (catch Exception e#
345 (println "can't import " ~class-symbol))))
347 (defn permissive-import [class-symbol]
348 (eval (list 'cortex.import/permissive-import* class-symbol)))
350 (defn selection-import [selection-fn]
351 (dorun
352 (map (comp permissive-import symbol)
353 (filter selection-fn
354 (map :name
355 swank.util.class-browse/available-classes)))))
357 (defn mega-import-jme3
358 "ALL the jme classes. For REPL use."
359 []
360 (selection-import
361 #(and
362 (.startsWith % "com.jme3.")
363 ;; Don't import the Lwjgl stuff since it can throw exceptions
364 ;; upon being loaded.
365 (not (re-matches #".*Lwjgl.*" %)))))
366 #+end_src
368 The =mega-import-jme3= is quite usefull for debugging purposes since
369 it allows completion for almost all of JME's classes
371 ** Simplification
372 *** World
374 It is comvienent to wrap the JME elements that deal with creating a
375 world, creation of basic objects, and Keyboard input with a nicer
376 interface (at least for my purposes).
378 #+srcname: world-inputs
379 #+begin_src clojure :results silent
380 (ns cortex.world)
381 (require 'cortex.import)
382 (use 'clojure.contrib.def)
383 (rlm.rlm-commands/help)
384 (cortex.import/mega-import-jme3)
386 (defvar *app-settings*
387 (doto (AppSettings. true)
388 (.setFullscreen false)
389 (.setTitle "Aurellem.")
390 ;; disable 32 bit stuff for now
391 ;;(.setAudioRenderer "Send")
392 )
393 "These settings control how the game is displayed on the screen for
394 debugging purposes. Use binding forms to change this if desired.
395 Full-screen mode does not work on some computers.")
397 (defn asset-manager
398 "returns a new, configured assetManager" []
399 (JmeSystem/newAssetManager
400 (.getResource
401 (.getContextClassLoader (Thread/currentThread))
402 "com/jme3/asset/Desktop.cfg")))
404 (defmacro no-exceptions
405 "Sweet relief like I never knew."
406 [& forms]
407 `(try ~@forms (catch Exception e# (.printStackTrace e#))))
409 (defn thread-exception-removal []
410 (println "removing exceptions from " (Thread/currentThread))
411 (.setUncaughtExceptionHandler
412 (Thread/currentThread)
413 (proxy [Thread$UncaughtExceptionHandler] []
414 (uncaughtException
415 [thread thrown]
416 (println "uncaught-exception thrown in " thread)
417 (println (.getMessage thrown))))))
419 (def println-repl (bound-fn [& args] (apply println args)))
421 (use '[pokemon [lpsolve :only [constant-map]]])
423 (defn no-op [& _])
425 (defn all-keys
426 "Construct a map of strings representing all the manual inputs from
427 either the keyboard or mouse."
428 []
429 (let [inputs (constant-map KeyInput)]
430 (assoc
431 (zipmap (map (fn [field]
432 (.toLowerCase (re-gsub #"_" "-" field))) (vals inputs))
433 (map (fn [val] (KeyTrigger. val)) (keys inputs)))
434 ;;explicitly add mouse controls
435 "mouse-left" (MouseButtonTrigger. 0)
436 "mouse-middle" (MouseButtonTrigger. 2)
437 "mouse-right" (MouseButtonTrigger. 1))))
439 (defn initialize-inputs
440 "more java-interop cruft to establish keybindings for a particular virtual world"
441 [game input-manager key-map]
442 (doall (map (fn [[name trigger]]
443 (.addMapping ^InputManager input-manager
444 name (into-array (class trigger) [trigger]))) key-map))
445 (doall (map (fn [name]
446 (.addListener ^InputManager input-manager game
447 (into-array String [name]))) (keys key-map))))
449 #+end_src
451 These functions are all for debug controlling of the world through
452 keyboard and mouse.
454 We reuse =constant-map= from =pokemon.lpsolve= to get the numerical
455 values for all the keys defined in the =KeyInput= class. The
456 documentation for =constant-map= is:
458 #+begin_src clojure :results output
459 (doc pokemon.lpsolve/constant-map)
460 #+end_src
462 #+results:
463 : -------------------------
464 : pokemon.lpsolve/constant-map
465 : ([class])
466 : Takes a class and creates a map of the static constant integer
467 : fields with their names. This helps with C wrappers where they have
468 : just defined a bunch of integer constants instead of enums
471 Then, =all-keys= converts the constant names like =KEY_J= to the more
472 clojure-like =key-j=, and returns a map from these keys to
473 jMonkeyEngine KeyTrigger objects, the use of which will soon become
474 apparent. =all-keys= also adds the three mouse button controls to the
475 map.
477 #+srcname: world
478 #+begin_src clojure :results silent
479 (in-ns 'cortex.world)
481 (defn traverse
482 "apply f to every non-node, deeply"
483 [f node]
484 (if (isa? (class node) Node)
485 (dorun (map (partial traverse f) (.getChildren node)))
486 (f node)))
488 (def gravity (Vector3f. 0 -9.81 0))
490 (defn world
491 [root-node key-map setup-fn update-fn]
492 (let [physics-manager (BulletAppState.)
493 shadow-renderer (BasicShadowRenderer. (asset-manager) (int 256))
494 ;;maybe use a better shadow renderer someday!
495 ;;shadow-renderer (PssmShadowRenderer. (asset-manager) 256 1)
496 ]
497 (doto
498 (proxy [SimpleApplication ActionListener] []
499 (simpleInitApp
500 []
501 (no-exceptions
502 (.setTimer this (IsoTimer. 60))
503 ;; Create key-map.
504 (.setFrustumFar (.getCamera this) 300)
505 (initialize-inputs this (.getInputManager this) (all-keys))
506 ;; Don't take control of the mouse
507 (org.lwjgl.input.Mouse/setGrabbed false)
508 ;; add all objects to the world
509 (.attachChild (.getRootNode this) root-node)
510 ;; enable physics
511 ;; add a physics manager
512 (.attach (.getStateManager this) physics-manager)
513 (.setGravity (.getPhysicsSpace physics-manager) gravity)
516 ;; go through every object and add it to the physics manager
517 ;; if relavant.
518 (traverse (fn [geom]
519 (dorun
520 (for [n (range (.getNumControls geom))]
521 (do
522 (println-repl "adding control " (.getControl geom n))
523 (.add (.getPhysicsSpace physics-manager)
524 (.getControl geom n))))))
525 (.getRootNode this))
526 ;;(.addAll (.getPhysicsSpace physics-manager) (.getRootNode this))
528 (setup-fn this)
529 (.setDirection shadow-renderer
530 (.normalizeLocal (Vector3f. -1 -1 -1)))
531 (.addProcessor (.getViewPort this) shadow-renderer)
532 (.setShadowMode (.getRootNode this) RenderQueue$ShadowMode/Off)
533 ))
534 (simpleUpdate
535 [tpf]
536 (no-exceptions
537 (update-fn this tpf)))
538 (onAction
539 [binding value tpf]
540 ;; whenever a key is pressed, call the function returned from
541 ;; key-map.
542 (no-exceptions
543 (if-let [react (key-map binding)]
544 (react this value)))))
545 ;; don't show a menu to change options.
547 (.setShowSettings false)
548 (.setPauseOnLostFocus false)
549 (.setSettings *app-settings*))))
551 (defn apply-map
552 "Like apply, but works for maps and functions that expect an implicit map
553 and nothing else as in (fn [& {}]).
554 ------- Example -------
555 (defn jjj [& {:keys [www] :or {www \"oh yeah\"} :as env}] (println www))
556 (apply-map jjj {:www \"whatever\"})
557 -->\"whatever\""
558 [fn m]
559 (apply fn (reduce #(into %1 %2) [] m)))
561 #+end_src
564 =world= is the most important function here.
565 *** TODO more documentation
567 #+srcname: world-shapes
568 #+begin_src clojure :results silent
569 (in-ns 'cortex.world)
570 (defrecord shape-description
571 [name
572 color
573 mass
574 friction
575 texture
576 material
577 position
578 rotation
579 shape
580 physical?])
582 (def base-shape
583 (shape-description.
584 "default-shape"
585 false
586 ;;ColorRGBA/Blue
587 1.0 ;; mass
588 1.0 ;; friction
589 ;; texture
590 "Textures/Terrain/BrickWall/BrickWall.jpg"
591 ;; material
592 "Common/MatDefs/Misc/Unshaded.j3md"
593 Vector3f/ZERO
594 Quaternion/IDENTITY
595 (Box. Vector3f/ZERO 0.5 0.5 0.5)
596 true))
598 (defn make-shape
599 [#^shape-description d]
600 (let [asset-manager (if (:asset-manager d) (:asset-manager d) (asset-manager))
601 mat (Material. asset-manager (:material d))
602 geom (Geometry. (:name d) (:shape d))]
603 (if (:texture d)
604 (let [key (TextureKey. (:texture d))]
605 (.setGenerateMips key true)
606 (.setTexture mat "ColorMap" (.loadTexture asset-manager key))))
607 (if (:color d) (.setColor mat "Color" (:color d)))
608 (.setMaterial geom mat)
609 (if-let [rotation (:rotation d)] (.rotate geom rotation))
610 (.setLocalTranslation geom (:position d))
611 (if (:physical? d)
612 (let [impact-shape (doto (GImpactCollisionShape.
613 (.getMesh geom)) (.setMargin 0))
614 physics-control (RigidBodyControl.
615 ;;impact-shape ;; comment to disable
616 (float (:mass d)))]
617 (.createJmeMesh impact-shape)
618 (.addControl geom physics-control)
619 ;;(.setSleepingThresholds physics-control (float 0) (float 0))
620 (.setFriction physics-control (:friction d))))
621 ;;the default is to keep this node in the physics engine forever.
622 ;;these commands must come after the control is added to the geometry.
623 ;;
624 geom))
626 (defn box
627 ([l w h & {:as options}]
628 (let [options (merge base-shape options)]
629 (make-shape (assoc options
630 :shape (Box. l w h)))))
631 ([] (box 0.5 0.5 0.5)))
633 (defn sphere
634 ([r & {:as options}]
635 (let [options (merge base-shape options)]
636 (make-shape (assoc options
637 :shape (Sphere. 32 32 (float r))))))
638 ([] (sphere 0.5)))
640 (defn add-element [game node]
641 (.addAll
642 (.getPhysicsSpace
643 (.getState
644 (.getStateManager game)
645 BulletAppState))
646 node)
647 (.attachChild (.getRootNode game) node))
649 (defn set-gravity*
650 [game gravity]
651 (traverse
652 (fn [geom]
653 (if-let
654 [control (.getControl geom RigidBodyControl)]
655 (do
656 (.setGravity control gravity)
657 (.applyImpulse control Vector3f/ZERO Vector3f/ZERO)
658 )))
659 (.getRootNode game)))
661 #+end_src
663 These are convienence functions for creating JME objects and
664 manipulating a world.
666 #+srcname: world-view
667 #+begin_src clojure :results silent
668 (in-ns 'cortex.world)
670 (defprotocol Viewable
671 (view [something]))
673 (extend-type com.jme3.scene.Geometry
674 Viewable
675 (view [geo]
676 (view (doto (Node.)(.attachChild geo)))))
678 (extend-type com.jme3.scene.Node
679 Viewable
680 (view [node]
681 (.start
682 (world node
683 {}
684 (fn [world]
685 (.enableDebug
686 (.getPhysicsSpace
687 (.getState
688 (.getStateManager world)
689 BulletAppState))
690 (asset-manager))
691 (set-gravity* world Vector3f/ZERO)
692 ;; (set-gravity* world (Vector3f. 0 (float -0.4) 0))
693 (let [sun (doto (DirectionalLight.)
694 (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))
695 (.setColor ColorRGBA/White))]
696 (.addLight (.getRootNode world) sun)))
697 no-op))))
699 (defn position-camera [game]
700 (doto (.getCamera game)
701 (.setLocation (Vector3f. 0 6 6))
702 (.lookAt Vector3f/ZERO (Vector3f. 0 1 0))))
704 #+end_src
706 Here I make the =Viewable= protocol and extend it to JME's types. Now
707 hello-world can be written as easily as:
709 #+begin_src clojure :results silent
710 (cortex.world/view (cortex.world/box))
711 #+end_src
713 ** Hello
714 Here are the jmonkeyengine "Hello" programs translated to clojure.
715 *** Hello Simple App
716 Here is the hello world example for jme3 in clojure.
717 It's a more or less direct translation from the java source
718 from
719 http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_simpleapplication.
721 Of note is the fact that since we don't have access to the
722 =AssetManager= via extendig =SimpleApplication=, we have to build one
723 ourselves.
725 #+srcname: hello-simple-app
726 #+begin_src clojure :results silent
727 (ns hello.hello-simple-app)
728 (require 'cortex.import)
729 (use 'clojure.contrib.def)
730 (rlm.rlm-commands/help)
731 (cortex.import/import-jme3)
732 (use 'cortex.world)
735 (def cube (Box. Vector3f/ZERO 1 1 1))
737 (def geom (Geometry. "Box" cube))
739 (def mat (Material. (asset-manager) "Common/MatDefs/Misc/Unshaded.j3md"))
741 (.setColor mat "Color" ColorRGBA/Blue)
743 (.setMaterial geom mat)
745 (defn simple-app []
746 (doto
747 (proxy [SimpleApplication] []
748 (simpleInitApp
749 []
750 ;; Don't take control of the mouse
751 (org.lwjgl.input.Mouse/setGrabbed false)
752 (.attachChild (.getRootNode this) geom)))
753 ;; don't show a menu to change options.
754 (.setShowSettings false)
755 (.setPauseOnLostFocus false)
756 (.setSettings *app-settings*)))
757 #+end_src
759 Running this program will begin a new jMonkeyEngine game which
760 displays a single blue cube.
762 #+begin_src clojure :exports code :results silent
763 (.start (hello.hello-simple-app/simple-app))
764 #+end_src
766 #+caption: the simplest JME game.
767 [[./images/simple-app.jpg]]
771 *** Hello Physics
772 From http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_physics
774 #+srcname: brick-wall-header
775 #+begin_src clojure :results silent
776 (ns hello.brick-wall)
777 (require 'cortex.import)
778 (use 'clojure.contrib.def)
779 (rlm.rlm-commands/help)
780 (cortex.import/mega-import-jme3)
781 (use '[pokemon [lpsolve :only [constant-map]]])
782 (use 'cortex.world)
783 #+end_src
785 #+srcname: brick-wall-body
786 #+begin_src clojure :results silent
787 (in-ns 'hello.brick-wall)
789 (defn floor
790 "make a sturdy, unmovable physical floor"
791 []
792 (box 20 1 20 :mass 0 :color false :position (Vector3f. 0 -2 0)))
794 (def brick-length 0.48)
795 (def brick-width 0.24)
796 (def brick-height 0.12)
799 (defn brick* [position]
800 (doto (box brick-length brick-height brick-width
801 :position position :name "brick"
802 :material "Common/MatDefs/Misc/Unshaded.j3md"
803 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
804 :mass 36)
805 (->
806 (.getMesh)
807 (.scaleTextureCoordinates (Vector2f. 1 0.5)))
808 ;;(.setShadowMode RenderQueue$ShadowMode/CastAndReceive)
809 )
810 )
812 (defn inception-brick-wall
813 "construct a physical brick wall"
814 []
815 (let [node (Node. "brick-wall")]
816 (dorun
817 (map (comp #(.attachChild node %) brick*)
818 (for
819 [x (range 15)
820 y (range 10)
821 z (range 1)]
822 (Vector3f.
823 (* brick-length x 1.03)
824 (* brick-width y y 10)
825 (* brick-height z)))))
826 node))
828 (defn gravity-toggle
829 [new-value]
830 (fn [game value]
831 (println-repl "set gravity to " new-value)
832 (if value
833 (set-gravity* game new-value)
834 (set-gravity* game gravity))))
836 (defn fire-cannon-ball []
837 (fn [game value]
838 (if (not value)
839 (let [camera (.getCamera game)
840 cannon-ball
841 (sphere 0.7
842 :material "Common/MatDefs/Misc/Unshaded.j3md"
843 :texture "Textures/PokeCopper.jpg"
844 :position
845 (.add (.getLocation camera)
846 (.mult (.getDirection camera) (float 1)))
847 :mass 3)] ;200 0.05
848 (.setShadowMode cannon-ball RenderQueue$ShadowMode/CastAndReceive)
849 (.setLinearVelocity
850 (.getControl cannon-ball RigidBodyControl)
851 (.mult (.getDirection camera) (float 50))) ;50
852 (add-element game cannon-ball)))))
854 (defn floor* []
855 (doto (box 10 0.1 5 :name "floor" ;10 0.1 5 ; 240 0.1 240
856 :material "Common/MatDefs/Misc/Unshaded.j3md"
857 :texture "Textures/Terrain/Pond/Pond.png"
858 :position (Vector3f. 0 -0.1 0 )
859 :mass 0)
860 (->
861 (.getMesh)
862 (.scaleTextureCoordinates (Vector2f. 3 6)));64 64
863 (->
864 (.getMaterial)
865 (.getTextureParam "ColorMap")
866 (.getTextureValue)
867 (.setWrap Texture$WrapMode/Repeat))
868 (.setShadowMode RenderQueue$ShadowMode/Receive)
869 ))
871 (defn brick-wall* []
872 (let [node (Node. "brick-wall")]
873 (dorun
874 (map
875 (comp #(.attachChild node %) brick*)
876 (for [y (range 15)
877 x (range 4)
878 z (range 1)]
879 (Vector3f.
880 (+ (* 2 x brick-length)
881 (if (even? (+ y z))
882 (/ brick-length 4) (/ brick-length -4)))
883 (+ (* brick-height (inc (* 2 y))))
884 (* 2 z brick-width) ))))
885 (.setShadowMode node RenderQueue$ShadowMode/CastAndReceive)
886 node))
888 (defn brick-wall-game-run []
889 (doto
890 (world
891 (doto (Node.) (.attachChild (floor*))
892 (.attachChild (brick-wall*))
893 )
894 {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))
895 "key-m" (gravity-toggle (Vector3f. 0 0 9.81))
896 "key-l" (gravity-toggle (Vector3f. 9.81 0 0))
897 "key-j" (gravity-toggle (Vector3f. -9.81 0 0))
898 "key-k" (gravity-toggle Vector3f/ZERO)
899 "key-u" (gravity-toggle (Vector3f. 0 9.81 0))
900 "key-o" (gravity-toggle (Vector3f. 0 -9.81 0))
901 "key-f" (fn[game value]
902 (if (not value) (add-element game (brick-wall*))))
903 "key-return" (fire-cannon-ball)}
904 position-camera
905 (fn [& _]))
906 (.start)))
907 #+end_src
909 #+begin_src clojure :results silent
910 (hello.brick-wall/brick-wall-game-run)
911 #+end_src
913 #+caption: the brick wall standing
914 [[./images/brick-wall-standing.jpg]]
916 #+caption: the brick wall after it has been knocked over by a "pok\eacute{}ball"
917 [[./images/brick-wall-knocked-down.jpg]]
919 *** Other Brick Games
920 #+srcname: other-games
921 #+begin_src clojure :results silent
922 (ns cortex.other-games
923 {:author "Dylan Holmes"})
924 (use 'cortex.world)
925 (use 'hello.brick-wall)
926 (use 'cortex.import)
927 (cortex.import/mega-import-jme3)
929 (defn scad [position]
930 (doto (box 0.1 0.1 0.1
931 :position position :name "brick"
932 :material "Common/MatDefs/Misc/Unshaded.j3md"
933 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
934 :mass 20)
935 (->
936 (.getMesh)
937 (.scaleTextureCoordinates (Vector2f. 1 0.5))
938 )
939 (-> (.getControl RigidBodyControl)
940 (.setLinearVelocity (Vector3f. 0 100 0))
941 )
943 ;;(.setShadowMode RenderQueue$ShadowMode/Cast)
944 ))
947 (defn shrapnel []
948 (let [node (Node. "explosion-day")]
949 (dorun
950 (map
951 (comp #(.attachChild node %) scad)
952 (for [y (range 15)
953 x (range 4)
954 z (range 1)]
955 (Vector3f.
956 (+ (* 2 x brick-height)
957 (if (even? (+ y z)) (/ brick-height 4) (/ brick-height -4)))
958 (+ (* brick-height (inc (* 2 y))))
959 (* 2 z brick-height) ))))
960 node))
963 (def domino-height 0.48)
964 (def domino-thickness 0.12)
965 (def domino-width 0.24)
967 (def domino-thickness 0.05)
968 (def domino-width 0.5)
969 (def domino-height 1)
971 (defn domino
972 ([position]
973 (domino position (Quaternion/IDENTITY)))
974 ([position rotation]
975 (doto (box domino-width domino-height domino-thickness
976 :position position :name "domino"
977 :material "Common/MatDefs/Misc/Unshaded.j3md"
978 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
979 :mass 1
980 :rotation rotation)
981 (.setShadowMode RenderQueue$ShadowMode/CastAndReceive)
982 )))
985 (defn domino-row []
986 (let [node (Node. "domino-row")]
987 (dorun
988 (map
989 (comp #(.attachChild node %) domino)
990 (for [
991 z (range 10)
992 x (range 5)
993 ]
994 (Vector3f.
995 (+ (* z domino-width) (* x 5 domino-width))
996 (/ domino-height 1)
997 (* -5.5 domino-thickness z) ))))
999 node))
1001 (defn domino-cycle []
1002 (let [node (Node. "domino-cycle")]
1003 (dorun
1004 (map
1005 (comp #(.attachChild node %) (partial apply domino) )
1006 (for [n (range 720)]
1007 (let [space (* domino-height 5.5)
1008 r (fn[n] (* (+ n 3) domino-width 0.5))
1009 t (fn[n] (reduce
1011 (map
1012 (fn dt[n] (/ space (* 2 (Math/PI) (r n))))
1013 (range n))))
1014 t (t n)
1015 r (r n)
1016 ct (Math/cos t)
1017 st (Math/sin t)
1019 (list
1020 (Vector3f.
1021 (* -1 r st)
1022 (/ domino-height 1)
1023 (* r ct))
1024 (.fromAngleAxis (Quaternion.)
1025 (- (/ 3.1415926 2) t) (Vector3f. 0 1 0))
1026 )))
1027 ))
1028 node))
1031 (defn domino-game-run []
1032 (doto
1033 (world
1034 (doto (Node.) (.attachChild (floor*))
1036 {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))
1037 "key-m" (gravity-toggle (Vector3f. 0 0 9.81))
1038 "key-l" (gravity-toggle (Vector3f. 9.81 0 0))
1039 "key-j" (gravity-toggle (Vector3f. -9.81 0 0))
1040 "key-k" (gravity-toggle (Vector3f. 0 9.81 0) )
1041 "key-u" (fn[g v] ((gravity-toggle (Vector3f. 0 -0 0)) g true))
1042 "key-o" (gravity-toggle (Vector3f. 0 -9.81 0))
1044 "key-space"
1045 (fn[game value]
1047 (if (not value)
1048 (let [d (domino (Vector3f. 0 (/ domino-height 0.25) 0)
1049 (.fromAngleAxis (Quaternion.)
1050 (/ Math/PI 2) (Vector3f. 0 1 0)))]
1051 (add-element game d))))
1052 "key-f"
1053 (fn[game value](if (not value) (add-element game (domino-cycle))))
1054 "key-return" (fire-cannon-ball)}
1055 position-camera
1056 (fn [& _]))
1057 (.start)))
1058 #+end_src
1060 #+begin_src clojure :results silent
1061 (cortex.other-games/domino-game-run)
1062 #+end_src
1064 #+caption: floating dominos
1065 [[./images/dominos.jpg]]
1067 *** Hello Loop
1068 #+srcname: hello-loop
1069 #+begin_src clojure :results silent
1070 (ns hello.loop)
1071 (use 'cortex.world)
1072 (use 'cortex.import)
1073 (cortex.import/mega-import-jme3)
1074 (rlm.rlm-commands/help)
1076 (defn blue-cube []
1077 (box 1 1 1
1078 :color ColorRGBA/Blue
1079 :texture false
1080 :material "Common/MatDefs/Misc/Unshaded.j3md"
1081 :name "blue-cube"
1082 :physical? false))
1084 (defn blue-cube-game []
1085 (let [cube (blue-cube)
1086 root (doto (Node.) (.attachChild cube))]
1087 (world root
1088 {}
1089 no-op
1090 (fn [game tpf]
1091 (.rotate cube 0.0 (* 2 tpf) 0.0)))))
1092 #+end_src
1094 *** Hello Collision
1096 #+srcname: hello-collision
1097 #+begin_src clojure :results silent
1098 (ns hello.collision)
1099 (use 'cortex.world)
1100 (use 'cortex.import)
1101 (use 'clojure.contrib.def)
1104 (cortex.import/mega-import-jme3)
1105 (rlm.rlm-commands/help)
1106 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])
1109 (defn environment []
1110 (let
1111 [scene-model
1112 (doto
1113 (.loadModel
1114 (doto (asset-manager)
1115 (.registerLocator
1116 "/home/r/cortex/assets/zips/town.zip" ZipLocator))
1117 "main.scene")
1118 (.setLocalScale (float 2.0)))
1119 collision-shape
1120 (CollisionShapeFactory/createMeshShape #^Node scene-model)
1121 landscape (RigidBodyControl. collision-shape 0)]
1122 (.setShadowMode scene-model RenderQueue$ShadowMode/CastAndReceive)
1123 (.addControl scene-model landscape)
1124 scene-model))
1126 (defn player-fn []
1127 (doto
1128 (CharacterControl.
1129 (CapsuleCollisionShape. (float 1.5) (float 6)(float 1))
1130 (float 0.05))
1131 (.setJumpSpeed 20)
1132 (.setFallSpeed 30)
1133 (.setGravity 30) ;30
1134 (.setPhysicsLocation (Vector3f. 0 10 0))))
1136 (defn lights []
1137 [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 1 1 1) (float 1))))
1138 (doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 0.7 0 1) (float 1))))
1139 (doto (DirectionalLight.)
1140 (.setColor (.mult ColorRGBA/White (float 0.9) ))
1141 (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])
1143 (defn night-lights []
1144 [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 0.275 0.467 0.784 1) (float 0.3))))
1145 (doto (DirectionalLight.)
1146 (.setColor (.mult ColorRGBA/White (float 0.2) ))
1147 (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])
1149 (def player (atom (player-fn)))
1151 (defn setup-fn [game]
1152 (dorun (map #(.addLight (.getRootNode game) %) (lights)))
1153 ;; set the color of the sky
1154 (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))
1155 ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)
1156 (doto (.getFlyByCamera game)
1157 (.setMoveSpeed (float 100))
1158 (.setRotationSpeed 3))
1159 (.add
1160 (.getPhysicsSpace
1161 (.getState (.getStateManager game) BulletAppState))
1162 @player)
1164 (doto (Node.) (.attachChild (.getRootNode game))
1165 (.attachChild (brick-wall*))
1171 (def walking-up? (atom false))
1172 (def walking-down? (atom false))
1173 (def walking-left? (atom false))
1174 (def walking-right? (atom false))
1176 (defn set-walk [walk-atom game value]
1177 ;;(println-repl "setting stuff to " value)
1178 (reset! walk-atom value))
1180 (defn responses []
1181 {"key-w" (partial set-walk walking-up?)
1182 "key-d" (partial set-walk walking-right?)
1183 "key-s" (partial set-walk walking-down?)
1184 "key-a" (partial set-walk walking-left?)
1185 "key-return" (fire-cannon-ball)
1186 "key-space" (fn [game value] (.jump @player))
1187 })
1189 (defn update-fn
1190 [game tpf]
1191 (let [camera (.getCamera game)
1192 cam-dir (.multLocal
1193 (.clone
1194 (.getDirection camera)) (float 0.6))
1195 cam-left (.multLocal
1196 (.clone
1197 (.getLeft camera)) (float 0.4))
1198 walk-direction (Vector3f. 0 0 0)]
1200 (cond
1201 @walking-up? (.addLocal walk-direction cam-dir)
1202 @walking-right? (.addLocal walk-direction (.negate cam-left))
1203 @walking-down? (.addLocal walk-direction (.negate cam-dir))
1204 @walking-left? (.addLocal walk-direction cam-left))
1205 (.setWalkDirection @player walk-direction)
1206 (.setLocation camera (.getPhysicsLocation @player))))
1208 (defn run-game []
1209 (.start
1210 (world (environment)
1211 (responses)
1212 setup-fn
1213 update-fn)))
1214 #+end_src
1216 *** Hello Terrain
1217 #+srcname: hello-terrain
1218 #+begin_src clojure :results silent
1219 (ns hello.terrain)
1220 (use 'cortex.world)
1221 (use 'cortex.import)
1222 (use 'clojure.contrib.def)
1223 (import jme3tools.converters.ImageToAwt)
1226 (cortex.import/mega-import-jme3)
1227 (rlm.rlm-commands/help)
1228 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])
1231 (defn setup-fn [type game]
1232 (.setMoveSpeed (.getFlyByCamera game) 50)
1233 (.setFrustumFar (.getCamera game) 10000)
1234 (let [env (environment type)
1235 cameras [(.getCamera game)]
1236 control (TerrainLodControl. env cameras)]
1237 ;;(.addControl env control)
1238 (.attachChild (.getRootNode game) env)))
1240 (defn environment [type]
1241 (let
1242 [mat_terrain
1243 (Material. (asset-manager) "Common/MatDefs/Terrain/Terrain.j3md")
1244 grass (.loadTexture (asset-manager) "Textures/Terrain/splat/grass.jpg")
1245 dirt (.loadTexture (asset-manager) "Textures/Terrain/splat/dirt.jpg")
1246 rock (.loadTexture (asset-manager) "Textures/Terrain/splat/road.jpg")
1247 heightmap-image (.loadTexture (asset-manager)
1248 ({:mountain "Textures/Terrain/splat/mountains512.png"
1249 :fortress "Textures/Terrain/splat/fortress512.png"
1250 }type))
1251 heightmap (ImageBasedHeightMap.
1252 (ImageToAwt/convert (.getImage heightmap-image) false true 0))
1253 terrain (do (.load heightmap)
1254 (TerrainQuad. "my terrain" 65 513 (.getHeightMap heightmap)))
1257 (dorun (map #(.setWrap % Texture$WrapMode/Repeat)
1258 [grass dirt rock]))
1260 (doto mat_terrain
1261 (.setTexture "Tex1" grass)
1262 (.setFloat "Tex1Scale" (float 64))
1264 (.setTexture "Tex2" dirt)
1265 (.setFloat "Tex2Scale" (float 32))
1267 (.setTexture "Tex3" rock)
1268 (.setFloat "Tex3Scale" (float 128))
1270 (.setTexture "Alpha"
1271 (.loadTexture
1272 (asset-manager)
1273 ({:mountain "Textures/Terrain/splat/alphamap.png"
1274 :fortress "Textures/Terrain/splat/alphamap2.png"} type))))
1276 (doto terrain
1277 (.setMaterial mat_terrain)
1278 (.setLocalTranslation 0 -100 0)
1279 (.setLocalScale 2 1 2))))
1283 (defn run-terrain-game [type]
1284 (.start
1285 (world
1286 (Node.)
1287 {}
1288 (partial setup-fn type)
1289 no-op)))
1290 #+end_src
1294 #+srcname: hello-animation
1295 #+begin_src clojure :results silent
1296 (ns hello.animation)
1297 (use 'cortex.world)
1298 (use 'cortex.import)
1299 (use 'clojure.contrib.def)
1300 (cortex.import/mega-import-jme3)
1301 (rlm.rlm-commands/help)
1302 (use '[hello [collision :only [lights]]])
1304 (defn stand
1305 [channel]
1306 (doto channel
1307 (.setAnim "stand" (float 0.5))
1308 (.setLoopMode LoopMode/DontLoop)
1309 (.setSpeed (float 1))))
1311 (defn anim-listener []
1312 (proxy [AnimEventListener] []
1313 (onAnimChange
1314 [control channel animation-name]
1315 (println-repl "RLM --- onAnimChange"))
1316 (onAnimCycleDone
1317 [control channel animation-name]
1318 (if (= animation-name "Walk")
1319 (stand channel)
1320 ))))
1322 (defn setup-fn [channel game]
1323 (dorun (map #(.addLight (.getRootNode game) %) (lights)))
1324 ;; set the color of the sky
1325 (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))
1326 ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)
1327 (.setAnim channel "stand")
1328 (doto (.getFlyByCamera game)
1329 (.setMoveSpeed (float 10))
1330 (.setRotationSpeed 1)))
1332 (defn walk [channel]
1333 (println-repl "zzz")
1334 (doto channel
1335 (.setAnim "Walk" (float 0.5))
1336 (.setLoopMode LoopMode/Loop)))
1339 (defn key-map [channel]
1340 {"key-space" (fn [game value]
1341 (if (not value)
1342 (walk channel)))})
1344 (defn player []
1345 (let [model (.loadModel (asset-manager) "Models/Oto/Oto.mesh.xml")
1346 control (.getControl model AnimControl)]
1347 (.setLocalScale model (float 0.5))
1348 (.clearListeners control)
1349 (.addListener control (anim-control))
1350 model))
1354 (defn run-anim-game []
1355 (let [ninja (player)
1356 control (.getControl ninja AnimControl)
1357 channel (.createChannel control)]
1358 (.start
1359 (world
1360 ninja
1361 (key-map channel)
1362 (partial setup-fn channel)
1363 no-op))))
1364 #+end_src
1366 *** Hello Materials
1367 #+srcname: material
1368 #+begin_src clojure :results silent
1369 (ns hello.material)
1370 (use 'cortex.world)
1371 (use 'cortex.import)
1372 (use 'clojure.contrib.def)
1373 (cortex.import/mega-import-jme3)
1374 (rlm.rlm-commands/help)
1376 (defn simple-cube []
1377 (box 1 1 1
1378 :position (Vector3f. -3 1.1 0)
1379 :material "Common/MatDefs/Misc/Unshaded.j3md"
1380 :texture "Interface/Logo/Monkey.jpg"
1381 :physical? false))
1383 (defn leaky-box []
1384 (box 1 1 1
1385 :position (Vector3f. 3 -1 0)
1386 :material "Common/MatDefs/Misc/ColoredTextured.j3md"
1387 :texture "Textures/ColoredTex/Monkey.png"
1388 :color (ColorRGBA. 1 0 1 1)
1389 :physical? false))
1391 (defn transparent-box []
1392 (doto
1393 (box 1 1 0.1
1394 :position Vector3f/ZERO
1395 :name "window frame"
1396 :material "Common/MatDefs/Misc/Unshaded.j3md"
1397 :texture "Textures/ColoredTex/Monkey.png"
1398 :physical? false)
1399 (-> (.getMaterial)
1400 (.getAdditionalRenderState)
1401 (.setBlendMode RenderState$BlendMode/Alpha))
1402 (.setQueueBucket RenderQueue$Bucket/Transparent)))
1404 (defn bumpy-sphere []
1405 (doto
1406 (sphere 2
1407 :position (Vector3f. 0 2 -2)
1408 :name "Shiny rock"
1409 :material "Common/MatDefs/Light/Lighting.j3md"
1410 :texture false
1411 :physical? false)
1412 (-> (.getMesh)
1413 (doto
1414 (.setTextureMode Sphere$TextureMode/Projected)
1415 (TangentBinormalGenerator/generate)))
1416 (-> (.getMaterial)
1417 (doto
1418 (.setTexture "DiffuseMap" (.loadTexture (asset-manager)
1419 "Textures/Terrain/Pond/Pond.png"))
1420 (.setTexture "NormalMap" (.loadTexture (asset-manager)
1421 "Textures/Terrain/Pond/Pond_normal.png"))
1422 (.setFloat "Shininess" (float 5))))
1423 (.rotate (float 1.6) 0 0)))
1426 (defn start-game []
1427 (.start
1428 (world
1429 (let [root (Node.)]
1430 (dorun (map #(.attachChild root %)
1431 [(simple-cube) (leaky-box) (transparent-box) (bumpy-sphere)]))
1432 root)
1433 {}
1434 (fn [world]
1435 (let [sun (doto (DirectionalLight.)
1436 (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))
1437 (.setColor ColorRGBA/White))]
1438 (.addLight (.getRootNode world) sun)))
1439 no-op
1440 )))
1441 #+end_src
1445 * The Body
1446 ** Eyes
1448 Ultimately I want to make creatures with eyes. Each eye can be
1449 independely moved and should see its own version of the world
1450 depending on where it is.
1451 #+srcname: eyes
1452 #+begin_src clojure
1453 (ns body.eye)
1454 (use 'cortex.world)
1455 (use 'cortex.import)
1456 (use 'clojure.contrib.def)
1457 (cortex.import/mega-import-jme3)
1458 (rlm.rlm-commands/help)
1459 (import java.nio.ByteBuffer)
1460 (import java.awt.image.BufferedImage)
1461 (import java.awt.Color)
1462 (import java.awt.Dimension)
1463 (import java.awt.Graphics)
1464 (import java.awt.Graphics2D)
1465 (import java.awt.event.WindowAdapter)
1466 (import java.awt.event.WindowEvent)
1467 (import java.awt.image.BufferedImage)
1468 (import java.nio.ByteBuffer)
1469 (import javax.swing.JFrame)
1470 (import javax.swing.JPanel)
1471 (import javax.swing.SwingUtilities)
1472 (import javax.swing.ImageIcon)
1473 (import javax.swing.JOptionPane)
1474 (import java.awt.image.ImageObserver)
1478 (defn scene-processor
1479 "deals with converting FrameBuffers to BufferedImages so
1480 that the continuation function can be defined only in terms
1481 of what it does with BufferedImages"
1482 [continuation]
1483 (let [byte-buffer (atom nil)
1484 renderer (atom nil)
1485 image (atom nil)]
1486 (proxy [SceneProcessor] []
1487 (initialize
1488 [renderManager viewPort]
1489 (let [cam (.getCamera viewPort)
1490 width (.getWidth cam)
1491 height (.getHeight cam)]
1492 (reset! renderer (.getRenderer renderManager))
1493 (reset! byte-buffer
1494 (BufferUtils/createByteBuffer
1495 (* width height 4)))
1496 (reset! image (BufferedImage. width height
1497 BufferedImage/TYPE_4BYTE_ABGR))))
1498 (isInitialized [] (not (nil? @byte-buffer)))
1499 (reshape [_ _ _])
1500 (preFrame [_])
1501 (postQueue [_])
1502 (postFrame
1503 [#^FrameBuffer fb]
1504 (.clear @byte-buffer)
1505 (.readFrameBuffer @renderer fb @byte-buffer)
1506 (Screenshots/convertScreenShot @byte-buffer @image)
1507 (continuation @image))
1508 (cleanup []))))
1510 (defn add-eye
1511 "Add an eye to the world, and call continuation on
1512 every frame produced"
1513 [world camera continuation]
1514 (let [width (.getWidth camera)
1515 height (.getHeight camera)
1516 render-manager (.getRenderManager world)
1517 viewport (.createMainView render-manager "eye-view" camera)]
1518 (doto viewport
1519 (.setBackgroundColor ColorRGBA/Black)
1520 (.setClearFlags true true true)
1521 (.addProcessor (scene-processor continuation))
1522 (.attachScene (.getRootNode world)))))
1524 (defn make-display-frame [display width height]
1525 (SwingUtilities/invokeLater
1526 (fn []
1527 (.setPreferredSize display (Dimension. width height))
1528 (doto (JFrame. "Eye Camera!")
1529 (-> (.getContentPane) (.add display))
1530 (.setDefaultCloseOperation JFrame/DISPOSE_ON_CLOSE)
1531 (.pack)
1532 (.setLocationRelativeTo nil)
1533 (.setResizable false)
1534 (.setVisible true)))))
1536 (defn image-monitor [#^BufferedImage image]
1537 (proxy [JPanel] []
1538 (paintComponent
1539 [g]
1540 (proxy-super paintComponent g)
1541 (locking image
1542 (.drawImage g image 0 0
1543 (proxy [ImageObserver]
1544 []
1545 (imageUpdate
1546 []
1547 (proxy-super imageUpdate))))))))
1549 (defn movie-image []
1550 (let [setup
1551 (runonce
1552 (fn [#^BufferedImage image]
1553 (let [width (.getWidth image)
1554 height (.getHeight image)
1555 display (image-monitor image)
1556 frame (make-display-frame display width height)]
1557 display)))]
1558 (fn [#^BufferedImage image]
1559 (.repaint (setup image)))))
1562 (defn observer
1563 "place thy eye!"
1564 [world camera]
1565 (let [eye camera
1566 width (.getWidth eye)
1567 height (.getHeight eye)]
1568 (no-exceptions
1569 (add-eye
1570 world
1571 eye
1572 (movie-image)))))
1573 #+end_src
1575 #+srcname: test-vision
1576 #+begin_src clojure
1578 (ns test.vision)
1579 (use 'cortex.world)
1580 (use 'cortex.import)
1581 (use 'clojure.contrib.def)
1582 (use 'body.eye)
1583 (cortex.import/mega-import-jme3)
1584 (rlm.rlm-commands/help)
1585 (import java.nio.ByteBuffer)
1586 (import java.awt.image.BufferedImage)
1587 (import java.awt.Color)
1588 (import java.awt.Dimension)
1589 (import java.awt.Graphics)
1590 (import java.awt.Graphics2D)
1591 (import java.awt.event.WindowAdapter)
1592 (import java.awt.event.WindowEvent)
1593 (import java.awt.image.BufferedImage)
1594 (import java.nio.ByteBuffer)
1595 (import javax.swing.JFrame)
1596 (import javax.swing.JPanel)
1597 (import javax.swing.SwingUtilities)
1598 (import javax.swing.ImageIcon)
1599 (import javax.swing.JOptionPane)
1600 (import java.awt.image.ImageObserver)
1603 (def width 200)
1604 (def height 200)
1606 (defn camera []
1607 (doto (Camera. width height)
1608 (.setFrustumPerspective 45 1 1 1000)
1609 (.setLocation (Vector3f. -3 0 -5))
1610 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1612 (defn camera2 []
1613 (doto (Camera. width height)
1614 (.setFrustumPerspective 45 1 1 1000)
1615 (.setLocation (Vector3f. 3 0 -5))
1616 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1618 (defn setup-fn [world]
1619 (let [eye (camera)
1620 width (.getWidth eye)
1621 height (.getHeight eye)]
1622 (no-exceptions
1623 (add-eye
1624 world
1625 eye
1626 (runonce visual))
1627 (add-eye
1628 world
1629 (camera2)
1630 (runonce visual)))))
1632 (defn spider-eye [position]
1633 (doto (Camera. 200 200 )
1634 (.setFrustumPerspective 45 1 1 1000)
1635 (.setLocation position)
1636 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1638 (defn setup-fn* [world]
1639 (let [eye (camera)
1640 width (.getWidth eye)
1641 height (.getHeight eye)]
1642 ;;(.setClearFlags (.getViewPort world) true true true)
1643 (observer world (.getCamera world))
1644 (observer world (spider-eye (Vector3f. 3 0 -5)))
1645 ;;(observer world (spider-eye (Vector3f. 0 0 -5)))
1646 ;; (observer world (spider-eye (Vector3f. -3 0 -5)))
1647 ;; (observer world (spider-eye (Vector3f. 0 3 -5)))
1648 ;; (observer world (spider-eye (Vector3f. 0 -3 -5)))
1649 ;; (observer world (spider-eye (Vector3f. 3 3 -5)))
1650 ;; (observer world (spider-eye (Vector3f. -3 3 -5)))
1651 ;; (observer world (spider-eye (Vector3f. 3 -3 -5)))
1652 ;; (observer world (spider-eye (Vector3f. -3 -3 -5)))
1655 world)
1657 (defn test-world []
1658 (let [thing (box 1 1 1 :physical? false)]
1659 (world
1660 (doto (Node.)
1661 (.attachChild thing))
1662 {}
1663 setup-fn
1664 (fn [world tpf]
1665 (.rotate thing (* tpf 0.2) 0 0)
1666 ))))
1669 #+end_src
1672 #+results: eyes
1673 : #'body.eye/test-world
1675 Note the use of continuation passing style for connecting the eye to a
1676 function to process the output. The example code will create two
1677 videos of the same rotating cube from different angles, sutiable for
1678 stereoscopic vision.
1685 * COMMENT code generation
1686 #+begin_src clojure :tangle ../src/cortex/import.clj
1687 <<import>>
1688 #+end_src
1690 #+begin_src clojure :tangle ../src/hello/brick_wall.clj
1691 <<brick-wall-header>>
1692 <<brick-wall-body>>
1693 #+end_src
1695 #+begin_src clojure :tangle ../src/hello/hello_simple_app.clj
1696 <<hello-simple-app>>
1697 #+end_src
1699 #+begin_src clojure :tangle ../src/cortex/world.clj
1700 <<world-inputs>>
1701 <<world>>
1702 <<world-shapes>>
1703 <<world-view>>
1704 #+end_src
1706 #+begin_src clojure :tangle ../src/cortex/other_games.clj
1707 <<other-games>>
1708 #+end_src
1710 #+begin_src clojure :tangle ../src/hello/loop.clj
1711 <<hello-loop>>
1712 #+end_src
1714 #+begin_src clojure :tangle ../src/hello/collision.clj
1715 <<hello-collision>>
1716 #+end_src
1718 #+begin_src clojure :tangle ../src/hello/terrain.clj
1719 <<hello-terrain>>
1720 #+end_src
1722 #+begin_src clojure :tangle ../src/hello/animation.clj
1723 <<hello-animation>>
1724 #+end_src
1726 #+begin_src clojure :tangle ../src/hello/material.clj
1727 <<material>>
1728 #+end_src
1730 #+begin_src clojure :tangle ../src/body/eye.clj
1731 <<eyes>>
1732 #+end_src
1734 #+begin_src clojure :tangle ../src/test/vision.clj
1735 <<test-vision>>
1736 #+end_src