view org/cortex.org @ 20:67d508a1e34d

going to update website/documentation
author Robert McIntyre <rlm@mit.edu>
date Sun, 23 Oct 2011 12:32:56 -0700
parents c32f3eb9fdeb
children 01e1427126af
line wrap: on
line source
1 #+title: Simulated Senses
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+description: Simulating senses for AI research using JMonkeyEngine3
5 #+SETUPFILE: ../../aurellem/org/setup.org
6 #+INCLUDE: ../../aurellem/org/level-0.org
7 #+babel: :mkdirp yes :noweb yes
9 * Background
10 Artificial Intelligence has tried and failed for more than half a
11 century to produce programs as flexible, creative, and “intelligent”
12 as the human mind itself. Clearly, we are still missing some important
13 ideas concerning intelligent programs or we would have strong AI
14 already. What idea could be missing?
16 When Turing first proposed his famous “Turing Test” in the
17 groundbreaking paper [[./sources/turing.pdf][/Computing Machines and Intelligence/]], he gave
18 little importance to how a computer program might interact with the
19 world:
21 #+BEGIN_QUOTE
22 \ldquo{}We need not be too concerned about the legs, eyes, etc. The example of
23 Miss Helen Keller shows that education can take place provided that
24 communication in both directions between teacher and pupil can take
25 place by some means or other.\rdquo{}
26 #+END_QUOTE
28 And from the example of Hellen Keller he went on to assume that the
29 only thing a fledgling AI program could need by way of communication
30 is a teletypewriter. But Hellen Keller did possess vision and hearing
31 for the first few months of her life, and her tactile sense was far
32 more rich than any text-stream could hope to achieve. She possessed a
33 body she could move freely, and had continual access to the real world
34 to learn from her actions.
36 I believe that our programs are suffering from too little sensory
37 input to become really intelligent. Imagine for a moment that you
38 lived in a world completely cut off form all sensory stimulation. You
39 have no eyes to see, no ears to hear, no mouth to speak. No body, no
40 taste, no feeling whatsoever. The only sense you get at all is a
41 single point of light, flickering on and off in the void. If this was
42 your life from birth, you would never learn anything, and could never
43 become intelligent. Actual humans placed in sensory deprivation
44 chambers experience hallucinations and can begin to loose their sense
45 of reality in as little as 15 minutes[sensory-deprivation]. Most of
46 the time, the programs we write are in exactly this situation. They do
47 not interface with cameras and microphones, and they do not control a
48 real or simulated body or interact with any sort of world.
51 * Simulation vs. Reality
52 I want demonstrate that multiple senses are what enable
53 intelligence. There are two ways of playing around with senses and
54 computer programs:
56 The first is to go entirely with simulation: virtual world, virtual
57 character, virtual senses. The advantages are that when everything is
58 a simulation, experiments in that simulation are absolutely
59 reproducible. It's also easier to change the character and world to
60 explore new situations and different sensory combinations.
63 ** Issues with Simulation
65 If the world is to be simulated on a computer, then not only do you
66 have to worry about whether the character's senses are rich enough to
67 learn from the world, but whether the world itself is rendered with
68 enough detail and realism to give enough working material to the
69 character's senses. To name just a few difficulties facing modern
70 physics simulators: destructibility of the environment, simulation of
71 water/other fluids, large areas, nonrigid bodies, lots of objects,
72 smoke. I don't know of any computer simulation that would allow a
73 character to take a rock and grind it into fine dust, then use that
74 dust to make a clay sculpture, at least not without spending years
75 calculating the interactions of every single small grain of
76 dust. Maybe a simulated world with today's limitations doesn't provide
77 enough richness for real intelligence to evolve.
79 ** Issues with Reality
81 The other approach for playing with senses is to hook your software up
82 to real cameras, microphones, robots, etc., and let it loose in the
83 real world. This has the advantage of eliminating concerns about
84 simulating the world at the expense of increasing the complexity of
85 implementing the senses. Instead of just grabbing the current rendered
86 frame for processing, you have to use an actual camera with real
87 lenses and interact with photons to get an image. It is much harder to
88 change the character, which is now partly a physical robot of some
89 sort, since doing so involves changing things around in the real world
90 instead of modifying lines of code. While the real world is very rich
91 and definitely provides enough stimulation for intelligence to develop
92 as evidenced by our own existence, it is also uncontrollable in the
93 sense that a particular situation cannot be recreated perfectly or
94 saved for later use. It is harder to conduct science because it is
95 harder to repeat an experiment. The worst thing about using the real
96 world instead of a simulation is the matter of time. Instead of
97 simulated time you get the constant and unstoppable flow of real
98 time. This severely limits the sorts of software you can use to
99 program the AI because all sense inputs must be handled in real
100 time. Complicated ideas may have to be implemented in hardware or may
101 simply be impossible given the current speed of our
102 processors. Contrast this with a simulation, in which the flow of time
103 in the simulated world can be slowed down to accommodate the
104 limitations of the character's programming. In terms of cost, doing
105 everything in software is far cheaper than building custom real-time
106 hardware. All you need is a laptop and some patience.
108 * Choose a Simulation Engine
110 Mainly because of issues with controlling the flow of time, I chose to
111 simulate both the world and the character. I set out to make a minimal
112 world in which I could embed a character with multiple senses. My main
113 goal is to make an environment where I can perform further experiments
114 in simulated senses.
116 As Carl Sagan once said, "If you wish to make an apple pie from
117 scratch, you must first invent the universe.” I examined many
118 different 3D environments to try and find something I would use as the
119 base for my simulation; eventually the choice came down to three
120 engines: the Quake II engine, the Source Engine, and jMonkeyEngine.
122 ** Quake II/Jake2
124 I spent a bit more than a month working with the Quake II Engine from
125 ID software to see if I could use it for my purposes. All the source
126 code was released by ID software into the Public Domain several years
127 ago, and as a result it has been ported and modified for many
128 different reasons. This engine was famous for its advanced use of
129 realistic shading and had decent and fast physics
130 simulation. Researchers at Princeton [[http://www.nature.com/nature/journal/v461/n7266/pdf/nature08499.pdf][used this code]] to study spatial
131 information encoding in the hippocampal cells of rats. Those
132 researchers created a special Quake II level that simulated a maze,
133 and added an interface where a mouse could run around inside a ball in
134 various directions to move the character in the simulated maze. They
135 measured hippocampal activity during this exercise to try and tease
136 out the method in which spatial data was stored in that area of the
137 brain. I find this promising because if a real living rat can interact
138 with a computer simulation of a maze in the same way as it interacts
139 with a real-world maze, then maybe that simulation is close enough to
140 reality that a simulated sense of vision and motor control interacting
141 with that simulation could reveal useful information about the real
142 thing. It happens that there is a Java port of the original C source
143 code called Jake2. The port demonstrates Java's OpenGL bindings and
144 runs anywhere from 90% to 105% as fast as the C version. After
145 reviewing much of the source of Jake2, I eventually rejected it
146 because the engine is too tied to the concept of a first-person
147 shooter game. One of the problems I had was that there does not seem
148 to be any easy way to attach multiple cameras to a single
149 character. There are also several physics clipping issues that are
150 corrected in a way that only applies to the main character and does
151 not apply to arbitrary objects. While there is a large community of
152 level modders, I couldn't find a community to support using the engine
153 to make new things.
155 ** Source Engine
157 The Source Engine evolved from the Quake II and Quake I engines and is
158 used by Valve in the Half-Life series of games. The physics simulation
159 in the Source Engine is quite accurate and probably the best out of
160 all the engines I investigated. There is also an extensive community
161 actively working with the engine. However, applications that use the
162 Source Engine must be written in C++, the code is not open, it only
163 runs on Windows, and the tools that come with the SDK to handle models
164 and textures are complicated and awkward to use.
166 ** jMonkeyEngine
168 jMonkeyEngine is a new library for creating games in Java. It uses
169 OpenGL to render to the screen and uses screengraphs to avoid drawing
170 things that do not appear on the screen. It has an active community
171 and several games in the pipeline. The engine was not built to serve
172 any particular game but is instead meant to be used for any 3D
173 game. After experimenting with each of these three engines and a few
174 others for about 2 months I settled on jMonkeyEngine. I chose it
175 because it had the most features out of all the open projects I looked
176 at, and because I could then write my code in Clojure, an
177 implementation of LISP that runs on the JVM.
179 * Setup
181 First, I checked out the source to jMonkeyEngine:
183 #+srcname: checkout
184 #+begin_src sh :results verbatim
185 svn checkout http://jmonkeyengine.googlecode.com/svn/trunk/engine jme3
186 #+end_src
188 #+results: checkout
189 : Checked out revision 7975.
192 Building jMonkeyEngine is easy enough:
194 #+srcname: build
195 #+begin_src sh :results verbatim
196 cd jme3
197 ant jar | tail -n 2
198 #+end_src
200 #+results: build
201 : BUILD SUCCESSFUL
202 : Total time: 15 seconds
205 Also build the javadoc:
207 #+srcname: javadoc
208 #+begin_src sh :results verbatim
209 cd jme3
210 ant javadoc | tail -n 2
211 #+end_src
213 #+results: javadoc
214 : BUILD SUCCESSFUL
215 : Total time: 12 seconds
217 Now, move the jars from the compilation into the project's lib folder.
219 #+srcname: move-jars
220 #+begin_src sh :results verbatim
221 mkdir -p lib
222 mkdir -p src
223 cp jme3/dist/jMonkeyEngine3.jar lib/
224 cp jme3/dist/lib/* lib/
225 ls lib
226 #+end_src
228 #+results: move-jars
229 #+begin_example
230 eventbus-1.4.jar
231 jbullet.jar
232 jheora-jst-debug-0.6.0.jar
233 jinput.jar
234 jME3-jbullet.jar
235 jME3-lwjgl-natives.jar
236 jME3-testdata.jar
237 jME3-test.jar
238 jMonkeyEngine3.jar
239 j-ogg-oggd.jar
240 j-ogg-vorbisd.jar
241 lwjgl.jar
242 nifty-1.3.jar
243 nifty-default-controls-1.3.jar
244 nifty-examples-1.3.jar
245 nifty-lwjgl-renderer-1.3.jar
246 nifty-openal-soundsystem-1.0.jar
247 nifty-style-black-1.3.jar
248 nifty-style-grey-1.0.jar
249 noise-0.0.1-SNAPSHOT.jar
250 stack-alloc.jar
251 vecmath.jar
252 xmlpull-xpp3-1.1.4c.jar
253 #+end_example
255 It's good to create a =assets= directory in the style that the
256 =AssetManager= will like.
258 #+srcname: create-assets
259 #+begin_src sh :results verbatim
260 mkdir -p assets
261 mkdir -p assets/Interface
262 mkdir -p assets/Materials
263 mkdir -p assets/MatDefs
264 mkdir -p assets/Models
265 mkdir -p assets/Scenes
266 mkdir -p assets/Shaders
267 mkdir -p assets/Sounds
268 mkdir -p assets/Textures
269 tree -L 1 assets
270 #+end_src
272 #+results: create-assets
273 #+begin_example
274 assets
275 |-- Interface
276 |-- MatDefs
277 |-- Materials
278 |-- Models
279 |-- Scenes
280 |-- Shaders
281 |-- Sounds
282 `-- Textures
284 8 directories, 0 files
285 #+end_example
288 The java classpath should have all the jars contained in the =lib=
289 directory as well as the src directory.
291 For example, here is the file I use to run my REPL for clojure.
293 #+include: "~/swank-all" src sh :exports code
295 The important thing here is that =cortex/lib/*=, =cortex/src=, and
296 =cortex/assets= appear on the classpath. (=cortex= is the base
297 directory of this project.)
299 #+srcname: pwd
300 #+begin_src sh
301 pwd
302 #+end_src
304 #+results: pwd
305 : /home/r/cortex
308 * Simulation Base
310 ** Imports
311 First, I'll import jme core classes.
312 #+srcname: import
313 #+begin_src clojure :results silent
314 (ns cortex.import
315 (:require swank.util.class-browse))
317 (defn import-jme3 []
318 (import '[com.jme3.system AppSettings JmeSystem])
319 (import '[com.jme3.app Application SimpleApplication])
320 (import 'com.jme3.material.Material)
321 (import '[com.jme3.math Vector3f ColorRGBA Quaternion Transform])
322 (import '[com.jme3.scene Node Geometry])
323 (import '[com.jme3.scene.shape Box Sphere Sphere$TextureMode])
324 (import 'com.jme3.font.BitmapText)
325 (import '[com.jme3.input KeyInput InputManager])
326 (import '[com.jme3.input.controls
327 ActionListener AnalogListener KeyTrigger MouseButtonTrigger])
328 (import '[com.jme3.asset AssetManager DesktopAssetManager] )
329 (import '[com.jme3.asset.plugins HttpZipLocator ZipLocator])
330 (import '[com.jme3.light PointLight DirectionalLight])
331 (import '[com.jme3.animation AnimControl Skeleton Bone])
332 (import '[com.jme3.bullet.collision.shapes
333 MeshCollisionShape SphereCollisionShape BoxCollisionShape])
334 (import 'com.jme3.renderer.queue.RenderQueue$ShadowMode)
335 (import 'jme3test.TestChooser)
336 (import '[com.jme3.bullet PhysicsTickListener PhysicsSpace])
337 (import '[com.jme3.bullet.joints SixDofJoint HingeJoint
338 SliderJoint Point2PointJoint ConeJoint]))
341 (defmacro permissive-import* [class-symbol]
342 `(try
343 (import ~class-symbol)
344 (catch Exception e#
345 (println "can't import " ~class-symbol))))
347 (defn permissive-import [class-symbol]
348 (eval (list 'cortex.import/permissive-import* class-symbol)))
350 (defn selection-import [selection-fn]
351 (dorun
352 (map (comp permissive-import symbol)
353 (filter selection-fn
354 (map :name
355 swank.util.class-browse/available-classes)))))
357 (defn mega-import-jme3
358 "ALL the jme classes. For REPL use."
359 []
360 (selection-import
361 #(and
362 (.startsWith % "com.jme3.")
363 ;; Don't import the Lwjgl stuff since it can throw exceptions
364 ;; upon being loaded.
365 (not (re-matches #".*Lwjgl.*" %)))))
366 #+end_src
368 The =mega-import-jme3= is quite usefull for debugging purposes since
369 it allows completion for almost all of JME's classes
371 ** Simplification
372 *** World
374 It is comvienent to wrap the JME elements that deal with creating a
375 world, creation of basic objects, and Keyboard input with a nicer
376 interface (at least for my purposes).
378 #+srcname: world-inputs
379 #+begin_src clojure :results silent
380 (ns cortex.world)
381 (require 'cortex.import)
382 (use 'clojure.contrib.def)
383 (rlm.rlm-commands/help)
384 (cortex.import/mega-import-jme3)
386 (defvar *app-settings*
387 (doto (AppSettings. true)
388 (.setFullscreen false)
389 (.setTitle "Aurellem.")
390 ;; disable 32 bit stuff for now
391 ;;(.setAudioRenderer "Send")
392 )
393 "These settings control how the game is displayed on the screen for
394 debugging purposes. Use binding forms to change this if desired.
395 Full-screen mode does not work on some computers.")
397 (defn asset-manager
398 "returns a new, configured assetManager" []
399 (JmeSystem/newAssetManager
400 (.getResource
401 (.getContextClassLoader (Thread/currentThread))
402 "com/jme3/asset/Desktop.cfg")))
404 (defmacro no-exceptions
405 "Sweet relief like I never knew."
406 [& forms]
407 `(try ~@forms (catch Exception e# (.printStackTrace e#))))
409 (defn thread-exception-removal []
410 (println "removing exceptions from " (Thread/currentThread))
411 (.setUncaughtExceptionHandler
412 (Thread/currentThread)
413 (proxy [Thread$UncaughtExceptionHandler] []
414 (uncaughtException
415 [thread thrown]
416 (println "uncaught-exception thrown in " thread)
417 (println (.getMessage thrown))))))
419 (def println-repl (bound-fn [& args] (apply println args)))
421 (use '[pokemon [lpsolve :only [constant-map]]])
423 (defn no-op [& _])
425 (defn all-keys
426 "Construct a map of strings representing all the manual inputs from
427 either the keyboard or mouse."
428 []
429 (let [inputs (constant-map KeyInput)]
430 (assoc
431 (zipmap (map (fn [field]
432 (.toLowerCase (re-gsub #"_" "-" field))) (vals inputs))
433 (map (fn [val] (KeyTrigger. val)) (keys inputs)))
434 ;;explicitly add mouse controls
435 "mouse-left" (MouseButtonTrigger. 0)
436 "mouse-middle" (MouseButtonTrigger. 2)
437 "mouse-right" (MouseButtonTrigger. 1))))
439 (defn initialize-inputs
440 "more java-interop cruft to establish keybindings for a particular virtual world"
441 [game input-manager key-map]
442 (doall (map (fn [[name trigger]]
443 (.addMapping ^InputManager input-manager
444 name (into-array (class trigger) [trigger]))) key-map))
445 (doall (map (fn [name]
446 (.addListener ^InputManager input-manager game
447 (into-array String [name]))) (keys key-map))))
449 #+end_src
451 These functions are all for debug controlling of the world through
452 keyboard and mouse.
454 We reuse =constant-map= from =pokemon.lpsolve= to get the numerical
455 values for all the keys defined in the =KeyInput= class. The
456 documentation for =constant-map= is:
458 #+begin_src clojure :results output
459 (doc pokemon.lpsolve/constant-map)
460 #+end_src
462 #+results:
463 : -------------------------
464 : pokemon.lpsolve/constant-map
465 : ([class])
466 : Takes a class and creates a map of the static constant integer
467 : fields with their names. This helps with C wrappers where they have
468 : just defined a bunch of integer constants instead of enums
471 Then, =all-keys= converts the constant names like =KEY_J= to the more
472 clojure-like =key-j=, and returns a map from these keys to
473 jMonkeyEngine KeyTrigger objects, the use of which will soon become
474 apparent. =all-keys= also adds the three mouse button controls to the
475 map.
477 #+srcname: world
478 #+begin_src clojure :results silent
479 (in-ns 'cortex.world)
481 (defn traverse
482 "apply f to every non-node, deeply"
483 [f node]
484 (if (isa? (class node) Node)
485 (dorun (map (partial traverse f) (.getChildren node)))
486 (f node)))
488 (def gravity (Vector3f. 0 -9.81 0))
490 (defn world
491 [root-node key-map setup-fn update-fn]
492 (let [physics-manager (BulletAppState.)
493 shadow-renderer (BasicShadowRenderer. (asset-manager) (int 256))
494 ;;maybe use a better shadow renderer someday!
495 ;;shadow-renderer (PssmShadowRenderer. (asset-manager) 256 1)
496 ]
497 (doto
498 (proxy [SimpleApplication ActionListener] []
499 (simpleInitApp
500 []
501 (no-exceptions
502 (.setTimer this (IsoTimer. 60))
503 ;; Create key-map.
504 (.setFrustumFar (.getCamera this) 300)
505 (initialize-inputs this (.getInputManager this) (all-keys))
506 ;; Don't take control of the mouse
507 (org.lwjgl.input.Mouse/setGrabbed false)
508 ;; add all objects to the world
509 (.attachChild (.getRootNode this) root-node)
510 ;; enable physics
511 ;; add a physics manager
512 (.attach (.getStateManager this) physics-manager)
513 (.setGravity (.getPhysicsSpace physics-manager) gravity)
516 ;; go through every object and add it to the physics manager
517 ;; if relavant.
518 (traverse (fn [geom]
519 (dorun
520 (for [n (range (.getNumControls geom))]
521 (do
522 (println-repl "adding control " (.getControl geom n))
523 (.add (.getPhysicsSpace physics-manager)
524 (.getControl geom n))))))
525 (.getRootNode this))
526 ;;(.addAll (.getPhysicsSpace physics-manager) (.getRootNode this))
528 (setup-fn this)
529 (.setDirection shadow-renderer
530 (.normalizeLocal (Vector3f. -1 -1 -1)))
531 (.addProcessor (.getViewPort this) shadow-renderer)
532 (.setShadowMode (.getRootNode this) RenderQueue$ShadowMode/Off)
533 ))
534 (simpleUpdate
535 [tpf]
536 (no-exceptions
537 (update-fn this tpf)))
538 (onAction
539 [binding value tpf]
540 ;; whenever a key is pressed, call the function returned from
541 ;; key-map.
542 (no-exceptions
543 (if-let [react (key-map binding)]
544 (react this value)))))
545 ;; don't show a menu to change options.
547 (.setShowSettings false)
548 (.setPauseOnLostFocus false)
549 (.setSettings *app-settings*))))
551 (defn apply-map
552 "Like apply, but works for maps and functions that expect an implicit map
553 and nothing else as in (fn [& {}]).
554 ------- Example -------
555 (defn jjj [& {:keys [www] :or {www \"oh yeah\"} :as env}] (println www))
556 (apply-map jjj {:www \"whatever\"})
557 -->\"whatever\""
558 [fn m]
559 (apply fn (reduce #(into %1 %2) [] m)))
561 #+end_src
564 =world= is the most important function here.
565 *** TODO more documentation
567 #+srcname: world-shapes
568 #+begin_src clojure :results silent
569 (in-ns 'cortex.world)
570 (defrecord shape-description
571 [name
572 color
573 mass
574 friction
575 texture
576 material
577 position
578 rotation
579 shape
580 physical?])
582 (def base-shape
583 (shape-description.
584 "default-shape"
585 false
586 ;;ColorRGBA/Blue
587 1.0 ;; mass
588 1.0 ;; friction
589 ;; texture
590 "Textures/Terrain/BrickWall/BrickWall.jpg"
591 ;; material
592 "Common/MatDefs/Misc/Unshaded.j3md"
593 Vector3f/ZERO
594 Quaternion/IDENTITY
595 (Box. Vector3f/ZERO 0.5 0.5 0.5)
596 true))
598 (defn make-shape
599 [#^shape-description d]
600 (let [asset-manager (if (:asset-manager d) (:asset-manager d) (asset-manager))
601 mat (Material. asset-manager (:material d))
602 geom (Geometry. (:name d) (:shape d))]
603 (if (:texture d)
604 (let [key (TextureKey. (:texture d))]
605 (.setGenerateMips key true)
606 (.setTexture mat "ColorMap" (.loadTexture asset-manager key))))
607 (if (:color d) (.setColor mat "Color" (:color d)))
608 (.setMaterial geom mat)
609 (if-let [rotation (:rotation d)] (.rotate geom rotation))
610 (.setLocalTranslation geom (:position d))
611 (if (:physical? d)
612 (let [impact-shape (doto (GImpactCollisionShape.
613 (.getMesh geom)) (.setMargin 0))
614 physics-control (RigidBodyControl.
615 ;;impact-shape ;; comment to disable
616 (float (:mass d)))]
617 (.createJmeMesh impact-shape)
618 (.addControl geom physics-control)
619 ;;(.setSleepingThresholds physics-control (float 0) (float 0))
620 (.setFriction physics-control (:friction d))))
621 ;;the default is to keep this node in the physics engine forever.
622 ;;these commands must come after the control is added to the geometry.
623 ;;
624 geom))
626 (defn box
627 ([l w h & {:as options}]
628 (let [options (merge base-shape options)]
629 (make-shape (assoc options
630 :shape (Box. l w h)))))
631 ([] (box 0.5 0.5 0.5)))
633 (defn sphere
634 ([r & {:as options}]
635 (let [options (merge base-shape options)]
636 (make-shape (assoc options
637 :shape (Sphere. 32 32 (float r))))))
638 ([] (sphere 0.5)))
640 (defn add-element
641 ([game element node]
642 (.addAll
643 (.getPhysicsSpace
644 (.getState
645 (.getStateManager game)
646 BulletAppState))
647 element)
648 (.attachChild node element))
649 ([game element]
650 (add-element game element (.getRootNode game))))
653 (defn set-gravity*
654 [game gravity]
655 (traverse
656 (fn [geom]
657 (if-let
658 [control (.getControl geom RigidBodyControl)]
659 (do
660 (.setGravity control gravity)
661 (.applyImpulse control Vector3f/ZERO Vector3f/ZERO)
662 )))
663 (.getRootNode game)))
665 #+end_src
667 These are convienence functions for creating JME objects and
668 manipulating a world.
670 #+srcname: world-view
671 #+begin_src clojure :results silent
672 (in-ns 'cortex.world)
674 (defprotocol Viewable
675 (view [something]))
677 (extend-type com.jme3.scene.Geometry
678 Viewable
679 (view [geo]
680 (view (doto (Node.)(.attachChild geo)))))
682 (extend-type com.jme3.scene.Node
683 Viewable
684 (view [node]
685 (.start
686 (world node
687 {}
688 (fn [world]
689 (.enableDebug
690 (.getPhysicsSpace
691 (.getState
692 (.getStateManager world)
693 BulletAppState))
694 (asset-manager))
695 (set-gravity* world Vector3f/ZERO)
696 ;; (set-gravity* world (Vector3f. 0 (float -0.4) 0))
697 (let [sun (doto (DirectionalLight.)
698 (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))
699 (.setColor ColorRGBA/White))]
700 (.addLight (.getRootNode world) sun)))
701 no-op))))
703 (defn position-camera [game]
704 (doto (.getCamera game)
705 (.setLocation (Vector3f. 0 6 6))
706 (.lookAt Vector3f/ZERO (Vector3f. 0 1 0))))
708 #+end_src
710 Here I make the =Viewable= protocol and extend it to JME's types. Now
711 hello-world can be written as easily as:
713 #+begin_src clojure :results silent
714 (cortex.world/view (cortex.world/box))
715 #+end_src
717 ** Hello
718 Here are the jmonkeyengine "Hello" programs translated to clojure.
719 *** Hello Simple App
720 Here is the hello world example for jme3 in clojure.
721 It's a more or less direct translation from the java source
722 from
723 http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_simpleapplication.
725 Of note is the fact that since we don't have access to the
726 =AssetManager= via extendig =SimpleApplication=, we have to build one
727 ourselves.
729 #+srcname: hello-simple-app
730 #+begin_src clojure :results silent
731 (ns hello.hello-simple-app)
732 (require 'cortex.import)
733 (use 'clojure.contrib.def)
734 (rlm.rlm-commands/help)
735 (cortex.import/import-jme3)
736 (use 'cortex.world)
739 (def cube (Box. Vector3f/ZERO 1 1 1))
741 (def geom (Geometry. "Box" cube))
743 (def mat (Material. (asset-manager) "Common/MatDefs/Misc/Unshaded.j3md"))
745 (.setColor mat "Color" ColorRGBA/Blue)
747 (.setMaterial geom mat)
749 (defn simple-app []
750 (doto
751 (proxy [SimpleApplication] []
752 (simpleInitApp
753 []
754 ;; Don't take control of the mouse
755 (org.lwjgl.input.Mouse/setGrabbed false)
756 (.attachChild (.getRootNode this) geom)))
757 ;; don't show a menu to change options.
758 (.setShowSettings false)
759 (.setPauseOnLostFocus false)
760 (.setSettings *app-settings*)))
761 #+end_src
763 Running this program will begin a new jMonkeyEngine game which
764 displays a single blue cube.
766 #+begin_src clojure :exports code :results silent
767 (.start (hello.hello-simple-app/simple-app))
768 #+end_src
770 #+caption: the simplest JME game.
771 [[./images/simple-app.jpg]]
775 *** Hello Physics
776 From http://jmonkeyengine.org/wiki/doku.php/jme3:beginner:hello_physics
778 #+srcname: brick-wall-header
779 #+begin_src clojure :results silent
780 (ns hello.brick-wall)
781 (require 'cortex.import)
782 (use 'clojure.contrib.def)
783 (rlm.rlm-commands/help)
784 (cortex.import/mega-import-jme3)
785 (use '[pokemon [lpsolve :only [constant-map]]])
786 (use 'cortex.world)
787 #+end_src
789 #+srcname: brick-wall-body
790 #+begin_src clojure :results silent
791 (in-ns 'hello.brick-wall)
793 (defn floor
794 "make a sturdy, unmovable physical floor"
795 []
796 (box 20 1 20 :mass 0 :color false :position (Vector3f. 0 -2 0)))
798 (def brick-length 0.48)
799 (def brick-width 0.24)
800 (def brick-height 0.12)
803 (defn brick* [position]
804 (doto (box brick-length brick-height brick-width
805 :position position :name "brick"
806 :material "Common/MatDefs/Misc/Unshaded.j3md"
807 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
808 :mass 36)
809 (->
810 (.getMesh)
811 (.scaleTextureCoordinates (Vector2f. 1 0.5)))
812 ;;(.setShadowMode RenderQueue$ShadowMode/CastAndReceive)
813 )
814 )
816 (defn inception-brick-wall
817 "construct a physical brick wall"
818 []
819 (let [node (Node. "brick-wall")]
820 (dorun
821 (map (comp #(.attachChild node %) brick*)
822 (for
823 [x (range 15)
824 y (range 10)
825 z (range 1)]
826 (Vector3f.
827 (* brick-length x 1.03)
828 (* brick-width y y 10)
829 (* brick-height z)))))
830 node))
832 (defn gravity-toggle
833 [new-value]
834 (fn [game value]
835 (println-repl "set gravity to " new-value)
836 (if value
837 (set-gravity* game new-value)
838 (set-gravity* game gravity))))
840 (defn fire-cannon-ball
841 ([node]
842 (fn [game value]
843 (if (not value)
844 (let [camera (.getCamera game)
845 cannon-ball
846 (sphere 0.7
847 :material "Common/MatDefs/Misc/Unshaded.j3md"
848 :texture "Textures/PokeCopper.jpg"
849 :position
850 (.add (.getLocation camera)
851 (.mult (.getDirection camera) (float 1)))
852 :mass 3)] ;200 0.05
853 (.setShadowMode cannon-ball RenderQueue$ShadowMode/CastAndReceive)
854 (.setLinearVelocity
855 (.getControl cannon-ball RigidBodyControl)
856 (.mult (.getDirection camera) (float 50))) ;50
857 (add-element game cannon-ball node)))))
858 ([]
859 (fire-cannon-ball false)))
862 (defn floor* []
863 (doto (box 10 0.1 5 :name "floor" ;10 0.1 5 ; 240 0.1 240
864 :material "Common/MatDefs/Misc/Unshaded.j3md"
865 :texture "Textures/Terrain/Pond/Pond.png"
866 :position (Vector3f. 0 -0.1 0 )
867 :mass 0)
868 (->
869 (.getMesh)
870 (.scaleTextureCoordinates (Vector2f. 3 6)));64 64
871 (->
872 (.getMaterial)
873 (.getTextureParam "ColorMap")
874 (.getTextureValue)
875 (.setWrap Texture$WrapMode/Repeat))
876 (.setShadowMode RenderQueue$ShadowMode/Receive)
877 ))
879 (defn brick-wall* []
880 (let [node (Node. "brick-wall")]
881 (dorun
882 (map
883 (comp #(.attachChild node %) brick*)
884 (for [y (range 15)
885 x (range 4)
886 z (range 1)]
887 (Vector3f.
888 (+ (* 2 x brick-length)
889 (if (even? (+ y z))
890 (/ brick-length 4) (/ brick-length -4)))
891 (+ (* brick-height (inc (* 2 y))))
892 (* 2 z brick-width) ))))
893 (.setShadowMode node RenderQueue$ShadowMode/CastAndReceive)
894 node))
896 (defn brick-wall-game-run []
897 (doto
898 (world
899 (doto (Node.) (.attachChild (floor*))
900 (.attachChild (brick-wall*))
901 )
902 {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))
903 "key-m" (gravity-toggle (Vector3f. 0 0 9.81))
904 "key-l" (gravity-toggle (Vector3f. 9.81 0 0))
905 "key-j" (gravity-toggle (Vector3f. -9.81 0 0))
906 "key-k" (gravity-toggle Vector3f/ZERO)
907 "key-u" (gravity-toggle (Vector3f. 0 9.81 0))
908 "key-o" (gravity-toggle (Vector3f. 0 -9.81 0))
909 "key-f" (fn[game value]
910 (if (not value) (add-element game (brick-wall*))))
911 "key-return" (fire-cannon-ball)}
912 position-camera
913 (fn [& _]))
914 (.start)))
915 #+end_src
917 #+begin_src clojure :results silent
918 (hello.brick-wall/brick-wall-game-run)
919 #+end_src
921 #+caption: the brick wall standing
922 [[./images/brick-wall-standing.jpg]]
924 #+caption: the brick wall after it has been knocked over by a "pok\eacute{}ball"
925 [[./images/brick-wall-knocked-down.jpg]]
927 *** Other Brick Games
928 #+srcname: other-games
929 #+begin_src clojure :results silent
930 (ns cortex.other-games
931 {:author "Dylan Holmes"})
932 (use 'cortex.world)
933 (use 'hello.brick-wall)
934 (use 'cortex.import)
935 (cortex.import/mega-import-jme3)
937 (defn scad [position]
938 (doto (box 0.1 0.1 0.1
939 :position position :name "brick"
940 :material "Common/MatDefs/Misc/Unshaded.j3md"
941 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
942 :mass 20)
943 (->
944 (.getMesh)
945 (.scaleTextureCoordinates (Vector2f. 1 0.5))
946 )
947 (-> (.getControl RigidBodyControl)
948 (.setLinearVelocity (Vector3f. 0 100 0))
949 )
951 ;;(.setShadowMode RenderQueue$ShadowMode/Cast)
952 ))
955 (defn shrapnel []
956 (let [node (Node. "explosion-day")]
957 (dorun
958 (map
959 (comp #(.attachChild node %) scad)
960 (for [y (range 15)
961 x (range 4)
962 z (range 1)]
963 (Vector3f.
964 (+ (* 2 x brick-height)
965 (if (even? (+ y z)) (/ brick-height 4) (/ brick-height -4)))
966 (+ (* brick-height (inc (* 2 y))))
967 (* 2 z brick-height) ))))
968 node))
971 (def domino-height 0.48)
972 (def domino-thickness 0.12)
973 (def domino-width 0.24)
975 (def domino-thickness 0.05)
976 (def domino-width 0.5)
977 (def domino-height 1)
979 (defn domino
980 ([position]
981 (domino position (Quaternion/IDENTITY)))
982 ([position rotation]
983 (doto (box domino-width domino-height domino-thickness
984 :position position :name "domino"
985 :material "Common/MatDefs/Misc/Unshaded.j3md"
986 :texture "Textures/Terrain/BrickWall/BrickWall.jpg"
987 :mass 1
988 :rotation rotation)
989 (.setShadowMode RenderQueue$ShadowMode/CastAndReceive)
990 )))
993 (defn domino-row []
994 (let [node (Node. "domino-row")]
995 (dorun
996 (map
997 (comp #(.attachChild node %) domino)
998 (for [
999 z (range 10)
1000 x (range 5)
1002 (Vector3f.
1003 (+ (* z domino-width) (* x 5 domino-width))
1004 (/ domino-height 1)
1005 (* -5.5 domino-thickness z) ))))
1007 node))
1009 (defn domino-cycle []
1010 (let [node (Node. "domino-cycle")]
1011 (dorun
1012 (map
1013 (comp #(.attachChild node %) (partial apply domino) )
1014 (for [n (range 720)]
1015 (let [space (* domino-height 5.5)
1016 r (fn[n] (* (+ n 3) domino-width 0.5))
1017 t (fn[n] (reduce
1019 (map
1020 (fn dt[n] (/ space (* 2 (Math/PI) (r n))))
1021 (range n))))
1022 t (t n)
1023 r (r n)
1024 ct (Math/cos t)
1025 st (Math/sin t)
1027 (list
1028 (Vector3f.
1029 (* -1 r st)
1030 (/ domino-height 1)
1031 (* r ct))
1032 (.fromAngleAxis (Quaternion.)
1033 (- (/ 3.1415926 2) t) (Vector3f. 0 1 0))
1034 )))
1035 ))
1036 node))
1039 (defn domino-game-run []
1040 (doto
1041 (world
1042 (doto (Node.) (.attachChild (floor*))
1044 {"key-i" (gravity-toggle (Vector3f. 0 0 -9.81))
1045 "key-m" (gravity-toggle (Vector3f. 0 0 9.81))
1046 "key-l" (gravity-toggle (Vector3f. 9.81 0 0))
1047 "key-j" (gravity-toggle (Vector3f. -9.81 0 0))
1048 "key-k" (gravity-toggle (Vector3f. 0 9.81 0) )
1049 "key-u" (fn[g v] ((gravity-toggle (Vector3f. 0 -0 0)) g true))
1050 "key-o" (gravity-toggle (Vector3f. 0 -9.81 0))
1052 "key-space"
1053 (fn[game value]
1055 (if (not value)
1056 (let [d (domino (Vector3f. 0 (/ domino-height 0.25) 0)
1057 (.fromAngleAxis (Quaternion.)
1058 (/ Math/PI 2) (Vector3f. 0 1 0)))]
1059 (add-element game d))))
1060 "key-f"
1061 (fn[game value](if (not value) (add-element game (domino-cycle))))
1062 "key-return" (fire-cannon-ball)}
1063 position-camera
1064 (fn [& _]))
1065 (.start)))
1066 #+end_src
1068 #+begin_src clojure :results silent
1069 (cortex.other-games/domino-game-run)
1070 #+end_src
1072 #+caption: floating dominos
1073 [[./images/dominos.jpg]]
1075 *** Hello Loop
1076 #+srcname: hello-loop
1077 #+begin_src clojure :results silent
1078 (ns hello.loop)
1079 (use 'cortex.world)
1080 (use 'cortex.import)
1081 (cortex.import/mega-import-jme3)
1082 (rlm.rlm-commands/help)
1084 (defn blue-cube []
1085 (box 1 1 1
1086 :color ColorRGBA/Blue
1087 :texture false
1088 :material "Common/MatDefs/Misc/Unshaded.j3md"
1089 :name "blue-cube"
1090 :physical? false))
1092 (defn blue-cube-game []
1093 (let [cube (blue-cube)
1094 root (doto (Node.) (.attachChild cube))]
1095 (world root
1096 {}
1097 no-op
1098 (fn [game tpf]
1099 (.rotate cube 0.0 (* 2 tpf) 0.0)))))
1100 #+end_src
1102 *** Hello Collision
1104 #+srcname: hello-collision
1105 #+begin_src clojure :results silent
1106 (ns hello.collision)
1107 (use 'cortex.world)
1108 (use 'cortex.import)
1109 (use 'clojure.contrib.def)
1112 (cortex.import/mega-import-jme3)
1113 (rlm.rlm-commands/help)
1114 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])
1117 (defn environment []
1118 (let
1119 [scene-model
1120 (doto
1121 (.loadModel
1122 (doto (asset-manager)
1123 (.registerLocator
1124 "/home/r/cortex/assets/zips/town.zip" ZipLocator))
1125 "main.scene")
1126 (.setLocalScale (float 2.0)))
1127 collision-shape
1128 (CollisionShapeFactory/createMeshShape #^Node scene-model)
1129 landscape (RigidBodyControl. collision-shape 0)]
1130 (.setShadowMode scene-model RenderQueue$ShadowMode/CastAndReceive)
1131 (.addControl scene-model landscape)
1132 scene-model))
1134 (defn player-fn []
1135 (doto
1136 (CharacterControl.
1137 (CapsuleCollisionShape. (float 1.5) (float 6)(float 1))
1138 (float 0.05))
1139 (.setJumpSpeed 20)
1140 (.setFallSpeed 30)
1141 (.setGravity 30) ;30
1142 (.setPhysicsLocation (Vector3f. 0 10 0))))
1144 (defn lights []
1145 [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 1 1 1) (float 1))))
1146 (doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 1 0.7 0 1) (float 1))))
1147 (doto (DirectionalLight.)
1148 (.setColor (.mult ColorRGBA/White (float 0.9) ))
1149 (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])
1151 (defn night-lights []
1152 [(doto (AmbientLight.) (.setColor (.mult (ColorRGBA. 0.275 0.467 0.784 1) (float 0.3))))
1153 (doto (DirectionalLight.)
1154 (.setColor (.mult ColorRGBA/White (float 0.2) ))
1155 (.setDirection (.normalizeLocal (Vector3f. 2.8 -28 2.8))))])
1157 (def player (atom (player-fn)))
1159 (defn setup-fn [game]
1160 (dorun (map #(.addLight (.getRootNode game) %) (lights)))
1161 ;; set the color of the sky
1162 (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))
1163 ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)
1164 (doto (.getFlyByCamera game)
1165 (.setMoveSpeed (float 100))
1166 (.setRotationSpeed 3))
1167 (.add
1168 (.getPhysicsSpace
1169 (.getState (.getStateManager game) BulletAppState))
1170 @player)
1172 (doto (Node.) (.attachChild (.getRootNode game))
1173 (.attachChild (brick-wall*))
1179 (def walking-up? (atom false))
1180 (def walking-down? (atom false))
1181 (def walking-left? (atom false))
1182 (def walking-right? (atom false))
1184 (defn set-walk [walk-atom game value]
1185 ;;(println-repl "setting stuff to " value)
1186 (reset! walk-atom value))
1188 (defn responses []
1189 {"key-w" (partial set-walk walking-up?)
1190 "key-d" (partial set-walk walking-right?)
1191 "key-s" (partial set-walk walking-down?)
1192 "key-a" (partial set-walk walking-left?)
1193 "key-return" (fire-cannon-ball)
1194 "key-space" (fn [game value] (.jump @player))
1195 })
1197 (defn update-fn
1198 [game tpf]
1199 (let [camera (.getCamera game)
1200 cam-dir (.multLocal
1201 (.clone
1202 (.getDirection camera)) (float 0.6))
1203 cam-left (.multLocal
1204 (.clone
1205 (.getLeft camera)) (float 0.4))
1206 walk-direction (Vector3f. 0 0 0)]
1208 (cond
1209 @walking-up? (.addLocal walk-direction cam-dir)
1210 @walking-right? (.addLocal walk-direction (.negate cam-left))
1211 @walking-down? (.addLocal walk-direction (.negate cam-dir))
1212 @walking-left? (.addLocal walk-direction cam-left))
1213 (.setWalkDirection @player walk-direction)
1214 (.setLocation camera (.getPhysicsLocation @player))))
1216 (defn run-game []
1217 (.start
1218 (world (environment)
1219 (responses)
1220 setup-fn
1221 update-fn)))
1222 #+end_src
1224 *** Hello Terrain
1225 #+srcname: hello-terrain
1226 #+begin_src clojure :results silent
1227 (ns hello.terrain)
1228 (use 'cortex.world)
1229 (use 'cortex.import)
1230 (use 'clojure.contrib.def)
1231 (import jme3tools.converters.ImageToAwt)
1234 (cortex.import/mega-import-jme3)
1235 (rlm.rlm-commands/help)
1236 (use '[hello [brick-wall :only [fire-cannon-ball brick-wall*]]])
1239 (defn setup-fn [type game]
1240 (.setMoveSpeed (.getFlyByCamera game) 50)
1241 (.setFrustumFar (.getCamera game) 10000)
1242 (let [env (environment type)
1243 cameras [(.getCamera game)]
1244 control (TerrainLodControl. env cameras)]
1245 ;;(.addControl env control)
1246 (.attachChild (.getRootNode game) env)))
1248 (defn environment [type]
1249 (let
1250 [mat_terrain
1251 (Material. (asset-manager) "Common/MatDefs/Terrain/Terrain.j3md")
1252 grass (.loadTexture (asset-manager) "Textures/Terrain/splat/grass.jpg")
1253 dirt (.loadTexture (asset-manager) "Textures/Terrain/splat/dirt.jpg")
1254 rock (.loadTexture (asset-manager) "Textures/Terrain/splat/road.jpg")
1255 heightmap-image (.loadTexture (asset-manager)
1256 ({:mountain "Textures/Terrain/splat/mountains512.png"
1257 :fortress "Textures/Terrain/splat/fortress512.png"
1258 }type))
1259 heightmap (ImageBasedHeightMap.
1260 (ImageToAwt/convert (.getImage heightmap-image) false true 0))
1261 terrain (do (.load heightmap)
1262 (TerrainQuad. "my terrain" 65 513 (.getHeightMap heightmap)))
1265 (dorun (map #(.setWrap % Texture$WrapMode/Repeat)
1266 [grass dirt rock]))
1268 (doto mat_terrain
1269 (.setTexture "Tex1" grass)
1270 (.setFloat "Tex1Scale" (float 64))
1272 (.setTexture "Tex2" dirt)
1273 (.setFloat "Tex2Scale" (float 32))
1275 (.setTexture "Tex3" rock)
1276 (.setFloat "Tex3Scale" (float 128))
1278 (.setTexture "Alpha"
1279 (.loadTexture
1280 (asset-manager)
1281 ({:mountain "Textures/Terrain/splat/alphamap.png"
1282 :fortress "Textures/Terrain/splat/alphamap2.png"} type))))
1284 (doto terrain
1285 (.setMaterial mat_terrain)
1286 (.setLocalTranslation 0 -100 0)
1287 (.setLocalScale 2 1 2))))
1291 (defn run-terrain-game [type]
1292 (.start
1293 (world
1294 (Node.)
1295 {}
1296 (partial setup-fn type)
1297 no-op)))
1298 #+end_src
1302 #+srcname: hello-animation
1303 #+begin_src clojure :results silent
1304 (ns hello.animation)
1305 (use 'cortex.world)
1306 (use 'cortex.import)
1307 (use 'clojure.contrib.def)
1308 (cortex.import/mega-import-jme3)
1309 (rlm.rlm-commands/help)
1310 (use '[hello [collision :only [lights]]])
1312 (defn stand
1313 [channel]
1314 (doto channel
1315 (.setAnim "stand" (float 0.5))
1316 (.setLoopMode LoopMode/DontLoop)
1317 (.setSpeed (float 1))))
1319 (defn anim-listener []
1320 (proxy [AnimEventListener] []
1321 (onAnimChange
1322 [control channel animation-name]
1323 (println-repl "RLM --- onAnimChange"))
1324 (onAnimCycleDone
1325 [control channel animation-name]
1326 (if (= animation-name "Walk")
1327 (stand channel)
1328 ))))
1330 (defn setup-fn [channel game]
1331 (dorun (map #(.addLight (.getRootNode game) %) (lights)))
1332 ;; set the color of the sky
1333 (.setBackgroundColor (.getViewPort game) (ColorRGBA. 0.3 0.4 0.9 1))
1334 ;(.setBackgroundColor (.getViewPort game) (ColorRGBA. 0 0 0 1)
1335 (.setAnim channel "stand")
1336 (doto (.getFlyByCamera game)
1337 (.setMoveSpeed (float 10))
1338 (.setRotationSpeed 1)))
1340 (defn walk [channel]
1341 (println-repl "zzz")
1342 (doto channel
1343 (.setAnim "Walk" (float 0.5))
1344 (.setLoopMode LoopMode/Loop)))
1347 (defn key-map [channel]
1348 {"key-space" (fn [game value]
1349 (if (not value)
1350 (walk channel)))})
1352 (defn player []
1353 (let [model (.loadModel (asset-manager) "Models/Oto/Oto.mesh.xml")
1354 control (.getControl model AnimControl)]
1355 (.setLocalScale model (float 0.5))
1356 (.clearListeners control)
1357 (.addListener control (anim-control))
1358 model))
1362 (defn run-anim-game []
1363 (let [ninja (player)
1364 control (.getControl ninja AnimControl)
1365 channel (.createChannel control)]
1366 (.start
1367 (world
1368 ninja
1369 (key-map channel)
1370 (partial setup-fn channel)
1371 no-op))))
1372 #+end_src
1374 *** Hello Materials
1375 #+srcname: material
1376 #+begin_src clojure :results silent
1377 (ns hello.material)
1378 (use 'cortex.world)
1379 (use 'cortex.import)
1380 (use 'clojure.contrib.def)
1381 (cortex.import/mega-import-jme3)
1382 (rlm.rlm-commands/help)
1384 (defn simple-cube []
1385 (box 1 1 1
1386 :position (Vector3f. -3 1.1 0)
1387 :material "Common/MatDefs/Misc/Unshaded.j3md"
1388 :texture "Interface/Logo/Monkey.jpg"
1389 :physical? false))
1391 (defn leaky-box []
1392 (box 1 1 1
1393 :position (Vector3f. 3 -1 0)
1394 :material "Common/MatDefs/Misc/ColoredTextured.j3md"
1395 :texture "Textures/ColoredTex/Monkey.png"
1396 :color (ColorRGBA. 1 0 1 1)
1397 :physical? false))
1399 (defn transparent-box []
1400 (doto
1401 (box 1 1 0.1
1402 :position Vector3f/ZERO
1403 :name "window frame"
1404 :material "Common/MatDefs/Misc/Unshaded.j3md"
1405 :texture "Textures/ColoredTex/Monkey.png"
1406 :physical? false)
1407 (-> (.getMaterial)
1408 (.getAdditionalRenderState)
1409 (.setBlendMode RenderState$BlendMode/Alpha))
1410 (.setQueueBucket RenderQueue$Bucket/Transparent)))
1412 (defn bumpy-sphere []
1413 (doto
1414 (sphere 2
1415 :position (Vector3f. 0 2 -2)
1416 :name "Shiny rock"
1417 :material "Common/MatDefs/Light/Lighting.j3md"
1418 :texture false
1419 :physical? false)
1420 (-> (.getMesh)
1421 (doto
1422 (.setTextureMode Sphere$TextureMode/Projected)
1423 (TangentBinormalGenerator/generate)))
1424 (-> (.getMaterial)
1425 (doto
1426 (.setTexture "DiffuseMap" (.loadTexture (asset-manager)
1427 "Textures/Terrain/Pond/Pond.png"))
1428 (.setTexture "NormalMap" (.loadTexture (asset-manager)
1429 "Textures/Terrain/Pond/Pond_normal.png"))
1430 (.setFloat "Shininess" (float 5))))
1431 (.rotate (float 1.6) 0 0)))
1434 (defn start-game []
1435 (.start
1436 (world
1437 (let [root (Node.)]
1438 (dorun (map #(.attachChild root %)
1439 [(simple-cube) (leaky-box) (transparent-box) (bumpy-sphere)]))
1440 root)
1441 {}
1442 (fn [world]
1443 (let [sun (doto (DirectionalLight.)
1444 (.setDirection (.normalizeLocal (Vector3f. 1 0 -2)))
1445 (.setColor ColorRGBA/White))]
1446 (.addLight (.getRootNode world) sun)))
1447 no-op
1448 )))
1449 #+end_src
1453 * The Body
1454 ** Eyes
1456 Ultimately I want to make creatures with eyes. Each eye can be
1457 independely moved and should see its own version of the world
1458 depending on where it is.
1459 #+srcname: eyes
1460 #+begin_src clojure
1461 (ns body.eye)
1462 (use 'cortex.world)
1463 (use 'cortex.import)
1464 (use 'clojure.contrib.def)
1465 (cortex.import/mega-import-jme3)
1466 (rlm.rlm-commands/help)
1467 (import java.nio.ByteBuffer)
1468 (import java.awt.image.BufferedImage)
1469 (import java.awt.Color)
1470 (import java.awt.Dimension)
1471 (import java.awt.Graphics)
1472 (import java.awt.Graphics2D)
1473 (import java.awt.event.WindowAdapter)
1474 (import java.awt.event.WindowEvent)
1475 (import java.awt.image.BufferedImage)
1476 (import java.nio.ByteBuffer)
1477 (import javax.swing.JFrame)
1478 (import javax.swing.JPanel)
1479 (import javax.swing.SwingUtilities)
1480 (import javax.swing.ImageIcon)
1481 (import javax.swing.JOptionPane)
1482 (import java.awt.image.ImageObserver)
1486 (defn scene-processor
1487 "deals with converting FrameBuffers to BufferedImages so
1488 that the continuation function can be defined only in terms
1489 of what it does with BufferedImages"
1490 [continuation]
1491 (let [byte-buffer (atom nil)
1492 renderer (atom nil)
1493 image (atom nil)]
1494 (proxy [SceneProcessor] []
1495 (initialize
1496 [renderManager viewPort]
1497 (let [cam (.getCamera viewPort)
1498 width (.getWidth cam)
1499 height (.getHeight cam)]
1500 (reset! renderer (.getRenderer renderManager))
1501 (reset! byte-buffer
1502 (BufferUtils/createByteBuffer
1503 (* width height 4)))
1504 (reset! image (BufferedImage. width height
1505 BufferedImage/TYPE_4BYTE_ABGR))))
1506 (isInitialized [] (not (nil? @byte-buffer)))
1507 (reshape [_ _ _])
1508 (preFrame [_])
1509 (postQueue [_])
1510 (postFrame
1511 [#^FrameBuffer fb]
1512 (.clear @byte-buffer)
1513 (.readFrameBuffer @renderer fb @byte-buffer)
1514 (Screenshots/convertScreenShot @byte-buffer @image)
1515 (continuation @image))
1516 (cleanup []))))
1518 (defn add-eye
1519 "Add an eye to the world, and call continuation on
1520 every frame produced"
1521 [world camera continuation]
1522 (let [width (.getWidth camera)
1523 height (.getHeight camera)
1524 render-manager (.getRenderManager world)
1525 viewport (.createMainView render-manager "eye-view" camera)]
1526 (doto viewport
1527 (.setBackgroundColor ColorRGBA/Black)
1528 (.setClearFlags true true true)
1529 (.addProcessor (scene-processor continuation))
1530 (.attachScene (.getRootNode world)))))
1532 (defn make-display-frame [display width height]
1533 (SwingUtilities/invokeLater
1534 (fn []
1535 (.setPreferredSize display (Dimension. width height))
1536 (doto (JFrame. "Eye Camera!")
1537 (-> (.getContentPane) (.add display))
1538 (.setDefaultCloseOperation JFrame/DISPOSE_ON_CLOSE)
1539 (.pack)
1540 (.setLocationRelativeTo nil)
1541 (.setResizable false)
1542 (.setVisible true)))))
1544 (defn image-monitor [#^BufferedImage image]
1545 (proxy [JPanel] []
1546 (paintComponent
1547 [g]
1548 (proxy-super paintComponent g)
1549 (locking image
1550 (.drawImage g image 0 0
1551 (proxy [ImageObserver]
1552 []
1553 (imageUpdate
1554 []
1555 (proxy-super imageUpdate))))))))
1557 (defn movie-image []
1558 (let [setup
1559 (runonce
1560 (fn [#^BufferedImage image]
1561 (let [width (.getWidth image)
1562 height (.getHeight image)
1563 display (image-monitor image)
1564 frame (make-display-frame display width height)]
1565 display)))]
1566 (fn [#^BufferedImage image]
1567 (.repaint (setup image)))))
1570 (defn observer
1571 "place thy eye!"
1572 [world camera]
1573 (let [eye camera
1574 width (.getWidth eye)
1575 height (.getHeight eye)]
1576 (no-exceptions
1577 (add-eye
1578 world
1579 eye
1580 (movie-image)))))
1581 #+end_src
1583 #+srcname: test-vision
1584 #+begin_src clojure
1586 (ns test.vision)
1587 (use 'cortex.world)
1588 (use 'cortex.import)
1589 (use 'clojure.contrib.def)
1590 (use 'body.eye)
1591 (cortex.import/mega-import-jme3)
1592 (rlm.rlm-commands/help)
1593 (import java.nio.ByteBuffer)
1594 (import java.awt.image.BufferedImage)
1595 (import java.awt.Color)
1596 (import java.awt.Dimension)
1597 (import java.awt.Graphics)
1598 (import java.awt.Graphics2D)
1599 (import java.awt.event.WindowAdapter)
1600 (import java.awt.event.WindowEvent)
1601 (import java.awt.image.BufferedImage)
1602 (import java.nio.ByteBuffer)
1603 (import javax.swing.JFrame)
1604 (import javax.swing.JPanel)
1605 (import javax.swing.SwingUtilities)
1606 (import javax.swing.ImageIcon)
1607 (import javax.swing.JOptionPane)
1608 (import java.awt.image.ImageObserver)
1611 (def width 200)
1612 (def height 200)
1614 (defn camera []
1615 (doto (Camera. width height)
1616 (.setFrustumPerspective 45 1 1 1000)
1617 (.setLocation (Vector3f. -3 0 -5))
1618 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1620 (defn camera2 []
1621 (doto (Camera. width height)
1622 (.setFrustumPerspective 45 1 1 1000)
1623 (.setLocation (Vector3f. 3 0 -5))
1624 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1626 (defn setup-fn [world]
1627 (let [eye (camera)
1628 width (.getWidth eye)
1629 height (.getHeight eye)]
1630 (no-exceptions
1631 (add-eye
1632 world
1633 eye
1634 (runonce visual))
1635 (add-eye
1636 world
1637 (camera2)
1638 (runonce visual)))))
1640 (defn spider-eye [position]
1641 (doto (Camera. 200 200 )
1642 (.setFrustumPerspective 45 1 1 1000)
1643 (.setLocation position)
1644 (.lookAt Vector3f/ZERO Vector3f/UNIT_Y)))
1646 (defn setup-fn* [world]
1647 (let [eye (camera)
1648 width (.getWidth eye)
1649 height (.getHeight eye)]
1650 ;;(.setClearFlags (.getViewPort world) true true true)
1651 (observer world (.getCamera world))
1652 (observer world (spider-eye (Vector3f. 3 0 -5)))
1653 ;;(observer world (spider-eye (Vector3f. 0 0 -5)))
1654 ;; (observer world (spider-eye (Vector3f. -3 0 -5)))
1655 ;; (observer world (spider-eye (Vector3f. 0 3 -5)))
1656 ;; (observer world (spider-eye (Vector3f. 0 -3 -5)))
1657 ;; (observer world (spider-eye (Vector3f. 3 3 -5)))
1658 ;; (observer world (spider-eye (Vector3f. -3 3 -5)))
1659 ;; (observer world (spider-eye (Vector3f. 3 -3 -5)))
1660 ;; (observer world (spider-eye (Vector3f. -3 -3 -5)))
1663 world)
1665 (defn test-world []
1666 (let [thing (box 1 1 1 :physical? false)]
1667 (world
1668 (doto (Node.)
1669 (.attachChild thing))
1670 {}
1671 setup-fn
1672 (fn [world tpf]
1673 (.rotate thing (* tpf 0.2) 0 0)
1674 ))))
1677 #+end_src
1680 #+results: eyes
1681 : #'body.eye/test-world
1683 Note the use of continuation passing style for connecting the eye to a
1684 function to process the output. The example code will create two
1685 videos of the same rotating cube from different angles, sutiable for
1686 stereoscopic vision.
1693 * COMMENT code generation
1694 #+begin_src clojure :tangle ../src/cortex/import.clj
1695 <<import>>
1696 #+end_src
1698 #+begin_src clojure :tangle ../src/hello/brick_wall.clj
1699 <<brick-wall-header>>
1700 <<brick-wall-body>>
1701 #+end_src
1703 #+begin_src clojure :tangle ../src/hello/hello_simple_app.clj
1704 <<hello-simple-app>>
1705 #+end_src
1707 #+begin_src clojure :tangle ../src/cortex/world.clj
1708 <<world-inputs>>
1709 <<world>>
1710 <<world-shapes>>
1711 <<world-view>>
1712 #+end_src
1714 #+begin_src clojure :tangle ../src/cortex/other_games.clj
1715 <<other-games>>
1716 #+end_src
1718 #+begin_src clojure :tangle ../src/hello/loop.clj
1719 <<hello-loop>>
1720 #+end_src
1722 #+begin_src clojure :tangle ../src/hello/collision.clj
1723 <<hello-collision>>
1724 #+end_src
1726 #+begin_src clojure :tangle ../src/hello/terrain.clj
1727 <<hello-terrain>>
1728 #+end_src
1730 #+begin_src clojure :tangle ../src/hello/animation.clj
1731 <<hello-animation>>
1732 #+end_src
1734 #+begin_src clojure :tangle ../src/hello/material.clj
1735 <<material>>
1736 #+end_src
1738 #+begin_src clojure :tangle ../src/body/eye.clj
1739 <<eyes>>
1740 #+end_src
1742 #+begin_src clojure :tangle ../src/test/vision.clj
1743 <<test-vision>>
1744 #+end_src