cortex: org/cortex.org comparison

comparison org/cortex.org @ 21:01e1427126af

splitting cortex into more manageable pieces

author	Robert McIntyre <rlm@mit.edu>
date	Sun, 23 Oct 2011 13:49:45 -0700
parents	c32f3eb9fdeb
children	cab2da252494

comparison

equal deleted inserted replaced

-:67d508a1e34d
+:01e1427126af
 #+author: Robert McIntyre
 #+email: rlm@mit.edu
 #+description: Simulating senses for AI research using JMonkeyEngine3
 #+SETUPFILE: ../../aurellem/org/setup.org
 #+INCLUDE: ../../aurellem/org/level-0.org
-#+babel: :mkdirp yes :noweb yes
+#+babel: :mkdirp yes :noweb yes :exports both
-* Background
-Artificial Intelligence has tried and failed for more than half a
-century to produce programs as flexible, creative, and “intelligent”
-as the human mind itself. Clearly, we are still missing some important
-ideas concerning intelligent programs or we would have strong AI
-already. What idea could be missing?
-When Turing first proposed his famous “Turing Test” in the
-groundbreaking paper [[./sources/turing.pdf][/Computing Machines and Intelligence/]], he gave
-little importance to how a computer program might interact with the
-world:
-#+BEGIN_QUOTE
-\ldquo{}We need not be too concerned about the legs, eyes, etc. The example of
-Miss Helen Keller shows that education can take place provided that
-communication in both directions between teacher and pupil can take
-place by some means or other.\rdquo{}
-#+END_QUOTE
-And from the example of Hellen Keller he went on to assume that the
-only thing a fledgling AI program could need by way of communication
-is a teletypewriter. But Hellen Keller did possess vision and hearing
-for the first few months of her life, and her tactile sense was far
-more rich than any text-stream could hope to achieve. She possessed a
-body she could move freely, and had continual access to the real world
-to learn from her actions.
-I believe that our programs are suffering from too little sensory
-input to become really intelligent. Imagine for a moment that you
-lived in a world completely cut off form all sensory stimulation. You
-have no eyes to see, no ears to hear, no mouth to speak. No body, no
-taste, no feeling whatsoever. The only sense you get at all is a
-single point of light, flickering on and off in the void. If this was
-your life from birth, you would never learn anything, and could never
-become intelligent. Actual humans placed in sensory deprivation
-chambers experience hallucinations and can begin to loose their sense
-of reality in as little as 15 minutes[sensory-deprivation]. Most of
-the time, the programs we write are in exactly this situation. They do
-not interface with cameras and microphones, and they do not control a
-real or simulated body or interact with any sort of world.
-* Simulation vs. Reality
-I want demonstrate that multiple senses are what enable
-intelligence. There are two ways of playing around with senses and
-computer programs:
-The first is to go entirely with simulation: virtual world, virtual
-character, virtual senses. The advantages are that when everything is
-a simulation, experiments in that simulation are absolutely
-reproducible. It's also easier to change the character and world to
-explore new situations and different sensory combinations.
-** Issues with Simulation
-If the world is to be simulated on a computer, then not only do you
-have to worry about whether the character's senses are rich enough to
-learn from the world, but whether the world itself is rendered with
-enough detail and realism to give enough working material to the
-character's senses. To name just a few difficulties facing modern
-physics simulators: destructibility of the environment, simulation of
-water/other fluids, large areas, nonrigid bodies, lots of objects,
-smoke. I don't know of any computer simulation that would allow a
-character to take a rock and grind it into fine dust, then use that
-dust to make a clay sculpture, at least not without spending years
-calculating the interactions of every single small grain of
-dust. Maybe a simulated world with today's limitations doesn't provide
-enough richness for real intelligence to evolve.
-** Issues with Reality
-The other approach for playing with senses is to hook your software up
-to real cameras, microphones, robots, etc., and let it loose in the
-real world. This has the advantage of eliminating concerns about
-simulating the world at the expense of increasing the complexity of
-implementing the senses. Instead of just grabbing the current rendered
-frame for processing, you have to use an actual camera with real
-lenses and interact with photons to get an image. It is much harder to
-change the character, which is now partly a physical robot of some
-sort, since doing so involves changing things around in the real world
-instead of modifying lines of code. While the real world is very rich
-and definitely provides enough stimulation for intelligence to develop
-as evidenced by our own existence, it is also uncontrollable in the
-sense that a particular situation cannot be recreated perfectly or
-saved for later use. It is harder to conduct science because it is
-harder to repeat an experiment. The worst thing about using the real
-world instead of a simulation is the matter of time. Instead of
-simulated time you get the constant and unstoppable flow of real
-time. This severely limits the sorts of software you can use to
-program the AI because all sense inputs must be handled in real
-time. Complicated ideas may have to be implemented in hardware or may
-simply be impossible given the current speed of our
-processors. Contrast this with a simulation, in which the flow of time
-in the simulated world can be slowed down to accommodate the
-limitations of the character's programming. In terms of cost, doing
-everything in software is far cheaper than building custom real-time
-hardware. All you need is a laptop and some patience.
-* Choose a Simulation Engine
-Mainly because of issues with controlling the flow of time, I chose to
-simulate both the world and the character. I set out to make a minimal
-world in which I could embed a character with multiple senses. My main
-goal is to make an environment where I can perform further experiments
-in simulated senses.
-As Carl Sagan once said, "If you wish to make an apple pie from
-scratch, you must first invent the universe.” I examined many
-different 3D environments to try and find something I would use as the
-base for my simulation; eventually the choice came down to three
-engines: the Quake II engine, the Source Engine, and jMonkeyEngine.
-** Quake II/Jake2
-I spent a bit more than a month working with the Quake II Engine from
-ID software to see if I could use it for my purposes. All the source
-code was released by ID software into the Public Domain several years
-ago, and as a result it has been ported and modified for many
-different reasons. This engine was famous for its advanced use of
-realistic shading and had decent and fast physics
-simulation. Researchers at Princeton [[http://www.nature.com/nature/journal/v461/n7266/pdf/nature08499.pdf][used this code]] to study spatial
-information encoding in the hippocampal cells of rats. Those
-researchers created a special Quake II level that simulated a maze,
-and added an interface where a mouse could run around inside a ball in
-various directions to move the character in the simulated maze. They
-measured hippocampal activity during this exercise to try and tease
-out the method in which spatial data was stored in that area of the
-brain. I find this promising because if a real living rat can interact
-with a computer simulation of a maze in the same way as it interacts
-with a real-world maze, then maybe that simulation is close enough to
-reality that a simulated sense of vision and motor control interacting
-with that simulation could reveal useful information about the real
-thing. It happens that there is a Java port of the original C source
-code called Jake2. The port demonstrates Java's OpenGL bindings and
-runs anywhere from 90% to 105% as fast as the C version. After
-reviewing much of the source of Jake2, I eventually rejected it
-because the engine is too tied to the concept of a first-person
-shooter game. One of the problems I had was that there does not seem
-to be any easy way to attach multiple cameras to a single
-character. There are also several physics clipping issues that are
-corrected in a way that only applies to the main character and does
-not apply to arbitrary objects. While there is a large community of
-level modders, I couldn't find a community to support using the engine
-to make new things.
-** Source Engine
-The Source Engine evolved from the Quake II and Quake I engines and is
-used by Valve in the Half-Life series of games. The physics simulation
-in the Source Engine is quite accurate and probably the best out of
-all the engines I investigated. There is also an extensive community
-actively working with the engine. However, applications that use the
-Source Engine must be written in C++, the code is not open, it only
-runs on Windows, and the tools that come with the SDK to handle models
-and textures are complicated and awkward to use.
-** jMonkeyEngine
-jMonkeyEngine is a new library for creating games in Java. It uses
-OpenGL to render to the screen and uses screengraphs to avoid drawing
-things that do not appear on the screen. It has an active community
-and several games in the pipeline. The engine was not built to serve
-any particular game but is instead meant to be used for any 3D
-game. After experimenting with each of these three engines and a few
-others for about 2 months I settled on jMonkeyEngine. I chose it
-because it had the most features out of all the open projects I looked
-at, and because I could then write my code in Clojure, an
-implementation of LISP that runs on the JVM.
-* Setup
-First, I checked out the source to jMonkeyEngine:
-#+srcname: checkout
-#+begin_src sh :results verbatim
-svn checkout http://jmonkeyengine.googlecode.com/svn/trunk/engine jme3
-#+end_src
-#+results: checkout
-: Checked out revision 7975.
-Building jMonkeyEngine is easy enough:
-#+srcname: build
-#+begin_src sh :results verbatim
-cd jme3
-ant jar | tail -n 2
-#+end_src
-#+results: build
-: BUILD SUCCESSFUL
-: Total time: 15 seconds
-Also build the javadoc:
-#+srcname: javadoc
-#+begin_src sh :results verbatim
-cd jme3
-ant javadoc | tail -n 2
-#+end_src
-#+results: javadoc
-: BUILD SUCCESSFUL
-: Total time: 12 seconds
-Now, move the jars from the compilation into the project's lib folder.
-#+srcname: move-jars
-#+begin_src sh :results verbatim
-mkdir -p lib
-mkdir -p src
-cp jme3/dist/jMonkeyEngine3.jar lib/
-cp jme3/dist/lib/* lib/
-ls lib
-#+end_src
-#+results: move-jars
-#+begin_example
-eventbus-1.4.jar
-jbullet.jar
-jheora-jst-debug-0.6.0.jar
-jinput.jar
-jME3-jbullet.jar
-jME3-lwjgl-natives.jar
-jME3-testdata.jar
-jME3-test.jar
-jMonkeyEngine3.jar
-j-ogg-oggd.jar
-j-ogg-vorbisd.jar
-lwjgl.jar
-nifty-1.3.jar
-nifty-default-controls-1.3.jar
-nifty-examples-1.3.jar
-nifty-lwjgl-renderer-1.3.jar
-nifty-openal-soundsystem-1.0.jar
-nifty-style-black-1.3.jar
-nifty-style-grey-1.0.jar
-noise-0.0.1-SNAPSHOT.jar
-stack-alloc.jar
-vecmath.jar
-xmlpull-xpp3-1.1.4c.jar
-#+end_example
-It's good to create a =assets= directory in the style that the
-=AssetManager= will like.
-#+srcname: create-assets
-#+begin_src sh :results verbatim
-mkdir -p assets
-mkdir -p assets/Interface
-mkdir -p assets/Materials
-mkdir -p assets/MatDefs
-mkdir -p assets/Models
-mkdir -p assets/Scenes
-mkdir -p assets/Shaders
-mkdir -p assets/Sounds
-mkdir -p assets/Textures
-tree -L 1 assets
-#+end_src
-#+results: create-assets
-#+begin_example
-assets
-|-- Interface
-|-- MatDefs
-|-- Materials
-|-- Models
-|-- Scenes
-|-- Shaders
-|-- Sounds
-`-- Textures
-8 directories, 0 files
-#+end_example
-The java classpath should have all the jars contained in the =lib=
-directory as well as the src directory.
-For example, here is the file I use to run my REPL for clojure.
-#+include: "~/swank-all" src sh :exports code
-The important thing here is that =cortex/lib/*=, =cortex/src=, and
-=cortex/assets= appear on the classpath. (=cortex= is the base
-directory of this project.)
-#+srcname: pwd
-#+begin_src sh
-pwd
-#+end_src
-#+results: pwd
-: /home/r/cortex
 * Simulation Base
 ** Imports
-First, I'll import jme core classes.
+jMonkeyEngine has a plethora of classes which can be overwhelming at
+first. So that I one can get right to coding, it's good to take the
+time right now and make a "import all" function which brings in all of
+the important jme3 classes. Once I'm happy with the general structure
+of a namespace I can deal with importing only the classes it actually
+needs.
 #+srcname: import
 #+begin_src clojure :results silent
 (ns cortex.import
 (:require swank.util.class-browse))
-(defn import-jme3 []
+(defn permissive-import
-(import '[com.jme3.system AppSettings JmeSystem])
+[classname]
-(import '[com.jme3.app Application SimpleApplication])
+(eval `(try (import '~classname)
-(import 'com.jme3.material.Material)
+(catch java.lang.Exception e#
-(import '[com.jme3.math Vector3f ColorRGBA Quaternion Transform])
+(println "couldn't import " '~classname))))
-(import '[com.jme3.scene Node  Geometry])
+classname)
-(import '[com.jme3.scene.shape Box Sphere Sphere$TextureMode])
-(import 'com.jme3.font.BitmapText)
+(defn jme-class? [classname]
-(import '[com.jme3.input KeyInput InputManager])
+(and
-(import '[com.jme3.input.controls
+(.startsWith classname "com.jme3.")
-	    ActionListener AnalogListener KeyTrigger MouseButtonTrigger])
+;; Don't import the Lwjgl stuff since it can throw exceptions
-(import '[com.jme3.asset AssetManager DesktopAssetManager] )
+;;  upon being loaded.
-(import '[com.jme3.asset.plugins HttpZipLocator ZipLocator])
+(not (re-matches #".*Lwjgl.*" classname))))
-(import '[com.jme3.light PointLight DirectionalLight])
-(import '[com.jme3.animation AnimControl Skeleton Bone])
+(defn jme-classes
-(import '[com.jme3.bullet.collision.shapes
+"returns a list of all jme3 classes"
-	    MeshCollisionShape SphereCollisionShape BoxCollisionShape])
+[]
-(import 'com.jme3.renderer.queue.RenderQueue$ShadowMode)
+(filter
-(import 'jme3test.TestChooser)
+jme-class?
-(import '[com.jme3.bullet PhysicsTickListener PhysicsSpace])
+(map :name
-(import '[com.jme3.bullet.joints SixDofJoint HingeJoint
+swank.util.class-browse/available-classes)))
-	    SliderJoint Point2PointJoint ConeJoint]))
-(defmacro permissive-import* [class-symbol]
-`(try
-(import ~class-symbol)
-(catch Exception e#
-	 (println "can't import " ~class-symbol))))
-(defn permissive-import [class-symbol]
-(eval (list 'cortex.import/permissive-import* class-symbol)))
-(defn selection-import [selection-fn]
-(dorun
-(map (comp permissive-import symbol)
-	(filter selection-fn
-		(map :name
-		     swank.util.class-browse/available-classes)))))
 (defn mega-import-jme3
-"ALL the jme classes. For REPL use."
+"Import ALL the jme classes. For REPL use."
 []
-(selection-import
+(doall
-#(and
+(map (comp permissive-import symbol) (jme-classes))))
-(.startsWith % "com.jme3.")
-;; Don't import the Lwjgl stuff since it can throw exceptions
-;; upon being loaded.
-(not (re-matches #".*Lwjgl.*" %)))))
 #+end_src
 The =mega-import-jme3= is quite usefull for debugging purposes since
-it allows completion for almost all of JME's classes
+it allows completion for almost all of JME's classes.
+Out of curiousity, let's see just how many classes =mega-import-jme3=
+imports:
+#+begin_src clojure :exports both
+(clojure.core/count (cortex.import/jme-classes))
+#+end_src
+#+results:
+: 955
 ** Simplification
 *** World
 It is comvienent to wrap the JME elements that deal with creating a
 #+begin_src clojure :results silent
 (ns hello.hello-simple-app)
 (require 'cortex.import)
 (use 'clojure.contrib.def)
 (rlm.rlm-commands/help)
-(cortex.import/import-jme3)
+(cortex.import/mega-import-jme3)
 (use 'cortex.world)
 (def cube (Box. Vector3f/ZERO 1 1 1))
 :mass 3)] ;200 0.05
 (.setShadowMode cannon-ball RenderQueue$ShadowMode/CastAndReceive)
 (.setLinearVelocity
 (.getControl cannon-ball RigidBodyControl)
 (.mult (.getDirection camera) (float 50))) ;50
-(add-element game cannon-ball node)))))
+(add-element game cannon-ball (if node node (.getRootNode game)))))))
 ([]
 (fire-cannon-ball false)))
 (defn floor* []

Mercurial > cortex

comparison org/cortex.org @ 21:01e1427126af