view thesis/user-guide.org @ 487:19b55aaf4462

complete first draft of user guide.
author Robert McIntyre <rlm@mit.edu>
date Sat, 29 Mar 2014 20:17:58 -0400
parents 6d460ac3f5d0
children
line wrap: on
line source
1 * Appendix: =CORTEX= User Guide
3 Those who write a thesis should endeavor to make their code not only
4 accessable, but actually useable, as a way to pay back the community
5 that made the thesis possible in the first place. This thesis would
6 not be possible without Free Software such as jMonkeyEngine3,
7 Blender, clojure, emacs, ffmpeg, and many other tools. That is why I
8 have included this user guide, in the hope that someone else might
9 find =CORTEX= useful.
11 ** Obtaining =CORTEX=
13 You can get cortex from its mercurial repository at
14 http://hg.bortreb.com/cortex. You may also download =CORTEX=
15 releases at http://aurellem.org/cortex/releases/. As a condition of
16 making this thesis, I have also provided Professor Winston the
17 =CORTEX= source, and he knows how to run the demos and get started.
18 You may also email me at =cortex@aurellem.org= and I may help where
19 I can.
21 ** Running =CORTEX=
23 =CORTEX= comes with README and INSTALL files that will guide you
24 through installation and running the test suite. In particular you
25 should look at test =cortex.test= which contains test suites that
26 run through all senses and multiple creatures.
28 ** Creating creatures
30 Creatures are created using /Blender/, a free 3D modeling program.
31 You will need Blender version 2.6 when using the =CORTEX= included
32 in this thesis. You create a =CORTEX= creature in a similiar manner
33 to modeling anything in Blender, except that you also create
34 several trees of empty nodes which define the creature's senses.
36 *** Mass
38 To give an object mass in =CORTEX=, add a ``mass'' metadata label
39 to the object with the mass in jMonkeyEngine units. Note that
40 setting the mass to 0 causes the object to be immovable.
42 *** Joints
44 Joints are created by creating an empty node named =joints= and
45 then creating any number of empty child nodes to represent your
46 creature's joints. The joint will automatically connect the
47 closest two physical objects. It will help to set the empty node's
48 display mode to ``Arrows'' so that you can clearly see the
49 direction of the axes.
51 Joint nodes should have the following metadata under the ``joint''
52 label:
54 #+BEGIN_SRC clojure
55 ;; ONE OF the following, under the label "joint":
56 {:type :point}
58 ;; OR
60 {:type :hinge
61 :limit [<limit-low> <limit-high>]
62 :axis (Vector3f. <x> <y> <z>)}
63 ;;(:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
65 ;; OR
67 {:type :cone
68 :limit-xz <lim-xz>
69 :limit-xy <lim-xy>
70 :twist <lim-twist>} ;(use XZY rotation mode in blender!)
71 #+END_SRC
73 *** Eyes
75 Eyes are created by creating an empty node named =eyes= and then
76 creating any number of empty child nodes to represent your
77 creature's eyes.
79 Eye nodes should have the following metadata under the ``eye''
80 label:
82 #+BEGIN_SRC clojure
83 {:red <red-retina-definition>
84 :blue <blue-retina-definition>
85 :green <green-retina-definition>
86 :all <all-retina-definition>
87 (<0xrrggbb> <custom-retina-image>)...
88 }
89 #+END_SRC
91 Any of the color channels may be omitted. You may also include
92 your own color selectors, and in fact :red is equivalent to
93 0xFF0000 and so forth. The eye will be placed at the same position
94 as the empty node and will bind to the neatest physical object.
95 The eye will point outward from the X-axis of the node, and ``up''
96 will be in the direction of the X-axis of the node. It will help
97 to set the empty node's display mode to ``Arrows'' so that you can
98 clearly see the direction of the axes.
100 Each retina file should contain white pixels whever you want to be
101 sensitive to your chosen color. If you want the entire field of
102 view, specify :all of 0xFFFFFF and a retinal map that is entirely
103 white.
105 Here is a sample retinal map:
107 #+caption: An example retinal profile image. White pixels are
108 #+caption: photo-sensitive elements. The distribution of white
109 #+caption: pixels is denser in the middle and falls off at the
110 #+caption: edges and is inspired by the human retina.
111 #+name: retina
112 #+ATTR_LaTeX: :width 7cm :placement [H]
113 [[./images/retina-small.png]]
115 *** Hearing
117 Ears are created by creating an empty node named =ears= and then
118 creating any number of empty child nodes to represent your
119 creature's ears.
121 Ear nodes do not require any metadata.
123 The ear will bind to and follow the closest physical node.
125 *** Touch
127 Touch is handled similarly to mass. To make a particular object
128 touch sensitive, add metadata of the following form under the
129 object's ``touch'' metadata field:
131 #+BEGIN_EXAMPLE
132 <touch-UV-map-file-name>
133 #+END_EXAMPLE
135 You may also include an optional ``scale'' metadata number to
136 specifiy the length of the touch feelers. The default is $0.1$,
137 and this is generally sufficient.
139 The touch UV should contain white pixels for each touch sensor.
141 Here is an example touch-uv map that approximates a human finger,
142 and its corresponding model.
144 #+caption: This is the tactile-sensor-profile for the upper segment
145 #+caption: of a fingertip. It defines regions of high touch sensitivity
146 #+caption: (where there are many white pixels) and regions of low
147 #+caption: sensitivity (where white pixels are sparse).
148 #+name: guide-fingertip-UV
149 #+ATTR_LaTeX: :width 9cm :placement [H]
150 [[./images/finger-UV.png]]
152 #+caption: The fingertip UV-image form above applied to a simple
153 #+caption: model of a fingertip.
154 #+name: guide-fingertip
155 #+ATTR_LaTeX: :width 9cm :placement [H]
156 [[./images/finger-2.png]]
158 *** Propriocepotion
160 Proprioception is tied to each joint node -- nothing special must
161 be done in a blender model to enable proprioception other than
162 creating joint nodes.
164 *** Muscles
166 Muscles are created by creating an empty node named =muscles= and
167 then creating any number of empty child nodes to represent your
168 creature's muscles.
171 Muscle nodes should have the following metadata under the
172 ``muscle'' label:
174 #+BEGIN_EXAMPLE
175 <muscle-profile-file-name>
176 #+END_EXAMPLE
178 Muscles should also have a ``strength'' metadata entry describing
179 the muscle's total strength at full activation.
181 Muscle profiles are simple images that contain the relative amount
182 of muscle power in each simulated alpha motor neuron. The width of
183 the image is the total size of the motor pool, and the redness of
184 each neuron is the relative power of that motor pool.
186 While the profile image can have any dimensions, only the first
187 line of pixels is used to define the muscle. Here is a sample
188 muscle profile image that defines a human-like muscle.
190 #+caption: A muscle profile image that describes the strengths
191 #+caption: of each motor neuron in a muscle. White is weakest
192 #+caption: and dark red is strongest. This particular pattern
193 #+caption: has weaker motor neurons at the beginning, just
194 #+caption: like human muscle.
195 #+name: muscle-recruit
196 #+ATTR_LaTeX: :width 7cm :placement [H]
197 [[./images/basic-muscle.png]]
199 Muscles twist the nearest physical object about the muscle node's
200 Z-axis. I recommend using the ``Single Arrow'' display mode for
201 muscles and using the right hand rule to determine which way the
202 muscle will twist. To make a segment that can twist in multiple
203 directions, create multiple, differently aligned muscles.
205 ** =CORTEX= API
207 These are the some functions exposed by =CORTEX= for creating
208 worlds and simulating creatures. These are in addition to
209 jMonkeyEngine3's extensive library, which is documented elsewhere.
211 *** Simulation
212 - =(world root-node key-map setup-fn update-fn)= :: create
213 a simulation.
214 - /root-node/ :: a =com.jme3.scene.Node= object which
215 contains all of the objects that should be in the
216 simulation.
218 - /key-map/ :: a map from strings describing keys to
219 functions that should be executed whenever that key is
220 pressed. the functions should take a SimpleApplication
221 object and a boolean value. The SimpleApplication is the
222 current simulation that is running, and the boolean is true
223 if the key is being pressed, and false if it is being
224 released. As an example,
225 #+BEGIN_SRC clojure
226 {"key-j" (fn [game value] (if value (println "key j pressed")))}
227 #+END_SRC
228 is a valid key-map which will cause the simulation to print
229 a message whenever the 'j' key on the keyboard is pressed.
231 - /setup-fn/ :: a function that takes a =SimpleApplication=
232 object. It is called once when initializing the simulation.
233 Use it to create things like lights, change the gravity,
234 initialize debug nodes, etc.
236 - /update-fn/ :: this function takes a =SimpleApplication=
237 object and a float and is called every frame of the
238 simulation. The float tells how many seconds is has been
239 since the last frame was rendered, according to whatever
240 clock jme is currently using. The default is to use IsoTimer
241 which will result in this value always being the same.
243 - =(position-camera world position rotation)= :: set the position
244 of the simulation's main camera.
246 - =(enable-debug world)= :: turn on debug wireframes for each
247 simulated object.
249 - =(set-gravity world gravity)= :: set the gravity of a running
250 simulation.
252 - =(box length width height & {options})= :: create a box in the
253 simulation. Options is a hash map specifying texture, mass,
254 etc. Possible options are =:name=, =:color=, =:mass=,
255 =:friction=, =:texture=, =:material=, =:position=,
256 =:rotation=, =:shape=, and =:physical?=.
258 - =(sphere radius & {options})= :: create a sphere in the simulation.
259 Options are the same as in =box=.
261 - =(load-blender-model file-name)= :: create a node structure
262 representing that described in a blender file.
264 - =(light-up-everything world)= :: distribute a standard compliment
265 of lights throught the simulation. Should be adequate for most
266 purposes.
268 - =(node-seq node)= :: return a recursuve list of the node's
269 children.
271 - =(nodify name children)= :: construct a node given a node-name and
272 desired children.
274 - =(add-element world element)= :: add an object to a running world
275 simulation.
277 - =(set-accuracy world accuracy)= :: change the accuracy of the
278 world's physics simulator.
280 - =(asset-manager)= :: get an /AssetManager/, a jMonkeyEngine
281 construct that is useful for loading textures and is required
282 for smooth interaction with jMonkeyEngine library functions.
284 - =(load-bullet)= :: unpack native libraries and initialize
285 blender. This function is required before other world building
286 functions are called.
289 *** Creature Manipulation / Import
291 - =(body! creature)= :: give the creature a physical body.
293 - =(vision! creature)= :: give the creature a sense of vision.
294 Returns a list of functions which will each, when called
295 during a simulation, return the vision data for the channel of
296 one of the eyes. The functions are ordered depending on the
297 alphabetical order of the names of the eye nodes in the
298 blender file. The data returned by the functions is a vector
299 containing the eye's /topology/, a vector of coordinates, and
300 the eye's /data/, a vector of RGB values filtered by the eye's
301 sensitivity.
303 - =(hearing! creature)= :: give the creature a sense of hearing.
304 Returns a list of functions, one for each ear, that when
305 called will return a frame's worth of hearing data for that
306 ear. The functions are ordered depending on the alphabetical
307 order of the names of the ear nodes in the blender file. The
308 data returned by the functions is an array PCM encoded wav
309 data.
311 - =(touch! creature)= :: give the creature a sense of touch. Returns
312 a single function that must be called with the /root node/ of
313 the world, and which will return a vector of /touch-data/
314 one entry for each touch sensitive component, each entry of
315 which contains a /topology/ that specifies the distribution of
316 touch sensors, and the /data/, which is a vector of
317 =[activation, length]= pairs for each touch hair.
319 - =(proprioception! creature)= :: give the creature the sense of
320 proprioception. Returns a list of functions, one for each
321 joint, that when called during a running simulation will
322 report the =[headnig, pitch, roll]= of the joint.
324 - =(movement! creature)= :: give the creature the power of movement.
325 Creates a list of functions, one for each muscle, that when
326 called with an integer, will set the recruitment of that
327 muscle to that integer, and will report the current power
328 being exerted by the muscle. Order of muscles is determined by
329 the alphabetical sort order of the names of the muscle nodes.
331 *** Visualization/Debug
333 - =(view-vision)= :: create a function that when called with a list
334 of visual data returned from the functions made by =vision!=,
335 will display that visual data on the screen.
337 - =(view-hearing)= :: same as =view-vision= but for hearing.
339 - =(view-touch)= :: same as =view-vision= but for touch.
341 - =(view-proprioception)= :: same as =view-vision= but for
342 proprioception.
344 - =(view-movement)= :: same as =view-vision= but for
345 proprioception.
347 - =(view anything)= :: =view= is a polymorphic function that allows
348 you to inspect almost anything you could reasonably expect to
349 be able to ``see'' in =CORTEX=.
351 - =(text anything)= :: =text= is a polymorphic function that allows
352 you to convert practically anything into a text string.
354 - =(println-repl anything)= :: print messages to clojure's repl
355 instead of the simulation's terminal window.
357 - =(mega-import-jme3)= :: for experimenting at the REPL. This
358 function will import all jMonkeyEngine3 classes for immediate
359 use.
361 - =(display-dialated-time world timer)= :: Shows the time as it is
362 flowing in the simulation on a HUD display.