Mercurial > cortex

comparison thesis/user-guide.org @ 486:6d460ac3f5d0

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
working on user guide.
author Robert McIntyre <rlm@mit.edu>
date Sat, 29 Mar 2014 20:12:52 -0400
parents c1e6b7221b2f
children 19b55aaf4462
comparison
equal deleted inserted replaced
485:ac953b562eab 486:6d460ac3f5d0
1 * Appendix: =CORTEX= User Guide 1 * Appendix: =CORTEX= User Guide
2 2
3 For future students who whould like to use =CORTEX= in their own 3 Those who write a thesis should endeavor to make their code not only
4 projects. 4 accessable, but actually useable, as a way to pay back the community
5 5 that made the thesis possible in the first place. This thesis would
6 6 not be possible without Free Software such as jMonkeyEngine3,
7 Blender, clojure, emacs, ffmpeg, and many other tools. That is why I
8 have included this user guide, in the hope that someone else might
9 find =CORTEX= useful.
10
11 ** Obtaining =CORTEX=
12
13 You can get cortex from its mercurial repository at
14 http://hg.bortreb.com/cortex. You may also download =CORTEX=
15 releases at http://aurellem.org/cortex/releases/. As a condition of
16 making this thesis, I have also provided Professor Winston the
17 =CORTEX= source, and he knows how to run the demos and get started.
18 You may also email me at =cortex@aurellem.org= and I may help where
19 I can.
20
21 ** Running =CORTEX=
22
23 =CORTEX= comes with README and INSTALL files that will guide you
24 through installation and running the test suite. In particular you
25 should look at test =cortex.test= which contains test suites that
26 run through all senses and multiple creatures.
27
28 ** Creating creatures
29
30 Creatures are created using /Blender/, a free 3D modeling program.
31 You will need Blender version 2.6 when using the =CORTEX= included
32 in this thesis. You create a =CORTEX= creature in a similiar manner
33 to modeling anything in Blender, except that you also create
34 several trees of empty nodes which define the creature's senses.
35
36 *** Mass
37
38 To give an object mass in =CORTEX=, add a ``mass'' metadata label
39 to the object with the mass in jMonkeyEngine units. Note that
40 setting the mass to 0 causes the object to be immovable.
41
42 *** Joints
43
44 Joints are created by creating an empty node named =joints= and
45 then creating any number of empty child nodes to represent your
46 creature's joints. The joint will automatically connect the
47 closest two physical objects. It will help to set the empty node's
48 display mode to ``Arrows'' so that you can clearly see the
49 direction of the axes.
50
51 Joint nodes should have the following metadata under the ``joint''
52 label:
53
54 #+BEGIN_SRC clojure
55 ;; ONE OF the following, under the label "joint":
56 {:type :point}
57
58 ;; OR
59
60 {:type :hinge
61 :limit [<limit-low> <limit-high>]
62 :axis (Vector3f. <x> <y> <z>)}
63 ;;(:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
64
65 ;; OR
66
67 {:type :cone
68 :limit-xz <lim-xz>
69 :limit-xy <lim-xy>
70 :twist <lim-twist>} ;(use XZY rotation mode in blender!)
71 #+END_SRC
72
73 *** Eyes
74
75 Eyes are created by creating an empty node named =eyes= and then
76 creating any number of empty child nodes to represent your
77 creature's eyes.
78
79 Eye nodes should have the following metadata under the ``eye''
80 label:
81
82 #+BEGIN_SRC clojure
83 {:red <red-retina-definition>
84 :blue <blue-retina-definition>
85 :green <green-retina-definition>
86 :all <all-retina-definition>
87 (<0xrrggbb> <custom-retina-image>)...
88 }
89 #+END_SRC
90
91 Any of the color channels may be omitted. You may also include
92 your own color selectors, and in fact :red is equivalent to
93 0xFF0000 and so forth. The eye will be placed at the same position
94 as the empty node and will bind to the neatest physical object.
95 The eye will point outward from the X-axis of the node, and ``up''
96 will be in the direction of the X-axis of the node. It will help
97 to set the empty node's display mode to ``Arrows'' so that you can
98 clearly see the direction of the axes.
99
100 Each retina file should contain white pixels whever you want to be
101 sensitive to your chosen color. If you want the entire field of
102 view, specify :all of 0xFFFFFF and a retinal map that is entirely
103 white.
104
105 Here is a sample retinal map:
106
107 #+caption: An example retinal profile image. White pixels are
108 #+caption: photo-sensitive elements. The distribution of white
109 #+caption: pixels is denser in the middle and falls off at the
110 #+caption: edges and is inspired by the human retina.
111 #+name: retina
112 #+ATTR_LaTeX: :width 7cm :placement [H]
113 [[./images/retina-small.png]]
114
115 *** Hearing
116
117 Ears are created by creating an empty node named =ears= and then
118 creating any number of empty child nodes to represent your
119 creature's ears.
120
121 Ear nodes do not require any metadata.
122
123 The ear will bind to and follow the closest physical node.
124
125 *** Touch
126
127 Touch is handled similarly to mass. To make a particular object
128 touch sensitive, add metadata of the following form under the
129 object's ``touch'' metadata field:
130
131 #+BEGIN_EXAMPLE
132 <touch-UV-map-file-name>
133 #+END_EXAMPLE
134
135 You may also include an optional ``scale'' metadata number to
136 specifiy the length of the touch feelers. The default is $0.1$,
137 and this is generally sufficient.
138
139 The touch UV should contain white pixels for each touch sensor.
140
141 Here is an example touch-uv map that approximates a human finger,
142 and its corresponding model.
143
144 #+caption: This is the tactile-sensor-profile for the upper segment
145 #+caption: of a fingertip. It defines regions of high touch sensitivity
146 #+caption: (where there are many white pixels) and regions of low
147 #+caption: sensitivity (where white pixels are sparse).
148 #+name: guide-fingertip-UV
149 #+ATTR_LaTeX: :width 9cm :placement [H]
150 [[./images/finger-UV.png]]
151
152 #+caption: The fingertip UV-image form above applied to a simple
153 #+caption: model of a fingertip.
154 #+name: guide-fingertip
155 #+ATTR_LaTeX: :width 9cm :placement [H]
156 [[./images/finger-2.png]]
157
158 *** Propriocepotion
159
160 Proprioception is tied to each joint node -- nothing special must
161 be done in a blender model to enable proprioception other than
162 creating joint nodes.
163
164 *** Muscles
165
166 Muscles are created by creating an empty node named =muscles= and
167 then creating any number of empty child nodes to represent your
168 creature's muscles.
169
170
171 Muscle nodes should have the following metadata under the
172 ``muscle'' label:
173
174 #+BEGIN_EXAMPLE
175 <muscle-profile-file-name>
176 #+END_EXAMPLE
177
178 Muscles should also have a ``strength'' metadata entry describing
179 the muscle's total strength at full activation.
180
181 Muscle profiles are simple images that contain the relative amount
182 of muscle power in each simulated alpha motor neuron. The width of
183 the image is the total size of the motor pool, and the redness of
184 each neuron is the relative power of that motor pool.
185
186 While the profile image can have any dimensions, only the first
187 line of pixels is used to define the muscle. Here is a sample
188 muscle profile image that defines a human-like muscle.
189
190 #+caption: A muscle profile image that describes the strengths
191 #+caption: of each motor neuron in a muscle. White is weakest
192 #+caption: and dark red is strongest. This particular pattern
193 #+caption: has weaker motor neurons at the beginning, just
194 #+caption: like human muscle.
195 #+name: muscle-recruit
196 #+ATTR_LaTeX: :width 7cm :placement [H]
197 [[./images/basic-muscle.png]]
198
199 Muscles twist the nearest physical object about the muscle node's
200 Z-axis. I recommend using the ``Single Arrow'' display mode for
201 muscles and using the right hand rule to determine which way the
202 muscle will twist. To make a segment that can twist in multiple
203 directions, create multiple, differently aligned muscles.
204
205 ** =CORTEX= API
206
207 These are the some functions exposed by =CORTEX= for creating
208 worlds and simulating creatures. These are in addition to
209 jMonkeyEngine3's extensive library, which is documented elsewhere.
210
211 *** Simulation
212 - =(world root-node key-map setup-fn update-fn)= :: create
213 a simulation.
214 - /root-node/ :: a =com.jme3.scene.Node= object which
215 contains all of the objects that should be in the
216 simulation.
217
218 - /key-map/ :: a map from strings describing keys to
219 functions that should be executed whenever that key is
220 pressed. the functions should take a SimpleApplication
221 object and a boolean value. The SimpleApplication is the
222 current simulation that is running, and the boolean is true
223 if the key is being pressed, and false if it is being
224 released. As an example,
225 #+BEGIN_SRC clojure
226 {"key-j" (fn [game value] (if value (println "key j pressed")))}
227 #+END_SRC
228 is a valid key-map which will cause the simulation to print
229 a message whenever the 'j' key on the keyboard is pressed.
230
231 - /setup-fn/ :: a function that takes a =SimpleApplication=
232 object. It is called once when initializing the simulation.
233 Use it to create things like lights, change the gravity,
234 initialize debug nodes, etc.
235
236 - /update-fn/ :: this function takes a =SimpleApplication=
237 object and a float and is called every frame of the
238 simulation. The float tells how many seconds is has been
239 since the last frame was rendered, according to whatever
240 clock jme is currently using. The default is to use IsoTimer
241 which will result in this value always being the same.
242
243
244 - =(position-camera world position rotation)= :: set the position
245 of the simulation's main camera.
246 - =(enable-debug world)= :: turn on debug wireframes for each
247 simulated object.
248 - =(set-gravity world gravity)= :: set the gravity of a running
249 simulation.
250 - =(box length width height & {options})= :: create a box in the
251 simulation. Options is a hash map specifying texture, mass,
252 etc. Possible options are =:name=, =:color=, =:mass=,
253 =:friction=, =:texture=, =:material=, =:position=,
254 =:rotation=, =:shape=, and =:physical?=.
255 - =(sphere radius & {options})= :: create a sphere in the simulation.
256 Options are the same as in =box=.
257 - =(load-blender-model file-name)= :: create a node structure
258 representing that described in a blender file.
259 - =(light-up-everything world)= :: distribute a standard compliment
260 of lights throught the simulation. Should be adequate for most
261 purposes.
262 - =(node-seq node)= :: return a recursuve list of the node's
263 children.
264 - =(nodify name children)= :: construct a node given a node-name and
265 desired children.
266 - =(add-element world element)= :: add an object to a running world
267 simulation.
268 - =(set-accuracy world accuracy)= :: change the accuracy of the
269 world's physics simulator.
270 - =(asset-manager)= :: get an /AssetManager/, a jMonkeyEngine
271 construct that is useful for loading textures and is required
272 for smooth interaction with jMonkeyEngine library functions.
273
274
275 *** Creature Manipulation / Import
276
277 - =(body! creature)= :: give the creature a physical body.
278 - =(vision! creature)= :: give the creature a sense of vision.
279 Returns a list of functions which will each, when called
280 during a simulation, return the vision data for the channel of
281 one of the eyes. The functions are ordered depending on the
282 alphabetical order of the names of the eye nodes in the
283 blender file. The data returned by the functions is a vector
284 containing the eye's /topology/, a vector of coordinates, and
285 the eye's /data/, a vector of RGB values filtered by the eye's
286 sensitivity.
287 - =(hearing! creature)= :: give the creature a sense of hearing.
288 Returns a list of functions, one for each ear, that when
289 called will return a frame's worth of hearing data for that
290 ear. The functions are ordered depending on the alphabetical
291 order of the names of the ear nodes in the blender file. The
292 data returned by the functions is an array PCM encoded wav
293 data.
294 - =(touch! creature)= :: give the creature a sense of touch. Returns
295 a single function that must be called with the /root node/ of
296 the world, and which will return a vector of /touch-data/
297 one entry for each touch sensitive component, each entry of
298 which contains a /topology/ that specifies the distribution of
299 touch sensors, and the /data/, which is a vector of
300 =[activation, length]= pairs for each touch hair.
301 - =(proprioception! creature)= :: give the creature the sense of
302 proprioception. Returns a list of functions, one for each
303 joint, that when called during a running simulation will
304 report the =[headnig, pitch, roll]= of the joint.
305 - =(movement! creature)= :: give the creature the power of movement.
306 Creates a list of functions, one for each muscle, that when
307 called with an integer, will set the recruitment of that
308 muscle to that integer, and will report the current power
309 being exerted by the muscle. Order of muscles is determined by
310 the alphabetical sort order of the names of the muscle nodes.
311
312 *** Visualization/Debug
313
314 - =(view-vision)= :: create a function that when called with a list
315 of visual data returned from the functions made by =vision!=,
316 will display that visual data on the screen.
317 - =(view-hearing)= :: same as =view-vision= but for hearing.
318 - =(view-touch)= :: same as =view-vision= but for touch.
319 - =(view-proprioception)= :: same as =view-vision= but for
320 proprioception.
321 - =(view-movement)= :: same as =view-vision= but for
322 proprioception.
323 - =(view anything)= :: =view= is a polymorphic function that allows
324 you to inspect almost anything you could reasonably expect to
325 be able to ``see'' in =CORTEX=.
326 - =(text anything)= :: =text= is a polymorphic function that allows
327 you to convert practically anything into a text string.
328 - =(println-repl anything)= :: print messages to clojure's repl
329 instead of the simulation's terminal window.
330
331 *** Misc
332
333 - =(load-bullet)= :: unpack native libraries and initialize
334 blender. This function is required before other world building
335 functions are called.
336 - =(mega-import-jme3)= :: for experimenting at the REPL. This
337 function will import all jMonkeyEngine3 classes for immediate
338 use.