view org/body.org @ 302:7e3938f40c52

added winston letter, corrected source listing for integration
author Robert McIntyre <rlm@mit.edu>
date Fri, 17 Feb 2012 13:03:58 -0700
parents 23aadf376e9d
children 7e7f8d6d9ec5
line wrap: on
line source
1 #+title: Building a Body
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+description: Simulating a body (movement, touch, propioception) in jMonkeyEngine3.
5 #+SETUPFILE: ../../aurellem/org/setup.org
6 #+INCLUDE: ../../aurellem/org/level-0.org
8 * Design Constraints
10 I use [[www.blender.org/][blender]] to design bodies. The design of the bodies is
11 determined by the requirements of the AI that will use them. The
12 bodies must be easy for an AI to sense and control, and they must be
13 relatively simple for jMonkeyEngine to compute.
15 # I'm a secret test! :P
16 ** Bag of Bones
18 How to create such a body? One option I ultimately rejected is to use
19 blender's [[http://wiki.blender.org/index.php/Doc:2.6/Manual/Rigging/Armatures][armature]] system. The idea would have been to define a mesh
20 which describes the creature's entire body. To this you add an
21 skeleton which deforms this mesh. This technique is used extensively
22 to model humans and create realistic animations. It is hard to use for
23 my purposes because it is difficult to update the creature's Physics
24 Collision Mesh in tandem with its Geometric Mesh under the influence
25 of the armature. Withouth this the creature will not be able to grab
26 things in its environment, and it won't be able to tell where its
27 physical body is by using its eyes. Also, armatures do not specify
28 any rotational limits for a joint, making it hard to model elbows,
29 shoulders, etc.
31 ** EVE
33 Instead of using the human-like "deformable bag of bones" approach, I
34 decided to base my body plans on the robot EVE from the movie wall-E.
36 #+caption: EVE from the movie WALL-E. This body plan turns out to be much better suited to my purposes than a more human-like one.
37 [[../images/Eve.jpg]]
39 EVE's body is composed of several rigid components that are held
40 together by invisible joint constraints. This is what I mean by
41 "eve-like". The main reason that I use eve-style bodies is so that
42 there will be correspondence between the AI's vision and the physical
43 presence of its body. Each individual section is simulated by a
44 separate rigid body that corresponds exactly with its visual
45 representation and does not change. Sections are connected by
46 invisible joints that are well supported in jMonkyeEngine. Bullet, the
47 physics backend for jMonkeyEngine, can efficiently simulate hundreds
48 of rigid bodies connected by joints. Sections do not have to stay as
49 one piece forever; they can be dynamically replaced with multiple
50 sections to simulate splitting in two. This could be used to simulate
51 retractable claws or EVE's hands, which are able to coalece into one
52 object in the movie.
54 * Solidifying the Body
56 Here is a hand designed eve-style in blender.
58 #+attr_html: width="755"
59 [[../images/hand-screenshot0.png]]
61 If we load it directly into jMonkeyEngine, we get this:
63 #+name: test-1
64 #+begin_src clojure
65 (def hand-path "Models/test-creature/hand.blend")
67 (defn hand [] (load-blender-model hand-path))
69 (defn setup [world]
70 (let [cam (.getCamera world)]
71 (println-repl cam)
72 (.setLocation
73 cam (Vector3f.
74 -6.9015837, 8.644911, 5.6043186))
75 (.setRotation
76 cam
77 (Quaternion.
78 0.14046453, 0.85894054, -0.34301838, 0.3533118)))
79 (light-up-everything world)
80 (.setTimer world (RatchetTimer. 60))
81 world)
83 (defn test-hand-1
84 ([] (test-hand-1 false))
85 ([record?]
86 (world (hand)
87 standard-debug-controls
88 (fn [world]
89 (if record?
90 (Capture/captureVideo
91 world
92 (File. "/home/r/proj/cortex/render/body/1")))
93 (setup world)) no-op)))
94 #+end_src
97 #+begin_src clojure :results silent
98 (.start (cortex.test.body/test-one))
99 #+end_src
101 #+begin_html
102 <div class="figure">
103 <center>
104 <video controls="controls" width="640">
105 <source src="../video/ghost-hand.ogg" type="video/ogg"
106 preload="none" poster="../images/aurellem-1280x480.png" />
107 </video>
108 </center>
109 <p>The hand model directly loaded from blender. It has no physical
110 presense in the simulation. </p>
111 </div>
112 #+end_html
114 You will notice that the hand has no physical presence -- it's a
115 hologram through which everything passes. Therefore, the first thing
116 to do is to make it solid. Blender has physics simulation on par with
117 jMonkeyEngine (they both use bullet as their physics backend), but it
118 can be difficult to translate between the two systems, so for now I
119 specify the mass of each object as meta-data in blender and construct
120 the physics shape based on the mesh in jMonkeyEngine.
122 #+name: body-1
123 #+begin_src clojure
124 (defn physical!
125 "Iterate through the nodes in creature and make them real physical
126 objects in the simulation."
127 [#^Node creature]
128 (dorun
129 (map
130 (fn [geom]
131 (let [physics-control
132 (RigidBodyControl.
133 (HullCollisionShape.
134 (.getMesh geom))
135 (if-let [mass (meta-data geom "mass")]
136 (do
137 (println-repl
138 "setting" (.getName geom) "mass to" (float mass))
139 (float mass))
140 (float 1)))]
141 (.addControl geom physics-control)))
142 (filter #(isa? (class %) Geometry )
143 (node-seq creature)))))
144 #+end_src
146 =physical!)= iterates through a creature's node structure, creating
147 CollisionShapes for each geometry with the mass specified in that
148 geometry's meta-data.
150 #+name: test-2
151 #+begin_src clojure
152 (in-ns 'cortex.test.body)
154 (def gravity-control
155 {"key-g" (fn [world _]
156 (set-gravity world (Vector3f. 0 -9.81 0)))
157 "key-u" (fn [world _] (set-gravity world Vector3f/ZERO))})
160 (defn floor []
161 (box 10 3 10 :position (Vector3f. 0 -10 0)
162 :color ColorRGBA/Gray :mass 0))
164 (defn test-hand-2
165 ([] (test-hand-2 false))
166 ([record?]
167 (world
168 (nodify
169 [(doto (hand)
170 (physical!))
171 (floor)])
172 (merge standard-debug-controls gravity-control)
173 (fn [world]
174 (if record?
175 (Capture/captureVideo
176 world (File. "/home/r/proj/cortex/render/body/2")))
177 (set-gravity world Vector3f/ZERO)
178 (setup world))
179 no-op)))
180 #+end_src
182 #+begin_html
183 <div class="figure">
184 <center>
185 <video controls="controls" width="640">
186 <source src="../video/crumbly-hand.ogg" type="video/ogg"
187 preload="none" poster="../images/aurellem-1280x480.png" />
188 </video>
189 </center>
190 <p>The hand now has a physical presence, but there is nothing to hold
191 it together.</p>
192 </div>
193 #+end_html
195 Now that's some progress.
197 * Joints
199 Obviously, an AI is not going to be doing much while lying in pieces
200 on the floor. So, the next step to making a proper body is to connect
201 those pieces together with joints. jMonkeyEngine has a large array of
202 joints available via bullet, such as Point2Point, Cone, Hinge, and a
203 generic Six Degree of Freedom joint, with or without spring
204 restitution.
206 Although it should be possible to specify the joints using blender's
207 physics system, and then automatically import them with jMonkeyEngine,
208 the support isn't there yet, and there are a few problems with bullet
209 itself that need to be solved before it can happen.
211 So, I will use the same system for specifying joints as I will do for
212 some senses. Each joint is specified by an empty node whose parent
213 has the name "joints". Their orientation and meta-data determine what
214 joint is created.
216 #+attr_html: width="755"
217 #+caption: Joints hack in blender. Each empty node here will be transformed into a joint in jMonkeyEngine
218 [[../images/hand-screenshot1.png]]
220 The empty node in the upper right, highlighted in yellow, is the
221 parent node of all the emptys which represent joints. The following
222 functions must do three things to translate these into real joints:
224 - Find the children of the "joints" node.
225 - Determine the two spatials the joint it meant to connect.
226 - Create the joint based on the meta-data of the empty node.
228 ** Finding the Joints
230 The higher order function =sense-nodes= from =cortex.sense= simplifies
231 the first task.
233 #+name: joints-2
234 #+begin_src clojure
235 (defvar
236 ^{:arglists '([creature])}
237 joints
238 (sense-nodes "joints")
239 "Return the children of the creature's \"joints\" node.")
240 #+end_src
242 ** Joint Targets and Orientation
244 This technique for finding a joint's targets is very similiar to
245 =cortex.sense/closest-node=. A small cube, centered around the
246 empty-node, grows exponentially until it intersects two /physical/
247 objects. The objects are ordered according to the joint's rotation,
248 with the first one being the object that has more negative coordinates
249 in the joint's reference frame. Since the objects must be physical,
250 the empty-node itself escapes detection. Because the objects must be
251 physical, =joint-targets= must be called /after/ =physical!= is
252 called.
254 #+name: joints-3
255 #+begin_src clojure
256 (defn joint-targets
257 "Return the two closest two objects to the joint object, ordered
258 from bottom to top according to the joint's rotation."
259 [#^Node parts #^Node joint]
260 (loop [radius (float 0.01)]
261 (let [results (CollisionResults.)]
262 (.collideWith
263 parts
264 (BoundingBox. (.getWorldTranslation joint)
265 radius radius radius) results)
266 (let [targets
267 (distinct
268 (map #(.getGeometry %) results))]
269 (if (>= (count targets) 2)
270 (sort-by
271 #(let [joint-ref-frame-position
272 (jme-to-blender
273 (.mult
274 (.inverse (.getWorldRotation joint))
275 (.subtract (.getWorldTranslation %)
276 (.getWorldTranslation joint))))]
277 (.dot (Vector3f. 1 1 1) joint-ref-frame-position))
278 (take 2 targets))
279 (recur (float (* radius 2))))))))
280 #+end_src
282 ** Generating Joints
284 This section of code iterates through all the different ways of
285 specifying joints using blender meta-data and converts each one to the
286 appropriate jMonkyeEngine joint.
288 #+name: joints-4
289 #+begin_src clojure
290 (defmulti joint-dispatch
291 "Translate blender pseudo-joints into real JME joints."
292 (fn [constraints & _]
293 (:type constraints)))
295 (defmethod joint-dispatch :point
296 [constraints control-a control-b pivot-a pivot-b rotation]
297 (println-repl "creating POINT2POINT joint")
298 ;; bullet's point2point joints are BROKEN, so we must use the
299 ;; generic 6DOF joint instead of an actual Point2Point joint!
301 ;; should be able to do this:
302 (comment
303 (Point2PointJoint.
304 control-a
305 control-b
306 pivot-a
307 pivot-b))
309 ;; but instead we must do this:
310 (println-repl "substuting 6DOF joint for POINT2POINT joint!")
311 (doto
312 (SixDofJoint.
313 control-a
314 control-b
315 pivot-a
316 pivot-b
317 false)
318 (.setLinearLowerLimit Vector3f/ZERO)
319 (.setLinearUpperLimit Vector3f/ZERO)))
321 (defmethod joint-dispatch :hinge
322 [constraints control-a control-b pivot-a pivot-b rotation]
323 (println-repl "creating HINGE joint")
324 (let [axis
325 (if-let
326 [axis (:axis constraints)]
327 axis
328 Vector3f/UNIT_X)
329 [limit-1 limit-2] (:limit constraints)
330 hinge-axis
331 (.mult
332 rotation
333 (blender-to-jme axis))]
334 (doto
335 (HingeJoint.
336 control-a
337 control-b
338 pivot-a
339 pivot-b
340 hinge-axis
341 hinge-axis)
342 (.setLimit limit-1 limit-2))))
344 (defmethod joint-dispatch :cone
345 [constraints control-a control-b pivot-a pivot-b rotation]
346 (let [limit-xz (:limit-xz constraints)
347 limit-xy (:limit-xy constraints)
348 twist (:twist constraints)]
350 (println-repl "creating CONE joint")
351 (println-repl rotation)
352 (println-repl
353 "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0)))
354 (println-repl
355 "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0)))
356 (println-repl
357 "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1)))
358 (doto
359 (ConeJoint.
360 control-a
361 control-b
362 pivot-a
363 pivot-b
364 rotation
365 rotation)
366 (.setLimit (float limit-xz)
367 (float limit-xy)
368 (float twist)))))
370 (defn connect
371 "Create a joint between 'obj-a and 'obj-b at the location of
372 'joint. The type of joint is determined by the metadata on 'joint.
374 Here are some examples:
375 {:type :point}
376 {:type :hinge :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}
377 (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
379 {:type :cone :limit-xz 0]
380 :limit-xy 0]
381 :twist 0]} (use XZY rotation mode in blender!)"
382 [#^Node obj-a #^Node obj-b #^Node joint]
383 (let [control-a (.getControl obj-a RigidBodyControl)
384 control-b (.getControl obj-b RigidBodyControl)
385 joint-center (.getWorldTranslation joint)
386 joint-rotation (.toRotationMatrix (.getWorldRotation joint))
387 pivot-a (world-to-local obj-a joint-center)
388 pivot-b (world-to-local obj-b joint-center)]
390 (if-let [constraints
391 (map-vals
392 eval
393 (read-string
394 (meta-data joint "joint")))]
395 ;; A side-effect of creating a joint registers
396 ;; it with both physics objects which in turn
397 ;; will register the joint with the physics system
398 ;; when the simulation is started.
399 (do
400 (println-repl "creating joint between"
401 (.getName obj-a) "and" (.getName obj-b))
402 (joint-dispatch constraints
403 control-a control-b
404 pivot-a pivot-b
405 joint-rotation))
406 (println-repl "could not find joint meta-data!"))))
407 #+end_src
409 Creating joints is now a matter of applying =connect= to each joint
410 node.
412 #+name: joints-5
413 #+begin_src clojure
414 (defn joints!
415 "Connect the solid parts of the creature with physical joints. The
416 joints are taken from the \"joints\" node in the creature."
417 [#^Node creature]
418 (dorun
419 (map
420 (fn [joint]
421 (let [[obj-a obj-b] (joint-targets creature joint)]
422 (connect obj-a obj-b joint)))
423 (joints creature))))
424 #+end_src
426 ** Round 3
428 Now we can test the hand in all its glory.
430 #+name: test-3
431 #+begin_src clojure
432 (in-ns 'cortex.test.body)
434 (def debug-control
435 {"key-h" (fn [world val]
436 (if val (enable-debug world)))})
438 (defn test-hand-3
439 ([] (test-hand-3 false))
440 ([record?]
441 (world
442 (nodify
443 [(doto (hand)
444 (physical!)
445 (joints!))
446 (floor)])
447 (merge standard-debug-controls debug-control
448 gravity-control)
449 (comp
450 #(Capture/captureVideo
451 % (File. "/home/r/proj/cortex/render/body/3"))
452 #(do (set-gravity % Vector3f/ZERO) %)
453 setup)
454 no-op)))
455 #+end_src
457 =physical!= makes the hand solid, then =joints!= connects each
458 piece together.
460 #+begin_html
461 <div class="figure">
462 <center>
463 <video controls="controls" width="640">
464 <source src="../video/full-hand.ogg" type="video/ogg"
465 preload="none" poster="../images/aurellem-1280x480.png" />
466 </video>
467 </center>
468 <p>Now the hand is physical and has joints.</p>
469 </div>
470 #+end_html
472 The joints are visualized as green connections between each segment
473 for debug purposes. You can see that they correspond to the empty
474 nodes in the blender file.
476 * Wrap-Up!
478 It is convienent to combine =physical!= and =joints!= into one
479 function that completely creates the creature's physical body.
481 #+name: joints-6
482 #+begin_src clojure
483 (defn body!
484 "Endow the creature with a physical body connected with joints. The
485 particulars of the joints and the masses of each pody part are
486 determined in blender."
487 [#^Node creature]
488 (physical! creature)
489 (joints! creature))
490 #+end_src
492 * The Worm
494 Going forward, I will use a model that is less complicated than the
495 hand. It has two segments and one joint, and I call it the worm. All
496 of the senses described in the following posts will be applied to this
497 worm.
499 #+name: test-4
500 #+begin_src clojure
501 (in-ns 'cortex.test.body)
503 (defn worm []
504 (load-blender-model
505 "Models/test-creature/worm.blend"))
507 (defn test-worm
508 ([] (test-worm false))
509 ([record?]
510 (let [timer (RatchetTimer. 60)]
511 (world
512 (nodify
513 [(doto (worm)
514 (body!))
515 (floor)])
516 (merge standard-debug-controls debug-control)
517 #(do
518 (speed-up %)
519 (light-up-everything %)
520 (.setTimer % timer)
521 (cortex.util/display-dialated-time % timer)
522 (if record?
523 (Capture/captureVideo
524 % (File. "/home/r/proj/cortex/render/body/4"))))
525 no-op))))
526 #+end_src
528 #+begin_html
529 <div class="figure">
530 <center>
531 <video controls="controls" width="640">
532 <source src="../video/worm-1.ogg" type="video/ogg"
533 preload="none" poster="../images/aurellem-1280x480.png" />
534 </video>
535 </center>
536 <p>This worm model will be the platform onto which future senses will
537 be grafted.</p>
538 </div>
539 #+end_html
541 * Headers
542 #+name: body-header
543 #+begin_src clojure
544 (ns cortex.body
545 "Assemble a physical creature using the definitions found in a
546 specially prepared blender file. Creates rigid bodies and joints so
547 that a creature can have a physical presense in the simulation."
548 {:author "Robert McIntyre"}
549 (:use (cortex world util sense))
550 (:use clojure.contrib.def)
551 (:import
552 (com.jme3.math Vector3f Quaternion Vector2f Matrix3f)
553 (com.jme3.bullet.joints
554 SixDofJoint Point2PointJoint HingeJoint ConeJoint)
555 com.jme3.bullet.control.RigidBodyControl
556 com.jme3.collision.CollisionResults
557 com.jme3.bounding.BoundingBox
558 com.jme3.scene.Node
559 com.jme3.scene.Geometry
560 com.jme3.bullet.collision.shapes.HullCollisionShape))
561 #+end_src
563 #+name: test-header
564 #+begin_src clojure
565 (ns cortex.test.body
566 (:use (cortex world util body))
567 (:import
568 (com.aurellem.capture Capture RatchetTimer)
569 (com.jme3.math Quaternion Vector3f ColorRGBA)
570 java.io.File))
571 #+end_src
573 * Source
574 - [[../src/cortex/body.clj][cortex.body]]
575 - [[../src/cortex/test/body.clj][cortex.test.body]]
576 - [[../assets/Models/test-creature/hand.blend][hand.blend]]
577 - [[../assets/Models/test-creature/palm.png][UV-map-1]]
578 - [[../assets/Models/test-creature/worm.blend][worm.blend]]
579 - [[../assets/Models/test-creature/retina-small.png][UV-map-1]]
580 - [[../assets/Models/test-creature/tip.png][UV-map-2]]
581 #+html: <ul> <li> <a href="../org/body.org">This org file</a> </li> </ul>
582 - [[http://hg.bortreb.com ][source-repository]]
584 * Next
585 The body I have made here exists without any senses or effectors. In
586 the [[./vision.org][next post]], I'll give the creature eyes.
588 * COMMENT Generate Source
589 #+begin_src clojure :tangle ../src/cortex/body.clj
590 <<body-header>>
591 <<body-1>>
592 <<joints-2>>
593 <<joints-3>>
594 <<joints-4>>
595 <<joints-5>>
596 <<joints-6>>
597 #+end_src
599 #+begin_src clojure :tangle ../src/cortex/test/body.clj
600 <<test-header>>
601 <<test-1>>
602 <<test-2>>
603 <<test-3>>
604 <<test-4>>
605 #+end_src