view org/body.org @ 269:bbd787e12025

added idea
author Robert McIntyre <rlm@mit.edu>
date Tue, 14 Feb 2012 17:54:32 -0600
parents 63dafe7365df
children c39b8b29a79e
line wrap: on
line source
1 #+title: Building a Body
2 #+author: Robert McIntyre
3 #+email: rlm@mit.edu
4 #+description: Simulating a body (movement, touch, propioception) in jMonkeyEngine3.
5 #+SETUPFILE: ../../aurellem/org/setup.org
6 #+INCLUDE: ../../aurellem/org/level-0.org
8 * Design Constraints
10 I use [[www.blender.org/][blender]] to design bodies. The design of the bodies is
11 determined by the requirements of the AI that will use them. The
12 bodies must be easy for an AI to sense and control, and they must be
13 relatively simple for jMonkeyEngine to compute.
15 # I'm a secret test! :P
16 ** Bag of Bones
18 How to create such a body? One option I ultimately rejected is to use
19 blender's [[http://wiki.blender.org/index.php/Doc:2.6/Manual/Rigging/Armatures][armature]] system. The idea would have been to define a mesh
20 which describes the creature's entire body. To this you add an
21 (skeleton) which deforms this mesh. This technique is used extensively
22 to model humans and create realistic animations. It is hard to use for
23 my purposes because it is difficult to update the creature's Physics
24 Collision Mesh in tandem with its Geometric Mesh under the influence
25 of the armature. Withouth this the creature will not be able to grab
26 things in its environment, and it won't be able to tell where its
27 physical body is by using its eyes. Also, armatures do not specify
28 any rotational limits for a joint, making it hard to model elbows,
29 shoulders, etc.
31 ** EVE
33 Instead of using the human-like "deformable bag of bones" approach, I
34 decided to base my body plans on the robot EVE from the movie wall-E.
36 #+caption: EVE from the movie WALL-E. This body plan turns out to be much better suited to my purposes than a more human-like one.
37 [[../images/Eve.jpg]]
39 EVE's body is composed of several rigid components that are held
40 together by invisible joint constraints. This is what I mean by
41 "eve-like". The main reason that I use eve-style bodies is so that
42 there will be correspondence between the AI's vision and the physical
43 presence of its body. Each individual section is simulated by a
44 separate rigid body that corresponds exactly with its visual
45 representation and does not change. Sections are connected by
46 invisible joints that are well supported in jMonkyeEngine. Bullet, the
47 physics backend for jMonkeyEngine, can efficiently simulate hundreds
48 of rigid bodies connected by joints. Sections do not have to stay as
49 one piece forever; they can be dynamically replaced with multiple
50 sections to simulate splitting in two. This could be used to simulate
51 retractable claws or EVE's hands, which are able to coalece into one
52 object in the movie.
54 * Solidifying the Body
56 Here is a hand designed eve-style in blender.
58 #+attr_html: width="755"
59 [[../images/hand-screenshot0.png]]
61 If we load it directly into jMonkeyEngine, we get this:
63 #+name: test-1
64 #+begin_src clojure
65 (def hand-path "Models/test-creature/hand.blend")
67 (defn hand [] (load-blender-model hand-path))
69 (defn setup [world]
70 (let [cam (.getCamera world)]
71 (println-repl cam)
72 (.setLocation
73 cam (Vector3f.
74 -6.9015837, 8.644911, 5.6043186))
75 (.setRotation
76 cam
77 (Quaternion.
78 0.14046453, 0.85894054, -0.34301838, 0.3533118)))
79 (light-up-everything world)
80 (.setTimer world (RatchetTimer. 60))
81 world)
83 (defn test-one []
84 (world (hand)
85 standard-debug-controls
86 (comp
87 #(Capture/captureVideo
88 % (File. "/home/r/proj/cortex/render/body/1"))
89 setup)
90 no-op))
91 #+end_src
94 #+begin_src clojure :results silent
95 (.start (cortex.test.body/test-one))
96 #+end_src
98 #+begin_html
99 <div class="figure">
100 <center>
101 <video controls="controls" width="640">
102 <source src="../video/ghost-hand.ogg" type="video/ogg"
103 preload="none" poster="../images/aurellem-1280x480.png" />
104 </video>
105 </center>
106 <p>The hand model directly loaded from blender. It has no physical
107 presense in the simulation. </p>
108 </div>
109 #+end_html
111 You will notice that the hand has no physical presence -- it's a
112 hologram through which everything passes. Therefore, the first thing
113 to do is to make it solid. Blender has physics simulation on par with
114 jMonkeyEngine (they both use bullet as their physics backend), but it
115 can be difficult to translate between the two systems, so for now I
116 specify the mass of each object as meta-data in blender and construct
117 the physics shape based on the mesh in jMonkeyEngine.
119 #+name: body-1
120 #+begin_src clojure
121 (defn physical!
122 "Iterate through the nodes in creature and make them real physical
123 objects in the simulation."
124 [#^Node creature]
125 (dorun
126 (map
127 (fn [geom]
128 (let [physics-control
129 (RigidBodyControl.
130 (HullCollisionShape.
131 (.getMesh geom))
132 (if-let [mass (meta-data geom "mass")]
133 (do
134 (println-repl
135 "setting" (.getName geom) "mass to" (float mass))
136 (float mass))
137 (float 1)))]
138 (.addControl geom physics-control)))
139 (filter #(isa? (class %) Geometry )
140 (node-seq creature)))))
141 #+end_src
143 =(physical!)= iterates through a creature's node structure, creating
144 CollisionShapes for each geometry with the mass specified in that
145 geometry's meta-data.
147 #+name: test-2
148 #+begin_src clojure
149 (in-ns 'cortex.test.body)
151 (def gravity-control
152 {"key-g" (fn [world _]
153 (set-gravity world (Vector3f. 0 -9.81 0)))
154 "key-u" (fn [world _] (set-gravity world Vector3f/ZERO))})
157 (defn floor []
158 (box 10 3 10 :position (Vector3f. 0 -10 0)
159 :color ColorRGBA/Gray :mass 0))
161 (defn test-two []
162 (world (nodify
163 [(doto (hand)
164 (physical!))
165 (floor)])
166 (merge standard-debug-controls gravity-control)
167 (comp
168 #(Capture/captureVideo
169 % (File. "/home/r/proj/cortex/render/body/2"))
170 #(do (set-gravity % Vector3f/ZERO) %)
171 setup)
172 no-op))
173 #+end_src
175 #+begin_html
176 <div class="figure">
177 <center>
178 <video controls="controls" width="640">
179 <source src="../video/crumbly-hand.ogg" type="video/ogg"
180 preload="none" poster="../images/aurellem-1280x480.png" />
181 </video>
182 </center>
183 <p>The hand now has a physical presence, but there is nothing to hold
184 it together.</p>
185 </div>
186 #+end_html
188 Now that's some progress.
190 * Joints
192 Obviously, an AI is not going to be doing much while lying in pieces
193 on the floor. So, the next step to making a proper body is to connect
194 those pieces together with joints. jMonkeyEngine has a large array of
195 joints available via bullet, such as Point2Point, Cone, Hinge, and a
196 generic Six Degree of Freedom joint, with or without spring
197 restitution.
199 Although it should be possible to specify the joints using blender's
200 physics system, and then automatically import them with jMonkeyEngine,
201 the support isn't there yet, and there are a few problems with bullet
202 itself that need to be solved before it can happen.
204 So, I will use the same system for specifying joints as I will do for
205 some senses. Each joint is specified by an empty node whose parent
206 has the name "joints". Their orientation and meta-data determine what
207 joint is created.
209 #+attr_html: width="755"
210 #+caption: Joints hack in blender. Each empty node here will be transformed into a joint in jMonkeyEngine
211 [[../images/hand-screenshot1.png]]
213 The empty node in the upper right, highlighted in yellow, is the
214 parent node of all the emptys which represent joints. The following
215 functions must do three things to translate these into real joints:
217 - Find the children of the "joints" node.
218 - Determine the two spatials the joint it meant to connect.
219 - Create the joint based on the meta-data of the empty node.
221 ** Finding the Joints
223 The higher order function =(sense-nodes)= from =cortex.sense= simplifies
224 the first task.
226 #+name: joints-2
227 #+begin_src clojure
228 (defvar
229 ^{:arglists '([creature])}
230 joints
231 (sense-nodes "joints")
232 "Return the children of the creature's \"joints\" node.")
233 #+end_src
236 ** Joint Targets and Orientation
238 This technique for finding a joint's targets is very similiar to
239 =(cortex.sense/closest-node)=. A small cube, centered around the
240 empty-node, grows exponentially until it intersects two /physical/
241 objects. The objects are ordered according to the joint's rotation,
242 with the first one being the object that has more negative coordinates
243 in the joint's reference frame. Since the objects must be physical,
244 the empty-node itself escapes detection. Because the objects must be
245 physical, =(joint-targets)= must be called /after/ =(physical!)= is
246 called.
248 #+name: joints-3
249 #+begin_src clojure
250 (defn joint-targets
251 "Return the two closest two objects to the joint object, ordered
252 from bottom to top according to the joint's rotation."
253 [#^Node parts #^Node joint]
254 (loop [radius (float 0.01)]
255 (let [results (CollisionResults.)]
256 (.collideWith
257 parts
258 (BoundingBox. (.getWorldTranslation joint)
259 radius radius radius) results)
260 (let [targets
261 (distinct
262 (map #(.getGeometry %) results))]
263 (if (>= (count targets) 2)
264 (sort-by
265 #(let [joint-ref-frame-position
266 (jme-to-blender
267 (.mult
268 (.inverse (.getWorldRotation joint))
269 (.subtract (.getWorldTranslation %)
270 (.getWorldTranslation joint))))]
271 (.dot (Vector3f. 1 1 1) joint-ref-frame-position))
272 (take 2 targets))
273 (recur (float (* radius 2))))))))
274 #+end_src
276 ** Generating Joints
278 This section of code iterates through all the different ways of
279 specifying joints using blender meta-data and converts each one to the
280 appropriate jMonkyeEngine joint.
282 #+name: joints-4
283 #+begin_src clojure
284 (defmulti joint-dispatch
285 "Translate blender pseudo-joints into real JME joints."
286 (fn [constraints & _]
287 (:type constraints)))
289 (defmethod joint-dispatch :point
290 [constraints control-a control-b pivot-a pivot-b rotation]
291 (println-repl "creating POINT2POINT joint")
292 ;; bullet's point2point joints are BROKEN, so we must use the
293 ;; generic 6DOF joint instead of an actual Point2Point joint!
295 ;; should be able to do this:
296 (comment
297 (Point2PointJoint.
298 control-a
299 control-b
300 pivot-a
301 pivot-b))
303 ;; but instead we must do this:
304 (println-repl "substuting 6DOF joint for POINT2POINT joint!")
305 (doto
306 (SixDofJoint.
307 control-a
308 control-b
309 pivot-a
310 pivot-b
311 false)
312 (.setLinearLowerLimit Vector3f/ZERO)
313 (.setLinearUpperLimit Vector3f/ZERO)))
315 (defmethod joint-dispatch :hinge
316 [constraints control-a control-b pivot-a pivot-b rotation]
317 (println-repl "creating HINGE joint")
318 (let [axis
319 (if-let
320 [axis (:axis constraints)]
321 axis
322 Vector3f/UNIT_X)
323 [limit-1 limit-2] (:limit constraints)
324 hinge-axis
325 (.mult
326 rotation
327 (blender-to-jme axis))]
328 (doto
329 (HingeJoint.
330 control-a
331 control-b
332 pivot-a
333 pivot-b
334 hinge-axis
335 hinge-axis)
336 (.setLimit limit-1 limit-2))))
338 (defmethod joint-dispatch :cone
339 [constraints control-a control-b pivot-a pivot-b rotation]
340 (let [limit-xz (:limit-xz constraints)
341 limit-xy (:limit-xy constraints)
342 twist (:twist constraints)]
344 (println-repl "creating CONE joint")
345 (println-repl rotation)
346 (println-repl
347 "UNIT_X --> " (.mult rotation (Vector3f. 1 0 0)))
348 (println-repl
349 "UNIT_Y --> " (.mult rotation (Vector3f. 0 1 0)))
350 (println-repl
351 "UNIT_Z --> " (.mult rotation (Vector3f. 0 0 1)))
352 (doto
353 (ConeJoint.
354 control-a
355 control-b
356 pivot-a
357 pivot-b
358 rotation
359 rotation)
360 (.setLimit (float limit-xz)
361 (float limit-xy)
362 (float twist)))))
364 (defn connect
365 "Create a joint between 'obj-a and 'obj-b at the location of
366 'joint. The type of joint is determined by the metadata on 'joint.
368 Here are some examples:
369 {:type :point}
370 {:type :hinge :limit [0 (/ Math/PI 2)] :axis (Vector3f. 0 1 0)}
371 (:axis defaults to (Vector3f. 1 0 0) if not provided for hinge joints)
373 {:type :cone :limit-xz 0]
374 :limit-xy 0]
375 :twist 0]} (use XZY rotation mode in blender!)"
376 [#^Node obj-a #^Node obj-b #^Node joint]
377 (let [control-a (.getControl obj-a RigidBodyControl)
378 control-b (.getControl obj-b RigidBodyControl)
379 joint-center (.getWorldTranslation joint)
380 joint-rotation (.toRotationMatrix (.getWorldRotation joint))
381 pivot-a (world-to-local obj-a joint-center)
382 pivot-b (world-to-local obj-b joint-center)]
384 (if-let [constraints
385 (map-vals
386 eval
387 (read-string
388 (meta-data joint "joint")))]
389 ;; A side-effect of creating a joint registers
390 ;; it with both physics objects which in turn
391 ;; will register the joint with the physics system
392 ;; when the simulation is started.
393 (do
394 (println-repl "creating joint between"
395 (.getName obj-a) "and" (.getName obj-b))
396 (joint-dispatch constraints
397 control-a control-b
398 pivot-a pivot-b
399 joint-rotation))
400 (println-repl "could not find joint meta-data!"))))
401 #+end_src
403 Creating joints is now a matter of applying =(connect)= to each joint
404 node.
406 #+name: joints-5
407 #+begin_src clojure
408 (defn joints!
409 "Connect the solid parts of the creature with physical joints. The
410 joints are taken from the \"joints\" node in the creature."
411 [#^Node creature]
412 (dorun
413 (map
414 (fn [joint]
415 (let [[obj-a obj-b] (joint-targets creature joint)]
416 (connect obj-a obj-b joint)))
417 (joints creature))))
418 #+end_src
421 ** Round 3
423 Now we can test the hand in all its glory.
425 #+name: test-3
426 #+begin_src clojure
427 (in-ns 'cortex.test.body)
429 (def debug-control
430 {"key-h" (fn [world val]
431 (if val (enable-debug world)))})
433 (defn test-three []
434 (world (nodify
435 [(doto (hand)
436 (physical!)
437 (joints!))
438 (floor)])
439 (merge standard-debug-controls debug-control
440 gravity-control)
441 (comp
442 #(Capture/captureVideo
443 % (File. "/home/r/proj/cortex/render/body/3"))
444 #(do (set-gravity % Vector3f/ZERO) %)
445 setup)
446 no-op))
447 #+end_src
449 =(physical!)= makes the hand solid, then =(joints!)= connects each
450 piece together.
452 #+begin_html
453 <div class="figure">
454 <center>
455 <video controls="controls" width="640">
456 <source src="../video/full-hand.ogg" type="video/ogg"
457 preload="none" poster="../images/aurellem-1280x480.png" />
458 </video>
459 </center>
460 <p>Now the hand is physical and has joints.</p>
461 </div>
462 #+end_html
464 The joints are visualized as green connections between each segment
465 for debug purposes. You can see that they correspond to the empty
466 nodes in the blender file.
468 * Wrap-Up!
470 It is convienent to combine =(physical!)= and =(joints!)= into one
471 function that completely creates the creature's physical body.
473 #+name: joints-6
474 #+begin_src clojure
475 (defn body!
476 "Endow the creature with a physical body connected with joints. The
477 particulars of the joints and the masses of each pody part are
478 determined in blender."
479 [#^Node creature]
480 (physical! creature)
481 (joints! creature))
482 #+end_src
484 * The Worm
486 Going forward, I will use a model that is less complicated than the
487 hand. It has two segments and one joint, and I call it the worm. All
488 of the senses described in the following posts will be applied to this
489 worm.
491 #+name: test-4
492 #+begin_src clojure
493 (in-ns 'cortex.test.body)
495 (defn worm []
496 (load-blender-model
497 "Models/test-creature/worm.blend"))
499 (defn worm-1 []
500 (let [timer (RatchetTimer. 60)]
501 (world
502 (nodify
503 [(doto (worm)
504 (body!))
505 (floor)])
506 (merge standard-debug-controls debug-control)
507 #(do
508 (speed-up %)
509 (light-up-everything %)
510 (.setTimer % timer)
511 (cortex.util/display-dialated-time % timer)
512 (Capture/captureVideo
513 % (File. "/home/r/proj/cortex/render/body/4")))
514 no-op)))
515 #+end_src
517 #+begin_html
518 <div class="figure">
519 <center>
520 <video controls="controls" width="640">
521 <source src="../video/worm-1.ogg" type="video/ogg"
522 preload="none" poster="../images/aurellem-1280x480.png" />
523 </video>
524 </center>
525 <p>This worm model will be the platform onto which future senses will
526 be grafted.</p>
527 </div>
528 #+end_html
530 * Headers
531 #+name: body-header
532 #+begin_src clojure
533 (ns cortex.body
534 "Assemble a physical creature using the definitions found in a
535 specially prepared blender file. Creates rigid bodies and joints so
536 that a creature can have a physical presense in the simulation."
537 {:author "Robert McIntyre"}
538 (:use (cortex world util sense))
539 (:use clojure.contrib.def)
540 (:import
541 (com.jme3.math Vector3f Quaternion Vector2f Matrix3f)
542 (com.jme3.bullet.joints
543 SixDofJoint Point2PointJoint HingeJoint ConeJoint)
544 com.jme3.bullet.control.RigidBodyControl
545 com.jme3.collision.CollisionResults
546 com.jme3.bounding.BoundingBox
547 com.jme3.scene.Node
548 com.jme3.scene.Geometry
549 com.jme3.bullet.collision.shapes.HullCollisionShape))
550 #+end_src
552 #+name: test-header
553 #+begin_src clojure
554 (ns cortex.test.body
555 (:use (cortex world util body))
556 (:import
557 (com.aurellem.capture Capture RatchetTimer)
558 (com.jme3.math Quaternion Vector3f ColorRGBA)
559 java.io.File))
560 #+end_src
562 * Source
563 - [[../src/cortex/body.clj][cortex.body]]
564 - [[../src/cortex/test/body.clj][cortex.test.body]]
565 - [[../assets/Models/test-creature/hand.blend][hand.blend]]
566 - [[../assets/Models/test-creature/palm.png][UV-map-1]]
567 - [[../assets/Models/test-creature/worm.blend][worm.blend]]
568 - [[../assets/Models/test-creature/retina-small.png][UV-map-1]]
569 - [[../assets/Models/test-creature/tip.png][UV-map-2]]
570 #+html: <ul> <li> <a href="../org/body.org">This org file</a> </li> </ul>
571 - [[http://hg.bortreb.com ][source-repository]]
573 * Next
574 The body I have made here exists without any senses or effectors. In
575 the [[./vision.org][next post]], I'll give the creature eyes.
577 * COMMENT Generate Source
578 #+begin_src clojure :tangle ../src/cortex/body.clj
579 <<body-header>>
580 <<body-1>>
581 <<joints-2>>
582 <<joints-3>>
583 <<joints-4>>
584 <<joints-5>>
585 <<joints-6>>
586 #+end_src
588 #+begin_src clojure :tangle ../src/cortex/test/body.clj
589 <<test-header>>
590 <<test-1>>
591 <<test-2>>
592 <<test-3>>
593 <<test-4>>
594 #+end_src