|
rlm@37
|
1 #+title: Simulated Sense of Touch
|
|
rlm@0
|
2 #+author: Robert McIntyre
|
|
rlm@0
|
3 #+email: rlm@mit.edu
|
|
rlm@37
|
4 #+description: Simulated touch for AI research using JMonkeyEngine and clojure.
|
|
rlm@37
|
5 #+keywords: simulation, tactile sense, jMonkeyEngine3, clojure
|
|
rlm@4
|
6 #+SETUPFILE: ../../aurellem/org/setup.org
|
|
rlm@4
|
7 #+INCLUDE: ../../aurellem/org/level-0.org
|
|
rlm@0
|
8
|
|
rlm@37
|
9 * Touch
|
|
rlm@0
|
10
|
|
rlm@226
|
11 Touch is critical to navigation and spatial reasoning and as such I
|
|
rlm@226
|
12 need a simulated version of it to give to my AI creatures.
|
|
rlm@0
|
13
|
|
rlm@306
|
14 Human skin has a wide array of touch sensors, each of which specialize
|
|
rlm@228
|
15 in detecting different vibrational modes and pressures. These sensors
|
|
rlm@228
|
16 can integrate a vast expanse of skin (i.e. your entire palm), or a
|
|
rlm@228
|
17 tiny patch of skin at the tip of your finger. The hairs of the skin
|
|
rlm@228
|
18 help detect objects before they even come into contact with the skin
|
|
rlm@247
|
19 proper.
|
|
rlm@228
|
20
|
|
rlm@248
|
21 However, touch in my simulated world can not exactly correspond to
|
|
rlm@248
|
22 human touch because my creatures are made out of completely rigid
|
|
rlm@248
|
23 segments that don't deform like human skin.
|
|
rlm@248
|
24
|
|
rlm@228
|
25 Instead of measuring deformation or vibration, I surround each rigid
|
|
rlm@247
|
26 part with a plenitude of hair-like objects (/feelers/) which do not
|
|
rlm@247
|
27 interact with the physical world. Physical objects can pass through
|
|
rlm@248
|
28 them with no effect. The feelers are able to tell when other objects
|
|
rlm@248
|
29 pass through them, and they constantly report how much of their extent
|
|
rlm@248
|
30 is covered. So even though the creature's body parts do not deform,
|
|
rlm@248
|
31 the feelers create a margin around those body parts which achieves a
|
|
rlm@248
|
32 sense of touch which is a hybrid between a human's sense of
|
|
rlm@248
|
33 deformation and sense from hairs.
|
|
rlm@228
|
34
|
|
rlm@306
|
35 Implementing touch in jMonkeyEngine follows a different technical route
|
|
rlm@228
|
36 than vision and hearing. Those two senses piggybacked off
|
|
rlm@228
|
37 jMonkeyEngine's 3D audio and video rendering subsystems. To simulate
|
|
rlm@247
|
38 touch, I use jMonkeyEngine's physics system to execute many small
|
|
rlm@247
|
39 collision detections, one for each feeler. The placement of the
|
|
rlm@247
|
40 feelers is determined by a UV-mapped image which shows where each
|
|
rlm@247
|
41 feeler should be on the 3D surface of the body.
|
|
rlm@228
|
42
|
|
rlm@229
|
43 * Defining Touch Meta-Data in Blender
|
|
rlm@229
|
44
|
|
rlm@245
|
45 Each geometry can have a single UV map which describes the position of
|
|
rlm@247
|
46 the feelers which will constitute its sense of touch. This image path
|
|
rlm@245
|
47 is stored under the "touch" key. The image itself is black and white,
|
|
rlm@247
|
48 with black meaning a feeler length of 0 (no feeler is present) and
|
|
rlm@247
|
49 white meaning a feeler length of =scale=, which is a float stored
|
|
rlm@247
|
50 under the key "scale".
|
|
rlm@229
|
51
|
|
rlm@231
|
52 #+name: meta-data
|
|
rlm@0
|
53 #+begin_src clojure
|
|
rlm@229
|
54 (defn tactile-sensor-profile
|
|
rlm@229
|
55 "Return the touch-sensor distribution image in BufferedImage format,
|
|
rlm@229
|
56 or nil if it does not exist."
|
|
rlm@229
|
57 [#^Geometry obj]
|
|
rlm@229
|
58 (if-let [image-path (meta-data obj "touch")]
|
|
rlm@229
|
59 (load-image image-path)))
|
|
rlm@233
|
60
|
|
rlm@233
|
61 (defn tactile-scale
|
|
rlm@247
|
62 "Return the length of each feeler. Default scale is 0.01
|
|
rlm@247
|
63 jMonkeyEngine units."
|
|
rlm@233
|
64 [#^Geometry obj]
|
|
rlm@233
|
65 (if-let [scale (meta-data obj "scale")]
|
|
rlm@233
|
66 scale 0.1))
|
|
rlm@228
|
67 #+end_src
|
|
rlm@156
|
68
|
|
rlm@246
|
69 Here is an example of a UV-map which specifies the position of touch
|
|
rlm@247
|
70 sensors along the surface of the upper segment of the worm.
|
|
rlm@229
|
71
|
|
rlm@246
|
72 #+attr_html: width=755
|
|
rlm@246
|
73 #+caption: This is the tactile-sensor-profile for the upper segment of the worm. It defines regions of high touch sensitivity (where there are many white pixels) and regions of low sensitivity (where white pixels are sparse).
|
|
rlm@246
|
74 [[../images/finger-UV.png]]
|
|
rlm@234
|
75
|
|
rlm@247
|
76 * Implementation Summary
|
|
rlm@247
|
77
|
|
rlm@247
|
78 To simulate touch there are three conceptual steps. For each solid
|
|
rlm@247
|
79 object in the creature, you first have to get UV image and scale
|
|
rlm@306
|
80 parameter which define the position and length of the feelers. Then,
|
|
rlm@430
|
81 you use the triangles which comprise the mesh and the UV data stored in
|
|
rlm@247
|
82 the mesh to determine the world-space position and orientation of each
|
|
rlm@248
|
83 feeler. Then once every frame, update these positions and orientations
|
|
rlm@248
|
84 to match the current position and orientation of the object, and use
|
|
rlm@247
|
85 physics collision detection to gather tactile data.
|
|
rlm@238
|
86
|
|
rlm@247
|
87 Extracting the meta-data has already been described. The third step,
|
|
rlm@273
|
88 physics collision detection, is handled in =touch-kernel=.
|
|
rlm@247
|
89 Translating the positions and orientations of the feelers from the
|
|
rlm@248
|
90 UV-map to world-space is itself a three-step process.
|
|
rlm@239
|
91
|
|
rlm@238
|
92 - Find the triangles which make up the mesh in pixel-space and in
|
|
rlm@273
|
93 world-space. =triangles= =pixel-triangles=.
|
|
rlm@239
|
94
|
|
rlm@247
|
95 - Find the coordinates of each feeler in world-space. These are the
|
|
rlm@273
|
96 origins of the feelers. =feeler-origins=.
|
|
rlm@239
|
97
|
|
rlm@238
|
98 - Calculate the normals of the triangles in world space, and add
|
|
rlm@238
|
99 them to each of the origins of the feelers. These are the
|
|
rlm@273
|
100 normalized coordinates of the tips of the feelers. =feeler-tips=.
|
|
rlm@239
|
101
|
|
rlm@247
|
102 * Triangle Math
|
|
rlm@306
|
103 ** Shrapnel Conversion Functions
|
|
rlm@239
|
104
|
|
rlm@247
|
105 #+name: triangles-1
|
|
rlm@247
|
106 #+begin_src clojure
|
|
rlm@247
|
107 (defn vector3f-seq [#^Vector3f v]
|
|
rlm@247
|
108 [(.getX v) (.getY v) (.getZ v)])
|
|
rlm@247
|
109
|
|
rlm@247
|
110 (defn triangle-seq [#^Triangle tri]
|
|
rlm@247
|
111 [(vector3f-seq (.get1 tri))
|
|
rlm@247
|
112 (vector3f-seq (.get2 tri))
|
|
rlm@247
|
113 (vector3f-seq (.get3 tri))])
|
|
rlm@247
|
114
|
|
rlm@247
|
115 (defn ->vector3f
|
|
rlm@247
|
116 ([coords] (Vector3f. (nth coords 0 0)
|
|
rlm@247
|
117 (nth coords 1 0)
|
|
rlm@247
|
118 (nth coords 2 0))))
|
|
rlm@247
|
119
|
|
rlm@247
|
120 (defn ->triangle [points]
|
|
rlm@247
|
121 (apply #(Triangle. %1 %2 %3) (map ->vector3f points)))
|
|
rlm@247
|
122 #+end_src
|
|
rlm@247
|
123
|
|
rlm@306
|
124 It is convenient to treat a =Triangle= as a vector of vectors, and a
|
|
rlm@255
|
125 =Vector2f= or =Vector3f= as vectors of floats. (->vector3f) and
|
|
rlm@273
|
126 (->triangle) undo the operations of =vector3f-seq= and
|
|
rlm@273
|
127 =triangle-seq=. If these classes implemented =Iterable= then =seq=
|
|
rlm@306
|
128 would work on them automatically.
|
|
rlm@248
|
129
|
|
rlm@247
|
130 ** Decomposing a 3D shape into Triangles
|
|
rlm@247
|
131
|
|
rlm@248
|
132 The rigid objects which make up a creature have an underlying
|
|
rlm@247
|
133 =Geometry=, which is a =Mesh= plus a =Material= and other important
|
|
rlm@248
|
134 data involved with displaying the object.
|
|
rlm@247
|
135
|
|
rlm@247
|
136 A =Mesh= is composed of =Triangles=, and each =Triangle= has three
|
|
rlm@306
|
137 vertices which have coordinates in world space and UV space.
|
|
rlm@247
|
138
|
|
rlm@430
|
139 Here, =triangles= gets all the world-space triangles which comprise a
|
|
rlm@273
|
140 mesh, while =pixel-triangles= gets those same triangles expressed in
|
|
rlm@247
|
141 pixel coordinates (which are UV coordinates scaled to fit the height
|
|
rlm@247
|
142 and width of the UV image).
|
|
rlm@247
|
143
|
|
rlm@247
|
144 #+name: triangles-2
|
|
rlm@247
|
145 #+begin_src clojure
|
|
rlm@247
|
146 (in-ns 'cortex.touch)
|
|
rlm@247
|
147 (defn triangle
|
|
rlm@247
|
148 "Get the triangle specified by triangle-index from the mesh."
|
|
rlm@247
|
149 [#^Geometry geo triangle-index]
|
|
rlm@247
|
150 (triangle-seq
|
|
rlm@247
|
151 (let [scratch (Triangle.)]
|
|
rlm@247
|
152 (.getTriangle (.getMesh geo) triangle-index scratch) scratch)))
|
|
rlm@247
|
153
|
|
rlm@247
|
154 (defn triangles
|
|
rlm@430
|
155 "Return a sequence of all the Triangles which comprise a given
|
|
rlm@247
|
156 Geometry."
|
|
rlm@247
|
157 [#^Geometry geo]
|
|
rlm@247
|
158 (map (partial triangle geo) (range (.getTriangleCount (.getMesh geo)))))
|
|
rlm@247
|
159
|
|
rlm@247
|
160 (defn triangle-vertex-indices
|
|
rlm@247
|
161 "Get the triangle vertex indices of a given triangle from a given
|
|
rlm@247
|
162 mesh."
|
|
rlm@247
|
163 [#^Mesh mesh triangle-index]
|
|
rlm@247
|
164 (let [indices (int-array 3)]
|
|
rlm@247
|
165 (.getTriangle mesh triangle-index indices)
|
|
rlm@247
|
166 (vec indices)))
|
|
rlm@247
|
167
|
|
rlm@247
|
168 (defn vertex-UV-coord
|
|
rlm@247
|
169 "Get the UV-coordinates of the vertex named by vertex-index"
|
|
rlm@247
|
170 [#^Mesh mesh vertex-index]
|
|
rlm@247
|
171 (let [UV-buffer
|
|
rlm@247
|
172 (.getData
|
|
rlm@247
|
173 (.getBuffer
|
|
rlm@247
|
174 mesh
|
|
rlm@247
|
175 VertexBuffer$Type/TexCoord))]
|
|
rlm@247
|
176 [(.get UV-buffer (* vertex-index 2))
|
|
rlm@247
|
177 (.get UV-buffer (+ 1 (* vertex-index 2)))]))
|
|
rlm@247
|
178
|
|
rlm@247
|
179 (defn pixel-triangle [#^Geometry geo image index]
|
|
rlm@247
|
180 (let [mesh (.getMesh geo)
|
|
rlm@247
|
181 width (.getWidth image)
|
|
rlm@247
|
182 height (.getHeight image)]
|
|
rlm@247
|
183 (vec (map (fn [[u v]] (vector (* width u) (* height v)))
|
|
rlm@247
|
184 (map (partial vertex-UV-coord mesh)
|
|
rlm@247
|
185 (triangle-vertex-indices mesh index))))))
|
|
rlm@247
|
186
|
|
rlm@248
|
187 (defn pixel-triangles
|
|
rlm@248
|
188 "The pixel-space triangles of the Geometry, in the same order as
|
|
rlm@248
|
189 (triangles geo)"
|
|
rlm@248
|
190 [#^Geometry geo image]
|
|
rlm@248
|
191 (let [height (.getHeight image)
|
|
rlm@248
|
192 width (.getWidth image)]
|
|
rlm@248
|
193 (map (partial pixel-triangle geo image)
|
|
rlm@248
|
194 (range (.getTriangleCount (.getMesh geo))))))
|
|
rlm@247
|
195 #+end_src
|
|
rlm@247
|
196 ** The Affine Transform from one Triangle to Another
|
|
rlm@247
|
197
|
|
rlm@273
|
198 =pixel-triangles= gives us the mesh triangles expressed in pixel
|
|
rlm@273
|
199 coordinates and =triangles= gives us the mesh triangles expressed in
|
|
rlm@247
|
200 world coordinates. The tactile-sensor-profile gives the position of
|
|
rlm@248
|
201 each feeler in pixel-space. In order to convert pixel-space
|
|
rlm@247
|
202 coordinates into world-space coordinates we need something that takes
|
|
rlm@247
|
203 coordinates on the surface of one triangle and gives the corresponding
|
|
rlm@247
|
204 coordinates on the surface of another triangle.
|
|
rlm@247
|
205
|
|
rlm@247
|
206 Triangles are [[http://mathworld.wolfram.com/AffineTransformation.html ][affine]], which means any triangle can be transformed into
|
|
rlm@247
|
207 any other by a combination of translation, scaling, and
|
|
rlm@248
|
208 rotation. The affine transformation from one triangle to another
|
|
rlm@247
|
209 is readily computable if the triangle is expressed in terms of a $4x4$
|
|
rlm@247
|
210 matrix.
|
|
rlm@247
|
211
|
|
rlm@247
|
212 \begin{bmatrix}
|
|
rlm@247
|
213 x_1 & x_2 & x_3 & n_x \\
|
|
rlm@247
|
214 y_1 & y_2 & y_3 & n_y \\
|
|
rlm@247
|
215 z_1 & z_2 & z_3 & n_z \\
|
|
rlm@247
|
216 1 & 1 & 1 & 1
|
|
rlm@247
|
217 \end{bmatrix}
|
|
rlm@247
|
218
|
|
rlm@306
|
219 Here, the first three columns of the matrix are the vertices of the
|
|
rlm@247
|
220 triangle. The last column is the right-handed unit normal of the
|
|
rlm@247
|
221 triangle.
|
|
rlm@247
|
222
|
|
rlm@247
|
223 With two triangles $T_{1}$ and $T_{2}$ each expressed as a matrix like
|
|
rlm@247
|
224 above, the affine transform from $T_{1}$ to $T_{2}$ is
|
|
rlm@247
|
225
|
|
rlm@247
|
226 $T_{2}T_{1}^{-1}$
|
|
rlm@247
|
227
|
|
rlm@306
|
228 The clojure code below recapitulates the formulas above, using
|
|
rlm@248
|
229 jMonkeyEngine's =Matrix4f= objects, which can describe any affine
|
|
rlm@248
|
230 transformation.
|
|
rlm@247
|
231
|
|
rlm@247
|
232 #+name: triangles-3
|
|
rlm@247
|
233 #+begin_src clojure
|
|
rlm@247
|
234 (in-ns 'cortex.touch)
|
|
rlm@247
|
235
|
|
rlm@247
|
236 (defn triangle->matrix4f
|
|
rlm@247
|
237 "Converts the triangle into a 4x4 matrix: The first three columns
|
|
rlm@247
|
238 contain the vertices of the triangle; the last contains the unit
|
|
rlm@247
|
239 normal of the triangle. The bottom row is filled with 1s."
|
|
rlm@247
|
240 [#^Triangle t]
|
|
rlm@247
|
241 (let [mat (Matrix4f.)
|
|
rlm@247
|
242 [vert-1 vert-2 vert-3]
|
|
rlm@430
|
243 (mapv #(.get t %) (range 3))
|
|
rlm@247
|
244 unit-normal (do (.calculateNormal t)(.getNormal t))
|
|
rlm@247
|
245 vertices [vert-1 vert-2 vert-3 unit-normal]]
|
|
rlm@247
|
246 (dorun
|
|
rlm@247
|
247 (for [row (range 4) col (range 3)]
|
|
rlm@247
|
248 (do
|
|
rlm@247
|
249 (.set mat col row (.get (vertices row) col))
|
|
rlm@247
|
250 (.set mat 3 row 1)))) mat))
|
|
rlm@247
|
251
|
|
rlm@247
|
252 (defn triangles->affine-transform
|
|
rlm@247
|
253 "Returns the affine transformation that converts each vertex in the
|
|
rlm@247
|
254 first triangle into the corresponding vertex in the second
|
|
rlm@247
|
255 triangle."
|
|
rlm@247
|
256 [#^Triangle tri-1 #^Triangle tri-2]
|
|
rlm@247
|
257 (.mult
|
|
rlm@247
|
258 (triangle->matrix4f tri-2)
|
|
rlm@247
|
259 (.invert (triangle->matrix4f tri-1))))
|
|
rlm@247
|
260 #+end_src
|
|
rlm@247
|
261 ** Triangle Boundaries
|
|
rlm@247
|
262
|
|
rlm@247
|
263 For efficiency's sake I will divide the tactile-profile image into
|
|
rlm@247
|
264 small squares which inscribe each pixel-triangle, then extract the
|
|
rlm@247
|
265 points which lie inside the triangle and map them to 3D-space using
|
|
rlm@273
|
266 =triangle-transform= above. To do this I need a function,
|
|
rlm@273
|
267 =convex-bounds= which finds the smallest box which inscribes a 2D
|
|
rlm@247
|
268 triangle.
|
|
rlm@247
|
269
|
|
rlm@273
|
270 =inside-triangle?= determines whether a point is inside a triangle
|
|
rlm@247
|
271 in 2D pixel-space.
|
|
rlm@247
|
272
|
|
rlm@247
|
273 #+name: triangles-4
|
|
rlm@247
|
274 #+begin_src clojure
|
|
rlm@247
|
275 (defn convex-bounds
|
|
rlm@247
|
276 "Returns the smallest square containing the given vertices, as a
|
|
rlm@247
|
277 vector of integers [left top width height]."
|
|
rlm@247
|
278 [verts]
|
|
rlm@247
|
279 (let [xs (map first verts)
|
|
rlm@247
|
280 ys (map second verts)
|
|
rlm@247
|
281 x0 (Math/floor (apply min xs))
|
|
rlm@247
|
282 y0 (Math/floor (apply min ys))
|
|
rlm@247
|
283 x1 (Math/ceil (apply max xs))
|
|
rlm@247
|
284 y1 (Math/ceil (apply max ys))]
|
|
rlm@247
|
285 [x0 y0 (- x1 x0) (- y1 y0)]))
|
|
rlm@247
|
286
|
|
rlm@247
|
287 (defn same-side?
|
|
rlm@247
|
288 "Given the points p1 and p2 and the reference point ref, is point p
|
|
rlm@247
|
289 on the same side of the line that goes through p1 and p2 as ref is?"
|
|
rlm@247
|
290 [p1 p2 ref p]
|
|
rlm@247
|
291 (<=
|
|
rlm@247
|
292 0
|
|
rlm@247
|
293 (.dot
|
|
rlm@247
|
294 (.cross (.subtract p2 p1) (.subtract p p1))
|
|
rlm@247
|
295 (.cross (.subtract p2 p1) (.subtract ref p1)))))
|
|
rlm@247
|
296
|
|
rlm@247
|
297 (defn inside-triangle?
|
|
rlm@247
|
298 "Is the point inside the triangle?"
|
|
rlm@247
|
299 {:author "Dylan Holmes"}
|
|
rlm@247
|
300 [#^Triangle tri #^Vector3f p]
|
|
rlm@247
|
301 (let [[vert-1 vert-2 vert-3] [(.get1 tri) (.get2 tri) (.get3 tri)]]
|
|
rlm@247
|
302 (and
|
|
rlm@247
|
303 (same-side? vert-1 vert-2 vert-3 p)
|
|
rlm@247
|
304 (same-side? vert-2 vert-3 vert-1 p)
|
|
rlm@247
|
305 (same-side? vert-3 vert-1 vert-2 p))))
|
|
rlm@247
|
306 #+end_src
|
|
rlm@247
|
307
|
|
rlm@247
|
308 * Feeler Coordinates
|
|
rlm@247
|
309
|
|
rlm@247
|
310 The triangle-related functions above make short work of calculating
|
|
rlm@247
|
311 the positions and orientations of each feeler in world-space.
|
|
rlm@247
|
312
|
|
rlm@247
|
313 #+name: sensors
|
|
rlm@247
|
314 #+begin_src clojure
|
|
rlm@247
|
315 (in-ns 'cortex.touch)
|
|
rlm@247
|
316
|
|
rlm@247
|
317 (defn feeler-pixel-coords
|
|
rlm@247
|
318 "Returns the coordinates of the feelers in pixel space in lists, one
|
|
rlm@247
|
319 list for each triangle, ordered in the same way as (triangles) and
|
|
rlm@247
|
320 (pixel-triangles)."
|
|
rlm@247
|
321 [#^Geometry geo image]
|
|
rlm@247
|
322 (map
|
|
rlm@247
|
323 (fn [pixel-triangle]
|
|
rlm@247
|
324 (filter
|
|
rlm@247
|
325 (fn [coord]
|
|
rlm@247
|
326 (inside-triangle? (->triangle pixel-triangle)
|
|
rlm@247
|
327 (->vector3f coord)))
|
|
rlm@247
|
328 (white-coordinates image (convex-bounds pixel-triangle))))
|
|
rlm@247
|
329 (pixel-triangles geo image)))
|
|
rlm@247
|
330
|
|
rlm@247
|
331 (defn feeler-world-coords
|
|
rlm@247
|
332 "Returns the coordinates of the feelers in world space in lists, one
|
|
rlm@247
|
333 list for each triangle, ordered in the same way as (triangles) and
|
|
rlm@247
|
334 (pixel-triangles)."
|
|
rlm@247
|
335 [#^Geometry geo image]
|
|
rlm@247
|
336 (let [transforms
|
|
rlm@247
|
337 (map #(triangles->affine-transform
|
|
rlm@247
|
338 (->triangle %1) (->triangle %2))
|
|
rlm@247
|
339 (pixel-triangles geo image)
|
|
rlm@247
|
340 (triangles geo))]
|
|
rlm@247
|
341 (map (fn [transform coords]
|
|
rlm@247
|
342 (map #(.mult transform (->vector3f %)) coords))
|
|
rlm@247
|
343 transforms (feeler-pixel-coords geo image))))
|
|
rlm@247
|
344
|
|
rlm@247
|
345 (defn feeler-origins
|
|
rlm@247
|
346 "The world space coordinates of the root of each feeler."
|
|
rlm@247
|
347 [#^Geometry geo image]
|
|
rlm@247
|
348 (reduce concat (feeler-world-coords geo image)))
|
|
rlm@247
|
349
|
|
rlm@247
|
350 (defn feeler-tips
|
|
rlm@247
|
351 "The world space coordinates of the tip of each feeler."
|
|
rlm@247
|
352 [#^Geometry geo image]
|
|
rlm@247
|
353 (let [world-coords (feeler-world-coords geo image)
|
|
rlm@247
|
354 normals
|
|
rlm@247
|
355 (map
|
|
rlm@247
|
356 (fn [triangle]
|
|
rlm@247
|
357 (.calculateNormal triangle)
|
|
rlm@247
|
358 (.clone (.getNormal triangle)))
|
|
rlm@247
|
359 (map ->triangle (triangles geo)))]
|
|
rlm@247
|
360
|
|
rlm@247
|
361 (mapcat (fn [origins normal]
|
|
rlm@247
|
362 (map #(.add % normal) origins))
|
|
rlm@247
|
363 world-coords normals)))
|
|
rlm@247
|
364
|
|
rlm@247
|
365 (defn touch-topology
|
|
rlm@247
|
366 "touch-topology? is not a function."
|
|
rlm@247
|
367 [#^Geometry geo image]
|
|
rlm@247
|
368 (collapse (reduce concat (feeler-pixel-coords geo image))))
|
|
rlm@247
|
369 #+end_src
|
|
rlm@247
|
370 * Simulated Touch
|
|
rlm@247
|
371
|
|
rlm@273
|
372 =touch-kernel= generates functions to be called from within a
|
|
rlm@247
|
373 simulation that perform the necessary physics collisions to collect
|
|
rlm@273
|
374 tactile data, and =touch!= recursively applies it to every node in
|
|
rlm@247
|
375 the creature.
|
|
rlm@238
|
376
|
|
rlm@233
|
377 #+name: kernel
|
|
rlm@233
|
378 #+begin_src clojure
|
|
rlm@233
|
379 (in-ns 'cortex.touch)
|
|
rlm@233
|
380
|
|
rlm@244
|
381 (defn set-ray [#^Ray ray #^Matrix4f transform
|
|
rlm@244
|
382 #^Vector3f origin #^Vector3f tip]
|
|
rlm@306
|
383 ;; Doing everything locally reduces garbage collection by enough to
|
|
rlm@243
|
384 ;; be worth it.
|
|
rlm@243
|
385 (.mult transform origin (.getOrigin ray))
|
|
rlm@243
|
386 (.mult transform tip (.getDirection ray))
|
|
rlm@249
|
387 (.subtractLocal (.getDirection ray) (.getOrigin ray))
|
|
rlm@249
|
388 (.normalizeLocal (.getDirection ray)))
|
|
rlm@242
|
389
|
|
rlm@249
|
390 (import com.jme3.math.FastMath)
|
|
rlm@249
|
391
|
|
rlm@233
|
392 (defn touch-kernel
|
|
rlm@234
|
393 "Constructs a function which will return tactile sensory data from
|
|
rlm@234
|
394 'geo when called from inside a running simulation"
|
|
rlm@234
|
395 [#^Geometry geo]
|
|
rlm@243
|
396 (if-let
|
|
rlm@243
|
397 [profile (tactile-sensor-profile geo)]
|
|
rlm@243
|
398 (let [ray-reference-origins (feeler-origins geo profile)
|
|
rlm@243
|
399 ray-reference-tips (feeler-tips geo profile)
|
|
rlm@244
|
400 ray-length (tactile-scale geo)
|
|
rlm@243
|
401 current-rays (map (fn [_] (Ray.)) ray-reference-origins)
|
|
rlm@249
|
402 topology (touch-topology geo profile)
|
|
rlm@249
|
403 correction (float (* ray-length -0.2))]
|
|
rlm@249
|
404
|
|
rlm@249
|
405 ;; slight tolerance for very close collisions.
|
|
rlm@249
|
406 (dorun
|
|
rlm@249
|
407 (map (fn [origin tip]
|
|
rlm@249
|
408 (.addLocal origin (.mult (.subtract tip origin)
|
|
rlm@249
|
409 correction)))
|
|
rlm@249
|
410 ray-reference-origins ray-reference-tips))
|
|
rlm@244
|
411 (dorun (map #(.setLimit % ray-length) current-rays))
|
|
rlm@233
|
412 (fn [node]
|
|
rlm@243
|
413 (let [transform (.getWorldMatrix geo)]
|
|
rlm@243
|
414 (dorun
|
|
rlm@244
|
415 (map (fn [ray ref-origin ref-tip]
|
|
rlm@244
|
416 (set-ray ray transform ref-origin ref-tip))
|
|
rlm@243
|
417 current-rays ray-reference-origins
|
|
rlm@244
|
418 ray-reference-tips))
|
|
rlm@233
|
419 (vector
|
|
rlm@243
|
420 topology
|
|
rlm@233
|
421 (vec
|
|
rlm@243
|
422 (for [ray current-rays]
|
|
rlm@233
|
423 (do
|
|
rlm@233
|
424 (let [results (CollisionResults.)]
|
|
rlm@233
|
425 (.collideWith node ray results)
|
|
rlm@233
|
426 (let [touch-objects
|
|
rlm@233
|
427 (filter #(not (= geo (.getGeometry %)))
|
|
rlm@249
|
428 results)
|
|
rlm@249
|
429 limit (.getLimit ray)]
|
|
rlm@233
|
430 [(if (empty? touch-objects)
|
|
rlm@249
|
431 limit
|
|
rlm@249
|
432 (let [response
|
|
rlm@249
|
433 (apply min (map #(.getDistance %)
|
|
rlm@249
|
434 touch-objects))]
|
|
rlm@249
|
435 (FastMath/clamp
|
|
rlm@249
|
436 (float
|
|
rlm@283
|
437 (if (> response limit) (float 0.0)
|
|
rlm@249
|
438 (+ response correction)))
|
|
rlm@249
|
439 (float 0.0)
|
|
rlm@249
|
440 limit)))
|
|
rlm@249
|
441 limit])))))))))))
|
|
rlm@233
|
442
|
|
rlm@233
|
443 (defn touch!
|
|
rlm@233
|
444 "Endow the creature with the sense of touch. Returns a sequence of
|
|
rlm@306
|
445 functions, one for each body part with a tactile-sensor-profile,
|
|
rlm@233
|
446 each of which when called returns sensory data for that body part."
|
|
rlm@233
|
447 [#^Node creature]
|
|
rlm@233
|
448 (filter
|
|
rlm@233
|
449 (comp not nil?)
|
|
rlm@233
|
450 (map touch-kernel
|
|
rlm@233
|
451 (filter #(isa? (class %) Geometry)
|
|
rlm@233
|
452 (node-seq creature)))))
|
|
rlm@233
|
453 #+end_src
|
|
rlm@233
|
454
|
|
rlm@249
|
455 #+results: kernel
|
|
rlm@249
|
456 : #'cortex.touch/touch!
|
|
rlm@249
|
457
|
|
rlm@247
|
458 * Visualizing Touch
|
|
rlm@238
|
459
|
|
rlm@249
|
460 Each feeler is represented in the image as a single pixel. The
|
|
rlm@306
|
461 greyscale value of each pixel represents how deep the feeler
|
|
rlm@249
|
462 represented by that pixel is inside another object. Black means that
|
|
rlm@249
|
463 nothing is touching the feeler, while white means that the feeler is
|
|
rlm@249
|
464 completely inside another object, which is presumably flush with the
|
|
rlm@249
|
465 surface of the triangle from which the feeler originates.
|
|
rlm@249
|
466
|
|
rlm@233
|
467 #+name: visualization
|
|
rlm@233
|
468 #+begin_src clojure
|
|
rlm@233
|
469 (in-ns 'cortex.touch)
|
|
rlm@233
|
470
|
|
rlm@233
|
471 (defn touch->gray
|
|
rlm@306
|
472 "Convert a pair of [distance, max-distance] into a gray-scale pixel."
|
|
rlm@233
|
473 [distance max-distance]
|
|
rlm@245
|
474 (gray (- 255 (rem (int (* 255 (/ distance max-distance))) 256))))
|
|
rlm@233
|
475
|
|
rlm@233
|
476 (defn view-touch
|
|
rlm@245
|
477 "Creates a function which accepts a list of touch sensor-data and
|
|
rlm@233
|
478 displays each element to the screen."
|
|
rlm@233
|
479 []
|
|
rlm@233
|
480 (view-sense
|
|
rlm@246
|
481 (fn [[coords sensor-data]]
|
|
rlm@233
|
482 (let [image (points->image coords)]
|
|
rlm@233
|
483 (dorun
|
|
rlm@233
|
484 (for [i (range (count coords))]
|
|
rlm@250
|
485 (.setRGB image ((coords i) 0) ((coords i) 1)
|
|
rlm@250
|
486 (apply touch->gray (sensor-data i)))))
|
|
rlm@283
|
487 image))))
|
|
rlm@233
|
488 #+end_src
|
|
rlm@249
|
489
|
|
rlm@249
|
490 #+results: visualization
|
|
rlm@249
|
491 : #'cortex.touch/view-touch
|
|
rlm@249
|
492
|
|
rlm@250
|
493 * Basic Test of Touch
|
|
rlm@249
|
494
|
|
rlm@249
|
495 The worm's sense of touch is a bit complicated, so for this basic test
|
|
rlm@249
|
496 I'll use a new creature --- a simple cube which has touch sensors
|
|
rlm@249
|
497 evenly distributed along each of its sides.
|
|
rlm@249
|
498
|
|
rlm@253
|
499 #+name: test-touch-0
|
|
rlm@249
|
500 #+begin_src clojure
|
|
rlm@249
|
501 (in-ns 'cortex.test.touch)
|
|
rlm@249
|
502
|
|
rlm@249
|
503 (defn touch-cube []
|
|
rlm@249
|
504 (load-blender-model "Models/test-touch/touch-cube.blend"))
|
|
rlm@249
|
505 #+end_src
|
|
rlm@249
|
506
|
|
rlm@253
|
507 ** The Touch Cube
|
|
rlm@249
|
508 #+begin_html
|
|
rlm@249
|
509 <div class="figure">
|
|
rlm@249
|
510 <center>
|
|
rlm@249
|
511 <video controls="controls" width="500">
|
|
rlm@249
|
512 <source src="../video/touch-cube.ogg" type="video/ogg"
|
|
rlm@249
|
513 preload="none" poster="../images/aurellem-1280x480.png" />
|
|
rlm@249
|
514 </video>
|
|
rlm@309
|
515 <br> <a href="http://youtu.be/aEao4m8meAI"> YouTube </a>
|
|
rlm@249
|
516 </center>
|
|
rlm@249
|
517 <p>A simple creature with evenly distributed touch sensors.</p>
|
|
rlm@249
|
518 </div>
|
|
rlm@249
|
519 #+end_html
|
|
rlm@249
|
520
|
|
rlm@249
|
521 The tactile-sensor-profile image for this simple creature looks like
|
|
rlm@249
|
522 this:
|
|
rlm@249
|
523
|
|
rlm@249
|
524 #+attr_html: width=500
|
|
rlm@249
|
525 #+caption: The distribution of feelers along the touch-cube. The colors of the faces are irrelevant; only the white pixels specify feelers.
|
|
rlm@249
|
526 [[../images/touch-profile.png]]
|
|
rlm@249
|
527
|
|
rlm@253
|
528 #+name: test-touch-1
|
|
rlm@249
|
529 #+begin_src clojure
|
|
rlm@249
|
530 (in-ns 'cortex.test.touch)
|
|
rlm@249
|
531
|
|
rlm@249
|
532 (defn test-basic-touch
|
|
rlm@321
|
533 "Testing touch:
|
|
rlm@321
|
534 You should see a cube fall onto a table. There is a cross-shaped
|
|
rlm@321
|
535 display which reports the cube's sensation of touch. This display
|
|
rlm@321
|
536 should change when the cube hits the table, and whenever you hit
|
|
rlm@321
|
537 the cube with balls.
|
|
rlm@321
|
538
|
|
rlm@321
|
539 Keys:
|
|
rlm@321
|
540 <space> : fire ball"
|
|
rlm@249
|
541 ([] (test-basic-touch false))
|
|
rlm@249
|
542 ([record?]
|
|
rlm@249
|
543 (let [the-cube (doto (touch-cube) (body!))
|
|
rlm@249
|
544 touch (touch! the-cube)
|
|
rlm@249
|
545 touch-display (view-touch)]
|
|
rlm@250
|
546 (world
|
|
rlm@250
|
547 (nodify [the-cube
|
|
rlm@250
|
548 (box 10 1 10 :position (Vector3f. 0 -10 0)
|
|
rlm@250
|
549 :color ColorRGBA/Gray :mass 0)])
|
|
rlm@250
|
550
|
|
rlm@250
|
551 standard-debug-controls
|
|
rlm@250
|
552
|
|
rlm@250
|
553 (fn [world]
|
|
rlm@340
|
554 (let [timer (IsoTimer. 60)]
|
|
rlm@340
|
555 (.setTimer world timer)
|
|
rlm@340
|
556 (display-dilated-time world timer))
|
|
rlm@250
|
557 (if record?
|
|
rlm@250
|
558 (Capture/captureVideo
|
|
rlm@250
|
559 world
|
|
rlm@250
|
560 (File. "/home/r/proj/cortex/render/touch-cube/main-view/")))
|
|
rlm@250
|
561 (speed-up world)
|
|
rlm@250
|
562 (light-up-everything world))
|
|
rlm@250
|
563
|
|
rlm@250
|
564 (fn [world tpf]
|
|
rlm@250
|
565 (touch-display
|
|
rlm@250
|
566 (map #(% (.getRootNode world)) touch)
|
|
rlm@250
|
567 (if record?
|
|
rlm@250
|
568 (File. "/home/r/proj/cortex/render/touch-cube/touch/"))))))))
|
|
rlm@250
|
569 #+end_src
|
|
rlm@249
|
570
|
|
rlm@340
|
571 #+results: test-touch-1
|
|
rlm@340
|
572 : #'cortex.test.touch/test-basic-touch
|
|
rlm@340
|
573
|
|
rlm@250
|
574 ** Basic Touch Demonstration
|
|
rlm@249
|
575
|
|
rlm@250
|
576 #+begin_html
|
|
rlm@250
|
577 <div class="figure">
|
|
rlm@250
|
578 <center>
|
|
rlm@250
|
579 <video controls="controls" width="755">
|
|
rlm@250
|
580 <source src="../video/basic-touch.ogg" type="video/ogg"
|
|
rlm@250
|
581 preload="none" poster="../images/aurellem-1280x480.png" />
|
|
rlm@250
|
582 </video>
|
|
rlm@309
|
583 <br> <a href="http://youtu.be/8xNEtD-a8f0"> YouTube </a>
|
|
rlm@250
|
584 </center>
|
|
rlm@250
|
585 <p>The simple creature responds to touch.</p>
|
|
rlm@250
|
586 </div>
|
|
rlm@250
|
587 #+end_html
|
|
rlm@249
|
588
|
|
rlm@250
|
589 ** Generating the Basic Touch Video
|
|
rlm@253
|
590 #+name: magick4
|
|
rlm@250
|
591 #+begin_src clojure
|
|
rlm@250
|
592 (ns cortex.video.magick4
|
|
rlm@250
|
593 (:import java.io.File)
|
|
rlm@316
|
594 (:use clojure.java.shell))
|
|
rlm@250
|
595
|
|
rlm@250
|
596 (defn images [path]
|
|
rlm@250
|
597 (sort (rest (file-seq (File. path)))))
|
|
rlm@250
|
598
|
|
rlm@250
|
599 (def base "/home/r/proj/cortex/render/touch-cube/")
|
|
rlm@250
|
600
|
|
rlm@250
|
601 (defn pics [file]
|
|
rlm@250
|
602 (images (str base file)))
|
|
rlm@250
|
603
|
|
rlm@250
|
604 (defn combine-images []
|
|
rlm@250
|
605 (let [main-view (pics "main-view")
|
|
rlm@250
|
606 touch (pics "touch/0")
|
|
rlm@250
|
607 background (repeat 9001 (File. (str base "background.png")))
|
|
rlm@250
|
608 targets (map
|
|
rlm@250
|
609 #(File. (str base "out/" (format "%07d.png" %)))
|
|
rlm@250
|
610 (range 0 (count main-view)))]
|
|
rlm@250
|
611 (dorun
|
|
rlm@250
|
612 (pmap
|
|
rlm@250
|
613 (comp
|
|
rlm@250
|
614 (fn [[background main-view touch target]]
|
|
rlm@250
|
615 (println target)
|
|
rlm@250
|
616 (sh "convert"
|
|
rlm@250
|
617 touch
|
|
rlm@250
|
618 "-resize" "x300"
|
|
rlm@250
|
619 "-rotate" "180"
|
|
rlm@250
|
620 background
|
|
rlm@250
|
621 "-swap" "0,1"
|
|
rlm@250
|
622 "-geometry" "+776+129"
|
|
rlm@250
|
623 "-composite"
|
|
rlm@250
|
624 main-view "-geometry" "+66+21" "-composite"
|
|
rlm@250
|
625 target))
|
|
rlm@250
|
626 (fn [& args] (map #(.getCanonicalPath %) args)))
|
|
rlm@250
|
627 background main-view touch targets))))
|
|
rlm@249
|
628 #+end_src
|
|
rlm@249
|
629
|
|
rlm@252
|
630 #+begin_src sh :results silent
|
|
rlm@312
|
631 cd ~/proj/cortex/render/touch-cube/
|
|
rlm@252
|
632 ffmpeg -r 60 -i out/%07d.png -b:v 9000k -c:v libtheora basic-touch.ogg
|
|
rlm@252
|
633 #+end_src
|
|
rlm@250
|
634
|
|
rlm@312
|
635 #+begin_src sh :results silent
|
|
rlm@312
|
636 cd ~/proj/cortex/render/touch-cube/
|
|
rlm@312
|
637 ffmpeg -r 30 -i blender-intro/%07d.png -b:v 9000k -c:v libtheora touch-cube.ogg
|
|
rlm@312
|
638 #+end_src
|
|
rlm@312
|
639
|
|
rlm@232
|
640 * Adding Touch to the Worm
|
|
rlm@232
|
641
|
|
rlm@253
|
642 #+name: test-touch-2
|
|
rlm@232
|
643 #+begin_src clojure
|
|
rlm@253
|
644 (in-ns 'cortex.test.touch)
|
|
rlm@232
|
645
|
|
rlm@283
|
646 (defn test-worm-touch
|
|
rlm@321
|
647 "Testing touch:
|
|
rlm@321
|
648 You will see the worm fall onto a table. There is a display which
|
|
rlm@321
|
649 reports the worm's sense of touch. It should change when the worm
|
|
rlm@321
|
650 hits the table and when you hit it with balls.
|
|
rlm@321
|
651
|
|
rlm@321
|
652 Keys:
|
|
rlm@321
|
653 <space> : fire ball"
|
|
rlm@283
|
654 ([] (test-worm-touch false))
|
|
rlm@253
|
655 ([record?]
|
|
rlm@253
|
656 (let [the-worm (doto (worm) (body!))
|
|
rlm@253
|
657 touch (touch! the-worm)
|
|
rlm@253
|
658 touch-display (view-touch)]
|
|
rlm@253
|
659 (world
|
|
rlm@253
|
660 (nodify [the-worm (floor)])
|
|
rlm@253
|
661 standard-debug-controls
|
|
rlm@253
|
662
|
|
rlm@253
|
663 (fn [world]
|
|
rlm@340
|
664 (let [timer (IsoTimer. 60)]
|
|
rlm@340
|
665 (.setTimer world timer)
|
|
rlm@340
|
666 (display-dilated-time world timer))
|
|
rlm@253
|
667 (if record?
|
|
rlm@253
|
668 (Capture/captureVideo
|
|
rlm@253
|
669 world
|
|
rlm@253
|
670 (File. "/home/r/proj/cortex/render/worm-touch/main-view/")))
|
|
rlm@253
|
671 (speed-up world)
|
|
rlm@253
|
672 (light-up-everything world))
|
|
rlm@232
|
673
|
|
rlm@253
|
674 (fn [world tpf]
|
|
rlm@253
|
675 (touch-display
|
|
rlm@253
|
676 (map #(% (.getRootNode world)) touch)
|
|
rlm@253
|
677 (if record?
|
|
rlm@253
|
678 (File. "/home/r/proj/cortex/render/worm-touch/touch/"))))))))
|
|
rlm@232
|
679 #+end_src
|
|
rlm@247
|
680
|
|
rlm@340
|
681 #+results: test-touch-2
|
|
rlm@340
|
682 : #'cortex.test.touch/test-worm-touch
|
|
rlm@340
|
683
|
|
rlm@253
|
684 ** Worm Touch Demonstration
|
|
rlm@253
|
685 #+begin_html
|
|
rlm@253
|
686 <div class="figure">
|
|
rlm@253
|
687 <center>
|
|
rlm@253
|
688 <video controls="controls" width="550">
|
|
rlm@253
|
689 <source src="../video/worm-touch.ogg" type="video/ogg"
|
|
rlm@253
|
690 preload="none" poster="../images/aurellem-1280x480.png" />
|
|
rlm@253
|
691 </video>
|
|
rlm@309
|
692 <br> <a href="http://youtu.be/RHx2wqzNVcU"> YouTube </a>
|
|
rlm@253
|
693 </center>
|
|
rlm@253
|
694 <p>The worm responds to touch.</p>
|
|
rlm@253
|
695 </div>
|
|
rlm@253
|
696 #+end_html
|
|
rlm@252
|
697
|
|
rlm@252
|
698
|
|
rlm@253
|
699 ** Generating the Worm Touch Video
|
|
rlm@253
|
700 #+name: magick5
|
|
rlm@253
|
701 #+begin_src clojure
|
|
rlm@253
|
702 (ns cortex.video.magick5
|
|
rlm@253
|
703 (:import java.io.File)
|
|
rlm@316
|
704 (:use clojure.java.shell))
|
|
rlm@253
|
705
|
|
rlm@253
|
706 (defn images [path]
|
|
rlm@253
|
707 (sort (rest (file-seq (File. path)))))
|
|
rlm@253
|
708
|
|
rlm@253
|
709 (def base "/home/r/proj/cortex/render/worm-touch/")
|
|
rlm@253
|
710
|
|
rlm@253
|
711 (defn pics [file]
|
|
rlm@253
|
712 (images (str base file)))
|
|
rlm@253
|
713
|
|
rlm@253
|
714 (defn combine-images []
|
|
rlm@253
|
715 (let [main-view (pics "main-view")
|
|
rlm@253
|
716 touch (pics "touch/0")
|
|
rlm@253
|
717 targets (map
|
|
rlm@253
|
718 #(File. (str base "out/" (format "%07d.png" %)))
|
|
rlm@253
|
719 (range 0 (count main-view)))]
|
|
rlm@253
|
720 (dorun
|
|
rlm@253
|
721 (pmap
|
|
rlm@253
|
722 (comp
|
|
rlm@253
|
723 (fn [[ main-view touch target]]
|
|
rlm@253
|
724 (println target)
|
|
rlm@253
|
725 (sh "convert"
|
|
rlm@253
|
726 main-view
|
|
rlm@253
|
727 touch "-geometry" "+0+0" "-composite"
|
|
rlm@253
|
728 target))
|
|
rlm@253
|
729 (fn [& args] (map #(.getCanonicalPath %) args)))
|
|
rlm@253
|
730 main-view touch targets))))
|
|
rlm@253
|
731 #+end_src
|
|
rlm@252
|
732
|
|
rlm@312
|
733 #+begin_src sh :results silent
|
|
rlm@312
|
734 cd ~/proj/cortex/render/worm-touch
|
|
rlm@312
|
735 ffmpeg -r 60 -i out/%07d.png -b:v 9000k -c:v libtheora worm-touch.ogg
|
|
rlm@312
|
736 #+end_src
|
|
rlm@312
|
737
|
|
rlm@247
|
738 * Headers
|
|
rlm@247
|
739
|
|
rlm@247
|
740 #+name: touch-header
|
|
rlm@247
|
741 #+begin_src clojure
|
|
rlm@247
|
742 (ns cortex.touch
|
|
rlm@247
|
743 "Simulate the sense of touch in jMonkeyEngine3. Enables any Geometry
|
|
rlm@247
|
744 to be outfitted with touch sensors with density determined by a UV
|
|
rlm@247
|
745 image. In this way a Geometry can know what parts of itself are
|
|
rlm@247
|
746 touching nearby objects. Reads specially prepared blender files to
|
|
rlm@247
|
747 construct this sense automatically."
|
|
rlm@247
|
748 {:author "Robert McIntyre"}
|
|
rlm@247
|
749 (:use (cortex world util sense))
|
|
rlm@247
|
750 (:import (com.jme3.scene Geometry Node Mesh))
|
|
rlm@247
|
751 (:import com.jme3.collision.CollisionResults)
|
|
rlm@247
|
752 (:import com.jme3.scene.VertexBuffer$Type)
|
|
rlm@247
|
753 (:import (com.jme3.math Triangle Vector3f Vector2f Ray Matrix4f)))
|
|
rlm@247
|
754 #+end_src
|
|
rlm@247
|
755
|
|
rlm@253
|
756 #+name: test-touch-header
|
|
rlm@253
|
757 #+begin_src clojure
|
|
rlm@253
|
758 (ns cortex.test.touch
|
|
rlm@253
|
759 (:use (cortex world util sense body touch))
|
|
rlm@253
|
760 (:use cortex.test.body)
|
|
rlm@340
|
761 (:import (com.aurellem.capture Capture IsoTimer))
|
|
rlm@253
|
762 (:import java.io.File)
|
|
rlm@253
|
763 (:import (com.jme3.math Vector3f ColorRGBA)))
|
|
rlm@253
|
764 #+end_src
|
|
rlm@253
|
765
|
|
rlm@340
|
766 #+results: test-touch-header
|
|
rlm@340
|
767 : com.jme3.math.ColorRGBA
|
|
rlm@340
|
768
|
|
rlm@228
|
769 * Source Listing
|
|
rlm@253
|
770 - [[../src/cortex/touch.clj][cortex.touch]]
|
|
rlm@253
|
771 - [[../src/cortex/test/touch.clj][cortex.test.touch]]
|
|
rlm@253
|
772 - [[../src/cortex/video/magick4.clj][cortex.video.magick4]]
|
|
rlm@253
|
773 - [[../src/cortex/video/magick5.clj][cortex.video.magick5]]
|
|
rlm@283
|
774 - [[../assets/Models/test-touch/touch-cube.blend][touch-cube.blend]]
|
|
rlm@253
|
775 #+html: <ul> <li> <a href="../org/touch.org">This org file</a> </li> </ul>
|
|
rlm@253
|
776 - [[http://hg.bortreb.com ][source-repository]]
|
|
rlm@253
|
777
|
|
rlm@254
|
778 * Next
|
|
rlm@332
|
779 So far I've implemented simulated Vision, Hearing, and
|
|
rlm@332
|
780 Touch, the most obvious and prominent senses that humans
|
|
rlm@332
|
781 have. Smell and Taste shall remain unimplemented for
|
|
rlm@332
|
782 now. This accounts for the "five senses" that feature so
|
|
rlm@332
|
783 prominently in our lives. But humans have far more than the
|
|
rlm@332
|
784 five main senses. There are internal chemical senses, pain
|
|
rlm@332
|
785 (which is *not* the same as touch), heat sensitivity, and
|
|
rlm@332
|
786 our sense of balance, among others. One extra sense is so
|
|
rlm@332
|
787 important that I must implement it to have a hope of making
|
|
rlm@332
|
788 creatures that can gracefully control their own bodies. It
|
|
rlm@332
|
789 is Proprioception, which is the sense of the location of
|
|
rlm@332
|
790 each body part in relation to the other body parts.
|
|
rlm@253
|
791
|
|
rlm@332
|
792 Close your eyes, and touch your nose with your right index
|
|
rlm@332
|
793 finger. How did you do it? You could not see your hand, and
|
|
rlm@332
|
794 neither your hand nor your nose could use the sense of touch
|
|
rlm@332
|
795 to guide the path of your hand. There are no sound cues,
|
|
rlm@332
|
796 and Taste and Smell certainly don't provide any help. You
|
|
rlm@332
|
797 know where your hand is without your other senses because of
|
|
rlm@332
|
798 Proprioception.
|
|
rlm@228
|
799
|
|
rlm@254
|
800 Onward to [[./proprioception.org][proprioception]]!
|
|
rlm@228
|
801
|
|
rlm@226
|
802 * COMMENT Code Generation
|
|
rlm@39
|
803 #+begin_src clojure :tangle ../src/cortex/touch.clj
|
|
rlm@231
|
804 <<touch-header>>
|
|
rlm@231
|
805 <<meta-data>>
|
|
rlm@231
|
806 <<triangles-1>>
|
|
rlm@247
|
807 <<triangles-2>>
|
|
rlm@231
|
808 <<triangles-3>>
|
|
rlm@231
|
809 <<triangles-4>>
|
|
rlm@231
|
810 <<sensors>>
|
|
rlm@231
|
811 <<kernel>>
|
|
rlm@231
|
812 <<visualization>>
|
|
rlm@0
|
813 #+end_src
|
|
rlm@0
|
814
|
|
rlm@253
|
815 #+begin_src clojure :tangle ../src/cortex/test/touch.clj
|
|
rlm@253
|
816 <<test-touch-header>>
|
|
rlm@253
|
817 <<test-touch-0>>
|
|
rlm@253
|
818 <<test-touch-1>>
|
|
rlm@253
|
819 <<test-touch-2>>
|
|
rlm@253
|
820 #+end_src
|
|
rlm@232
|
821
|
|
rlm@253
|
822 #+begin_src clojure :tangle ../src/cortex/video/magick4.clj
|
|
rlm@253
|
823 <<magick4>>
|
|
rlm@39
|
824 #+end_src
|
|
rlm@39
|
825
|
|
rlm@253
|
826 #+begin_src clojure :tangle ../src/cortex/video/magick5.clj
|
|
rlm@253
|
827 <<magick5>>
|
|
rlm@253
|
828 #+end_src
|
