Mercurial > cortex

comparison thesis/cortex.org @ 477:ba54df21fc7c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
complete first draft of touch.
author Robert McIntyre <rlm@mit.edu>
date Fri, 28 Mar 2014 22:51:14 -0400
parents 5a15611fbb9f
children a5480a06d5fe
comparison
equal deleted inserted replaced
476:5a15611fbb9f 477:ba54df21fc7c
1552 1552
1553 This system of hearing has also been co-opted by the 1553 This system of hearing has also been co-opted by the
1554 jMonkeyEngine3 community and is used to record audio for demo 1554 jMonkeyEngine3 community and is used to record audio for demo
1555 videos. 1555 videos.
1556 1556
1557 ** Touch uses hundreds of hair-like elements 1557 ** COMMENT Touch uses hundreds of hair-like elements
1558 1558
1559 Touch is critical to navigation and spatial reasoning and as such I 1559 Touch is critical to navigation and spatial reasoning and as such I
1560 need a simulated version of it to give to my AI creatures. 1560 need a simulated version of it to give to my AI creatures.
1561 1561
1562 Human skin has a wide array of touch sensors, each of which 1562 Human skin has a wide array of touch sensors, each of which
1586 simulate touch, I use jMonkeyEngine's physics system to execute 1586 simulate touch, I use jMonkeyEngine's physics system to execute
1587 many small collision detections, one for each feeler. The placement 1587 many small collision detections, one for each feeler. The placement
1588 of the feelers is determined by a UV-mapped image which shows where 1588 of the feelers is determined by a UV-mapped image which shows where
1589 each feeler should be on the 3D surface of the body. 1589 each feeler should be on the 3D surface of the body.
1590 1590
1591 *** COMMENT Defining Touch Meta-Data in Blender 1591 *** Defining Touch Meta-Data in Blender
1592 1592
1593 Each geometry can have a single UV map which describes the 1593 Each geometry can have a single UV map which describes the
1594 position of the feelers which will constitute its sense of touch. 1594 position of the feelers which will constitute its sense of touch.
1595 This image path is stored under the ``touch'' key. The image itself 1595 This image path is stored under the ``touch'' key. The image itself
1596 is black and white, with black meaning a feeler length of 0 (no 1596 is black and white, with black meaning a feeler length of 0 (no
1600 #+caption: Touch does not use empty nodes, to store metadata, 1600 #+caption: Touch does not use empty nodes, to store metadata,
1601 #+caption: because the metadata of each solid part of a 1601 #+caption: because the metadata of each solid part of a
1602 #+caption: creature's body is sufficient. 1602 #+caption: creature's body is sufficient.
1603 #+name: touch-meta-data 1603 #+name: touch-meta-data
1604 #+begin_listing clojure 1604 #+begin_listing clojure
1605 #+BEGIN_SRC clojure
1605 (defn tactile-sensor-profile 1606 (defn tactile-sensor-profile
1606 "Return the touch-sensor distribution image in BufferedImage format, 1607 "Return the touch-sensor distribution image in BufferedImage format,
1607 or nil if it does not exist." 1608 or nil if it does not exist."
1608 [#^Geometry obj] 1609 [#^Geometry obj]
1609 (if-let [image-path (meta-data obj "touch")] 1610 (if-let [image-path (meta-data obj "touch")]
1613 "Return the length of each feeler. Default scale is 0.01 1614 "Return the length of each feeler. Default scale is 0.01
1614 jMonkeyEngine units." 1615 jMonkeyEngine units."
1615 [#^Geometry obj] 1616 [#^Geometry obj]
1616 (if-let [scale (meta-data obj "scale")] 1617 (if-let [scale (meta-data obj "scale")]
1617 scale 0.1)) 1618 scale 0.1))
1619 #+END_SRC
1618 #+end_listing 1620 #+end_listing
1619 1621
1620 Here is an example of a UV-map which specifies the position of 1622 Here is an example of a UV-map which specifies the position of
1621 touch sensors along the surface of the upper segment of a fingertip. 1623 touch sensors along the surface of the upper segment of a fingertip.
1622
1623 1624
1624 #+caption: This is the tactile-sensor-profile for the upper segment 1625 #+caption: This is the tactile-sensor-profile for the upper segment
1625 #+caption: of a fingertip. It defines regions of high touch sensitivity 1626 #+caption: of a fingertip. It defines regions of high touch sensitivity
1626 #+caption: (where there are many white pixels) and regions of low 1627 #+caption: (where there are many white pixels) and regions of low
1627 #+caption: sensitivity (where white pixels are sparse). 1628 #+caption: sensitivity (where white pixels are sparse).
1628 #+name: fimgertip-UV 1629 #+name: fimgertip-UV
1629 #+ATTR_LaTeX: :width 10cm 1630 #+ATTR_LaTeX: :width 13cm
1630 [[../images/finger-UV.png]] 1631 [[./images/finger-UV.png]]
1631 1632
1632 *** COMMENT Implementation Summary 1633 *** Implementation Summary
1633 1634
1634 To simulate touch there are three conceptual steps. For each solid 1635 To simulate touch there are three conceptual steps. For each solid
1635 object in the creature, you first have to get UV image and scale 1636 object in the creature, you first have to get UV image and scale
1636 parameter which define the position and length of the feelers. 1637 parameter which define the position and length of the feelers.
1637 Then, you use the triangles which comprise the mesh and the UV 1638 Then, you use the triangles which comprise the mesh and the UV
1655 - Calculate the normals of the triangles in world space, and add 1656 - Calculate the normals of the triangles in world space, and add
1656 them to each of the origins of the feelers. These are the 1657 them to each of the origins of the feelers. These are the
1657 normalized coordinates of the tips of the feelers. 1658 normalized coordinates of the tips of the feelers.
1658 (=feeler-tips=). 1659 (=feeler-tips=).
1659 1660
1660 *** COMMENT Triangle Math 1661 *** Triangle Math
1661 1662
1662 The rigid objects which make up a creature have an underlying 1663 The rigid objects which make up a creature have an underlying
1663 =Geometry=, which is a =Mesh= plus a =Material= and other 1664 =Geometry=, which is a =Mesh= plus a =Material= and other
1664 important data involved with displaying the object. 1665 important data involved with displaying the object.
1665 1666
1673 1674
1674 #+caption: Programs to extract triangles from a geometry and get 1675 #+caption: Programs to extract triangles from a geometry and get
1675 #+caption: their verticies in both world and UV-coordinates. 1676 #+caption: their verticies in both world and UV-coordinates.
1676 #+name: get-triangles 1677 #+name: get-triangles
1677 #+begin_listing clojure 1678 #+begin_listing clojure
1679 #+BEGIN_SRC clojure
1678 (defn triangle 1680 (defn triangle
1679 "Get the triangle specified by triangle-index from the mesh." 1681 "Get the triangle specified by triangle-index from the mesh."
1680 [#^Geometry geo triangle-index] 1682 [#^Geometry geo triangle-index]
1681 (triangle-seq 1683 (triangle-seq
1682 (let [scratch (Triangle.)] 1684 (let [scratch (Triangle.)]
1721 [#^Geometry geo image] 1723 [#^Geometry geo image]
1722 (let [height (.getHeight image) 1724 (let [height (.getHeight image)
1723 width (.getWidth image)] 1725 width (.getWidth image)]
1724 (map (partial pixel-triangle geo image) 1726 (map (partial pixel-triangle geo image)
1725 (range (.getTriangleCount (.getMesh geo)))))) 1727 (range (.getTriangleCount (.getMesh geo))))))
1728 #+END_SRC
1726 #+end_listing 1729 #+end_listing
1727 1730
1728 *** The Affine Transform from one Triangle to Another 1731 *** The Affine Transform from one Triangle to Another
1729 1732
1730 =pixel-triangles= gives us the mesh triangles expressed in pixel 1733 =pixel-triangles= gives us the mesh triangles expressed in pixel
1794 (triangle->matrix4f tri-2) 1797 (triangle->matrix4f tri-2)
1795 (.invert (triangle->matrix4f tri-1)))) 1798 (.invert (triangle->matrix4f tri-1))))
1796 #+END_SRC 1799 #+END_SRC
1797 #+end_listing 1800 #+end_listing
1798 1801
1799 *** COMMENT Triangle Boundaries 1802 *** Triangle Boundaries
1800 1803
1801 For efficiency's sake I will divide the tactile-profile image into 1804 For efficiency's sake I will divide the tactile-profile image into
1802 small squares which inscribe each pixel-triangle, then extract the 1805 small squares which inscribe each pixel-triangle, then extract the
1803 points which lie inside the triangle and map them to 3D-space using 1806 points which lie inside the triangle and map them to 3D-space using
1804 =triangle-transform= above. To do this I need a function, 1807 =triangle-transform= above. To do this I need a function,
1845 (same-side? vert-2 vert-3 vert-1 p) 1848 (same-side? vert-2 vert-3 vert-1 p)
1846 (same-side? vert-3 vert-1 vert-2 p)))) 1849 (same-side? vert-3 vert-1 vert-2 p))))
1847 #+END_SRC 1850 #+END_SRC
1848 #+end_listing 1851 #+end_listing
1849 1852
1850 *** COMMENT Feeler Coordinates 1853 *** Feeler Coordinates
1851 1854
1852 The triangle-related functions above make short work of 1855 The triangle-related functions above make short work of
1853 calculating the positions and orientations of each feeler in 1856 calculating the positions and orientations of each feeler in
1854 world-space. 1857 world-space.
1855 1858
1917 [#^Geometry geo image] 1920 [#^Geometry geo image]
1918 (collapse (reduce concat (feeler-pixel-coords geo image)))) 1921 (collapse (reduce concat (feeler-pixel-coords geo image))))
1919 #+END_SRC 1922 #+END_SRC
1920 #+end_listing 1923 #+end_listing
1921 1924
1922 *** COMMENT Simulated Touch 1925 *** Simulated Touch
1923 1926
1924 Now that the functions to construct feelers are complete, 1927 Now that the functions to construct feelers are complete,
1925 =touch-kernel= generates functions to be called from within a 1928 =touch-kernel= generates functions to be called from within a
1926 simulation that perform the necessary physics collisions to 1929 simulation that perform the necessary physics collisions to
1927 collect tactile data, and =touch!= recursively applies it to every 1930 collect tactile data, and =touch!= recursively applies it to every