rlm@302: Dear Professor Winston,
rlm@302: 
rlm@302: I'm ready for you to look through the work that I've done so far. It's
rlm@302: a sequence of posts describing the different simulated senses I've
rlm@302: implemented, with videos. 
rlm@302: 
rlm@302: It's "blocks world reloaded", because like you say, you need multiple
rlm@302: senses to enable intelligence.
rlm@302: 
rlm@302: Please look through the videos and skim the text and tell me what
rlm@302: you think:
rlm@302: 
rlm@302: Introduction:
rlm@302: http://aurellem.org/cortex/html/intro.html
rlm@302: http://aurellem.org/cortex/html/sense.html
rlm@302: 
rlm@302: http://aurellem.org/cortex/html/body.html  -- simulated physical bodies
rlm@302: http://aurellem.org/cortex/html/vision.html -- simulated eyes
rlm@302: http://aurellem.org/cortex/html/hearing.html -- simulated ears
rlm@302: http://aurellem.org/cortex/html/touch.html   -- simulated skin/hairs
rlm@302: http://aurellem.org/cortex/html/proprioception.html -- simulated proprioception
rlm@302: http://aurellem.org/cortex/html/movement.html       -- simulated muscles
rlm@302: http://aurellem.org/cortex/html/integration.html  -- full demonstration 
rlm@302: 
rlm@302: In particular, look at the video at
rlm@302: http://aurellem.org/cortex/html/integration.html.  It shows a
rlm@302: simulated hand equipped with all of the senses I've built so far.
rlm@302: 
rlm@302: There's some more background information and full source code at 
rlm@302: http://aurellem.org
rlm@302: 
rlm@302: If you can't see a video, let me know and I'll upload it to YouTube so
rlm@302: you can see it.
rlm@302: 
rlm@302: 
rlm@302: 
rlm@302: 
rlm@302: Now, I need your help moving forward. Can I use this work as a base
rlm@302: for a Masters thesis with you when I come back to MIT this coming Fall?
rlm@302: What critiques and project ideas do you have after looking through
rlm@302: what I've done so far?
rlm@302: 
rlm@302: I have some ideas on where I can go with this project but I think you
rlm@302: will have some better ones.
rlm@302: 
rlm@302: Here are some possible projects I might do with this as a base that I
rlm@302: think would be worthy Masters projects.
rlm@302: 
rlm@302:  - HACKER for writing muscle-control programs : Presented with
rlm@302:    low-level muscle control/ sense API, generate higher level programs
rlm@302:    for accomplishing various stated goals. Example goals might be
rlm@302:    "extend all your fingers" or "move your hand into the area with
rlm@302:    blue light" or "decrease the angle of this joint".  It would be
rlm@302:    like Sussman's HACKER, except it would operate with much more data
rlm@302:    in a more realistic world.  Start off with "calisthenics" to
rlm@302:    develop subroutines over the motor control API.  This would be the
rlm@302:    "spinal chord" of a more intelligent creature.
rlm@302: 
rlm@302:  - Create hundreds of small creatures and have them do simple
rlm@302:    simulated swarm tasks.
rlm@302:  
rlm@302:  - A joint that learns what sort of joint it (cone, point-to-point,
rlm@302:    hinge, etc.) is by correlating exploratory muscle movements with
rlm@302:    vision.
rlm@302: 
rlm@302:  - Something with cross-modal clustering using the rich sense
rlm@302:    data. This might prove difficult due to the higher dimensionality
rlm@302:    of my senses.
rlm@302: 
rlm@302:  - Simulated Imagination --- this would involve a creature with an
rlm@302:    effector which creates an /entire new sub-simulation/ where the
rlm@302:    creature has direct control over placement/creation of objects via
rlm@302:    simulated telekinesis. The creature observes this sub-world through
rlm@302:    it's normal senses and uses its observations to make predictions
rlm@302:    about it's top level world.
rlm@302: 
rlm@302:  - Hook it up with Genesis --- I could make a "semantic effector"
rlm@302:    which marks objects/sensory states with semantic information. In
rlm@302:    conjunction with Simulated Imagination, and HACKER-like motor
rlm@302:    control, Genesis might be able to ask simple questions like "stack
rlm@302:    two blocks together and hit the top one with your hand; does the
rlm@302:    bottom block move?" and the system could answer "yes".  This would
rlm@302:    be rather complicated to do and involves many of the above
rlm@302:    projects, but there may be a way to scale it down to Master's
rlm@302:    thesis size.
rlm@302: 
rlm@302:  - Make a virtual computer in the virtual world which with which the
rlm@302:    creature interacts using its fingers to press keys on a virtual
rlm@302:    keyboard. The creature can access the internet, watch videos, take
rlm@302:    over the world, anything it wants. (This is probably not worthy of
rlm@302:    a Masters project, I just thought it was a neat idea. It's possible
rlm@302:    to use videos/etc in the simulated world at any rate.)
rlm@302: 
rlm@302: 
rlm@302: I can't wait to hear your critiques and ideas.  If you think I
rlm@302: shouldn't use this system as a base and should instead do something
rlm@302: else, that's fine too. 
rlm@302: 
rlm@302: On a related note, can I be considered for the position of TA next
rlm@302: year for 6.034 or 6.xxx?
rlm@302: 
rlm@302: sincerely,
rlm@302: --Robert McIntyre