1.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     1.2 +++ b/org/goodlit.org	Tue Jun 03 13:24:58 2014 -0400
     1.3 @@ -0,0 +1,114 @@
     1.4 +#+TITLE: Good readables
     1.5 +#+AUTHOR: Dylan Holmes
     1.6 +#+EMAIL:     rlm@mit.edu
     1.7 +#+SETUPFILE: ../../aurellem/org/setup.org
     1.8 +#+INCLUDE:   ../../aurellem/org/level-0.org
     1.9 +#+MATHJAX: align:"left" mathml:t path:"http://www.aurellem.org/MathJax/MathJax.js"
    1.10 +#+
    1.11 +
    1.12 +* Non-narrative books
    1.13 +| Title                                                          | Author                | About                                                                                                                                                                                     |
    1.14 +|----------------------------------------------------------------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    1.15 +| Formal Theories of the Commonsense World                       | Hobbs \amp{} Moore    | Essays about the rules of intuitive physics---how we naturally reason about the physical world before we learn about (for example) molecules, inverse square laws, and quantum mechanics. |
    1.16 +| Vision                                                         | Marr                  | Describes successes in computer vision, and outlines effective policies for thinking about and researching intelligence.                                                                  |
    1.17 +| The Structure and Interpretation of Classical Mechanics (SICM) | Sussman \amp{} Wisdom | This book uses programming as a tool for analysing otherwise-intractable physical systems, and as a medium for explicit, unambiguous expression of ideas\mdash{}as an alternative to mathematical notation, the meanings of which often depend on implicit conventions.  |
    1.18 +| The Computer Revolution in Philosophy                          | Sloman                | Elucidates the essential role of philosophy in science, and the newfound revolutionary role of computers and computational language in philosophy (especially in philosophy of mind).     |
    1.19 +| Freedom Evolves                                                | Dennett               | An eminently readable book on how free will can develop within a deterministic universe in general, and in mechanistic robots in particular.                                              |
    1.20 +| Philosophical Investigations                                   | Wittgenstein          | A seminal, if disconnected, philosophical treatise which describes how meanings of sentences arise through learned, lazily-evaluated \ldquo{}choreographies\rdquo{}.             |
    1.21 +| The Emotion Machine                                            | Minsky                | A wonderful compendium of ideas about the construction of intelligence, presented in clever, non-technical language.                                                                      |
    1.22 +| The Myth of Mental Illness                                     | Szasz                 | A thought-provoking look at mental illness/treatment as a form of social control.                                                                                                         |
    1.23 +| BUGS in Writing                                                | Dupre                 | A whimsical and erudite handbook for writing clearly, with an emphasis on writing for science/technology.                                                                                 |
    1.24 +| The Visual Display of Quantitative Information                 | Tufte                 | A learned tour of good practices for organizing information and  communicating large amounts of it with minimal clutter.                                                                  |
    1.25 +| The Algebra of Programming                                     | Odgen \amp{} de Moor  | An abstract but deep textbook which describes how programs can be described, combined, optimized, and analyzed using a new kind of algebra.                                               |
    1.26 +| Probability Theory: The logic of science                       | Jaynes                | A heterodoxical textbook which presents probability theory as a kind of logical inference.                                                                                                |
    1.27 +| Category Theory                                                | Awodey                | Friendly and thorough introduction to the ideas of category theory \mdash{} less imposing than (but a good companion to) MacLane's /Categories for the Working Mathematician/.            |
    1.28 +| Algebra: Chapter 0                                             | Aluffi                | Friendly and thorough introduction to the ideas of abstract algebra\mdash{} less imposing than MacLane \amp{} Birkhoff's /Algebra/.                                                       |
    1.29 +| Principles of Quantum Mechanics                                | Shankar               | A careful, solid introduction to quantum mechanics. Less chatty than Griffith's /Introduction to Quantum Mechanics/.                                                                |
    1.30 +  
    1.31 +  
    1.32 +* COMMENT Articles, etc.
    1.33 +
    1.34 +| Title                                 | Author   | About                                                                                                            |
    1.35 +|---------------------------------------+----------+------------------------------------------------------------------------------------------------------------------|
    1.36 +| A Computer Model of Skill Acquisition | Sussman  | Sussman explores learning how to do things as the process of developing and debugging programs to do them. He finds that this paradigm is an effective way to think about learning; for example, with each lesson, his program learns something new. He incidentally also concludes that automatic program-writers are impracticible, because ideas are often developed in a /cycle/ of suggesting/coding/debugging. |
    1.37 +| A Universal Banach Space              | Leinster | Very brief, but neat: integration is a \ldquo{}universal object\rdquo{} (in the terminology of Category Theory.) |
    1.38 +|                                       |          |                                                                                                                  |
    1.39 +
    1.40 +
    1.41 +* Narratives
    1.42 +| Title                                | Author              | About                                                                                                                                                                                                                                                                                            |
    1.43 +|--------------------------------------+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    1.44 +| The Adventures of Sherlock Holmes    | Doyle, Arthur Conan | An austere and brilliant detective who possesses razor-sharp skills in observation, analysis of character, and collecting mundane facts. I particularly like the Sherlock Holmes novels because Holmes's talents are apparently attainable through practice.  |
    1.45 +| The Lives of Christopher Chant       | Jones, Diana Wynne  | An excellent story, with wonderously fun magic and enjoyable characters. Part of the /Chrestomanci/ series.                                                                                                                                                                                      |
    1.46 +| The Hitchhiker's Guide to the Galaxy | Adams, Douglas      | A subtly hilarous existentialist novel about how bureaucratic incompetence leads to the destruction of planet Earth, and about the lone unfortunate British man who survives.                                                                                                                    |
    1.47 +| Matter                               | Banks, Iain         | An epic science fiction novel. As with all novels in the /Culture/ series, /Matter/ is remarkable for the following: (1) Tangibly portraying the vastness of space and the incomprehensibility of advanced intelligence, (2) Artful diction, (3) Fearless use of unorthodox narrative structure. |
    1.48 +| The Magicians                        | Grossman, Lev       | An approximate analogy: \ldquo{}The Magicians\rdquo{} is to \ldquo{}Narnia\rdquo{} as \ldquo{}Wicked\rdquo{} is to \ldquo{}Oz\rdquo{}.  A gritty realistic book about a perennially morose overachiever and his adventures in relentlessly unsentimental places.                                 |
    1.49 +
    1.50 +# Aaron Sloman
    1.51 +# Dennet's humor book
    1.52 +# Hofstadter's I Am a Strange Loop
    1.53 +
    1.54 +# Woolf's The Waves
    1.55 +
    1.56 +
    1.57 +
    1.58 +
    1.59 +# * Books Relating to Artificial Intelligence
    1.60 +
    1.61 +# ** The Emotion Machine, by Marvin Minsky
    1.62 +# ** Formal Theories of the Commmonsense World, by Hobbs and Moore
    1.63 +# ([[http://books.google.com/books/about/Formal_Theories_of_the_Commonsense_World.html?id=aUO2PCw-vdgC][link to Formal Theories]])
    1.64 +
    1.65 +# ** Vision, by David Marr
    1.66 +
    1.67 +# ** Philosophical Investigations, by Ludwig Wittgenstein
    1.68 +
    1.69 +# ** Probability Theory: The Logic of Science, by Edwin Jaynes
    1.70 +
    1.71 +# ** The Algebra of Programming, by Odgen and de Moor
    1.72 +
    1.73 +
    1.74 +# * Other Exemplary Books
    1.75 +
    1.76 +# ** Real and Complex Analysis, by Walter Rudin
    1.77 +# This book is a model of terseness in exposition --- it's theoretically
    1.78 +# elegant, though pedagogically impenetrable.
    1.79 +
    1.80 +# ** Category Theory, by Steve Awodey
    1.81 +# This book is a friendly and exciting introduction to category theory.
    1.82 +
    1.83 +# ** Principles of Quantum Mechanics, by Ramamurti Shankar
    1.84 +# Rigorous yet engaging.
    1.85 +
    1.86 +# ** Algebra: Chapter 0, by Aluffi
    1.87 +
    1.88 +# ** BUGS in Writing, by Lynn Dupr\eacute{}
    1.89 +
    1.90 +# ** The Visual Display of Quantitative Information, by Edward Tufte
    1.91 +
    1.92 +# # Piaget
    1.93 +# # Hans Freudenthal
    1.94 +# ** How to Solve It, by Polya
    1.95 +
    1.96 +   
    1.97 +# #* Interesting Papers
    1.98 +
    1.99 +# #** A Universal Banach Space, by Tom Leinster
   1.100 +# A short paper in which the set of Lebesgue-measureable functions on
   1.101 +# [0,1] emerges as the initial object in a category of  Banach
   1.102 +# spaces, and in which integration is a catamorphism. 
   1.103 +
   1.104 +# #** The Eudoxus Real numbers, by R. Arthan
   1.105 +# This paper constructs the real numbers in terms of
   1.106 +# proportionalities.
   1.107 +
   1.108 +
   1.109 +# http://arxiv.org/abs/math/0405454
   1.110 +
   1.111 +# #** Tolerance Space Theory and Some Applications, by Sossinsky
   1.112 +
   1.113 +# #Tolerance spaces
   1.114 +# #The Eudoxus real numbers
   1.115 +
   1.116 +
   1.117 +# # AARON SLOMAN

     2.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     2.2 +++ b/org/moon.org	Tue Jun 03 13:24:58 2014 -0400
     2.3 @@ -0,0 +1,20 @@
     2.4 +#+BEGIN_VERSE
     2.5 +Heaven is the darkness of a moon in a mirror
     2.6 +In whose dark glass, long-lost love refracts into a promise
     2.7 +And our incipient sorrows witness their imminent conclusion
     2.8 +In mere gray inches of earth.
     2.9 +
    2.10 +But before this framed firmament, we can merely stand and sigh
    2.11 +And see our lives begun knotted with their own undoing
    2.12 +Before this stygian wall, this shroud of narcissus.
    2.13 +
    2.14 +Yet its stillness is not our stillness,
    2.15 +And beyond the burnished gleaming edges of its flat sky
    2.16 +A new sun rises
    2.17 +Gilding the vast shoreline of our memory
    2.18 +And the dawn-fogged peaks of our unflagging aspiration.
    2.19 +#+END_VERSE
    2.20 +
    2.21 +# aspire next.
    2.22 +#  towards which we steadfastly tread.
    2.23 +# steadfast aspiration.

     3.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     3.2 +++ b/org/sloman-old.html	Tue Jun 03 13:24:58 2014 -0400
     3.3 @@ -0,0 +1,1348 @@
     3.4 +<?xml version="1.0" encoding="utf-8"?>
     3.5 +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
     3.6 +               "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
     3.7 +<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
     3.8 +<head>
     3.9 +<title>Transcript of Aaron Sloman - Artificial Intelligence - Psychology - Oxford Interview</title>
    3.10 +<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
    3.11 +<meta name="title" content="Transcript of Aaron Sloman - Artificial Intelligence - Psychology - Oxford Interview"/>
    3.12 +<meta name="generator" content="Org-mode"/>
    3.13 +<meta name="generated" content="2013-10-04 18:49:53 UTC"/>
    3.14 +<meta name="author" content="Dylan Holmes"/>
    3.15 +<meta name="description" content=""/>
    3.16 +<meta name="keywords" content=""/>
    3.17 +<style type="text/css">
    3.18 + <!--/*--><![CDATA[/*><!--*/
    3.19 +  html { font-family: Times, serif; font-size: 12pt; }
    3.20 +  .title  { text-align: center; }
    3.21 +  .todo   { color: red; }
    3.22 +  .done   { color: green; }
    3.23 +  .tag    { background-color: #add8e6; font-weight:normal }
    3.24 +  .target { }
    3.25 +  .timestamp { color: #bebebe; }
    3.26 +  .timestamp-kwd { color: #5f9ea0; }
    3.27 +  .right  {margin-left:auto; margin-right:0px;  text-align:right;}
    3.28 +  .left   {margin-left:0px;  margin-right:auto; text-align:left;}
    3.29 +  .center {margin-left:auto; margin-right:auto; text-align:center;}
    3.30 +  p.verse { margin-left: 3% }
    3.31 +  pre {
    3.32 +	border: 1pt solid #AEBDCC;
    3.33 +	background-color: #F3F5F7;
    3.34 +	padding: 5pt;
    3.35 +	font-family: courier, monospace;
    3.36 +        font-size: 90%;
    3.37 +        overflow:auto;
    3.38 +  }
    3.39 +  table { border-collapse: collapse; }
    3.40 +  td, th { vertical-align: top;  }
    3.41 +  th.right  { text-align:center;  }
    3.42 +  th.left   { text-align:center;   }
    3.43 +  th.center { text-align:center; }
    3.44 +  td.right  { text-align:right;  }
    3.45 +  td.left   { text-align:left;   }
    3.46 +  td.center { text-align:center; }
    3.47 +  dt { font-weight: bold; }
    3.48 +  div.figure { padding: 0.5em; }
    3.49 +  div.figure p { text-align: center; }
    3.50 +  div.inlinetask {
    3.51 +    padding:10px;
    3.52 +    border:2px solid gray;
    3.53 +    margin:10px;
    3.54 +    background: #ffffcc;
    3.55 +  }
    3.56 +  textarea { overflow-x: auto; }
    3.57 +  .linenr { font-size:smaller }
    3.58 +  .code-highlighted {background-color:#ffff00;}
    3.59 +  .org-info-js_info-navigation { border-style:none; }
    3.60 +  #org-info-js_console-label { font-size:10px; font-weight:bold;
    3.61 +                               white-space:nowrap; }
    3.62 +  .org-info-js_search-highlight {background-color:#ffff00; color:#000000;
    3.63 +                                 font-weight:bold; }
    3.64 +  /*]]>*/-->
    3.65 +</style>
    3.66 +<link rel="stylesheet" type="text/css" href="../css/sloman.css" /> 
    3.67 +<script type="text/javascript">
    3.68 +<!--/*--><![CDATA[/*><!--*/
    3.69 + function CodeHighlightOn(elem, id)
    3.70 + {
    3.71 +   var target = document.getElementById(id);
    3.72 +   if(null != target) {
    3.73 +     elem.cacheClassElem = elem.className;
    3.74 +     elem.cacheClassTarget = target.className;
    3.75 +     target.className = "code-highlighted";
    3.76 +     elem.className   = "code-highlighted";
    3.77 +   }
    3.78 + }
    3.79 + function CodeHighlightOff(elem, id)
    3.80 + {
    3.81 +   var target = document.getElementById(id);
    3.82 +   if(elem.cacheClassElem)
    3.83 +     elem.className = elem.cacheClassElem;
    3.84 +   if(elem.cacheClassTarget)
    3.85 +     target.className = elem.cacheClassTarget;
    3.86 + }
    3.87 +/*]]>*///-->
    3.88 +</script>
    3.89 +
    3.90 +</head>
    3.91 +<body>
    3.92 +
    3.93 +
    3.94 +<div id="content">
    3.95 +<h1 class="title">Transcript of Aaron Sloman - Artificial Intelligence - Psychology - Oxford Interview</h1>
    3.96 +
    3.97 +
    3.98 +<blockquote>
    3.99 +
   3.100 +
   3.101 +
   3.102 +
   3.103 +
   3.104 +
   3.105 +
   3.106 +
   3.107 +
   3.108 +
   3.109 +
   3.110 +
   3.111 +
   3.112 +
   3.113 +
   3.114 +
   3.115 +<p>
   3.116 +<b>Editor's note:</b> This is a working draft transcript which I made of
   3.117 +<a href="http://www.youtube.com/watch?feature=player_detailpage&amp;v=iuH8dC7Snno">this nice interview</a> of Aaron Sloman. Having just finished one
   3.118 +iteration of transcription, I still need to go in and clean up the
   3.119 +formatting and fix the parts that I misheard, so you can expect the
   3.120 +text to improve significantly in the near future.
   3.121 +</p>
   3.122 +<p>
   3.123 +To the extent that this is my work, you have my permission to make
   3.124 +copies of this transcript for your own purposes. Also, feel free to
   3.125 +e-mail me with comments or corrections.
   3.126 +</p>
   3.127 +<p>
   3.128 +You can send mail to <code>transcript@aurellem.org</code>.
   3.129 +</p>
   3.130 +<p>
   3.131 +Cheers,
   3.132 +</p>
   3.133 +<p>
   3.134 +&mdash;Dylan
   3.135 +</p>
   3.136 +</blockquote>
   3.137 +
   3.138 +
   3.139 +
   3.140 +
   3.141 +
   3.142 +<div id="table-of-contents">
   3.143 +<h2>Table of Contents</h2>
   3.144 +<div id="text-table-of-contents">
   3.145 +<ul>
   3.146 +<li><a href="#sec-1">1 Introduction</a>
   3.147 +<ul>
   3.148 +<li><a href="#sec-1-1">1.1 Aaron Sloman evolves into a philosopher of AI</a></li>
   3.149 +<li><a href="#sec-1-2">1.2 AI is hard, in part because there are tempting non-problems.</a></li>
   3.150 +</ul>
   3.151 +</li>
   3.152 +<li><a href="#sec-2">2 What problems of intelligence did evolution solve?</a>
   3.153 +<ul>
   3.154 +<li><a href="#sec-2-1">2.1 Intelligence consists of solutions to many evolutionary problems; no single development (e.g. communication) was key to human-level intelligence.</a></li>
   3.155 +<li><a href="#sec-2-2">2.2 Speculation about how communication might have evolved from internal lanagues.</a></li>
   3.156 +</ul>
   3.157 +</li>
   3.158 +<li><a href="#sec-3">3 How do language and internal states relate to AI?</a>
   3.159 +<ul>
   3.160 +<li><a href="#sec-3-1">3.1 In AI, false assumptions can lead investigators astray.</a></li>
   3.161 +<li><a href="#sec-3-2">3.2 Example: Vision is not just about finding surfaces, but about finding affordances.</a></li>
   3.162 +<li><a href="#sec-3-3">3.3 Online and offline intelligence</a></li>
   3.163 +<li><a href="#sec-3-4">3.4 Example: Even toddlers use sophisticated geometric knowledge</a></li>
   3.164 +</ul>
   3.165 +</li>
   3.166 +<li><a href="#sec-4">4 Animal intelligence</a>
   3.167 +<ul>
   3.168 +<li><a href="#sec-4-1">4.1 The priority is <i>cataloguing</i> what competences have evolved, not ranking them.</a></li>
   3.169 +<li><a href="#sec-4-2">4.2 AI can be used to test philosophical theories</a></li>
   3.170 +</ul>
   3.171 +</li>
   3.172 +<li><a href="#sec-5">5 Is abstract general intelligence feasible?</a>
   3.173 +<ul>
   3.174 +<li><a href="#sec-5-1">5.1 It's misleading to compare the brain and its neurons to a computer made of transistors</a></li>
   3.175 +<li><a href="#sec-5-2">5.2 For example, brains may rely heavily on chemical information processing</a></li>
   3.176 +<li><a href="#sec-5-3">5.3 Brain algorithms may simply be optimized for certain kinds of information processing other than bit manipulations</a></li>
   3.177 +<li><a href="#sec-5-4">5.4 Example: find the shortest path by dangling strings</a></li>
   3.178 +<li><a href="#sec-5-5">5.5 In sum, we know surprisingly little about the kinds of problems that evolution solved, and the manner in which they were solved.</a></li>
   3.179 +</ul>
   3.180 +</li>
   3.181 +<li><a href="#sec-6">6 A singularity of cognitive catch-up</a>
   3.182 +<ul>
   3.183 +<li><a href="#sec-6-1">6.1 What if it will take a lifetime to learn enough to make something new?</a></li>
   3.184 +</ul>
   3.185 +</li>
   3.186 +<li><a href="#sec-7">7 Spatial reasoning: a difficult problem</a>
   3.187 +<ul>
   3.188 +<li><a href="#sec-7-1">7.1 Example: Spatial proof that the angles of any triangle add up to a half-circle</a></li>
   3.189 +<li><a href="#sec-7-2">7.2 Geometric results are fundamentally different than experimental results in chemistry or physics.</a></li>
   3.190 +</ul>
   3.191 +</li>
   3.192 +<li><a href="#sec-8">8 Is near-term artificial general intelligence likely?</a>
   3.193 +<ul>
   3.194 +<li><a href="#sec-8-1">8.1 Two interpretations: a single mechanism for all problems, or many mechanisms unified in one program.</a></li>
   3.195 +</ul>
   3.196 +</li>
   3.197 +<li><a href="#sec-9">9 Abstract General Intelligence impacts</a></li>
   3.198 +</ul>
   3.199 +</div>
   3.200 +</div>
   3.201 +
   3.202 +<div id="outline-container-1" class="outline-2">
   3.203 +<h2 id="sec-1"><span class="section-number-2">1</span> Introduction</h2>
   3.204 +<div class="outline-text-2" id="text-1">
   3.205 +
   3.206 +
   3.207 +
   3.208 +</div>
   3.209 +
   3.210 +<div id="outline-container-1-1" class="outline-3">
   3.211 +<h3 id="sec-1-1"><span class="section-number-3">1.1</span> Aaron Sloman evolves into a philosopher of AI</h3>
   3.212 +<div class="outline-text-3" id="text-1-1">
   3.213 +
   3.214 +<p>[0:09] My name is Aaron Sloman. My first degree many years ago in
   3.215 +Capetown University was in Physics and Mathematics, and I intended to
   3.216 +go and be a mathematician. I came to Oxford and encountered
   3.217 +philosophers &mdash; I had started reading philosophy and discussing
   3.218 +philosophy before then, and then I found that there were philosophers
   3.219 +who said things about mathematics that I thought were wrong, so
   3.220 +gradually got more and more involved in [philosophy] discussions and
   3.221 +switched to doing philosophy DPhil. Then I became a philosophy
   3.222 +lecturer and about six years later, I was introduced to artificial
   3.223 +intelligence when I was a lecturer at Sussex University in philosophy
   3.224 +and I very soon became convinced that the best way to make progress in
   3.225 +both areas of philosophy (including philosophy of mathematics which I
   3.226 +felt i hadn't dealt with adequately in my DPhil) about the philosophy
   3.227 +of mathematics, philosophy of mind, philsophy of language and all
   3.228 +those things&mdash;the best way was to try to design and test working
   3.229 +fragments of mind and maybe eventually put them all together but
   3.230 +initially just working fragments that would do various things.
   3.231 +</p>
   3.232 +<p>
   3.233 +[1:12] And I learned to program and ~ with various other people
   3.234 +including ~Margaret Boden whom you've interviewed, developed&mdash;helped
   3.235 +develop an undergraduate degree in AI and other things and also began
   3.236 +to do research in AI and so on which I thought of as doing philosophy,
   3.237 +primarily.
   3.238 +</p>
   3.239 +<p>
   3.240 +[1:29] And then I later moved to the University of Birmingham and I
   3.241 +was there &mdash; I came in 1991 &mdash; and I've been retired for a while but
   3.242 +I'm not interested in golf or gardening so I just go on doing full
   3.243 +time research and my department is happy to keep me on without paying
   3.244 +me and provide space and resources and I come, meeting bright people
   3.245 +at conferences and try to learn and make progress if I can.
   3.246 +</p>
   3.247 +</div>
   3.248 +
   3.249 +</div>
   3.250 +
   3.251 +<div id="outline-container-1-2" class="outline-3">
   3.252 +<h3 id="sec-1-2"><span class="section-number-3">1.2</span> AI is hard, in part because there are tempting non-problems.</h3>
   3.253 +<div class="outline-text-3" id="text-1-2">
   3.254 +
   3.255 +
   3.256 +<p>
   3.257 +One of the things I learnt and understood more and more over the many
   3.258 +years &mdash; forty years or so since I first encountered AI &mdash; is how
   3.259 +hard the problems are, and in part that's because it's very often
   3.260 +tempting to <i>think</i> the problem is something different from what it
   3.261 +actually is, and then people design solutions to the non-problems, and
   3.262 +I think of most of my work now as just helping to clarify what the
   3.263 +problems are: what is it that we're trying to explain &mdash; and maybe
   3.264 +this is leading into what you wanted to talk about:
   3.265 +</p>
   3.266 +<p>
   3.267 +I now think that one of the ways of getting a deep understanding of
   3.268 +that is to find out what were the problems that biological evolution
   3.269 +solved, because we are a product of <i>many</i> solutions to <i>many</i>
   3.270 +problems, and if we just try to go in and work out what the whole
   3.271 +system is doing, we may get it all wrong, or badly wrong.
   3.272 +</p>
   3.273 +
   3.274 +</div>
   3.275 +</div>
   3.276 +
   3.277 +</div>
   3.278 +
   3.279 +<div id="outline-container-2" class="outline-2">
   3.280 +<h2 id="sec-2"><span class="section-number-2">2</span> What problems of intelligence did evolution solve?</h2>
   3.281 +<div class="outline-text-2" id="text-2">
   3.282 +
   3.283 +
   3.284 +
   3.285 +</div>
   3.286 +
   3.287 +<div id="outline-container-2-1" class="outline-3">
   3.288 +<h3 id="sec-2-1"><span class="section-number-3">2.1</span> Intelligence consists of solutions to many evolutionary problems; no single development (e.g. communication) was key to human-level intelligence.</h3>
   3.289 +<div class="outline-text-3" id="text-2-1">
   3.290 +
   3.291 +
   3.292 +<p>
   3.293 +[2:57] Well, first I would challenge that we are the dominant
   3.294 +species. I know it looks like that but actually if you count biomass,
   3.295 +if you count number of species, if you count number of individuals,
   3.296 +the dominant species are microbes &mdash; maybe not one of them but anyway
   3.297 +they're the ones who dominate in that sense, and furthermore we are
   3.298 +mostly &mdash; we are largely composed of microbes, without which we
   3.299 +wouldn't survive.
   3.300 +</p>
   3.301 +
   3.302 +<p>
   3.303 +[3:27] But there are things that make humans (you could say) best at
   3.304 +those things, or worst at those things, but it's a combination.  And I
   3.305 +think it was a collection of developments of which there isn't any
   3.306 +single one. [] there might be, some people say, human language which
   3.307 +changed everything. By our human language, they mean human
   3.308 +communication in words, but I think that was a later development from
   3.309 +what must have started as the use of <i>internal</i> forms of
   3.310 +representation &mdash; which are there in nest-building birds, in
   3.311 +pre-verbal children, in hunting mammals &mdash; because you can't take in
   3.312 +information about a complex structured environment in which things can
   3.313 +change and you may have to be able to work out what's possible and
   3.314 +what isn't possible, without having some way of representing the
   3.315 +components of the environment, their relationships, the kinds of
   3.316 +things they can and can't do, the kinds of things you might or might
   3.317 +not be able to do &mdash; and <i>that</i> kind of capability needs internal
   3.318 +languages, and I and colleagues [at Birmingham] have been referring to
   3.319 +them as generalized languages because some people object to
   3.320 +referring&hellip;to using language to refer to something that isn't used
   3.321 +for communication. But from that viewpoint, not only humans but many
   3.322 +other animals developed abilities to do things to their environment to
   3.323 +make them more friendly to themselves, which depended on being able to
   3.324 +represent possible futures, possible actions, and work out what's the
   3.325 +best thing to do.
   3.326 +</p>
   3.327 +<p>
   3.328 +[5:13] And nest-building in corvids for instance&mdash;crows, magpies,
   3.329 + [hawks], and so on &mdash; are way beyond what current robots can do, and
   3.330 + in fact I think most humans would be challenged if they had to go and
   3.331 + find a collection of twigs, one at a time, maybe bring them with just
   3.332 + one hand &mdash; or with your mouth &mdash; and assemble them into a
   3.333 + structure that, you know, is shaped like a nest, and is fairly rigid,
   3.334 + and you could trust your eggs in them when wind blows. But they're
   3.335 + doing it, and so &hellip; they're not our evolutionary ancestors, but
   3.336 + they're an indication &mdash; and that example is an indication &mdash; of
   3.337 + what must have evolved in order to provide control over the
   3.338 + environment in <i>that</i> species.
   3.339 +</p>
   3.340 +</div>
   3.341 +
   3.342 +</div>
   3.343 +
   3.344 +<div id="outline-container-2-2" class="outline-3">
   3.345 +<h3 id="sec-2-2"><span class="section-number-3">2.2</span> Speculation about how communication might have evolved from internal lanagues.</h3>
   3.346 +<div class="outline-text-3" id="text-2-2">
   3.347 +
   3.348 +<p>[5:56] And I think hunting mammals, fruit-picking mammals, mammals
   3.349 +that can rearrange parts of the environment, provide shelters, needed
   3.350 +to have &hellip;. also needed to have ways of representing possible
   3.351 +futures, not just what's there in the environment. I think at a later
   3.352 +stage, that developed into a form of communication, or rather the
   3.353 +<i>internal</i> forms of representation became usable as a basis for
   3.354 +providing [context] to be communicated. And that happened, I think,
   3.355 +initially through performing actions that expressed intentions, and
   3.356 +probably led to situtations where an action (for instance, moving some
   3.357 +large object) was performed more easily, or more successfully, or more
   3.358 +accurately if it was done collaboratively. So someone who had worked
   3.359 +out what to do might start doing it, and then a conspecific might be
   3.360 +able to work out what the intention is, because that person has the
   3.361 +<i>same</i> forms of representation and can build theories about what's
   3.362 +going on, and might then be able to help.
   3.363 +</p>
   3.364 +<p>
   3.365 +[7:11] You can imagine that if that started happening more (a lot of
   3.366 +collaboration based on inferred intentions and plans) then sometimes
   3.367 +the inferences might be obscure and difficult, so the <i>actions</i> might
   3.368 +be enhanced to provide signals as to what the intention is, and what
   3.369 +the best way is to help, and so on.
   3.370 +</p>
   3.371 +<p>
   3.372 +[7:35] So, this is all handwaving and wild speculation, but I think
   3.373 +it's consistent with a large collection of facts which one can look at
   3.374 +&mdash; and find if one looks for them, but one won't know if [some]one
   3.375 +doesn't look for them &mdash; about the way children, for instance, who
   3.376 +can't yet talk, communicate, and the things they'll do, like going to
   3.377 +the mother and turning the face to point in the direction where the
   3.378 +child wants it to look and so on; that's an extreme version of action
   3.379 +indicating intention.
   3.380 +</p>
   3.381 +<p>
   3.382 +[8:03] Anyway. That's a very long roundabout answer to one conjecture
   3.383 +that the use of communicative language is what gave humans their
   3.384 +unique power to create and destroy and whatever, and I'm saying that
   3.385 +if by that you mean <i>communicative</i> language, then I'm saying there
   3.386 +was something before that which was <i>non</i>-communicative language, and I
   3.387 +suspect that noncommunicative language continues to play a deep role
   3.388 +in <i>all</i> human perception &mdash;in mathematical and scientific reasoning, in
   3.389 +problem solving &mdash; and we don't understand very much about it.
   3.390 +</p>
   3.391 +<p>
   3.392 +[8:48]
   3.393 +I'm sure there's a lot more to be said about the development of
   3.394 +different kinds of senses, the development of brain structures and
   3.395 +mechanisms is above all that, but perhaps I've droned on long enough
   3.396 +on that question.
   3.397 +</p>
   3.398 +
   3.399 +</div>
   3.400 +</div>
   3.401 +
   3.402 +</div>
   3.403 +
   3.404 +<div id="outline-container-3" class="outline-2">
   3.405 +<h2 id="sec-3"><span class="section-number-2">3</span> How do language and internal states relate to AI?</h2>
   3.406 +<div class="outline-text-2" id="text-3">
   3.407 +
   3.408 +
   3.409 +<p>
   3.410 +[9:09] Well, I think most of the human and animal capabilities that
   3.411 +I've been referring to are not yet to be found in current robots or
   3.412 +[computing] systems, and I think there are two reasons for that: one
   3.413 +is that it's intrinsically very difficult; I think that in particular
   3.414 +it may turn out that the forms of information processing that one can
   3.415 +implement on digital computers as we currently know them may not be as
   3.416 +well suited to performing some of these tasks as other kinds of
   3.417 +computing about which we don't know so much &mdash; for example, I think
   3.418 +there may be important special features about <i>chemical</i> computers
   3.419 +which we might [talk about in a little bit? find out about]. 
   3.420 +</p>
   3.421 +
   3.422 +</div>
   3.423 +
   3.424 +<div id="outline-container-3-1" class="outline-3">
   3.425 +<h3 id="sec-3-1"><span class="section-number-3">3.1</span> In AI, false assumptions can lead investigators astray.</h3>
   3.426 +<div class="outline-text-3" id="text-3-1">
   3.427 +
   3.428 +<p>[9:57] So, one of the problems then is that the tasks are hard &hellip; but
   3.429 +there's a deeper problem as to why AI hasn't made a great deal of
   3.430 +progress on these problems that I'm talking about, and that is that
   3.431 +most AI researchers assume things&mdash;and this is not just AI
   3.432 +researchers, but [also] philsophers, and psychologists, and people
   3.433 +studying animal behavior&mdash;make assumptions about what it is that
   3.434 +animals or humans do, for instance make assumptions about what vision
   3.435 +is for, or assumptions about what motivation is and how motivation
   3.436 +works, or assumptions about how learning works, and then they try ---
   3.437 +the AI people try &mdash; to model [or] build systems that perform those
   3.438 +assumed functions. So if you get the <i>functions</i> wrong, then even if
   3.439 +you implement some of the functions that you're trying to implement,
   3.440 +they won't necessarily perform the tasks that the initial objective
   3.441 +was to imitate, for instance the tasks that humans, and nest-building
   3.442 +birds, and monkeys and so on can perform. 
   3.443 +</p>
   3.444 +</div>
   3.445 +
   3.446 +</div>
   3.447 +
   3.448 +<div id="outline-container-3-2" class="outline-3">
   3.449 +<h3 id="sec-3-2"><span class="section-number-3">3.2</span> Example: Vision is not just about finding surfaces, but about finding affordances.</h3>
   3.450 +<div class="outline-text-3" id="text-3-2">
   3.451 +
   3.452 +<p>[11:09] I'll give you a simple example &mdash; well, maybe not so simple,
   3.453 +but &mdash; It's often assumed that the function of vision in humans (and
   3.454 +in other animals with good eyesight and so on) is to take in optical
   3.455 +information that hits the retina, and form into the (maybe changing
   3.456 +&mdash; or, really, in our case definitely changing) patterns of
   3.457 +illumination where there are sensory receptors that detect those
   3.458 +patterns, and then somehow from that information (plus maybe other
   3.459 +information gained from head movement or from comparisons between two
   3.460 +eyes) to work out what there was in the environment that produced
   3.461 +those patterns, and that is often taken to mean &ldquo;where were the
   3.462 +surfaces off which the light bounced before it came to me&rdquo;. So
   3.463 +you essentially think of the task of the visual system as being to
   3.464 +reverse the image formation process: so the 3D structure's there, the
   3.465 +lens causes the image to form in the retina, and then the brain goes
   3.466 +back to a model of that 3D structure there. That's a very plausible
   3.467 +theory about vision, and it may be that that's a <i>subset</i> of what
   3.468 +human vision does, but I think James Gibson pointed out that that kind
   3.469 +of thing is not necessarily going to be very useful for an organism,
   3.470 +and it's very unlikely that that's the main function of perception in
   3.471 +general, namely to produce some physical description of what's out
   3.472 +there.
   3.473 +</p>
   3.474 +<p>
   3.475 +[12:37] What does an animal <i>need</i>? It needs to know what it can do,
   3.476 +what it can't do, what the consequences of its actions will be
   3.477 +&hellip;. so, he introduced the word <i>affordance</i>, so from his point of
   3.478 +view, the function of vision, perception, are to inform the organism
   3.479 +of what the <i>affordances</i> are for action, where that would mean what
   3.480 +the animal, <i>given</i> its morphology (what it can do with its mouth, its
   3.481 +limbs, and so on, and the ways it can move) what it can do, what its
   3.482 +needs are, what the obstacles are, and how the environment supports or
   3.483 +obstructs those possible actions.
   3.484 +</p>
   3.485 +<p>
   3.486 +[13:15] And that's a very different collection of information
   3.487 +structures that you need from, say, &ldquo;where are all the
   3.488 +surfaces?&rdquo;: if you've got all the surfaces, <i>deriving</i> the
   3.489 +affordances would still be a major task. So, if you think of the
   3.490 +perceptual system as primarily (for biological organisms) being
   3.491 +devices that provide information about affordances and so on, then the
   3.492 +tasks look very different. And most of the people working, doing
   3.493 +research on computer vision in robots, I think haven't taken all that
   3.494 +on board, so they're trying to get machines to do things which, even
   3.495 +if they were successful, would not make the robots very intelligent
   3.496 +(and in fact, even the ones they're trying to do are not really easy
   3.497 +to do, and they don't succeed very well&mdash; although, there's progress;
   3.498 +I shouldn't disparage it too much.)
   3.499 +</p>
   3.500 +</div>
   3.501 +
   3.502 +</div>
   3.503 +
   3.504 +<div id="outline-container-3-3" class="outline-3">
   3.505 +<h3 id="sec-3-3"><span class="section-number-3">3.3</span> Online and offline intelligence</h3>
   3.506 +<div class="outline-text-3" id="text-3-3">
   3.507 +
   3.508 +
   3.509 +<p>
   3.510 +[14:10] It gets more complex as animals get more sophisticated. So, I
   3.511 +like to make a distinction between online intelligence and offline
   3.512 +intelligence. So, for example, if I want to pick something up &mdash; like
   3.513 +this leaf &lt;he plucks a leaf from the table&gt; &mdash; I was able to select
   3.514 +it from all the others in there, and while moving my hand towards it,
   3.515 +I was able to guide its trajectory, making sure it was going roughly
   3.516 +in the right direction &mdash; as opposed to going out there, which
   3.517 +wouldn't have been able to pick it up &mdash; and these two fingers ended
   3.518 +up with a portion of the leaf between them, so that I was able to tell
   3.519 +when I'm ready to do that &lt;he clamps the leaf between two fingers&gt;
   3.520 +and at that point, I clamped my fingers and then I could pick up the
   3.521 +leaf. 
   3.522 +</p>
   3.523 +<p>
   3.524 +[14:54] Whereas, &mdash; and that's an example of online intelligence:
   3.525 +during the performance of an action (both from the stage where it's
   3.526 +initiated, and during the intermediate stages, and where it's
   3.527 +completed) I'm taking in information relevant to controlling all those
   3.528 +stages, and that relevant information keeps changing. That means I
   3.529 +need stores of transient information which gets discarded almost
   3.530 +immediately and replaced or something. That's online intelligence. And
   3.531 +there are many forms; that's just one example, and Gibson discussed
   3.532 +quite a lot of examples which I won't try to replicate now.
   3.533 +</p>
   3.534 +<p>
   3.535 +[15:30] But in offline intelligence, you're not necessarily actually
   3.536 +<i>performing</i> the actions when you're using your intelligence; you're
   3.537 +thinking about <i>possible</i> actions. So, for instance, I could think
   3.538 +about how fast or by what route I would get back to the lecture room
   3.539 +if I wanted to [get to the next talk] or something. And I know where
   3.540 +the door is, roughly speaking, and I know roughly which route I would
   3.541 +take, when I go out, I should go to the left or to the right, because
   3.542 +I've stored information about where the spaces are, where the
   3.543 +buildings are, where the door was that we came out &mdash; but in using
   3.544 +that information to think about that route, I'm not actually
   3.545 +performing the action. I'm not even <i>simulating</i> it in detail: the
   3.546 +precise details of direction and speed and when to clamp my fingers,
   3.547 +or when to contract my leg muscles when walking, are all irrelevant to
   3.548 +thinking about a good route, or thinking about the potential things
   3.549 +that might happen on the way. Or what would be a good place to meet
   3.550 +someone who I think [for an acquaintance in particular] &mdash; [barber]
   3.551 +or something &mdash; I don't necessarily have to work out exactly <i>where</i>
   3.552 +the person's going to stand, or from what angle I would recognize
   3.553 +them, and so on.
   3.554 +</p>
   3.555 +<p>
   3.556 +[16:46] So, offline intelligence &mdash; which I think became not just a
   3.557 +human competence; I think there are other animals that have aspects of
   3.558 +it: Squirrels are very impressive as you watch them. Gray squirrels at
   3.559 +any rate, as you watch them defeating squirrel-proof birdfeeders, seem
   3.560 +to have a lot of that [offline intelligence], as well as the online
   3.561 +intelligence when they eventually perform the action they've worked
   3.562 +out [] that will get them to the nuts. 
   3.563 +</p>
   3.564 +<p>
   3.565 +[17:16] And I think that what happened during our evolution is that
   3.566 +mechanisms for acquiring and processing and storing and manipulating
   3.567 +information that is more and more remote from the performance of
   3.568 +actions developed. An example is taking in information about where
   3.569 +locations are that you might need to go to infrequently: There's a
   3.570 +store of a particular type of material that's good for building on
   3.571 +roofs of houses or something out around there in some
   3.572 +direction. There's a good place to get water somewhere in another
   3.573 +direction. There are people that you'd like to go and visit in
   3.574 +another place, and so on. 
   3.575 +</p>
   3.576 +<p>
   3.577 +[17:59] So taking in information about an extended environment and
   3.578 +building it into a structure that you can make use of for different
   3.579 +purposes is another example of offline intelligence. And when we do
   3.580 +that, we sometimes use only our brains, but in modern times, we also
   3.581 +learned how to make maps on paper and walls and so on. And it's not
   3.582 +clear whether the stuff inside our heads has the same structures as
   3.583 +the maps we make on paper: the maps on paper have a different
   3.584 +function; they may be used to communicate with others, or meant for
   3.585 +<i>looking</i> at, whereas the stuff in your head you don't <i>look</i> at; you
   3.586 +use it in some other way.
   3.587 +</p>
   3.588 +<p>
   3.589 +[18:46] So, what I'm getting at is that there's a great deal of human
   3.590 +intelligence (and animal intelligence) which is involved in what's
   3.591 +possible in the future, what exists in distant places, what might have
   3.592 +happened in the past (sometimes you need to know why something is as
   3.593 +it is, because that might be relevant to what you should or shouldn't
   3.594 +do in the future, and so on), and I think there was something about
   3.595 +human evolution that extended that offline intelligence way beyond
   3.596 +that of animals. And I don't think it was <i>just</i> human language, (but
   3.597 +human language had something to do with it) but I think there was
   3.598 +something else that came earlier than language which involves the
   3.599 +ability to use your offline intelligence to discover something that
   3.600 +has a rich mathematical structure. 
   3.601 +</p>
   3.602 +</div>
   3.603 +
   3.604 +</div>
   3.605 +
   3.606 +<div id="outline-container-3-4" class="outline-3">
   3.607 +<h3 id="sec-3-4"><a name="example-gap" id="example-gap"></a><span class="section-number-3">3.4</span> Example: Even toddlers use sophisticated geometric knowledge</h3>
   3.608 +<div class="outline-text-3" id="text-3-4">
   3.609 +
   3.610 +<p>[19:44] I'll give you a simple example: if you look through a gap, you
   3.611 +can see something that's on the other side of the gap. Now, you
   3.612 +<i>might</i> see what you want to see, or you might see only part of it. If
   3.613 +you want to see more of it, which way would you move? Well, you could
   3.614 +either move <i>sideways</i>, and see through the gap&mdash;and see it roughly
   3.615 +the same amount but a different part of it [if it's a ????], or you
   3.616 +could move <i>towards</i> the gap and then your view will widen as you
   3.617 +approach the gap. Now, there's a bit of mathematics in there, insofar
   3.618 +as you are implicitly assuming that information travels in straight
   3.619 +lines, and as you go closer to a gap, the straight lines that you can
   3.620 +draw from where you are through the gap, widen as you approach that
   3.621 +gap. Now, there's a kind of theorem of Euclidean geometry in there
   3.622 +which I'm not going to try to state very precisely (and as far as I
   3.623 +know, wasn't stated explicitly in Euclidean geometry) but it's
   3.624 +something every toddler&mdash; human toddler&mdash;learns. (Maybe other
   3.625 +animals also know it, I don't know.) But there are many more things,
   3.626 +actions to perform, to get you more information about things, actions
   3.627 +to perform to conceal information from other people, actions that will
   3.628 +enable you to operate, to act on a rigid object in one place in order
   3.629 +to produce an effect on another place. So, there's a lot of stuff that
   3.630 +involves lines and rotations and angles and speeds and so on that I
   3.631 +think humans (maybe, to a lesser extent, other animals) develop the
   3.632 +ability to think about in a generic way. That means that you could
   3.633 +take out the generalizations from the particular contexts and then
   3.634 +re-use them in a new contexts in ways that I think are not yet
   3.635 +represented at all in AI and in theories of human learning in any []
   3.636 +way &mdash; although some people are trying to study learning of mathematics.
   3.637 +</p>
   3.638 +</div>
   3.639 +</div>
   3.640 +
   3.641 +</div>
   3.642 +
   3.643 +<div id="outline-container-4" class="outline-2">
   3.644 +<h2 id="sec-4"><span class="section-number-2">4</span> Animal intelligence</h2>
   3.645 +<div class="outline-text-2" id="text-4">
   3.646 +
   3.647 +
   3.648 +
   3.649 +</div>
   3.650 +
   3.651 +<div id="outline-container-4-1" class="outline-3">
   3.652 +<h3 id="sec-4-1"><span class="section-number-3">4.1</span> The priority is <i>cataloguing</i> what competences have evolved, not ranking them.</h3>
   3.653 +<div class="outline-text-3" id="text-4-1">
   3.654 +
   3.655 +<p>[22:03] I wasn't going to challenge the claim that humans can do more
   3.656 +sophisticated forms of [tracking], just to mention that there are some
   3.657 +things that other animals can do which are in some ways comparable,
   3.658 +and some ways superior to [things] that humans can do. In particular,
   3.659 +there are species of birds and also, I think, some rodents ---
   3.660 +squirrels, or something &mdash; I don't know enough about the variety ---
   3.661 +that can hide nuts and remember where they've hidden them, and go back
   3.662 +to them. And there have been tests which show that some birds are able
   3.663 +to hide tens &mdash; you know, [eighteen] or something nuts &mdash; and to
   3.664 +remember which ones have been taken, which ones haven't, and so
   3.665 +on. And I suspect most humans can't do that. I wouldn't want to say
   3.666 +categorically that maybe we couldn't, because humans are very
   3.667 +[varied], and also [a few] people can develop particular competences
   3.668 +through training. But it's certainly not something I can do.
   3.669 +</p>
   3.670 +
   3.671 +</div>
   3.672 +
   3.673 +</div>
   3.674 +
   3.675 +<div id="outline-container-4-2" class="outline-3">
   3.676 +<h3 id="sec-4-2"><span class="section-number-3">4.2</span> AI can be used to test philosophical theories</h3>
   3.677 +<div class="outline-text-3" id="text-4-2">
   3.678 +
   3.679 +<p>[23:01] But I also would like to say that I am not myself particularly
   3.680 +interested in trying to align animal intelligences according to any
   3.681 +kind of scale of superiority; I'm just trying to understand what it
   3.682 +was that biological evolution produced, and how it works, and I'm
   3.683 +interested in AI <i>mainly</i> because I think that when one comes up with
   3.684 +theories about how these things work, one needs to have some way of
   3.685 +testing the theory. And AI provides ways of implementing and testing
   3.686 +theories that were not previously available: Immanuel Kant was trying
   3.687 +to come up with theories about how minds work, but he didn't have any
   3.688 +kind of a mechanism that he could build to test his theory about the
   3.689 +nature of mathematical knowledge, for instance, or how concepts were
   3.690 +developed from babyhood onward. Whereas now, if we do develop a
   3.691 +theory, we have a criterion of adequacy, namely it should be precise
   3.692 +enough and rich enough and detailed to enable a model to be
   3.693 +built. And then we can see if it works. 
   3.694 +</p>
   3.695 +<p>
   3.696 +[24:07] If it works, it doesn't mean we've proved that the theory is
   3.697 +correct; it just shows it's a candidate. And if it doesn't work, then
   3.698 +it's not a candidate as it stands; it would need to be modified in
   3.699 +some way.
   3.700 +</p>
   3.701 +</div>
   3.702 +</div>
   3.703 +
   3.704 +</div>
   3.705 +
   3.706 +<div id="outline-container-5" class="outline-2">
   3.707 +<h2 id="sec-5"><span class="section-number-2">5</span> Is abstract general intelligence feasible?</h2>
   3.708 +<div class="outline-text-2" id="text-5">
   3.709 +
   3.710 +
   3.711 +
   3.712 +</div>
   3.713 +
   3.714 +<div id="outline-container-5-1" class="outline-3">
   3.715 +<h3 id="sec-5-1"><span class="section-number-3">5.1</span> It's misleading to compare the brain and its neurons to a computer made of transistors</h3>
   3.716 +<div class="outline-text-3" id="text-5-1">
   3.717 +
   3.718 +<p>[24:27] I think there's a lot of optimism based on false clues:
   3.719 +the&hellip;for example, one of the false clues is to count the number of
   3.720 +neurons in the brain, and then talk about the number of transistors
   3.721 +you can fit into a computer or something, and then compare them. It
   3.722 +might turn out that the study of the way synapses work (which leads
   3.723 +some people to say that a typical synapse [] in the human brain has
   3.724 +computational power comparable to the Internet a few years ago,
   3.725 +because of the number of different molecules that are doing things,
   3.726 +the variety of types of things that are being done in those molecular
   3.727 +interactions, and the speed at which they happen, if you somehow count
   3.728 +up the number of operations per second or something, then you get
   3.729 +these comparable figures).
   3.730 +</p>
   3.731 +</div>
   3.732 +
   3.733 +</div>
   3.734 +
   3.735 +<div id="outline-container-5-2" class="outline-3">
   3.736 +<h3 id="sec-5-2"><span class="section-number-3">5.2</span> For example, brains may rely heavily on chemical information processing</h3>
   3.737 +<div class="outline-text-3" id="text-5-2">
   3.738 +
   3.739 +<p>Now even if the details aren't right, there may just be a lot of
   3.740 +information processing that&hellip;going on in brains at the <i>molecular</i>
   3.741 +level, not the neural level. Then, if that's the case, the processing
   3.742 +units will be orders of magnitude larger in number than the number of
   3.743 +neurons. And it's certainly the case that all the original biological
   3.744 +forms of information processing were chemical; there weren't brains
   3.745 +around, and still aren't in most microbes. And even when humans grow
   3.746 +their brains, the process of starting from a fertilized egg and
   3.747 +producing this rich and complex structure is, for much of the time,
   3.748 +under the control of chemical computations, chemical information
   3.749 +processing&mdash;of course combined with physical sorts of materials and
   3.750 +energy and so on as well.
   3.751 +</p>
   3.752 +<p>
   3.753 +[26:25] So it would seem very strange if all that capability was
   3.754 +something thrown away when you've got a brain and all the information
   3.755 +processing, the [challenges that were handled in making a brain],
   3.756 +&hellip; This is handwaving on my part; I'm just saying that we <i>might</i>
   3.757 +learn that what brains do is not what we think they do, and that
   3.758 +problems of replicating them are not what we think they are, solely in
   3.759 +terms of numerical estimate of time scales, the number of components,
   3.760 +and so on.
   3.761 +</p>
   3.762 +</div>
   3.763 +
   3.764 +</div>
   3.765 +
   3.766 +<div id="outline-container-5-3" class="outline-3">
   3.767 +<h3 id="sec-5-3"><span class="section-number-3">5.3</span> Brain algorithms may simply be optimized for certain kinds of information processing other than bit manipulations</h3>
   3.768 +<div class="outline-text-3" id="text-5-3">
   3.769 +
   3.770 +<p>[26:56] But apart from that, the other basis of skepticism concerns
   3.771 +how well we understand what the problems are. I think there are many
   3.772 +people who try to formalize the problems of designing an intelligent
   3.773 +system in terms of streams of information thought of as bit streams or
   3.774 +collections of bit streams, and they think of as the problems of
   3.775 +intelligence as being the construction or detection of patterns in
   3.776 +those, and perhaps not just detection of patterns, but detection of
   3.777 +patterns that are useable for sending <i>out</i> streams to control motors
   3.778 +and so on in order to []. And that way of conceptualizing the problem
   3.779 +may lead on the one hand to oversimplification, so that the things
   3.780 +that <i>would</i> be achieved, if those goals were achieved, maybe much
   3.781 +simpler, in some ways inadequate. Or the replication of human
   3.782 +intelligence, or the matching of human intelligence&mdash;or for that
   3.783 +matter, squirrel intelligence&mdash;but in another way, it may also make
   3.784 +the problem harder: it may be that some of the kinds of things that
   3.785 +biological evolution has achieved can't be done that way. And one of
   3.786 +the ways that might turn out to be the case is not because it's not
   3.787 +impossible in principle to do some of the information processing on
   3.788 +artificial computers-based-on-transistors and other bit-manipulating
   3.789 +[]&mdash;but it may just be that the computational complexity of solving
   3.790 +problems, processes, or finding solutions to complex problems, are
   3.791 +much greater and therefore you might need a much larger universe than
   3.792 +we have available in order to do things.
   3.793 +</p>
   3.794 +</div>
   3.795 +
   3.796 +</div>
   3.797 +
   3.798 +<div id="outline-container-5-4" class="outline-3">
   3.799 +<h3 id="sec-5-4"><span class="section-number-3">5.4</span> Example: find the shortest path by dangling strings</h3>
   3.800 +<div class="outline-text-3" id="text-5-4">
   3.801 +
   3.802 +<p>[28:55] Then if the underlying mechanisms were different, the
   3.803 +information processing mechanisms, they might be better tailored to
   3.804 +particular sorts of computation. There's a [] example, which is
   3.805 +finding the shortest route if you've got a collection of roads, and
   3.806 +they may be curved roads, and lots of tangled routes from A to B to C,
   3.807 +and so on. And if you start at A and you want to get to Z &mdash; a place
   3.808 +somewhere on that map &mdash; the process of finding the shortest route
   3.809 +will involve searching through all these different possibilities and
   3.810 +rejecting some that are longer than others and so on. But if you make
   3.811 +a model of that map out of string, where these strings are all laid
   3.812 +out on the maps and so have the lengths of the routes. Then if you
   3.813 +hold the two knots in the string &ndash; it's a network of string &mdash; which
   3.814 +correspond to the start point and end point, then <i>pull</i>, then the
   3.815 +bits of string that you're left with in a straight line will give you
   3.816 +the shortest route, and that process of pulling just gets you the
   3.817 +solution very rapidly in a parallel computation, where all the others
   3.818 +just hang by the wayside, so to speak.
   3.819 +</p>
   3.820 +</div>
   3.821 +
   3.822 +</div>
   3.823 +
   3.824 +<div id="outline-container-5-5" class="outline-3">
   3.825 +<h3 id="sec-5-5"><span class="section-number-3">5.5</span> In sum, we know surprisingly little about the kinds of problems that evolution solved, and the manner in which they were solved.</h3>
   3.826 +<div class="outline-text-3" id="text-5-5">
   3.827 +
   3.828 +<p>[30:15] Now, I'm not saying brains can build networks of string and
   3.829 +pull them or anything like that; that's just an illustration of how if
   3.830 +you have the right representation, correctly implemented&mdash;or suitably
   3.831 +implemented&mdash;for a problem, then you can avoid very combinatorially
   3.832 +complex searches, which will maybe grow exponentially with the number
   3.833 +of components in your map, whereas with this thing, the time it takes
   3.834 +won't depend on how many strings you've [got on the map]; you just
   3.835 +pull, and it will depend only on the shortest route that exists in
   3.836 +there. Even if that shortest route wasn't obvious on the original map.
   3.837 +</p>
   3.838 +
   3.839 +<p>
   3.840 +[30:59] So that's a rather long-winded way of formulating the
   3.841 +conjecture which&mdash;of supporting, a roundabout way of supporting the
   3.842 +conjecture that there may be something about the way molecules perform
   3.843 +computations where they have the combination of continuous change as
   3.844 +things move through space and come together and move apart, and
   3.845 +whatever &mdash; and also snap into states that then persist, so [as you
   3.846 +learn from] quantum mechanics, you can have stable molecular
   3.847 +structures which are quite hard to separate, and then in catalytic
   3.848 +processes you can separate them, or extreme temperatures, or strong
   3.849 +forces, but they may nevertheless be able to move very rapidly in some
   3.850 +conditions in order to perform computations.
   3.851 +</p>
   3.852 +<p>
   3.853 +[31:49] Now there may be things about that kind of structure that
   3.854 +enable searching for solutions to <i>certain</i> classes of problems to be
   3.855 +done much more efficiently (by brain) than anything we could do with
   3.856 +computers. It's just an open question.
   3.857 +</p>
   3.858 +<p>
   3.859 +[32:04] So it <i>might</i> turn out that we need new kinds of technology
   3.860 +that aren't on the horizon in order to replicate the functions that
   3.861 +animal brains perform &mdash;or, it might not. I just don't know. I'm not
   3.862 +claiming that there's strong evidence for that; I'm just saying that
   3.863 +it might turn out that way, partly because I think we know less than
   3.864 +many people think we know about what biological evolution achieved.
   3.865 +</p>
   3.866 +<p>
   3.867 +[32:28] There are some other possibilities: we may just find out that
   3.868 +there are shortcuts no one ever thought of, and it will all happen
   3.869 +much more quickly&mdash;I have an open mind; I'd be surprised, but it
   3.870 +could turn up. There <i>is</i> something that worries me much more than the
   3.871 +singularity that most people talk about, which is machines achieving
   3.872 +human-level intelligence and perhaps taking over [the] planet or
   3.873 +something. There's what I call the <i>singularity of cognitive catch-up</i> &hellip;
   3.874 +</p>
   3.875 +</div>
   3.876 +</div>
   3.877 +
   3.878 +</div>
   3.879 +
   3.880 +<div id="outline-container-6" class="outline-2">
   3.881 +<h2 id="sec-6"><span class="section-number-2">6</span> A singularity of cognitive catch-up</h2>
   3.882 +<div class="outline-text-2" id="text-6">
   3.883 +
   3.884 +
   3.885 +
   3.886 +</div>
   3.887 +
   3.888 +<div id="outline-container-6-1" class="outline-3">
   3.889 +<h3 id="sec-6-1"><span class="section-number-3">6.1</span> What if it will take a lifetime to learn enough to make something new?</h3>
   3.890 +<div class="outline-text-3" id="text-6-1">
   3.891 +
   3.892 +<p>&hellip; SCC, singularity of cognitive catch-up, which I think we're close
   3.893 +to, or maybe have already reached&mdash;I'll explain what I mean by
   3.894 +that. One of the products of biological evolution&mdash;and this is one of
   3.895 +the answers to your earlier questions which I didn't get on to&mdash;is
   3.896 +that humans have not only the ability to make discoveries that none of
   3.897 +their ancestors have ever made, but to shorten the time required for
   3.898 +similar achievements to be reached by their offspring and their
   3.899 +descendants. So once we, for instance, worked out ways of complex
   3.900 +computations, or ways of building houses, or ways of finding our way
   3.901 +around, we don't need&hellip;our children don't need to work it out for
   3.902 +themselves by the same lengthy trial and error procedure; we can help
   3.903 +them get there much faster.
   3.904 +</p>
   3.905 +<p>
   3.906 +Okay, well, what I've been referring to as the singularity of
   3.907 +cognitive catch-up depends on the fact that&mdash;fairly obvious, and it's
   3.908 +often been commented on&mdash;that in case of humans, it's not necessary
   3.909 +for each generation to learn what previous generations learned <i>in the same way</i>. And we can speed up learning once something has been
   3.910 +learned, [it is able to] be learned by new people. And that has meant
   3.911 +that the social processes that support that kind of education of the
   3.912 +young can enormously accelerate what would have taken&hellip;perhaps
   3.913 +thousands [or] millions of years for evolution to produce, can happen in
   3.914 +a much shorter time. 
   3.915 +</p>
   3.916 +
   3.917 +<p>
   3.918 +[34:54] But here's the catch: in order for a new advance to happen ---
   3.919 +so for something new to be discovered that wasn't there before, like
   3.920 +Newtonian mechanics, or the theory of relativity, or Beethoven's music
   3.921 +or [style] or whatever &mdash; the individuals have to have traversed a
   3.922 +significant amount of what their ancestors have learned, even if they
   3.923 +do it much faster than their ancestors, to get to the point where they
   3.924 +can see the gaps, the possibilities for going further than their
   3.925 +ancestors, or their parents or whatever, have done.
   3.926 +</p>
   3.927 +<p>
   3.928 +[35:27] Now in the case of knowledge of science, mathematics,
   3.929 +philosophy, engineering and so on, there's been a lot of accumulated
   3.930 +knowledge. And humans are living a <i>bit</i> longer than they used to, but
   3.931 +they're still living for [whatever it is], a hundred years, or for
   3.932 +most people, less than that. So you can imagine that there might come
   3.933 +a time when in a normal human lifespan, it's not possible for anyone
   3.934 +to learn enough to understand the scope and limits of what's already
   3.935 +been achieved in order to see the potential for going beyond it and to
   3.936 +build on what's already been done to make that&hellip;those future steps.
   3.937 +</p>
   3.938 +<p>
   3.939 +[36:10] So if we reach that stage, we will have reached the
   3.940 +singularity of cognitive catch-up because the process of education
   3.941 +that enables individuals to learn faster than their ancestors did is
   3.942 +the catching-up process, and it may just be that we at some point
   3.943 +reach a point where catching up can only happen within a lifetime of
   3.944 +an individual, and after that they're dead and they can't go
   3.945 +beyond. And I have some evidence that there's a lot of that around
   3.946 +because I see a lot of people coming up with what <i>they</i> think of as
   3.947 +new ideas which they've struggled to come up with, but actually they
   3.948 +just haven't taken in some of what was&hellip;some of what was done [] by
   3.949 +other people, in other places before them. And I think that despite
   3.950 +the availability of search engines which make it <i>easier</i> for people
   3.951 +to get the information&mdash;for instance, when I was a student, if I
   3.952 +wanted to find out what other people had done in the field, it was a
   3.953 +laborious process&mdash;going to the library, getting books, and
   3.954 +&mdash;whereas now, I can often do things in seconds that would have taken
   3.955 +hours. So that means that if seconds [are needed] for that kind of
   3.956 +work, my lifespan has been extended by a factor of ten or
   3.957 +something. So maybe that <i>delays</i> the singularity, but it may not
   3.958 +delay it enough. But that's an open question; I don't know. And it may
   3.959 +just be that in some areas, this is more of a problem than others. For
   3.960 +instance, it may be that in some kinds of engineering, we're handing
   3.961 +over more and more of the work to machines anyways and they can go on
   3.962 +doing it.  So for instance, most of the production of computers now is
   3.963 +done by a computer-controlled machine&mdash;although some of the design
   3.964 +work is done by humans&mdash; a lot of <i>detail</i> of the design is done by
   3.965 +computers, and they produce the next generation, which then produces
   3.966 +the next generation, and so on.
   3.967 +</p>
   3.968 +<p>
   3.969 +[37:57] I don't know if humans can go on having major advances, so
   3.970 +it'll be kind of sad if we can't.
   3.971 +</p>
   3.972 +</div>
   3.973 +</div>
   3.974 +
   3.975 +</div>
   3.976 +
   3.977 +<div id="outline-container-7" class="outline-2">
   3.978 +<h2 id="sec-7"><span class="section-number-2">7</span> Spatial reasoning: a difficult problem</h2>
   3.979 +<div class="outline-text-2" id="text-7">
   3.980 +
   3.981 +
   3.982 +<p>
   3.983 +[38:15] Okay, well, there are different problems [ ] mathematics, and
   3.984 +they have to do with properties. So for instance a lot of mathematics
   3.985 +that can be expressed in terms of logical structures or algebraic
   3.986 +structures and those are pretty well suited for manipulation and&hellip;on
   3.987 +computers, and if a problem can be specified using the
   3.988 +logical/algebraic notation, and the solution method requires creating
   3.989 +something in that sort of notation, then computers are pretty good,
   3.990 +and there are lots of mathematical tools around&mdash;there are theorem
   3.991 +provers and theorem checkers, and all kinds of things, which couldn't
   3.992 +have existed fifty, sixty years ago, and they will continue getting
   3.993 +better.
   3.994 +</p>
   3.995 +
   3.996 +<p>
   3.997 +But there was something that I was <a href="#sec-3-4">alluding to earlier</a> when I gave the
   3.998 +example of how you can reason about what you will see by changing your
   3.999 +position in relation to a door, where what you are doing is using your
  3.1000 +grasp of spatial structures and how as one spatial relationship
  3.1001 +changes namely you come closer to the door or move sideways and
  3.1002 +parallel to the wall or whatever, other spatial relationships change
  3.1003 +in parallel, so the lines from your eyes through to other parts of
  3.1004 +the&hellip;parts of the room on the other side of the doorway change,
  3.1005 +spread out more as you go towards  the doorway, and as you move
  3.1006 +sideways, they don't spread out differently, but focus on different
  3.1007 +parts of the internal &hellip; that they access different parts of the
  3.1008 +&hellip; of the room.
  3.1009 +</p>
  3.1010 +<p>
  3.1011 +Now, those are examples of ways of thinking about relationships and
  3.1012 +changing relationships which are not the same as thinking about what
  3.1013 +happens if I replace this symbol with that symbol, or if I substitute
  3.1014 +this expression in that expression in a logical formula.  And at the
  3.1015 +moment, I do not believe that there is anything in AI amongst the
  3.1016 +mathematical reasoning community, the theorem-proving community, that
  3.1017 +can model the processes that go on when a young child starts learning
  3.1018 +to do Euclidean geometry and is taught things about&mdash;for instance, I
  3.1019 +can give you a proof that the angles of any triangle add up to a
  3.1020 +straight line, 180 degrees. 
  3.1021 +</p>
  3.1022 +
  3.1023 +</div>
  3.1024 +
  3.1025 +<div id="outline-container-7-1" class="outline-3">
  3.1026 +<h3 id="sec-7-1"><span class="section-number-3">7.1</span> Example: Spatial proof that the angles of any triangle add up to a half-circle</h3>
  3.1027 +<div class="outline-text-3" id="text-7-1">
  3.1028 +
  3.1029 +<p>There are standard proofs which involves starting with one triangle,
  3.1030 +then adding a line parallel to the base one of my former students,
  3.1031 +Mary Pardoe, came up with which I will demonstrate with this &lt;he holds
  3.1032 +up a pen&gt; &mdash; can you see it? If I have a triangle here that's got
  3.1033 +three sides, if I put this thing on it, on one side &mdash; let's say the
  3.1034 +bottom&mdash;I can rotate it until it lies along the second&hellip;another
  3.1035 +side, and then maybe move it up to the other end ~. Then I can rotate
  3.1036 +it again, until it lies on the third side, and move it back to the
  3.1037 +other end. And then I'll rotate it again and it'll eventually end up
  3.1038 +on the original side, but it will have changed the direction it's
  3.1039 +pointing in &mdash; and it won't have crossed over itself so it will have
  3.1040 +gone through a half-circle, and that says that the three angles of a
  3.1041 +triangle add up to the rotations of half a circle, which is a
  3.1042 +beautiful kind of proof and almost anyone can understand it. Some
  3.1043 +mathematicians don't like it, because they say it hides some of the
  3.1044 +assumptions, but nevertheless, as far as I'm concerned, it's an
  3.1045 +example of a human ability to do reasoning which, once you've
  3.1046 +understood it, you can see will apply to any triangle &mdash; it's got to
  3.1047 +be a planar triangle &mdash; not a triangle on a globe, because then the
  3.1048 +angles can add up to more than &hellip; you can have three <i>right</i> angles
  3.1049 +if you have an equator&hellip;a line on the equator, and a line going up to
  3.1050 +to the north pole of the earth, and then you have a right angle and
  3.1051 +then another line going down to the equator, and you have a right
  3.1052 +angle, right angle, right angle, and they add up to more than a
  3.1053 +straight line. But that's because the triangle isn't in the plane,
  3.1054 +it's on a curved surface. In fact, that's one of the
  3.1055 +differences&hellip;definitional differences you can take between planar and
  3.1056 +curved surfaces: how much the angles of a triangle add up to. But our
  3.1057 +ability to <i>visualize</i> and notice the generality in that process, and
  3.1058 +see that you're going to be able to do the same thing using triangles
  3.1059 +that stretch in all sorts of ways, or if it's a million times as
  3.1060 +large, or if it's made&hellip;you know, written on, on&hellip;if it's drawn in
  3.1061 +different colors or whatever &mdash; none of that's going to make any
  3.1062 +difference to the essence of that process. And that ability to see
  3.1063 +the commonality in a spatial structure which enables you to draw some
  3.1064 +conclusions with complete certainty&mdash;subject to the possibility that
  3.1065 +sometimes you make mistakes, but when you make mistakes, you can
  3.1066 +discover them, as has happened in the history of geometrical theorem
  3.1067 +proving. Imre Lakatos had a wonderful book called <a href="http://en.wikipedia.org/wiki/Proofs_and_Refutations"><i>Proofs and Refutations</i></a> &mdash; which I won't try to summarize &mdash; but he has
  3.1068 +examples: mistakes were made; that was because people didn't always
  3.1069 +realize there were subtle subcases which had slightly different
  3.1070 +properties, and they didn't take account of that. But once they're
  3.1071 +noticed, you rectify that. 
  3.1072 +</p>
  3.1073 +</div>
  3.1074 +
  3.1075 +</div>
  3.1076 +
  3.1077 +<div id="outline-container-7-2" class="outline-3">
  3.1078 +<h3 id="sec-7-2"><span class="section-number-3">7.2</span> Geometric results are fundamentally different than experimental results in chemistry or physics.</h3>
  3.1079 +<div class="outline-text-3" id="text-7-2">
  3.1080 +
  3.1081 +<p>[43:28] But it's not the same as doing experiments in chemistry and
  3.1082 +physics, where you can't be sure it'll be the same on [] or at a high
  3.1083 +temperature, or in a very strong magnetic field &mdash; with geometric
  3.1084 +reasoning, in some sense you've got the full information in front of
  3.1085 +you; even if you don't always notice an important part of it. So, that
  3.1086 +kind of reasoning (as far as I know) is not implemented anywhere in a
  3.1087 +computer. And most people who do research on trying to model
  3.1088 +mathematical reasoning, don't pay any attention to that, because of
  3.1089 +&hellip; they just don't think about it. They start from somewhere else,
  3.1090 +maybe because of how they were educated. I was taught Euclidean
  3.1091 +geometry at school. Were you?
  3.1092 +</p>
  3.1093 +<p>
  3.1094 +(Adam ford: Yeah)
  3.1095 +</p>
  3.1096 +<p>
  3.1097 +Many people are not now. Instead they're taught set theory, and
  3.1098 +logic, and arithmetic, and [algebra], and so on. And so they don't use
  3.1099 +that bit of their brains, without which we wouldn't have built any of
  3.1100 +the cathedrals, and all sorts of things we now depend on.
  3.1101 +</p>
  3.1102 +</div>
  3.1103 +</div>
  3.1104 +
  3.1105 +</div>
  3.1106 +
  3.1107 +<div id="outline-container-8" class="outline-2">
  3.1108 +<h2 id="sec-8"><span class="section-number-2">8</span> Is near-term artificial general intelligence likely?</h2>
  3.1109 +<div class="outline-text-2" id="text-8">
  3.1110 +
  3.1111 +
  3.1112 +
  3.1113 +</div>
  3.1114 +
  3.1115 +<div id="outline-container-8-1" class="outline-3">
  3.1116 +<h3 id="sec-8-1"><span class="section-number-3">8.1</span> Two interpretations: a single mechanism for all problems, or many mechanisms unified in one program.</h3>
  3.1117 +<div class="outline-text-3" id="text-8-1">
  3.1118 +
  3.1119 +
  3.1120 +<p>
  3.1121 +[44:35] Well, this relates to what's meant by general. And when I
  3.1122 +first encountered the AGI community, I thought that what they all
  3.1123 +meant by general intelligence was <i>uniform</i> intelligence ---
  3.1124 +intelligence based on some common simple (maybe not so simple, but)
  3.1125 +single powerful mechanism or principle of inference. And there are
  3.1126 +some people in the community who are trying to produce things like
  3.1127 +that, often in connection with algorithmic information theory and
  3.1128 +computability of information, and so on. But there's another sense of
  3.1129 +general which means that the system of general intelligence can do
  3.1130 +lots of different things, like perceive things, understand language,
  3.1131 +move around, make things, and so on &mdash; perhaps even enjoy a joke;
  3.1132 +that's something that's not nearly on the horizon, as far as I
  3.1133 +know. Enjoying a joke isn't the same as being able to make laughing
  3.1134 +noises. 
  3.1135 +</p>
  3.1136 +<p>
  3.1137 +Given, then, that there are these two notions of general
  3.1138 +intelligence&mdash;there's one that looks for one uniform, possibly
  3.1139 +simple, mechanism or collection of ideas and notations and algorithms,
  3.1140 +that will deal with any problem that's solvable &mdash; and the other
  3.1141 +that's general in the sense that it can do lots of different things
  3.1142 +that are combined into an integrated architecture (which raises lots
  3.1143 +of questions about how you combine these things and make them work
  3.1144 +together) and we humans, certainly, are of the second kind: we do all
  3.1145 +sorts of different things, and other animals also seem to be of the
  3.1146 +second kind, perhaps not as general as humans. Now, it may turn out
  3.1147 +that in some near future time, who knows&mdash;decades, a few
  3.1148 +decades&mdash;you'll be able to get machines that are capable of solving
  3.1149 +in a time that will depend on the nature of the problem, but any
  3.1150 +problem that is solvable, and they will be able to do it in some sort
  3.1151 +of tractable time &mdash; of course, there are some problems that are
  3.1152 +solvable that would require a larger universe and a longer history
  3.1153 +than the history of the universe, but apart from that constraint,
  3.1154 +these machines will be able to do anything [].  But to be able to do
  3.1155 +some of the kinds of things that humans can do, like the kinds of
  3.1156 +geometrical reasoning where you look at the shape and you abstract
  3.1157 +away from the precise angles and sizes and shapes and so on, and
  3.1158 +realize there's something general here, as must have happened when our
  3.1159 +ancestors first made the discoveries that eventually put together in
  3.1160 +Euclidean geometry. 
  3.1161 +</p>
  3.1162 +<p>
  3.1163 +It may be that that requires mechanisms of a kind that we don't know
  3.1164 +anything about at the moment. Maybe brains are using molecules and
  3.1165 +rearranging molecules in some way that supports that kind of
  3.1166 +reasoning. I'm not saying they are &mdash; I don't know, I just don't see
  3.1167 +any simple&hellip;any obvious way to map that kind of reasoning capability
  3.1168 +onto what we currently do on computers. There is&mdash;and I just
  3.1169 +mentioned this briefly beforehand&mdash;there is a kind of thing that's
  3.1170 +sometimes thought of as a major step in that direction, namely you can
  3.1171 +build a machine (or a software system) that can represent some
  3.1172 +geometrical structure, and then be told about some change that's going
  3.1173 +to happen to it, and it can predict in great detail what'll
  3.1174 +happen. And this happens for instance in game engines, where you say
  3.1175 +we have all these blocks on the table and I'll drop one other block,
  3.1176 +and then [the thing] uses Newton's laws and properties of rigidity of
  3.1177 +the parts and the elasticity and also stuff about geometries and space
  3.1178 +and so on, to give you a very accurate representation of what'll
  3.1179 +happen when this brick lands on this pile of things, [it'll bounce and
  3.1180 +go off, and so on]. And you just, with more memory and more CPU power,
  3.1181 +you can increase the accuracy&mdash; but that's totally different than
  3.1182 +looking at <i>one</i> example, and working out what will happen in a whole
  3.1183 +<i>range</i> of cases at a higher level of abstraction, whereas the game
  3.1184 +engine does it in great detail for <i>just</i> this case, with <i>just</i> those
  3.1185 +precise things, and it won't even know what the generalizations are
  3.1186 +that it's using that would apply to others []. So, in that sense, [we]
  3.1187 +may get AGI &mdash; artificial general intelligence &mdash; pretty soon, but
  3.1188 +it'll be limited in what it can do. And the other kind of general
  3.1189 +intelligence which combines all sorts of different things, including
  3.1190 +human spatial geometrical reasoning, and maybe other things, like the
  3.1191 +ability to find things funny, and to appreciate artistic features and
  3.1192 +other things may need forms of pattern-mechanism, and I have an open
  3.1193 +mind about that.
  3.1194 +</p>
  3.1195 +</div>
  3.1196 +</div>
  3.1197 +
  3.1198 +</div>
  3.1199 +
  3.1200 +<div id="outline-container-9" class="outline-2">
  3.1201 +<h2 id="sec-9"><span class="section-number-2">9</span> Abstract General Intelligence impacts</h2>
  3.1202 +<div class="outline-text-2" id="text-9">
  3.1203 +
  3.1204 +
  3.1205 +<p>
  3.1206 +[49:53] Well, as far as the first type's concerned, it could be useful
  3.1207 +for all kinds of applications &mdash; there are people who worry about
  3.1208 +where there's a system that has that type of intelligence, might in
  3.1209 +some sense take over control of the planet. Well, humans often do
  3.1210 +stupid things, and they might do something stupid that would lead to
  3.1211 +disaster, but I think it's more likely that there would be other
  3.1212 +things [] lead to disaster&mdash; population problems, using up all the
  3.1213 +resources, destroying ecosystems, and whatever. But certainly it would
  3.1214 +go on being useful to have these calculating devices. Now, as for the
  3.1215 +second kind of them, I don't know&mdash;if we succeeded at putting
  3.1216 +together all the parts that we find in humans, we might just make an
  3.1217 +artificial human, and then we might have some of them as your friends,
  3.1218 +and some of them we might not like, and some of them might become
  3.1219 +teachers or whatever, composers &mdash; but that raises a question: could
  3.1220 +they, in some sense, be superior to us, in their learning
  3.1221 +capabilities, their understanding of human nature, or maybe their
  3.1222 +wickedness or whatever &mdash; these are all issues in which I expect the
  3.1223 +best science fiction writers would give better answers than anything I
  3.1224 +could do, but I did once fantasize when I [back] in 1978, that perhaps
  3.1225 +if we achieved that kind of thing, that they would be wise, and gentle
  3.1226 +and kind, and realize that humans are an inferior species that, you
  3.1227 +know, have some good features, so they'd keep us in some kind of
  3.1228 +secluded&hellip;restrictive kind of environment, keep us away from
  3.1229 +dangerous weapons, and so on. And find ways of cohabitating with
  3.1230 +us. But that's just fantasy.
  3.1231 +</p>
  3.1232 +<p>
  3.1233 +Adam Ford: Awesome. Yeah, there's an interesting story <i>With Folded Hands</i> where [the computers] want to take care of us and want to
  3.1234 +reduce suffering and end up lobotomizing everybody [but] keeping them
  3.1235 +alive so as to reduce the suffering. 
  3.1236 +</p>
  3.1237 +<p>
  3.1238 +Aaron Sloman: Not all that different from <i>Brave New World</i>, where it
  3.1239 +was done with drugs and so on, but different humans are given
  3.1240 +different roles in that system, yeah.
  3.1241 +</p>
  3.1242 +<p>
  3.1243 +There's also <i>The Time Machine</i>, H.G. Wells, where the &hellip; in the
  3.1244 +distant future, humans have split in two: the Eloi, I think they were
  3.1245 +called, they lived underground, they were the [] ones, and then&mdash;no,
  3.1246 +the Morlocks lived underground; Eloi lived on the planet; they were
  3.1247 +pleasant and pretty but not very bright, and so on, and they were fed
  3.1248 +on by &hellip;
  3.1249 +</p>
  3.1250 +<p>
  3.1251 +Adam Ford: [] in the future.
  3.1252 +</p>
  3.1253 +<p>
  3.1254 +Aaron Sloman: As I was saying, if you ask science fiction writers,
  3.1255 +you'll probably come up with a wide variety of interesting answers. 
  3.1256 +</p>
  3.1257 +<p>
  3.1258 +Adam Ford: I certainly have; I've spoken to [] of Birmingham, and
  3.1259 +Sean Williams, &hellip; who else? 
  3.1260 +</p>
  3.1261 +<p>
  3.1262 +Aaron Sloman: Did you ever read a story by E.M. Forrester called <i>The Machine Stops</i> &mdash; very short story, it's <a href="http://archive.ncsa.illinois.edu/prajlich/forster.html">on the Internet somewhere</a>
  3.1263 +&mdash; it's about a time when people sitting &hellip; and this was written in
  3.1264 +about [1914 ] so it's about&hellip;over a hundred years ago &hellip; people are
  3.1265 +in their rooms, they sit in front of screens, and they type things,
  3.1266 +and they communicate with one another that way, and they don't meet;
  3.1267 +they have debates, and they give lectures to their audiences that way,
  3.1268 +and then there's a woman whose son says &ldquo;I'd like to see
  3.1269 +you&rdquo; and she says &ldquo;What's the point? You've got me at
  3.1270 +this point &rdquo; but he wants to come and talk to her &mdash; I won't
  3.1271 +tell you how it ends, but.
  3.1272 +</p>
  3.1273 +<p>
  3.1274 +Adam Ford: Reminds me of the Internet.
  3.1275 +</p>
  3.1276 +<p>
  3.1277 +Aaron Sloman: Well, yes; he invented &hellip; it was just extraordinary
  3.1278 +that he was able to do that, before most of the components that we
  3.1279 +need for it existed.
  3.1280 +</p>
  3.1281 +<p>
  3.1282 +Adam Ford: [Another person who did that] was Vernor Vinge [] <i>True Names</i>. 
  3.1283 +</p>
  3.1284 +<p>
  3.1285 +Aaron Sloman: When was that written?
  3.1286 +</p>
  3.1287 +<p>
  3.1288 +Adam Ford: The seventies.
  3.1289 +</p>
  3.1290 +<p>
  3.1291 +Aaron Sloman: Okay, well a lot of the technology was already around
  3.1292 +then. The original bits of internet were working, in about 1973, I was
  3.1293 +sitting &hellip; 1974, I was sitting at Sussex University trying to
  3.1294 +use&hellip;learn LOGO, the programming language, to decide whether it was
  3.1295 +going to be useful for teaching AI, and I was sitting [] paper
  3.1296 +teletype, there was paper coming out, transmitting ten characters a
  3.1297 +second from Sussex to UCL computer lab by telegraph cable, from there
  3.1298 +to somewhere in Norway via another cable, from there by satellite to
  3.1299 +California to a computer Xerox [] research center where they had
  3.1300 +implemented a computer with a LOGO system on it, with someone I had
  3.1301 +met previously in Edinburgh, Danny Bobrow, and he allowed me to have
  3.1302 +access to this sytem. So there I was typing. And furthermore, it was
  3.1303 +duplex typing, so every character I typed didn't show up on my
  3.1304 +terminal until it had gone all the way there and echoed back, so I
  3.1305 +would type, and the characters would come back four seconds later.
  3.1306 +</p>
  3.1307 +<p>
  3.1308 +[55:26] But that was the Internet, and I think Vernor Vinge was
  3.1309 +writing after that kind of thing had already started, but I don't
  3.1310 +know. Anyway.
  3.1311 +</p>
  3.1312 +<p>
  3.1313 +[55:41] Another&hellip;I mentioned H.G. Wells, <i>The Time Machine</i>. I
  3.1314 +recently discovered, because <a href="http://en.wikipedia.org/wiki/David_Lodge_(author)">David Lodge</a> had written a sort of
  3.1315 +semi-novel about him, that he had invented Wikipedia, in advance&mdash; he
  3.1316 +had this notion of an encyclopedia that was free to everybody, and
  3.1317 +everybody could contribute and [collaborate on it]. So, go to the
  3.1318 +science fiction writers to find out the future &mdash; well, a range of
  3.1319 +possible futures.
  3.1320 +</p>
  3.1321 +<p>
  3.1322 +Adam Ford: Well the thing is with science fiction writers, they have
  3.1323 +to maintain some sort of interest for their readers, after all the
  3.1324 +science fiction which reaches us is the stuff that publishers want to
  3.1325 +sell, and so there's a little bit of a &hellip; a bias towards making a
  3.1326 +plot device there, and so the dramatic sort of appeals to our
  3.1327 +amygdala, our lizard brain; we'll sort of stay there obviously to some
  3.1328 +extent. But I think that they do come up with sort of amazing ideas; I
  3.1329 +think it's worth trying to make these predictions; I think that we
  3.1330 +should more time on strategic forecasting, I mean take that seriously.
  3.1331 +</p>
  3.1332 +<p>
  3.1333 +Aaron Sloman: Well, I'm happy to leave that to others; I just want to
  3.1334 +try to understand these problems that bother me about how things
  3.1335 +work. And it may be that some would say that's irresponsible if I
  3.1336 +don't think about what the implications will be. Well, understanding
  3.1337 +how humans work <i>might</i> enable us to make [] humans &mdash; I suspect it
  3.1338 +wont happen in this century; I think it's going to be too difficult.
  3.1339 +</p></div>
  3.1340 +</div>
  3.1341 +</div>
  3.1342 +
  3.1343 +<div id="postamble">
  3.1344 +<p class="date">Date: 2013-10-04 18:49:53 UTC</p>
  3.1345 +<p class="author">Author: Dylan Holmes</p>
  3.1346 +<p class="creator">Org version 7.7 with Emacs version 23</p>
  3.1347 +<a href="http://validator.w3.org/check?uri=referer">Validate XHTML 1.0</a>
  3.1348 +
  3.1349 +</div>
  3.1350 +</body>
  3.1351 +</html>

     4.1 --- a/org/sloman.org	Tue Jun 03 13:23:23 2014 -0400
     4.2 +++ b/org/sloman.org	Tue Jun 03 13:24:58 2014 -0400
     4.3 @@ -4,22 +4,19 @@
     4.4  #+STYLE: <link rel="stylesheet" type="text/css" href="../css/sloman.css" /> 
     4.5  
     4.6  
     4.7 +#+begin_quote
     4.8 +*Update* (13 Oct): Aaron Sloman has produced an improved version of
     4.9 + this transcript, which includes follow-up thoughts and links to
    4.10 + related works. It is available on his website here:
    4.11 + [[http://www.cs.bham.ac.uk/research/projects/cogaff/movies/transcript-interview.html]].
    4.12 +
    4.13 +
    4.14 +This draft will remain available here for historical purposes.
    4.15 +#+end_quote
    4.16 +
    4.17  #+BEGIN_QUOTE
    4.18  
    4.19  
    4.20 -
    4.21 -
    4.22 -
    4.23 -
    4.24 -
    4.25 -
    4.26 -
    4.27 -
    4.28 -
    4.29 -
    4.30 -
    4.31 -
    4.32 -
    4.33  *Editor's note:* This is a working draft transcript which I made of
    4.34  [[http://www.youtube.com/watch?feature=player_detailpage&v=iuH8dC7Snno][this nice interview]] of Aaron Sloman. Having just finished one
    4.35  iteration of transcription, I still need to go in and clean up the
    4.36 @@ -30,6 +27,11 @@
    4.37  copies of this transcript for your own purposes. Also, feel free to
    4.38  e-mail me with comments or corrections.
    4.39  
    4.40 +(Addendum: This transcription is licensed by Aaron Sloman and Dylan Holmes, as
    4.41 + indicated here:
    4.42 + http://www.cs.bham.ac.uk/research/projects/cogaff/movies/transcript-interview.html#license)
    4.43 +
    4.44 +
    4.45  You can send mail to =transcript@aurellem.org=.
    4.46  
    4.47  Cheers,
    4.48 @@ -39,6 +41,7 @@
    4.49  
    4.50  
    4.51  
    4.52 +
    4.53  * Introduction
    4.54  
    4.55  ** Aaron Sloman evolves into a philosopher of AI