Mercurial > audio-send
changeset 1:c41d773a85fb
moved org files, ignored html files
| author | Robert McIntyre <rlm@mit.edu> |
|---|---|
| date | Tue, 25 Oct 2011 13:03:35 -0700 |
| parents | f9476ff7637e |
| children | 82b9267a3e19 |
| files | .hgignore bugs-in-openal.org device_details.org ear.html ear.org org/bugs-in-openal.org org/device_details.org org/ear.org org/todo.org todo.org |
| diffstat | 10 files changed, 278 insertions(+), 544 deletions(-) [+] |
line wrap: on
line diff
1.1 --- a/.hgignore Tue Oct 25 13:02:31 2011 -0700 1.2 +++ b/.hgignore Tue Oct 25 13:03:35 2011 -0700 1.3 @@ -1,3 +1,3 @@ 1.4 syntax: glob 1.5 build* 1.6 - 1.7 +html*
2.1 --- a/bugs-in-openal.org Tue Oct 25 13:02:31 2011 -0700 2.2 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 2.3 @@ -1,50 +0,0 @@ 2.4 -* bugs in OpenAl-soft 2.5 - 2.6 -Wave files produced by wave writer do not have the correct header. 2.7 - 2.8 -something is wrong with 2.9 - 2.10 -#+begin_src conf 2.11 -drivers = null, 2.12 -#+end_src 2.13 - 2.14 -It only produces the single null option when queried with 2.15 -ALC_ALL_DEVICES_SPECIFIER, where it should produce much more output. 2.16 - 2.17 -specifically, 2.18 - 2.19 ------------------------------- 2.20 -#+begin_src conf 2.21 -drivers = oss, 2.22 -#+end_src 2.23 - 2.24 -#+begin_src text 2.25 -0: OSS Software 2.26 -1: PulseAudio Software 2.27 -2: ALSA Software on default 2.28 -3: ALSA Software on HDA Intel [ALC272 Analog] (hw:0,0) 2.29 -4: ALSA Software on HDA Intel [ALC272 Digital] (hw:0,1) 2.30 -5: ALSA Software on HDA Intel [INTEL HDMI] (hw:0,3) 2.31 -6: PortAudio Software 2.32 -7: Wave File Writer 2.33 -#+end_src 2.34 ------------------------------- 2.35 -#+begin_src conf 2.36 -drivers = pulse, 2.37 -#+end_src 2.38 - 2.39 -#+begin_src text 2.40 -Devices 2.41 -0: Internal Audio Analog Stereo 2.42 -#+end_src 2.43 ------------------------------- 2.44 -#+begin_src conf 2.45 -drivers = wave, 2.46 -#+end_src 2.47 - 2.48 -#+begin_src text 2.49 -Devices 2.50 -0: Wave File Writer 2.51 -#+end_src 2.52 ------------------------------- 2.53 -
3.1 --- a/device_details.org Tue Oct 25 13:02:31 2011 -0700 3.2 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 3.3 @@ -1,37 +0,0 @@ 3.4 -* a device has a two names, 3.5 - - *name*, which is the filename and which appears before all of the functions. 3.6 - - *trueName*, which is a string that is used to summon the device. 3.7 - 3.8 - 3.9 -Every device has the *<name>_data* struct, which is where all the 3.10 -device specific state is located. 3.11 - 3.12 -In the function *<name>_open_playback*, the device is initialized, 3.13 -here the trueName is set, and the devices data is initialized. 3.14 - 3.15 -The ALCdevice's ExtraData field must always be cast to to the actual 3.16 -data struct that the device uses. 3.17 - 3.18 - 3.19 - 3.20 -the =open_playback= function is where everything is initialized, 3.21 - 3.22 - 3.23 -but the =reset_playback= is where the actual thread which is the heart 3.24 -of the device is started. 3.25 - 3.26 - 3.27 -every device has a <name>Proc function which takes a pointer to an 3.28 -ALCdevice and is run in a separate thread. It is this procedure that 3.29 -actually does stuff with sound data. 3.30 - 3.31 - 3.32 -It is the ALContext which controls the particulars of a listener. 3.33 - 3.34 -The context contains the device. 3.35 - 3.36 -* so do we need a new context for each listener, alone with their own 3.37 -devices? 3.38 - 3.39 -However, the device also maintains a *list* of contexts... 3.40 -
4.1 --- a/ear.html Tue Oct 25 13:02:31 2011 -0700 4.2 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 4.3 @@ -1,266 +0,0 @@ 4.4 -<?xml version="1.0" encoding="utf-8"?> 4.5 -<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 4.6 - "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> 4.7 -<html xmlns="http://www.w3.org/1999/xhtml" 4.8 -lang="en" xml:lang="en"> 4.9 -<head> 4.10 -<title>The EARS!</title> 4.11 -<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> 4.12 -<meta name="generator" content="Org-mode"/> 4.13 -<meta name="generated" content="2011-09-21 16:38:27 MDT"/> 4.14 -<meta name="author" content="Robert McIntyre"/> 4.15 -<meta name="description" content="Simulating multiple listeners and the sense of hearing in uMonkeyEngine3"/> 4.16 -<meta name="keywords" content=""/> 4.17 -<style type="text/css"> 4.18 - <!--/*--><![CDATA[/*><!--*/ 4.19 - html { font-family: Times, serif; font-size: 12pt; } 4.20 - .title { text-align: center; } 4.21 - .todo { color: red; } 4.22 - .done { color: green; } 4.23 - .tag { background-color: #add8e6; font-weight:normal } 4.24 - .target { } 4.25 - .timestamp { color: #bebebe; } 4.26 - .timestamp-kwd { color: #5f9ea0; } 4.27 - .right {margin-left:auto; margin-right:0px; text-align:right;} 4.28 - .left {margin-left:0px; margin-right:auto; text-align:left;} 4.29 - .center {margin-left:auto; margin-right:auto; text-align:center;} 4.30 - p.verse { margin-left: 3% } 4.31 - pre { 4.32 - border: 1pt solid #AEBDCC; 4.33 - background-color: #F3F5F7; 4.34 - padding: 5pt; 4.35 - font-family: courier, monospace; 4.36 - font-size: 90%; 4.37 - overflow:auto; 4.38 - } 4.39 - table { border-collapse: collapse; } 4.40 - td, th { vertical-align: top; } 4.41 - th.right { text-align:center; } 4.42 - th.left { text-align:center; } 4.43 - th.center { text-align:center; } 4.44 - td.right { text-align:right; } 4.45 - td.left { text-align:left; } 4.46 - td.center { text-align:center; } 4.47 - dt { font-weight: bold; } 4.48 - div.figure { padding: 0.5em; } 4.49 - div.figure p { text-align: center; } 4.50 - textarea { overflow-x: auto; } 4.51 - .linenr { font-size:smaller } 4.52 - .code-highlighted {background-color:#ffff00;} 4.53 - .org-info-js_info-navigation { border-style:none; } 4.54 - #org-info-js_console-label { font-size:10px; font-weight:bold; 4.55 - white-space:nowrap; } 4.56 - .org-info-js_search-highlight {background-color:#ffff00; color:#000000; 4.57 - font-weight:bold; } 4.58 - /*]]>*/--> 4.59 -</style> 4.60 -<link rel="stylesheet" type="text/css" href="../aurellem/src/css/aurellem.css"/> 4.61 -<script type="text/javascript"> 4.62 -<!--/*--><![CDATA[/*><!--*/ 4.63 - function CodeHighlightOn(elem, id) 4.64 - { 4.65 - var target = document.getElementById(id); 4.66 - if(null != target) { 4.67 - elem.cacheClassElem = elem.className; 4.68 - elem.cacheClassTarget = target.className; 4.69 - target.className = "code-highlighted"; 4.70 - elem.className = "code-highlighted"; 4.71 - } 4.72 - } 4.73 - function CodeHighlightOff(elem, id) 4.74 - { 4.75 - var target = document.getElementById(id); 4.76 - if(elem.cacheClassElem) 4.77 - elem.className = elem.cacheClassElem; 4.78 - if(elem.cacheClassTarget) 4.79 - target.className = elem.cacheClassTarget; 4.80 - } 4.81 -/*]]>*///--> 4.82 -</script> 4.83 - 4.84 -</head> 4.85 -<body> 4.86 - 4.87 -<div id="content"> 4.88 - 4.89 - 4.90 - 4.91 -<div class="header"> 4.92 - <div class="float-right"> 4.93 - <!-- 4.94 - <form> 4.95 - <input type="text"/><input type="submit" value="search the blog »"/> 4.96 - </form> 4.97 - --> 4.98 - </div> 4.99 - 4.100 - <h1>aurellem <em>☉</em></h1> 4.101 - <ul class="nav"> 4.102 - <li><a href="/">read the blog »</a></li> 4.103 - <!-- li><a href="#">learn about us »</a></li--> 4.104 - </ul> 4.105 -</div> 4.106 - 4.107 -<h1 class="title">The EARS!</h1> 4.108 -<div class="author">Written by <author>Robert McIntyre</author></div> 4.109 - 4.110 - 4.111 - 4.112 - 4.113 - 4.114 - 4.115 - 4.116 - 4.117 - 4.118 - 4.119 -<div id="table-of-contents"> 4.120 -<h2>Table of Contents</h2> 4.121 -<div id="text-table-of-contents"> 4.122 -<ul> 4.123 -<li><a href="#sec-1">1 Ears! </a></li> 4.124 -</ul> 4.125 -</div> 4.126 -</div> 4.127 - 4.128 -<div id="outline-container-1" class="outline-2"> 4.129 -<h2 id="sec-1"><span class="section-number-2">1</span> Ears! </h2> 4.130 -<div class="outline-text-2" id="text-1"> 4.131 - 4.132 - 4.133 -<p> 4.134 -I want to be able to place ears in a similiar manner to how I place 4.135 -the eyes. I want to be able to place ears in a unique spatial 4.136 -position, and recieve as output at every tick the FFT of whatever 4.137 -signals are happening at that point. 4.138 -</p> 4.139 - 4.140 - 4.141 - 4.142 -<pre class="src src-clojure">(<span style="color: #9b30ff;">ns</span> body.ear) 4.143 -(<span style="color: #da70d6;">use</span> 'cortex.world) 4.144 -(<span style="color: #da70d6;">use</span> 'cortex.import) 4.145 -(<span style="color: #da70d6;">use</span> 'clojure.contrib.def) 4.146 -(cortex.import/mega-import-jme3) 4.147 -(rlm.rlm-commands/help) 4.148 -(<span style="color: #9b30ff;">import</span> java.nio.ByteBuffer) 4.149 -(<span style="color: #9b30ff;">import</span> java.awt.image.BufferedImage) 4.150 -(<span style="color: #9b30ff;">import</span> java.awt.Color) 4.151 -(<span style="color: #9b30ff;">import</span> java.awt.Dimension) 4.152 -(<span style="color: #9b30ff;">import</span> java.awt.Graphics) 4.153 -(<span style="color: #9b30ff;">import</span> java.awt.Graphics2D) 4.154 -(<span style="color: #9b30ff;">import</span> java.awt.event.WindowAdapter) 4.155 -(<span style="color: #9b30ff;">import</span> java.awt.event.WindowEvent) 4.156 -(<span style="color: #9b30ff;">import</span> java.awt.image.BufferedImage) 4.157 -(<span style="color: #9b30ff;">import</span> java.nio.ByteBuffer) 4.158 -(<span style="color: #9b30ff;">import</span> javax.swing.JFrame) 4.159 -(<span style="color: #9b30ff;">import</span> javax.swing.JPanel) 4.160 -(<span style="color: #9b30ff;">import</span> javax.swing.SwingUtilities) 4.161 -(<span style="color: #9b30ff;">import</span> javax.swing.ImageIcon) 4.162 -(<span style="color: #9b30ff;">import</span> javax.swing.JOptionPane) 4.163 -(<span style="color: #9b30ff;">import</span> java.awt.image.ImageObserver) 4.164 -</pre> 4.165 - 4.166 - 4.167 - 4.168 - 4.169 -<p> 4.170 -JMonkeyEngine3's audio system works as follows: 4.171 -first, an appropiate audio renderer is created during initialization 4.172 -and depending on the context. On my computer, this is the 4.173 -LwjglAudioRenderer. 4.174 -</p> 4.175 -<p> 4.176 -The LwjglAudioRenderer sets a few internal state variables depending 4.177 -on what capabilities the audio system has. 4.178 -</p> 4.179 -<p> 4.180 -may very well need to make my own AudioRenderer 4.181 -</p> 4.182 - 4.183 - 4.184 - 4.185 -<pre class="src src-clojure">(<span style="color: #9b30ff;">in-ns</span> 'body.ear) 4.186 -(<span style="color: #9b30ff;">import</span> 'com.jme3.capture.SoundProcessor) 4.187 - 4.188 - 4.189 -(<span style="color: #9b30ff;">defn</span> <span style="color: #0000ff;">sound-processor</span> 4.190 - <span style="color: #EB4264;">"deals with converting ByteBuffers into Arrays of bytes so that the</span> 4.191 -<span style="color: #EB4264;"> continuation functions can be defined in terms of immutable stuff."</span> 4.192 - [continuation] 4.193 - (<span style="color: #da70d6;">proxy</span> [SoundProcessor] [] 4.194 - (cleanup []) 4.195 - (process 4.196 - [<span style="color: #228b22;">#^ByteBuffer</span> audioSamples numSamples] 4.197 - (no-exceptions 4.198 - (<span style="color: #9b30ff;">let</span> [byte-array (<span style="color: #da70d6;">byte-array</span> numSamples)] 4.199 - (.get audioSamples byte-array 0 numSamples) 4.200 - (continuation 4.201 - (<span style="color: #da70d6;">vec</span> byte-array))))))) 4.202 - 4.203 - 4.204 -(<span style="color: #9b30ff;">defn</span> <span style="color: #0000ff;">add-ear</span> 4.205 - <span style="color: #EB4264;">"add an ear to the world. The continuation function will be called</span> 4.206 -<span style="color: #EB4264;"> on the FFT or the sounds which the ear hears in the given</span> 4.207 -<span style="color: #EB4264;"> timeframe. Sound is 3D."</span> 4.208 - [world listener continuation] 4.209 - (<span style="color: #9b30ff;">let</span> [renderer (.getAudioRenderer world)] 4.210 - (.addListener renderer listener) 4.211 - (.registerSoundProcessor renderer listener 4.212 - (sound-processor continuation)) 4.213 - listener)) 4.214 - 4.215 -</pre> 4.216 - 4.217 - 4.218 - 4.219 - 4.220 - 4.221 - 4.222 - 4.223 -<pre class="src src-clojure">(<span style="color: #9b30ff;">ns</span> test.hearing) 4.224 -(<span style="color: #da70d6;">use</span> 'cortex.world) 4.225 -(<span style="color: #da70d6;">use</span> 'cortex.import) 4.226 -(<span style="color: #da70d6;">use</span> 'clojure.contrib.def) 4.227 -(<span style="color: #da70d6;">use</span> 'body.ear) 4.228 -(cortex.import/mega-import-jme3) 4.229 -(rlm.rlm-commands/help) 4.230 - 4.231 -(<span style="color: #9b30ff;">defn</span> <span style="color: #0000ff;">setup-fn</span> [world] 4.232 - (<span style="color: #9b30ff;">let</span> [listener (Listener.)] 4.233 - (add-ear world listener #(<span style="color: #da70d6;">println</span> (<span style="color: #da70d6;">nth</span> % 0))))) 4.234 - 4.235 -(<span style="color: #9b30ff;">defn</span> <span style="color: #0000ff;">play-sound</span> [node world value] 4.236 - (<span style="color: #9b30ff;">if</span> (<span style="color: #da70d6;">not</span> value) 4.237 - (<span style="color: #9b30ff;">do</span> 4.238 - (.playSource (.getAudioRenderer world) node)))) 4.239 - 4.240 -(<span style="color: #9b30ff;">defn</span> <span style="color: #0000ff;">test-world</span> [] 4.241 - (<span style="color: #9b30ff;">let</span> [node1 (AudioNode. (asset-manager) <span style="color: #3AA221;">"Sounds/pure.wav"</span> false false)] 4.242 - (world 4.243 - (Node.) 4.244 - {<span style="color: #3AA221;">"key-space"</span> (<span style="color: #da70d6;">partial</span> play-sound node1)} 4.245 - setup-fn 4.246 - no-op 4.247 - ))) 4.248 - 4.249 - 4.250 -</pre> 4.251 - 4.252 - 4.253 - 4.254 - 4.255 - 4.256 - 4.257 - 4.258 - 4.259 -</div> 4.260 -</div> 4.261 -<div id="postamble"> 4.262 -<p class="date">Date: 2011-09-21 16:38:27 MDT</p> 4.263 -<p class="author">Author: Robert McIntyre</p> 4.264 -<p class="creator">Org version 7.6 with Emacs version 23</p> 4.265 -<a href="http://validator.w3.org/check?uri=referer">Validate XHTML 1.0</a> 4.266 -</div> 4.267 -</div> 4.268 -</body> 4.269 -</html>
5.1 --- a/ear.org Tue Oct 25 13:02:31 2011 -0700 5.2 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 5.3 @@ -1,137 +0,0 @@ 5.4 -#+title: The EARS! 5.5 -#+author: Robert McIntyre 5.6 -#+email: rlm@mit.edu 5.7 -#+MATHJAX: align:"left" mathml:t path:"../aurellem/src/MathJax/MathJax.js" 5.8 -#+STYLE: <link rel="stylesheet" type="text/css" href="../aurellem/src/css/aurellem.css"/> 5.9 -#+BABEL: :exports both :noweb yes :cache no :mkdirp yes 5.10 -#+INCLUDE: ../aurellem/src/templates/level-0.org 5.11 -#+description: Simulating multiple listeners and the sense of hearing in uMonkeyEngine3 5.12 - 5.13 - 5.14 - 5.15 - 5.16 -* Ears! 5.17 - 5.18 -I want to be able to place ears in a similiar manner to how I place 5.19 -the eyes. I want to be able to place ears in a unique spatial 5.20 -position, and recieve as output at every tick the FFT of whatever 5.21 -signals are happening at that point. 5.22 - 5.23 -#+srcname: ear-header 5.24 -#+begin_src clojure 5.25 -(ns body.ear) 5.26 -(use 'cortex.world) 5.27 -(use 'cortex.import) 5.28 -(use 'clojure.contrib.def) 5.29 -(cortex.import/mega-import-jme3) 5.30 -(rlm.rlm-commands/help) 5.31 -(import java.nio.ByteBuffer) 5.32 -(import java.awt.image.BufferedImage) 5.33 -(import java.awt.Color) 5.34 -(import java.awt.Dimension) 5.35 -(import java.awt.Graphics) 5.36 -(import java.awt.Graphics2D) 5.37 -(import java.awt.event.WindowAdapter) 5.38 -(import java.awt.event.WindowEvent) 5.39 -(import java.awt.image.BufferedImage) 5.40 -(import java.nio.ByteBuffer) 5.41 -(import javax.swing.JFrame) 5.42 -(import javax.swing.JPanel) 5.43 -(import javax.swing.SwingUtilities) 5.44 -(import javax.swing.ImageIcon) 5.45 -(import javax.swing.JOptionPane) 5.46 -(import java.awt.image.ImageObserver) 5.47 -#+end_src 5.48 - 5.49 -JMonkeyEngine3's audio system works as follows: 5.50 -first, an appropiate audio renderer is created during initialization 5.51 -and depending on the context. On my computer, this is the 5.52 -LwjglAudioRenderer. 5.53 - 5.54 -The LwjglAudioRenderer sets a few internal state variables depending 5.55 -on what capabilities the audio system has. 5.56 - 5.57 -may very well need to make my own AudioRenderer 5.58 - 5.59 -#+srcname: ear-body-1 5.60 -#+begin_src clojure :results silent 5.61 -(in-ns 'body.ear) 5.62 -(import 'com.jme3.capture.SoundProcessor) 5.63 - 5.64 - 5.65 -(defn sound-processor 5.66 - "deals with converting ByteBuffers into Arrays of bytes so that the 5.67 - continuation functions can be defined in terms of immutable stuff." 5.68 - [continuation] 5.69 - (proxy [SoundProcessor] [] 5.70 - (cleanup []) 5.71 - (process 5.72 - [#^ByteBuffer audioSamples numSamples] 5.73 - (no-exceptions 5.74 - (let [byte-array (byte-array numSamples)] 5.75 - (.get audioSamples byte-array 0 numSamples) 5.76 - (continuation 5.77 - (vec byte-array))))))) 5.78 - 5.79 - 5.80 -(defn add-ear 5.81 - "add an ear to the world. The continuation function will be called 5.82 - on the FFT or the sounds which the ear hears in the given 5.83 - timeframe. Sound is 3D." 5.84 - [world listener continuation] 5.85 - (let [renderer (.getAudioRenderer world)] 5.86 - (.addListener renderer listener) 5.87 - (.registerSoundProcessor renderer listener 5.88 - (sound-processor continuation)) 5.89 - listener)) 5.90 - 5.91 -#+end_src 5.92 - 5.93 - 5.94 - 5.95 -#+srcname: test-hearing 5.96 -#+begin_src clojure :results silent 5.97 -(ns test.hearing) 5.98 -(use 'cortex.world) 5.99 -(use 'cortex.import) 5.100 -(use 'clojure.contrib.def) 5.101 -(use 'body.ear) 5.102 -(cortex.import/mega-import-jme3) 5.103 -(rlm.rlm-commands/help) 5.104 - 5.105 -(defn setup-fn [world] 5.106 - (let [listener (Listener.)] 5.107 - (add-ear world listener #(println (nth % 0))))) 5.108 - 5.109 -(defn play-sound [node world value] 5.110 - (if (not value) 5.111 - (do 5.112 - (.playSource (.getAudioRenderer world) node)))) 5.113 - 5.114 -(defn test-world [] 5.115 - (let [node1 (AudioNode. (asset-manager) "Sounds/pure.wav" false false)] 5.116 - (world 5.117 - (Node.) 5.118 - {"key-space" (partial play-sound node1)} 5.119 - setup-fn 5.120 - no-op 5.121 - ))) 5.122 - 5.123 - 5.124 -#+end_src 5.125 - 5.126 - 5.127 - 5.128 -* COMMENT Code Generation 5.129 - 5.130 -#+begin_src clojure :tangle /home/r/cortex/src/body/ear.clj 5.131 -<<ear-header>> 5.132 -<<ear-body-1>> 5.133 -#+end_src 5.134 - 5.135 - 5.136 -#+begin_src clojure :tangle /home/r/cortex/src/test/hearing.clj 5.137 -<<test-hearing>> 5.138 -#+end_src 5.139 - 5.140 -
6.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 6.2 +++ b/org/bugs-in-openal.org Tue Oct 25 13:03:35 2011 -0700 6.3 @@ -0,0 +1,50 @@ 6.4 +* bugs in OpenAl-soft 6.5 + 6.6 +Wave files produced by wave writer do not have the correct header. 6.7 + 6.8 +something is wrong with 6.9 + 6.10 +#+begin_src conf 6.11 +drivers = null, 6.12 +#+end_src 6.13 + 6.14 +It only produces the single null option when queried with 6.15 +ALC_ALL_DEVICES_SPECIFIER, where it should produce much more output. 6.16 + 6.17 +specifically, 6.18 + 6.19 +------------------------------ 6.20 +#+begin_src conf 6.21 +drivers = oss, 6.22 +#+end_src 6.23 + 6.24 +#+begin_src text 6.25 +0: OSS Software 6.26 +1: PulseAudio Software 6.27 +2: ALSA Software on default 6.28 +3: ALSA Software on HDA Intel [ALC272 Analog] (hw:0,0) 6.29 +4: ALSA Software on HDA Intel [ALC272 Digital] (hw:0,1) 6.30 +5: ALSA Software on HDA Intel [INTEL HDMI] (hw:0,3) 6.31 +6: PortAudio Software 6.32 +7: Wave File Writer 6.33 +#+end_src 6.34 +------------------------------ 6.35 +#+begin_src conf 6.36 +drivers = pulse, 6.37 +#+end_src 6.38 + 6.39 +#+begin_src text 6.40 +Devices 6.41 +0: Internal Audio Analog Stereo 6.42 +#+end_src 6.43 +------------------------------ 6.44 +#+begin_src conf 6.45 +drivers = wave, 6.46 +#+end_src 6.47 + 6.48 +#+begin_src text 6.49 +Devices 6.50 +0: Wave File Writer 6.51 +#+end_src 6.52 +------------------------------ 6.53 +
7.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 7.2 +++ b/org/device_details.org Tue Oct 25 13:03:35 2011 -0700 7.3 @@ -0,0 +1,37 @@ 7.4 +* a device has a two names, 7.5 + - *name*, which is the filename and which appears before all of the functions. 7.6 + - *trueName*, which is a string that is used to summon the device. 7.7 + 7.8 + 7.9 +Every device has the *<name>_data* struct, which is where all the 7.10 +device specific state is located. 7.11 + 7.12 +In the function *<name>_open_playback*, the device is initialized, 7.13 +here the trueName is set, and the devices data is initialized. 7.14 + 7.15 +The ALCdevice's ExtraData field must always be cast to to the actual 7.16 +data struct that the device uses. 7.17 + 7.18 + 7.19 + 7.20 +the =open_playback= function is where everything is initialized, 7.21 + 7.22 + 7.23 +but the =reset_playback= is where the actual thread which is the heart 7.24 +of the device is started. 7.25 + 7.26 + 7.27 +every device has a <name>Proc function which takes a pointer to an 7.28 +ALCdevice and is run in a separate thread. It is this procedure that 7.29 +actually does stuff with sound data. 7.30 + 7.31 + 7.32 +It is the ALContext which controls the particulars of a listener. 7.33 + 7.34 +The context contains the device. 7.35 + 7.36 +* so do we need a new context for each listener, alone with their own 7.37 +devices? 7.38 + 7.39 +However, the device also maintains a *list* of contexts... 7.40 +
8.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 8.2 +++ b/org/ear.org Tue Oct 25 13:03:35 2011 -0700 8.3 @@ -0,0 +1,137 @@ 8.4 +#+title: The EARS! 8.5 +#+author: Robert McIntyre 8.6 +#+email: rlm@mit.edu 8.7 +#+MATHJAX: align:"left" mathml:t path:"../aurellem/src/MathJax/MathJax.js" 8.8 +#+STYLE: <link rel="stylesheet" type="text/css" href="../aurellem/src/css/aurellem.css"/> 8.9 +#+BABEL: :exports both :noweb yes :cache no :mkdirp yes 8.10 +#+INCLUDE: ../aurellem/src/templates/level-0.org 8.11 +#+description: Simulating multiple listeners and the sense of hearing in uMonkeyEngine3 8.12 + 8.13 + 8.14 + 8.15 + 8.16 +* Ears! 8.17 + 8.18 +I want to be able to place ears in a similiar manner to how I place 8.19 +the eyes. I want to be able to place ears in a unique spatial 8.20 +position, and recieve as output at every tick the FFT of whatever 8.21 +signals are happening at that point. 8.22 + 8.23 +#+srcname: ear-header 8.24 +#+begin_src clojure 8.25 +(ns body.ear) 8.26 +(use 'cortex.world) 8.27 +(use 'cortex.import) 8.28 +(use 'clojure.contrib.def) 8.29 +(cortex.import/mega-import-jme3) 8.30 +(rlm.rlm-commands/help) 8.31 +(import java.nio.ByteBuffer) 8.32 +(import java.awt.image.BufferedImage) 8.33 +(import java.awt.Color) 8.34 +(import java.awt.Dimension) 8.35 +(import java.awt.Graphics) 8.36 +(import java.awt.Graphics2D) 8.37 +(import java.awt.event.WindowAdapter) 8.38 +(import java.awt.event.WindowEvent) 8.39 +(import java.awt.image.BufferedImage) 8.40 +(import java.nio.ByteBuffer) 8.41 +(import javax.swing.JFrame) 8.42 +(import javax.swing.JPanel) 8.43 +(import javax.swing.SwingUtilities) 8.44 +(import javax.swing.ImageIcon) 8.45 +(import javax.swing.JOptionPane) 8.46 +(import java.awt.image.ImageObserver) 8.47 +#+end_src 8.48 + 8.49 +JMonkeyEngine3's audio system works as follows: 8.50 +first, an appropiate audio renderer is created during initialization 8.51 +and depending on the context. On my computer, this is the 8.52 +LwjglAudioRenderer. 8.53 + 8.54 +The LwjglAudioRenderer sets a few internal state variables depending 8.55 +on what capabilities the audio system has. 8.56 + 8.57 +may very well need to make my own AudioRenderer 8.58 + 8.59 +#+srcname: ear-body-1 8.60 +#+begin_src clojure :results silent 8.61 +(in-ns 'body.ear) 8.62 +(import 'com.jme3.capture.SoundProcessor) 8.63 + 8.64 + 8.65 +(defn sound-processor 8.66 + "deals with converting ByteBuffers into Arrays of bytes so that the 8.67 + continuation functions can be defined in terms of immutable stuff." 8.68 + [continuation] 8.69 + (proxy [SoundProcessor] [] 8.70 + (cleanup []) 8.71 + (process 8.72 + [#^ByteBuffer audioSamples numSamples] 8.73 + (no-exceptions 8.74 + (let [byte-array (byte-array numSamples)] 8.75 + (.get audioSamples byte-array 0 numSamples) 8.76 + (continuation 8.77 + (vec byte-array))))))) 8.78 + 8.79 + 8.80 +(defn add-ear 8.81 + "add an ear to the world. The continuation function will be called 8.82 + on the FFT or the sounds which the ear hears in the given 8.83 + timeframe. Sound is 3D." 8.84 + [world listener continuation] 8.85 + (let [renderer (.getAudioRenderer world)] 8.86 + (.addListener renderer listener) 8.87 + (.registerSoundProcessor renderer listener 8.88 + (sound-processor continuation)) 8.89 + listener)) 8.90 + 8.91 +#+end_src 8.92 + 8.93 + 8.94 + 8.95 +#+srcname: test-hearing 8.96 +#+begin_src clojure :results silent 8.97 +(ns test.hearing) 8.98 +(use 'cortex.world) 8.99 +(use 'cortex.import) 8.100 +(use 'clojure.contrib.def) 8.101 +(use 'body.ear) 8.102 +(cortex.import/mega-import-jme3) 8.103 +(rlm.rlm-commands/help) 8.104 + 8.105 +(defn setup-fn [world] 8.106 + (let [listener (Listener.)] 8.107 + (add-ear world listener #(println (nth % 0))))) 8.108 + 8.109 +(defn play-sound [node world value] 8.110 + (if (not value) 8.111 + (do 8.112 + (.playSource (.getAudioRenderer world) node)))) 8.113 + 8.114 +(defn test-world [] 8.115 + (let [node1 (AudioNode. (asset-manager) "Sounds/pure.wav" false false)] 8.116 + (world 8.117 + (Node.) 8.118 + {"key-space" (partial play-sound node1)} 8.119 + setup-fn 8.120 + no-op 8.121 + ))) 8.122 + 8.123 + 8.124 +#+end_src 8.125 + 8.126 + 8.127 + 8.128 +* COMMENT Code Generation 8.129 + 8.130 +#+begin_src clojure :tangle /home/r/cortex/src/body/ear.clj 8.131 +<<ear-header>> 8.132 +<<ear-body-1>> 8.133 +#+end_src 8.134 + 8.135 + 8.136 +#+begin_src clojure :tangle /home/r/cortex/src/test/hearing.clj 8.137 +<<test-hearing>> 8.138 +#+end_src 8.139 + 8.140 +
9.1 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 9.2 +++ b/org/todo.org Tue Oct 25 13:03:35 2011 -0700 9.3 @@ -0,0 +1,53 @@ 9.4 +* Tasks to do relating to the Aurellem recorder device. 9.5 + 9.6 + 9.7 + 9.8 +** Cleanup 9.9 +- [ ] come up with a good name for the recording device. 9.10 +- [X] shift stepping functions to =nstep= 9.11 +- [ ] make a new project that Uses openal-soft but is not inside it 9.12 +- [ ] transfer all of my stuff to this project (everntually, we'll hava a "record" artifact) 9.13 +- [ ] put everything into an org file 9.14 +- [ ] make a post on aurellem 9.15 +- [ ] make a post on jMonkeyEngine forums 9.16 + 9.17 +** New Features 9.18 +- [ ] enable multiple listeners / multiple data 9.19 +- [ ] add support for streaming sources. 9.20 + 9.21 + 9.22 +** Java Integration 9.23 +- [ ] watch out for playSourceInstance and make sure it's handled correctly! 9.24 +- [ ] figure out how to compile for all platforms 9.25 +- [ ] get jMonkeyEngine to auto-load the new artifacts. 9.26 +- [X] properly extend the AudioRenderer instead of copying everything over. 9.27 +- [ ] combine with video recorder to make an easy-to-use thing for the main case. 9.28 + 9.29 +** Clojure Integration 9.30 +- [ ] abstract to closure =(ear)= function. 9.31 + 9.32 +** Testing 9.33 +- [X] make sure it works for effects (NOT GOING TO DO!), jMonkeyEngine doesn't do it well. 9.34 +- [ ] make an automated test using two listeners that are in the exact same area. 9.35 + 9.36 + 9.37 +** Conversion to multiple listeners. 9.38 +- [ ] make every function call manage context switching automatically 9.39 +- [ ] gather up all data necessary for context switching in a struct. 9.40 +- [ ] nmake a map/list of these structures in the ExtraData device field. 9.41 +- [ ] 9.42 + 9.43 + 9.44 + 9.45 + 9.46 + 9.47 + 9.48 + 9.49 + 9.50 + 9.51 + 9.52 + 9.53 + 9.54 + 9.55 + 9.56 +
10.1 --- a/todo.org Tue Oct 25 13:02:31 2011 -0700 10.2 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 10.3 @@ -1,53 +0,0 @@ 10.4 -* Tasks to do relating to the Aurellem recorder device. 10.5 - 10.6 - 10.7 - 10.8 -** Cleanup 10.9 -- [ ] come up with a good name for the recording device. 10.10 -- [X] shift stepping functions to =nstep= 10.11 -- [ ] make a new project that Uses openal-soft but is not inside it 10.12 -- [ ] transfer all of my stuff to this project (everntually, we'll hava a "record" artifact) 10.13 -- [ ] put everything into an org file 10.14 -- [ ] make a post on aurellem 10.15 -- [ ] make a post on jMonkeyEngine forums 10.16 - 10.17 -** New Features 10.18 -- [ ] enable multiple listeners / multiple data 10.19 -- [ ] add support for streaming sources. 10.20 - 10.21 - 10.22 -** Java Integration 10.23 -- [ ] watch out for playSourceInstance and make sure it's handled correctly! 10.24 -- [ ] figure out how to compile for all platforms 10.25 -- [ ] get jMonkeyEngine to auto-load the new artifacts. 10.26 -- [X] properly extend the AudioRenderer instead of copying everything over. 10.27 -- [ ] combine with video recorder to make an easy-to-use thing for the main case. 10.28 - 10.29 -** Clojure Integration 10.30 -- [ ] abstract to closure =(ear)= function. 10.31 - 10.32 -** Testing 10.33 -- [X] make sure it works for effects (NOT GOING TO DO!), jMonkeyEngine doesn't do it well. 10.34 -- [ ] make an automated test using two listeners that are in the exact same area. 10.35 - 10.36 - 10.37 -** Conversion to multiple listeners. 10.38 -- [ ] make every function call manage context switching automatically 10.39 -- [ ] gather up all data necessary for context switching in a struct. 10.40 -- [ ] nmake a map/list of these structures in the ExtraData device field. 10.41 -- [ ] 10.42 - 10.43 - 10.44 - 10.45 - 10.46 - 10.47 - 10.48 - 10.49 - 10.50 - 10.51 - 10.52 - 10.53 - 10.54 - 10.55 - 10.56 -