aurellem

The EARS!

Written by Robert McIntyre

Table of Contents

1 Ears!

I want to be able to place ears in a similiar manner to how I place the eyes. I want to be able to place ears in a unique spatial position, and recieve as output at every tick the FFT of whatever signals are happening at that point.

(ns body.ear)
(use 'cortex.world)
(use 'cortex.import)
(use 'clojure.contrib.def)
(cortex.import/mega-import-jme3)
(rlm.rlm-commands/help)
(import java.nio.ByteBuffer)
(import java.awt.image.BufferedImage)
(import java.awt.Color)
(import java.awt.Dimension)
(import java.awt.Graphics)
(import java.awt.Graphics2D)
(import java.awt.event.WindowAdapter)
(import java.awt.event.WindowEvent)
(import java.awt.image.BufferedImage)
(import java.nio.ByteBuffer)
(import javax.swing.JFrame)
(import javax.swing.JPanel)
(import javax.swing.SwingUtilities)
(import javax.swing.ImageIcon)
(import javax.swing.JOptionPane)
(import java.awt.image.ImageObserver)

JMonkeyEngine3's audio system works as follows: first, an appropiate audio renderer is created during initialization and depending on the context. On my computer, this is the LwjglAudioRenderer.

The LwjglAudioRenderer sets a few internal state variables depending on what capabilities the audio system has.

may very well need to make my own AudioRenderer

(in-ns 'body.ear)
(import 'com.jme3.capture.SoundProcessor)


(defn sound-processor
  "deals with converting ByteBuffers into Arrays of bytes so that the
  continuation functions can be defined in terms of immutable stuff."
  [continuation]
  (proxy [SoundProcessor] []
    (cleanup [])
    (process
      [#^ByteBuffer audioSamples numSamples]
      (no-exceptions
       (let [byte-array (byte-array numSamples)]
         (.get audioSamples byte-array 0 numSamples)
         (continuation
          (vec byte-array)))))))
         

(defn add-ear 
  "add an ear to the world.  The continuation function will be called
  on the FFT or the sounds which the ear hears in the given
  timeframe. Sound is 3D."
  [world listener continuation]
  (let [renderer (.getAudioRenderer world)]
    (.addListener renderer listener)
    (.registerSoundProcessor renderer listener
                             (sound-processor continuation))
    listener))

(ns test.hearing)
(use 'cortex.world)
(use 'cortex.import)
(use 'clojure.contrib.def)
(use 'body.ear)
(cortex.import/mega-import-jme3)
(rlm.rlm-commands/help)

(defn setup-fn [world]
  (let [listener (Listener.)]
    (add-ear world listener #(println (nth % 0)))))
  
(defn play-sound [node world value]
  (if (not value)
    (do
      (.playSource (.getAudioRenderer world) node))))

(defn test-world []
  (let [node1 (AudioNode. (asset-manager) "Sounds/pure.wav" false false)]
    (world
     (Node.)
     {"key-space" (partial play-sound node1)}
     setup-fn
     no-op
     )))
    

Date: 2011-09-21 16:38:27 MDT

Author: Robert McIntyre

Org version 7.6 with Emacs version 23

Validate XHTML 1.0