Detect microphone volume (blowing into microphone)

In my application I would like to detect if a user is blowing into the microphone. I'd like to achieve this by recording the sound with the microphone and check the volume, knowing that this is far from perfect as any noise could trigger it after all.

I read the documentation regarding sound recording and tuner and tried to use it for my purposes. I also have been studying the SimpleTuner example coded shipped with the Corona SDK. But it seems to me that the values proved by the tuner are just not usable to detect the volume correctly.

I have tested an implementation with Objective-C that uses AVAudioRecorder, which works well. You can find the example here:
http://mobileorchard.com/tutorial-detecting-when-a-user-blows-into-the-mic/

The AVAudioRecorder provides the method peakPowerForChannel, which can be used to detect the volume. It returns the measured power in decibels, ranging from -160 dB (lowest volume) to 0 dB (max volume).

Could someone from the Ansca team please explain, what number exactly is returned from the getTunerVolume() function? Is the function using the same method from AVAudioRecorder to detect the recorded volume on iOS devices?

The documentation for getTunerVolume says: Get the last calculated volume number, the mean squared value of the samples in the current audio buffer using sample values scaled to be in the range [-1..1]. To get a more meaningful range of values, convert this to a log scale using for example 10*math.log( volume ).

But I have no clue what that should mean exactly (especially the: mean squared value of the samples in the current audio buffer using sample values scaled to be in the range [-1..1]). The raw output value is changing between 0.4 and 0.6, but seems not really affected by any input on the microphone (in big contrast to the native Objective-C code I tested with).

Further it is stated that convert this to a log scale using for example 10*math.log( volume ). But in the SimpleTuner example code the factor 20 is used instead of 10 local v = 20*math.log(r:getTunerVolume()). Apart from that on most websites I found regarding this topic they use log10 and not log2.

Again, could please someone from the Ansca dev team shed some light on this topic and clarify the documentation regarding the getTunerVolume function? Or did any of the developers here have success using the tuner for such a purpose?

Thanks,
Thomas

I Would like to get some info on this subject as well..
Anyone?

Hi. I'm new to corona and am also wondering if a blow into the microphone is possible.

Here is some code to demonstrate a way to use Corona to detect sound. All it does is change the color of some text and is very simple-minded but hopefully this will help you get started.

It keeps a running average so it can adapt to different background noise levels
Every frame, it
- gets the sound level value
- converts it to dB
- compares each new value with the value from the last frame
- if the difference is a big enough jump, it's a sound onset
- if the difference is a big enough drop, it's a sound offset
- updates the running average.

-- Create an object to access audio input features
local r = media.newRecording()
r:startRecording()
r:startTuner()  
 
-- Create an object to hold display items
local g = display.newGroup()
 
-- Simple sound detector  
-- This just changes the color of a text label in response to a detected sound.
-- There are two threshold quantities here that could be used to adjust the sensitivity,
-- one is the amount of change that detects sound onset, the other is for sound offset.
-- You could connect these to a UI element to allow the user to adjust the sensitivity.
local soundDetector = display.newText( "............", 0, 0, nil, 22 )
soundDetector:setReferencePoint( display.TopLeftReferencePoint)
soundDetector:setTextColor( 255,255,255, 150 )
soundDetector.x = display.contentWidth/2 
soundDetector.y = 0.8*display.contentHeight 
local lastV = 0
local threshold = {}  
threshold.pos = 2.5   -- adjust these
threshold.neg = 1     -- to change sensitivity
 
function soundDetector:enterFrame( event )
    local v = r:getTunerVolume()
    if v == 0 then
        return
    end
    -- Convert RMS power to dB scale
    v = 20 * 0.301 * math.log(v)
    soundDetector.text = string.format("%4.2f", v - lastV )    
    if v > lastV + threshold.pos then
        -- Detected a level increase
        soundDetector:setTextColor( 255,255,100, 255 )
        lastV = v
    else
        if v < lastV - threshold.neg  then    
            -- Detected a level drop
            soundDetector:setTextColor( 255,255,255, 150 )            
            lastV = v
        else
            -- Adapt the background level
            -- Simple running average 
            lastV = 0.5 * v + 0.5 * lastV
        end
    end
end
Runtime:addEventListener( "enterFrame", soundDetector );
g:insert(soundDetector)

Another limitation is that you can't currently play sound back at the same time as you are recording. So no ocarinas or trumpets with Corona just yet.

Thanks for the info. So the sound detection from the mic isn't possible on Android using Corona?

@ptotheaul - no, not yet on Android because sound recording doesn't currently work. [edit] Actually I just tried this sample code on a Droid X and it does work!

@snarla Ah. What version of android are you running? Could it have something to do with that? Wait! You're staff! Can we figure out if this is now supported?

Hi all,

I'm glad the thread finally gains traction. I did some testing with the code provided by Snarla (thanks again for that). I added another output text field for the raw input value from getTunerVolume and set the positive threshold down to 2 to get better result on my testing devices (iPhone 3G and iPod touch 4th gen), see the adjusted code below.

I did a lot of testing in completely different environment with the Corona code below and the program mentioned in my original post based on AVAudioRecorder from the Cocoa library (Objective-C). My observation is as follows and I'm curious if others have similar experience.

The code works in very silent environment with no sudden disturbing noise. The raw value is between 0.2 and 0.3 in that case.
If there is just some slight background noise (some running computers in office, typing/clicking sounds) the raw value is fluctuating a lot between 0.4 and 0.6 and it either no blow is recognized or many false once (no blowing, just some noise).
In a subway with people around (talking or not) and noise from the train itself the value is constantly around 0.5 to 5.8., so no blow can be detected.
If the background is really loud (I attended a concert and tested the program) the raw value is constantly around 0.5 to 0.6. The good thing is that it wasn't fluctuating and thus not detecting wrong blows, but I really cannot understand why even in such a loud environment the value never grew bigger than around 0.6.

So from my experience it seems that the calculated value for getTunerVolume is way to sensitive. I also tested it on my mac book, there the input has much lower values compared to the iPhone, but also a lot of fluctuation from just office background noise (no people talking) and constantly detects wrong blows. Can anyone else confirm this behavior?

The values read from AVAudioRecorder behave much differently. Even in a subway with noisy people the peak value very seldom reaches 0 (which for AVAudioRecorder means highest volume). Only at the concert I got constantly the peak value 0.

@Snarla could you give us some details regarding getTunerVolume and your testing code? Please see my questions below.

What is the highest raw value you measure with an iOS testing device? I never saw any value beyond 0.62 no matter how loud the surrounding are or how much I blow into the microphone.
From what I read so far regarding sound pressure/level comparision (eg. http://www.animations.physics.unsw.edu.au/jw/dB.htm) the formula to compare 2 sound levels is as follows:
1
20 * log10 * (p2/p1)
I am getting the same issues here. Actually on the new ipod touch the volume just stays to the max. It never drops below .560.
It makes it unusable. It's way too sensitive.

Any fix for getting the microphone volume?
I've tried the demo and all the examples, also tried looking to the raw values but honestly I don't understand if it's just working at random or it's too sensitive.

Since it's a problem reported about 4 months ago I'm sure you've fixed, just can't find the correct way to get volume value.

Please ansca, Carlos, everyone just explain us how the get a correct volume value, otherwise admit this feature is just not working

Hi Shedder,

I don't think there has been any update on this issue. Generally they never acknowledged that there is something wrong with the tuner getVolume function or at least explain in detail why the function is behaving the way it does (again, from my experience it appears to be much too sensitive). Thus after spending a lot of time I simply had to give up on detecting microphone volume and moved on without this feature. It would be nice though if this will be fixed in a future version, but right now I'm not holding my breath.

Thomas

Hi Teichmann,
I'm not even so sure it's just too sensitive. On my iPhone 3G it looks completely random, if I'm in some place with no sounds or in the subway, the values jump around in the same way.
It's useless. I mean, how can you possibly do something with that function?

It's frustrating... I had a request from a client about making an app that needed to measure noise levels.. I checked the Corona API and I found the getVolume so I told him, of course, I can do it.

Well.. now I'm turning down that job, since the getVolume function is a joke. And I don't understand why Ansca is not fixing it.. I mean, it should not be that hard or time consuming!

BTW, I've already turned down another job because it needed a multi-line input textbox (not possible with Corona)... and the new UI library? it was "almost" ready months ago...

I like Corona, honestly, I also wrote about it on Ansca blog, but now I'm definitely considering starting to learn Obj C and switch to xCode since it's just missing too many basic features.

It's ok if you guys can't add support of 3D or 2.5D, shaders or image overlay methods and filters.. but at least getting the Mic Volume? The audio panning (no pitch, 3d effects.. just left and right)? Multiline input boxes? Good listeners for maps? A way to communicate with web views (the workarounds are not practical at all)? Retina texts (native)?
Those are basic features..

I understand you're working on fixing Android performance right now, but we, developers, should have access to Corona Roadmap. We need to know if you're really working on a certain feature and an estimate on when it's going to be ready.
We are your customers, but we also have clients, we need to know what we can do and when we'll be able to offer a certain feature.

Hey does the code to detect sound work well for simulator. Because the values are continuously fluctuating whether there is any sound or not. I want to know how to get a more stable value for volume as I don't want it to keep fluctuating. This volume value will be used as input for display function.

Please help guys.

Hey does the code to detect sound work well for simulator. Because the values are continuously fluctuating whether there is any sound or not. I want to know how to get a more stable value for volume as I don't want it to keep fluctuating. This volume value will be used as input for display function.

Please help guys.

Hey did u find any solution to the problem. I am currently working on creating an app that requires to determine the volume. Also the value of the volume cant keep fluctuating. Please help. I need to create this app as part of my final thesis and I have less then 2 weeks to wrap it up this part is holding me back , would really appreciate if you could guide me.

Awaiting your reply :) and thanks in advance.

Actually, at least from my tests, getTunerVolume seems to work on Simulators (Corona and xCode), it's probably too sensitive but it works.

The problem is that it doesn't work on devices!

I mean, if I launch the SimpleTuner example on the simulator and make some noise, the values change accordingly.. however if I build it and install on device (tested on iPhone 3G and iPad) the values are basically random... it's basically useless.

I've submitted a bug report few days ago, let's see..

I just found this thread and I wanted to bump it up to the top again, as I thought it would be cool to try audio to control a player by using the Iphone4's two mic's to detect where the sound is coming from and move player accordingly.

This would work in just like how you use the accelerometer to tilt and move player.

Now, from I can read from the posts in this thread I could only assume that CoronaSDK is not "there" just yet to support this type of controls.

lano78

Is there a way in which I can get the recorded data to buffers (e.g. double buffering) instead of to a file.
I need to analyze the data, like in a spectrum analyzer.
Thanks

Hi folks,

It's great to see so much interest in sound input in the Corona community!

Ansca may eventually be able to devote resources to developing more powerful audio analysis features, but for now we have to make do with what we have, i.e. the Tuner API (which was designed for pitch detection, not for general audio analysis).

In the code sample I provided, please take note of the threshold parameters and the inline comments. Most likely you *will* need to figure out a way to adjust them to work with your particular input device(s). Please also take notice of the simple running average used to compute the background level. This is another part we hope you will play with and adjust and improve for your own purposes.

A really good "blowing" detector should not just respond to volume increases and decreases, but also to the frequency content of the sound: ideally it should be able to tell the difference between blowing and speech, for example. This type of feature detection is not available in Corona at the moment.

The raw values from getTunerVolume() depend on the microphone, the device, and the testing environment and are NOT very useful without being converted to some kind of dB scale. So, don't worry if they don't seem to be changing very much. That is why this example is written to respond mainly to *changes* in the input. In my testing I've found that for some reason an iPod with a mic is noticeably less sensitive than a 3GS with its built-in mic.

@teichmann you are technically right about the log conversion, properly the factor should be log base 10 of e which is 0.434 or so, not 0.301 (oops). BUT you can just drop this factor altogether and save yourself some processor load, since this code is just detecting level changes and the threshold can be adjusted. Also the dB scale here is dBfs (dB relative to full scale, which is 1.0 in this case). Also, no this is not the same as what you get from AVAudioRecorder. The metering functions in AVAudioRecorder probably also perform additional processing.

@rony_sha: Sorry, this is not possible in Corona at the moment.

hi snarla,

I tried out your code and got the same effect as shredder. The values are just all over - no matter if I shout, put the iphone in front of loud music or in a silent room.

Do you know why this is?

As I said, you need to convert to dB and process the data in some way to detect features. You cannot use the raw values in any kind of absolute way.

@shedder, your inputs are welcome and there is a forum specifically for the roadmap and feature requests.

@snarla, thanks for the example. Just to be sure I compiled and tried again on 3 devices (iPhone 3G, iPad 1st gen, iPod touch 4g) as well as the simulator.
I also decided to print the v value ( v = 20 * 0.434 * math.log(v) ) and the raw r:getTunerVolume() to see what's happening.

From my tests the v value does indeed change when it detects a volume level change. The default values are different on each device I tested, but react correctly.

The problem is that, I think, getTunerVolume reacts to a volume change from complete silence to very tiny noises, but it's not correctly measuring louder levels.

For example, if in complete silence I got a value at around 0.3/0.4, then I turn on gradually a speaker playing white noise, the value rises suddenly to 0.5/0.6... and then it stays there.
It definitely looks like it's saturating or capping the input level.

A very different result from the Voice Memos app that respond beautifully.

Please, give a look to this other post:
http://developer.anscamobile.com/forum/2011/08/06/gettunervolume-not-returning-expected-values

Duff333 is playing a white noise sound that start silent and get gradually louder and recording getTunerVolume values. The value starts rapidly rising and then it just stop at 0.58. Definitely looks like a saturated input.

snarla, can you tell me if that looks correct to you?
Hi Snarla,

I'm happy more developers joined this discussion and appreciate your time to reply. But I don't understand how it is possible that everyone reports similar problems and observations, while you are saying it works for you.
- I don't know in what environment you are testing, in an absolutely quiet laboratory or a normal office. But the absolute value of the getTunerVolume function is around 0.55 to 0.6 in the office I work. There are just a few desktop computers running while nobody is even talking, no music etc. If I go to a very quiet room, the value is still around 0.3. So as I said already back in January and others mentioned as well, the function is totally oversensitive to even small noise in the background (or saturated, however you call it).
- As I wrote in my 2 bug report filings (cases 2579 and 7388): if the input is already saturated around 0.55 - 0.6 and start to blow into the microphone the value from getTunerVolume gets even *lower*. Can you please explain me, how is this possible? I tested on iPhone 3G and iPod touch 4th generation with built-in microphones and a iPod touch 3rd generation with external microphone. All behave the same.
- What about the issue that it is not possible on iOS to record sound and play sound simultaneously? Is there any plan to fix this issue?
You wrote in one of your answers: "Ansca may eventually be able to devote resources to developing more powerful audio analysis features, but for now we have to make do with what we have, i.e. the Tuner API (which was designed for pitch detection, not for general audio analysis)."

To me it sounds like the API is ok if you are trying to detect the pitch of the input, for example for tuning up a music instrument. But for doing anything about volume detection, the API is just not there yet. If we can all agree on this, then it would be a nice move from Ansca to write this clearly in the API documentation.

I'm sorry for my sour tone, I do like Corona in a lot of ways. But this is one of the cases that frustrate me a lot. I just cannot understand why Ansca is providing this API if it is not working properly as many are reporting, but just saying take it or leave it (while strangely, it works so much better on Android). If Ansca doesn't have the resources or time to fix it then at least state clearly that it is broken and update the documentation. Because we developers also have clients or employers we need to answer to.

I mean look at the excellent thread Shedder pointed to (big thanks for that), do we really need any other indications that the funcktion is broken on iOS right now?
http://developer.anscamobile.com/forum/2011/08/06/gettunervolume-not-returning-expected-values

Ok, now I feel a bit better :-). Snarla, please don't take any offence, that's not my intention. But Ansca really should take a clear stance on this issue.

Best,
Thomas
As Thomas said, definitely we don't want to offend anyone.
It's just that the getTunerVolume problem has been reported many months ago, and at today, there hasn't been a clear explanation from Ansca.

Honestly, I think getTunerVolume is broken.
But, if that's not the case and the problem is that somehow we are doing it wrong, don't know how to correctly adjust the parameters or how to interpret the raw data coming from the API, please Ansca, write a new blog post with some code showing the DB level or even better, a VU meter that works exactly like the Voice Memo App.

That would definitely, once and for all, put a stop to the getTunerVolume issue.

I agree with the people here. The sample code does not work on the device. If you can, please give a more detailed explanation and example on how we can use the output of getTunerVolume... instead of just telling us that we are doing it wrong. That would be greatly appreciated.
Thanks

Can you define precisely what it would mean to "work"? How precisely would you measure this?

If you could try to see if you can identify precisely what your technical needs are and then let us know, that would help a lot more than just saying "I want to make a cool app that does (some fancy thing)" (although that is good to hear about too)

Our stance is that we make stuff available to you to see if you can do cool stuff with it. YMMV :) We're always listening to your helpful feedback. We can't always act on it immediately.

Hi Snarla,

Could you please have a look at the analysis from duff333 in this thread? I think it shows quite well, why we all think the getTunerVolume output is not useful.

http://developer.anscamobile.com/forum/2011/08/06/gettunervolume-not-returning-expected-values

"I want to make a cool app that does (some fancy thing)"

Honestly I think none of us was writing anything like that. On the contrary, I feel I spend a lot of time to write precise questions and findings from the testing, but you never bother to really answer to each of them. I list them in bullet points to make them very easily identifiable.

As I said, I think Corona is doing a lot of things right, that's why many developers are quite happy with it. But saying "Our stance is that we make stuff available to you to see if you can do cool stuff with it" while not fixing things that are apparently broken (please, proof us otherwise if we are all wrong) is quite a statement for paying customers.

Thomas

Hi Snarla,

This is one thing that doesnt "work" ( well for me ):
http://developer.anscamobile.com/content/simple-tuner

What I expected from this is, that it will read out the volume of the sounds that the microphone detects. Person shouting on the microphone= 10. No person shouting and just background noise= 5. Something like that.

But the volume on this example just goes up and down within a certain range, even if I blow, shout, speak, put it infront of the speakers when playing music.

If what I expected this application to do is wrong, please let me know.
There is no cool stuff happening here, I just want to know how to use Corona.

It sounds like you are looking for a VU metering function. The tuner's getVolume function is not designed to be a VU meter and the project is intended to demonstrate how to use the Tuner API for pitch detection.

The Simple Tuner example, besides frequency, is also showing "volume".
In the code that line is commented with "-- convert to db" .
Is it showing decibels detected by the iPhone? If that so, the values are wrong.

Gettunerfrequency returns the frequency, but GetTunerVolume what is supposed to do?
What is exactly returning?
How are we supposed to use it in real world apps?

I could definitely build a VU meter if it didn't saturate the input level like explained in this topic:
http://developer.anscamobile.com/content/simple-tuner

But anyway.. I give up, clearly there's no interest by Ansca to acknowledge or resolve this problem (hey, there's NO problem, right?)

Hi Snarla,

If that is the case, I believe that the "volume" should be removed from the example. I don't know if the other developers were mislead to thinking that it detects the volume correctly - but I sure was.

Thanks

views:6391 update：2011/9/27 8:54:05


Label: corona android iphone ipad building error inmobi problem display screen View image tableView failed object physics touch button game lua config event listener