I am trying to improve the tensorflow micro_speech example for sparkfun_edge.
The tensorflow code contains the following lines:
// Temporary gain emulation to deal with too-quiet audio on prototype boards.
const int16_t kAdcSampleGain = 10;
It looks like at least my board that I picked up last week does not actually need this workaround and oversaturates with it and exacerbate issues caused by inaccurate kAdcSampleDC.
With gain emulation disabled the example actually recognized a few times “No” - something that practically never happened with gain emulation.
I am still working through some issues but would eventually like to upstream my changes to tensorflow - however this leads to the following questions:
How many different versions of the board are in circulation (prototype?, TF conference? Release?)
What exactly are the hardware differences ( So I can try to auto detect or implement automatic gain control )
Feb 25: Change Mic gain to 75x : This is Pete’s requested final gain value
Feb 12 : Change to 100x gain with 1M ohm resistors for mics
and the fact that TensorFlow Dev Summit was March 6-7 - can I assume all boards outside a few internal prototypes ( Sparfun and Google) contain the 75x gain?
Nice work being thorough or your TF PR. Yes you are correct – all the Edge boards that are in consumer’s hands have the 75x gain setup. The prototypes mentioned in the code were limited to a few samples that we sent to Google so that they could begin writing the example code ahead of launch.
Together with reducing average_window_duration_ms to 500 in recognize_commands.h my board can now recognizes “yes” and “no” most of the time. ( I suspect my “no” is just too short - but haven’t really though this through yet)
ups@:
I am trying to improve the tensorflow micro_speech example for sparkfun_edge.
The tensorflow code contains the following lines:
// Temporary gain emulation to deal with too-quiet audio on prototype boards.
const int16_t kAdcSampleGain = 10;
It looks like at least my board that I picked up last week does not actually need this workaround and oversaturates with it and exacerbate issues caused by inaccurate kAdcSampleDC.
With gain emulation disabled the example actually recognized a few times “No” - something that practically never happened with gain emulation.
I am still working through some issues but would eventually like to upstream my changes to tensorflow - however this leads to the following questions:
How many different versions of the board are in circulation (prototype?, TF conference? Release?)
What exactly are the hardware differences ( So I can try to auto detect or implement automatic gain control )
Thanks
Stephan
Hello Stephan,
How can I be sure the audio samples from the mics are on correct range? I could e.g. log the values in audio_provider.cc but I dont what is correct and what is not. If I change the settings like you have done, I still have problems. I think the audio samples are scaled wrong and I try to set them on correct range. Should the range be like [0, 255]?
Do you know what is the purpose of micro_features_generator.cc? They are remapping the audio samples to same range?
The ADC is 14bit and uses a MCU internally generated reference voltage. (2V if I remember it right)
The microphones are coupled through capacitors to amplifiers that are connected to the ADC.
The amplifier is a simple op-amp amplifying the signal and using a voltage divider connected to the supply voltage to generate a virtual zero.
There are two issues with the design that I verified dumping audio data by stealing the code form the TF/micro person_detection example to dump frame buffers. (And modifying the audio module to also dump supply voltage)
The virtual zero for audio moves based on supply voltage. Supply voltage changes on load changes (more so powered from serial adapter than from battery) - I could observe the impact of the blue inference led blinking in both voltage and audio data capture.
The audio data captured indicate audio voltage peeks going above the ADC range and audio lows running into non-linearity near 0V.
The audio provider converts these vales to signed 16 integers and the current tensorflow workarounds for the prototype board cause saturation of the audio data.
The micro_features_generator performs an FFT on overlapping 30ms time slices - extracts and bins frequency energies in a way to emulate human sensitivity and produces a picture where the X axis is time and the Y axis are the frequency bins.
Stefan, have you noticed there is Edge 2 board now available which has different microphones. Maybe they solved some mic problems which you also found. I have the old and new board. The new board is delivered with a demo software which shows audio frequencies from microphones. I think Edge 2 board microphones work well based on this demo. I would really like to try the new board and speech detection asap. But I don’t know yet how to flash the new board because they removed button 14 from the board, there is only reset button…
I bought a few Artemis boards (Not the Sparkfun Edge V2) that have a PCM microphone on board - but have not found a good reason to play with them yet.
I believe the flashing is now triggered by receiving the right magic strings over serial in a short time window after reset.
( And the USB to serial adapter automatically causes a reset on opening the device from the host ).
As far as I know this is the standard boot loader from theA MCU SDK.
The apollo3evb target may work for you if the LED and PCM pins match up with your board - if not they should contain all software needed. (PCM capture)
Stephan… you are a code whispering logic analyzing monster! (Or you read the source code, but that’s still not entirely common behavior and so I am impressed)
Yes there is indeed a new bootloader that operates with a UART handshake shortly after reset. But the original bootloader is still in play one layer deeper as a failsafe (and for advanced users to get a little more flash space)
A word of caution to anyone trying out TensorFlow speech detection on the new (Artemis based) boards - it won’t work out of the box yet. I need to get a PR through the TF team and also debug a little weirdness. If you want to try it for yourself use the github link below as an example and remember to use the new uploading tools that will work with Artemis.
I am really interested in seeing if I can get the Tensorflow Speech detection working on my Artemis Red Board. I have downloaded the SparkFun Tensorflow fork from above and was able to get things compile and flashed.
After I reset the board, none of the LEDs light up. I do get the following on the serial console:
FPU Enabled.
PDM DMA Threshold = 16
Anything obvious I should try? Is the TensorFlow example working on Artemis yet?
The symptoms you are describing are the same that I have found. I don’t have a fix yet b/c I am too occupied with other projects to start diving into the tensorflow portion of the code. What I mean to suggest is that if you’ve gotten it properly set up for your RedBoard Artemis then you should be able to see audio data being captured (add debug statements with the error reporter) and see that it is hanging up (HardFault_Handler) in audio data processing. Why there is a discrepancy between Edge and any other Apollo3-based board at that point is beyond me.
I have reached out to some of the TFLM (TensorFlow Lite Micro) guys at google and asked for assistance. So far I haven’t heard a peep. We may need a community effort to dig into this one.
Can you explain w/ just a little more detail what process you used? E.g. “I added a sparkfun_redboard_artemis target, using the updated BSPs, am compiling on the command line with AmbiqSuite SDK 2.2.0, and am uploading the binary w/ TOOL_NAME”
I’m just eager to get as much info as possible. Do you have a GitHub account? It would help bring visibility to the PR if you contributed to the discussion there too Thanks!
And I’m going to keep posting the link to the PR till the end of days:
Awesome! Okay, I wanted to make sure I wasn’t missing something. I will go start-up a thread in the Artemis forum on the steps I took. Right now I am using the commands for the Edge 2, so I probably have to recraft them to point to the BSP for the RedBoard.
I had been lurking on the linked PR, but I can jump in once I get some findings… hopefully shortly!
Thanks!