secret
Created

libav how to encode audio

  • Download Gist
audio_test.c
C
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73
bool AV::addAudioFrame(unsigned char* buffer, int nsamples, int nchannels) {
AVCodecContext* c = ct.as->codec;
AVPacket packet = {0}; // data and size must be '0' (allocation is done for you :> )
AVFrame* frame = avcodec_alloc_frame();
int got_packet = 0;
// BUFFER HANDLING
int samples_stored = av_audio_fifo_write(ct.afifo, (void**)&buffer, nsamples);
if(samples_stored != nsamples) {
return false;
}
int nstored = av_audio_fifo_size(ct.afifo);
if(nstored < c->frame_size) {
return false;
}
 
av_init_packet(&packet);
int use_nsamples = c->frame_size;
frame->nb_samples = use_nsamples; // <-- important, must be set before avcodec_fill_audio_frame
// GET DATA FROM BUFFER
int num_bytes = av_samples_get_buffer_size(NULL, c->channels, use_nsamples, c->sample_fmt, 0);
uint8_t* my_buffer = (uint8_t*)av_malloc(num_bytes);
uint8_t** my_ptr = &my_buffer;
int nread = av_audio_fifo_read(ct.afifo, (void**)my_ptr, use_nsamples);
if(nread != use_nsamples) {
printf("We only read: %d but we wanted to read %d samples.\n", nread, use_nsamples);
av_free(my_buffer);
return false;
}
// FILL
int fill_result = avcodec_fill_audio_frame(
frame
,c->channels
,c->sample_fmt
,(uint8_t*)my_buffer
,num_bytes
,1
);
if(fill_result != 0) {
char buf[1024];
av_strerror(fill_result, buf, 1024);
printf("av error: %s\n",buf);
av_free(my_buffer);
return false;
}
 
// ENCODE
int enc_result = avcodec_encode_audio2(c, &packet, frame, &got_packet);
packet.stream_index = ct.as->index;
if(!got_packet) {
av_free(my_buffer);
return false;
}
if(enc_result < 0) {
char buf[1024];
av_strerror(enc_result, buf, 1024);
printf("av error: %s\n",buf);
}
 
// WRITE
if(av_interleaved_write_frame(ct.c, &packet) != 0) {
printf("Cannot write audio frame.\n");
av_free(my_buffer);
return false;
}
 
av_free(my_buffer);
return true;
}
mail.txt
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
Hi!
 
I'm using libav to encode raw RGB24 frames to h264 and muxing it to flv. This works
all fine and I've streamed for more then 48 hours w/o any problems! My next step
is to add audio to the stream. I'll be capturing live audio and I want to encode it
in real time using speex, mp3 or nelly moser.
 
Background info
----------------------
I'm new to digital audio and therefore I might be doing things wrong. But basically my application
gets a "float" buffer with interleaved audio. The buffer contains 256 samples per channel,
and I have 2 channels. Because I might be mixing terminology, this is how I use the
data:
 
 
// input = array with audio samples
// bufferSize = 256
// nChannels = 2
void audioIn(float * input, int bufferSize, int nChannels) {
// convert from float to S16
short* buf = new signed short[bufferSize * 2];
for(int i = 0; i < bufferSize; ++i) { // loop over all samples
int dx = i * 2;
buf[dx + 0] = (float)input[dx + 0] * numeric_limits<short>::max(); // convert frame of the first channel
buf[dx + 1] = (float)input[dx + 1] * numeric_limits<short>::max(); // convert frame of the second channel
}
 
// add this to the libav wrapper.
av.addAudioFrame((unsigned char*)buf, bufferSize, nChannels);
delete[] buf;
}
 
Now that I have a buffer, where each sample is 16 bits, I pass this short* buffer, to my
wrapper "av.addAudioFrame()" function. In this function I create a buffer, before I encode
the audio. From what I read, the AVCodecContext of the audio encoder sets the frame_size.
This frame_size must match the number of samples in the buffer when calling avcodec_encode_audio2().
Why I think this, is because of what is documented here:
http://libav.org/doxygen/master/group__lavc__encoding.html#ga93a49fbd0973b216dcb8a8c5dffe1d82
 
Then, especially the line: "If it is not set, frame->nb_samples must be equal to avctx->frame_size
for all frames except the last." (Please correct me here if I'm wrong about this).
 
After encoding I call av_interleaved_write_frame() to actually write the frame.
When I use mp3 as codec my application runs for about 1-2 minutes and then my server, which is
receiving the video/audio stream (flv, tcp), disconnects with a message "Frame too large: 14485504".
Also in my logs I see several messages from libav: "Stream over/underflow detected."
 
Questions:
------------
- There quite some bits I'm not sure of, even when going through the source code of libav and therefore
I hope if someone has an working example of encoding audio which comes from a buffer which which
comes from "outside" libav (i.e. your own application).
 
- As I wrote above I need to keep track of a buffer before I can encode.
Does someone else has some code which does this? I'm using AVAudioFifo now.
 
- I compiled with --enable-debug=3 and disable optimizations, but I'm not seeing any
debug information. How can I make libav more verbose?
 
Thanks!
roxlu

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.