OS X/iOS - konwersja częstotliwości próbkowania bufora przy użyciu AudioConverterFillComplexBuffer

Question

Piszę backend CoreAudio dla audio library called XAL. Bufory wejściowe mogą mieć różne częstotliwości próbkowania. Używam pojedynczej jednostki audio dla wyjścia. Pomysł polega na przekształceniu buforów i wymieszaniu ich przed wysłaniem do jednostki audio.

Wszystko działa tak długo, jak bufor wejściowy ma takie same właściwości (częstotliwość próbkowania, liczba kanałów itp.) Jako wyjściowy zespół audio. W związku z tym część mieszająca działa.

Jednak utknąłem z częstotliwością próbkowania i przeliczaniem liczby kanałów. Z tego co wymyśliłem, jest to najłatwiejsze do zrobienia dzięki API Audio Converter Services. Udało mi się skonstruować konwerter; chodzi o to, że format wyjściowy jest taki sam jak format jednostki wyjściowej, ale ewentualnie dostosowany do celów konwertera.

Konwerter audio został pomyślnie skonstruowany, ale po wywołaniu AudioConverterFillComplexBuffer(), otrzymuję komunikat o błędzie wyjścia -50.

Chciałbym, gdybym mógł uzyskać inny zestaw gałek ocznych na temat tego kodu. Problem prawdopodobnie znajduje się poniżej AudioConverterNew(). Zmienna stream zawiera dane przychodzących (i wysyłanych) buforów, a streamSize zawiera bajt danych przychodzących (i wychodzących) bufora.

Co zrobiłem źle?

void CoreAudio_AudioManager::_convertStream(Buffer* buffer, unsigned char** stream, int *streamSize) 
{ 
    if (buffer->getBitsPerSample() != unitDescription.mBitsPerChannel || 
     buffer->getChannels() != unitDescription.mChannelsPerFrame || 
     buffer->getSamplingRate() != unitDescription.mSampleRate) 
    { 
     printf("INPUT STREAM SIZE: %d\n", *streamSize); 
     // describe the input format's description 
     AudioStreamBasicDescription inputDescription; 
     memset(&inputDescription, 0, sizeof(inputDescription)); 
     inputDescription.mFormatID = kAudioFormatLinearPCM; 
     inputDescription.mFormatFlags = kLinearPCMFormatFlagIsPacked | kLinearPCMFormatFlagIsSignedInteger; 
     inputDescription.mChannelsPerFrame = buffer->getChannels(); 
     inputDescription.mSampleRate = buffer->getSamplingRate(); 
     inputDescription.mBitsPerChannel = buffer->getBitsPerSample(); 
     inputDescription.mBytesPerFrame = (inputDescription.mBitsPerChannel * inputDescription.mChannelsPerFrame)/8; 
     inputDescription.mFramesPerPacket = 1; //*streamSize/inputDescription.mBytesPerFrame; 
     inputDescription.mBytesPerPacket = inputDescription.mBytesPerFrame * inputDescription.mFramesPerPacket; 
     printf("INPUT : %lu bytes per packet for sample rate %g, channels %d\n", inputDescription.mBytesPerPacket, inputDescription.mSampleRate, inputDescription.mChannelsPerFrame); 

     // copy conversion output format's description from the 
     // output audio unit's description. 
     // then adjust framesPerPacket to match the input we'll be passing. 

     // framecount of our input stream is based on the input bytecount. 
     // output stream will have same number of frames, but different 
     // number of bytes. 
     AudioStreamBasicDescription outputDescription = unitDescription; 
     outputDescription.mFramesPerPacket = 1; //inputDescription.mFramesPerPacket; 
     outputDescription.mBytesPerPacket = outputDescription.mBytesPerFrame * outputDescription.mFramesPerPacket; 
     printf("OUTPUT : %lu bytes per packet for sample rate %g, channels %d\n", outputDescription.mBytesPerPacket, outputDescription.mSampleRate, outputDescription.mChannelsPerFrame); 

     // create an audio converter 
     AudioConverterRef audioConverter; 
     OSStatus acCreationResult = AudioConverterNew(&inputDescription, &outputDescription, &audioConverter); 
     printf("Created audio converter %p (status: %d)\n", audioConverter, acCreationResult); 
     if(!audioConverter) 
     { 
      // bail out 
      free(*stream); 
      *streamSize = 0; 
      *stream = (unsigned char*)malloc(0); 
      return; 
     } 

     // calculate number of bytes required for output of input stream. 
     // allocate buffer of adequate size. 
     UInt32 outputBytes = outputDescription.mBytesPerPacket * (*streamSize/inputDescription.mBytesPerFrame); // outputDescription.mFramesPerPacket * outputDescription.mBytesPerFrame; 
     unsigned char *outputBuffer = (unsigned char*)malloc(outputBytes); 
     memset(outputBuffer, 0, outputBytes); 
     printf("OUTPUT BYTES : %d\n", outputBytes); 

     // describe input data we'll pass into converter 
     AudioBuffer inputBuffer; 
     inputBuffer.mNumberChannels = inputDescription.mChannelsPerFrame; 
     inputBuffer.mDataByteSize = *streamSize; 
     inputBuffer.mData = *stream; 

     // describe output data buffers into which we can receive data. 
     AudioBufferList outputBufferList; 
     outputBufferList.mNumberBuffers = 1; 
     outputBufferList.mBuffers[0].mNumberChannels = outputDescription.mChannelsPerFrame; 
     outputBufferList.mBuffers[0].mDataByteSize = outputBytes; 
     outputBufferList.mBuffers[0].mData = outputBuffer; 

     // set output data packet size 
     UInt32 outputDataPacketSize = outputDescription.mBytesPerPacket; 

     // convert 
     OSStatus result = AudioConverterFillComplexBuffer(audioConverter, /* AudioConverterRef inAudioConverter */ 
                  CoreAudio_AudioManager::_converterComplexInputDataProc, /* AudioConverterComplexInputDataProc inInputDataProc */ 
                  &inputBuffer, /* void *inInputDataProcUserData */ 
                  &outputDataPacketSize, /* UInt32 *ioOutputDataPacketSize */ 
                  &outputBufferList, /* AudioBufferList *outOutputData */ 
                  NULL /* AudioStreamPacketDescription *outPacketDescription */ 
                 ); 
     printf("Result: %d wheee\n", result); 

     // change "stream" to describe our output buffer. 
     // even if error occured, we'd rather have silence than unconverted audio. 
     free(*stream); 
     *stream = outputBuffer; 
     *streamSize = outputBytes; 

     // dispose of the audio converter 
     AudioConverterDispose(audioConverter); 
    } 
} 


OSStatus CoreAudio_AudioManager::_converterComplexInputDataProc(AudioConverterRef inAudioConverter, 
                   UInt32* ioNumberDataPackets, 
                   AudioBufferList* ioData, 
                   AudioStreamPacketDescription** ioDataPacketDescription, 
                   void* inUserData) 
{ 
    printf("Converter\n"); 
    if(*ioNumberDataPackets != 1) 
    { 
     xal::log("_converterComplexInputDataProc cannot provide input data; invalid number of packets requested"); 
     *ioNumberDataPackets = 0; 
     ioData->mNumberBuffers = 0; 
     return -50; 
    } 

    *ioNumberDataPackets = 1; 
    ioData->mNumberBuffers = 1; 
    ioData->mBuffers[0] = *(AudioBuffer*)inUserData; 

    *ioDataPacketDescription = NULL; 

    return 0; 
} 

Answer 1

kod Praca dla konwersji częstotliwości próbkowania Core Audio i kanału konwersji zliczania, używając audio Services Konwerter (teraz dostępny jako część BSD-licensed XAL audio library):

void CoreAudio_AudioManager::_convertStream(Buffer* buffer, unsigned char** stream, int *streamSize) 
{ 
    if (buffer->getBitsPerSample() != unitDescription.mBitsPerChannel || 
     buffer->getChannels() != unitDescription.mChannelsPerFrame || 
     buffer->getSamplingRate() != unitDescription.mSampleRate) 
    { 
     // describe the input format's description 
     AudioStreamBasicDescription inputDescription; 
     memset(&inputDescription, 0, sizeof(inputDescription)); 
     inputDescription.mFormatID = kAudioFormatLinearPCM; 
     inputDescription.mFormatFlags = kLinearPCMFormatFlagIsPacked | kLinearPCMFormatFlagIsSignedInteger; 
     inputDescription.mChannelsPerFrame = buffer->getChannels(); 
     inputDescription.mSampleRate = buffer->getSamplingRate(); 
     inputDescription.mBitsPerChannel = buffer->getBitsPerSample(); 
     inputDescription.mBytesPerFrame = (inputDescription.mBitsPerChannel * inputDescription.mChannelsPerFrame)/8; 
     inputDescription.mFramesPerPacket = 1; //*streamSize/inputDescription.mBytesPerFrame; 
     inputDescription.mBytesPerPacket = inputDescription.mBytesPerFrame * inputDescription.mFramesPerPacket; 

     // copy conversion output format's description from the 
     // output audio unit's description. 
     // then adjust framesPerPacket to match the input we'll be passing. 

     // framecount of our input stream is based on the input bytecount. 
     // output stream will have same number of frames, but different 
     // number of bytes. 
     AudioStreamBasicDescription outputDescription = unitDescription; 
     outputDescription.mFramesPerPacket = 1; //inputDescription.mFramesPerPacket; 
     outputDescription.mBytesPerPacket = outputDescription.mBytesPerFrame * outputDescription.mFramesPerPacket; 

     // create an audio converter 
     AudioConverterRef audioConverter; 
     OSStatus acCreationResult = AudioConverterNew(&inputDescription, &outputDescription, &audioConverter); 
     if(!audioConverter) 
     { 
      // bail out 
      free(*stream); 
      *streamSize = 0; 
      *stream = (unsigned char*)malloc(0); 
      return; 
     } 

     // calculate number of bytes required for output of input stream. 
     // allocate buffer of adequate size. 
     UInt32 outputBytes = outputDescription.mBytesPerPacket * (*streamSize/inputDescription.mBytesPerPacket); // outputDescription.mFramesPerPacket * outputDescription.mBytesPerFrame; 
     unsigned char *outputBuffer = (unsigned char*)malloc(outputBytes); 
     memset(outputBuffer, 0, outputBytes); 

     // describe input data we'll pass into converter 
     AudioBuffer inputBuffer; 
     inputBuffer.mNumberChannels = inputDescription.mChannelsPerFrame; 
     inputBuffer.mDataByteSize = *streamSize; 
     inputBuffer.mData = *stream; 

     // describe output data buffers into which we can receive data. 
     AudioBufferList outputBufferList; 
     outputBufferList.mNumberBuffers = 1; 
     outputBufferList.mBuffers[0].mNumberChannels = outputDescription.mChannelsPerFrame; 
     outputBufferList.mBuffers[0].mDataByteSize = outputBytes; 
     outputBufferList.mBuffers[0].mData = outputBuffer; 

     // set output data packet size 
     UInt32 outputDataPacketSize = outputBytes/outputDescription.mBytesPerPacket; 

     // fill class members with data that we'll pass into 
     // the InputDataProc 
     _converter_currentBuffer = &inputBuffer; 
     _converter_currentInputDescription = inputDescription; 

     // convert 
     OSStatus result = AudioConverterFillComplexBuffer(audioConverter, /* AudioConverterRef inAudioConverter */ 
                  CoreAudio_AudioManager::_converterComplexInputDataProc, /* AudioConverterComplexInputDataProc inInputDataProc */ 
                  this, /* void *inInputDataProcUserData */ 
                  &outputDataPacketSize, /* UInt32 *ioOutputDataPacketSize */ 
                  &outputBufferList, /* AudioBufferList *outOutputData */ 
                  NULL /* AudioStreamPacketDescription *outPacketDescription */ 
                 ); 

     // change "stream" to describe our output buffer. 
     // even if error occured, we'd rather have silence than unconverted audio. 
     free(*stream); 
     *stream = outputBuffer; 
     *streamSize = outputBytes; 

     // dispose of the audio converter 
     AudioConverterDispose(audioConverter); 
    } 
} 


OSStatus CoreAudio_AudioManager::_converterComplexInputDataProc(AudioConverterRef inAudioConverter, 
                   UInt32* ioNumberDataPackets, 
                   AudioBufferList* ioData, 
                   AudioStreamPacketDescription** ioDataPacketDescription, 
                   void* inUserData) 
{ 
    if(ioDataPacketDescription) 
    { 
     xal::log("_converterComplexInputDataProc cannot provide input data; it doesn't know how to provide packet descriptions"); 
     *ioDataPacketDescription = NULL; 
     *ioNumberDataPackets = 0; 
     ioData->mNumberBuffers = 0; 
     return 501; 
    } 

    CoreAudio_AudioManager *self = (CoreAudio_AudioManager*)inUserData; 

    ioData->mNumberBuffers = 1; 
    ioData->mBuffers[0] = *(self->_converter_currentBuffer); 

    *ioNumberDataPackets = ioData->mBuffers[0].mDataByteSize/self->_converter_currentInputDescription.mBytesPerPacket; 
    return 0; 
} 

W nagłówku, jako część CoreAudio_AudioManager klasa, tu są odpowiednie zmienne instancji:

AudioStreamBasicDescription unitDescription; 
    AudioBuffer *_converter_currentBuffer; 
    AudioStreamBasicDescription _converter_currentInputDescription; 

---

Kilka miesięcy później patrzę na to i zdałem sobie sprawę, że nie udokumentowałem zmian.

Jeśli jesteś zainteresowany tym, co zmiany były:

• spojrzenie na funkcję zwrotną CoreAudio_AudioManager::_converterComplexInputDataProc
• jeden musi prawidłowo określić liczbę pakietów wyjściowych do ioNumberDataPackets
• to wymagało wprowadzenia nowego zmienne instancji do przechowywania zarówno bufora (poprzedni inUserData), jak i opisu wejściowego (używanego do obliczania liczby pakietów, które mają być przesłane do konwertera Core Audio)
• to obliczenie pakietów "wyjściowych" (dostarczanych do konwertera) odbywa się w oparciu o ilość danych, które otrzymało nasze wywołanie zwrotne, oraz liczbę bajtów na pakiet, który format wejściowy zawiera:

Mamy nadzieję, że ta zmiana pomoże przyszły czytelnik (łącznie ze mną)!