Understanding GStreamer: gstaudiomixerutils

This is my first article of GStreamer in my blog. I was wondering whether to write in Spanish or in English. Since I had no luck while finding documentation of this topic (besides the official manual) even in English, therefore, writing in english ensures to target a wider audience.

About Gstreamer

I want to write a simple application that records audio from the sound card using GStreamer.I have managed to do that BUT: It reads a lot of garbage that I don't know where it comes from; also, I want to be able to select whether to record from line-in, cdrom, or microphone. So a little more understanding of GStreamer is required here.

GStreamer's power come mostly from all the wealth of plugins available that allows GStreamer to handle almost everything that may come up. The rest of it's power comes from it's core design although it is not the scope of this post to discuss GStreamer's core design.

The Mixer Interface

The way GStreamer supports the soundcard mixer is through its Base plugin package. This package includes the following libraries:

• Audio Library
• CDDA Library
• FloatCast Librery
• Interfaces library
• Network Buffer Library
• Riff Media Library
• RTP Library
• Tag Support Library
• Base Utils Library
• Video Library

From all this libraries, of special interest are the Audio Library and the Interfaces library, which contain the following elements.

Audio Library

gstaudio - Support library for audio elements

gstaudioclock - Helper object for implementing audio clocks

gstaudiofilter - Base class for simple audio filters

gstaudiomixerutils - utility functions to find available audio mixers from the plugin registry

gstbaseaudiosink - Base class for audio sinks

gstbaseaudiosrc - Base class for audio sources

gstaudiosink - Simple base class for audio sinks

gstaudiosrc - Simple base class for audio sources

gstmultichannel -

gstringbuffer - Base class for audio ringbuffer implementations

Interfaces Library

gstcolorbalance - Interface for adjusting color balance settings

gstcolorbalancechannel -

gstmixer - Interface for elements that provide mixer operations

gstmixeroptions -

gstmixertrack -

gstnavigation -

gstpropertyprobe -

gsttuner - Interface for elements providing tuner operations

gsttunerchannel -

gsttunernorm -

gstvideoorientation - Interface for elements providing video orientation controls

gstxoverlay - Interface for setting/getting a Window on elements supporting it

First Example: listing the available mixers

Of all these elements we will focus on gstaudiomixerutils from the Audio Library. This sub-library contains a utility function: gst_audio_default_registry_mixer_filter(). This function explores all the GStreamer Registry looking for factory elements that provide a mixer interface. When it finishes traversing all the registry it will return a GLib doubly linked list (GList) with all the available GstElements available that provide an audio Mixer Interface. Additionally, the function gst_audio_default_registry_mixer_filter() provides a way to call some "callback" whenever it finds a suitable GstElement that provide an audio Mixer interface. which we will not use in this article.

All right!! let's get our hands dirty. The following code does this

1. Initialize Gstreamer
2. Print information about Gstreamer
3. Traverse the default plugin registry in order to find any usable audio mixer. It returns a GList
4. Walk through the list printing all available information of each mixer.
5. Clean up memory and exit.

When we walk across the list of mixers (actually a list of instances of GstElement) we call the function print_mixer_information() which does this:

1. Receives a pointer of the data element of the element of the GList. We try to cast it to the type GstElement.
2. If it is an GstElement we get the GstElementFactory associated with it.
   
3. Once that we manage to get the GstElementFactory associated with our mixer element, we procede to printi the detailed information for this plugin.
4. Finally it frees each element by setting it to it's NULL state and then decrementing it's counting reference.

The source code is reproduced here:

/* mixer_scan.c
 * Copyright (c) 2007 Noe Misael Nieto Arroyo
 * 
 * This program will get a list of available mixers for the system
 * compile with:
 * gcc -o mixer_scan mixer_scan.c `pkg-config --libs --cflags glib-2.0 gstreamer-plugins-base-0.10` -lgstinterfaces-0.10 -lgstaudio-0.10
 */

#include <gst/gst.h>
#include <gst/interfaces/mixer.h>
#include <glib.h>

void print_mixer_information(gpointer data, gpointer extra_data){
GstElement *mixer = GST_ELEMENT(data);
GstElementFactory *fac;

if (GST_IS_ELEMENT(mixer)){
  g_print("GstElement name: %s\n",GST_ELEMENT_NAME(mixer));
  /*The GstElement has a GstElementFactory associated. This 
        GstElementFactory has, as well, another structure associated:
        GstElementDetails.
        Maybe there is another way:
            GST_ELEMENT_GET_CLASS(element)->details
       */
   fac = gst_element_get_factory(mixer);
   g_print("More details about this element:\n\t%s: %s\n\t%s: %s\n\t%s: %s\n\t%s: %s\n",
            "Long name: ", gst_element_factory_get_longname(fac),
            "klass: ", gst_element_factory_get_klass(fac),
            "Description: ", gst_element_factory_get_description(fac),
            "Author: ", gst_element_factory_get_author(fac));
   /*We must free each element by setting it to it's NULL state and then
        decrementing it's counting reference.*/
   gst_element_set_state(mixer,GST_STATE_NULL);
   g_object_unref(G_OBJECT(mixer));
}
}



int main (int argc, char **argv){
guint major,minor,micro,nano;
GList *mixers=NULL;

/*GStreamer Library Initalization*/
gst_init(&argc,&argv);
gst_version(&major,&minor,&micro,&nano);

g_print("mixer_scan by Noe Nieto. Copyright (c) 2007.\n");
g_print("This program will try to list the available mixers and ");
g_print("display it's information in the console.\n");
g_print("This version was linked against GStreamer version %d.%d.%d-%d\n",
              major,minor,micro,nano);

g_print("\n**Traverse the default plugin registry in order of plugin rank and find usable audio mixer ...\n");

mixers = gst_audio_default_registry_mixer_filter(NULL, /*No filter function*/
                                                    FALSE, /*We want all the available mixers*/
                                                    NULL); /*No extra data, 'cos no filter function*/
if (!mixers) {
g_print("**There are no registered mixers in the default plugin registry.\n");
return(1);
}

/**Walk through the list printing all available information*/
/* gst_audio_default_registry_mixer_filter() creates it's elements using
     gst_element_factory_create()*/
g_list_foreach(mixers,print_mixer_information,NULL);

/**All GstElements set to NULL state and decreased it's reference count*/
g_list_free(mixers);
return 0;
}

Now it is time to show the results:

tzicatl@MXP6LKY91:~/Programar/GStreamer-0.10$ ./mixer_scan
mixer_scan by Noe Nieto. Copyright (c) 2007.
This program will try to list the available mixers and display it's information in the console.
This version was linked against GStreamer version 0.10.12-0

**Traverse the default plugin registry in order of plugin rank and find usable audio mixer ...
GstElement name: alsamixerelement0
More details about this element:
      Long name: : Alsa mixer
      klass: : Generic/Audio
      Description: : Control sound input and output levels with ALSA
      Author: : Leif Johnson <leif@ambient.2y.net>
GstElement name: ossmixerelement0
More details about this element:
      Long name: : OSS Mixer
      klass: : Generic/Audio
      Description: : Control sound input and output levels with OSS
      Author: : Andrew Vander Wingo <wingo@pobox.com>
tzicatl@MXP6LKY91:~/Programar/GStreamer-0.10$

Conclusion
GStreamer is a powerful framework for multimedia applicacion. Once installed and configured (which most Linux distros already do), Gstreamer can serve very well for multimedia applications.

My interests focus on Data Acquisition, signal processing and lately speech processing. GStreamer seems to be the path to follow to develop professional applications that cover those special areas.