Technical Blog

Create an AR.Drone graphical application

This article is a tutorial to learn the basics of the ARDrone SDK.

It shows the steps to create a minimalist graphical application, with the following features :

  • Display of the drone’s video
  • Buttons to take off / land

If you’re not familiar with the AR.Drone SDK, you should read first my previous Introduction to the AR.Drone SDK.

The AR.Drone Tool

The AR.Drone Tool is the easiest way to create an AR.Drone application on the Linux platform. Parrot provides a basic project skeleton, including all the code needed to initialize the system, gather video and other kind of information transmitted by the drone.

Those files are located in the directory Examples/Linux/sdk_demo/Sources. The files are the following:

  • ardrone_testing_tool.[ch]: the initialization code.
  • Navadata/navdata.[ch]: the navigation data (gyroscope and altimeter information, battery level etc.)
  • Video/video_stage.[ch]: handling of the video stream.
  • UI/gamepad.[ch]: handling of the joystick.
  • UI/ui.[ch]: unknown! (almost empty file).

To create an application, the main task is to customize several functions:

  • ardrone_testing_tool.c: ardrone_tool_init_custom() is called at the initialization of the application, and ardrone_tool_shutdown_custom() when leaving.
  • Navdata/navdata.c: demo_navdata_client_process() is called every time the drone transmits navigation data.
  • Video/video_stage.c: output_gtk_stage_transform() is called every time an image acquired by the drone is received and decoded.

Step 1: Creating the user interface

In this article, I will use GTK. The build system provided by Parrot is designed to use GTK, and the required flags are already present.

In this first step, we will simply create a basic interface with three main widgets:

  • An image widget to display the camera’s video
  • A button to take off
  • A button to land

I created two new files: UI/gui.c and UI/gui.h (click to expand).

#ifndef GUI_H_
# define GUI_H_

# include <gtk/gtk.h>
typedef struct gui
{
  GtkWidget *window;
  GtkWidget *start;
  GtkWidget *stop;
  GtkWidget *box;
  GtkWidget *cam;
} gui_t;

gui_t *get_gui();

void init_gui(int argc, char **argv);

#endif
#include <stdlib.h>
#include "gui.h"

gui_t *gui = NULL;

gui_t *get_gui()
{
  return gui;
}

/* If the drone is landed, only start is clickable,
   if the drone is in the air, only stop is clickable
*/
static void toggleButtonsState(void)
{
  gboolean start_state = gtk_widget_get_sensitive(gui->start);

  gtk_widget_set_sensitive(gui->start, !start_state);
  gtk_widget_set_sensitive(gui->stop, start_state);
}

static void buttons_callback( GtkWidget *widget,
			      gpointer   data )
{
    // FIXME: make the drone start
}

static void on_destroy(GtkWidget *widget, gpointer data)
{
  vp_os_free(gui);
  gtk_main_quit();
}

void init_gui(int argc, char **argv)
{
  gui = vp_os_malloc(sizeof (gui_t));

  g_thread_init(NULL);
  gdk_threads_init();
  gtk_init(&argc, &argv);

  gui->window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
  g_signal_connect(G_OBJECT(gui->window),
		   "destroy",
		   G_CALLBACK(on_destroy),
		   NULL);
  gui->box = gtk_vbox_new(FALSE, 10);
  gtk_container_add(GTK_CONTAINER(gui->window),
		    gui->box);
  gui->cam = gtk_image_new();
  gtk_box_pack_start(GTK_BOX(gui->box), gui->cam, FALSE, TRUE, 0);

  gui->start = gtk_button_new_with_label("Start");
  g_signal_connect (gui->start, "clicked",
		      G_CALLBACK (buttons_callback), NULL);
  gui->stop = gtk_button_new_with_label("Stop");
  g_signal_connect (gui->stop, "clicked",
		      G_CALLBACK (buttons_callback), NULL);
	gtk_widget_set_sensitive(gui->start, TRUE);
  gtk_widget_set_sensitive(gui->stop, FALSE);

  gtk_box_pack_start(GTK_BOX(gui->box), gui->start, TRUE, TRUE, 0);
  gtk_box_pack_start(GTK_BOX(gui->box), gui->stop, TRUE, TRUE, 0);

  gtk_widget_show_all(gui->window);
}

You may have noticed that I am using vp_os_malloc() and vp_os_free() instead of the usual standard library functions. It is required by the SDK. If you don’t do so, you will get error messages like this one:

In function `init_gui’:
gui.c:(.text+0×21): undefined reference to `please_use_vp_os_malloc’

We need to update the Makefile to compile our file with the rest of the SDK. Edit sdk_demo/Build/Makefile and find these lines:

GENERIC_BINARIES_COMMON_SOURCE_FILES+=			\
   UI/ui.c  \
   UI/gamepad.c \
   Navdata/navdata.c    \
   Video/video_stage.c

Add a reference to our files, so the lines now looks like this:

GENERIC_BINARIES_COMMON_SOURCE_FILES+=			\
   UI/ui.c  \
   UI/gui.c \
   UI/gamepad.c \
   Navdata/navdata.c    \
   Video/video_stage.c

Step 2: creating the GUI thread

Now that we have the code to create the GUI, we need to call it. Parrot’s SDK includes a way to create threads with macros. We will use it to create a thread dedicated to our user interface. This work is done in ardrone_testing_tool.c.

The first modification is to include the GUI header. Add the following include:

#include "UI/gui.h"

Then we have to define our thread function, using specific macros. After the include, add this code:

DEFINE_THREAD_ROUTINE(gui, data) /* gui is the routine's name */
{
  gdk_threads_enter();
  gtk_main();
  gdk_threads_leave();
}

Then, we need to customize the ardrone_tool_init_custom function. We add the following lines:

  init_gui(argc, argv); /* Creating the GUI */
  START_THREAD(gui, NULL); /* Starting the GUI thread */

We also custom the ardrone_tool_shutdown_custom function. We need to add:

  JOIN_THREAD(gui);

Finally, we have to add our thread in the Thread Table.
We need to add

THREAD_TABLE_ENTRY(gui, 20)

in the following block:

BEGIN_THREAD_TABLE
  THREAD_TABLE_ENTRY( ardrone_control, 20 )
  THREAD_TABLE_ENTRY( navdata_update, 20 )
  THREAD_TABLE_ENTRY( video_stage, 20 )
END_THREAD_TABLE

More information about the thread management can be found in the Developer’s Guide.

Displaying the camera images

Every time the drone transmits to the computer an image, the output_gtk_stage_transform function in Video/video_stage.c is called.

The function I developed is the following one:

C_RESULT output_gtk_stage_transform( void *cfg, vp_api_io_data_t *in, vp_api_io_data_t *out)
{
  vp_os_mutex_lock(&video_update_lock);
  // Get a reference to the last decoded picture
  pixbuf_data      = (uint8_t*)in->buffers[0];
  vp_os_mutex_unlock(&video_update_lock);

  gdk_threads_enter();
  // GdkPixbuf structure to store the displayed picture
  static GdkPixbuf *pixbuf = NULL;

  if(pixbuf!=NULL)
    {
      g_object_unref(pixbuf);
      pixbuf=NULL;
    }

  // Creating the GdkPixbuf from the transmited data
  pixbuf = gdk_pixbuf_new_from_data(pixbuf_data,
				    GDK_COLORSPACE_RGB,
				    FALSE,   // No alpha channel
				    8,       // 8 bits per pixel
				    320,     // Image width
				    288,     // Image height
				    320 * 3, // New pixel every 3 bytes (3channel per pixel)
				    NULL,    // Function pointers
				    NULL);

  gui_t *gui = get_gui();
  if (gui && gui->cam) // Displaying the image
    gtk_image_set_from_pixbuf(GTK_IMAGE(gui->cam), pixbuf);
  gdk_threads_leave();

  return (SUCCESS);
}

Drone’s take off and landing

The two buttons in the graphical interface are currently binded to empty functions. We need to go back to UI/gui.c and edit the buttons_callback function.We use the following code:

static void buttons_callback(GtkWidget *widget,
                             gpointer   data )
{
    static int value = 1;
    ardrone_tool_set_ui_pad_start(value);
    if (value)
      g_print("Taking off");
    else
      g_print("Landing");
    value = (value + 1) % 2;
    toggleButtonsState(); // We want only one button to be clickable
}

We need to include the file containing ardrone_tool_set_ui_pad_start prototype:

#include <ardrone_tool/UI/ardrone_input.h>

The ardrone_tool_set_ui_pad_start(value) is the AR.Drone’s function used to take off or land the drone. If value is 1 the drone takes off, and lands is value is 0.

Now that all these modifications has been done, we have an application with buttons to take off and land, able to display the drone’s video.

  • Alberto Martín

    you are my savior!! Fantastic tutorial!! Is perfect for me, Thank you

    Regards from Spain

  • Alessandro Benini

    Hi,
    I’m trying to do these modifications, but I get the following errors (video_stage.c):

    cc Video/video_stage.c
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c: In function ‘output_gtk_stage_transform’:
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:80: warning: implicit declaration of function ‘gdk_threads_enter’
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:82: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:82: error: ‘pixbuf’ undeclared (first use in this function)
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:82: error: (Each undeclared identifier is reported only once
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:82: error: for each function it appears in.)
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:86: warning: implicit declaration of function ‘g_object_unref’
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:91: warning: implicit declaration of function ‘gdk_pixbuf_new_from_data’
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:92: error: ‘GDK_COLORSPACE_RGB’ undeclared (first use in this function)
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:101: error: ‘gui_t’ undeclared (first use in this function)
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:101: error: ‘gui’ undeclared (first use in this function)
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:101: warning: implicit declaration of function ‘get_gui’
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:103: warning: implicit declaration of function ‘gtk_image_set_from_pixbuf’
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:103: warning: implicit declaration of function ‘GTK_IMAGE’
    /home/alex/Desktop/ARDrone_OpenCV/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:104: warning: implicit declaration of function ‘gdk_threads_leave’
    make[3]: *** [../../Soft/Build/targets_versions/linux_sdk_demo_PROD_MODE_ffmpeg_Intel_Linux_2.6.32-33-generic_GNU_Linux_gcc_4.4.3/Video/video_stage.o] Error 1
    make[2]: *** [all] Error 2
    make[1]: *** [build_app] Error 2
    make: *** [linux_sdk_demo] Error 2

    I think the problem is gdk_threads_enter() but I’m not able to fix it. Can you help me?

    Regards

    Alex

  • Anonymous

    It might be the problem described here: http://gauth.fr/2011/09/introduction-to-the-ar-drone-sdk/ (error undefined reference to symbol ‘some_gtk_function’).

  • Alessandro Benini

    Hi Gauth69,
    first of all, thank you for your fast reply!

    I recompiled the code with the gtk flag that you suggested but it doesn’t still work. I tried to compile the original demos (with and without gui) and they work. Can you help me, please?

    Best Regards

    Alessandro

  • Alessandro Benini

    I’ve just overwrote the output_gtk_stage_transform function

  • Anonymous

    I don’t have that much idea… aren’t you missing some includes? You need to include gdk/gdk.h

  • Mcives

    Hi Gauth

    Firstly thanks for all of these walk throughs.

    I am currently getting alot of errors and I think it is due to a lack of gdk.h I was wondering if you know where this is or how to create it?

    Many Thanks

  • Anonymous

    You can use commands like find or locate, for instance:
    find /usr -name ‘gdk.h’

  • Mcives

    Thanks for the fast reply

    I wasn’t looking at a high enough level

  • Andres Silva

    Hi, when I was compiling your example was presented this error:

    cc Video/video_stage.c
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c: In function ‘output_gtk_stage_transform’:
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:68: warning: implicit declaration of function ‘gdk_threads_enter’
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:70: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:70: error: ‘pixbuf’ undeclared (first use in this function)
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:70: error: (Each undeclared identifier is reported only once
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:70: error: for each function it appears in.)
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:74: warning: implicit declaration of function ‘g_object_unref’
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:79: warning: implicit declaration of function ‘gdk_pixbuf_new_from_data’
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:80: error: ‘GDK_COLORSPACE_RGB’ undeclared (first use in this function)
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:89: error: ‘gui_t’ undeclared (first use in this function)
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:89: error: ‘gui’ undeclared (first use in this function)
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:89: warning: implicit declaration of function ‘get_gui’
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:91: warning: implicit declaration of function ‘gtk_image_set_from_pixbuf’
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:91: warning: implicit declaration of function ‘GTK_IMAGE’
    /home/andres/AF/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:92: warning: implicit declaration of function ‘gdk_threads_leave’
    make[3]: *** [../../Soft/Build/targets_versions/linux_sdk_demo_PROD_MODE_ffmpeg_Intel_Linux_2.6.35-32-generic-pae_GNU_Linux_gcc_4.4.5/Video/video_stage.o] Error 1
    make[2]: *** [all] Error 2
    make[1]: *** [build_app] Error 2
    make: *** [linux_sdk_demo] Error 2
    andres@felipe:~/AF/Examples/Linux/sdk_demo/Build$

    ¿Can you help me?

  • hannafi marouan

    Thank you for your efforts, i have the same problem i included #include in video_stage.c but the the type get_t is always unknown, the problem is in this line: gui_t *gui = get_gui(); inthe file video_stage.c.

    This is my output after doing a make in Examples/Linux/sdk_demo.
    ——————————————————————–
    Checking required Ubuntu packages …
    Building ARDroneTool/Lib
    cc Video/video_stage.c
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c: In function ‘output_gtk_stage_transform’:
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:91:3: error: unknown type name ‘gui_t’
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:91:3: warning: implicit declaration of function ‘get_gui’ [-Wimplicit-function-declaration]
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:91:16: warning: initialization makes pointer from integer without a cast [enabled by default]
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:92:17: error: request for member ‘cam’ in something not a structure or union
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:93:5: warning: implicit declaration of function ‘gtk_image_set_from_pixbuf’ [-Wimplicit-function-declaration]
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:93:5: warning: implicit declaration of function ‘GTK_IMAGE’ [-Wimplicit-function-declaration]
    /home/hannouf/Bureau/ARDrone_SDK_Version_1_8_20110726/Examples/Linux/sdk_demo/Build/../Sources/Video/video_stage.c:93:44: error: request for member ‘cam’ in something not a structure or union
    make[3]: *** [../../Soft/Build/targets_versions/linux_sdk_demo_PROD_MODE_vlib_Linux_3.2.0-23-generic_GNU_Linux_gcc_4.6.3/Video/video_stage.o] Error 1
    make[2]: *** [all] Error 2
    make[1]: *** [build_app] Error 2
    make: *** [linux_sdk_demo] Error 2
    ——————————————————–
    can you help me please ?

  • Hannafi Marouan

    It’s okay i found the problem thank you anyway !!

  • Jason Tuskanov

    Hi,

    First, thank you for this great tutorial. It is much more clear than Parrot’s user guide. Unfortunately, it doesn’t seem to work with Ardrone2 and SDK 2.0

    The gui window contains only 2 buttons (no image from the camera), and the function output_gtk_stage_transform() is never called.

    Do other people experience the same problem ?
    Do you have any idea why this problem occurs ?

  • Darshan

    Hey Hannafi,
    I am trying to run the same GUI app but while compiling I am getting the error:

    No rule to make target `../../Soft/Build/targets_versions/linux_sdk_demo_PROD_MODE_vlib_Linux_3.2.0-29-generic-pae_GNU_Linux_usrbingcc_4.6.3/UI/ui.c.d’, needed by `all’. Stop.

    Can you help me find that error since i noticed you had solved the issue. Thank you

  • Holymoo

    I’m also using Ardrone2 and SDK 2.0. Unfortunately, I can’t get it to compile at all. It appears the problem stems from the fact that some new functions are missing the argc and argv parameters.

  • Hannafi Marouan

    hey Darshan,
    i forgot how i did it may be you must chmod 777 video_stage.c or .h but i’m not sure.

  • likith

    sir,can you please make a video for creating graphical interface for sdk_2_0_1,so that i can have a clear idea…

  • HawxOne97

    I am getting the same error and do not understand how to fix it. Can someone help?

  • Malabika Sarbadhikary

    Libs already extracted
    Building target static
    Architecture i686 is already built
    Creating universal static lib file from architectures i686
    Build done.
    33[31mChecking required Ubuntu packages ...33[0m
    ok.
    33[32;01mBuilding ARDroneTool/Lib33[0m
    33[32;01mBuilding ARDroneTool/Lib33[0m
    cc ardrone_testing_tool.c
    In file included from /home/anirban/Desktop/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/ardrone_testing_tool.c:14:0:
    /*********************/Desktop/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/UI/gui.c: In function ‘init_gui’:
    /*********************/Desktop/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/UI/gui.c:47:3: warning: ‘g_thread_init’ is deprecated (declared at /usr/include/glib-2.0/glib/deprecated/gthread.h:260) [-Wdeprecated-declarations]
    /***********************/Desktop/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/ardrone_testing_tool.c: In function ‘ardrone_tool_init_custom’:
    /*******************/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/ardrone_testing_tool.c:55:13: error: ‘argc’ undeclared (first use in this function)
    ********************/Desktop/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/ardrone_testing_tool.c:55:13: note: each undeclared identifier is reported only once for each function it appears in
    /home/anirban/Desktop/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/ardrone_testing_tool.c:55:18: error: ‘argv’ undeclared (first use in this function)
    ******************/ARDrone_SDK_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/ardrone_testing_tool.c: In function ‘thread_gui’:
    *************ARDrone_SDK/_2_0_1/Examples/Linux/sdk_demo/Build/../Sources/ardrone_testing_tool.c:37:1: warning: control reaches end of non-void function [-Wreturn-type]
    make[3]: *** [../../Soft/Build/targets_versions/linux_sdk_demo_PROD_MODE_vlib_Linux_3.8.0-26-generic_GNU_Linux_usrbingcc_4.7.3/ardrone_testing_tool.o] Error 1
    make[2]: *** [all] Error 2
    make[1]: *** [build_app] Error 2
    make: *** [linux_sdk_demo] Error 2

    I’m getting this error. Please help me to fix it.

  • Anand

    I am usind AR.Drone SDK 2.0.
    I have no idea about “ardrone_tool_set_ui_pad_start”.
    where to include these lines

    #include

  • Anand

    I have found the solution.

  • lingowei

    me too ,
    it is no image,and something different from API2.0

  • lintong

    Hi. Could anyone please help? When I compiled the sdk_demo. The error appears at this line: init_gui(argc, argv); /* Creating the GUI */

    error says argc and argv is not declared.

  • lintong

    yes I have the same problem! The error appears at this line: init_gui(argc, argv); /* Creating the GUI */

    did you slove it?

  • phodrone

    Why the video doesn’t work for me ? I’m testing your C files.

  • Ameer Hamza

    have you found the solution?

  • Ameer Hamza

    Can you explain how to create the thread function? I want to make a simple thread in which i want to give a fix value of altitude? can you explain how to define thread function?

  • Martin

    Were you able to find a solution?