Technical Blog

How to spider a HTML page with jQuery

There are a lot of cases where you would like to parse HTML pages from your JS scripts to extract data. jQuery provides functions to help to do so.

Getting the page content

We use the jQuery.get method. It looks like an asynchronous php’s file_get_contents function.
It works this way:

$.get(url, function (data) {
  // data is the content of the URL.
});
Same Origin Policy

The Same Origin Policy makes that if you do $.get with an url different from your server, you will get the following error:

XMLHttpRequest cannot load http://www.some_page.com. Origin null is not allowed by Access-Control-Allow-Origin

To make it work, the trick is to use a proxy. Create the following php file:

<?php
header('Access-Control-Allow-Origin: *');

if (isset($_GET['url']) && preg_match('`^http://`', $_GET['url'])) {
   echo file_get_contents($_GET['url']);
}
?>

Then, instead of doing

$.get("http://www.example.com", ...

use it this way:

$.get("http://www.my_website/path/to/proxy.php?url=http://www.example.com", ...

The proxy file is not required to be on the same server as your javascript file.

Parsing the data

With jQuery

We can build a jQuery structure from the downloaded page:

$.get(url, function (data) {
  $page = $(data);
})

Then, we can obtain all the data we want using jQuery selectors and methods. For example, to get the text from all the cells with the class “example”:

var res = [];
$("td.example", $page).each(function(){
  res.push($(this).text());
});

However, this technique has a big disadvantage. If the HTML downloaded by $.get contains error, It might fail to build the structure, and we won’t able to use the selectors.

If the downloaded page contains scripts or stylesheet, they will be downloaded and executed.

With regular expressions

The second possibility is to use regular expressions, as we get the page’s data as a string. We can extract the text from the cells with the class “example” this way:

res = data.match(/<td.*?class=["'][^"']*example(\s|["'])[^>]*>(.*?)</td>/gim)[2];

We can also mix both techniques: building a jQuery structure from a part of the page matched by a regex, or using a regex in a jQuery node.

Conclusion

This article should get you started in order to spider pages directly from the browser. It is particularly helpful to build extensions for browsers (Safari, Chrome, Firefox) or with add-ons like GreaseMonkey. It allows to aggregate data from different sources to build more powerful application.


Get a color name from any RGB combination

The aim of this script is to get a color name from any RGB combination. There are over 16 million combinations, so we cannot have one name for each combination. However, we can get the nearest known color.

Demo

Input
Closest

The color picker comes from David Durman’s website.

How it works

The hexadecimal value of a color represent a three-coordinates point: Red, Green and Blue. The first thing we need is a data set. I use this list from Wikipedia. I parsed it with jQuery, and built an array of labels points (r, g, b and color name) that I serialized in JSON. I finally put it in a seperate file.

The second step is to classify the color. We put this new point in our 3D space, and locate the closest labeled point. This algorithm is called kNN (k Nearest Neighbor). Here we use k=1, as we only have on element per class (color name).

Possible evolutions

The results of this technique are impacted by two main factors:

  • The dataset
  • The metric

We can change the dataset to modify the results. We can reduce it, to only use generic color names (green, brown, yellow etc.), instead of precise names (Blizzard Blue, Pakistan green, Stil de grain yellow, etc.). We can also add more data, so the density of points in the space will be higher, and the results more accurate. A lot of lists are available on the web.

Instead of using a lot of exotic color names, a solution could be to use several points for each label (color name). For instance, we could use a limited set of color names (red, green, blue, etc) but have a lot of points associated to these labels. For example, different points like #07250b (that is, in my opinion, missclassified) and #51f665 could share a same label: green. The problem is to have the dataset. We could build one from pages like this one, removing the numbers from each color name, so multiple points share the same name.
With several points for each label, It also will give us the possibility of using a 3NN, for example, instead of a 1NN classifier. It should impact also on the results, but I’m not sure it will really improve them.

The metric, or distance function, is the way we compute the distance between two points. In this technique, I use the euclidian metric, but a lot of other distances exists. Articles can be found about color metrics.

One possibility is to change our 3D space. Here we use the RGB color space, but we could also use different spaces like CMYK, HSV etc.

Finally, we could also get more information about our classification process. We could display the decision function. Instead of displaying all the existing colors in the picker, we could display for each pixel the color of the closest labeled point. It would be a good way to see the dataset density, and detect zones where the density is to low.

How to use it

You need first to download color_classifier.js. It requires jQuery to load the dataset, but you can avoid it by moving dataset.js content in color_classifier.js.
The first step is to build the classifier:

window.classifier = new ColorClassifier();
get_dataset('dataset.js', function (data){
    window.classifier.learn(data);
});

Then we can classify colors:

var result_name = window.classifier.classify("#aaf000");

Control the AR.Drone LEDs

The AR.Drone has four LEDs, one per rotor.

Currently, we can only use predefined animations (around 20), but maybe we will be able to use user-defined sequences in a future version of the SDK.

Which function to use?

We use the ardrone_at_set_led_animation function to control the LEDs. The prototype is defined in ARDroneLib/Soft/Common/ardrone_api.h:

void ardrone_at_set_led_animation (
    LED_ANIMATION_IDS anim_id,
    float32_t freq,
    uint32_t duration_sec);
  • anim_id: set to one of the animations defined by the SDK. It is detailed further down in this article.
  • freq: the frequency (hertz) of the animation, or the inverse of the period (seconds). For example, if it is set to 0.25, the period is 4: the animation will be completed in 4 seconds. The higher the value is, the faster is the animation.
  • duration_sec: the duration in seconds. If its value is superior to the time of execution of the animation, it will loop. If the value is set to 0, the animation will loop infinitely.

The different animations

The anim_id is one of the values defined in ARDroneLib/Soft/Common/led_animation.h. They are several kinds of animations:

Animation’s Name Display
BLINK_GREEN_RED
BLINK_GREEN
BLINK_RED
BLINK_ORANGE
SNAKE_GREEN_RED
FIRE
STANDARD
RED
GREEN
RED_SNAKE
BLANK
Animation’s Name Display
RIGHT_MISSILE
LEFT_MISSILE
DOUBLE_MISSILE
FRONT_LEFT_GREEN_OTHERS_RED
FRONT_RIGHT_GREEN_OTHERS_RED
REAR_RIGHT_GREEN_OTHERS_RED
REAR_LEFT_GREEN_OTHERS_RED
LEFT_GREEN_RIGHT_RED
LEFT_RED_RIGHT_GREEN
BLINK_STANDARD

Understanding the led_animation.h file

The file ARDroneLib/Soft/Common/led_animation.h contains the declaration of the different animations. It works like a kind of enum: each identifier refers to an ID that will be transmitted to the drone. Then, it will play the animation settings associated to this ID. Therefore, we can’t define our own sequences.
For example:

LED_ANIMATION(BLINK_STANDARD, {0,2,{{0x00,500},{0xA5,500}}})

The LED_ANIMATION macro is explained in the file:

LED_ANIMATION(#name, {#nb_cycle,#nb_state,{{#led_pattern1,#delay1},{#led_pattern2,#delay2},{...,...}}})
#name = name, example : BLINK
#nb_cycle = number of times the animation is played (0 means infinite), example : 3
#nb_state = number of led patterns in the animation, example : 2
#led_pattern = led bitfield (G1 | R1 | G2 | R2 | G3 | R3 | G4 | R4), example : 0xAA all green led turned on
#delay = delay in ms for the associated led pattern, example : 500

In our example, the animation’s ID is BLINK_STANDARD. It loops infinitely and contains two states: 0×00 during 0.5 seconds, and 0xA5 during 0.5 seconds.
0xA5 is hexadecimal code. Its binary form is: 1010 0101. It is mapped this way:

1 0 1 0 0 1 0 1
G1 R1 G2 R2 G3 R3 G4 R4

Therefore, we can see that:

  • LEDs 1 and 2 are green
  • LEDs 3 and 4 are red

The results will be this:

The same way, in the following state, 0×00, all the lights are turned off.

To use this animation, we use the following code:

ardrone_at_set_led_animation(BLINK_STANDARD, 0.25, 6);

In this example, all the LEDs will switch between the two states every two seconds (period of 4 seconds) during 6 seconds.

Appendix: list of the animations

The list can be found in the following file: ARDroneLib/Soft/Common/led_animation.h

LED_ANIMATION(BLINK_GREEN_RED,               { 0,2, { {0x55,500},{0xAA,500} } } )
LED_ANIMATION(BLINK_GREEN,                   { 0,2, { {0x00,500},{0xAA,500} } } )
LED_ANIMATION(BLINK_RED,                     { 0,2, { {0x55,500},{0x00,500} } } )
LED_ANIMATION(BLINK_ORANGE,                  { 0,2, { {0xFF,500},{0x00,500} } } )
LED_ANIMATION(SNAKE_GREEN_RED,               { 0,8, { {0x90,200},{0x48,200},{0x24,200},{0x12,200},{0x9,200},{0x84,200},{0x42,200},{0x21,200}}})
LED_ANIMATION(FIRE,                          { 0,2, { {0x35,50},{0xC5,50} } } )
LED_ANIMATION(STANDARD,                      { 1,1, { {0xA5,100} } } )
LED_ANIMATION(RED,                           { 1,1, { {0x55,100} } } )
LED_ANIMATION(GREEN,                         { 1,1, { {0xAA,100} } } )
LED_ANIMATION(RED_SNAKE,                     { 0,4, { {0x40,500},{0x10,500},{0x04,500},{0x01,500}}})
LED_ANIMATION(BLANK,                         { 1,1, { {0x00,100} } } )
LED_ANIMATION(RIGHT_MISSILE,                 { 1,5, { {0x00,500},{0x04,300},{0x1C,100},{0x30,300},{0x00,500}}})
LED_ANIMATION(LEFT_MISSILE,                  { 1,5, { {0x00,500},{0x01,300},{0x43,100},{0xC0,300},{0x00,500}}})
LED_ANIMATION(DOUBLE_MISSILE,                { 1,5, { {0x00,500},{0x05,300},{0x5F,100},{0xF0,300},{0x00,500}}})
LED_ANIMATION(FRONT_LEFT_GREEN_OTHERS_RED,   { 1,1, { {0x95,100} } } )
LED_ANIMATION(FRONT_RIGHT_GREEN_OTHERS_RED,  { 1,1, { {0x65,100} } } )
LED_ANIMATION(REAR_RIGHT_GREEN_OTHERS_RED,   { 1,1, { {0x59,100} } } )
LED_ANIMATION(REAR_LEFT_GREEN_OTHERS_RED,    { 1,1, { {0x56,100} } } )
LED_ANIMATION(LEFT_GREEN_RIGHT_RED,          { 1,1, { {0x96,100} } } )
LED_ANIMATION(LEFT_RED_RIGHT_GREEN,          { 1,1, { {0x69,100} } } )
LED_ANIMATION(BLINK_STANDARD,                { 0,2, { {0x00,500},{0xA5,500} } } )


Create a video with the AR.Drone

The AR.Drone is an efficient source of images: it can fly, being remotely controlled, etc. We will see in this article how to create avi files from its cameras. We will use OpenCV to create it, so you’ll probably need first to take a look at Use OpenCV with the AR.Drone SDK.

OpenCV code

We will use the CvVideoWriter structure to build our avi file.
Firstly, we need a function to initialize it.

CvVideoWriter *init_video_writer(char *fname)
{
  int isColor = 1;
  int fps     = 30;
  int frameW = 320;
  int frameH = 240;
  return cvCreateVideoWriter(fname, // with avi extension
                             CV_FOURCC('D', 'I', 'V', 'X'), //MPEG4
                             fps,
                             cvSize(frameW,frameH),
                             isColor);
}

This feature is handled in my project with a button. I added to functions that are call by the button’s callback :

static CvVideoWriter *video_writer = 0;
void init_video(void)
{
  video_writer = init_video_writer();
}

void stop_video(void)
{
  // Necessary to have a valid avi file
  cvReleaseVideoWriter(&video_writer);
}

I added a function to add a frame to the video:

inline void add_frame(IplImage *img)
{
  if (video_writer)
    cvWriteFrame(video_writer, img);
}

Finally, we need to call the add_frame function every time we receive a new frame from the drone. I added it in the output_gtk_stage_transform function in Video/video_stage.c.
Underneath the code creating the OpenCV image, I added

if (/* video saving is enabled by the user */)
    add_frame(img);

Handling different frame rates

The AR.Drone has two cameras, using two different frame rates:

  • The frontal camera has 15 FPS
  • The vertical camera has 60 FPS

In the previous code, video was created with 30FPS. Therefore, one camera will look to slow, and the other one to fast. Therefore, we can update the function this way:

CvVideoWriter *init_video_writer(char *fname, int fps)
{
  int isColor = 1;
  int fps     = fps;
  int frameW = 320;
  int frameH = 240;
  return cvCreateVideoWriter(fname, // with avi extension
                             CV_FOURCC('D', 'I', 'V', 'X'), // //MPEG4
                             fps,
                             cvSize(frameW,frameH),
                             isColor);
}

Then, we can use two ways to call it:

video_writer = init_video_writer("out_horizontal.avi", 15);

or

video_writer = init_video_writer("out_vertical.avi", 60);

Refer to this page for more information about possible codecs.


Use OpenCV with the AR.Drone SDK

OpenCV (Open Source Computer Vision Library) is a powerful image processing library. I will detail in this post how to use it with the AR.Drone’s C SDK.

Compiling the AR.Drone SDK with OpenCV

The first step is to install OpenCV. If you’re using Ubuntu, you may refer to this page.

Once the library is installed, we need to modify the Makefile to add the correct flags. We will edit sdk_demo/Build/Makefile.

To add the correct cflags, find the line:

GENERIC_INCLUDES:=$(addprefix -I,$(GENERIC_INCLUDES))

and add underneath:

GENERIC_INCLUDES += `pkg-config --cflags opencv` 

To add the correct libraries, change the following line:

GENERIC_LIBS=-lpc_ardrone -lgtk-x11-2.0 -lrt

to

GENERIC_LIBS=-lpc_ardrone -lgtk-x11-2.0 -lrt `pkg-config --libs opencv`

Creating an OpenCV image from the drone’s image

We need to update the output_gtk_stage_transform in Video/video_stage.c to transform the data received from the drone to an IplImage, the OpenCV image structure. First, we need to add some includes:

#include "cv.h"
#include "highgui.h" // if you want to display images with OpenCV functions

We will use a method close to what we did to create a GdkPixbuf:

IplImage *ipl_image_from_data(uint8_t* data)
{
  IplImage *currframe;
  IplImage *dst;

  currframe = cvCreateImage(cvSize(320,240), IPL_DEPTH_8U, 3);
  dst = cvCreateImage(cvSize(320,240), IPL_DEPTH_8U, 3);

  currframe->imageData = data;
  cvCvtColor(currframe, dst, CV_BGR2RGB);
  cvReleaseImage(&currframe);
  return dst;
}

We call it from output_gtk_stage_transform in Video/video_stage.c:

IplImage *img = ipl_image_from_data((uint8_t*)in->buffers[0], 1);

Vertical camera handling

As detailed in a previous article, the images captured with the vertical camera has a lower size than the horizontal camera. The data transmitted has the same size in both cases, but with empty pixels. I updated the ipl_image_from_data:

IplImage *ipl_image_from_data(uint8_t* data, int reduced_image)
{
  IplImage *currframe;
  IplImage *dst;

  if (!reduced_image)
  {
    currframe = cvCreateImage(cvSize(320,240), IPL_DEPTH_8U, 3);
    dst = cvCreateImage(cvSize(320,240), IPL_DEPTH_8U, 3);
  }
  else
  {
    currframe = cvCreateImage(cvSize(176, 144), IPL_DEPTH_8U, 3);
    dst = cvCreateImage(cvSize(176,144), IPL_DEPTH_8U, 3);
    currframe->widthStep = 320*3;
  }

  currframe->imageData = data;
  cvCvtColor(currframe, dst, CV_BGR2RGB);
  cvReleaseImage(&currframe);
  return dst;
}

The trick is the same as detailed in the previous article. We set that each new line starts every 320*3 bytes, but we only use 176*3 byes per line.

Converting OpenCV images to GdkPixbuf

If you’re using a GTK interface as detailed in previous articles, you may want to display the OpenCV image inside your GTK Window. To do this, I use the following function to create a GdkPixbuf structure that can be displayed by GTK:

GdkPixbuf* pixbuf_from_opencv(IplImage *img, int resize)
{
  IplImage* converted = cvCreateImage(cvSize(img->width, img->height), IPL_DEPTH_8U, 3);
  cvCvtColor(img, converted, CV_BGR2RGB);

  GdkPixbuf* res = gdk_pixbuf_new_from_data(converted->imageData,
                                           GDK_COLORSPACE_RGB,
                                           FALSE,
                                           8,
                                           converted->width,
                                           converted->height,
                                           converted->widthStep,
                                           NULL,
                                           NULL);
  if (resize)
    res = gdk_pixbuf_scale_simple(res, 320, 240, GDK_INTERP_BILINEAR);

  return res;
}

Switch camera with the AR.Drone SDK

The AR.Drone has two cameras, but we can only use one at a time. Therefore, we can need to change the camera whose images are sent by the drone.

The different channels

The different channels are declared in ARDroneLib/Soft/Common/ardrone_api.h with the following enum:

enum ZAP_VIDEO_CHANNEL {
  ZAP_CHANNEL_FIRST = 0,
  ZAP_CHANNEL_HORI = ZAP_CHANNEL_FIRST,
  ZAP_CHANNEL_VERT,
  ZAP_CHANNEL_LARGE_HORI_SMALL_VERT,
  ZAP_CHANNEL_LARGE_VERT_SMALL_HORI,
  ZAP_CHANNEL_LAST = ZAP_CHANNEL_LARGE_VERT_SMALL_HORI,
  ZAP_CHANNEL_NEXT
}

They are 4 main values to select the video stream:
ZAP_CHANNEL_HORI

ZAP_CHANNEL_VERT

ZAP_CHANNEL_LARGE_HORI_SMALL_VERT

ZAP_CHANNEL_LARGE_VERT_SMALL_HORI

Note: images come from the Doc folder of the AR.Drone SDK.

Changing the camera

We use the following code to select the camera:

ZAP_VIDEO_CHANNEL channel = /* some channel, ex: ZAP_CHANNEL_LARGE_HORI_SMALL_VERT*/;
ARDRONE_TOOL_CONFIGURATION_ADDEVENT (video_channel, &channel, cb);

The cb parameter

cb is a pointer to a function called when the command is executed. It is defined in the Developer’s Guide:

The callback function type is void (*callBack)(unsigned int success). The configuration tool will call the callback function after any attempt to set the configuration, with zero as the parameter in case of failure, and one in case of success. In case of failure, the tool will automatically retry after an amount of time.

Handling the different resolutions

The vertical camera has a smaller resolution: width and height are only half of the horizontal (frontal) camera. Nevertheless, the data transmitted by the drone have the same size. Therefore, if you display the image, only a quarter of the image will be the camera’s image, and the rest will be green pixels. To create a Pixbuf structure containing only the good pixels, use the following code:

buf = gdk_pixbuf_new_from_data(pixbuf_data,
                               GDK_COLORSPACE_RGB,
                               FALSE,
                               8,
                               176,
                               144,
                               320 * 3,
                               NULL,
                               NULL);

This trick is to give as parameter the correct reduced width of 176 pixels, but to indicate that each new line starts after 320 * 3 pixels, so it will skip useless pixels. The height parameter is also reduced to 144, so the bottom of the image, filled with green pixels, is ignored.

Next, you can enlarge the image to have the same size as the horizontal camera:

pixbuf = gdk_pixbuf_scale_simple (pixbuf,
                                  320,
                                  240,
                                  GDK_INTERP_BILINEAR) ;

Create an AR.Drone graphical application

This article is a tutorial to learn the basics of the ARDrone SDK.

It shows the steps to create a minimalist graphical application, with the following features :

  • Display of the drone’s video
  • Buttons to take off / land

If you’re not familiar with the AR.Drone SDK, you should read first my previous Introduction to the AR.Drone SDK.

The AR.Drone Tool

The AR.Drone Tool is the easiest way to create an AR.Drone application on the Linux platform. Parrot provides a basic project skeleton, including all the code needed to initialize the system, gather video and other kind of information transmitted by the drone.

Those files are located in the directory Examples/Linux/sdk_demo/Sources. The files are the following:

  • ardrone_testing_tool.[ch]: the initialization code.
  • Navadata/navdata.[ch]: the navigation data (gyroscope and altimeter information, battery level etc.)
  • Video/video_stage.[ch]: handling of the video stream.
  • UI/gamepad.[ch]: handling of the joystick.
  • UI/ui.[ch]: unknown! (almost empty file).

To create an application, the main task is to customize several functions:

  • ardrone_testing_tool.c: ardrone_tool_init_custom() is called at the initialization of the application, and ardrone_tool_shutdown_custom() when leaving.
  • Navdata/navdata.c: demo_navdata_client_process() is called every time the drone transmits navigation data.
  • Video/video_stage.c: output_gtk_stage_transform() is called every time an image acquired by the drone is received and decoded.

Step 1: Creating the user interface

In this article, I will use GTK. The build system provided by Parrot is designed to use GTK, and the required flags are already present.

In this first step, we will simply create a basic interface with three main widgets:

  • An image widget to display the camera’s video
  • A button to take off
  • A button to land

I created two new files: UI/gui.c and UI/gui.h (click to expand).

#ifndef GUI_H_
# define GUI_H_

# include <gtk/gtk.h>
typedef struct gui
{
  GtkWidget *window;
  GtkWidget *start;
  GtkWidget *stop;
  GtkWidget *box;
  GtkWidget *cam;
} gui_t;

gui_t *get_gui();

void init_gui(int argc, char **argv);

#endif
#include <stdlib.h>
#include "gui.h"

gui_t *gui = NULL;

gui_t *get_gui()
{
  return gui;
}

/* If the drone is landed, only start is clickable,
   if the drone is in the air, only stop is clickable
*/
static void toggleButtonsState(void)
{
  gboolean start_state = gtk_widget_get_sensitive(gui->start);

  gtk_widget_set_sensitive(gui->start, !start_state);
  gtk_widget_set_sensitive(gui->stop, start_state);
}

static void buttons_callback( GtkWidget *widget,
			      gpointer   data )
{
    // FIXME: make the drone start
}

static void on_destroy(GtkWidget *widget, gpointer data)
{
  vp_os_free(gui);
  gtk_main_quit();
}

void init_gui(int argc, char **argv)
{
  gui = vp_os_malloc(sizeof (gui_t));

  g_thread_init(NULL);
  gdk_threads_init();
  gtk_init(&argc, &argv);

  gui->window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
  g_signal_connect(G_OBJECT(gui->window),
		   "destroy",
		   G_CALLBACK(on_destroy),
		   NULL);
  gui->box = gtk_vbox_new(FALSE, 10);
  gtk_container_add(GTK_CONTAINER(gui->window),
		    gui->box);
  gui->cam = gtk_image_new();
  gtk_box_pack_start(GTK_BOX(gui->box), gui->cam, FALSE, TRUE, 0);

  gui->start = gtk_button_new_with_label("Start");
  g_signal_connect (gui->start, "clicked",
		      G_CALLBACK (buttons_callback), NULL);
  gui->stop = gtk_button_new_with_label("Stop");
  g_signal_connect (gui->stop, "clicked",
		      G_CALLBACK (buttons_callback), NULL);
	gtk_widget_set_sensitive(gui->start, TRUE);
  gtk_widget_set_sensitive(gui->stop, FALSE);

  gtk_box_pack_start(GTK_BOX(gui->box), gui->start, TRUE, TRUE, 0);
  gtk_box_pack_start(GTK_BOX(gui->box), gui->stop, TRUE, TRUE, 0);

  gtk_widget_show_all(gui->window);
}

You may have noticed that I am using vp_os_malloc() and vp_os_free() instead of the usual standard library functions. It is required by the SDK. If you don’t do so, you will get error messages like this one:

In function `init_gui’:
gui.c:(.text+0×21): undefined reference to `please_use_vp_os_malloc’

We need to update the Makefile to compile our file with the rest of the SDK. Edit sdk_demo/Build/Makefile and find these lines:

GENERIC_BINARIES_COMMON_SOURCE_FILES+=			\
   UI/ui.c  \
   UI/gamepad.c \
   Navdata/navdata.c    \
   Video/video_stage.c

Add a reference to our files, so the lines now looks like this:

GENERIC_BINARIES_COMMON_SOURCE_FILES+=			\
   UI/ui.c  \
   UI/gui.c \
   UI/gamepad.c \
   Navdata/navdata.c    \
   Video/video_stage.c

Step 2: creating the GUI thread

Now that we have the code to create the GUI, we need to call it. Parrot’s SDK includes a way to create threads with macros. We will use it to create a thread dedicated to our user interface. This work is done in ardrone_testing_tool.c.

The first modification is to include the GUI header. Add the following include:

#include "UI/gui.h"

Then we have to define our thread function, using specific macros. After the include, add this code:

DEFINE_THREAD_ROUTINE(gui, data) /* gui is the routine's name */
{
  gdk_threads_enter();
  gtk_main();
  gdk_threads_leave();
}

Then, we need to customize the ardrone_tool_init_custom function. We add the following lines:

  init_gui(argc, argv); /* Creating the GUI */
  START_THREAD(gui, NULL); /* Starting the GUI thread */

We also custom the ardrone_tool_shutdown_custom function. We need to add:

  JOIN_THREAD(gui);

Finally, we have to add our thread in the Thread Table.
We need to add

THREAD_TABLE_ENTRY(gui, 20)

in the following block:

BEGIN_THREAD_TABLE
  THREAD_TABLE_ENTRY( ardrone_control, 20 )
  THREAD_TABLE_ENTRY( navdata_update, 20 )
  THREAD_TABLE_ENTRY( video_stage, 20 )
END_THREAD_TABLE

More information about the thread management can be found in the Developer’s Guide.

Displaying the camera images

Every time the drone transmits to the computer an image, the output_gtk_stage_transform function in Video/video_stage.c is called.

The function I developed is the following one:

C_RESULT output_gtk_stage_transform( void *cfg, vp_api_io_data_t *in, vp_api_io_data_t *out)
{
  vp_os_mutex_lock(&video_update_lock);
  // Get a reference to the last decoded picture
  pixbuf_data      = (uint8_t*)in->buffers[0];
  vp_os_mutex_unlock(&video_update_lock);

  gdk_threads_enter();
  // GdkPixbuf structure to store the displayed picture
  static GdkPixbuf *pixbuf = NULL;

  if(pixbuf!=NULL)
    {
      g_object_unref(pixbuf);
      pixbuf=NULL;
    }

  // Creating the GdkPixbuf from the transmited data
  pixbuf = gdk_pixbuf_new_from_data(pixbuf_data,
				    GDK_COLORSPACE_RGB,
				    FALSE,   // No alpha channel
				    8,       // 8 bits per pixel
				    320,     // Image width
				    288,     // Image height
				    320 * 3, // New pixel every 3 bytes (3channel per pixel)
				    NULL,    // Function pointers
				    NULL);

  gui_t *gui = get_gui();
  if (gui && gui->cam) // Displaying the image
    gtk_image_set_from_pixbuf(GTK_IMAGE(gui->cam), pixbuf);
  gdk_threads_leave();

  return (SUCCESS);
}

Drone’s take off and landing

The two buttons in the graphical interface are currently binded to empty functions. We need to go back to UI/gui.c and edit the buttons_callback function.We use the following code:

static void buttons_callback(GtkWidget *widget,
                             gpointer   data )
{
    static int value = 1;
    ardrone_tool_set_ui_pad_start(value);
    if (value)
      g_print("Taking off");
    else
      g_print("Landing");
    value = (value + 1) % 2;
    toggleButtonsState(); // We want only one button to be clickable
}

We need to include the file containing ardrone_tool_set_ui_pad_start prototype:

#include <ardrone_tool/UI/ardrone_input.h>

The ardrone_tool_set_ui_pad_start(value) is the AR.Drone’s function used to take off or land the drone. If value is 1 the drone takes off, and lands is value is 0.

Now that all these modifications has been done, we have an application with buttons to take off and land, able to display the drone’s video.


Introduction to the AR.Drone SDK

What is the AR.Drone ?

The AR.Drone is a quadricopter created by Parrot. It can be be purchased in stores like Amazon.com at a price of around 300 dollars. It is mainly used as a flying video game, and several games has been released using augmented reality.

The drone has two cameras: one frontal and one vertical. The technical specifications can be found here.
The drone can be controlled using any Wifi device. When the drone is turned on, it automatically creates a ad-hoc wifi. The controlling device connects to the wifi, and communicates with the drone. Parrot developed an application for the iPhone and the Android Phones, named AR.FreeFlight. More applications created by other developers can be found on the AppStore, etc.

The SDK

Parrot released a SDK to help developers creating innovation applications using the drone. The SDK is available for iOS, Android, Linux and Windows. Linux and Windows SDK are using the C programming language. In this article, I will focus on the Linux SDK. The specifications of the communication’s protocol used by the drone is also available. Therefore, it is possible to create a new SDK instead of using Parrot’s one.

The SDK and the Developer’s Guide can be found in the AR.Drone Open API Platform. In this article, I am using the version 1.7. Several ways exist to create an application using the AR.Drone. In the developer guide, Parrot recommends to use the AR.Drone Tool, a framework designed to create easily applications.

SDK structure

The root of the archive contains several files and directory:

  • ARDroneAPI.dox: doxygen file, used to generate the documentation.
  • ARDroneLib: AR.Drone library (communication with the drone, video codecs etc.)
  • ControlEngine: files specific to the iPhone.
  • Docs: folder where the documentation is generated
  • Examples: folder containing demonstration code for each platform and the ARDrone Tool.

The archive contains more files than we need. We can remove the ControlEngine directory, and all the directories in Examples except the Linux one.

Building the examples

The examples are useful for several reasons:

  • To check that the system has all the needed libraries.
  • To check that the computer can communicate with the drone

The first step is to install the missing package. If you’re using Ubuntu, a script was developed to install them automatically. ARDroneLib/Soft/Build/check_dependencies.sh must be executed with the root privilege. OK is displayed when all the packages are installed.
When all the packages are installed, we are ready to compile the libraries and the examples:

cd Examples/Linux
make

If you get the following error message:

In file included from ..//VP_Com/linux/vp_com_serial.c:33:
..//VP_Com/vp_com_serial.h:22: error: expected ‘)’ before ‘*’ token

you need to edit the file ARDroneLib/Soft/Build/custom.makefile and change the line

USE_LINUX=no

to

USE_LINUX=yes

We need to connect to the wifi created by the drone when It is turned on. Its name is ardrone_XXXXX, with XXXXX the ID of the drone.
By default, only two IP addresses are used:

  • 192.168.1.1 is the drone.
  • 192.168.1.2 is the device that communicates with it.

Thefore, when we are connected to the wifi, we need to set our IP address. We can use the following command:

sudo ifconfig wlan0 192.168.1.2 netmask 255.255.255.0

If the computer address is not 192.168.1.2, the drone won’t be able to communicate with the computer.
Several examples have been compiled. The most useful one is Build/Release/ardrone_navigation. This graphical application is able to display all the information transmitted by the drone:

  • Camera’s images
  • Battery status
  • Gyroscopic data

The application is able to control the drone movement using a Joystick. Refer to the Developer’s Guide for more information.

Common troubleshooting

Error:

In file included from ..//VP_Com/linux/vp_com_serial.c:33:
..//VP_Com/vp_com_serial.h:22: error: expected ‘)’ before ‘*’ token

Solution:
Edit the file ARDroneLib/Soft/Build/custom.makefile and change the line

USE_LINUX=no

to

USE_LINUX=yes

Error:

undefined reference to symbol ‘some_gtk_function’

Solution:

Some flags are probably missing in the Makefile. A way to fix it is to edit the file Examples/Linux/sdk_demo/Build/Makefile. To add the correct flags, change the following line:

GENERIC_LIBS=-lpc_ardrone -lgtk-x11-2.0 -lrt

to

GENERIC_LIBS=-lpc_ardrone -lgtk-x11-2.0 -lrt `pkg-config --libs gtk+2.0`

Error:

Timeout when reading navdatas – resending a navdata request on port 5554

Solution:
One possibility is that your computer’s IP is not 192.168.1.2. You can set it with the following command line:

sudo ifconfig wlan0 192.168.1.2 netmask 255.255.255.0

Error:
The navigation data always contains zeros:
Solution
This is an error I had with VMWare. I changed my IP (in this case, the eth0 IP) to 192.168.1.3 and since then, I don’t have any more errors like this one. I use the bridged mode to connect to the drone via AirPort.
Error:
The check_dependencies.sh file is not installing the packages.
Solution:
Several problems can exist:

  • You are not using Ubuntu, and therefore the script cannot work. You need to manually install required packages.
  • You are not running the script with the root privilege. Prefix your command line by sudo.
  • The file temporary_file located in the same folder exists and indicates to the script that all the packages are installed. It can happen if you got an archive from an other computer, instead of the official SDK. You simply need to remove this file.

Golf results notifier

Golf Result Notifier is a tool I developed to help following professional golf tournaments. The main golf tours (PGA Tour, European Tour, LPGA Tour, etc.) provide live leaderboards, updated every time a player holes out. This is a great way to follow the tournaments, but it requires to take a look on a regular basis to the rankings. Therefore, I decided to develop a program that notifies every time a player’s score is updated. It pops up notifications like these:

The program is developed with Python.

Accessing to data

The crucial step is to get the data. I developed my program to handle two tours: PGA Tour and European Tour.
On a user aspect, the way to access to those data is easy: going to the tour’s website, and click on leaderboard.

With a program, the problem is different.
Firstly, the tours uses two techniques. The PGA Tour uses one url all the season: http://www.pgatour.com/leaderboard, but the European Tour uses a different url for each tournament. The format is: 

http://www.europeantour.com/europeantour/season=[Year]/tournamentid=[TournamentId]/leaderboard/index.html

For example: http://www.europeantour.com/europeantour/season=2011/tournamentid=2011013/leaderboard/index.html. Note that we use the Leaderboard tab, and not the Results tab when the tournament is not over.

To handle the European version, the program simply download the tour’s main page, and finds the URL with the following regex:

file = get_file("http://www.europeantour.com/index.html")
# Getting the box containing the URL
containerPattern = r'id="ETContainer_thisWeek".*?>(.*?)id="STContainer_thisWeek'
data = re.findall(containerPattern, file, re.M|re.I|re.S)[0]
# Getting the current tournament season and id
leaderboardPattern = r'href="/europeantour/season=(.*?)/tournamentid=(.*?)/'
season, tournament_id = re.findall(leaderboardPattern, data, re.M|re.I|re.S)[0]

Now that we have those URLs, we can think that we only need to download the file and parse it with a regex. Unfortunately, it is now working this way. When we download those pages, we only obtain the page’s layout, and no information about the rankings and the results. Therefore, we need to inspect these pages in our browser, to find out where the data are.

To do that, I am using FireBug, a Firefox plugin. I opened http://www.pgatour.com/leaderboard with Firebug opened to the Networking panel. I checked the files list, and detected a JSon file that contains all the results. The pattern is:

http://www.pgatour.com/15s/.element/ssi/auto/3.0/sdms/leaderboards/r[TournamentId]/data/current/leaderboard-1-all.json

The only changing part is the tournament id. This number can be found in the HTML of http://www.pgatour.com/leaderboard, and I get it with a simple regex.

The european tour works almost the same way. I  found that the results are stored in an HTML file. The URL pattern is:

http://www.europeantour.com/europeantour/season=[Year]/tournamentid=[TournamentID]/library/leaderboard/_leaderboard_v2.html

Extracting the player’s data

The data sources are very different in the two cases :

  • A JSON file (PGA Tour)
  • An HTML file (European Tour)
The first one is very easy to parse: I directly used Python’s JSON package. The loads function transforms the string stored in the file to a Python’s structure (dictionaries). I used a Javascript Beautifier to display the JSON file in a readable way, to be able to understand the structure of the file.
For example, let’s take this leaderboard. The JSON file looks like this:

{
    "lb": {
        "lt": "April 3, 2011 - 5:57 p.m.", /* local time */
        "tid": "020", /* tournament id */
        "c": { /* course information */
            "c": {
                "sl": "1",
                "h": [{ /* hole one distance and par */
                    "d": "397",
                    "p": "4",
                    "n": "1"
                },
                /*...*/],
                "n": "Redstone GC Tournament Course" /* Course name */
            }
        },
        "sd": "Thursday Mar. 31 - Sunday Apr. 3, 2011", /* tournament date */
        "tn": "Shell Houston Open", /* tournament name */
        "pds": { /* leaderboard */
            "p": [{ /* First player is the leader */
                "fcr": "19", /* Current Fedex Cup Rank */
                "sh": "1", /* Current ranking */
                "pfcr": "2", /* Projected Fedex Cup Rank */
                "id": "01810", /* PGATour.com player ID */
                "ln": "Mickelson", /* last name */
                "stat": { /* player statistics: average drive distance, driving accuracy... */
                    "s": [{
                        "id": "101",
                        "v": "323.5"
                    }, /*...*/
                    ]
                },
                "fcp": "548", /* Fedex Cup Points */
                "t": "F", /* "Thru" information, here F: player finished his round */
                "pfct": "500", /* Projected Event Points */
                "crs": "-7", /* Current round's score */
                "fn": "Phil", /* First name */
                "f": "USA", /* Country */
                "tp": "-20", /* Tournament score */
                "rn": "4", /* Current round */
                "pfcp": "1048", /* Fedex Cup points */
                "rs": { /* Round scores */
                    "r": [{
                        "sc": "70", /* Score */
                        "t": "12:40pm", /* Tee Time */
                        "rn": "1" /* Round number */
                    }, /*...*/
                    }],
                }
            }, {
                /* next player ... */

In my Python code, I can access the results like this:

file_content = get_file(filepath)
obj = json.loads(file_content)
tournament_name = obj["lb"]["tn"]
local_time = obj["lb"]["lt"]
results = {}
for p in obj["lb"]["pds"]["p"]:
    data = {"hole" : p["t"], "score": p["tp"],
            "pos": p["p"], "today": p["crs"],
             "id" : p["id"]}
    results[p["fn"] + " " + p["ln"]] = data
In the HTML case, I had to parse myself the file. I used several regular expressions to extract the data, and build the data structure.
The first step is to extract the leaderboard content. The first regex is the following one:

leaderboardPattern = r'id="lbl">.*?<tbody>(.*?)</tbody>'
leaderboard = re.findall(leaderboardPattern, file, re.M|re.I|re.S)[0]

Then I want to get all the lines (one per player):

playerPattern = r'<tr\s* id="(.*?)"[^>]+>(.*?)</tr>'
players = re.findall(playerPattern, leaderboard, re.M|re.I|re.S)

Finally, for each player I can extract the data and build the data structure.
For example, this line:

has the following html:

<tr id="798" class="o 798 sel">
  <td class="l"></td>
  <td class="mk">1</td>
  <td class="o60">77</td>
  <td>118</td>
  <td><img src="/imgml/flags/23x17/flag_fra.gif" tag="FRA" alt="FRA" title="France" width="23" height="17"></td>
  <td class="">-</td>
  <td class="b">1</td>
  <td class="nm">JACQUELIN, Raphaël</td>
  <td>-12</td><td>18</td><td>-3</td>
  <td class="rnd ">66</td><td class="rnd ">69</td>
  <td class="rnd ">69</td><td class="rnd ">68</td>
  <td>272</td>
  <td class="r"></td>
</tr>

I process this way:

results = {}
for p in players:
    id = p[0]
    pattern = r'class="b">([^<]+)<.*?"nm">([^<]+)<.*?<td>(.*?)</td>.*?<td>(.*?)</td>.*?<td>(.*?)<'
    infos = re.findall(pattern, p[1], re.M|re.I|re.S)
    if len(infos) == 0:
        continue
    infos = infos[0]
    data = {"hole" : infos[3], "score": infos[2],
            "pos": infos[0], "today": infos[4],
            "id" : id}
    results[infos[1]] = data

Notifying score modifications

Displaying a notification

Now that we have the latest information, we can compare it to the previous one, player by player. We compare the current score to the old one. If it changed, we notify it. Then, we update the stored leaderboard, wait 30 seconds and start again this process.

To display the notification, I use Growl, a mac application. To create a notification, I decided to use the growlnotify application. It is a command-line application that helps displaying notifications. A Python’s module is also available to perform the same task.
Growlnotify commands line works this way:

growlnotify -m "Message" -t "Title" --image "image_path"

In my python code, I use it this way:

os.system(command_line)

Getting the picture

When a notification pops, I like to have the picture of the player. Therefore, I added a feature that downloads the picture of a player from its ID.
In both cases (PGA Tour and European Tour), I analyzed the URL of player’s pictures. On the European Tour website, a picture is displayed when clicking on a player’s name on the leaderboard. I easily figured out with FireBug’s HTML tab that the pattern is:

 http://www.europeantour.com/imgml/players/[PlayerID].jpg

On the PGA Tour website, the pictures can be found in the Personnal tab of the player’s profile. The URL can be built from the player’s ID this way:

playerID = int(playerID)
a = playerID / 10000
b = (playerID / 100) % 100
c = playerID % 100
a,b,c = str(a).zfill(2), str(b).zfill(2), str(c).zfill(2)
url = "http://i.cdn.turner.com/pgatour/players/%s/%s/%s/images/headshot-96x109.jpg" % (a, b, c)

To avoid downloading every time the picture, I added a simple cache system:

localFname = "cache/pga-" + playerID + ".png"
# Check if we have the picture
if os.path.exists(localFname):
    return localFname
# Dowloading the picture
os.system("/sw/bin/wget %s -O %s  1>/dev/null 2>/dev/null" % (url, localFname))
return localFname # local path to image, given as parameter to growlnotify