Test

Migrating to a Mac

Have been really busy latelly, righting paper for CHI 2011 and Eurographics 2011, also, I bought a Mac, and am now going through the pain of converting code to another platform.

I usually use openCV 2.1 with OpenFrameworks, this is a pain in MacOS because OpenFrameworks only works in 32bit "i386" format. OpenCv on the other hand is quite easy to install in 64-bit mode but hard on 32.

To do that follow this link:

http://opencv.willowgarage.com/wiki/Mac_OS_X_OpenCV_Port

forget the MacPorts installation, use the CMake, and try several configurations with ccmake ( the gui, not the normal cmake ), especially remove ffmpeg and 3rdparties and put i386 in the architecture. It will probably give some error in the compile, but try another combination untill you get it!

Mobile HCI 2010

Tomorrow and in the rest of the week I will be attending (partially as student volunteer) to Mobile HCI 2010 which is held here in Lisbon.

Migrating to a Mac

New computer, new mac, new ways of doing old things, migrating the work should take some days...

Further updates soon…

I have been neglecting this blog for some time now, lot’s of work as been done, I will try to update more often.

Image Mosaicking from Video

 

I’ve been experimenting with image mosaicking for some time now. I finally achieved some interesting results.

In this experiment I recorded a video of a landscape inside my faculty.

The algorithm to create these panoramic images is something like this:

   1:  draw first frame image

   2:   

   3:  For each image

   4:      find important points on previous image

   5:      find the same points on current image

   6:      remove invalid points ( too close from the border, too different from average)

   7:      find homography rotation matrix( RANSAC method )

   8:      Warp image with rotation matrix

   9:      Blend current image rotated above the other images

  10:  endfor

 

The above example was created with OpenFrameworks and OpenCV using the following functions:

   1:  /* detect good points to track, such as corners*/

   2:  cvGoodFeaturesToTrack( firstImageGray, eig, temp, points[0], &pointsCount,

   3:                                     TRACK_QUALITY, MIN_DISTANCE, 0, 3, 0, 0.04 );

   4:   

   5:  /* detect the same points on second image */

   6:  cvCalcOpticalFlowPyrLK( firstImageGray, secondImageGray, prev_pyramid, pyramid,

   7:                  points[0], points[1], pointsCount, cvSize(WIN_SIZE,WIN_SIZE ),3, status, 0,

   8:                  cvTermCriteria(CV_TERMCRIT_ITER|CV_TERMCRIT_EPS,2000,0.001), flags );

   9:   

  10:  /*Find Homography*/

  11:  CvMat _imagePoints1 = cvMat(1, pointsCount, CV_32FC2, &points[0][0] );

  12:     

  13:  CvMat _imagePoints2 = cvMat(1, pointsCount, CV_32FC2, &points[1][0] );

  14:   

  15:  CvMat*    warp_mat = cvCreateMat(3,3,CV_32FC1);

  16:   

  17:  cvFindHomography( &_imagePoints2, &_imagePoints1,warp_mat, CV_RANSAC, 1 );

  18:   

  19:  /*Warp Image*/

  20:  cvWarpPerspective( secondImage, secondImageRot, warp_mat );

  21:   

  22:  /*blend image: copy only the parts that are not black*/

  23:  cvCvtColor(secondImageRot, secondImageRotGray, CV_BGR2GRAY );

  24:  cvErode(secondImageRotGray,secondImageRotGray);

  25:  cvCopy( secondImageRot, mainImage, secondImageRotGray  );

  26:   

These are separated bits of code. If you want the full code please send me an email.

rui.nobrega [at] di.fct.unl.pt

Conference calendar

Today I updated my personal calendar of conferences that I want to follow and possibly publish.

 

Computer Vision:
CVPR
3DPVT
more:
http://iris.usc.edu/Information/Iris-Conferences.html

Computer Graphics:
SIGGraph
EuroGraphics
more:
http://graphicsconferences.wikidot.com/upcoming-conferences

HCI:
CHI
Interact
UIST
TEI
more:
http://www.interaction-design.org/calendar/
http://www.conference-service.com/conferences/human-computer-interaction.html
http://www.sigchi.org/conferences/calendarofevents.html

Multimedia:
ACM MM
PCM
more:
http://sigmm.utdallas.edu/Conference%20Calendar/

Less important but Important:
SIGIR   (information retrieval)
GIR   (geographic information retrieval)
ICISP   (image and signal processing)
WIAMIS  (Image analysis for multimedia information systems)
ISMAR (augmented reality )
ACE
AVI
MobileHCI

Delaunay triangulation and Hough line detection

In March I made some experiences with Delaunay triangulation and Hough Line Detection using OpenCv. Here’s the results:

CAT

On February 4, 2010 was held in my department the Comissão de Acompanhamento de Tese. A reunion to evaluate the goals of my PHD thesis. I thing it went pretty well.

The commission that follows my PHD is composed by:

Joao Antonio Madeiras Pereira  IST – UTL

Manuel João Toscano Próspero dos Santos  FCT – UNL

Nuno Manuel Robalo Correia  FCT – UNL   ( advisor )

Luís Manuel Marques da Costa Caires  FCT – UNL  ( president of CITI )

Side Projects – Multi-touch Table

In this last months I’ve been involved in several side projects non-related with my PHD. Some of them are personal some are continuations of my previous work with multi-touch technologies.

My latest work is a multi-touch table for the Joana Vasconcelo’s exposition. This is a famous artist, at least in Portugal and UK. The exposition is being held at the Berardo Museum which is a modern art museum at CCB (Centro Cultural de Belém) in Lisbon.

It was a demanding job, we as researchers are very much used to do multi-touch stuff in laboratories. In a lab you don’t have to care about safety regulations, design, looks, random users or … babies on top of your table!

Here’s the link for the official page of the project with photos and specifications

The first challenge was to discover someone that could do a very large table using mirrors. We knew how to do it, but with no precision. This guy from Lusoscreen had some ideas and know-how with projectors and mirror paper. They made the projection system.

The second challenge was to decide what technique we would use to detect multiple fingers. There are two main approaches to do this cheaply using Laser Planes or the FTIR approach with LED’s.

Our first approach was to try a table with Laser Planes. It is an easier technique for large surfaces and it is very easy to use because the user has to do almost no pressure for it to work. Just put 4 lasers in each corner and you have a table. The surface can be glass with some kind of opacity. Although it works very well it has some problems, the lack of pressure means that almost anything can activate a touch. This includes the user clothing such as sleeves or the palm of the hand. The other problem is the use of Infra-red light lasers. These lasers are very dangerous when pointed directly to the eyes. That was not the case in the multi-touch table, the lasers are diffused in a plane parallel to the surface. BUT, and there always a but, it is not idiot proof. Some idiot could lean his head into the surface and stare at the lasers for hours and become blind.  This is a very unlikely situation, BUT, the table would be in a exposition filled with kids, and there could be someone complaining about it, so we decided NO lasers this solution is not good. We opted then with the FTIR technique which is harder to do but uses inoffensive LEDs. The FTIR table involved a large research in materials and hours painting compliant surfaces with silicone. But it was worth it.  Comparing LLP and FTIR in terms of detection, I would consider the FTIR better for tables and the LLP better for walls due to the sleeve problem.

The last challenge was to give the table a look that the artists form the exposition would accept. Giving them an acrylic surface with tracing paper on top is not a solution! People want smooth things with round corners such as the IPhone and the color of the Nintendo Wii! If you deliver a wooden box they will hate it, even if it has multi-touch, broadband internet and delivers coffee cups and muffins! With the help of the Joana Vasconcelo’s Atelier we turned the table into the nice sweet box that you can see bellow.

The software worked in the inauguration and has been continuously updated and is holding the volume of interaction. According to yesterday's news paper the exposition has been visited by almost 30 000 people in three weeks!

 

Related Work

In the last months I have been exploring the OpenCV library. It is a great library with several general purpose algorithms for computer vision. One of the problems is that it has become somewhat outdated. New discoveries and new projects and libraries have risen and should be explored.

Today I went exploring the web for further computer vision projects.

OpenVis3D is a library that takes advantage of stereo vision to create a 3D representation of the scene

http://code.google.com/p/openvis3d/

 

List of software and code for computer vision:

http://www.cs.cmu.edu/~cil/v-source.html

 

Real time dense stereo:

http://people.csail.mit.edu/demirdji/download/index.html

 

Panoramic and photo stiching tools

http://www.all-in-one.ee/~dersch/

 

Image stiching

http://sharpstitch.sourceforge.net/

Panoramic vision with a webcam

Yesterday I’ve continued my experiments with openCV, this is a small demo that creates a panoramic vision using a simple webcam. It works best if the camera if the camera is attach to some support and moved using only rotation.

This demo takes advantage of the functions explained in the previous post but presents the new camera frames in an approximated position using the average displacement of the common points of the current and previous frames.

 

An interesting blog with useful function for OpenCV is this one: http://blog.weisu.org/search/label/OpenCV

Finding common points in two consecutive frames or stereo vision.

This was a little test program to understand how to work with the flow functions from OpenCV.

This example is based on the lkdemo demo from OpenCV.

In the first image I detect the most important points from the image such as corners using the cvGoodFeaturesToTrack function.

In the second image the same points were tracked using the cvCalcOpticalFlowPyrLK function.

OpenCV and OpenFrameworks integration

Everyone that has used OpenFramewoks knows that there is an add-on that implements some functions from the OpenCV library. Most of these functions include basic image processing, blob tracking and face recognition. But what about the rest of the library? What if I want to use a Corner Detection function from OpenCv and still display the result in OpenFrameworks?

Bellow there is a simple code to use ofImage, ofxCvGrayscaleImage and ofxCvColorImage from OpenFrameworks and use them with OpenCV functions such as cvPreCornerDetect or cvDilate.

Using the OpenCV example in OpenFrameworks, extra declarations in testApp.h:

ofImage                    loader;
ofxCvGrayscaleImage        cornersImg;
ofxCvColorImage            colorImg; 

in testApp.cpp these are the setup and draw functions.

void testApp::setup(){
  loader.loadImage("t01-esq.png");  //load image
  colorImg.allocate( loader.width,loader.height);
  cornersImg.allocate( loader.width,loader.height);
  colorImg.setFromPixels(loader.getPixels(), loader.width,loader.height);
  //one channel 8bits values between 0 and 255
  IplImage *gray = cvCreateImage(cvSize(loader.width,loader.height), IPL_DEPTH_8U, 1);
  //one channel 32bits values between 0.0 and 1.0
  IplImage *corners = cvCreateImage(cvSize(loader.width,loader.height), IPL_DEPTH_32F, 1);
  IplImage *dilatedCorners = cvCreateImage(cvSize(loader.width,loader.height), IPL_DEPTH_8U, 1);
  //convert to grayScale
  cvCvtColor(colorImg.getCvImage(), gray, CV_BGR2GRAY);
  cvPreCornerDetect( gray, corners, 7 );
  cvDilate(corners,corners,0,1); 
 
    //convert from float do unsigned int by multiplying every falue by 255
  cvConvertScale( corners, dilatedCorners, 255 );
  cornersImg = dilatedCorners;
}  
void testApp::draw(){
  colorImg.draw(20,20);
  cornersImg.draw( colorImg.width + 40, 20 );
}

the final result is the following application:

Note that to convert an ofxCvColorImage to IPLImage we use setFromPixels, to convert an IPLImage to grayscale or color we use cvCvtColor, and to convert the type of the image from IPL_DEPTH_8U to IPL_DEPTH_32F and vice-versa we use cvConvertScale. To convert from IPLImage to ofxCvColorImage the image must be in the IPL_DEPTH_8U format with 3 channels, and just use “=”.

Some OpenCv functions require that the image to be in 32 bit float format. For further information read the OpenCV reference. There is no need to include additional files, OpenFrameworks already includes the OpenCV headers.