First step in getting the image translation to work... reading in and writing to pgm color image files.


I'm guessing that (0, 0) is supposed to be in the middle, not the bottom left hand corner for the first non-segv trial for removing the camera perspective.


I remembered to clear the color buffer this time. Also the location is mapped to a different (0, 0). The real width and hieght are now used instead of maxVal.
For complete, albeit formated poorly, source code to this point, click here.


Those damn parenthises. They are matched correctly (well maybe) now.
Source Code


The first image is the original. The second is the formula applied once. The second is the correction applied twice. Notice that after each correction the image is flipped. Thus, the poor second image is somewhat back to being square.
Source code




New method. Need to adjust the calculation of rsw (row shift width).
Source code


'Tis a bit rough, but at least the corners line up.
Source code


Hurrah. It works!!! Next step, mirror the output.
Source code


Mirrored output and loops to control the Moire interference patterns.
Source code


New and improved showColor() and setCPixel() functions. Improved usability with setCPixel() using RGB_INT instead of RGB and showColor() drawing 8 pixels (vs. 1) in a pass.
Source code

I think using the Moravec detector for finding corners will work. However, I need the image in grayscale to pass to the algorithm.
Source code


In theory I'm in the process of finding the edges. This same exact binary works fine on the linux Xserver, but xhosted to a sun machine dies for an unknown reason. After copying the chain code from viewer.c and upgrading it to color from black and white with a few other minor changes (avoiding segv) there is still some error in what is considered a boundry.
Source code


I added some mouse functionality to display the coordinate when the left mouse button is pressed. The chain codes are now color coded.
Source code


This is one of those things that really does suck. Thanks to gdb for this one.
Source code
gdb output

The error reported in the previous entry is reproduceable. The culperate line seems to be line 42.

#define MAX_CHAINS 5
If the value specified evaluates to a two digit number it works. If the value specified evaluates to a one digit number it crashes. This is really messed up.

Changing it to a constant definition like

const int MAX_CHAINS = 5;

is subject to the same wierdness as with the preprocessor directive.

Well, I have done the masticistic thing of making the code conform to the strictest C specification.
Source code
Note (4/28/2000): At this point there was in fact an error in the Makefile causing the -Wall and -fstrict-aliasing options to be ignored. It is not for a while yet following this chronological order that the code follows strict C standards.

IT GOES!!!

The code does work, albeit very slowly right now. Notice the recalculation of img_pxl_avg() for every loop at this time on line 270 (will definiatly fix soon).

The preceding MAX_CHAINS problem is still a problem. The image is actually grabbed before the code finishes becuase it wants to run many chains around the image.
Source code


Major cleanup of the code is in progress for this update. Still a long way to go though. Functionality is equivalent to previous log entry.
Source code

Lots to report for this update. Thanks to Andy and his memory leak tracking skills to find that funky error I was getting inside of malloc().

The obvious change is that the major axis is being found correctly. The rest of the lines aren't correct, but that is because the code was originally designed to identify keys, not tic-tac-toe boards.

With the updated checkThings() function, only one chain code is being used. It can use up to 10, but it is determinng object from background much more reliably.
Source code


Now this is a major development! The outline of the board. And it only takes about a *second* to calculate too. (Goal achieved Feb. 27 '00 1:30am.)
Source code


Major progress made in the speed of execution. The showColor() function has been completly rewritten using glDrawPixels().

There are two modes now. The new mode is a speed test. It loops, redrawing the screen and looking for the object (chaincodes). Each iteration only takes about a second. To access this speed test on line 17 the BUFFERED constant needs to be defined.
Source code


In theory the memory leaks are plugged. I'm actually calling free() now.
Source code

First, I am now able to read in compressed color pgm image files. This means files take up less space and load a lot faster.

Currently, the code identifies the object most likely to be the tic-tac-toe board. Unfortunatly, as the second picture shows, it does not yet realize that four pixels next to each other in the shape of square will not be the tic-tac-toe board.

The remaining changes are cosmetic. Mostly involving changes to the menu items available from a right mouse click.
Source code



I was working on detecting the pieces when I ran it with this image. Time to look at that board detecting algorithm again...
Source code


Fixed the boarder detection problem from previous entry. Currently, working on detecting the marbles. The colored dots indicate starting points for the search. Also, some mouse usability has been fixed, although it is not completly bug free yet...
Source code


Lots has changed over break. Things are broken up a little now. It makes changing things a little easier now.
research.c
research.h
chaincode.c
chaincode.h
linked_list.c
linked_list.h
I' m currently trying to find pieces. Unfortunatly, the inside line detecting algorithm sucks. So, that is my current focus...


The outside egde points for the board are being found fairly well now. Three of the four middle points too, but I don't see what is up with the fourth.
Source Code


I've added labels to display the (x, y) values of the edge/corner points. Also, I extend the inside lines back to the lines between the corners.

In reference to the previous entries uncorrect point. Note that the line it ends is also off in that same direction...
Source Code


The miscalculation error is fixed (round off errors).
Source Code


I ran the code (same as above) with a different image. This may prove interesting to fix, because the marble is getting in the way.

However, it should be noted that the marble on the middle left side is being correctly identified.


It works!!!!!!!!!!!!!!!!!
The pieces (or lack thereof) are being correctly found for the test images.
Source Code
middle has no piece
corner 1 has no piece
corner 2 has no piece
corner 3 has no piece
corner 4 has no piece
side 1-3 has no piece
side 1-4 has no piece
side 2-4 has no piece
side 2-3 has no piece

middle has a piece
corner 1 has no piece
corner 2 has no piece
corner 3 has no piece
corner 4 has no piece
side 1-3 has a piece
side 1-4 has no piece
side 2-4 has no piece
side 2-3 has a piece


I've added comments and some general code cleanup.
Source Code

It's been a while since I've touched the vision code itself. However, I am sending data over the network and displaying the data in the window. I still have to figure out what color scheme (RGB or YUV) I'm getting the data in originally, hence the messed up image shown.


Live data feeds are working! The calculation code doesn't chrash when it runs in sie mode!! The code even finds the board!!! And it is doing so using the YUV color scheme.

Only one week until the research conference...


After running the code with live data I had to go back to the drawing board on some of the edge and color detection code. After fixing some bugs and one major overhall as to how the board is stored in memory the code correctly determined the teams in the following.

56 hours untill the research confernce presentation...

The first column is the team, the second is the weighted value used to determine which team. ([0][0] = lower right, [0][2] = upper right, [2][0] = lower left, [2][2] = upper left)

pieces[0][0]: 2  64
pieces[0][1]: 1  113
pieces[0][2]: 2  43
pieces[1][0]: 1  140
pieces[1][1]: 2  72
pieces[1][2]: 2  52
pieces[2][0]: 2  68
pieces[2][1]: 1  125
pieces[2][2]: 1  99



April 28, 2000. 12:30am. 12 hours until defense.

The code works in its entirety. From the Frame Grabber to the arms.

The main source file is no longer called research.c but is ttt_vision.c.
The "first frame of garbage data" bug has been fixed now. This speeds up startup time becuase these computations are not being wasted.

Timming code is now in place inside the ttt_vision and shell_ttt programs. In the process of writing the timeing code I shaved about 6 seconds per frame off in calculation. One frame now takes about 10 seconds inside of the vision program itself. Considering at one point it took longer just to draw one frame to a window, this is really cool.

The vision program is fully multithreaded now. There are still two modes of execution too. If it is run as a stand alone program then only one thread is started. If in this mode the user chooses the "Buffer" selection then the program simulates what it does in SIE mode. In this the code does create a new thread. If the user turns off buffering then this thread is destroyed. If the vision program is started in SIE mode then one thread handles glut events and the other the socket connection and calculation.

Team detection has been rewritten. Shelley can now play as red or green. The code now uses differences between RGB instead of color wieghts.

Original Image

Preprocessing:
1. Perspective Correction
2. Edge Detection
3. Corner Identification
5. Moravec Filter

Processing:
6. Game Detection
7. Detect Pieces

Source Code:
ttt_vision.c
ttt_vision.h
ttt_vision_protocol.h
common.h
colors.h
pgm.c
pgm.h
linked_list.c
linked_list.h
chaincode.c
chaincode.h
Makefile
Ttt_Vision.C
Ttt_Vision.H
Makefile

To view the pages I used for my defense presentation clicke here.