home | projects | about

Wednesday, May 14, 2008

fwd: a tool for peer to peer port forwarding

Today I am releasing fwd, a tool for port forwarding between peers, through the firewalls and NATs separating the peers. Fwd achieves this without the need for firewall configuration. The only requirement is that each peer is running an instance of fwd.

To set up a forwarding session, fwd negotiates a peer to peer tunnel between the local and remote peers using a technique called Interactive Connectivity Establishment (ICE). Forwarded data is sent through this tunnel.

Fwd is cross platform and runs on Mac, Linux, and Windows. Read more and download fwd here.

Wednesday, April 23, 2008

Fast 2D graphics w/OpenGL ES

Say you have an existing 2D game with a frame rate that generates new frames totally in game code, and you are porting to the increasing popular OpenGL ES v1.1 api. You can read about OpenGL ES from the public specification here:

http://www.khronos.org/opengles/spec/

Let's assume, hypothetically, that your game already has optimized 2D graphics code, and only needs a way to get a composed frame to the screen quickly (this is the case for Warfare Incorporated). There are two overall approaches:

1. Turn all your graphics objects into OpenGL texture objects, then compose your frame by texture mapping these into a color buffer.

2. Keep all your optimized 2D graphics code. When a frame is ready, turn it into a texture with OpenGL ES, and texture map this into a color buffer.

There are pros and cons to each of these. #1 will mean a lot of code change: if you have optimized 2D graphics code already, it's possible your graphics are stored in a custom format, and that drawing a particular object is done in a custom way. Porting the format, and the drawing for every object over to OpenGL ES api is a lot of work. However, it will likely be the fastest executing if the device your game is running on has a graphics accelerator. It is also likely to take the least amount of battery life.

Approach #2 is the least amount of work: it means you keep all your 2D graphics code and data formats. When a composed frame is ready you "upload" it to OpenGL ES, and have it draw that to the screen. The problem with this is that uploading a texture is often slow. First, regular texture objects aren't designed to dynamically update. When a texture is uploaded it is converted into a gpu specific native format that is optimized for fast texture mapping. This "conversion" is quite slow. Putting this texture conversion into the middle of your game's frame rate won't work.

Thankfully, OpenGL ES has created the concept of a "PBuffer". A PBuffer is both a surface for rendering onto, that can also be used as a pixel buffer for a texture. It is assumed that PBuffers dynamically update, so using a PBuffer as a texture is designed to be inexpensive. They key thing for your 2D game, is that uploading a new texture into your PBuffer is also fairly inexpensive, using glTexSubImage2D().

Once your frame is composed in your PBuffer, you'll want to draw it onto your windowed surface. OpenGL ES v1.1 has a nice "extension api" called glDrawTex(). This is the fastest and easiest way of drawing a textured quad. Use this to draw your PBuffer contents to your window surface. Before you call glDrawTex(), be sure to set the clipping rectangle with glTexParameter / GL_TEXTURE_CROP_RECT_OES. Finally, your platform specific swap buffers api will copy your color buffer to the associated native window. This whole process will give you decent frame rates with minimal change to your existing game. To recap:

1. Use glOrtho to set up a parallel projection, useful for your 2D game.
2. Create a PBuffer surface and associate it with a texture name. Use powers of 2 dimensions.
3. Use glTexSubImage to update only the parts of the PBuffer that are changing (worst case, the whole frame).
4. Once the PBuffer is ready, draw it to your window surface by setting the cropping rect and calling glDrawTex.
5. Swap buffers. Goto 3.

I will post more information about this approach in June.

Tuesday, March 04, 2008

Using Leopard's RSS-fed picture screen saver

I have a lot of pictures - actually it's my wife who takes them all. I mainly ensure they are accessible, safely backed up on and off site, and useful in various ways. They are stored / accessed from a home Linux machine. So when my wife asked to see them on her Mac in a screen saver, I was pleased to discover that Leopard had an RSS based picture screen saver built in. This should be a snap I thought. First thing - our pictures are simply stored on the Linux machine's file system in a large directory tree. I needed a way to 1> serve an RSS feed, and 2> make the pictures in the feed accessible from http. In the process of writing a script to do this I found out a few tidbits about Leopard's RSS screen saver:

- The first thing I did was make a single RSS feed of all the pictures - 60K or so. This just didn't work. (a) For some reason, the screen saver takes forever to validate the large RSS document I enter. It literally hung and after 5 minutes I killed it. (b) every picture in the RSS feed is read right away, rather than reading the picture just before it is displayed. I decided a smaller number of images (< 1000) randomly chosen every 24 hours from the archive, would be a better approach.

- The screen saver ignores the EXIF orientation setting. Modern cameras know the orientation of the photo because it is recorded in the image itself how the camera was being held at the time the picture was taken. I added support to orient the image properly. Doing this required storing the normalized image in a local cache (because I didn't want to modify the original). While doing this, I added support for resizing to a desired size. There is no need to make the image larger than the screen it is displayed on, and resizing makes the file sizes much smaller than the original typically. This is helpful since the screen saver downloads and caches all the items in the RSS feed, so smaller images use less disk space on the Mac.

- Originally I named the pictures in the normalized image cache by index. When the images were changed, the same names were used since it was index based. This caused a problem because the screen saver cached the images by name. So now the images are named based on md5 sum, ensuring uniqueness.

- Finally, I noticed that using an enclosure in the RSS item for the picture caused the RSS screen saver to log a warning message to the console utility about not being able to find the "img node". I changed the script to not use enclosures.

BuildRss is the result, a simple script that works around the Leopard's RSS picture screen saver's various behaviors. I run this script nightly from a cron job to update with a new batch of pictures. You can download the script from here.

Sunday, December 24, 2006

Link-Backup v 0.8

New version of Link-Backup released, with a few enhancements:

v0.8 12/24/2006 scottlu
- allow backup of any file while it is changing
- added --verbose logging to tree building
- minor --verify command fix

Link-Backup can be found here.

Sunday, September 03, 2006

Link-Backup v 0.7

v 0.7 09/02/2006 scottlu
- Ignore pipe, socket, and device file types
- Added --ssh-i to select ssh id file to use (see ssh -i) (Damien Mascord)
- Added --ssh-C to perform ssh compression (see ssh -C) (David Precious)
- Added --ssh-p to specify remote port (see ssh -p) (David Precious)

A friendly user reported a failed backup, and on close inspection it turned out that link-backup was attempting to backup atypical file types such as pipes, sockets, and devices. Now link-backup knows how to ignore those types of files.

Added --ssh-C on request to compress the transfer stream. If you're mainly backing up already compressed files such as pictures and music, --ssh-C is likely not to be a win. For other types of files, it'll save some bandwidth.

Added --ssh-i and ssh-p on request. Good options, thanks for the suggestions and code!

Get Link-Backup here.

Sunday, June 18, 2006

Link-Backup v 0.6

v 0.6 06/17/2006 scottlu
- Ignore broken symlinks and other failed stats during filelist creation
- Added --lock, which ensures only one backup to a given dest can occur at a time
- Released viewlb.cgi, a web based backup viewer.

Special thanks to Joe Beda for the lock feature and backup viewer.

Get Link-Backup here.

Saturday, April 15, 2006

Link-Backup v 0.5

v 0.5 04/15/2006 scottlu

- Added 'latest' link from Joe Beda (thanks Joe!). Now your latest backup is linked from "latest" so it's easy to access.
- Fixed --verify. It wasn't specifying the remote machine correctly.

Read about Link-Backup here.