March 2008

Mac/BlackBerry SDK

The BlackBerry is a very popular mobile communications device. Official Mac support from Research in Motion is limited to providing the PocketMac utility as a free download. With no official SDK the Mac/BlackBerry ecosystem has seen very little activity.

ZBlackBerry is a suite of code that implements the BlackBerry USB communications protocol in a generic fashion. A few hundred lines of code let Macs use that protocol. A few hundred more allow multiple Mac applications to talk to a single BlackBerry simultaneously, something that has not been possible till now.

Continue reading "Mac/BlackBerry SDK" »

October 2007

Embedding Lua in Zen

Lua is a very nice little programming language, which combines a clean C-ish syntax with the power of Scheme. Rather than requiring a particular programming style it provides building blocks that allow one to work in any combination of object-oriented, functional or imperative styles. Lua is something of a hidden gem, having found a keen but un-publicized audience amongst game developers who generally use it for scripting in-game behavior. My interest in it is to provide customizability of behavior in the Zen home automation system.

Continue reading "Embedding Lua in Zen" »

January 2006

WebDAV

WebDAV is an extension to the HTTP protocol. It is the basis for Apple's iDisk and Windows' Web Folders, standard features of Mac OS X and Windows XP. It is thus the easiest way for a server to make data available to a client machine without requiring that client software be installed first.

ZooLib provides a generic WebDAV server. Your application need only implement subclasses of ZNodeRep to represent nodes in your desired hierarchy. Standard subclasses of ZNodeRep let you lay one tree over another, or expose part of the server's file system to clients.

Continue reading "WebDAV" »

May 2005

ZTSoup: UI-Friendly TupleBase Access

The tuplebase API is well-suited to data processing needs. But it's clumsy as the mechanism by which data in a tuplebase is to be presented and maintained in a live UI, rather than web pages or generated reports.

ZTSoup gets its name and inspiration from the Apple Newton concept. A ZTSoup is backed by the same data as a tuplebase, but that data is accessed by instantiating a ZTCrouton for each tuple that code is interested in, and a ZTSieve for the result set of any query of interest. Why the funny names? A soup has croutons floating in it, interesting ones are sieved out of it.

Continue reading "ZTSoup: UI-Friendly TupleBase Access" »

May 2003

ZDCPixmapBlit

ZDCPixmapBlit uses templates to generate the code for source/destination and source/matte/destination blitting, matte/destination filling, and destination-only munging, using the four Porter/Duff composition operators Copy, Over, In and Plus.

October 2002

Tuplebase

ZooLib's tuplebase is derived from the tuplespace concept initially explored in the Linda coordination language, another well known derivation of which is Sun's JavaSpaces system. Whereas JavaSpaces is Java-only and relies on many of that languages's features, ZooLib's tuplebase works today with C++ and Java, and is well suited to work with other languages.

Continue reading "Tuplebase" »

October 2002

ZDC_ZooLib: Portable Graphics

ZooLib defines and implements a graphics API that produces identical results across all supported platforms. It is geared towards creating user interface elements and so is pixel-based rather than geometric so that a programmer can know precisely which pixels will be touched by a drawing operation. It supports pixel-plotting, lines of any thickness, text, rectangles, rounded rectangles, ovals, arbitrary regions and the drawing of masked-pixmaps. ZDC_ZooLib is an implementation of this API using no OS facilities at all, and thus can be used in server applications without the difficulties that normally poses (gaining access to a window or graphics server from a low-privilege process).

The implementation uses operations on ZBigRegion instances to decide which pixels to touch, decomposes the regions into rectangles and then calls ZDCPixmapBlit to actually do the work.

IFF and QuickTime File Formats

IFF is a venerable data meta-format, introduced in 1985 by Electronic Arts as a standard framing mechanism for multimedia data. In short it defines a nested chunked format, where the file as a whole is considered to be a chunk. A chunk has as its first four bytes a tag that indicates the type of data in the payload, a 32 bit (four bytes again) count of the number of bytes in the payload, and then that number bytes being the payload itself. Some chunk types are known to contain zero or more other chunks in their payload, so an arbitrarily complex hiearchy can be established. There are other details, but chunk types and sizes are IFF's essentials. QuickTime's file format is almost the same, except that the size comes first, and includes the eight bytes of the chunk header.

IFF/QT are simpler than most file formats, but can still be fiddly to work with. To make it easier for me to parse and generate QT files manually (on platforms which don't have QT libraries) I put together a suite of ZooLib streams that take care of all the bookkeeping.

Tuples Defined Rigorously

For most of my career I've been very suspicious of dynamic data representations. After all, what's the point of having a compiler if it isn't provided with enough information about the shape of the data being manipulated to tell you when your code is going wrong. However, that really only works for data created and consumed within a single application. In the mid-90s, every C++ framework worth its salt had a huge bunch of code dedicated to turning arbitrary C++ objects into something that could be serialized and regenerated as objects later; and in fact that's about as far as most people got with CORBA before giving up.

But most objects in a C++ program, if they're objects at all, simply don't need to be serialized. The ones that do have disparate needs, and special cases abound. The approach I've found most flexible and least intrustive is to provide ZTuple. In this context a tuple is isomorphic to a LISP a-list, Python or Cocoa dictionary, Perl or Ruby Hash or to a Java Map. It's simply a list of name/value pairs, where values can be primitives (strings, numbers, raw bytes etc), tuples or lists of values.

ZooLib provides a suite of facilities that read and write tuples to binary streams, generate and parse a well-defined and easy to read text format, and that can read and write appropriate data formats as tuples.

January 2002

ZStreamMUX

There's a maxim, a citation for which I can't locate now, which pours scorn on the idea of multiplexing streams over TCP. It kinda made sense when I first saw it, but in these days of ornery system adminstrators and their firewalls that sometimes allow connections seemingly on the roll of a die, there's a lot to be said for the all or nothing of getting a connection to a server, and then sharing it locally.

I wrote ZStreamMUX after coming across an early draft of WebMUX. ZStreamMUX takes a read stream and a write stream, and runs a lightweight protocol over them to support multiple independent sessions. The overall interface is similar to that provided by sockets. The protocol is asynchronous, and uses buffer credits to prevent deadlock at the protocol level.

June 2001

BlockStore: File System in a File

Applications often have to satisfy two conflicting requirements. On the one hand the data created by a user has a structure whose parts should be managed independently of one another, ideally with each piece placed in its own file. On the other hand users like to think of their data as a single entity which can be copied, emailed and backed up in its entirety. Although Mac OS X has the notion of a bundle, a specially marked directory which behaves like a single file when manipulated by the Finder, other operating systems do not.

Continue reading "BlockStore: File System in a File" »

March 2001

Assets: Portable resources

I picked the name 'asset' as an alternative to 'resource', a term that already has too many disparate meanings on different platforms. That said, assets are used in the same situations that MacOS/Win32/BeOS resources would be, although the mechanism is more flexible.

The data making up an asset tree is directly usable by any processor, big or little-endian. It can be kept in a file, loaded into RAM, memory-mapped from disk or accessed from a stream.

December 2000

ZooLib

ZooLib is an Open Source (MIT License) C++ library that makes it easy to write one set of source and build an application for Windows, Mac and UNIX. It provides a foundation suite of facilities that in essence form a virtual operating system API, and wide range of higher-level facilities that build on that foundation.

Continue reading "ZooLib" »

September 2000

ZStreamR_Boundary

ZStreamR_Boundary is a ZooLib stream derivative that uses Boyer-Moore-Horspool to take a source stream and efficiently provide to a caller only that data preceding a boundary. This is very useful when parsing MIME multipart streams, which can be nested arbitrarily and otherwise get very fiddly to deal with.

August 1999

NPainter

NPainter is a suite of classes that provides a MacPaint-like interface, but supporting indexed and true color, across all platforms supported by ZooLib.

Continue reading "NPainter" »

May 1999

Windoids: Windows Within Windows...

Measurement in Motion 1.0 had an interesting interface. A document contained a set of measurements, and various objects that could display and modify those measurements. The objects were represented as 'windoids', windows within the document window. MiM 1.0 was a Mac application, so the windoids could depend only on ZooLib's graphics API, with no help from the OS. When demand for a Windows version became overwhelming I implemented a full windowing system that conformed to ZooLib's ZOSWindow API, so that any code that worked within a regular OS window could instead be hosted in a windoid. QuickTime by this point was writing to the screen buffer asynchronously, so any change in the visible region of a windoid is notified before it happens and after its completed, somewhat like the BeOS BDirectWindow mechanism.

August 1998

ZBigRegion

In porting ZooLib to BeOS I found that BeOS's region API was missing key features. Exclusive or and equivalence testing could be handled by composing more primitive operations, but insetting was trickier. So I dug through the X source and found that their implementation of insetting was totally generic, basically shifting the source region for each set bit in the inset distance and accumulating a union (for expanding) or intersection (for contracting).

But BRegion was also quite slow. The internal representation was pretty close to that used by the X code, being a list of non-overlapping rectangles organized into bands. BRegion did not always maintain the same tight constraints as the X code, but it still proved possible to significantly improve performance by manipulating BRegion's internal data using the X code.

By this point I was intimately familiar with X regions and it was a small step to take the X source and rework it, the result being ZBigRegion - a portable region implementation using 32 bit coordinates. Combined with ZDCPixmapBlit it forms the basis of ZDC_ZooLib.

January 1996

ZDBase: Portable Database Engine

ZDBase is a relatively simple database storage engine. It's more of an alternative to Berkeley DB (as it was) than a replacement for MySQL. It supports an arbitrary number of tables, records and typed/named fields within those records. Each table maintains as many indices as desired, and the schema can be updated at any time. All data is kept in a blockstore, so everything lives in a single file, and the code and data are portable across platforms.

Continue reading "ZDBase: Portable Database Engine" »

Project Archives