March 2008

Mac/BlackBerry SDK

The BlackBerry is a very popular mobile communications device. Official Mac support from Research in Motion is limited to providing the PocketMac utility as a free download. With no official SDK the Mac/BlackBerry ecosystem has seen very little activity.

ZBlackBerry is a suite of code that implements the BlackBerry USB communications protocol in a generic fashion. A few hundred lines of code let Macs use that protocol. A few hundred more allow multiple Mac applications to talk to a single BlackBerry simultaneously, something that has not been possible till now.

Continue reading "Mac/BlackBerry SDK" »

January 2006

WebDAV

WebDAV is an extension to the HTTP protocol. It is the basis for Apple's iDisk and Windows' Web Folders, standard features of Mac OS X and Windows XP. It is thus the easiest way for a server to make data available to a client machine without requiring that client software be installed first.

ZooLib provides a generic WebDAV server. Your application need only implement subclasses of ZNodeRep to represent nodes in your desired hierarchy. Standard subclasses of ZNodeRep let you lay one tree over another, or expose part of the server's file system to clients.

Continue reading "WebDAV" »

December 2005

Windows Services

UNIX does not make an explicit distinction between programs that are servers vs programs invoked by a user. Each flavor of UNIX provides a mechanism for the management of these programs, generally by using naming and placement conventions of descriptor files in the file-system.

In contrast, Windows NT/XP supports servers with a specialized API and management tools. Windows services are packaged as regular applications, but when they are invoked by the service framework they register a callback with that framework to which status and control messages can be delivered.

Continue reading "Windows Services" »

May 2005

ZTSoup: UI-Friendly TupleBase Access

The tuplebase API is well-suited to data processing needs. But it's clumsy as the mechanism by which data in a tuplebase is to be presented and maintained in a live UI, rather than web pages or generated reports.

ZTSoup gets its name and inspiration from the Apple Newton concept. A ZTSoup is backed by the same data as a tuplebase, but that data is accessed by instantiating a ZTCrouton for each tuple that code is interested in, and a ZTSieve for the result set of any query of interest. Why the funny names? A soup has croutons floating in it, interesting ones are sieved out of it.

Continue reading "ZTSoup: UI-Friendly TupleBase Access" »

May 2004

Java Tuplebase Access

Initially I provided Java access to a tuplebase instance by implementing Java classes whose most interesting methods were marked as native, and thus invoked via JNI. This was very powerful because Java could use any tuplebase implementation simply by calling the appropriate factory function and I could expose any existing C++ functionality simply by implementing the appropriate JNI glue.

Continue reading "Java Tuplebase Access" »

May 2003

ZDCPixmapBlit

ZDCPixmapBlit uses templates to generate the code for source/destination and source/matte/destination blitting, matte/destination filling, and destination-only munging, using the four Porter/Duff composition operators Copy, Over, In and Plus.

November 2002

Electric Fence and Threads

Electric Fence is an immensely useful library written by Bruce Perens. It uses the standard UNIX memory mapping API to put guard pages on either side of memory allocations, thus causing a segment violation as soon as your code writes where it shouldn't. I made a few small changes to make it possible to use from a preemptively multi-threaded application.

November 2002

Web Browser Plugins

ZooLib's UI code requires only that there be a ZFakeWindow-derivative at the top of the enclosure hierarchy. In the distant past ZooLib included implementations of ZFakeWindow for Mac control panel cdevs, HyperCard XCMD windows, MacroMind Director XObject windows, Zoom closures, MacApp views and of course still does for ZOSWindows.

It was thus relatively straightforward to implement ZFakeWindow_NSPlugin, which translates between the Netscape browser plugin and the ZFakeWindow APIs. What was actually more difficult was finding a decent implementation of the plugin glue code and header files. In 2002 there wasn't anything that would work with current compilers and with current browsers, so most of the effort was in putting together ZNSPlugin, which is a usable implementation of the glue.

October 2002

Tuplebase

ZooLib's tuplebase is derived from the tuplespace concept initially explored in the Linda coordination language, another well known derivation of which is Sun's JavaSpaces system. Whereas JavaSpaces is Java-only and relies on many of that languages's features, ZooLib's tuplebase works today with C++ and Java, and is well suited to work with other languages.

Continue reading "Tuplebase" »

October 2002

ZDC_ZooLib: Portable Graphics

ZooLib defines and implements a graphics API that produces identical results across all supported platforms. It is geared towards creating user interface elements and so is pixel-based rather than geometric so that a programmer can know precisely which pixels will be touched by a drawing operation. It supports pixel-plotting, lines of any thickness, text, rectangles, rounded rectangles, ovals, arbitrary regions and the drawing of masked-pixmaps. ZDC_ZooLib is an implementation of this API using no OS facilities at all, and thus can be used in server applications without the difficulties that normally poses (gaining access to a window or graphics server from a low-privilege process).

The implementation uses operations on ZBigRegion instances to decide which pixels to touch, decomposes the regions into rectangles and then calls ZDCPixmapBlit to actually do the work.

IFF and QuickTime File Formats

IFF is a venerable data meta-format, introduced in 1985 by Electronic Arts as a standard framing mechanism for multimedia data. In short it defines a nested chunked format, where the file as a whole is considered to be a chunk. A chunk has as its first four bytes a tag that indicates the type of data in the payload, a 32 bit (four bytes again) count of the number of bytes in the payload, and then that number bytes being the payload itself. Some chunk types are known to contain zero or more other chunks in their payload, so an arbitrarily complex hiearchy can be established. There are other details, but chunk types and sizes are IFF's essentials. QuickTime's file format is almost the same, except that the size comes first, and includes the eight bytes of the chunk header.

IFF/QT are simpler than most file formats, but can still be fiddly to work with. To make it easier for me to parse and generate QT files manually (on platforms which don't have QT libraries) I put together a suite of ZooLib streams that take care of all the bookkeeping.

Tuples Defined Rigorously

For most of my career I've been very suspicious of dynamic data representations. After all, what's the point of having a compiler if it isn't provided with enough information about the shape of the data being manipulated to tell you when your code is going wrong. However, that really only works for data created and consumed within a single application. In the mid-90s, every C++ framework worth its salt had a huge bunch of code dedicated to turning arbitrary C++ objects into something that could be serialized and regenerated as objects later; and in fact that's about as far as most people got with CORBA before giving up.

But most objects in a C++ program, if they're objects at all, simply don't need to be serialized. The ones that do have disparate needs, and special cases abound. The approach I've found most flexible and least intrustive is to provide ZTuple. In this context a tuple is isomorphic to a LISP a-list, Python or Cocoa dictionary, Perl or Ruby Hash or to a Java Map. It's simply a list of name/value pairs, where values can be primitives (strings, numbers, raw bytes etc), tuples or lists of values.

ZooLib provides a suite of facilities that read and write tuples to binary streams, generate and parse a well-defined and easy to read text format, and that can read and write appropriate data formats as tuples.

April 2002

Files

Most of my work till this point had not required 'interesting' operations with file systems, being restricted to creating and opening files in externally determined locations, then reading and writing the files' contents. When it became necessary to ennumerate the contents of directories, and to deal with permissions and locking I took the opportunity to define an API that would be consistent across Windows, Mac and UNIX whilst cleanly accomodating their differences.

Continue reading "Files" »

January 2002

ZStreamMUX

There's a maxim, a citation for which I can't locate now, which pours scorn on the idea of multiplexing streams over TCP. It kinda made sense when I first saw it, but in these days of ornery system adminstrators and their firewalls that sometimes allow connections seemingly on the roll of a die, there's a lot to be said for the all or nothing of getting a connection to a server, and then sharing it locally.

I wrote ZStreamMUX after coming across an early draft of WebMUX. ZStreamMUX takes a read stream and a write stream, and runs a lightweight protocol over them to support multiple independent sessions. The overall interface is similar to that provided by sockets. The protocol is asynchronous, and uses buffer credits to prevent deadlock at the protocol level.

June 2001

BlockStore: File System in a File

Applications often have to satisfy two conflicting requirements. On the one hand the data created by a user has a structure whose parts should be managed independently of one another, ideally with each piece placed in its own file. On the other hand users like to think of their data as a single entity which can be copied, emailed and backed up in its entirety. Although Mac OS X has the notion of a bundle, a specially marked directory which behaves like a single file when manipulated by the Finder, other operating systems do not.

Continue reading "BlockStore: File System in a File" »

March 2001

Assets: Portable resources

I picked the name 'asset' as an alternative to 'resource', a term that already has too many disparate meanings on different platforms. That said, assets are used in the same situations that MacOS/Win32/BeOS resources would be, although the mechanism is more flexible.

The data making up an asset tree is directly usable by any processor, big or little-endian. It can be kept in a file, loaded into RAM, memory-mapped from disk or accessed from a stream.

December 2000

ZooLib

ZooLib is an Open Source (MIT License) C++ library that makes it easy to write one set of source and build an application for Windows, Mac and UNIX. It provides a foundation suite of facilities that in essence form a virtual operating system API, and wide range of higher-level facilities that build on that foundation.

Continue reading "ZooLib" »

September 2000

ZStreamR_Boundary

ZStreamR_Boundary is a ZooLib stream derivative that uses Boyer-Moore-Horspool to take a source stream and efficiently provide to a caller only that data preceding a boundary. This is very useful when parsing MIME multipart streams, which can be nested arbitrarily and otherwise get very fiddly to deal with.

December 1999

BeOS Audio Driver for Sony N505VE

Putting BeOS on a machine that had previously run only Windows was a revelation. Elegant, efficient, and the rarely needed reboots took only seconds. But Be had a small team and their hardware support was focused on the standard modes of common hardware. Laptop hardware tends to be anything but standard. The Sony N505VE was a very lightweight laptop with a good sized screen. Its sound hardware was powerful, but unique and at the time undocumented, and so was entirely unsupported by BeOS. I wrote and open sourced a BeOS audio driver that used the YMF-744B's legacy SoundBlaster emulation to provide 8 bit audio output.

August 1999

NPainter

NPainter is a suite of classes that provides a MacPaint-like interface, but supporting indexed and true color, across all platforms supported by ZooLib.

Continue reading "NPainter" »

May 1999

Windoids: Windows Within Windows...

Measurement in Motion 1.0 had an interesting interface. A document contained a set of measurements, and various objects that could display and modify those measurements. The objects were represented as 'windoids', windows within the document window. MiM 1.0 was a Mac application, so the windoids could depend only on ZooLib's graphics API, with no help from the OS. When demand for a Windows version became overwhelming I implemented a full windowing system that conformed to ZooLib's ZOSWindow API, so that any code that worked within a regular OS window could instead be hosted in a windoid. QuickTime by this point was writing to the screen buffer asynchronously, so any change in the visible region of a windoid is notified before it happens and after its completed, somewhat like the BeOS BDirectWindow mechanism.

August 1998

ZBigRegion

In porting ZooLib to BeOS I found that BeOS's region API was missing key features. Exclusive or and equivalence testing could be handled by composing more primitive operations, but insetting was trickier. So I dug through the X source and found that their implementation of insetting was totally generic, basically shifting the source region for each set bit in the inset distance and accumulating a union (for expanding) or intersection (for contracting).

But BRegion was also quite slow. The internal representation was pretty close to that used by the X code, being a list of non-overlapping rectangles organized into bands. BRegion did not always maintain the same tight constraints as the X code, but it still proved possible to significantly improve performance by manipulating BRegion's internal data using the X code.

By this point I was intimately familiar with X regions and it was a small step to take the X source and rework it, the result being ZBigRegion - a portable region implementation using 32 bit coordinates. Combined with ZDCPixmapBlit it forms the basis of ZDC_ZooLib.

January 1998

ZFiber: Threads At Interrupt Time

Fibers, at least in this context, are threads implemented using setjmp/longjmp to transfer control in a manner reminiscent of co-routines. They are a generalization of the stack-swapping used by NetPhone to run in the constrained environment it experienced when its deferred tasks were scheduled when an application with a tiny stack was current (yes Print Monitor, I'm talking about you).

Continue reading "ZFiber: Threads At Interrupt Time" »

September 1997

ZOSWindow: Abstracting OS Window APIs

ZooLib had always maintained the ZFakeWindow abstract interface between user interface elements and the hosting environment. This allowed ZooLib UI widgets to be used within MacApp applications, standalone applications, Control Panels, XCMD windows and XObject windows. It worked very well.

However, when the portability axis was across platforms rather than hosting environments there was the potential for a lot of replication and ugly code in ZWindow, the standalone application derivative of ZFakeWindow.

So, I abstracted the interface to OS-windows, placing it in ZOSWindow, and modified ZWindow to use a ZOSWindow, rather than be conditionally compiled for different platforms.

January 1996

ZDBase: Portable Database Engine

ZDBase is a relatively simple database storage engine. It's more of an alternative to Berkeley DB (as it was) than a replacement for MySQL. It supports an arbitrary number of tables, records and typed/named fields within those records. Each table maintains as many indices as desired, and the schema can be updated at any time. All data is kept in a blockstore, so everything lives in a single file, and the code and data are portable across platforms.

Continue reading "ZDBase: Portable Database Engine" »

Project Archives