February 2014

WireOver - Fast, Free, Secure File Sending

WireOver is a Mac, Windows and Linux application for sending and receiving files. It's easy to use and very low friction, requiring only entry of an email address and click on a received web link to set up. Receiving an unsolicited send is even easier, as acting on the receipt of the notifying email naturally associates that email with the recipient.

Transfers (by subscribers) use perfect forward secrecy, and can be entirely asynchronous, so sender and receiver need never be online at the same time. Of course if they are, then transmission goes as fast as the physical network allows.

We built the backend and client mainly in Python, supporting Mac and Windows back to 10.5 and XP respectively. For Linux we support 32 and 64 bit Ubuntu 12 and later.

Much of my focus has been on keeping OS variations from impacting the main body of the code. To that end, we're using wxWidgets to abstract the GUI, and I've implemented such native code as needed to interface with unwrapped aspects of the OS and third-party libraries. In some cases the fluent Python bindings to OS facilities just aren't fast enough, and native code is used there also.

March 2012

SPARKvue Augmented Reality

Working with Sally Ride Science and PASCO I extended the desktop version of SPARKvue to support augmented reality views of the experimental setup.

In this enhanced version of SPARKvue sensors are tagged with fiduciary markers. The video from a camera pointed at the experimental setup is shown beneath normal UI elements, and measurements from sensors are drawn into the video as it runs.

November 2011

MOTOACTV HTTP Proxy

The MOTOACTV communicates with servers to share and manage its user's fitness information. The device is very small and interacting with a complex network configuration UI is not viable. So, when tethered, it communicates through an HTTP proxy I wrote that runs on the user's desktop machine.

This is not my first full HTTP proxy; that was the desktop component of iMobimac Modem. In this case there were a couple of requirements that made things interesting.

Generally a proxy is a long-running system service, and it builds and evolves knowledge of what outbound paths and upstream proxies are available and usable. This proxy launches when the device connects, and shuts down when it quits, so any knowledge it can acquire is likely to be stale.

The approach I took was to try everything at once -- initiate connections on every combination of network interface, proxy name and resolved IP address, and use whichever wins the race. Thereafter it starts with the last thing that worked, and only scatterguns after a timeout. Interestingly this makes regular desktop browsing more responsive too, but it's pretty abusive to the network infrastructure and is only safe for idempotent GETs.

The other interesting feature was handling Proxy Auto Config (PAC) files. These are chunks of javascript that take a desired host name and return the name/address of the proxy that should handle the request. I'd never come across one in the wild, but they're very common in corporate networks where there's free rein inside the firewall, but all external access is via proxies. The system facilities on OS X are sufficient that the task can be handled by calling CFNetworkExecuteProxyAutoConfigurationURL. Windows on the other hand can almost do the job. The WinHTTP library can handle a PAC specified with a network URL, but not with a local file. So when we see that the PAC is a local file we fabricate an HTTP URL referencing a port we're handling, and any request on that port returns the content of the PAC file.

December 2008

Zorap, Web-Based Video Chat

Zorap web pages are multi-party multimedia environments. Video from your webcam and audio from your microphone, photos, music and video from your own computer are shared to tens of your friends via a Zorap Server. Web content, including YouTube and other video sites are similarly shared, all in an exciting and customizable drag-and-drop interface.

Electric Magic implemented the Mac-specific portions of the Zorap web plugin, supporting Safari, Firefox and other modern web browsers.

March 2008

Mac/BlackBerry SDK

The BlackBerry is a very popular mobile communications device. Official Mac support from Research in Motion is limited to providing the PocketMac utility as a free download. With no official SDK the Mac/BlackBerry ecosystem has seen very little activity.

ZBlackBerry is a suite of code that implements the BlackBerry USB communications protocol in a generic fashion. A few hundred lines of code let Macs use that protocol. A few hundred more allow multiple Mac applications to talk to a single BlackBerry simultaneously, something that has not been possible till now.

ZBlackBerry can also make a BlackBerry connected to a PC accessible via the same API as is used when it's connected directly to a Mac. This lets Mac application developers run their BlackBerry application under the Windows-based debugger and still have it communicate with their Mac application. This is a crucial capability when writing anything more than the most trivial application.

ZBlackBerry is part of ZooLib, our open source C++ library.

May 2006

Knowledge Forum 4.6

Knowledge Forum 4.6's minor version number change belies just how different from its predecessor it really is.

Under the hood KF 4.6 has moved from using ZDBase for its backing store, instead using a tuplebase. This makes it possible to split HTML page generation into separate processes, potentially running on multiple front end machines. It also restores support for a rich client application, now written in Java, using ZTSoup to efficiently communicate changes in the tuplebase, whether made by other clients or by the web interface.

December 2005

Windows Services

UNIX does not make an explicit distinction between programs that are servers vs programs invoked by a user. Each flavor of UNIX provides a mechanism for the management of these programs, generally by using naming and placement conventions of descriptor files in the file-system.

In contrast, Windows NT/XP supports servers with a specialized API and management tools. Windows services are packaged as regular applications, but when they are invoked by the service framework they register a callback with that framework to which status and control messages can be delivered.

Turning an application into a service is straightforward, but involves a certain amount of boiler-plate code, and requires that the application be structured to be controllable by a thread started by the Service framework. The ZooLib class ZWinService implements the boiler-plate, and at its simplest requires only that a Run and Stop method be implemented on a derivation of ZWinService in order to connect an application to the service framework.

May 2004

Java Tuplebase Access

Initially I provided Java access to a tuplebase instance by implementing Java classes whose most interesting methods were marked as native, and thus invoked via JNI. This was very powerful because Java could use any tuplebase implementation simply by calling the appropriate factory function and I could expose any existing C++ functionality simply by implementing the appropriate JNI glue.

But native code can't be included with unsigned applets, so I ported ZTBRep_Client from C++ to Java. ZTBRep_Client is the most useful tuplebase implementation in that it talks to a ZTBServer instance on the end of a comms link, generally a network socket. It's also fairly large, and the async nature of the protocol makes things somewhat tricky. The normal Java synchronization model is based around monitors, which are not directly useful for implementing certain types of concurrent processing, so I implemented condition vars and mutexes to provide more appropriate tools.

August 2003

Knowledge Forum 4.5

Knowledge Forum 4.5 was a significant refinement of Knowledge Forum 4.0, with a much richer web interface, although still constrained by the need to support Netscape 4.x-era web browsers.

New features included support for upload and download of files as attachments to views and notes, headings within views, full text and attribute-based searches, and notification by email. It also supported graphs of useful metrics, implemented across platforms using ZDC_ZooLib.

On the back end, a significant change was moving from the original blockstore implementation to one based on B+Trees that uses a neat upward propagting copy-on-write of nodes to make it virtually bullet-proof - only a physical failure of the backing store can cause data loss.

November 2002

Web Browser Plugins

ZooLib's UI code requires only that there be a ZFakeWindow-derivative at the top of the enclosure hierarchy. In the distant past ZooLib included implementations of ZFakeWindow for Mac control panel cdevs, HyperCard XCMD windows, MacroMind Director XObject windows, Zoom closures, MacApp views and of course still does for ZOSWindows.

It was thus relatively straightforward to implement ZFakeWindow_NSPlugin, which translates between the Netscape browser plugin and the ZFakeWindow APIs. What was actually more difficult was finding a decent implementation of the plugin glue code and header files. In 2002 there wasn't anything that would work with current compilers and with current browsers, so most of the effort was in putting together ZNSPlugin, which is a usable implementation of the glue.

September 2002

Knowledge Forum 4.0

Knowledge Forum 4.0 took the radical direction of being web-only. We'd had some support for web access by virtue of a perl program that used the client's communications protocol to talk directly to the server, but perl wasn't pre-installed on Mac OS (Classic) or Windows, and it had some performance problems. So we ported the perl software to C++ and incorporated it into the server directly.

For a time we had native client and web access running simultaneously, but never in a released version. The 3.x client remains buildable as of May 2006, but still talks only to a 3.x server.

April 2002

Files

Most of my work till this point had not required 'interesting' operations with file systems, being restricted to creating and opening files in externally determined locations, then reading and writing the files' contents. When it became necessary to ennumerate the contents of directories, and to deal with permissions and locking I took the opportunity to define an API that would be consistent across Windows, Mac and UNIX whilst cleanly accomodating their differences.

The ZooLib file API doesn't address more esoteric file systems like VMS or IBM's MVS. It models a hierarchical system, with accomodations for special roots (like Windows' UNC paths and Mac Classic's multiple volumes with the same name). Files are byte-oriented, and as with other stream operations ZooLib's file Open and Create return a ref counted streamer object, and optionally an error enum providing more detail in the case of failure. The representation of a pointer to a node in a file system also follows ZooLib's practice of a using a smart object with value semantics.

The ZFile suite was the first place I formalized the notion of a 'trail'. This was another case where it was important to come up with a new name that was suggestive of an existing notion, but distinct enough from it that its different semantics could be defined. Picture a tree; a trail is simply the list of steps to be taken to navigate from some node to some other node. A step is either to traverse the link from the node to its parent, or to traverse the link from the node to one of its children. We represent a step as a string. An empty string indicates a 'bounce', a traversal to the parent of the node. Any other string indicates a traversal to the node with that name. Whether the name is attached to the child node, or attached to the link to the child node doesn't matter in this model.

The notion of a trail through a tree shows up in assets and in the handling of HTTP requests. Theoretically it could also be applied to tuples but I'm not quite ready to harmonize everything to that extent.

January 2000

Knowledge Forum 3.0

Knowledge Forum 3.0 took advantage of major enhancements in ZooLib that let the client be released for Windows as well as Mac OS. Much of Knowledge Forum 2.0 had been built around the Mac-only Zoom framework, and so had to be reimplemented. The parts of Knowledge Forum 2.0 built with ZooLib were simply carried forward, with refinements and enhancements.

By this time ZooLib abstracted Mac OS, Windows and UNIX, so the server could be hosted on all three OSes.
The text editing in Knowledge Forum 3.0's note windows used the PAIGE text engine, which was a fairly crusty 60,000 lines by this time, but was portable across Mac and Windows. So although the rest of the client could be built for UNIX, without the text editor it wasn't particularly useful and we never released a UNIX client, although that wasn't a big deal in 2000.
Unfortunately we also had to abandon Kevin Parichan's excellent painting implementation, as it was also Mac-specific. Instead we used ZooLib's NPainter, which used only ZooLib features and was thus completely portable.

September 1997

ZOSWindow: Abstracting OS Window APIs

ZooLib had always maintained the ZFakeWindow abstract interface between user interface elements and the hosting environment. This allowed ZooLib UI widgets to be used within MacApp applications, standalone applications, Control Panels, XCMD windows and XObject windows. It worked very well.

However, when the portability axis was across platforms rather than hosting environments there was the potential for a lot of replication and ugly code in ZWindow, the standalone application derivative of ZFakeWindow.

So, I abstracted the interface to OS-windows, placing it in ZOSWindow, and modified ZWindow to use a ZOSWindow, rather than be conditionally compiled for different platforms.

April 1996

DigiPhone

The publishers of Digiphone, a VOIP application for Windows, took a shortcut to getting Mac compatibility - they (indirectly) bought NetPhone from Electric Magic, and had me add support for DigiPhone's communications protocol and audio codec.

Project Archives