Random bits from Mike Fisk

Traveler's Manifesto

2009-06-09T17:22:00.005-06:00

When I check in to a hotel, there are few simple (cheap) things I'm looking for:

A desk with power outlets on the desk (e.g. in the lamp) for my laptop and cell phone
Decent lighting, especially near the desk
A shower I can stand up in
A shower head that is high enough to get my hair wet
A bed at least close to the same length as me (e.g. King only)

I just checked into a pretty new Hyatt Place "suite" that looks flashy with its 42" HDTV, media center, couch with ottoman, and granite wet bar, but it manages to fail at all of the first three items. I thought business hotels had figured this out by now.

Spot the Terrorist

2007-09-29T18:02:00.000-06:00

There are many definitions for "terrorist." I like this one: "one who utilizes the systematic use of violence and intimidation to achieve political objectives." But who's using intimidation and violence in these scenarios:

The dictator or the president who lies about the dictator's plans and invades his country resulting in over a million deaths
The student wearing a circuit board, or the police who threatened to kill her
The people calling for peace or the people saying that terrorists will win
The people placing advertisements or the people who think they're bombs

If you are scared all the time, even when people aren't trying to hurt you, then you are letting the terrorists win. Instead, hold the American ideals above all and refuse to give in to the terrorists and fear mongers. Don't elect candidates who run on a platform of fear.

Those Who Sacrifice Liberty For Security Deserve Neither. --- Benjamin Franklin

How to make sure I don't buy your product

2007-07-22T10:26:00.000-06:00

Promise to sue anybody who reveals flaws in it. HID Global has done this with their RFID-based proximity cards. Knowing about vulnerabilities in your access control devices would be a real bummer. Blissful ignorance would surely be preferable (not). HID's catchphrase is "HID -- The Trusted Brand". I guess in the absence of facts, your customers will just have to trust you.
When asked if you support platform X, tell me I should use a "real OS". BMC is an enterprise software company that brought some sales engineers on site just to tell me that. That was around 1996 and the OS was Linux. On their website today, Linux is the third item on their "Featured Technologies" list. Ironically, "Mainframe Management", their historical focus, is number four. I'm glad they've caught on, but I think I'll look for software from a company with "real vision".

Handheld AJAX

2007-06-05T15:06:00.000-06:00

For those of us trying to provide network application services to mobile PDAs and Smart Phones these days, we have a few options for how to write and structure apps:

Platform-specific client-server (BlackBerry, Palm, Windows, Symbio, etc.)
Synchronous browser
Synchronous remote display (e.g. VNC)
Java Client-server (J2ME Midlets)
Asynchronous browser (AJAX)

Platform-specific apps (#1) are a show-stopper for me --- there are too many platforms and they're too different. A plain browser (#2) is ubiquitous, but the high latency and low bandwidth of a lot of mobile networks makes it pretty painful to use. A remote display technology like VNC (#3) would suffer even more from network issues.

J2ME (#4) is a pretty nice technology --- although I don't think a lot of the VM implementations are designed to run multiple non-game applications in a user-friendly way. No offense to Java, but the world seems to like AJAX apps (#5) better, so for mobile e-mail, I'm going to try out Claros Mini.

The other tie-breaker is that options #1 and #4 require local storage and enable offline operation. Option #5 can optionally do that using technology like Google Gears, or it can work in a purely stateless mode to the client.

Windows Mobile Redux

2007-06-02T15:43:00.000-06:00

I finally broke down and bought a T-Mobile Dash (a branded HTC s620) running Windows Mobile. It has WiFi, GPRS/EDGE, 4 bands, Bluetooth 2.0, a QWERTY keyboard, QVGA screen, Micro SD, 1.3MP camera, stereo, etc. This device is better than the Palm I tried a few months ago and better than the Window Mobile iPaq I bought at work some time ago. But there is still plenty of work to do:

For Microsoft:

Upgrading the OS from 5.0 to 6.0 erases all of the files. What kind of lame OS upgrade is that?
There is no built-in functionality to backup to SD card (see above)
I can route laptops via a Bluetooth Personal Area Network (PAN) using the "Internet Sharing" app, but I have to poke that app each time I bring up my laptop --- and it only supports one laptop at a time. Better yet, I would like to bring-up the Dash's WiFi in AP mode and skip Bluetooth altogether.
The "ClearVue" Microsoft Office viewer apps are only viewers. I can't even make a simple spreadsheet. They're copyright Westtek, LLC. If Microsoft can't even write editors for MS office docs, it's definitely time to switch to OpenOffice.

For T-Mobile:

T-mobile doesn't expose the upgrade download to the phone's browser -- only to full-blown browsers.
T-mobile still wants to sell you SMS service in addition to Internet, so they want you to use their IM client that routes all messages over SMS. It's a good use of SMS, except that you have to pay separately for it. AOL is all too happy to fuel this business model, so they don't advertise any mobile IM clients. I'll take my IM over the Internet, thank you very much. OctroTalk is a free Jabber client with support for transports for AIM, etc. and works quite nicely. I don't know how T-Mobile expects you to IM if you're in a location served only by WiFi.
T-mobile really wants to capitalize on their HotSpot business unit, so they really want you to pay $10/mo extra for HotSpot access. You have to work hard to get a plan without it.

For other software authors:

The Java VM that comes on the Dash (Jeodek by Esmertec AG) only runs one app at a time and doesn't let you permanently allow an app to use the network without being prompted.
Google provides a nice Java client for Gmail, but it doesn't alert me when I have new messages or let me download attachments. The built-in POP/IMAP client might be nice, but Gmail doesn't support IMAP and POP would get me out of sync. Help, Google!
Minimo, the Firefox variant for mobiles, doesn't even successfully start for me, although others report better luck on the same device.

The best news is that vendors are getting better about providing "over-the-air (OTA)" software installs rather than just installers for a Windows PC that is supposed to then sync the app to the mobile. And T-Mobile EDGE service is giving me just shy of 200kb/s, which isn't too shabby for this sort of device --- broadband for my 486 with 1024x768 screen was 128kb/s ISDN. Although round-trip latency averages 900ms (ranges between 400ms and 2600ms), which hurts TCP and makes tty apps difficult.

The Green Grid

2007-04-20T20:41:00.000-06:00

Marketplace had a story on a tech consortium called The Green Grid, which is creating standards for measuring and improving server-room efficiency. The accompanying figure, which occurs in their white paper The Green Grid Opportunity, shows how 35% of the electricity used by a server can be lost to power and head conversions. Another rule of thumb says that cooling the machine room may double your electricity costs.

And kudos to Marketplace for providing text transcripts, as well as audio, of their segments.

Wikanalysis

2007-02-11T21:58:00.000-07:00

While channel surfing this weekend, I stumbled onto the documentary Press for Truth, which is based in large part on the 9/11 timeline that has been built at cooperativeresearch.org. The timeline is entirely based on open-source literature and every bit has a citation (usually multiple citations). I found it more useful than PBS's America Rebuilds.

How Often to Upgrade your "Enterprise Linux"

2007-02-02T13:52:00.000-07:00

When you deploy a Linux distribution in your enterprise, you have to keep in mind that it's different than deploying just an operating system. A good Linux distribution comes with all of the application software that you have to install and manage separately with a commercial operating system like Windows:

Office suite
Browser
E-mail client
PDF viewer
SSH Client
Compilers and interpreters (C, C++, Java, Perl, Python, FORTRAN, etc)
Streaming media player
Desktop publishing software
Instant messaging software
Diagram software
Image editing software
Photo management software
Drivers (network, printer, video, etc.)

So imagine if you installed all of the software that every user in the enterprise needed and then froze everything for a year or two. Certainly your IT shop would enjoy not having to deploy new software all the time. But the users and departments in many enterprises would revolt. Inevitably some project will have a legitimate business case for using some new feature.

So don't expect to install and freeze a Linux distribution across your enterprise for very long. The good news is that upgrading your distribution upgrades all of these applications at once and that your distribution provider should have done extensive integration testing for you. To maximize the value you get from your distribution, you should choose one that does frequent releases, keeps pace with upstream software releases, and makes upgrades easy. Unfortunately, some of the "enterprise" Linux distributions like RHEL are only released every 18 to 24 months. Debian's "stable" releases have been just as far apart. These "enterprise" distributions have very long-term support (7 years for RHEL).

I argue that what an enterprise needs is frequent high-quality releases that are easy to upgrade between --- not long-term support. Ubuntu offers an excellent compromise by making a release every 6 months or so, but also by providing a "Long-Term Support" release about every 2 years.

Not paying attention

2006-12-26T09:34:00.001-07:00

I continue to be underwhelmed by the hardware and software engineering in the mobile device market. A year or two ago I got a top-of-the-line hires iPaq running Windows Mobile (or whatever they call WinCE now). After that, I swore off Windows Mobile. For Christmas, my wife gave me a Palm LifeDrive. I haven't had a Palm since they were made by US Robotics about 10 years ago. I'm surprised how many things they've missed:

Single-threaded blocking operation. When the web browser is partway through receiving a page, the device is totally wedged. You can't stop the download, scroll the screen, or switch apps. In a few (or several) seconds, it may handle your input event. I'm not asking for threads, just a reasonable event loop and some non-blocking functions.

Memory Protection. Palms run on ARM chips that have MMUs, but individual apps routinely crash the whole system. In fact, it's less reliable than my ancient Palm.

Hardware Packaging. My LifeDrive is probably supposed to compete with iPods. But it's significantly thicker than my 2nd-generation 30GB iPod and an absolute brick compared to today's 30GB iPods. And they only put 4GB inside. Yes, it has a much larger screen, it's a touch screen, and it has decent speakers, but I really think they could have made this device half as thick.

Standard Connectors. My Motorola RAZR phone made me realize that every gadget should use a mini-USB connector for DC power and data. But Palm (and Compaq) stick with these proprietary brittle plastic connectors and a custom USB cable to connect to them.

It won't help the hardware problems, but ACCESS (the parent company of PalmSource, the software spin-off of Palm, Inc.) is (finally) developing a Linux-based Palm environment. I installed a community-developed Linux on my Palm, but the WiFi and Bluetooth aren't supported yet, so it's not all that functional. But the UI was much more compact and the PIM apps look just fine.

Anyway, I returned the Palm because it was basically useless for me.

Illogic First, Security Second (or third or...)

2006-07-25T19:19:00.000-06:00

The place that I work is supposed to be very conscientious about safety and security. Why? Well, because our sponsors manage based on metrics and compliance rather than harder to quantify measures of success and effectiveness, so they fixate on things like days of lost work and such.

But sometimes the place I work can't see the forest for the trees. As a cost savings matter, they are removing the security guards that guard the entrance to where I work. As a result, we, the technical staff, are being held responsible for enforcing physical security. This means making sure that people don't follow other people through, and generally acting like your a bouncer. We're not really thrilled that it's now our responsibility to be part-time security guards. If this was just a normal office building, it might be tolerable, but we have classified information and materials in the area and our security perimeter is presumably supposed to provide some challenge to a professional military or terrorist attack. For example, the guards that we've replaced carry pretty large guns around.

To make matters worse, the cost savings from eliminating guards at our building is being drastically overwhelmed by the creation of a new "toll plaza" of guards checking vehicle traffic to make sure you don't have a Ryder truck full of fertilizer. Of course, you'll be able to run that "toll booths" and quickly detonate your fertilizer anyway. So in my estimation it's net loss in safety/security and an increase in cost.

After it became clear that official complaining wasn't going to fix anything, I decided to declare a "Stop Work" today. Every employee is empowered (in fact obligated) to declare such a thing when they see unsafe or insecure behavior. The apparent official reaction was to over-rule my "stop work" without any resolution, as far as I can tell.

So what is the real safety and security culture here? Well, on my way home tonight, the traffic backs up on the main street and one guy partially blocks the intersection (with a light) to our parking lot. I honk at him for breaking the law by obstructing the intersection and blocking those of us who now have a green light. His reaction is to get irate and yell at me. That seems to sum up my day pretty well.

Get a Second Life

2006-06-11T20:23:00.000-06:00

It's nice that Google puts their colloquia up on the web for people to watch. They had an interesting talk from the folks at Second Life, an online VR environment that has created an actual economy of its own. The talk made it sound pretty interesting and I've played with the Flatland environment, so I thought I'd take a look. There's a lot of collective work that's been put into it, but to be honest, I couldn't find any of the neat things they described in the talk. But the talk is definitely worth watching.

Battle of the Lambdas

2006-05-31T21:21:00.000-06:00

My colleagues and I are debating the relative merits of C++ and Python for an upcoming programming project. As an example of the "codability" of both languges, here's how I would implement function passing to iterate over a container in a thread-safe way:

Python:


class AtomicList:
    def __init__(self):
         _list = []
        _lock = threading.Lock()

    def append(self, x):
         _list += x

    def foreach(self, f):
        self._lock.acquire()
        for x in _list: f(x)  
        self._lock.unlock()

class Element:
    def __init__(self, x):
       self.value = x

    def printer(self):
        print self.x
   
MyList.append(Element(1))
MyList.append(Element(2))

MyList.foreach( lambda x: x.printer )

C++ (with STL), the typing is static and the code really isn't any longer or more verbose. But the biggest problem is the bind() syntax and the fact that C++ wasn't designed to syntactically support lambdas and it's only because of templates and the type inference that a C++ compiler has to implement that you can trick it into doing these things.


template <class T>
class AtomicList : LockClass {
    std::vector<t> _list;

    void append(T x) {  
        _list.push_back(x);
    }
    template <class C>
    void foreach(C x) {
        LockClass::lock();
        std::for_each(_list->begin(), _list->end(), x);
        LockClass::unlock()
    }
};

template<class>
class Element {
  public:
    Element (T x) : val(x) {}

    void printer(int x) {
        printf("%d\n", x);
    }
};

MyList<foo> intList;
intList.append(Element(1));
intList.append(Element(2));
intList.foreach( bind(&Foo::printer, _1) );

Higher-Order Programming and/or Dynamic Typing

2006-05-29T21:52:00.000-06:00

What do the Cocoa, Qt, and Glib/Gtk GUI toolkits all have in common? All three are event-driven frameworks and all three provide the capability to register callbacks and object attributes dynamically using string names. And all three use plain C or C++ for most of their core libraries.

While Qt does use C++ templates, it doesn't use them for Signals and Slots, their event mechanism. Instead, Qt's Meta Object System uses a C++ preprocessor to insert code that allows object introspection and performs a lot of type checking at runtime. They point out that modern C++ compilers could do most or all of this with templates, but they have portability concerns.

Apple's Cocoa uses the SmallTalk-like messaging of Objective C to allow the sending of arbitrarily-named signals between objects.

Glib uses a plain C runtime in which almost everything seems to be a "void *" with (hopefully) some run-time checking. In my opinion, glib duplicates too many things. For instance, instead of using (and defining when not provided by the system) ISO/IEC 988:1999 stdint.h types, Glib defines its own "gushort" and the like. Because there is almost no syntactic sugar (aside from a few basic macros), the Glib syntax is pretty cumbersome.

Why do I care? Well, for two reasons. First, smacq objects have attributes (fields in my terminology) that are nominally described by strings, but in fact all operations in the run-time use numeric identifiers for efficiency. Numeric identifiers are assigned at run-time in no predefined order, so modules have to explicitly lookup the number for the field(s) they want. They are supposed to then save the numeric identifier and use it for the duration of the program. Is the efficiency gain of numeric identifiers worth the pain of this added translation step? I'm not sure. Can I provide some clever syntactic sugar to make it easier to do these lookups? Probably.

Second, there are a lot of container data types in the smacq implementation (mainly using STL classes). To make access to these data structures thread-safe without trusting the caller to lock and unlock objects, I want to use higher-order programming where the caller provides a basic block of code to be applied to each element in the container. In C terminology, that would be a callback function. But I don't want to have unchecked "void *" pointers running around to encapsulate callback parameters.

So instead, I'm using the Boost Lambda Library which is pretty close to letting me pass lambda functions into my containers' foreach methods.

One thing that's not obvious about C++ templates is that you can declare a function (not just a class and method) that is truly polymorphic --- it works on any type. When the compiler comes across a place in the code where such a function is called, it determines whether or not the types being used with the function are adequate (e.g. provide the right methods and such). If so, it generates an instance of the function specifically for that type. Multiple instances may be created for different types. Each of those instances has its own copy of the code, but at run-time there is no type-checking or delayed binding that has to be done.

As a result, a C++ compiler these days has a pretty powerful type-inference engine in order to deal with both templates and overloading. The Boost Lambda Library makes extensive use of this. Unfortunately, C++ still wasn't designed for lambdas and higher-order programming, so the syntax is painful and there are significant limitations.

So how does this fully C++ approach compare to the dynamic approaches of the GUI toolkits. Pretty well, I think. They require that you have a callback function (or even a whole object) that is declared like any other function. That's syntactically annoying for one-liners. Type checking on the callbacks seems to be much weaker (even when including run-time checks).

The only lingering question I'd like to see is a performance analysis of replicating statically typed code versus performing simple type checks or delayed binding in a smaller code base. The prevailing wisdom seems to be that smaller is faster. If you look at the question of function inlining, which also increases code size, the ratio between CPU performance and memory performance seems to be wide enough (and purportedly widening) that non-trivial functions shouldn't be inlined.

Progress and yet, no progress

2006-04-10T10:03:00.001-06:00

Suppose you have a simple goal in mind: You want your Mac laptop to be able to securely mount a filesystem on your home server, maybe a Linux box. The Mac can mount all kinds of network file systems: NFS, FTP, SMB, WebDAV. But what if you want some strong authentication and privacy? Well, FTP is clear-text only, SMB has some better authentication, if configured right, but I believe is subject to man-in-the-middle attacks. NFS can use Kerberos, but you have to setup the infrastructure, which is a bit much for home. WebDAV can use digest authentication and can use SSL certificates to avoid man-in-the-middle attacks and, uniquely, to provide privacy.

Great, so I tried using WebDAV, but I can't find a WebDAV server for Linux that lets users modify files in their home directories. Instead, it seems that the assumption is that you want WebDAV access control to be orthoginal to native filesystem permissions.

So basically, there's no easy way I can export a filesystem securely to my Mac.

There is the Davenport servlet that serves up SMB filesystems with WebDAV. Combined with Samba, that might work, but is more complicated than it needs to be.

There's also the Perl Net::DAV::Server library, which purports to do this, but doesn't seem to work, at least when using the MacOS Finder as the client.

Finally, I can use FTP over MindTerm SSH's transparent FTP to SFTP bridge, but that requires that I start up SSH and authenticate in advance.

Better Co-routines in Python

2006-03-24T06:54:00.000-07:00

The upcoming version of Python, Python 2.5, will include more full support for co-routines by enhancing Generators, which already have a yield statement which allows them to yield execution to the caller. The new additions allow better exception handling and, more importantly, the ability for data to be passed back and forth when a Generator yields or is resumed.

This is an excellent addition (although I'm not sure I like the name of the send() method to resume a generator).

Dirty Little Secrets of SSL Certificate Authorities

2006-02-18T09:07:00.001-07:00

There was recently an incident where a Phishing scam acquired a certificate binding www.mountain-america.com to "www.mountain-america.com". That violates the intent of X.509 certs to bind a common name to a real-world organization. For example, the certificate for my bank binds www.lanb.com to "Los Alamos National Bank". But apparently, in the name of making a buck and making SSL cert issuance fast and easy, CAs like GeoTrust are now issuing certs that just bind a domain name to a domain name. That provides some protection against man-in-the-middle attacks against your site, but doesn't leave the user with any way to tell who you are.

The other thing that many people don't realize is that SSL authentication in browsers is, like many things, only as strong as the weakest link. My late-model Firefox browser has pre-loaded CA certs from 39 different organizations. If any of those organizations issue a bad cert, it doesn't matter how good the others are.

This came up many years ago when I was trying to buy a cert from Thawte, the South African company (since sold to Verisign, an implicitly trustworthy American company), for Los Alamos National Laboratory. Some of the security folks were weary of trusting a South African company. It was hard to explain to them, that we're not trusting anybody by having them sign our private key and issue a certificate. It's all the users and their browsers that have to trust Thawte, but the browsers implicitly do.

So practically speaking, what assumptions can you make about SSL certs these days? About the only thing you can tell is that the domain-name in the browser's URL is really the domain name that you are talking to. This provides some end-to-end protection against DNS attacks and other man-in-the-middle attacks. But don't assume squat about the legal entity you're talking to. If you actually open the cert and look at it, you should look at the Organization field and trust it only if you trust the Organization of the Issuer (the certificate authority).

Cisco

2006-02-16T17:56:00.000-07:00

I just had the displeasure of spending the day with a team of Cisco reps and engineers. Maybe it's to be expected, but they're pretty institutionally conceited. They're not all that current with their competitors capabilities, but are quick to dismiss them in every way. And they're no longer a networking company. They're like Microsoft and are in the client and software business now. I wanted specs on their protocols and APIs for their Cisco Trust Agent, their RADIUS server, and their GAME protocol for communicating between the RADIUS server and third-party systems. They're not interested in sharing any of those things with customers. They describe them as "open standards", but that means that if you sign a contract and NDA as a business partner, then they'll give you access. I asked if their APIs came with sample open-source code and started to get a lecture on how you shouldn't trust open source code for security systems. Too bad they're not a networking company anymore.

Security Vendor Tidbits

2006-01-17T22:19:00.000-07:00

Arbor Network's PeakFlow now uses agents on authentication servers to associate user identities with IP addresses for correlation with flow data.

Authenex has joined the token market with a USB fob that includes reader-less smart card support, AES encryption, and sequence-based one-time passwords, and an online challenge-response protocol.

U.S. Customs Opening International Mail

2006-01-16T08:49:00.000-07:00

For those of you keeping track of your diminishing civil liberties, here's another:

U.S. Customs Opening International Mail: "Reuters is reporting that Customs and Border Protection is opening international mail coming into the U.S. without warrant. "

Experiments in social aggregation

2005-12-31T15:14:00.000-07:00

I spent some time today playing with some of the new news/blog aggregation services (e.g. search engines) on the web. TailRank is a "weblog indexing zeitgeist".

Topix.net and Findory.com takes a Google News approach to things, but with more locality features and the ability to look only at Blogs, Gadgets, and some other categories that Google doesn't have. MemeoRandom only has categories for Politics and Tech but does a nice job at combining mainstream articles and Blog entries.

Digg takes a manual, but distributed, approach where user voting is used to rank stories in their single list of stories. Basically it's a popularity-based Slashdot (rather than a strictly chronological one).

Tag-based sites rely on people categorizing their own links using freeform category names. Sites like del.icio.us, Furl, and BlinkList depend on people storing their bookmarks online, while Technorati uses tags embedded in Blogs all over the web.

Frankly, I don't find the notion of categorizing my bookmarks as very interesting. But I do like Technorati's approach of harvesting additional meta-data from Blogs and other "news" sites.

On the separation of software stack and operating system

2005-12-28T09:18:00.000-07:00

Modern Linux distributions are popular because they provide a good operating system and applications bundled together. In fact, a lot of us switched from operating systems like Solaris which did not provide a good bundle of applications. But what if we decoupled OS and application again? What if I want to run OpenSolaris or OS/X as my operating system because of support for specific hardware platforms? Or what if I have to run a specific release of RHEL because that is the only thing what my hosting service provides? Or what if for some strange reason, I need to run on Windows?

In each of those cases, what I need is an application stack that is
independently supported and regression tested and agnostic to underlying operating system. Open-source application stacks now exist from OpenPKG and SpikeSource (based primarily on OpenPKG, but offers extensive commercial support). OpenLogic's BlueGlue takes a similar approach to distributing a complete development environment for software developers. Other companies like SourceLabs focus on open-source web services environments.

Amateurs and Professionals: Blogs, Open Source, and Startups

2005-12-18T21:58:00.000-07:00

IT Conversations has an excellent talk by Paul Graham at OSCON 2005. Paul compares the motivations and effectiveness of traditional, salaried "professionals" such as "software engineers" and "reporters" to individual contributors who work on open source software, blogs, and early-stage startups. He notes that these "amateurs" buck the standards of the 20th century by working out of their garages and being incredibly productive compared to professionals based in sterile offices who coast through their days with meetings and interruptions. Paul also points out that "amateur" in its original Latin and French meaning was about being motivated by passion.

Windows Mobile

2005-12-18T21:25:00.000-07:00

I have this really nice piece of hardware, an iPaq 4700 with a full 480x640 color screen. Unfortunately, it's next to useless because it runs Windows Mobile. (The Linux kernel boots on it, but not all of the drivers are working yet.) Windows Mobile (formerly known as Win CE) is a real piece of work. In the user interface department, it wastes huge amounts of screen real-estate with decorations taken directly from the Windows desktop UI. The browser can only do one thing at a time (single window, no tabs, only one download at a time). Even worse, pretty much all Windows Mobile software is designed to be installed on a Windows desktop and then sychronized onto the mobile device. So much for being "mobile" if you have to be tethered to a PC with a cable. Since the device has both WiFi and Bluetooth, it certainly doesn't need a PC for connectivity. And even if I was willing to use a PC, I don't have a Windows PC!

oneSIS

2005-11-17T15:45:00.000-07:00

oneSIS is a tool for managing shared-filesystem (e.g. NFS root) heterogeneous clusters. The basic idea is like the old Sun install CDs --- it's a normal system image that's read-only and all of the files that are writable (e.g. /var/run) are symlinks into /tmp, which is populated at boot time. oneSIS uses the same approach for files that need to be different on different classes of machines: the file on the server is made into a symlink to /tmp, which then gets a link to the proper file on the server.

Sandia developed it and Ron Minnich's group plans to use it to manage filesystem their Linux Xcpu system.

Cyber Infrastructure

2005-11-17T15:23:00.000-07:00

Cyber Infrastructure (CI) is the US name for what the UK likes to call e-Science. It's basically the enabling technologies that let scientists get work done using networks. Some people would call it the natural progress of the Grid, which focused more on how to run jobs and use HPC resources, but not so much on Computer Supported Cooperative Work. I ran into a friend of mine from UCSD who is working on the CIchannel.org, a video archive of presentations on CI. Neat stuff.

Agility

2005-10-29T08:59:00.000-06:00

I listened to a couple of good podcasts on agile software engineering methods. Alistair Cockburn gave a nice overview of Agile Software Development and Scott Ambler's "Are you Agile or are you Fragile?" was a good follow-up. Agile methods are typically iterative. Before each iteration, the stakeholders prioritize what they want done. The implementers then bite-off some of the items from the top of the list to implement during the next iteration. Because there is not a lot of time spent doing top-down whole-project up-front designs, iterations are frequently expected to involve refactoring of existing code to support the implementation of new functionality.

These concepts resonate with my experience, but I've always worked in groups that do "systems engineering" or "systems integration" rather than large-scale software development. We have all the same issues of how to allocate resources and deal with frequent requirements changes. I'd like to see more case studies in these other areas. But intuitively I agree with the wisdom that agility is competitive advantage and lack of agility means you're not satisfying your stakeholders. And in my world forecasting requirements gets harder by the day.

Zimbra and Client-Side Storage for JavaScript

2005-10-27T21:43:00.000-06:00

Zimbra is an open-source (mozilla-like license) AJAX web application suite that uses popular open-source servers in the backend. It takes Tooltips to their natural extreme. Anything that can be identified becomes hoverable: people names do directory lookups, times and dates do calendar lookups, addresses do Google Maps lookups, etc.

So this is pretty cool stuff for a networked collaboration suite. But how excited should we all get about AJAX? It has some serious limitations in terms of caching data for disconnected operation. Most browsers would be hard-pressed to cache even the JavaScript itself. A heavy-weight IMAP client can store and syncronize data and JavaScript can't (yet). So what extensions would be required to securely consume client-side storage? The storage could be sandboxed from all other storage on the host computer. Access to the storage could be limited to only authorized JavaScripts.

But the more natural way for a web app to store things is to a web server. So maybe the answer is not language extenstions, but that every client should run an HTTP-based (maybe WebDAV) server that will store and/or cache web objects.

You're next computer: $1 and a 24-month service agreement

2005-10-26T20:52:00.000-06:00

We recently drove out to Davis, CA to pick-up some horses that Gina imported from Germany. With the better part of 4 days worth of driving, I decided to load-up the iPod. I had been buying audiobooks from Audible.com, but used iTunes' relatively new support for podcasts to find some interesting non-fiction to listen to. Books and podcasts are an excellent way to keep your brain awake during hours of driving (much more effective than music). I'll blog some interesting highlights here and this is the first one:

The wonderful IT Conversations series of Podcasts had an interview with Sun's COO Jonathan Schwartz where he talks about Sun's business model and that of his competitors. He makes some interesting points about the role of operating systems and software as give-away technology that enables you to sell services --- the same model in which cell phones and set-top boxes are given away for free in exchange for the opportunity to sell you content and services. From a business perspective, you look at your Net Present Value rather than just looking at the sales-price of the system.

Crapped in the Hallway

2005-10-26T20:03:00.000-06:00

Marty Bahamonde of FEMA recently testified in front of a Senate hearing about FEMA and Hurricane Katrina. He supplied printed exhibits of pictures and correspondence while he was in New Orleans. As reported by the AP, Michael Brown's Press Secretary sent the following e-mail:

Also, it is very important that time is allowed for Mr. Brown to eat dinner. Gievn (sic) that Baton Rouge is back to normal, restaurants are getting busy. He needs much more that (sic) 20 or 30 minutes. We now have traffic to encounter to get to and from a location of his choise (sic), followed by wait service from the restaurant staff, eating, etc.

To the coworker who forwarded the e-mail to him, Marty replied:

OH MY GOD!!!!!!!! No won't go any further, too easy of a target. Just tell her that I just ate an MRE and crapped in the hallway of the Superdome along with 30,000 other close friends so I understand her concern about busy restaurants. Maybe tonight I will have time to move my pebbles on the parking garage floor so they don't stab me in the back while I try to sleep.

3D Rendering APIs

2005-09-10T14:19:00.000-06:00

LWN references a nice survey of the current rendering infrastructure used on Linux and other X-Windows platforms: The State of Linux Graphics.

Operating Systems at LANL

2005-09-10T13:08:00.000-06:00

A milestone in the rejuvenation of OS research at LANL has occurred. I attended a talk by the Performance & Architecture Lab that surveyed some of their postdocs' recent work on things such as incremental kernel-based process checkpointing that only checkpoints dirtied pages and page replacement algorithms to prevent thrashing.

Data Models

2005-09-04T20:01:00.000-06:00

The Unix-based open source software community is an interesting barometer. Some say that it doesn't innovate. I wouldn't go that far, but I would say that a lot of projects are designed to be the last project in an area. By that, I mean that they "finish" the concept, make it efficient, make it portable, make it all-inclusive, and remove the bullshit. So I find it interesting when hyped-up concepts show up in these projects.

I recently noticed that data models are making their way into open source projects. Gnucash is moving towards SQL access for all data. They have spun-off the Query Object Framework (QOF) library for performing simple SQL queries on in-memory GObjects. The goal is for QOF to transparently interface with and/or cache results from backend SQL databases. Another related project is Data with Interaction (DWI), which provides an Object-Relational Mapping (ORM). ORM has been popularized by Java projects like JBoss. Basically, you specify (typically in an XML markup) how your data is to be interacted with and the system builds an API (and GUI) as appropriate. I had dismissed the JBoss approach as being too painful for the benefit. Maybe I'm wrong.

Asyncronous Publishing and Browsing

2005-09-04T15:32:00.000-06:00

At SIGCOMM this year, it was clear that there is some popularity to the idea of adding DHTs to the set of is ubiquitous protocols that applications are built on. The notion is that people don't want to just exchange packets, but that huge classes of applications want to treat the network as a giant, shared hash table. This idea might seem comfortable for people who like programming in languages like Python, where the hash table (dictionary) is the fundamental data structure.

My first reaction was that this is a bad idea. But a few semantics are crucial to many Internet users, but not provided by IP. These semantics are requisite for frequently-disconnected operation (especially common for mobile devices and people) is problematic. As Vint Cerf and Bob Kahn noted at their Turing Award Lecture, the Internet was not specifically designed for disconnected operation. Some application layers, notably e-mail, have addressed this problem. E-mail is asynchronous with queuing at both ends and intermediate hops. It is reliable. It uses global addresses that are not associated with routing. And when everything is online, it's near real-time.

Usenet news is another powerful example of offline message distribution. Messages are posted to "newsgroups" that have their own hierarchical name space. For those of you too young to be familiar with Usenet infrastructure, basically you sent a post to your local server. Periodically, your server would connect to a pre-defined set of other servers and exchange any messages that only one side or the other had. Originally this was done using uucp messaging, which may have involved calling each of the other servers with a modem. Periodically, servers delete old messages to free up storage.

The asynchronous web

Now what if I wanted the web to have some of these features? What if I wanted to click on something and, if I wasn't on the network, or the provider wasn't on the network, have my browser queue that request and execute it later. The browser could keep a table of contents for these pages that have been downloaded and are ready for use. It would probably keep the page that I clicked on to jog my memory why I downloaded the page. Prefetching, as now implemented by Firefox and Google, would be more important in a disconnected environment.

But those browser changes only implement one side of the asynchrony. What if the client and server were never up at the same time? Should there be some stateful network service (unlike IP packet forwarding) that can be used for message queuing? A centralized or distributed service could provide for the storage and distribution of content, rather than having everybody on the Internet connect to your personal web server. The best solution with the current publishing model is to register a personal domain name and then use a hosting service to serve up your content for that domain. You can manually move your domain and content from one service to another. If you don't purchase your own domain and use a cheaper (or free) service like Blogger.com, yahoo.com, geocities.com, then you're stuck using their name to identify your content and you don't have identifier portability, you can't move your content elsewhere without renaming it (similar to the problem of having to change phone numbers if you switch to a new phone company).

Portable Identifiers

And while we're on the subject of identifiers, there's a crusade for object identifiers (URNs, ISBNs, DOIs) that are permanent identifiers for documents, regardless of who is currently publishing there or where (an ISBN is the even if you get the book from your library, Amazon.com, or straight from the publisher). These IDs could be hierarchical and memorable (like e-mail addresses), centralized (like ISBN numbers) or purely random (like MD5 hashes). I encourage all authors to use DOI's in their citations.

Many peer-to-peer system use an MD5 hash of a file as its unique identifier. Similarly, I could post my e-mail or Blog entries as documents identified by a pseudo-random ID number (e-mail clients already assign Message-IDs) and then just send references of those documents to you. One departure from many peer-to-peer systems, however, is the need for dynamic content. For example, I would then like to publish a rendezvous page (e.g. what's currently at http://mikefisk.bogger.com/) that has a constant identifier --- regardless of the hash of the full document.

Publishing in the Future

Should you have to buy a domain name and setup a hosting service in order to publish content? This is a significant barrier to entry. Is there an economic model for a different namespace and protocol for publishing things? The peer-to-peer community is examining distributed services where you provide what you can, when you can, and so does everybody else. There is a lot of evidence that for popular content, protocols like BitTorrent can make sense. Would this scale to less-popular content?

And that brings us back to the DHT. If I can publish content based on a pseudo-random key (such as an md5 hash of the content), maybe it is a good supporting layer for message queuing and object delivery. Maybe my asynchronous browser would be better off using opendht:// URLs instead of http:// URLs.

Katrina

2005-09-04T12:07:00.000-06:00

Every bozo has an opinion about the social response to Hurricane Katrina, which recently hit New Orleans and other portions of the gulf coast. Not long before this event, I happened to see a Nova documentary all about it. In case you've been hiding under a dry rock, the problem is that New Orleans is below sea level and sinking as the swamp land that it was built on continues to dry out from being protected from Lake Pontchartrain. So if you overflow or break a levy, the city drowns. That's what happened.

Federal response: Why is that New Orleans seems to be the only place that the Bush administration was not willing to invade. I heard one excuse that they were waiting for law enforcement to get a handle on looting before the national guard could come in and help with rescue and clean-up. This is the National Guard --- the state militia! A disaster has been declared. Martial law has been declared. Just send them in already. (By Saturday, this finally seems to have happened).

Local response: I know that it's hot and humid and that everybody is hungry, thirsty, and in many cases sick. I also think it's okay to loot groceries under these circumstances. Some news footage shows guys running out of stores with new clothes. That seems over the line. But the stories of street gangs, shots being fired, etc --- why do we seem to handle this so much more poorly than third-world countries. My guess is that we're more spoiled and have a bigger sense of entitlement.

Woe is the Stateful Web

2005-07-21T19:47:00.000-06:00

Today I had to deal (again) with linking to the ACID web application for viewing Snort alerts. ACID has this nice "feature" that you can't just link to query results. First you have to go to the blank query page and get a cookie assigned to you. So now it's time to restate some of my issues with web developers these days:

First, too many web developers have no understanding of SQL injection, shell injection, and other all-too-common vulnerabilities in their code. I recently had to deal with the aftermath of some code written by the local instructor of a web programming course at the university. I shudder to think about what his students have learned and not learned....

Second, most web developers think that they can and should write web apps the same way you write a one-tier application that contains UI, business logic, and backend all in one tidy package. To approximate this, they use paradigms that depend on server-side state persisting between connections. Now I'm not a fan of cookies either, but form data (hidden or otherwise) is a perfectly valid way to preserve untrusted state in the client. And because it's on the client, there is no storage requirement for the application provider and no need to arbitrarily expire state. Take the ubiquitous shopping cart as an example. If your site doesn't keep track of this across sessions (e.g. stored in a database by username), you might as well have the client keep track of it from page to page. When the user finally asks to view their cart, you validate all of the saved input (e.g. part numbers) and display the results.

The combination of these two mis-understandings leads to a particularly common flaw that drives me nuts. When programmers validate input on one page and then keep passing it as a trusted value in a hidden tag in subsequent forms. Any network programmer should treat the client as a malicious --- either because it's an attacker, or just because it may have arbitrary software bugs that you need to tolerate somewhat gracefully.

HTTP was designed as a stateless protocol. There are powerful benefits to stateless protocols: fault tolerance, parallelization, etc. But you have to use it right. If you don't, you break those things and create security vulnerabilities. The web is a different beast from traditional user interfaces and traditional programming methods shouldn't be forced down its throat.

Virtual PC: as bad as the real thing

2005-06-26T13:55:00.000-06:00

So here I am trying to do some "simple" things in Virtual PC. Unfortunately, I'm a prisoner to my software since my version of "simple" isn't expected by Microsoft:

For example, I have "shared network" enabled (think NAT) for virtual hosts. Unfortunately, I have no way to make my Mac talk to the virtual hosts. They're in the 192.168.131 subnet, which has no route or virtual network interface. So packets to the virtual host go out the default route and into the ether. But a Linux virtual host can talk to the Windows virtual host.

And by the way, there appears to be no way to change the subnet used by Virtual PC's DHCP server.

Virtual PC also takes over the Mac firewall in order to do its own thing --- so much for having a firewall. What a nice "feature".

How the hell do I do Alt-F2 to change a Linux virtual console?

Bring on the VMware for x86 MacOS...

Asynchonous event scheduling

2005-06-25T19:36:00.000-06:00

I just re-read the fine paper "Multiprocessor Support for Event-Driven Programs" from USENIX 03. This is a must-read for any systems person. The paper describes the implementation of libasync-smp. A few things come to mind when relating this work to my pet project, SMACQ:

First, both use the asychronous event-driven design pattern. Both use two-level work queues: libasync-smp has "cpu" callbacks and "fd" callbacks, where SMACQ has "consume" callbacks and "produce" callbacks. But the SMACQ scheduler doesn't currently have a main select() loop to determine when "produce" callbacks should be run. Instead the focus is on consuming what you have and only then producing new data. This implies that input file descriptors have to have sufficient buffering on their own --- something that is difficult if you're a web server accept()ing TCP connections at very high speed.

Second, it is unfortunate that parallelism in SMACQ is limited both by SmacqModules, which aren't supposed to have to be thread-safe, and DtsObjects, which aren't thread-safe at the moment (but have been in the past, and easily could be in the future). Coloring (in the paper's sense) events by SmacqModule and implementing locking in DtsObjects is sufficient for thread-safety (outside of the core). However, affinity by DtsObject is likely to be more useful than affinity by SmacqModule since a call with a DtsObject argument is all but guaranteed to want to use that object, while not all modules have local state, and many that do have state that does not change (and is therefore easily cached). In contrast DtsObjects get re-used and modified frequently. Thus, a multi-threaded SMACQ scheduler should probably map events to a queue using the DtsObject (the pointer value divided by the alignment, probably) rather than the SmacqModule. However, this means that both SmacqModules and DtsObjects need to be locked. This is doable, but not very pleasing.

The other approach to parallelism in SMACQ is closer to the SEDA work since it breaks execution graphs into sub-graphs that can be run independently with objects copied from one graph to another at the places the graph is partitioned. This is the approach that Bill Weiss is attempting to implement with GASnet. The advantage is that no locking is required and shared-memory isn't required. The disadvantage is that it only takes advantage of certain, very embarassing, sorts of parallelism.

The return of the Interactive White Board

2005-06-24T12:05:00.000-06:00

Around 1994 we covertly installed white-board material on the entire back wall of our office at the TCC. The 12 foot by 8 foot white-board became a place for semi-permanent notes like "_{\ni}" (the LaTeX syntax for a subscripted "such that"), todo lists, and random notes to us from visitors. Shortly thereafter, I made a CGI script called the "Interactive Whiteboard", the contents of which are archived for posterity by the Way-Back Machine.

From 1995 to 1998, I occaisonally updated a "What's Happening" page on my website.

10 years later, I'll try this "blog" thing....