The arbitrary syntax-directed editor seemed like a good goal once. Now, I'm not so sure.

As I wrote earlier, SDEs can prevent a certain class of input errors, but they have problems of their own. One issue is that they're generally tied to a particular programming language. I was curious to understand the gory details of that connection, so I cobbled together a bit of a project. I wanted to create a GUI-driven program editor for the arbitrary language, which would then be trivially customized to work with any actual language.

It turned out to be non-trivial.

At once, it's clear that an SDE needs to know the grammar of the language it's working on. But that's really not enough. The ultimate goal in an arbitrary SDE is not the compilation of the code, but rather its presentation to a separate compiler. That means you've got to convert a mass of "code notions" or "anotated syntax nodes" (pick your poison) into the corresponding, working, output. In the case of Python, that means a pretty-printer that knows how to indent code. You can see the wide space for variation.

It's not just a pretty-printer. See, bugs can still happen, but now your diagnostic messages are useless unless you can map code line numbers to ASN ids and look to see what broke in your editor. Nobody expects the arbitrary editor (except for C, which has directives to set the file and line number as reported by the compiler and debugger), so it's extra work to track down problems when they do happen. Still, for the faithful, this might be a minor trade-off in exchange for the prospect of tracking down far fewer bugs.

Another major issue is the screen display. What should an arbitrary SDE look like? Well, it's hard to say. An entire project would contain anotated nodes which represent entire files of output, and even the directories they're in. It would also contain nodes that represent double-quoted strings in Perl, which can contain arbitrary program code embedded within them. It's probably more difficult to pretty-print Perl than C for this and other reasons, but the on-screen display really needs to be founded in the idea of a pretty-printer for the target language. In a graphical environment, the precise mechanics of display may be different, but one way or another each syntax node must be placed and rendered in a careful arrangement with respect to all others. Furthermore, the mechanics of interactivity must be respected. How, for example, would the programmer select a particular line of code, or an expression within it? How would changes be made? And how many of these decisions really depend on the larger context in which the programmer is working?

Storing and operating on a tree structure is actually not that hard. Solving the complex user-interface dynamics of displaying and editing a computer program in an arbitrary language, but using only graphical interface elements to do so, is a rather difficult challenge. Putting the whole affair in a web browser is even more difficult.

You could argue for some sort of generalization to block, line, and span structures with suitable layout templates. This approach could work in some respects, but I haven't gotten that deep yet. In any event, it seems like it's rather complicated, and it multiplies the ways in which AST nodes (or their types as known to the programming system) must be customizable.

Another significant issue is response time for the user interface. You have about one tenth of a second for most things to seem done. Therefore, don't hit the network often, and try to do asynchronous write-back to a server-side main store for what you cache (lots of) locally. Use a large local workspace with enough function to keep the programmer busy for a few minutes at a time. There could be a tremendous amount of javascript, XUL, and XBL, or you might just settle for a text editor and CVS.

So for now, I'm going to settle.