Data Determinism

[Note: this is a post from a now defunct blog. I’m repub­lish­ing it because of this recent arti­cle about Google, “Google design­er leaves, blam­ing data-cen­trism”. It is not a defense of what appears to be Google’s data fetishism (but who can blame them?); it just got me thinkin’ …]

I want to say that the usu­al prac­tice of begin­ning with the user inter­face, as the arti­fact that guides the con­ver­sa­tion between clients and devel­op­ers, is wrong. It looks good and sounds right: The client will be using the inter­face, right? The code is a black box that ought to be sub­ject to intense refac­tor­ing, right?

True all, but we also know that the stack of appli­ca­tion devel­op­ment rests on the data­base lay­er, moves into the var­i­ous lay­ers of code and so-called busi­ness log­ic, and ends with the user inter­face. The lay­er that has the most effect on what is pos­si­ble to code or dis­play down­stream is the data­base lay­er. That is, what you choose as a data­base for­mat and mod­el will con­strain what you can do on the pre­sen­ta­tion end, where­as pre­sen­ta­tion tech­nolo­gies rarely have effects in the oppo­site direc­tion (a pos­si­ble excep­tion is Flash before Flex). This stack holds even if you’re not using MVC; it’s more or less encod­ed in the way our cur­rent soft­ware tools work. For exam­ple, even if you code in pure JavaScript, your stack is going to begin with JSON or XML or the DOM, and end with CSS.

Call this view “data deter­min­ism.” As such, it prob­a­bly suf­fers from the same crit­i­cisms that have been lev­eled at oth­er forms of deter­min­ism, such as his­tor­i­cal mate­ri­al­ism, which holds that infra­struc­tre (work behav­iors, tech­nolo­gies, etc.) deter­mine super­struc­tire (reli­gious beliefs, laws, etc.). I’ll accept that, if you (the crit­ic) accept that the data lev­el at least con­strains the oth­er lev­els, and that the oth­er lev­els, to have an effect, must be able to mod­i­fy the data lev­el. Well, then, there’s the rub: once the data mod­el is writ­ten, it does­n’t get changed a whole lot. The web design­ers don’t have a lot to talk to the DBAs about, and the two groups rarely know how to have a con­ver­sa­tion. In fact, with enter­prise data­bas­es, you get the tail-wag­ging-the-dog effect: “We can’t do that because the data­base only accepts this kind of data.” So there.

Any­way, I believe that the con­ver­sa­tion with clients should begin with the data mod­el, using per­haps sim­pli­fied E‑R dia­grams, but ulti­mate­ly get­ting at a kind of ontol­ogy. What are the salient cat­e­gories and rela­tions and process­es that describe the domain in ques­tion? This is a con­ver­sa­tion that clients can have with devel­op­ers, and usu­al­ly it is a great con­ver­sa­tion, and not con­strained by an arbi­trary visu­al arti­fact that can direct con­ver­sa­tion along a false groove.

A cou­ple of prin­ci­ples fol­low from data determinism:

1. The require­ments process should be pre­ced­ed (replaced?) by an ontol­ogy-dis­cov­ery process. And the prop­er method to use here is ethnography.
2. The data­base should be designed with flex­i­bil­i­ty in mind. I pre­fer very sim­ple seman­tic web struc­tures (triples, graphs, etc.) that can be fil­tered by more spe­cif­ic ontol­ogy lay­ers. This process I want to dis­cuss in anoth­er post or two.
3. After this is in place, then the dis­cus­sion should move to the lev­el of visu­al arti­facts, such as inter­faces. Both the client and devel­op­er will have a bet­ter idea of what is possible.

This is essen­tial­ly a cod­i­fi­ca­tion of the appli­ca­tion devel­op­ment process I’ve devel­oped with clients in acad­e­mia, where I have devel­oped sev­er­al web-based appli­ca­tions for human­i­ties com­put­ing projects.

Look­ing back at this post, I see that it has sig­nif­i­cance for two things that cur­rent­ly occu­py my mind: Edpunk and the RAW DATA NOW move­ment (both, inter­est­ing­ly, rep­re­sent­ed by folks at UMW …)

1. Data deter­min­ism pro­vides an under-the-hood ratio­nale for Edupunk: Enter­prise apps tend to hide data and the data­base, reify­ing it into a nat­ur­al, immutable con­di­tion that inter­faces and behav­iors have to con­form to. Think of Black­board. Also con­sid­er that it was over the data mod­el that they took Desire2learn to court.

2. Data detem­i­nism also helps explain why data, among all the things we call “infor­ma­tion,” needs to be free (in spite of the fact that it appar­ent­ly does not want to be). RAW DATA NOW, as Tim Bern­ers-Lee recent­ly exhort­ed the auidence at TED. Because if it deter­mines every­thing, we need to have access to it — raw, and with­out undue medi­a­tion by nice-look­ing inter­faces or toothy EULAs.

Posted in Software Design | 5 Comments »