Today I made a New Interface for Textual Expression, as part of a continuation of my master’s thesis research. Strokeweight is an interface that correlates gestures of drawing to gestures of text. Click below to view a demonstration video on Vimeo. Technical details can be found after the jump.

Strokeweight: Writing with fruit and Dunsany from Adam Parrish on Vimeo.

(The source text used in the demonstration video is Lord Dunsany’s Poltarnees, Beholder of Ocean, which has nothing to do with fruit. I don’t know why I drew fruit. I guess I drew fruit because it’s easy to draw, and also delicious.)

Here’s how it works.

Strokeweight requires a two-dimensional array of strings. I created this array by running a short story by Lord Dunsany through a Markov chain algorithm, and then using the Markov chain to populate the cells in the array. The algorithm I used works like this: A random seed token is placed in a random grid cell, and this token is passed to the Markov chain to generate values for the surrounding cells. The tokens in those cells are passed to the Markov chain to generate values for the cells surrounding them, and so on recursively, until the entire grid is filled. You end up with a two-dimensional grid that has tree-like “strands” of Markov-compliant text running through it in different directions.

The “Markov grid” thereby created is loaded into a Processing applet, which associates the position of stylus (or the mouse) to a position in the grid. The string contained in that position on the grid is sent (via OSC) to a Cocoa application that actually displays the text. Lifting the stylus from the tablet (or releasing the mouse button) tells the text display to insert a new line; leaving the pen up for more than a second or so will generate a second new line.

Here’s how the interface is set up.

The relationship between the drawing and the text is not literal in any way, of course, but it does manage to capture a number of drawing-related gestures and translate them into textual “gestures”:

• Small, slow movements of the stylus tend to move along (relatively) coherent sequences of cells, and therefore generate (relatively) coherent text
• Large, fast movements skip across sequences and create less coherent text
• Detail work in which the stylus is repeatedly lifted from the tablet, then replaced, creates staccato enjambments
• Lifting the stylus to “rest” effectively creates a new stanza

Overall, I’m satisfied with the results, especially for a day’s worth of work. I’d like to think about making adjustments to incorporate stylus pressure, and maybe adjust the “markov grid” algorithm to create coherent text more frequently (or ditch the concept altogether).

Bonus for people who have read this far: a complete reading of the poem written in the demonstration video, read by yours truly (192kbps MP3). I did this mostly to test out ITP’s swank new audio booth.