Bubbles_glow

≡ Literary Systems ≡

 
Corner_fold
  • Citation
  • Extension
  • Response
  • Comment
Hopo_catalog
Literary Systems
 
Flowchart_grey_24

Information Theory for Literature

Information theory, sparked by Claude Shannon, is used to quantify textual meaning. At it's heart lies the notion of entropy, which, in terms of information theory (not thermodynamics) means the amount of text needed to convey a given message. It looks at ways of encoding a certain symbolic text into a system of different symbols, which are often binary code.

The beauty of information theory is that it chooses to talk about meaning as being composed of other symbols, and by seeing it in that way can say a great many productive things about how to convey it. It neatly dodges the philosophical question. All of the standard information theoretical examples involve mathematical models of coin tosses, balls hidden under boxes and the like. But when the meaning in question is language, not data, and more than just that, literary — how is it possible to define the entropy of a sentence that forces you to reconsider your interpretation of an entire novel? It calls to mind the old Antoine de Saint-Exupéry adage:
"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."
The practice of writing has always been concerned with discovering the different potential ways to express your ideas, and choosing the best one among them. Perhaps great works of literature approach the entropy of the ideas that they convey. If there is nothing left to take away, and you have achieved Saint-Exupéry's perfection, Claude Shannon's entropy, then you have a work packed dense with meaning, brimming with ideas. When a text would suffer from any continued revision, then that text is done.


 
There's an analogue to the semiotic sign here, in which the signifier is the symbol that points to the signified. The information is the message of symbols which points to the meaning.

Complicating this, at least in code, is the fact that there are often many level of signs, pointing downward to the ultimate meaning, the binary which will be executed. "loop do" sparks off two possible chains of meaning. The first one, in the mind of the programmer, stands for a concept of repetition that she will combine with other such functional concepts. For the computer, at the level of what the text will become, it is a signifier for the bytecode that it will soon be translated into. The bytecode is, in turn, a signifier that points at the machine language into which it will be assembled.

The so-called "generations" of programming languages, the great leaps forward, have occurred at these fission lines. A new language builds itself on top of the old one, creating a new system of symbolic signifiers to point at the old words as the signified.