Wednesday, November 22, 2017
Explain yourself, Siri, or Alex or Watson or any other AI that does interesting/amazing things and we don't know how it does it
From the NYTimes:
It has become commonplace to hear that machines, armed with machine learning, can outperform humans at decidedly human tasks, from playing Go to playing “Jeopardy!” We assume that is because computers simply have more data-crunching power than our soggy three-pound brains. Kosinski’s results suggested something stranger: that artificial intelligences often excel by developing whole new ways of seeing, or even thinking, that are inscrutable to us. It’s a more profound version of what’s often called the “black box” problem — the inability to discern exactly what machines are doing when they’re teaching themselves novel skills — and it has become a central concern in artificial-intelligence research. In many arenas, A.I. methods have advanced with startling speed; deep neural networks can now detect certain kinds of cancer as accurately as a human. But human doctors still have to make the decisions — and they won’t trust an A.I. unless it can explain itself.This isn’t merely a theoretical concern. In 2018, the European Union will begin enforcing a law requiring that any decision made by a machine be readily explainable, on penalty of fines that could cost companies like Google and Facebook billions of dollars. The law was written to be powerful and broad and fails to define what constitutes a satisfying explanation or how exactly those explanations are to be reached. It represents a rare case in which a law has managed to leap into a future that academics and tech companies are just beginning to devote concentrated effort to understanding. As researchers at Oxford dryly noted, the law “could require a complete overhaul of standard and widely used algorithmic techniques” — techniques already permeating our everyday lives.
And so we have a new research field, explainable A.I., or X.A.I.
Its goal is to make machines able to account for the things they learn, in ways that we can understand. But that goal, of course, raises the fundamental question of whether the world a machine sees can be made to match our own.
One expert, David Gunning, asserts:
“The real secret is finding a way to put labels on the concepts inside a deep neural net,” he says. If the concepts inside can be labeled, then they can be used for reasoning — just like those expert systems were supposed to do in A.I.’s first wave.
To create a neural net that can reveal its inner workings, the researchers in Gunning’s portfolio are pursuing a number of different paths. Some of these are technically ingenious — for example, designing new kinds of deep neural networks made up of smaller, more easily understood modules, which can fit together like Legos to accomplish complex tasks.
Makes sense. That's what the brain does, isn't it? Except that the network in even a small patch of neural tissue is huge in comparison to deep learning nets.
Perhaps language will help:
Five years ago, Darrell and some colleagues had a novel idea for letting an A.I. teach itself how to describe the contents of a picture. First, they created two deep neural networks: one dedicated to image recognition and another to translating languages. Then they lashed these two together and fed them thousands of images that had captions attached to them. As the first network learned to recognize the objects in a picture, the second simply watched what was happening in the first, then learned to associate certain words with the activity it saw. Working together, the two networks could identify the features of each picture, then label them. Soon after, Darrell was presenting some different work to a group of computer scientists when someone in the audience raised a hand, complaining that the techniques he was describing would never be explainable. Darrell, without a second thought, said, Sure — but you could make it explainable by once again lashing two deep neural networks together, one to do the task and one to describe it.Darrell’s previous work had piggybacked on pictures that were already captioned. What he was now proposing was creating a new data set and using it in a novel way. Let’s say you had thousands of videos of baseball highlights. An image-recognition network could be trained to spot the players, the ball and everything happening on the field, but it wouldn’t have the words to label what they were. But you might then create a new data set, in which volunteers had written sentences describing the contents of every video. Once combined, the two networks should then be able to answer queries like “Show me all the double plays involving the Boston Red Sox” — and could potentially show you what cues, like the logos on uniforms, it used to figure out who the Boston Red Sox are.
I wonder if these people could make sense of some obscure notes I wrote up a decade ago:
Attractor Nets, Series I: Notes Toward a New Theory of Mind, Logic and Dynamics in Relational Networks, Working Paper, 52 pp.Abstract: These notes explore the use of Sydney Lamb’s relational network notion for linguistics to represent the logical structure of complex collection of attractor landscapes (as in Walter Freeman’s account of neuro-dynamics). Given a sufficiently large system, such as a vertebrate nervous system, one might want to think of the attractor net as itself being a dynamical system, one at a higher order than that of the dynamical systems realized at the neuronal level. A mind is a fluid attractor net of fractional dimensionality over a neural net whose behavior displays complex dynamics in a state space of unbounded dimensionality. The attractor-net moves from one discrete state (frame) to another while the underlying neural net moves continuously through its state space.Attractor Nets 2011: Diagrams for a New Theory of Mind, Working Paper, 55 pp.Abstract: These diagrams explore the use of Sydney Lamb’s relational network notion for linguistics to represent the logical structure of complex collection of attractor landscapes (as in Walter Freeman’s account of neuro-dynamics). Given a sufficiently large system, such as a vertebrate nervous system, one might want to think of the attractor net as itself being a dynamical system, one at a higher order than that of the dynamical systems realized at the neuronal level. Constructions include: variety ('is-a' inheritance), simple movements, counting and place notation, orientation in time and space, language, learning.Introduction: This is a series of diagrams based on the informal ideas presented inAttractor Nets, Series I: NotesToward a New Theory of Mind, Logic and Dynamics in Relational Networks, which explains the notational conventions and discusses the constructions. These diagrams should be used in conjunction with that document, which contains and discusses many of them. In particular, the diagrams in the first three sections are without annotation, but they are explained in the AttractorNets paper.
The rest of the diagrams are annotated, but depend on ideas developed in the attractor nets paper.
The discussions of Variety and Fragments of Language compare the current notation, based on thework of Sydney Lamb, with a more conventional notion. In Lamb’s notation, nodes are logicaloperators (and, or) while in the more conventional notation nodes are concepts. The Lamb-basednotation is more complex, but also fuller.
And, we might as well toss these notes in as well:
From Associative Nets to the Fluid Mind. Working Paper. October 2013, 16 pp.Abstract: We can think of the mind as a network that’s fluid on several scales of viscosity. Some things change very slowly, on a scale of months to years. Other things change rapidly, in milliseconds or seconds. And other processes are in between. The microscale dynamic properties of the mind at any time are context dependent. Under some conditions it will function as a highly structured cognitive network; the details of the network will of course depend on the exact conditions, both internal (including chemical) and external (what’s the “load” on the mind?). Under other conditions the mind will function more like a loose associative net. These notes explore these notions in a very informal way.
Tuesday, November 21, 2017
I've been digging out old MacPaint images over in Twitter, so I thought I'd bump this to the top of the queue.This is a slightly off-angle photograph of a painting I did in the summer of 1981:
When I started it I had a simple formal problem in mind, to do a painting that used a full range of colors. Id been doing paintings that leaned toward blues and reds and paintings that leaned toward blues and greens, but none that had all three in prominent use. That was the problem I started with in that painting. I approached it right off the bat by painting that rainbow arc of color patches across the top and right side. I then filled in the rest with appropriate imagery. Note the three worlds separated by the squid's tentacles: the yellow sky with the bluish sun, the forest with blue sky and stream, and the underwater scene with the strange ET-like face.
A couple years later I got a Macintosh and decided to realize that same image in the very limited medium of MacPaint, which gave me only white dots and black dots, no grays, much less color. Here's the final image:
Been Down So Long It Looks Like Up to Me, by Richard Fariña, 1966: “coming on like the Hallelujah Chorus done by 200 kazoo players with perfect pitch... hilarious, chilling, sexy, profound, maniacal, beautiful and outrageous all at the same time”–Thomas Pynchon.
I’m not going to try summarizing the novel. Just think about the title. What could that possibly mean? Well, I’m thinking it’s the story of my life.
Isn’t it the story of everyone’s life, up to a point, which you may not have yet arrived at, who knows?
I’ve lived under a cloud most of my adult life. The clouds, I think, are breaking up. The sun is shining through. & it’s not that the light hurts, because it doesn’t, but that it’s disorienting. I don’t know the world from this POV.
* * * * *
* * * * *
Are you familiar with the sociological concept of a reference group? It’s a group which you use as a standard for judging your behavior and accomplishments. When I got my PhD in English Literature, those people, academic literary critics, became my reference group. I started out strong, with good articles in good publications, and then that came to an end. I knew, of course, that my work was very different from standard literary criticism, and that caused problems, for me, not for them.
But I worked on literature, and they work on literature, no one else, so what choice have I had. They’re my reference group.
But maybe not. As I said yesterday, at last, someone’s interested in the technical work I did 40 years ago. And they’re not literary critics. They’re gamers.
Maybe I can stop worrying about academic literary criticism. They’re certainly not interested in my technical world, even those interested in cognitive criticism and computational criticism (two very different groups, BTW) have little use for it. And I can’t see making much headway with the descriptive folks, either. They seem more interested in theorizing description than in actually doing it. So we really don’t have anything to talk about. They’re not going to tumble to my ring-composition work. It’s actual description rather than theoretical throat-clearing in preparation for description at some later date.
So, let’s just bracket academic literary criticism for awhile. That profession is no longer a reference group for me. Let’s see if I can get somewhere with the gamers.
* * * * *
That’s one thing. And, in a way, it’s secondary. The big thing is that I think I’m finally going to be able to deliver on a task I set myself four and a half decades ago: to come to terms with, to understand, in some sense, the mechanisms underlying Coleridge’s “Kubla Khan”.
I talked about “Kubla Khan” in the presentation I delivered at HEX01 (First Workshop on the History of Expressive Systems), only a week ago today (early in the morning). At the end James Ryan, one of the organizers, asked me whether I would get back to “Kubla Khan”. I forget exactly what I said, but it was something like “maybe/I hope to/someday/yes”. That was the short answer. The long answer isn’t really that long, but it was too long to give in that context.
The long answer is that I long ago made “Kubla Khan” my touchstone, my personal reference point, my North Star. I judge my intellectual progress by what it tells me about “Kubla Khan”. So I’ve thought about the poem – and it’s relation to “This Lime-Tree Bower My Prison” – off and on for most of my adult life. I did my MA thesis on it in 1972, published an updated version of that in 1985 (“Articulate Vision: A Structuralist Reading of “Kubla Khan”), and a considerably more sophisticated account in 2003 (“Kubla Khan” and the Embodied Mind). I count that last as considerable progress, but still, a way to go with no sense of just how far or even in what direction. In 2013 I put up a working paper, STC, Poetic Form, and a Glimpse of the Mind, in which I did a comparison between “Kubla Khan” and “This Lime-Tree Bower My Prison” that was considerably more detailed and sophisticated than the one I’d published way back in MLN in 1981, “Metaphoric and Metonymic Invariance: Two Examples from Coleridge” (obviously, my first formal publication on “Kubla Khan”). So, I’ve been through the poem five times in my career, including my unpublished master’s thesis. And, yes, to answer Ryan’s question, I hope to get to it again. But just when, I don’t know.
Well, a day or two later I got back to it. And I’ve declared the problem to be solved. Of course, there’s something of a gap between the declaration and the actual solution. I know that. And it’s not so much the solution that I’m after, but a sure sense of the terms in which a solution is likely to be found. That’s where I’m at.
Monday, November 20, 2017
— James Ryan (@xfoml) November 14, 2017
That's from the First Workshop on the History of Expressive Systems. The image you see on the screen originated in my computer in Hoboken, NJ, and was being viewed, via Skype, in Funchal, Madeira, Portugal.
These people aren't literary critics. They're into gaming. That is to say, they are interested in stories, in creating interactive stories, and they think in computational terms. And that's how I've been thinking about literary texts for over 40 years. I can talk to them about Shakespeare, Coleridge, and Conrad in computational terms, but also Francis Ford Coppola, Walt Disney, King Kong, Gojira, and others. They need to know what I know, and vice versa.
The diagram in that image is from my 1976 article, Cognitive Networks and Literary Semantics, MLN 91: 1976, 952-982. Given the importance of MLN as a journal, and the fact that that particular issue was a special one commemorating 100 years of publication, I figured it would mark the beginning of a spectacular academic career. WRONG! Oh, the intellectual work's been good, at times even thrilling, but the literary academy wanted to go to Kansas (though that is not, perhaps, how they thought of it) and I wanted to go to the moon.
Have I found some fellow astronauts?
Sunday, November 19, 2017
As you can see from my most recent post, I have acquired a rather exotic women's shoe. I was walking to the library when I spotted it on the sidewalk. Apparently discarded, a single shoe, left foot, size 7, "Kiss & Tell" – How's THAT for branding? There's a label on the sole at the instep that says, "All Man-Made Material Made in China". What does that mean? I understand "Made in China", but "All Man-Made Material" is ambiguous. Does it mean that all the materials are man-made (and they're made in China), so that the suede uppers are actually some artificial suede substance? Of does it mean that the man-made materials were made in China (the sole and heel are plastic) but the rest might well be natural? If so, was it also assembled in China?
Anyhow, as soon as I saw the shoe one of those little light-bulbs went off above my head:
So I grabbed it and put it in my backpack and then continued on to the library to return my film, Miyazaki's Castle in the Sky, and pick up my book, King Kong: The History of a Movie Icon from Fay Wray to Peter Jackson. There's a connection, you see, between King Kong and that shoe. King Kong died on the Empire State Building, right? Why not pose the shoe with the Empire State Building. Like this perhaps:
Notice that, from this angle, that size 7 woman's shoe is larger than that phallic whatsiewhoseit across the river.
Then I realized that these aren't the only photos of shoes I've got. For example, I found these hanging outside the improvised shack of some homeless person:
And then we have the stash of women's shoes that my friend Wayquay is selling at The Ruins JC. Mostly women's shoes, but not all of them. I suppose we could say these baby booties (made by Wayquay herself) aren't shoes, strictly speaking, but they serve the same function, no?
And I've got other shoe shots as well, like these:
This, of course, is a minor sport.
Anyhow, I figured that, with these latest shots of the green shoe – I've got more that I haven't uploaded, and I plan to take more photos as well (perhaps in Narnia) – I should create a tag here at New Savanna (shoes) to capture those shots and write up a brief post acknowledging the importance of shoes.
Isn't that green just gorgeous! That shoe's the greatest prop ever!
Bumping this to the top of the queue as it is relevant to current news about sexual harassment and rape in America.A couple of weeks ago I watched my way through the first season of Bonanza, a TV western that I had watched in my youth. As some of you may know, it was one of the most popular shows on television at the time and ran for 14 seasons from 1959 to 1973. It was set in Nevada in the 1860s and centered on the Cartwright family, father Ben and his three adult male sons, proprietors of the Ponderosa, a large cattle ranch bordering on lake Tahoe and near Virginia City, a mining town.
This post consists of two casual notes about the show. The first concerns sexual violence against women and the second is about the structure of (political) power.
Rape in the Old West
After I’d watched about a dozen episodes a thought struck me: there’s a lot of sexual violence against women in this show. That thought stayed with me to the end of the season, 34 episodes. I wasn’t taking notes or keeping count but I’d say that half the episodes depicted sexual violence.
What do I mean? As I said, I wasn’t keeping notes, but typically a man would embrace a woman and try to kiss her. She would resist but he wouldn’t stop. At this point either the camera would cut away, leaving us to imagine what happened next, or a Good Guy, such as one of the Cartwrights, would come along and rescue the woman. The violence wasn’t nearly as graphic as we’d see on Deadwood, set in a similar place a decade later, but then Deadwood wasn’t made for a family audience watching on primetime network television back in the day when network television was much more important than it is now. Bonanza WAS made for a family audience.
Was this typical of primetime television back in the 1960s? I don’t know, but I suspect it was more common than I remember. Was this typical of westerns? I don’t know.
I know that it is not typical of The West Wing, a more recent and very different television show that I’m now watching. As you may know The West Wing is a political drama set in the west wing of the White House, which contains offices for the President, Vice President, and high-level staffers. While sexual violence comes up as a topic every now and then, the show doesn’t have a lot of scenes where a man forces himself on a woman. In fact I cannot think of one such scene off hand, and I’ve been through the whole run of the show.
Why then is it so common in Bonanza? Westerns are typically set in a world where the rule of law is tenuous. Westerns are about violence: cattle rustling, disputes over land, conflicts with Indians (aka Native Americans), bank robbery and other forms of theft, and, in the case of Bonanza, rape. What about other TV Westerns? I don’t know off hand; though I watched many TV Westerns when I was young, I don’t recall many of them.
Whatever the more general case, the first season of Bonanza was concerned about sexual violence against women. In some cases the women were saloon girls, prostitutes I (now) assume, though that was certainly not explicit (unlike Deadwood). In other cases the women were married or simply single; in one episode two Indian women were raped at a trading post.
Were the other 13 seasons like this? I don’t know and I don’t intend to watch them. It would be interesting to know, though.
Saturday, November 18, 2017
Greggs forced to apologise after PR shots promoting advent calendar replace baby Jesus with a SAUSAGE ROLL pic.twitter.com/q0qwd4ISxh— Andrew Bloch (@AndrewBloch) November 15, 2017
LONDON — A British bakery chain has apologized after creating a Nativity scene in which the baby Jesus, surrounded by three wise men, was replaced by a sausage roll.
Friday, November 17, 2017
I’m thinking about what would be involved in making a game that would play-out as a ring composition. Given that ring composition is a specific form of plot it is possible that it could not be adapted to open-ended gameplay. But that is by no means obvious to me. Ring-composition narratives have a an internal logic that is oriented toward certain kinds of problematics, if you will. I’ve spent enough time dealing with ring composition that I’m beginning to get a sense of how they work & what they’re for. The idea would be to set up a game world so that successful play would take the form of a ring, but no tightly specified sequence of actions is required. What’s important is the high-level pattern.
Why King Kong?
Because it’s there?
No, seriously. I’ve developed a pretty sophisticated understanding of the 1933 film. I’ve seen the 2005 Peter Jackson remake and the 1976 remake, but I don’t remember much about either, which is mostly a statement about what I remember, not an aesthetic judgment. I’ve also seen Kong: Skull Island, which is a different kind of beast. For that matter, I’ve seen a bunch of Planet of the Apes films and, of course, a whole bunch of action-adventure films in jungle settings involving big animals. It’s a world I’m comfortable with.
But it’s the 1933 film that interests me. That’s the one that started the whole business. It’s an astonishing film. And it’s ring form. It’s not, as I’ve observed in several posts, that I believe there’s some special magic in ring-composition – a key to all mythologies, an Open Sesame! that will lay the wonders of the world before my feet – rather, it’s something specific to look for. And, having found various examples, I can compare and contrast them to get a sense of what’s going on.
I am aware that various games have been based on the character and even one or more of the specific films (e.g. Peter Jackson’s King Kong). I may well want to investigate one or may of them at some point. But I need to do some thinking before I get to that point. Just what thinking I need to do...
... THAT’s what I’m trying to figure out.
Policy, Strategy, Tactics and cultural psychodynamics
In On War Carl von Clauswitz made a tripartite distinction between policy, strategy, and tactics. National political policy established the objectives to be achieved by military strategy; strategy established the objectives to be achieved by tactics; and tactics governed the actual deployment of troops and equipment on a day-by-day basis. I find this a generally useful way of thinking about a variety of things. One might, for example, think of an action-adventure game (like Jackson’s Kong) as a strategic level adventure realized, at various points, though tactical level actions.
But what of policy? Northrop Frye has observed that a certain kind of comedy has a three-part plot: “One is the period of preparation . . . Another is the period of license and confusion of values . . . Third is the period of festivity itself” (A Natural Perspective, p. 73). If you drop the third part, you get a tragedy.
Well, the 1933 King Kong more or less follows that model, where the Skull Island sequence is that “period of license and confusion of values”. What would happen if Kong failed to defeat T-Rex and, consequently, R-Rex killed Darrow? The possibility of the existing happy ending, in which Darrow and Driscoll are destined to be married, disappears. Are we left with a tragic ending?
This, it seems to me, is a very high-level consideration. The battle between Kong and T-Rex is a tactical-level action, but the consequence is up there at the policy level, which is where I’d put the difference between tragedy.
That’s something I have in fact investigated  in the case of three Shakespeare plays, Much Ado About Nothing (a comedy), Othello (a tragedy), and The Winter’s Tale (a romance). In all these plays the shape of the dramatic action follows from the same situation, a man mistakenly believes that his beloved has betrayed him. These plays differ systematically in the types of characters they have in various roles, their age, social position, and temperament. I’m inclined to think that, if this is what you want of a game, then you need to give the user Sims-like powers of set-up at the very beginning.
Timothy Gowers, in TLS, "The end of an error?", opens by discussing three cases:
- In 1998 an article appeared in the Lancet (on the link between MMR vaccine and autism), was found to be deeply flawed, but not retracted until 2010.
- Three articles posted to asXiv proving an important piece of mathematics. The articles were difficult and a bit sketchy, but others went to work and cleaned things up.
- A couple months ago a different article was posted to asXive. It was clear and complete and within days a serious flaw was found and the original claim was retracted.
The first case involved formal peer review whereas the second two involved review by peers, but not the standard prepublication process typical of the formal literature. The first case was a disaster while the second two were successes. In light of such cases, should we reconsider the role of formal peer review in academic publication? Has the Internet rendered it obsolete?
Gowers notes that prepublication in arXive is how mathematicians now disseminate ideas and establish priority, "Of course, different disciplines have different needs and very different publishing cultures." It may well remain useful in some disciplines, but no longer for mathematics. Peer review is defended with respect to three functions:
- Reliability: "The first is that it is supposed to ensure reliability: if you read something in the peer-reviewed literature, you can have some confidence that it is correct."
- Weed out the chaff: "... a vast amount of academic literature is being produced all the time, most of it not deserving of our attention, and the peer-review system saves us time by selecting the most important articles. It also enables us to make quick judgements about the work of other academics."
- Feedback to the author: "If you submit a serious paper to a serious journal, then whether or not it is accepted, it has at least been read, and if you are lucky you receive valuable advice about how to improve it."
Gowers then goes on to discuss these functions, mostly in terms of the (scientific) disciplines he's familiar with. Reliability is important in mathematics because researchers build on the work of others in a fairly 'tight' way. This is not so much the case in literary criticism, which is as close to a home discipline as I've got. He notes:
In some disciplines, the formal peer-review system appears to have failed on a huge scale. This is particularly true of articles about scientific experiments where the conclusions are statistical in nature.
Nor, for that matter:
...does formal peer review seem to manage very well to stop wrong ideas from spreading outside academia. Climate change deniers are not put off by their lack of representation in respectable academic journals. Drugs policy bears little relation to the harm that drugs actually cause. An economics paper that supported austerity-based policies influenced several governments before it was discovered to contain some very basic mistakes that invalidated it.
As for weeding out the chaff:
Again, the way things already are in mathematics suggests that this would not be as serious a problem as it at first looks. Far more papers are posted to the arXiv than I have time to check through, even if I restrict myself to a few areas of particular interest. But I use a recently created website called arxivist.com, which puts each day’s preprints in what it judges to be their order of interest to me.
As for saving time, yes, we certainly can "make quick judgements of publication lists by looking at the names of journals" but this "encourages the measurement culture that has infected academia, with all its well-known adverse consequences." As for the feedback function, that
is much more obviously valuable, especially in some subjects. Remarkably, we have arrived at a system where academics feel a moral obligation to perform the thankless task of reviewing the work of other academics, anonymously and unpaid. This undoubtedly makes the literature better than it would otherwise have been, and ensures at least one reader for each paper.
While other feedback mechanisms are possible "it is less clear how they could become widely adopted." Thus one could add a comment function to preprint servers but "only a small minority of preprints are actually worth commenting on" and
there is no moral pressure to do so. Throwing away the current system risks throwing away all the social capital associated with it and leaving us impoverished as a result.
That is a strong argument against an abrupt change to a new system, but it is not an argument against a gradual one. And it is not unrealistic to hope for gradual change.
I've added another article to my online stash. Title above. Download from:
Citation: William Benzon. Metaphoric and Metonymic Invariance: Two Examples from Coleridge. MLN 96: 1097 - 1105, 1981. DOI: 10.2307/2906237
Citation: William Benzon. Metaphoric and Metonymic Invariance: Two Examples from Coleridge. MLN 96: 1097 - 1105, 1981. DOI: 10.2307/2906237
Abstract: “Kubla Khan and “This Lime-Tree Bower My Prison” are two very different poems by the same poet. But they share the same two-part structure, and they share imagery as well. The roaring dell of “Lime-Tree” corresponds to the savage chasm of “Kubla Khan.” The concern with sight and sound manifest in “Kubla Khan” shows up in “Lime-Tree Bower” in the image of the creeking rook flying across the sun. And the way in which both Charles and the poet have access to that sight gives it a role similar to the sunny dome and caves of ice in “Kubla Khan,” where both the poet and his audience are linked through the image. These two poems share the same world. But they take radically different paths through it. One path is regulated by metonymy and unfolds though two consciousness moving through different parts of the same landscape. The other path is regulated by metaphor and so unfolds in two different worlds linked by a common image; the path it takes through these worlds is, however, the same.
I. Forms of Invariance – 1097
II. Metonymic Invariance – 1099
III. Metaphoric Invariance – 1001
IV. Toward Poetic Grammar – 1103
NOTES – 1105