CHAPTER THREE FRAMEWORKS FOR THE ANALYSIS OF VISUAL DISPLAY 3.1 Preliminaries to the Frameworks 3.1.1 The Social Semiotic / Sociological Approach versus the Psychological Approach As may be clear from Chapter Two, SFL takes a social, cultural, or “inter-organism” view of language rather than a mental, psychological, or “intra-organism” perspective (Halliday 1978: 12-16). In the words of Halliday, in the sociolinguistic perspective, “we take account of the fact that people not only speak, but that they speak to each other” (1978: 57; my emphasis). Likewise, in studying semiotic systems other than language, SFL-informed models take a social semiotic perspective rather than a psychological one. They are concerned with how the cultural resources of painting, dancing, sculpture and so forth evolve to make meaning in society, together with language. However, a cursory review of the literature shows that a particularly strong influence in the study of illustrations in educational materials has been the psychological approach1. This approach is characterized by, and centres around, the experiment. When the researcher is interested in, for instance, whether an illustrated text is more effective to learning than a non-illustrated text, he or she selects two groups of subjects. One group (the experimental group) is given the illustrated material and the other group (the control group) is given the non-illustrated, linguistic only material. After some time of learning, either in the laboratory or in the normal 55 classroom, the two groups of subjects take a test. The performance of the two groups in the test is used to help researchers determine whether to accept or refute the original hypothesis. In any serious experiment, the hypothesis is built within a theoretical framework and both the subjects participating in the experiment and the materials used are selected to meet certain criteria. It may be the case that psychological studies of illustrations can tell us whether or not a type of instructional material or methodology is likely to produce positive effects on learning in terms of the amount of recognition, recall, retention or comprehension of selected items in the material. They tell us, for instance, what type of colour or shape combination is more likely to be recognized by the learner. To put it another way, such studies can reveal the workings of the human brain, that is what the brain is capable of doing and how it is functioning. On the other hand, the psychological studies not reveal the social and cultural aspects of the illustrations in educational materials. Nor they show how these non-linguistic materials have evolved as resources for meaning, together with the linguistic materials, and how the human child develops these resources in the process of his or her socialization and education in the community. Further, in the context of science and science education, as Lemke (2002a) explains, But mentalist psychology does not have much to say about science as a social activity, as a system of interdependent actions and practices that produce scientific statements and theories… It has little to say to us about how to model the practices and activities of science for our students, how to help them learn to integrate these various practices into some semblance of the ways that scientists act or how to learn to be even peripheral participants in the world-spanning activities of scientific communities. 56 As discussed in Chapter One, if one of the goals of EST (English for Science and Technology) teaching to non-native speakers of English is the familiarization with the linguistic and non-linguistic conventions in international scientific and technological settings, then the inadequacy of the psychological approaches becomes obvious and we need therefore to incorporate the social semiotic approach to the study of illustrations in science textbooks. 3.1.2 Social Semiotic Approaches Before we describe the SFL-informed approaches to visual semiotics, we need to comment on an approach to the study of visual display in science that is more in line with social semiotic approach than with the mentalist approach. This is the sociological or ethnomethodological approach, exemplified by Lynch and Woolgar (1990) and Goodwin (2001). As the name of the approach indicates, this approach seeks to find out how in an actual work setting the participants construct, use and circulate the visual images (together with other semiotic resources such as language, gaze, gesture) as an essential part of the task they are charged with performing. Goodwin (2001: 167) has succinctly termed work-related visual practice as “professional vision”. Examples of professional vision include field archeologists “systematically classifying the colour of the dirt they are excavating”, map-making and a video tape of some scene used in court as evidence for a certain verdict (Goodwin 2001: 167-177). Although sociological studies of the use of visual displays have made significant contributions to our knowledge and understanding, they often seem to lack a coherent framework to explain how the various visual displays make meaning in 57 their natural and social settings. They have shown us what is happening through video tape recordings, verbal accounts and historical documents, but they have not been explicit enough about the systems and functions that underlie the use of visual images. It is for an adequate description of the systems and functions in the visual displays in science that we need to turn to social semiotic approaches. There have been mainly three important SFL-informed studies of visual and multimodal semiotics: O’Toole (1994; 1995) in the visual arts, Kress and van Leeuwen (1996) and Kress (2000a) in the public communication, and Lemke (e.g. 1998a; 2000; 2003) in science and science education. I summarize the major contributions of each study below, as a preparation for the proposal of my models for the analysis of biology texts. 3.1.2.1 O’Toole’s Model (1994; 1995) O’Toole (1994) marks a milestone in the efforts of systemic-functional linguists to apply Halliday’s SFL to semiotics other than language, in this case, the displayed art, including painting, sculpture, and architecture. The model he proposes for the analysis of painting is reproduced in Table 3.1. O’Toole’s (1994) model is based on three theses. First, “every piece of communication has three main functions: 1) to engage our attention and interest, 2) to convey some information about reality, and 3) to structure these into a coherent textual form” (1994: 5), be it in the form of language, visual display, dance, or music. That is, semiotic codes other than language have also to fulfil, simultaneously, what SFL refers to as the metafunctions of language: ideational, interpersonal and textual 58 Function REPRESENTATIONAL MODAL COMPOSITIONAL Unit Narrative themes Scenes Portrayals Interplay of episodes Rhythm Gaze Frame Light Perspective EPISODE Actions, events Agents --patients --goals Focal/side sequence Interplay of actions Relative Prominence Scale Centrality Interplay of Modalities Relative position in work Alignment Interplay of forms Coherence FIGURE Character Object Act/Stance/Gesture Clothing Components Gaze Contrast: Scale Stance Line Characterization Light Colour Relative position in episode Parallelism/Opposition Subframing Part of body/object Natural form Stylization Cohesion: Reference (Parallel/Contrast/Rhythm) WORK MEMBER Modality Gestalt: Framing Horizontals Verticals Diagonals Proportion Geometry Line Rhythm Colour Table 3.1 Functions and systems in painting (Source: O’Toole 1994: 24) 59 metafunctions. There is an important proviso, as O’Toole (1994: 232) explains, “[h]owever, certain artists or schools of painting seem to foreground one of these functions to a very marked degree”, although this also applies to some linguistic text. That is, there is a monofunctional tendency in some acts of communication. Secondly, the three functions spread throughout all ranks or units in the piece of communication. In the case of painting, the above Table recognizes the ranks of Work, Episode, Figure, and Member, each consisting of one or more of the next lower rank (except the rank of Member), Work consisting of one or more Episodes, Episode consisting of one or more Figures, and so on, the way language has a hierarchy of ranks: Clause, Group / Phrase and Word (Halliday 1973: 141). It is also obvious that in some cases, a Work may have only one Episode, Figure, and Member. Thirdly, like language, displayed arts bear an intimate relationship with the context of situation and context of culture. That is, the particular selections from the options in all three metafunctions in visual display realize the context in which the work of art is created and used, that is, a particular contextual configuration of field, tenor and mode. These three features set O’Toole’s approach on a sound social semiotic ground and make it a rigorous, coherent and powerful model. Hence in my models for the biology texts, I take O’Toole’s (1994) model as the point of departure. O’Toole (1995) analyses and interprets Frank Hinder’s painting Flight into Egypt using basically the same framework: a trio-functional analysis and an interpretation that relates to its social context of creation and other discourses of art: art history, art critics and teaching. What is noteworthy is that this functions and systems chart (1995: 162-163) is more detailed than the chart given in O’Toole (1994: 24). For instance, the chart (1995) recognizes five ranks rather than the four ranks as in the 60 1994 chart. The additional rank is School / Genre, referring to the typical selection from all three metafunctions that characterize a particular type of painting, e.g. Realism, Impressionism, Expressionism, Futurism, Constructivism, Surrealism, Cubism, Baroque and the like. To be accurate, this is not a separate rank, but a collection of features that some paintings tend to show or foreground in their selections of the systems and options in the three metafunctions. That is to say, Baroque school, for instance, tends to depict a certain subject matter (ideational), adopt a particular interpersonal stance and have a particular means of textual coherence. Such descriptions follow the trio-functional analysis rather than predetermine it (O’Toole 1995: 161). In addition, the systems in the 1995 chart are also more detailed than those in the 1994 chart. For instance, in the Modal function at the rank of Work, the 1994 chart recognizes only “Gaze”, whereas the 1995 chart lists under “Gaze”: ‘Eyework’, ‘Paths’, ‘Rhythms’ and Intermediaries. 3.1.2.2 Kress and van Leeuwen (1996) and Kress (2000a) Kress and van Leeuwen (1996: 15; original emphasis) first draw attention to the emergence of the multimodal semiotic landscape in contemporary public communication, such as newspapers, magazines, advertisements and so on, where there involves “a complex interplay of written text, images and other graphic elements” and where “these elements combine together into visual designs, by means of layout”. In view of such recent and vibrant changes in public communication, Kress (2000a) advocates a departure from the language-based theory of communication in an attempt to account for the multimodal landscape. He (2000a: 183) writes: 61 [T]here are the strongest possible reasons for taking a completely fresh look at this landscape, and for setting a quite new agenda of human semiosis in the domain of communication and representation. Such an agenda has, as some of its most urgent elements, the requirement for a theorisation and a description of the full range of semiotic modes in use in a particular society; a full understanding of the potentials and limitations of all these modes; of their present use in a society; of their potentials for their interaction and interrelation with each other; and an understanding of their place and function in our imaginings of the future. That is, whereas O’Toole (1994; 1995) focuses on the displayed art, i.e. the refined “high-brow” images of painting, sculpture and architecture, Kress and van Leeuwen (1996) are concerned with the visual design of public communication, i.e. not only the images, the linguistic text, but also their interaction, and Kress (2000a) is concerned with the total multimodal communication sphere: images, written texts, sounds, actions and other semiotic resources. The basic view in Kress and van Leeuwen (1996) is that “language and visual communication both realize the same more fundamental and far-reaching systems of meaning that constitute our cultures, but that each does so by means of its own specific forms, and independently”(1996: 17). By this they mean that the metafunctions that are identified as applicable to language also apply to the visual communication, although they are realized in the visual by different and independent forms. Indeed, the bulk of their book is devoted to the exploration of the visual grammar that realizes the ideational (Chapters and 3), interpersonal (Chapters and 5) and textual (Chapter 6) metafunctions. However, Kress and van Leeuwen (1996: 17) believe that “each medium has its own possibilities and limitations of meaning” and that “[n]ot everything that can be realized in language can also be realized by means of images, or vice versa” (1996: 17). 62 Of particular interest is their discussion of the principles of composition, the textual metafunction. They propose three simultaneous systems of visual composition: the system of INFORMATION VALUE, whether an element is placed in the left or right, top or bottom, centre or margin of the visual page has “specific informational values” (1996: 183)2; the system of SALIENCE, the placement of the elements either in the foreground or background, their relative size, contrasts in tonal value (or colour), differences in sharpness, etc. serve to “attract the viewer’s attention to different degrees” (1996: 183); and the system of FRAMING, the “presence or absence of framing devices (realized by elements which create dividing lines, or by actual frame lines) disconnects or connects elements of the image, signifying that they belong or not belong together in some sense” (1996: 183). Further, they hold that “[t]hese three principles of composition apply not just to single pictures, they apply also to composite visuals, visuals which combine text and image, and, perhaps, other graphic elements, be it on a page or on a television or computer screen” and that in analyzing the multimodal texts we need to “look at the whole page as an integrated text” (1996: 183; original emphasis). If we compare O’Toole’s model with Kress and van Leeuwen’s approach to the visual images, we find that both take Halliday’s SFL as points of departure, but that each has different emphasis. O’Toole tries to show that the system and function at various ranks in language can be adapted to the study of visual arts, the framework in general is applicable to visual studies, while Kress and van Leeuwen (1996) attempt to show that, at a much more delicate level, the process types in language, for instance, can find their equivalents or lack of equivalents in visual display. O’Toole stresses the applicability of SFL in general orientation while Kress and van Leeuwen (1996) claim a comparability at a more specific level. For instance, Kress and van Leeuwen (1996) 63 identify the visual Narrative and Conceptual process types and subtypes, while O’Toole (1994) lists at the rank of Work only Narrative themes (equivalent to Kress and van Leeuwen’s (1996) Narrative processes, but lacking in specificity), Scenes, Portrayals (equivalent to Kress and van Leeuwen’s (1996) Conceptual processes, but again lacking in specificity) and Interplay of episodes. In the discussion of textual metafunction, O’Toole (1994) draws attention to Framing, Horizontals, Verticals, Diagonals and so on at the rank of Work. Kress and van Leeuwen (1996), on the other hand, divide the pictorial space into Left and Right, Top and Bottom, and in some cultures Centre and Margin, and assign different information values to each of the pictorial zones. One can, of course, notice that O’Toole’s Horizontals are equivalent to Kress and van Leeuwen’s (1996) Left and Right, but the communicative values assigned to such pictorial spaces are completely different. O’Toole (1994: 23) says of Horizontals that “[w]ithin the frame, … forms are related to the horizontal axis and the vertical axis, both of which contribute to stability and harmony, while their relation to the diagonal axes tends to create energy and dynamism”, whereas Kress and van Leeuwen (1996) treat of Left as the pictorial realization of what is Given, and Right of what is New. Another difference between the two approaches to the study of visual images is that in O’Toole’s work the notion of rank scale, which derives from Halliday’s (1973: 141) chart for language, is crucial. Indeed, in all the functions and systems charts proposed by O’Toole (1994; 1995) there is a rank scale. In Kress and van Leeuwen (1996) and Kress (2000a), the notion of rank is implicit. O’Toole (1994; 1995) and Kress and van Leeuwen (1996) form the two comprehensive resources for the study of multisemiotic texts. 64 The semiotic demands of the discipline not stop here. In cell biology, in particular, recourse to non-linguistic semiotic resources has been necessary since Robert Hooke (1635-1703) first drew a picture of the “cell” seen under his microscope as reported in Micrographia (1665). This time-honoured morphological approach to the studies of the cell, with the help of a light microscope and an electron microscope, has recently culminated in what we know as the ultrastructure of the cell. To communicate what was observed under the microscope, the cell biologists have developed a range of devices, including light micrographs, electron micrographs, and schematic drawings, each of which has several sub-types, depending on the techniques adopted. More recently, however, cell biologists have attempted to investigate the biochemical basis of the structure and function of the living cell. Rather than merely describe the mechanical or morphological features of the cellular life, this new approach seeks to account for the cell and cell activity in terms of the structure and function of its chemical components, the four major families of small organic molecules (sugars, fatty acids, amino acids and nucleotides) and the macromolecules (polysaccharides, lipids, proteins and nucleic acids). Most of the macromolecules normally exist as specific biologically significant three-dimensional structures called conformations, for example, the double helix for DNA, the extended chain conformation for cellulose and the α-helix, β-pleated sheet, β-turn and loop conformations for proteins. As noted by McMurry and Castellion (1999: xvi), “[u]nderstanding many aspects of chemistry – such as the specificity and selectivity of enzymes, or the action of drugs – requires understanding the three-dimensional nature of molecules”. That is, the introduction of biochemistry means that the semiotic demands of the discipline have exponentially increased so that natural language, however important it may be, is inadequate as a single resource. As a result, other 70 semiotic means such as chemical notation, ball-and-stick models, space-filling models, animations, video recordings and so forth have evolved for communicative purposes. Natural language alone has been inadequate with morphological research; it is naturally insufficient as a means to describe both the morphological and the biochemical. 3.2.2 Theoretical Frameworks for the Analysis of Biology Texts Myers (1990: 233-249) identifies, “in terms of realism and abstraction” (1990: 247), five categories of visual displays in a sociobiology text: photographs, drawings, maps, graphs / models / tables, and imaginary figures (1990: 234)5. The first three types have some reference to our everyday visual experience while “Graphs, models, and tables redefine space, … so that each mark has meaning only in relation to the presentation of the claim” (1990: 235). In many textbooks on the molecular study of the cell, one of which is ECB, biochemical symbolism constitutes yet another semiotic resource, modelled after algebra (Hoffmann 1993: 27-28; Knight 1992: 176-179, 1996: 135). In what follows, I present the frameworks for the analysis of schematic drawings (Section 3.2.2.1), tables (Section 3.2.2.2), and statistical graphs (Section 3.2.2.3). The reason for identifying these various kinds of visual display is that they are different in form and function; they have evolved to perform different functions. Lines, dots, curves, and colours, if organized differently, have come to mean different things. 71 3.2.2.1 Framework for Analysing Schematic Drawings By schematic drawings I refer to those that are designed to depict in a simplified way some scene or process, actual or imaginary. The functions and systems chart for the analysis of schematic drawings is displayed in Table 3.2. Although the rank scale in the chart follows O’Toole (1994: 24), the functions and systems are not, unsurprisingly, identical. For instance, in O’Toole’s (1994) model, in the Modal function at the ranks of Work and Figure, Gaze is an important means deployed by artists to attract the attention of the viewer. In the biological schematic drawings I have analysed, Gaze does not appear to figure as an important resource. More importantly, in the Compositional function, unlike in paintings where usually little more than a title is provided to indicate what is depicted, in scientific illustrations, Labelling appears frequently. This feature is related to the pedagogic use of the schematic drawing. An important part of a biology student’s training is to learn to recognize the shapes of components of an organism and learn how these components are named by the scientific community; for example, a certain shape is named the “stem”, or “root”, or “microtubule”. Labels and Leaders provide in part the means for the enculturation of the learner into the discipline of biology. The Representational meaning of the schematic drawing is what Lemke (1998a) calls the “topological” meaning, especially, the Shape, Colour, Size, Spatial relation to each other and to the whole structure, and Action. Such meanings are also typological in that they fall into categories; for instance, the Shape is round, square, rectangular, and so on. But the predominant aspect of these meanings in biology is topological where 72 Function REPRESENTATIONAL MODAL COMPOSITIONAL Unit Overall shape; Components of the structure; Whole process; Phases of the process Frame; Size; Scale; Perspective; Full colour or black and white; Colour contrast; Shade or light Gestalt: Framing, Horizontals, Verticals, and Diagonals; Proportion; Geometry; Colour; Drawing’s relation to running text: Spatial and Colour; Labelling: Positioning, Colouring and Leaders Shape; Colour; Size; Spatial relation to each other, and to the structure; Actions, events Relative Prominence: Colour, amount of detail; Centrality; Lettering (for label and caption): type size, style (serif or san serif), Weight; Line and arrow width; Numerical sequence Relative position in the structure or process; Colour contrast between components WORK EPISODE Table 3.2 Functions and systems in schematic drawing (Adapted from O’Toole 1994: 24) 73 Function REPRESENTATIONAL MODAL COMPOSITIONAL Unit FIGURE MEMBER Components; Acts Contrast: Scale, Line, Light, Colour; Omission of detail Natural form: Shape, colour, etc. and spatial relationship to other components Stylization; Conventionalization Relative position in the component or phase; Colour contrast or similarity; Subframing Cohesion: Parallel/Contrast in Shape and Colour; Reference through language Table 3.2 (Continued) 74 the irregularity defies any linguistic encoding except in the most general terms. The exact Spatial relations and the moment-to-moment movement in space can best be shown in a drawing or video recording rather than by verbal description. 3.2.2.2 Framework for Analyzing Tables Gove et al (1986: 2324) define a table as “a systematic arrangement (as of numerical values) usu. in parallel rows or columns for ready reference”. This definition draws attention to several features of a table. First, a table is arranged in rows or columns on a printed page or part of the page. That is, it makes use of the spatial resources of the horizontal (rows) and the vertical (columns) of the printed page, foreclosing the semiotic potential of other spatial relationships, such as the diagonal, circular, centremargin, etc. Second, a table has evolved to present a set of related numerical values or facts. Thirdly, a table realizes the two functions of the information unit (Given and New) of natural language by fixed spatial means on the printed page so that the position a lexicogrammatical item occupies in the table signifies whether it is Given or New. This facilitates a reader’s information retrieval and the comparison between more than one set of information. There are different types and styles of table. The Publication Manual of the American Psychological Association (2001) recommends that a table consist of a title, headings (column head, stubhead, and optionally table spanner, column spanner), rules, table body and notes to table. The framework for the analysis of the APA style numerical tables is presented in Table 3.36. 75 Functions Units REPRESENTATIONAL TABLE CELL PERIPHERY (excluding Notes to the table) Typological meaning (TRANSITIVITY); Topological meaning (quantity, degree, etc.) Number MODIFICATION (epithet function, enumeration); unit of measurement; Circumstantial features: Time, Frequency, etc. MODAL COMPOSITIONAL Declarative; Lettering: type size; style (serif or san serif); weight; Frame; line width Vertical; Horizontal; Spacing between columns and between rows; Lines. Textual ellipsis Lettering; the number of decimal places; Colour, shading ATTITUDE of the nominal groups; positioning; line width of the rule Vertical aligning; Horizontal aligning; Placement relative to the body of the table. Table 3.3 Functions and systems for numerical tables Table 3.3 has a rank scale consisting of Table, Cell and Periphery. Cell is the body of the table and Periphery provides essential information on how to read the table and includes title, headings (column head, stubhead, table spanner, column spanner), table number, and notes to the table (not analyzed here). Below I discuss some features of a table as a meaning-making resource, particularly its textual metafunction. Natural language has its own resources for the textual metafunction (see Halliday 1994: 334 and Section 2.2.3 for a summary). Relevant here is the information structure. By “information” is meant “the tension between what is already known or predictable and what is new or unpredictable” (Halliday 1994: 296). In spoken 76 language, the information unit is realized as a pitch contour so that a listener can normally get a clue as to what he or she needs to take as New. In written language, however, such prosodic, paralinguistic and indexical features are absent (see Halliday 1989 [1985]: 30-32) and thus ambiguity may result. Take the clause “A single-celled yeast can divide every 90-120 minutes” as an example. Since written language does not have an explicit indicator for the information structure, the way spoken language does, there are more than one ways of interpreting the clause. That is, written language cannot create the level of newsworthiness of some part of the information desired by some human activities for some purposes. To overcome possible ambiguity and meet the increasing demand of the textual metafunction, new resources are needed to fix and facilitate the Given ^ New structure; among them is a table. Three points may be noted about the table. First of all, a table is a demarcation of a visual space into the top-bottom and left-right quadrants as in: Top / left Top / right Bottom / left Bottom / right Second, such a demarcation of space is conventionally assigned Given ^ New structure, that is, either Top is Given and Bottom is New (i.e. the column head is on the Top and the body on the Bottom, a vertical orientation), or Left is Given and Right is New (i.e. the stubhead is on the left and the body on the right, a horizontal orientation), or both the Left and the Top are Given, the Right and the Bottom are New (i.e. the column head is on the Top and the stubhead on the left, the body is below the column head and to the right of the stubhead). By “conventionally assigned” I mean that there 77 is no absolutely correct orientation, one or the other might be more popular at a given time or in a given community. But for a given table such an orientation is fixed and cannot be changed any more. Thirdly, experiential functions take up each slot created by the grid, completing the table and bringing it to life, as it were. The experiential meaning of the New (in the body of the table) can be numerical (in numerical tables) or verbal (in word tables). The nature of the experiential meaning is not what tells us whether a spatial-linguistic composite is a table or not. As long as some experiential meaning fills up the slot, any meaning (Actor, Process, Circumstance and so on), then the table is alive. It is necessary to note, however, that the sequence the experiential items are arranged is often such that some regular patterns can be displayed in the table. An excellent example is the periodic table of the elements in chemistry. Dimitri Mendeléev (18341907) arranged all the known elements by groups and periods. The principles are that the element put in one group (one column) has the same number of electrons in its outer shell and that the atomic number of the elements in each period (one row) increases by one unit from the left to the right. By knowing an element’s position in the periodic table, we know its electronic configuration and hence its chemical properties. Thus in a table, the textual meaning of Given ^ New combines with the spatial arrangement of Left and Right and Top and Bottom, and with the experiential meaning. Textual meaning (e.g. Given ^ New) and experiential meaning are lexicogrammatical, common to spoken and written language, while the spatial arrangement is written only, making use of the print or written media. A table as an entity embodying the above three features simultaneously is a powerful means of making meaning. It is good for comparing and listing data and 78 gives a reader a particularly strong impact. No longer does the reader or writer have to look around for the New in a piece of information; whatever is put in the New slot (or the body of the table) is always and automatically New, and of course whatever is in the Given slot is always Given. The information structure is routinized. Many tables also deploy textual ellipsis so that both the headings and the body contain a minimum number of lexicogrammatical items (see Lemke 1998a: 96-101 and Baldry 2000b: 47-48 for a discussion). The reason for this is that ellipsis carries the Given ^ New structure to its extreme: if only two lexical items are presented, one the Given and the other the New, the possibility of ambiguity (between which is Given and which is New, and within the New which part is to be given most attention to) is completely eradicated. In other words, ellipsis maximizes the contrast between Given and New by omitting all irrelevant information, resulting in the bare thematic items: there is nothing else to distract the reader. Herein resides the power and clarity of a table. It may be noted that the numbers in the Cell of a table take an absolute stance toward the reality they construct; there is no Modalisation at all, meaning that whatever is included in the Cell appears as absolutely correct. The textual organization of a word table is similar to that of the numerical table. Like a numerical table, a word table stresses the division of the Given and New of a clause so that the New can be recognized at a glance and the New of one clause can be easily compared and contrasted with that of another clause. However, as illustrated in Chapter below, a word table may create a succession of Given and New. As well, in a word table, there may not be so much textual ellipsis as in a numerical table. The interpersonal and ideational meanings of word tables are also different from those of numerical tables in that the clauses or clause fragments in word 79 tables are the same with verbal language, complete with all three metafunctions except the tabular demarcation of Given and New. 3.2.2.3 Framework for Analyzing Graphs Statistical graphs for frequency distributions, which include bar graphs, histograms, frequency polygons and so on, derive from data tables, which in turn originate from linguistic and mathematical expressions of some quantitative relation between a set of variables. In most cases, statistical graphs make use of the coordinate system, that is, the horizontal x-axis designating the independent variable and the vertical y-axis designating the dependent variable stand for the Given, and the space circumscribed by the two axes is the New where the relations between the two variables are shown. Also the y-axis represents the quantitative information, ratio- or interval- scale data, which is capable of being scaled. This means that the y-axis’s quantitative values can be turned into visually perceptible heights which can be compared visually: the higher the bar or point, the higher the value of the y variable for the corresponding x variable. There is nothing in the height of the bar per se except its assigned meaning concerning the quantitative value. That is, a statistical graph, quite unlike a photograph, is an abstract theoretical entity although it may have material form: a photograph resembles the perceptible object while a statistical graph constructs a theoretical object, which may be invisible to human vision prior to its material formation. The framework for analysing statistical graphs, adapted from O’Halloran (1996: 161), is presented in Table 3.4. 80 Function REPRESENTATIONAL MODAL COMPOSITIONAL Unit GRAPH EPISODE Statistical reality: topological meanings, such as trends, continuous co-variations, correlation and frequency distribution; Comparisons of patterns of variation Accompanying text in the form of Caption, Title and Labelling which are emphasized by Size, Positioning, Underlining and Font; Colour, Line width, Shading, Line Solidarity, Arrows; Curvature; Perspective; Framing; Scale; Style of production; Directionality Gestalt: Framing, Horizontals, Verticals and Diagonals; Positioning; Use of Lines, Curves and Bars; Inter-connections established through symbolism and language for the labelling of Participants and Processes; Cohesion: links to the running text Change, or Relations between Figures Prominence of interplay Labelling of interplay Table 3.4 Functions and systems in graphs (Adapted from O’Halloran 1996: 161) 81 Function REPRESENTATIONAL MODAL COMPOSITIONAL Unit FIGURE PART Participants; Circumstantial features; Portrayal of co-variation associated with Process as a Curve, Line or Bar Prominence of individual figures; Displayed trend of process through Line, Bar, Curve Labelling of Figures through symbolism and/or language; Portrayal of Process between Participants as Axes and Figure with relative Positioning and Size of Figure and salient features as displayed by Lines, Curves, Bars, Colour, Line width, and Shadings Title; Axes, Scale, Arrows; Labels; Lines, Curves, Intersection points; Slope of Parts of the Figure Stylization; Cohesion: Parallelism, Contrast; Reference through language and/or symbolism Conventionalization Table 3.4 (Continued) 82 Compared with that for natural language, the Representational meaning of a statistical graph is more specialized in that it deals only with the relative numerical relationships between two sets of variables, or how an attribute of some entities, for example, the height or weight of school children, is distributed among a sample or a population, the latter being in essence a comparison between the entities in terms of the attribute. This visually expressed topological relationship powerfully complements the semantics of natural language, which is typically typologically oriented (Lemke 1998a). Depending on the nature of the variable designated by the x-axis, a curve within a coordinate system may mean either a material process, as the output of the crop increasing or decreasing, or a relational process, as the comparison between the entities. Interpersonally and Experientially, although in principle a graph is as reliable or unreliable as the data that informs its compilation and, for that matter, is subject to the semiotic choices made in the display (for example, the selection of the scale), whenever a student encounters a graph in the textbook, he or she is normally expected to believe it rather than doubt it. That is, the graph carries with it a self-authenticating power and a high modality. Further, as noted by O’Halloran (1999b: 18), with exceptions “[interpersonal] strategies for engaging the viewer of the mathematical visual display not operate through nuance as found in forms of art, but rather select for a direct unmarked command, ‘look here’”. Compositionally, x- and y- axes provide the basis or grounding where the New is expressed in the form of Curve, Line or Bar. The axes “contribute to stability and harmony” (O’Toole 1994: 23), while the Curve, Line or Bar “create[s] energy and dynamism” (1994: 23). 83 3.2.3 Reading Path in Multimodal Text Another crucial question with a multisemiotic text is the reading path it may create for its hypothetical reader. Underlying this question is the recognition of the page as a textual unit where various semiotic resources make meaning (Baldry 2000b: 42). As O’Halloran (1999a: 322) points out, “[w]ith multisemiotic texts, the most important stage is a step-by-step analysis of the text through the reading path determined by the choices within different semiotic codes”. It is to be noted that the reading path in a multisemiotic text identified by O’Halloran (1999a: 322) is not linear, from left to right, or from top to bottom, but typically follows some specific sequence (see also Kress and van Leeuwen 1996: 218 ff.,1998: 205-209; Kress 2003: 156-160). As will be illustrated below in the analysis of multimodal texts, there seem to be two aspects to the reading path: the intersemiotic aspect, i.e. how the reader is expected to shift his or her attention from one semiotic mode to another, and the intrasemiotic aspect, i.e. how the reader is expected to move from one component to another within one semiotic mode. Very often the intersemiotic aspect of the reading path in an introductory textbook is that after a brief “modal ‘scanning’ of the page” (Kress 2003: 159), or a quick perusal of visually salient elements, usually images, the reader moves from the verbal text (expressed by specific typographical features) to the non-linguistic resources and then to the verbal text again, thus following a back-andforth reading sequence. Initially the visual image on the page exerts a strong impact upon the reader through choices such as Colour and Framing. After the initial visual impact subsides, however, he or she normally begins to study the verbal text and may later be linguistically instructed to study the visual image in greater detail. The relative privilege the verbal text enjoys in the reading path is partially explained by the fact that 84 at this stage of the student’s education it is largely through the verbal language in the running text that he or she is instructed explicitly when to view and study the nonlinguistic resources and how to interpret them. In other words, how a multisemiotic text “indicate[s] to the reader / viewer the possible ways of reading the text and the relative information priority to be assigned to the different component parts of the overall visual composition” (Baldry 2000b: 42) and how a reader / viewer is expected to respond to the text constitute a visual semiotic strategy to realise the educational goals within which a multimodal text is constructed and interpreted. As the context of situation and culture (Halliday 1978) within which the text operates changes, the reading path, along with the nature of the semiotic resources employed, also differs. For example, Kress and van Leeuwen (1996: 219) and Lemke (1996: 216) have observed that many scientists have non-sequential reading habits and that many books are not designed “to be read only in the strict linear order in which the text appears on the pages” (Lemke 1996: 216). This applies to established scholars or scientists, people who are beyond their basic “military training” periods (Knight 1992: 6, 143) and for whom texts are created and read for research purposes rather than learning purposes. Students of biology or literature or any other field need eventually to learn non-sequential reading. While they are still students, however, they may need in the multimodal textbook page a clear, pre-coded reading path in order to enter the paradigms of contemporary science (Kuhn 1996) and the practices and conventions that characterize scientific activity. 85 [...]... that whatever is included in the Cell appears as absolutely correct The textual organization of a word table is similar to that of the numerical table Like a numerical table, a word table stresses the division of the Given and New of a clause so that the New can be recognized at a glance and the New of one clause can be easily compared and contrasted with that of another clause However, as illustrated... rather than by verbal description 3. 2.2.2 Framework for Analyzing Tables Gove et al (1986: 232 4) define a table as a systematic arrangement (as of numerical values) usu in parallel rows or columns for ready reference” This definition draws attention to several features of a table First, a table is arranged in rows or columns on a printed page or part of the page That is, it makes use of the spatial... inquiry of the discipline and its methodological approaches, biology texts have always been multimodal, that is, deploying a range of semiotic resources in addition to natural language The reason for this is clear: natural language alone cannot adequately communicate or construct the process and product of observation and experimentation; the potential of natural language as a typologicallyoriented semiotic... tabular demarcation of Given and New 3. 2.2 .3 Framework for Analyzing Graphs Statistical graphs for frequency distributions, which include bar graphs, histograms, frequency polygons and so on, derive from data tables, which in turn originate from linguistic and mathematical expressions of some quantitative relation between a set of variables In most cases, statistical graphs make use of the coordinate... morphological research; it is naturally insufficient as a means to describe both the morphological and the biochemical 3. 2.2 Theoretical Frameworks for the Analysis of Biology Texts Myers (1990: 233 -249) identifies, “in terms of realism and abstraction” (1990: 247), five categories of visual displays in a sociobiology text: photographs, drawings, maps, graphs / models / tables, and imaginary figures (1990: 234 )5... illustrated in Chapter 5 below, a word table may create a succession of Given and New As well, in a word table, there may not be so much textual ellipsis as in a numerical table The interpersonal and ideational meanings of word tables are also different from those of numerical tables in that the clauses or clause fragments in word 79 tables are the same with verbal language, complete with all three metafunctions... meaning Textual meaning (e.g Given ^ New) and experiential meaning are lexicogrammatical, common to spoken and written language, while the spatial arrangement is written only, making use of the print or written media A table as an entity embodying the above three features simultaneously is a powerful means of making meaning It is good for comparing and listing data and 78 gives a reader a particularly strong... REPRESENTATIONAL MODAL COMPOSITIONAL Unit FIGURE PART Participants; Circumstantial features; Portrayal of co-variation associated with Process as a Curve, Line or Bar Prominence of individual figures; Displayed trend of process through Line, Bar, Curve Labelling of Figures through symbolism and/or language; Portrayal of Process between Participants as Axes and Figure with relative Positioning and Size of. .. means that the semiotic demands of the discipline have exponentially increased so that natural language, however important it may be, is inadequate as a single resource As a result, other 70 semiotic means such as chemical notation, ball-and-stick models, space-filling models, animations, video recordings and so forth have evolved for communicative purposes Natural language alone has been inadequate... Horizontal aligning; Placement relative to the body of the table Table 3. 3 Functions and systems for numerical tables Table 3. 3 has a rank scale consisting of Table, Cell and Periphery Cell is the body of the table and Periphery provides essential information on how to read the table and includes title, headings (column head, stubhead, table spanner, column spanner), table number, and notes to the table . using basically the same framework: a trio -functional analysis and an interpretation that relates to its social context of creation and other discourses of art: art history, art critics and teaching depict a certain subject matter (ideational), adopt a particular interpersonal stance and have a particular means of textual coherence. Such descriptions follow the trio -functional analysis rather. resources in addition to natural language. The reason for this is clear: natural language alone cannot adequately communicate or construct the process and product of observation and experimentation;