Unaspiring Writer Articles About


Article / Multimodal Communication

The Terminology of Multimodal Communication

Roy K. Humble / 17 July 2020

This is Part 2 of a four-part series on Multimodal Communication that's intended as a supplement to The Humble Argument. Part 1 provides a general introduction to multimodal communication and its terminology. This articles introduces the central concepts and terminology of multimodal communication. Part 3 provides an overview of the general multimodal expectations that college professors have for your college paper. Part 4 wraps things up with more specific guidelines for different academic disciplines and some help with using visual elements within college papers.

To learn the basics of multimodal communication and apply them in college, we have to start with the concepts involved and that means learning the technical terms that scholars use to discuss them.

We'll tackle this in three waves of vocabulary building:

  1. The Rhetorical Situation:The vocabulary in this section defines the general elements of communication that are present whenever people share ideas that they want others to accept.
  2. The Modes of Multimodal Communication:This section focuses on the five main modes of communicating ideas or information to others. It ends with related terms that are useful for understanding what makes multimodal communication more or less effective as you move from one rhetorical situation to another.
  3. Other Important Terms: This final section offers a few other important terms that don't easily fit into the first two sections. They had to go somewhere.

1. The Rhetorical Situation

Rhetoric is communication that tries to influence the thinking of others. The rhetorical situation is when and where rhetoric happens.

Last night, for example, you were in a rhetorical situation when a Nike ad on TV tried to convince you that you should buy a new pair of running shoes even though you don't actually run, not by choice. This morning, your roommate created a rhetorical situation when she told you that you should put on a coat because it's cold outside and starting to rain. She was trying to change your mind about what you should wear, which was basketball shorts and a tee shirt. And as you sat down with your homework this afternoon, you found yourself in yet another rhetorical situation because it required you to write an essay that presents your opinion about a poem, an essay that your English professor will read, consider, and grade.

In The Humble Argument, the rhetorical situation is presented practically as the situation surrounding a college essay. Here, we look at the rhetorical situation is more theoretically so that it can provide a more flexible framework for understanding all kinds of multimodal communication.

The Main Ingredients

The examples above of running shoe ad, roommate's warning, and poetry paper all have three main ingredients, and these are the core ingredients for any rhetorical situation:

These three main ingredients in the rhetorical situation imply two other elements that are also worth considering:

Rhetorical Situation Analysis

Here's how these five ingredients form what unnamed experts claim is the second most painful rhetorical situation of all time, the job interview. The first most painful rhetorical situation is buying a used car.

A woman sits before a table of four skeptical interviewers.

These five ingredients are all present as Greek works in the explanation of communication from the ancient Greek philosopher Aristotle more than two thousand years ago. Western scholars have tinkering with these concepts ever since and have translated his Greek terms into more useful modern terms, but to be honest, nobody's really improved much upon Aristotle. What he described two thousand years ago is still the rhetorical situation because these are still the ingredients that are present when people try to change each other's minds.

The only thing that's really changed since Aristotle's time is that technological advancements have given us new ways to share our ideas with others — new modes of communication. In Aristotle's day, the spoken word was the main mode of communication and real time speeches or discussions were the main rhetorical situation. After the printing press became an important part of the world, the written word was king and allowed for asynchronous rhetorical situations between writers and readers.

Now that personal computers are so prevalent, there isn't really a king anymore. Personal computers allow us to combine images and words and sounds and even videos into a single digital text. The Internet allows us to weave resources from around the world into that same single text. At the same time, though, we continue to write papers and go to job interviews and simply argue with each other over coffee.

The Rhetorical Situation and Multimodal Communication

The rhetorical situation for multimodal communication in the writing classroom works like this:

Whatever the particulars of this rhetorical situation may have been, the result is that the senders have changed the thinking of the receivers over time. Multimodal is now the Big Thing in college writing. This is why college writers like yourself are now being asked to make videos and websites and posters instead of or in addition to old-fashioned academic essays. This is why entire departments of sentence-loving English teachers are being asked to teach low-grade versions of design, podcasting, videography, and public speaking in their writing classes.

This happens from time to time in English departments. When I was a young writing teacher, formal logic and Greek terminology was the next Big Thing among writing teachers. Slightly before then, young writing teachers took the conflict-resolution ideas of a marriage counselor, Carl Rogers, and turned them into the next Big Thing. Slightly after that, discourse communities were the next Big Thing. That was followed by the great hypertext scare, when the introduction of the World Wide Web made English visionaries predict that soon every college papers would be riddled with hypertext links, like at Wikipedia.

Eventually, multimodal communication will be claimed by the Speech and Communication department where it belongs, and young writing teachers will discover a new Big Thing to rally around and force upon their elders. Until then, however, it's here for you to learn, and learn it you will. As you wade into this, there are three things for you to keep in mind:

  1. Even though most writing teachers aren't particularly well qualified to teach multimodal communication, yours probably is. And even if yours isn't, you should still pay attention to them and to this guide because understanding the basics will help you be a more thoughtful receiver of multimodal messages and a better multimodal communicator.
  2. No matter what the Big Thing is at any given moment in college writing, the rhetorical situation is going to be a part of it. It isn't going anywhere, and it's always going to useful to you as a sender of messages:
    • As a sender of messages, it helps you consider your purpose in sending a particular message. Is it a noble purpose, one that's worth your time and creative energy and careful thinking?
    • If so, you can then consider the message. Is it a valid message? Is it reasonable? Can you explain it in sufficient detail and defend it with adequate evidence?
    • If so, then consider your audience, the receivers. What is the best way to communicate that message to them? (Here's where the different modes of communication come into play most prominently.) What does your audience expect from you in this rhetorical situation? What do you know about their preferences and values?
    • Do you share any common ground within the broader setting of this rhetorical situation? The more carefully you consider this and the other elements of the rhetorical situation, the more effective your communication will become.
  3. As a receiver of messages, your consideration of the rhetorical situation is a kind of self-defense against bad ideas trying to get into your brain.
    • Who is the sender of the message? What's the purpose of sending it to you? What's in for them if you let it influence my thinking? Those three questions will karate chop 99% of all advertising messages and expose them as manipulative attacks on your otherwise good judgment.
    • If you find the sender somewhat credible, take a look at the message itself. Does the main idea make sense? Has the sender provided sufficient, credible evidence for it? Are the reasons logical? What different modes of communication are working together here? Is there anything shady going on? Is a picture of a vegan bodybuilder telling you that you'll get a killer body by not eating meat?
    • Is the message relying on ideas or sentiments from the current setting to make its point? If so, is that appropriate or manipulative?. It usually doesn't take much questioning to expose shoddy thinking, and that helps you to keep bad ideas from becoming your own.

2. The Modes of Multimodal Communication

According to multimodal communication theorists, there are five main modes or modalities or ways of communication — linguistic, visual, aural, special, and gestural. We'll take a briefly look at each of these modes and how they might show up in a multimodal text.

Linguistic Mode

This is where you use your words — written or spoken. Using your words includes several important activities that can help to make your communication more effective, such as these:

For college papers, this is the most important mode for you to use. The more you understand it, the more effective your papers will be in presenting your message and convincing your audience that your ideas make sense. For other types of communication — speeches, posters, videos — it's still important but less so.

Visual Mode

The visual mode, as you might expect, refers to what you and your audiences sees. This means images, of course, but in a college paper, it can also include a lot of other features that help you to format and present your words so that they are easier to for readers to understand:

The visual mode also includes different ways to help readers focus on the key elements of illustrations, photographs, and images-based types of communication like videos or websites.

Aural Mode

This is what your audience hears. This is the primary mode of communication with public speaking, podcasts, radio, and other speech-based types of communication, but it's also an important ingredient in videos and film. This model extends the spoken words of the linguistic mode and the images of the visual modal with many other sound elements:

Spatial Mode

This mode presents information through the physical arrangement of images and information.

This is especially important when the communication is three-dimensional. How you arrange the chairs for your presentation to a class, for example, affects how that audience will respond. If you arrange them in a circle, you imply that you are connected to your audience and invite them to participate in the communication. If you arrange the chairs in rows that you stand in front of, you imply that you are the expert and that your audience should sit back, shut up, and receive your ideas.

The spatial mode also matters in two-dimensional forms of communication. How you divide website information into pages and tabs shows your readers how your information or ideas are structured and helps them to navigate to easily within that web-based presentation. If you put two illustrations next to each other on a page in your essay, the proximity of those images to each other suggests to readers that they are related. If you make one large and the other images small, it implies that one is more important than the other.

Gestural Mode

This is the way that people communication information nonverbally through facial expressions, body language, hand gestures, and touching.

If you cross your arms and say "I love you," for example, then your audience is getting a mixed message. The linguistic mode tells them that you love them, which is nice. However, the gestural mode tells them that you're also not 100% happy with them. They could do a better job helping out around the apartment. If nothing else, they could put on a coat before going outside in the rain.

You don't have any gestural modes with your college papers — except when you turn it in, perhaps, by slamming it down your professor's desk or something. However, this is really important to consider as you interact with others or as you watch the interaction of others.

3. Other Important Terms

In any rhetorical situation, the point of contact between the sender and receiver is a message, an idea. However, that idea is not transmitted directly from one brain to another. The idea has to first be translated into a tangible form — a television ad, a spoken suggestion, or a written paper. The vocabulary of multimedia communication offers a few important terms that relate to the tangible form of a message.

Text

Traditionally, the word "text" has describes something like a book or newspaper article or poem that is made with written words. With multimodal communication, however, written words are just one of many options, so "text" here refers to any kind of communication created with any or several or all of the following five modes. A speech is a text. A poster is a text. A video or podcast or graphic novel or Internet meme is a text. In the three examples from early in this section, the television ad, spoken suggestion, and poetry essay are all texts, too.

Medium

This one is tricky, so pay attention. The medium refers to whatever technology or mechanism is used to transmit the text from the sender to the receiver. It's the go-between.

So with a television ad text, which is basically a short film, the idea presented by that text is that you, friend, should buy some running shoes. The medium of television is the technology that transmits the text from the sender to the receiver.

With a message to put on a coat, your roommate could have texted you the text of that message on her phone. In that case, the text would have been a text, which is cool, and the medium would have been smartphone technology. She could have written you a note and left it on the back door. In that case, the text would be the note itself, and the medium would have been paper with sticky stuff on the back of it. However, instead of those impersonal media, your thoughtful roommate used the medium of direct, face-to-face conversation to transmit her message. She translated her idea into words that she sent to your ears with her voice. That was kind of her.

With the essay you are trying to write about that poem, the message will be your opinion of the poem. The text will be the essay you write that brilliantly explains and defends your opinion. The medium you will use to transmit your text will be ink on paper. You will print your words onto clean, white paper just as the assignment instructions require.

Affordance

This term refers to the strengths or potential uses of a given medium — the sort of messages that a medium allows you to transmit more effectively. Television, for example, is great for combining visual information, sound, and words into a cohesive message that is received and digested immediately. Face-to-face speaking is great for direct, face-to-face communication and allows you to add support to the message with gestures, such as your roommate's hands on her hips and frown. Paper is a great medium for messages that are received when the receiver has time to finally sit down and read the words on the page, probably after the kids are finally in bed.

Genre and Conventions

In the broader world outside the multimodal writing classroom, the term "genre" (ZHAHN ruh) usually refers to different types of creative works — written texts, music, or movies, for example. Fiction is a genre for writing, and nonfiction and poetry are genres. Blues is a genre of music. Horror and romantic comedies are two genres of movies.

What distinguishes one genre of writing, music, or movie from another is that each genre has different shared characteristics. As receivers of these texts, we expect that any text belong to a particular genre will have the features of that genre. You expect a fictional book to follow the convention of telling you made-up stories about made-up characters. You expect poetry to rhyme and be about love — or maybe you have a broader expectation after you take a poetry class, or maybe not. You expect a romantic comedy to end happily — horror films, not so much.

These expectations are known as conventions. Conventions are the rules that the texts of any given genre must follow.

With multimodal communication in college, "genre" can apply to any kind of multimodal of text, written or otherwise — class presentation, PowerPoint slides, research paper, or website, for example. This includes conventions for the message itself and the different modes of communication with the text.

Outside the college classroom, you'll find even more genres for multimodal communication — phone calls, texts, email, conversations with old people, and more. You might not have considered the conventions for these genres, but you still know what they are. You know how to follow them, and you can tell when someone violates them.

Every social media platform is its own genre, too. The conventions for Facebook are different from the conventions for Twitter or Instagram or whatever TikTok is. You'll find that the most effective posts on those platforms are the ones that follow those conventions in creative ways, not the ones that break them.

This discussion continues in Part 3 of this series, "The Multimodal College Paper."