Multimodality Satellite Talks: November – December 2017

Day: Thursday 2nd November 2017

Time: 3-4pm

Place: Parkinson SR (B.11)


Multimodal knowledge communication in contemporary academic and corporate context

Carmen Daniela Maier, University of Aarhus


We are witnessing today a ground-breaking expansion of multimodal genres that affect the communication of specialized knowledge in academic and corporate contexts in ways unforeseen before. By adopting a conceptual framework born at the intersection of multimodality, intertextuality, resemiotization and remediation, in my present research work, I explore genres belonging to academic and corporate contexts.

My analytical focus is on the tensions that appear between the domain specific knowledge and genre specific knowledge in transitional, intermediate, borrowed or new genres; for example, in academic visual essays, in academic video essays, in various corporate films, etc. I also examine how specialized knowledge from various domains of activity is more and more successfully communicated within sites of appearance and dissemination whose rationale is both educational and commercial; for example, in the multimedia kits of symposiums for academic and corporate communities.

This interest in the rapid and on-going proliferation of various multimodal generic forms is motivated by my wish to explain how the ways in which knowledge is created, communicated and disseminated is radically transformed when new generic structures, purposes and even networks are appropriated across professional domains, semiotic modes and media.



Carmen Daniela Maier, PhD, is Associate Professor at the Department of Management, Corporate Communication Research Section, School of Business and Social Sciences, Aarhus University, Denmark. She has published work related to corporate communication, multimodality, social semiotics, knowledge communication and sustainability. In the field of multimodality, her research work also includes being a member of the editorial board of peer-reviewed journal, Multimodal Communication. She is also the co-editor of Interactions, Images and Texts: A Reader in Multimodality (De Gruyter). Her latest publications include the chapter Exploring Organizational Heritage Identity: The Multimodal Communication from the book New studies in multimodality: Conceptual and methodological elaborations (Bloomsbury).

Her research work conducted together with her colleagues has brought her three international prizes: Distinguished Award STC NY Metro Chapter (2008), International Technical Publications Competition Merit Award Winner (2008) and Enrique Alcaraz Research Award (2015).




Day: Thursday 16th November 2017

Time: 2-3pm

Place: Baines Wing SR (1.15)


Multimodality and Discourse Systems

Ilaria Moschini, University of Florence


The talk stems out of my research on the hybridization of US political and institutional discourse with the language of the web (e.g. Moschini 2016, 2017), where I have underlined how the exploration of the ideological load of the context of use (which is never neutral) is crucial to fully ‘unpack’ the realization of meaning in “webridized” discourses. Digitally mediated textualities appear to be shaped – along with the discursive practices favored by the digital platforms or by the communities of use – by the values encoded in such platforms. And those values are not only the ones related to the globalization of markets (Fairclough 2011), but also the ones related to the aesthetics and to the worldviews of the people who designed them (Coleman 2013).

Through a selection of examples, the talk will critically argue for the necessity of combining the multimodal exploration of how meaning is created with the investigation of the governing philosophies of the “Discourse Systems” (Scollon, Wong Scollon & Jones, 2012 [1995]), that is the worldviews which underpin digital platforms. Indeed, personal computers, the Internet and social media are cultural artifacts that originate out of a peculiar historical blending of US liberal culture, DIY ethos and Sci-Fi Fan Culture (Isaacson 2014; Turner 2006) and that still present deep traces of such an ideational architecture.



Coleman, G.E. (2013). Coding Freedom. The Ethics and Aesthetics of Hacking. Princeton-Oxford: Princeton UP.

Fairclough, N. (2011). “Discourse hybridity and social change in Critical Discourse Analysis”. In Srikant Sarangi, Vanda Polese and Giuditta Caliendo (eds.), Genre(s) on the Move: Hybridisation and Discourse Change in Specialised Communication. Napoli: Edizioni Scientifiche Italiane, pp. 11-26.

Isaacson, W. (2014). The Innovators. How a Group of Hackers, Geniuses and Geeks Created the Digital Revolution. New York: Simon & Schuster.

Moschini, I. (2016). “The Participatory Stance of the White House on Facebook: A Critical Multimodal Analysis”. Multimodal Communication, 5 (1), pp. 41-55.

— (2017). “A Digital ‘Meeting Place’? A Socio-semiotic and Multimodal Analysis of the Social Hub”. In Rita Salvi and Judith Turnbull, The Discursive Construal of Trust in the Dynamics of Knowledge Diffusion. Newcastle Upon Tyne (UK): Cambridge Scholars Publishing, pp. 184-205.

Scollon, R., Wong Scollon, S. & Jones, R.H. (2012 [1995]). Intercultural Communication. A Discourse Approach. Malden, MA & Oxford: Wiley-Blackwell.

Turner, F. (2006). From Counterculture to Cyberculture. Stewart Brand, the Whole Earth Network and the Rise of Digital Utopianism. Chicago (IL): University of Chicago Press.



Ilaria Moschini is tenured Assistant Professor in English Linguistics at the University of Florence (Italy), Department of Languages, Literatures and Intercultural Studies. Her main research interests are US political and institutional language, media/new media language and US culture that she investigates using a framework that combines systemic functional linguistics and critical discourse analysis with a socio-semiotic approach to multimodality. She has published several essays on the linguistic/semiotic analysis of texts from different discursive areas (politics, media and advertising) and a volume on the evolution of American imaginary. More information at:




Day: Thursday 23rd November 2017

Time: 3-5pm

Place: Baines Wing SR (2.06)


2 talks:

  • “This is chaos, baby!” Epistemologies, transcription methods and research agendas in video-based communication, by Maria Grazia Sindoni, University of Messina
  • Multimodal Communication, Politics and Social Media: Verbal and Visual Language in Donald Trump’s Facebook Page, by Massimiliano Demata, University of Bari


“This is chaos, baby!” Epistemologies, transcription methods and research agendas in video-based communication

Maria Grazia Sindoni, University of Messina


The analysis of spontaneous interaction is a research priority within social and linguistic sciences (cf. Bezemer and Jewitt 2010: 181), such as conversation analysis (Sacks, Schegloff, and Jefferson 1974; Sacks 1992), interactional sociology (Goffman 1981), interactional sociolinguistics (Gumpertz 1999), linguistic anthropology (Duranti 1997), micro-ethnography (Erickson 2004) and linguistic ethnography (Creese 2008). However, synchronous video-based communication has called for a profound rethinking of theories and methods (Sindoni 2013). New concerns have started to circulate since the early 2000s (Norris 2004; Herring 2013), thus encouraging the shaping of an emerging field of analysis, that has contributed to identify the somewhat blurry boundaries of the research arena of web-based video interactions.

In this talk, I will critically review some of these concerns, with reference to three main areas:

  1. epistemologies, or why existing theories should be reshaped or new frameworks of analysis should be developed and consequent definition of the field/s and data (e.g. what are data and relevant semiotic resources/modes?);
  2. transcription and annotation methods, or how to collect and handle data (e.g., ethical issues and data contamination), considering that transcription and annotation are unavoidable methodological questions when dealing with multimodal data (e.g., manual vs. semi-manual or automatic transcription; text-based vs. multimodal-based annotation; mono-resource vs. multiple resource transcription);
  3. research agendas, or to which end research questions should be framed and how to prioritize goals, so as to make sure that findings can be tested and verifiable, and experiments are replicable.

To address these interrelated questions, I will adopt a top-down approach and draw upon different data sets to the end of exploring some of the most challenging areas in web-based video research galaxy, with a view to possible applicative contexts, such as in learning/teaching environments.



Bezemer, J. and Jewitt, C. (2010), Multimodal Analysis: Key Issues, in Litosseliti, L. (2010, ed.), Research Methods in Linguistics, Continuum, London & New York, pp. 180-197.

Creese, A. (2008), Linguistic Ethnography, in King, K. A. and Hornberger, N. H.  (2008, eds.), Encyclopedia of

Language and Education, 2nd Edition, Volume 10: Research Methods in Language and Education, Springer Science Business Media LLC, New York, pp. 229-241.

Duranti, A. (1997), Linguistic Anthropology, Cambridge University Press, Cambridge.

Erickson, F. (2004), Talk and Social Theory: Ecologies of Speaking and Listening in Everyday Life, Polity Press, Cambridge.

Goffman, E. (1981), Forms of Talk, Blackwell, Oxford.

Gumperz, J. J. (1999), On Interactional Sociolinguistic Method, in Sarangi, S. and Roberts, C. (1999, eds.), Talk,

Work and Institutional Order: Discourse in Medical, Mediation and Management Settings, Mouton de Gruyter, Berlin, pp. 453-471.

Herring, S. C. (2013), Discourse in Web 2.0: Familiar, Reconfigured, and Emergent, in Tannen, D. and Trester, A. M. (2013, eds.), Georgetown University Round Table on Languages and Linguistics 2011: Discourse 2.0: Language and New Media, Georgetown University Press, Washington DC, pp. 1-25.

Norris, S. (2004), Analyzing Multimodal Interaction. A Methodological Framework, Routledge, London & New York.

Sacks, H. (1992), Lectures on Conversation, Blackwell, Oxford.

Schegloff, E. A., Jefferson, G. and Sacks, H. (1977), The Preference for Self-Correction in the Organization of Repair in Conversation, in «Language», 53, pp. 361-382.

Sindoni, M. G. (2013), Spoken and Written Discourse in Online Interactions. A Multimodal Approach. Routledge, London & New York.




Maria Grazia Sindoni, PhD, is Associate Professor in English Linguistics and Translation at the University of Messina (Italy). She has published four books, articles in national and international journals, and edited two books. Her most recent book is “Spoken and Written Discourse in Online Interactions. A Multimodal Approach” (Routledge, London & New York, 2013). Her main research interests include systemic-functional linguistics, multimodality, testing and assessment, corpus linguistics, critical discourse analysis, theories of semiosis of communication and computer-mediated interaction.



Multimodal Communication, Politics and Social Media: Verbal and Visual Language in Donald Trump’s Facebook Page

Massimiliano Demata Università di Bari


According to a multimodal view of meaning, language is no longer considered central in the communication of meaning in the written form.  Rather, multimodality involves the simultaneous use of different semiotic systems, such as language, layout, images, etc., to create and communicate meaning. Facebook is made by units of meaning, such as “posts”, which are constituted by the interaction of verbal and non-verbal modes.  While verbal language is still central in the users’ experience of Facebook, non-verbal semiotic systems, such as visual images, are becoming increasingly central in the scope of the social platform. Meanings “are shaped by the norms and rules operating at the moment of sign-making, influenced by the motivations and interests of a sign-maker in a specific social context” (Jewitt 2013, pp. 15-16).  On the basis of this assumption, this talk will discuss how social meanings are encoded in Trump’s Facebook page, and how relations of power are played out in it.



Massimiliano Demata is Assistant Professor of English Language and Translation at the University of Bari, Italy. He took his DPhil in English at St Cross College, Oxford in 1999 and was a Fulbright Research Scholar at Yale University (1999) and Indiana University (2014). In 2008 he published his monograph, Representations of War and Terrorism. The Ideology and Language of George W. Bush.  He has published extensively on the language of British and American media

and politics, Computer-mediated communication and translation and ideology.  His current research focuses on social media and Multimodality in the context of American politics.




Day: Thursday 30th November 2017

Time: 3-4pm

Place: Baines Wing SR (2.15)


Persuasive technology: a new facet of digitality

Sandra Petroni University of Rome “Tor Vergata”


Digital textuality involves a wide range of text types today. Most of them are “remediated” discursive practices (Bolter 2001, Bolter and Grusin 1999), e.g. digital news reports, articles, interviews, video-conferences, etc.  Others are “prototypical constructions” (Santini 2007, Petroni 2011), i.e., new instantiations of emerging genres, or better, genres still in formation, often without a name, a label, and not fully standardised and acknowledged, such as a wiki text.

What gives rise to these new meaning-making processes, in particular to these new forms and potential pattern structures, are the technological capabilities and the functionality of the medium itself (Shepherd & Watters 1999). On the one hand, these affordances shape the way social actors (users) interact in digital settings (web pages or sites, social networks, collaborative productions, or mobile applications) by creating new forms of social practices. In digital contexts, in fact, user agency has been profoundly transformed thanks to the presence of the two potentialities of interaction and interactivity. All the new digital practices and communicative exchanges are mediated and filtered through the new digital technologies. Moving from real contexts to digital ones implies the presence of mediating technologies, embodied by interfaces, which re-model the nature of user agency in terms of agentic value, identity, social positioning, etc.. On the other, these technologies provide tools that can affect people’s attitude and engagement and push them to carry out digital actions, such as posting, commenting or sharing but also logging in and clicking on links which, additionally, imply evaluation (Martin & White 2007).

The aim of this talk is to show how “persuasive” technologies (Fogg 2009), along with the digital actions they endorse, are created and used to encage users in these settings and to affect what people think and do.



BOLTER, J.D. (2001). Writing Space. Hillsdale, NJ: Lawrence Erlbaum Asssociates;

BOLTER, J.D., GRUSIN, R. (1999). Remediation. Understanding New Media. Cambridge MA: MIT Press;

FOGG, B.J. (2009). “The new rules of persuasion” available online at; accessed March 2017;

MARTIN, J.R., WHITE, P.R.R. (2007). The language of evaluation. New York: Palgrave Macmillan

PETRONI, S. (2011) Language in the Multimodal Web Domain. Toronto-Rome: Legas-Aracne;

SANTINI, M. (2007). “Characterizing genres of web pages: Genre hybridism and individualization” available online at:; accessed April 2010.

SHEPHERD, M., WATTERS, C. (1999). “The functionality attribute of cybergenres”. Proceedings of the 32nd Hawaii International Conference on System Sciences. Los Alamitos CA: IEEE-Computer Society.


Biosketch. Sandra Petroni is Associate Professor of English Linguistics and teaches at the Department of Humanities of the University of Rome “Tor Vergata”, on the Languages in the Information Society Degree Course. Her research fields are critical discourse analysis, multimodality, digital communication, CMC,  and specialized discourse – in particular the Information and Communication Technology (ICT) discourse.




Day: Thursday 7th December 2017

Time: 1-2pm

Place: Baines Wing SR 2.16



Rescuing the Image. Re-conceptualizing the Role of Visuality in Eighteenth-Century Acts of Reading.

Alessio Mattana (University of Leeds) and Giacomo Savani (University of Leicester)


Abstract: Contemporary critical studies on the culture of the long eighteenth century have tended, across different disciplines, to overlook the function and impact of images for the readers of the age. In this work-in-progress talk, Giacomo (Roman archaeology) and Alessio (English literature) investigate and problematize the importance of visuality in eighteenth-century acts of reading. By focusing on two preliminary case studies – that of a misleading etching depicting a set of Roman baths and that of Robert Hooke’s book on microscopic observation Micrographia (1665) – they aim at fostering discussion by questioning accepted critical practices and sketching possible research leads to advance a more complex understanding of the nexus between image and text.



Alessio Mattana is a doctoral student in English. His research interests lie in the intersection between literature and natural philosophy in the long eighteenth-century, with particular focus on Newtonian philosophy and the wide genre of ‘history’.


Giacomo Savani has recently obtained a PhD in Roman Archaeology at the University of Leicester. His research interests lie in the field of Roman social and cultural history, Roman art and architecture, and the reception of the Greco-Roman culture in Early Modern Europe.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s