"Cybersecurity Computer-Based Training and Technical Communication Design"

About the Author

Lysandwr McNary graduated from New Mexico Tech in spring of 2013 with a Bachelor of Science degree in Technical Communication, after years spent in varied pursuits, from book publisher to United States Army Psychological Operations Specialist. Her time with the US Army, including two years in Iraq, provided many opportunities for studying cybersecurity, communication, and human behavior.

Abstract

Literature Review

e-Learning Tools and Principles of Design

Technical communicators who create digital technical communication instruments such as web sites and e-Learning (computer or mobile based training) modules are increasingly challenged by evolving technology. They must not only keep up to date with the changing arsenal of techniques and technologies, but also discern the most effective modalities. This is especially true for e-Learning: Clark and Meyer warned in their foundational instructional science (2008) text of the dangers in over or under use of technology in ways that defeat knowledge transfer and procedural compliance. The rise of disciplines and design principles such as information architecture, games research, and virtual world creation (to list just a few) have radically affected the technical communication environment. As training development advances overall, the available construction options for complex, critical learning instruments require deeper examination as to suitability and effectiveness. Cybersecurity is a field where human behavior is a major factor in exploitable vulnerabilities, which can lead to criminal or malicious activity, and effective training proactively reduces risk. One could reasonably expect that organizations producing e-Learning or CBT modules specifically in fulfillment of FISMA training requirements in the rapidly advancing field of cybersecurity would look to development and assessment tools applied by technical communicators.

Information architecture has become the primary method for defining and organizing digital instruments. Generally, the result is a hierarchical navigation structure that presents logical pathways through the information space and allows effective information retrieval and use.

Crystal (2007) addressed fundamental principles in information architecture as applied to user information needs, and so illustrates how pervasively those constructs affect the work of technical communicators. In outlining the larger purpose of the article, which is to explain and promote the faceted classification method of organizing information for navigation, Crystal emphasized how information architecture has tended to apply classic principles of information organization to the human-computer interface. As design becomes more user-centered, and content organized in multiple navigation paths, Crystal argued that the commonly used hierarchic modes are overly rigid and single-focused. He promoted a faceted architecture, with the inherently subjective (even when critically judged) nature of information classified by users as well as designers. He reviewed extensive research on user-criteria for sorting and navigation judgments, and applies those to the information architecture framework to support dynamic interaction access.

The critical analysis Crystal performed is representative of how technical communicators must review techniques for best practices, especially as technology evolves, and continuously examine user interaction success. A decade or so ago, information architecture was an obscure if not unknown terminology—now the discipline is foundational to digital communication creation. The faceted framework premise appears to provide a more dynamic user interface, with user-relevant criteria driving the design. The instrument created is an organization of information objects, rather than a collection of pages and or links.

In continuing that line of thought, Ford and Mott (2007) addressed the role of technical communicators in the creation of information objects. They also recognized the appropriateness of the information architecture label to many technical communicators’ roles and responsibilities in recent years; discussing required skills, knowledge of latest technology, and ability to effectively understand and use the tools (primarily software) that best generate usable objects. They supported the requirement of effectiveness and user-relevance by emphasizing ISO 13407 (Human-centered design processes for interactive systems) standards in design and usability modeling for technical communicators. These standards require developers to “understand and specify the context of use; specify the users and organizational requirements; produce design solutions; evaluate designs against requirements.”(“ISO 13407” 2006)

Moving from the general aspect of technical communicators as digital media instrument developers to the specific of e-Learning creation, the issues of user-relevance and user-interface are of even higher priority. The pursuit of effective, successful training requires thorough examination of training goals and learner vulnerabilities. Are training objectives simple and repetitive? Are desired outcomes based on rote memorization, behavioral change, and/or comprehension and solution of complex situations? Will learners need to apply some mix of skills and newly acquired knowledge to achieve training success, and what are the consequences of failure? Padmanabhan (2009) described design approaches for technical communicators that focus on goal-based scenarios and support active learning by the participant.

Significantly, Padmanabhan differentiated between less complex instructional tasks that can be effectively communicated with traditional, linear materials; and critical learning situations where long-term recall is required and that have external repercussions. A consequence-free learning environment allows learners to make errors in order to develop a deep knowledge base, ideally providing sufficient feedback to learn from those errors while understanding the application to “real-world” events.

In developing digital learning environments, technical communication technology has perhaps most dramatically changed in the realm of creating virtual worlds. The Society for Technical Communications journal Technical Communication devoted an entire issue to the concept, and Williams (2008), as guest editor, addressed this most advanced of high-tech tools available in digital media development. He asked questions that technical communicators must consider: What happens when the information organization is out of the designer’s hands and under the control of the user, because they have so much ability to choose? How will designers apply information architecture to information that is spatially organized? How can the participation and interaction factors (often with each other) of users be used to advantage? These are the human factors most affected by rapidly developing technology.

Padmahan (2008) examined the relevance of human design in virtual worlds and connects those human factors to the success or failure of design. Human performance efficiency is paramount to success in the virtual world environment, and technical communicators must consider how content, tasks, and “presence” in the world affect usability. Usability and that sense of presence are a significant factor: If actions that would take place in the real life scenario cannot be fully replicated or convincingly substituted for in the virtual, effective use is diminished.

Bronack, et al. (2008) took the virtual world question further, outlining specific requirements technical communicators must consider in order to design effective virtual worlds:

• Comprehensive, thematic design of space. Design that consistently renders an identifiable environment provides for shared concepts and enables communication. Using metaphors to construct recognizable themes serves “as a key bridge in the process of human reasoning and understanding abstractions. (262)”

• Promoting a sense of presence (environmental, personal, and social.) Presence is a “precursor to interactivity—among people and between people and information (262)” and prompts participant engagement. Evidence of participant interaction (a door left open in a virtual building), avatar use, and outlets for communication are methods of creating presence.

• Consideration of human behaviors in online social environments. With novel ways available to interact in virtual world environments, technical communicators need to consider the impact of context and community modeling on participant behavior.

Not all online training environments are social environments, but there is a social aspect to human behavior in a virtual world –the sense of shared reality, of being able to communicate with others even indirectly. Bronack, et al. also pointed out the degree to which virtual world participants rely on classic face-to-face type behavior with their avatars, for example moving to line of sight when communicating even though it is clearly unnecessary. Participants also interact differently with other entities they believe to be human-operated as opposed to computer-operated. This presents particular challenges when considering and testing virtual world design for cybersecurity training, where security requirements so rigidly bind the human interaction element even at the lowest level of information awareness training.

Araki and Carliner (2008) reviewed literature addressing the differences between gaming and social virtual worlds, and how technical and education content is communicated in either. Their material highlights the virtual world value of combining simulated learning with reaping the benefits of developing skills, a primary tenet of game-oriented virtual worlds, and points out the complexity of creating professional avatars. They largely ignore the effective professional training aspect of virtual world environments, but Araki and Carliner did bring up IBM’s guidelines for professional behavior and British Petroleum’s Second Life (Linden Lab’s social virtual world) employee ethics and compliance program in-world site. In-game rewards as stimulus for progress are another element of game theory explored in social virtual worlds as well.

Cybersecurity and CBT

As noted, computer based training (CBT) or e-Learning modules developed specifically for information security in compliance with FISMA require adherence to NIST guidelines (2009) even at the lowest level of information awareness. As previously described, there are four federal entities designated as ISS LOB Tier 1 Awareness Training Shared Service Centers (SSCs), certified to provide awareness training material via web, computer, or delivered in person. Their training material is intended for all users of federal information and information systems.

A baseline of effective training modalities and their usability results would therefore seem a necessity. Yet a review of the literature regarding CBT training element evaluation doesn’t demonstrate that sufficient research has been performed to define best practices and techniques specifically for the development of information awareness training modules.

Greitzer, et al. (2008) defined and addressed the “insider threat”, where deliberate or unintentional actions by personnel within the security perimeter create exploitable vulnerabilities leading to cybercrime. They pointed out the clear need for compliance training in information assurance standards, and the need for organizations to take active rather than reactive steps regarding the increasing potential risk. In 2004 to 2006 surveys, companies reporting insider cybercrime events increased by nearly one-third. At the Pacific Northwest National Laboratory (PNNL) CYBERCIEGE training system, researchers are developing complex, interactive CBT involving serious gaming approaches, in addition to examining game-based training effectiveness. However, their simulated scenarios have involved limited small group models, and they have yet to assess the full learning success of the training. In addition, this training is primarily to satisfy Tier 2 and higher users, rather than the “audience of everyone” NIST mandates for Tier 1 as outlined previously.

In an examination of common myths and misperceptions regarding cybersecurity and information awareness, Furman et al. (2012) interviewed participants who needed to be alert to online and computer security threats. They found participants lacked the skills required to effectively maintain cybersecurity. Weaknesses in this study include that usage data was self-reported; that the interview population was relatively small and not necessarily representative; and that no clear correlation with any specific training effectiveness was drawn. Rather, Furman et al. reached the simple overall conclusions that misperceptions are common and training is insufficient.

Greitzer, et al. in the 2007 article “Cognitive Science Implications for Enhancing Training Effectiveness in a Serious Gaming Context” stated that “what is lacking is an active learning paradigm—grounded in principles of cognition—that helps ensure that students learn the functional value of the material by working directly with the content” (2:3). They examined the 'serious gaming' methodology of creating training to check effectiveness by applying cognitive learning principles. In this study of the game-based cybersecurity training system CYBERCIEGE, they followed up participant evaluations with heuristic evaluations by education experts. But as in the previously quoted paper, the study groups are small and narrow in user skill range.

These studies show that effectiveness of gaming, goal-oriented design, and virtual world use for cybersecurity training are not entirely untested. That said, we can see how limited the testing has been. Professionally trained technical communicators can and should apply e-Learning and design principles more extensively to CBT cybersecurity training than has been seen so far. This apparent deficiency may be due in part to a lack of TC awareness by the cybersecurity industry, which this essay hopes to address.

Assessing usability and effectiveness is a constant challenge. Ideally, usability research identifies the elements to be assessed, then how best to assess them, based on target audience needs and capabilities. Experts in any field are generally expected to expedite skill acquisition by novices, and identifying issues a learner would experience in performing a new task or completing a training module would seem an obvious application of expertise. However, when subject matter experts test and assess CBT training they may not adequately analyze usability by target users due to lack of empathy.

Lentz and deJong (2008) demonstrated that there is some evidence that experts use cognitive shortcuts that actually interfere with predicting novice behavior and identifying user issues. They described a study where cell phone users with various levels of experience and expertise (from cell company employees to novices) predicted how long a novice user would require to complete a set list of actions with an unfamiliar phone model. None of the respondent groups were very accurate in their predictions, but the expert users were farthest off by a factor of 200% (compared to 33% by the intermediate user group.) A second study described by Lentz and de Jong in the same paper resulted in IT experts greatly overestimating the ability of laypersons to demonstrate technical knowledge, which certainly applies to how usability of cybersecurity training may be inaccurately assessed by IT experts.

Training assessment research in cybersecurity ranges from the highly simplistic (“Do you know what phishing is now?”) to highly technical overviews by experts who may be more focused on the complexity of knowledge involved than how capable users are of absorbing the cognitive load. When Abawajy and Kim (2010) analyzed cybersecurity training for performance results, they observed that security and information awareness requirements do not presently have a well-developed and tested training delivery method. They attempt to identify to some degree the most effective delivery vehicle(s), but don’t go far beyond identifying issues and benefits with various systems and methods. They discuss steps taken by training designers to engage users with web-based and game-based security awareness training that are obviously within the realm of technical communicators, such as well-applied graphics, in-module assessments, and challenging content. But they don’t appear to identify a clear target audience and analyze appropriate needs and abilities before building their testing system and developing assessment questions.

Yoon et al. discussed how to evaluate and address the critical training requirements of government agencies (with Partridge in their 2008 paper), and how Purdue University developed a computer-based “decision support” prototype (with the participation of the Indiana Department of Transportation, the Indiana State Police, and the Indiana Department of Homeland Security) to allow complex training simulations. The case study looked at four transportation security drills over two years in order to assess the effectiveness of the prototype as a training tool. In these mock drills, the CBT was used as part of the ongoing scenario as opposed to training in advance of the objective testing.

In their follow up 2010 paper, they further outlined the pre-testing (by survey, field observations, and interviews) and technology considerations that lead into development of the collaborative CBT. The collaborative computer training (CCT) prototype developers examined combined interactive media, game theory, and virtual reality applications. The overall perceived effectiveness of the training tool is assessed, but the study does not delve into identifying specific aspects that were particularly effective in communicating relevant material, increasing knowledge retention, and ensuring behavioral compliance to desired objectives. Also, although the CCT prototype is certainly an example of a well-assessed CBT tool, the real-time usage factor moves it away from the asymmetrical and generally non-collaborative training commonly developed for information assurance.

Information awareness CBT is a high-demand product; however, due in part to a lack of cross communication (judging by papers produced in the field) between computer science/information technology professionals and technical communicators, acknowledged best practices of instructional digital instrument design and general technical communications are not applied industry-wide. Organizations would benefit from technical communicators well-educated in the specifics of cybersecurity training needs, and technical communicators would benefit from a thorough knowledge of FISMA and NIST mandates and training requirements.

Pages: 1· 2· 3· 4· 5· 6· 7· 8· 9· 10· 11· 12

Posted by on May 21, 2021 in article, Issue 9.2