Help Design does matter! Supporting knowledge development with design patterns and social computing


Silke Schworm, Markus Heckner


1 Leave a comment on Absatz 1 0 Help systems are meant to support learners in solving problems and tasks they cannot solve on their own. However, currently many help systems of standard software fail to reach these goals. A major problem is a gap between the intentions of the help system designers and the objectives of help users: The support offered by online help systems is often not perceived to be helpful. This originates in a lack of comprehensibility of the help content, due to its text-based format, and often a very technical style of writing. To overcome these problems, an online help system for a statistics software package was developed by keeping a strong focus on the users and their specific goals and tasks during the design of the system. The proposed system provides (a) content in problem-oriented units which were (b) each structured following the structure of design patterns, by providing a description of the problem, a solution to the problem as well as a link to related help topics. (c) Social Tagging was added to improve retrieval of help content while (d) a commenting function was included to provide a feedback channel between authors and users, which can be used to improve the contents after the deployment of the system.

2 Leave a comment on Absatz 2 0 A controlled experiment was conducted focusing on the effects of the implemented design pattern (problem-oriented units and animated screen captures to visualize the problem solution). Two groups were compared. One working with the newly developed help system, one with the help system regularly integrated into the software package. Thirty-six students of educational sciences where asked to solve complex tasks in the problem domain of the testing of. For the first task they were allowed to use the help system. Afterwards a transfer task had to be done without the help of the system. Results showed significant advantages for our proposed system, leading to less learning time, higher learning outcome and a higher acceptance by the users.

1     Considerable weaknesses of current help systems

3 Leave a comment on Absatz 3 0 Adequate help seeking and effective help use are metacognitive learning strategies. They form an important prerequisite for successful learning at university (Aleven, McLaren & Koedinger, 2006). In practice, however, many students ignore their need for help or they ask for help in a way, which does not foster their learning processes (Schworm & Fischer, 2006). This effect does not only occur in classroom teaching, but also when learning in computer-based settings. Existing help systems have considerable shortcomings, which can be ascribed to the (1) comprehensibility of the content and to (b) the format of help output.

4 Leave a comment on Absatz 4 0 Most outputs of a help query include lengthy step-by-step instructions, which take much time and effort to process, especially for novices. Users often get frustrated because of the large amount of detailed information (Renkl, 2002; Tidwell, 2006). In addition, users’ vocabulary differs from that of the designers (Furnas, Landauer, Gomez, & Dumais, 1987). Help provided by the system often does not adequately fit the learners’ prior knowledge and is written in technical jargon (Wittwer & Renkl, 2008). However, the language of an effective help system should be adapted to the prerequisites of the learners (Clark & Murphy, 1982). According to the obvious expert novice communication gap, there is also a need for adjusting the help systems’ instructional explanations and solution descriptions to learners’ help-seeking goals (Nückles, Ertelt, Wittwer & Renkl, 2007). Thirdly, novice users often have difficulties in providing an appropriate and clear-cut support request. This already concerns the early problem formulation stage of the help seeking process and research shows that the preciseness of problem formulation within the context of web-based computer assistance is influenced by the computer expertise of the help seeker (Nückles et al., 2007). Support systems should allow users some freedom in their choice of search terms by accepting different synonymous formulations of help requests.

5 Leave a comment on Absatz 5 0 Regarding formal aspects of help design, a first shortcoming is that help systems are usually presented in separate windows or new application sites on the internet. This “split-source” format can cause users to lose sight of their original problem, because they have to switch between multiple windows. Mapping the aspects of the task to the information of the help output and keeping the given instructional steps in mind by switching between the help system and the application can lead to extraneous cognitive load which prevents learning (Sweller, Merrienboer & Paas, 1998). Furthermore, there is a deficiency in the representation of help content. Even though several studies showed the helpfulness of graphical representation (Mayer, 2005; Schnotz & Bannert, 2003) and the usefulness of multiple representations for learning (Ainsworth, 2006), most help systems are still presented in a text-centric way. However, visuals (either animated or static graphics) can be used to clarify or to concretize complex issues or tasks and can thus promote learning and performance (Harrison, 1995). Existing help systems often give the user a set of screenshots to describe a procedure or to show specific menu options. However, if too many screenshots are necessary to give a clear description of a workflow, this can lead to irrelevant cognitive load (Sweller et al., 1998).

6 Leave a comment on Absatz 6 0 Finally, the hypermedia phenomenon of being “lost in hyperspace” (Conklin, 1987) is a difficulty that can as well occur within a web-based help system. The navigation structure of many help systems is often not easy to understand, because of the hypermedia structure of the help files. Browsing through the presented hyperlinks complicates the traceability of the help request and the users often get lost in dead ends or cannot return to the starting point of their search.

2     Requirements for successful online help seeking

7 Leave a comment on Absatz 7 0 Online help systems need to fulfill various requirements in order to be used successfully: Users need to find the appropriate help, the help artifact needs to be structured and presented appropriately, a feedback mechanism must be present so that the system can be improved through continuous user input. These groups of requirements were used to design the proposed help system. First: Finding Help. To construct an efficient and user oriented support system the designer has to consider all aspects of information- or help-seeking behavior. Therefore the information presented in help files has to be well structured and labeled suitably so that users can find and understand the topics provided by the system. By integrating a tagging function users can actively cooperate on the support system. Tags reflect the vocabulary of users and will therefore be better understood by users, as the specialized vocabulary of experts. User-generated tags are therefore particularly useful when the necessary expertise to formulate appropriate help requests is missing. Furthermore, navigation within the help system should not cause additional difficulties. The purpose of the search function should be to enable users to find the information that they are looking for using a language they are familiar with. However, in cases in which users cannot find the desired information by browsing and searching, asking should be possible within the bounds of the system. That means if problems during help use occur the user should be able to communicate them without leaving the system.

8 Leave a comment on Absatz 8 0 A second requirement of a help system is the efficient structure and presentation of help. According to Alexander (1977) a pattern describes a step-by-step instruction to solve a specific problem that occurs again and again in the domain of architecture. Those instructions contained experience-based solutions for common design problems in a language easily understandable for novices. Accordingly, a pattern generally describes a model solution on an abstract level. However, patterns do not stand alone, but rather build on each other and include cross-references between them. The resulting network is called a pattern language (Alexander, Ishikawa & Silverstein, 1977). Since the 1970s the concept of design patterns has frequently been adapted from different IT-domains (Borchers, 2001; Gamma, Helm, Johnson, & Vlissides, 2005; Tidwell, 2006; van Duyne, Landay, & Hong, 2003). The common idea of this concept has always been, to make workable and successful solutions by experts accessible and understandable for laymen (Borchers, 2001). This idea can also be applied to the context of help system design. Since the users of software are usually faced with similar, recurring, typical problems, design patterns are suitable for representing the solutions. Each pattern presents a solution to one given problem. Furthermore, the different patterns are highly interrelated to provide the user all the information that is necessary to accomplish a specific task. Patterns have a definite and consistent structure, which means each pattern contains the same parts in the same sequence with the same layout (Borchers, 2001; Gamma et al., 2005). There are four crucial elements of a pattern (Gamma et al., 2005): name, problem, solution and consequences. Consistent structure facilitates finding the relevant information and thus minimizes cognitive load (cf. Mayer, 2005).

9 Leave a comment on Absatz 9 0 A third requirement of a help system is enabling feedback and participation.

10 Leave a comment on Absatz 10 0 An online support system that will meet the standards of Web 2.0 must abandon the idea of a static and final product. Instead, by the implementation of Web 2.0 elements the idea of a dynamic support system will be pursued and the continual development and improvement are emphasized. Dynamic help systems are gradually developed by experts and users. Schauer (2005) defined, among others, user contributed value and co-creation to be two core issues of the social web. Since the development of technologies like blogging tools and wiki engines learners can participate in the construction process without a substantial level of specialized knowledge by easily publishing content in the internet (O’Reilly, 2005). Blogs are frequently updated web pages and list the articles they contain (referred to as posts) in reverse-chronological order (Nardi, Schiano, & Gumbrecht, 2004). A particularly useful feature of blogs for the context of help design is the ability of communication between blog-author and blog-reader respectively help-content-designer and help-content-user. Here, users can leave comments that include questions or suggestions to written blog posts. The above mentioned dynamics of the help system evolves from the collaboration between designers and users and blogging offers the possibility to bridge the communication gap between them. In addition, commentaries on a help issue can give feedback on the perceived quality of the help artifact. The help users may as well paraphrase the content using their own expressions and thereby help other users to better understand the help issue and the help designer to improve it.

3     Research questions

11 Leave a comment on Absatz 11 0 The help system is intended to be used within the problem solving process. Often several months or even years lie between the formal education and the practical application of the acquired skills, e.g. between the course on empirical data analysis and its application as part of a thesis project. A help system which is developed to avoid the weaknesses mentioned above is intended to be used within the statistics education of students of educational sciences and psychology who use statistics software for the analysis of their empirical data for example in the context of a master thesis. Even though students have some formal education on the statistics software they often appear to not be able to apply the software to their current problem. As students are considered to work on their master thesis without further support of a teacher, the software’s help system is expected to be an important help-source.

12 Leave a comment on Absatz 12 0 Designing the help system with a structure borrowed from design patterns and including elements of social computing is expected to offer help highly adapted to the learners needs regarding content and format of the presented help. According to our arguments stated above using the newly developed help system should be more appreciated by the learners and result in less learning time and better learning results.

13 Leave a comment on Absatz 13 0 The following hypotheses are stated:

14 Leave a comment on Absatz 14 0 The proposed help system

15 Leave a comment on Absatz 15 0 1.    reduces the time required for task solution compared to a conventional help-system.

16 Leave a comment on Absatz 16 0 2.    leads to better task solutions than a conventional help system.

17 Leave a comment on Absatz 17 0 3.    leads to better transfer results than a conventional help system.

18 Leave a comment on Absatz 18 0 4.    is more accepted by the learners than the software’s original help system.

4     Methods

19 Leave a comment on Absatz 19 0 Thirty-six students volunteered to take part in this study (N = 36; mean age: 22.19 years (SD = 2.01; 31 female and 5 male participants). All participants were students of the 4th term of the bachelor degree program “Educational Science” of the University of Regensburg. They all had introductory courses in statistics and quantitative research methods that also contained a short introduction to the software. However, competences concerning the use of the software can be considered to be rather low. Thus, it was not expectable to have any “highly skilled” participants in this sample. Former statistical training of the participants and the resulting abilities in using the software is known and prior knowledge was regarded to be rather low. Self-assessment was chosen just to reassure this premise. This consideration was confirmed by the students’ self-judgment on their prior knowledge about the statistics software. We implemented a control group design with the experimental group working with the newly designed online help and a control group working with the program’s standard help system. During the study the students learned basic skills on questionnaire data analysis using the statistics software. They were randomly assigned to one of the experimental conditions (n = 18 in each group). All participants of the experimental groups received the same learning task. The conditions differed concerning the help system available during the learning phase.

20 Leave a comment on Absatz 20 0 The participants worked in sessions of approximately 90 minutes. The session started with a learning task on the analysis of questionnaire data. When doing the learning tasks, the participants were allowed to use the help system whenever necessary. There were no restrictions of learning time. The experimental group worked with the newly developed help system. The control group used the standard help system, which is part of the software. The help system introduced here was built using WordPress (http://www.wordpress.org) an open source weblog system, enriched by additional modules developed by the authors.

21 Leave a comment on Absatz 21 0 The system contains two navigation levels. According to the organization principle, the menu structure at the left sidebar contains the help items grouped by topics. Additionally there is a constant link at the top of the screen that directly points back to the starting page. The position of the user within the system is always highlighted in the menu bar.

22 Leave a comment on Absatz 22 0 Each help page is structured according to the design pattern approach:

23 Leave a comment on Absatz 23 0 ·    Title. The title contains the key concept of the relevant procedure (e.g. recoding variables)

24 Leave a comment on Absatz 24 0 ·    Problem description of the anticipated problem of the learner the help issue is written for. Presuming that especially novices often do not know the relevant statistic procedure they are looking for, the problem description gives an example of a most common statistical problem that requires this procedure (e.g. “To avoid response biases in questionnaire data items are often formulated inversely. To agree upon this item has to be considered to be a disagreement upon the underling construct. To do statistical analyses all variables have to be coded in a consistent polarity”)

25 Leave a comment on Absatz 25 0 ·    List of related problems in case the current problem does not fully meet the learners needs (e.g. “Compare: Compute aggregated variables”)

26 Leave a comment on Absatz 26 0 ·    A step by step instruction where the users can chose according to their level of prior knowledge whether (1) to see an animation of the necessary steps or (2) to simply read a textual description. Here animated screen captures have been implemented to especially help novices reducing cognitive load by a split source format. The animation shows the software screen and the relevant procedure. Signaling of the most important parts of the screen (e.g. submenus and buttons to be clicked on) additionally reduces load for the low experienced user. The textual description for the expert user simply contains the step-by-step instruction comparable to the content of the standard help system.

27 Leave a comment on Absatz 27 0 ·    A comment box that enables the learner to directly contact the help author

28 Leave a comment on Absatz 28 0 ·    A tagging function to gradually improve retrieval

29 Leave a comment on Absatz 29 0 The standard help system showed on the left hand side the complete navigation structure. As it contains far more issues the participants of the control group received the starting page of the relevant help issue for each task to reduce time on task artifacts due to larger help content. The step-by-step description of the standard help contained exactly the same procedural steps as those in the experimental help system. However, the page does neither contain a concrete example were the procedure may be required nor an animated instruction.

30 Leave a comment on Absatz 30 0 Participants of both groups had to solve a learning task. During the learning task they were allowed to use the help-system. The learning task required the analyses of statistical data, which was included into a scenario. Then the participants were given two research questions they had to answer and the statistical problems they needed to solve to do so (transcoding variables, aggregating scales, checking reliability, calculating and interpreting correlations and t-tests). A maximum of 25 points could be achieved.

31 Leave a comment on Absatz 31 0 The transfer task was structured according to the structure of the learning task. The cover story was altered and a new set of data was offered. The tasks that had to be solved were formulated equivalently, but now, the help system was no longer available. Here as well a maximum score of 25 points was achievable.

32 Leave a comment on Absatz 32 0 At last there was a questionnaire. This instrument included some demographic questions, a question about the learners’ perceived prior knowledge on the statistics software as well as 17 questions concerning the acceptance of the help-system (e.g. “The content of the help was easy to understand”). The prior knowledge item had to be answered on a Likert scale from 1 to 5 with 1 indicating a low prior knowledge and 5 indicating high prior knowledge (“How do you appraise your skills on using this statistics software?”). Acceptance items had to be answered on a Likert scale from 1 to 5, too, with 1 indicating a positive and 5 indicating a negative judgment. We obtained a Cronbach’s Alpha of .92. The questionnaire as well assessed learners help use by the question: “When working on the five tasks. On how many of the tasks did you use the help-system?”

5     Results

33 Leave a comment on Absatz 33 0 The groups did not significantly differ in their self-judgment on prior knowledge on the statistics software, F(1, 34) = 1.15, p > .10, eta2 = .03 (experimental group, M = 1.39, SD = 0.50; control group, M = 1.22, SD = 0.43).

34 Leave a comment on Absatz 34 0 When solving the learning task both groups used the help system to an equally high extent (experimental group, M = 4.83, SD = 0.38; control group, M = 4.94, SD = 0.24). The high mean in both groups indicates an adequate need for help as intended by the relatively high difficulty of the task. There were no significant differences between the groups, F(1, 34) = 0.91, p >.10, eta2 = .03.

35 Leave a comment on Absatz 35 0 Firstly we assessed the impact of help design on learning time. Time was assessed in minutes (starting with after giving the instruction and ending with the collection of the task solutions). There were significant differences in time on task for the learning task, F(1, 34) = 27.67, p <.001, eta2 = .45, with the experimental group needing significantly less time (M = 30.56, SD = 3.38) than the control group (M = 43.61, SD = 9.97). There were no significant differences in time on task for the transfer task, F(1, 34) = 1.8, p >.10, eta2 = .05.

36 Leave a comment on Absatz 36 0 Secondly we assessed the impact of help design on learning outcome. The performed ANOVA showed a significant effect of the treatment conditions on learning task results, F(1, 34) = 59.57, p <.001, eta2 = .64.

37 Leave a comment on Absatz 37 0 The treatment conditions had a significant effect on transfer test performance as well, F(1,34) = 30.18, p < .001, eta2 = .47. The learners of the experimental group, showed significant higher transfer performance (M = 19.44, SD = 5.28) than those of the control group (M = 10.33, SD = 4.65).

38 Leave a comment on Absatz 38 0 Finally we assessed the impact of help design on the acceptance of the help-system. The performed ANOVA showed a significant effect of the treatment conditions on the acceptance of the help-system, F(1, 33) = 84.60, p < .01, eta2 = .71. The participants who worked with the newly developed help-system evaluated their help-system far more positively (M = 1.65, SD = 0.33) than the participants who worked with the software’s help-system (M = 3.20, SD = 0.63).

6     Discussion

39 Leave a comment on Absatz 39 0 According to Puustinen and Rouet (2009) it is so far assumed that an important difference between mere information search and help-seeking is the addressing of a human helper in case of help-seeking. However, behind every computer-based help system there is a person who designed and developed the system and there are other users working with the system. Implementation social web elements makes the existence of human support either by the help designer or by peers more salient an enables the implementation of a help system which is gradually adapted to users’ needs.

40 Leave a comment on Absatz 40 0 The study presented evaluated a help system that was intended to ease learners’ efforts on browsing and search within the help system by implementing the system according to the design pattern approach and including a tagging feature. The possibility to ask was integrated within the boundaries of the system by including a commenting function, which enables the interaction between the help designer and his or her target group. Additionally the system considered different help needs of users with different prior knowledge, by offering animated screen-captures showing the concrete help activity within the actual software. Thus the inexperienced user was not forced to match the content of the help-system to the actual software screen. The help system lead to positive effects on all the assessed dependent variables: The learners who worked with the new system needed less time to solve the learning task and showed better results in the learning task as well as in the transfer task where the help system was no longer available. Additionally, the help system was highly accepted by the learners. Thus the newly developed system scored on learning time, learning results and acceptance of the system. As we equalized the experimental groups concerning search effort for finding the appropriate help page by offering them the relevant starting page for the task at hand, differences in time on task should not be drawn back on the mere total amount of information of the help system but clearly is a result of its design aspects. However, some methodological shortcomings of the study have to be considered. The framework of the study did not allow using the full potential of our borrowing form of design patterns. For reasons of the planned experiment the page did not yet contain extended explanations for expert performers and a list of related problems and even though there was a tagging and blogging function implemented within the system a cross-sectional-study did not give any opportunity for the students to actually profit from any tags and comments. As the learners had low prior knowledge they surely have particularly profited from the animated screen captures that showed them in an integrated format the several steps necessary to solve their problem. We expect the given problem formulation to facilitate the understanding of the help content and its relevance for solving the task at hand. As the problem formulation is purposely written in plain sentences and easily to understand, especially novices should profit from this feature that clearly distinguishes the new system from the standard system.


41 Leave a comment on Absatz 41 0 Ainsworth, S. E. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16, 183-198.

42 Leave a comment on Absatz 42 0 Aleven, V., McLaren, B. M. & Koedinger, K. R. (2006). Toward computer-based tutoring of help-seeking skills. In S. A. Karabenick & R. S. Newman (Eds.), Help seeking in academic settings: Goals, groups, and contexts (pp. 259-296). Mahwah: Erlbaum.

43 Leave a comment on Absatz 43 0 Alexander, C., Ishikawa, S. & Silverstein, M. (1977). A Pattern Language: Towns, Buildings, Construction. New York: Oxford University Press.

44 Leave a comment on Absatz 44 0 Borchers, J. (2001). A pattern approach to interaction design. Chichester: Wiley.

45 Leave a comment on Absatz 45 0 Clark, H. H. & Murphy, G. L. (1982). Audience design in meaning and reference. In L. J. F. & K. W. (Eds.), Language and comprehension (pp. 287-299). Amsterdam: North-Holland Publishing Company.

46 Leave a comment on Absatz 46 0 Conklin, J. (1987). Hypertext: An Introduction and Survey. Computer, 20(9), 17-41.

47 Leave a comment on Absatz 47 0 ISO 9241-210:2010 (2010). Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems.

48 Leave a comment on Absatz 48 0 Furnas, G. W., Landauer, T. K., Gomez, L. M. & Dumais, S. T. (1987). The vocabulary problem in human-system communication. Communications of the. ACM, 30(11), 964-971.

49 Leave a comment on Absatz 49 0 Gamma, E., Helm, R., Johnson, R. & Vlissides, J. (2005). Design Patterns. Boston: Addison-Wesley.

50 Leave a comment on Absatz 50 0 Harrison, S. M. (1995). A Comparison of Still, Animated, or Nonillustrated On-Line Help with Written or Spoken Instructions in a Graphical User Interface. In I. R. Katz, R. L. Mack, L. Marks, M. B. Rosson & J. Nielsen (Eds.), Proceedings of the ACM CHI 95 Human Factors in Computing Systems Conference (pp. 82-89). Denver: Colorado.

51 Leave a comment on Absatz 51 0 Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 31-48). New York: Cambridge University Press.

52 Leave a comment on Absatz 52 0 Nardi, B., Schiano, D. J. & Gumbrecht, M. (2004). Blogging as social activity, or, would you let 900 million people read your diary? In Proceedings of the 2004 ACM conference on Computer Supported Cooperative Work (CSCW). Chicago, Illinois, USA: ACM.

53 Leave a comment on Absatz 53 0 Nückles, M., Ertelt, A., Wittwer, J. & Renkl, A. (2007). Scripting Laypersons’ Problem Descriptions in Internet-Based Communication with Experts. In F. Fischer, I. Kollar, H. Mandl, J.M. Haake (Eds.), Scripting Computer-Supported Collaborative Learning (pp. 73-89). New York: Springer Science + Business Media.

54 Leave a comment on Absatz 54 0 O’Reilly, T. (2005). What is web 2.0 design patterns and business models for the next generation of software. http://www.oreilly.com/pub/a/oreilly/tim/news/2005/09/30/ what-is-web-20.html.

55 Leave a comment on Absatz 55 0 Puustinen, M. & Rouet J.-F. (2009). Learning with new technologies: Help seeking and information searching revisited. Computers and Education, 53(4), 1014-1019.

56 Leave a comment on Absatz 56 0 Renkl, A. (2002). Learning from worked-out examples: Instructional explanations supplement self-explanations. Learning & Instruction, 12, 529-556.

57 Leave a comment on Absatz 57 0 Schauer, B. (2005). What put the ’2’ in web 2.0? http://adaptivepath.com/images/ publications/essays/What_puts_the_2_in_Web_20.pdf

58 Leave a comment on Absatz 58 0 Schnotz, W. & Bannert, M. (2003). Construction and Interference in learning from multiple representations. Learning and Instruction, 13, 141-156.

59 Leave a comment on Absatz 59 0 Schworm, S. & Fischer F. (2006). Academic Help Seeking. In H. Mandl & H. F. Friedrich (Eds.), Handbuch Lernstrategien (pp. 282-297). Göttingen: Hogrefe.

60 Leave a comment on Absatz 60 0 Sweller, J., Merrienboer, J. J. G. & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10, 251-296.

61 Leave a comment on Absatz 61 0 Tidwell, J. (2006). Designing Interfaces. Sebastopol: O’Reilly.

62 Leave a comment on Absatz 62 0 Van Duyne, D., Landay, J. & Hong, J. (2003). The design of sites. Patterns, principles, and processes for crafting a customer-centered Web experience. Boston: Addison Wesley.

63 Leave a comment on Absatz 63 0 Wittwer, J., & Renkl, A. (2008). Why instructional explanations often do not work: A framework for understanding the effectiveness of instructional explanations. Educational Psychologist, 43, 49-64.