Publications

Jeong, E. & Ryu, H. (2016). Melodic Contour Identification Reflects the Cognitive Threshold of Aging, Frontiers in Aging Neuroscience

Abstract
Cognitive decline is a natural phenomenon of aging. Although there exists a consensus that sensitivity to acoustic features of music is associated with such decline, no solid evidence has yet shown that structural elements and contexts of music explain this loss of cognitive performance. This study examined the extent and the type of cognitive decline that is related to the contour identification task (CIT) using tones with different pitches (i.e., melodic contours). Both younger and older adult groups participated in the CIT given in three listening conditions (i.e., focused, selective, and alternating). Behavioral data (accuracy and response times) and hemodynamic reactions were measured using functional near-infrared spectroscopy (fNIRS).

Our findings showed cognitive declines in the older adult group but with a subtle difference from the younger adult group. The accuracy of the melodic CITs given in the target-like distraction task (CIT2) was significantly lower than that in the environmental noise (CIT1) condition in the older adult group, indicating that CIT2 may be a benchmark test for age-specific cognitive decline. The fNIRS findings also agreed with this interpretation, revealing significant increases in oxygenated hemoglobin (oxyHb) concentration in the younger (p < 0.05 for d(pre - on task); p < 0.01 for d(on – post task)) rather than the older adult group (n.s for d(pre - on task); n.s for d(on – post task)). We further concluded that the oxyHb difference was present in the brain regions near the right dorsolateral prefrontal cortex. Taken together, these findings suggest that CIT2 (i.e., the melodic contour task in the target-like distraction) is an optimized task that could indicate the degree and type of age-related cognitive decline.

Link to read more, http://www.ncbi.nlm.nih.gov/pubmed/27378907

Lee, H., Parsons, D., Kwon, G., Kim, J., Petrova, K., Jeong, E., & H. Ryu. (2016). Cooperation begins: Encouraging critical thinking skills through cooperative reciprocity using a mobile learning game, 97, 97-115

Abstract
Mobile learning has the advantage of being able to be used within and between contexts and can also be seamlessly integrated into broader learning experiences that include other forms of learning. Such experiences can assist in the development of cognitive and collaborative skills by encouraging learners to work together to solve problems, see others’ perspectives and cooperatively find creative and critical solutions. This paper describes a serious mobile learning game designed to allow participants to play the role of business consultants to an organisation facing some serious challenges. It uses mixed reality resources to lead the players through a realistic scenario, providing them with physical, cognitive and collaborative challenges. Following mobile learning, the learners demonstrate their critical insights into the learning content by creating a consulting presentation in the classroom. Our study contrasts group cooperation where each learner is given asymmetric learning contents, with a cooperative group with two single learners given symmetric (identical) learning contents.We present the results of an experiment designed to measure the effectiveness of asymmetric learning content in fostering cooperative critical thinking, as examined by content and conversation analysis whilst preparing the consulting presentation. We found that the implicit cooperation condition e cooperative reciprocity, triggered by the asymmetric learning contents – was important for maximising critical thinking skills.
Our first research goal was to see how critical thinking skills developed in cooperative mobile learning groups. Additionally, the second research goal revolved around how deficient knowledge motivates the cooperative learning processes. To examine this we constructed two cooperative learning groups, each with different game playing modes, along with a third (control) group of solo learners. When evaluating their critical thinking skills, as revealed by conversation analysis, therewere significant differences among the three groups.We found that the solo learners usually identified only simple organisational structures such as single links with other departments, using ideas or proposing solutions based on the given learning materials. Further, the cooperative groups where the experimental condition encouraged cooperative reciprocity performed better than the cooperative groups operating without this condition. This was particularly evident in our analysis of their ability to link ideas. Of course, some existing learning activities (e.g., group projects as adopted in many universities) are able to firmly build upon this cognitive skill. However, it has not been extensively studied in contextual, mixed-reality mobile learning, where the learning design is intended to increase engagement and motivation in the process of cooperative learning. In conclusion, it can be seen that cooperative knowledge construction requires that learners recognise the value of the knowledge to be shared, both to share their own knowledge with others, and to gain knowledge in return. The key to success in interactive cooperative learning is thus that self-interest should quickly align with collective interest. Therefore learning design, such as the mobile and blended approach described in this article, must consider how this alignment can be created (for instance, our learning design – a mobile learning game blended with subsequent classroom activities – sufficiently motivated the learners to align with a collective interest). Knowledge asymmetry that allows more communication and cooperation might be a way forward in designing such cooperative mobile learning games.

Link to read more, http://dx.doi.org/10.1016/j.compedu.2016.03.006

Kim, H., Jang, S., & Kwon, G (2016) An Analysis on the Verbal Spatial Knowledge Expressions of Caller on the 119 Emergency Call Recordings, HCI Korea 2016

Abstract

We analyzed the current 119 emergency call recordings from Gyeonggi Northern Fire Headquarters in the view point of length of time and type of spatial knowledge.
From the analysis, we distill three partially meaningful implications. First, A Caller tends to express Spatial knowledge at the beginning. Second, Landmark establishing common ground, is economical to identify location. Lastly, Map is inefficient in verifying location.

Keywords

Emergency call, Emergency dispatcher, Conversation analysis, Spatial knowledge.

Lee, K., Kim, J., Kim, J., Lee, I., Kim, J. & Kim, J (2016) 3D Finger map for all:Design and research challenges. HCI Korea 2016

Abstract

The present study introduces how to co-design a 3D finger map coupling with two purposes; one is to enhance independent mobility for the blind and the visually impaired; the other is to improve public awareness of the necessity of that support. In order that, we have a particular interest of how the blind perceive spatial information when walking. Observations and co-design workshop have been made. Relevant design issues and usability issues will be discussed.

Keywords

Finger Map, tactile information, visual information, 3d Printing, visual impairment, assistance device.

Lee, A., & Ryu, H. (2016) My mental scrapbook: What to store, what to remember, and what to retrieve in the lifelog data. HCI Korea 2016

Abstract

Like a ‘pentimento’ showing an artist’s mindchanging in the progress of painting, we reinterpret our experiences based on our current status and thoughts. Recent HCI studies suggested the lifelog system should provide something beyond prompt veridical recall, by which it can generate persistent user experience. In this study, we carried out a longitudinal user study to figure out the important attributes of one’s own meaningful experience in order to develop conceptual design guidelines for lifelog system.

Keywords

Reconstructive memory, photo elicitation, lifelog, qualitative study, reference to (past) future events

Jeong, E., & Ryu, H. (2016). Nonverbal auditory working memory: Can music indicate the capacity? Brain & Cognition, 105, 9-21

Abstract:
Different working memory (WM) mechanisms that underlie words, tones, and timbres have been proposed in previous studies. In this regard, the present study developed a WM test with nonverbal sounds and compared it to the conventional verbal WM test. A total of twenty-five, non-music major, right-handed college students were presented with four different types of sounds (words, syllables, pitches, timbres) that varied from two to eight digits in length. Both accuracy and oxygenated hemoglobin (oxyHb) were measured. The results showed significant effects of number of targets on accuracy and sound type on oxyHb. A further analysis showed prefrontal asymmetry with pitch being processed by the right hemisphere (RH) and timbre by the left hemisphere (LH). These findings suggest a potential for employing musical sounds (i.e., pitch and timbre) as a complementary stimuli for conventional nonverbal WM tests, which can additionally examine its asymmetrical roles in the prefrontal regions.
Our findings indicate the potential use of nonverbal sounds (i.e., music) as an assessment medium for working memory. A relatively weaker hemodynamic activation shown in both pitch and timbre processing implies a certain benefit for some individuals who have dysfunctions in cognition such as mild cognitive impairments (MCI) or Alzheimer’s disease.

Link to read more, http://dx.doi.org/10.1016/j.bandc.2016.03.003

Ryu, H., Kim, N., Lee, J., & D. Shin (2016), An Affordance-based Model of Human Action Selection in a Human-Machine Interaction System with Cognitive Interpretations, International Journal of Human-Computer Interaction, 32(5), 402-414

Abstract

Current technology is not sufficient to automate all desired tasks. Human-machine interaction (HMI) has thus become a key control and design factor for tasks requiring human-level decision-making or information synthesis. Such processes require a formal representation of human actions (including decision-making) when modeling HMI systems; however, successful prescriptive approaches to this end have still been elusive. This paper extends the affordance-based finite state automata (FSA) model, conditioning human prior experience and natural loss of task knowledge. The new model draws upon both reinforcement learning and natural memory decay for decision-making on action choice. An empirical study is carried out to specify how action choice is affected or updated by reinforcement learning based on past experience, and Wickelren’s decay function is jointly employed to predict human decision-making behavior

The model considers two critical factors in the semantic condition that plays an important role in interpreting the human operator’s action choice behaviors. The basic activation rate is associated with the initial activation of task knowledge and reflects the human operator’s past experience, and the memory decay corresponds to the natural failure to recall adequate task knowledge as time elapses. By incorporating the two components, the extended model contributes to investigate how a human operator is attracted to a particular action, which is not considered in the previous affordance-based FSA model.

Link to read more, http://dx.doi.org/10.1080/10447318.2016.1157678

Lee. A., Song, K., Kim, J., Gwon, G., & Ryu, H. (2015). Fingerstroke time estimates for touchscreen-based mobile gaming interaction. Human Movement Science, 44, 211-224.

Abstract
The growing popularity of gaming applications and ever-faster mobile carrier networks have called attention to an intriguing issue that is closely related to command input performance. A challenging mirroring game service, which simultaneously provides game service to both PC and mobile phone users, allows them to play games against each other with very different control interfaces. Thus, for efficient mobile game design, it is essential to apply a new predictive model for measuring how potential touch input compares to the PC interfaces. The present study empirically tests the keystroke-level model (KLM) for predicting the time performance of basic interaction controls on the touch-sensitive smartphone interface (i.e., tapping, pointing, dragging, and flicking). A modified KLM, tentatively called the fingerstroke-level model (FLM), is proposed using time estimates on regression models.

Keywords
Mobile game; Fingerstroke-level model (FLM); Regression model; Finger movement

Lee, A., Kim, J., & Ryu, H. (2015). Mobile Learning: The Next Generation- Exploring the Potentials of Mobile Learning for Stroke Patients: RehabMaster Mobile. Routledge
Song, H., Jang, S., & Kim, J. (2015) Development of gamification toolkit for new buisness of firm: focusing on manufacturing company’s servitization theory. KOSIME 2015
Seo, K., Han, G., Lee, H., Ryu, H., & Kim, J. (2015). Design Requirements for PT-tv (Play Therapy with TV): An Observational Study on Play Therapy and TV Viewing.TVX 2015 (WIP)

Abstract
Television (TV) is hard to be separated from our daily lives. Many infants and toddlers are in perpetual contact with TV and/or video content. Recent studies have focused on what characteristics of TV content would affect children’s language development. We are interested in the developmental play therapy performed by pediatricians and how this can be translated into the TV content design to enhance their language development. An observational study was conducted for three weeks at the Infants Care Center. The behavioral patterns during the play therapy and TV viewing were compared and the design requirements for the play therapy with TV (PT-tv) were proposed.

Keywords
PT-tv; design requirement; developmental play therapy;TV viewing; infant and toddler; language development

Lee, A. (2015). Challenges for Wearable Camera: Understanding of the meaning behind photo-taking. CHI 2015 (SRC-Second Place)

Abstract
The wearable camera industry is facing low adoption rates due to concerns over the amount of data the devices collect and the inability to differentiate from mobile phones and digital cameras. To improve adoption rates, the perception of the wearable camera should be changed. This research attempts to portray mobile cameras as tools for personal experience sharing. A 50-day study was conducted to determine what types of experiences are meaningful for the users. These factors should be considered when designing wearable cameras for personal re-experiencing system.

Keywords
photos, experience-sharing, wearable camera

Kim, J., & Ryu, H. (2014). Emotional information transfer mechanism design using tactile information: based on research about relationship between vibration and Arousal-Valence. HCI Korea 2015

Abstract
Tactile information has been used to deliver emotional information in a variety of field such as mobile phone, theater, etc. However, there are only minor efforts of how to design the vibration properties, which is the most important fact of the tactile information system, in order to generate the tactile information effectively. This research proposes reference point through an experimental study which is about the effects of vibration properties on the valence-arousal of the emotional information.

Keyword
Vibration, Emotion, Valence, Arousal, Self-Assessment Manikin, SAM

Lee, A., Kim,J., &Ryu, H. (2014).A study for designing a wearable camera as an experience-sharing and exchanging channel. HCI Korea 2015

Abstract
As the target user of wearable camera has enlarged, the way to deal with the voluminous collected data and deliver it to the users is a prerequisite to overcome in the industry. The aim of this paper is to figure out the design attributes of one’s meaningful experience based on in-depth interview and suggest the design guideline for the wearable camera as an experience-sharing and experienceexchanging channel.

Keyword
Wearable Camera, Image based lifelog, Personal meaningful contents of experience.

Lee, J., & Ryu, H. (2014). Patent-based ideation method : A patent analysis for technology and design convergence. HCI Korea 2015.

Abstract
An awareness of the importance of design and its role has increased in recent years and approaches in terms of design has increased. However, there are not much methods or framework to make convergence between the designs and existing or new technologies. In this study, a methodology of design and technology convergence is proposed using patents including design patents. And this methodology is applied to “Bladeless fan” of Dyson for tracing of Dyson’s
patent process. This study proposes this process to be utilized in one of new ideation method processes in product/service development process.

Keyword
Design technology Convergence, PatentAnalysis, Design rights

Lee, J., Seo, T., & Ryu, H. (2014). A study of dynamic information visualization for implicit learning: focusing on measuring cognitive load using fNIRS. HCI Korea 2015.

Abstract
Designing dynamic info-vis (information visualization) has been much dealt with by HCI researchers and practitioners, but clear guideline has yet been provided for designing/allocating specific dynamic properties. This study, as part of an effort for it, tried to experimentally figure out how different dynamic properties in info-vis influence one’s information acquirement level considering one of human’s innate cognitive capability, ‘implicit learning’. Experiments proceed in two steps. First step is to confirm that two different dynamic properties in bar graph are considered ‘implicit’. And second step is to verify whether the implicit dynamic properties is learnt unwittingly so that it affects one’s information acquisition. Functional Near-InfraRed Spectoscopy (fNIRS) is employed to measure participants’ cognitive load while they process the same bar graph information with different dynamic attributes, thereby we suggest a guideline for dynamic info-vis embracing human’s innate characteristic.

Keywords
Human Factors, Dynamic information visualization, Implicit Learning, fNIRS, User Interface

Seo, K., Kim, J., Ryu, H., & Jang, S. (2014). RehabMaster: A Pervasive rehabilitation platform for stroke patients and their caregivers. Pervasive health—state-of-the-art and beyond. London: Springer.

Abstract
As aging society is coming, concern about long-term disability (e.g., stroke, dementia,
and so forth) is dramatically increasing. Especially, stroke is the leading cause of
long-term disability that hinders post-stroke patients going back to their normal life
. In their normal treatment process, i.e., post-stroke rehabilitation process,
the hospitalization period (N.B. the state-covering insurance in Korea accepts only
8 weeks hospitalization for stroke patients) is over, most of inpatients get back to
their homes and are prescribed with a home exercise therapy program. However,
the outpatients are getting usually worse than their hospitalization period when they
get back to the medical center for check-ups in every 4 weeks. One of the major
reasons for the deterioration is their weakened determination compared to the compulsory
exercise program in their hospitalization period. In this context, a number of
researches have done about serious games, to foster post-stroke patients’ motivation
in performing the home exercise therapy.

Lee, J., Choi, G., Kim, J., & Ryu, H. (2014) Development of features selection technique using users’ false belief. Journal of the HCI Society of Korea, 09(02), 33-40.
Seo, K. (2014). Autonomy-based rehabilitation design: balancing capability and complexity. ACM CHI 2014 (SRC-First Place)

Abstract
Autonomy is critical to harness post-stroke patient’s motivation in an unsupervised home-based rehabilitation program. The aim of this study is to share our experiences of designing the home-based rehabilitation platform for post-stroke patients, RehabMaster, particularly for, what kinds of autonomy supports should be of value for the post-stroke patients. Usefulness of the autonomy-based rehabilitation design was studied by a two-week home-based rehabilitation session with sixteen post-stroke patients and further design issues for autonomy were discussed.

S. I. Petersen & H. Ryu. (2015). Gamification in concept design: Applying market mechanisms to enhance innovation and predict concept performance, Journal of Design, Business & Society, 1(1), 95-110

Abstract
Predicting the success of a potential breakthrough innovative business opportunity in New Product Development (NPD) is notoriously difficult and is synonymous with high market and technology risk. As a consequence, NPD organizations favour concurrent exploration of multiple concepts where the most promising concepts are evaluated and selected at each stage-gate. The two challenges associated with this approach are ensuring a sufficiently wide exploration of the market–technology space, followed by assessing the potential of breakthrough innovations. Individual designers usually conduct early exploration independently or within design teams, applying an organic and analogous search approach. In contrast, the project team members collectively make decisions for early design concept selection, applying a non-analytical and unstructured decision-making process. The combination of grounded exploratory study and hypothesis-driven study examines the use of gamification and, in particular, how gamification can assist in promoting the exploration of breakthrough innovative concepts in the early design concept phase, combined with concurrent and early identification of the most promising candidates.

Studies conducted consisted of a pilot study at Copenhagen Business School where we identified migration of risk in the business phase and two other studies conducted in the Design Technology and Innovation Management project courses at Hanyang University in Seoul, South Korea. The last two studies were a baseline study and an experimentation study, where we observed and collected teams’ market and technology risk-preferences, teams’ and external panel’s grading as well as Design Quality Criteria scoring. In the second study, we established the baseline for risk-­preference migration and in the third study we implemented eight weeks of gamification, applying the Prediction Market approach.

The findings showed that gamification promotes extended risk-taking, can assess team-confidence and decision-making ability and acts as an early indicator of the performance of the final concepts. Suggestions for future studies include testing of gamification on industry projects and making it operational by adapting the approach to the NPD culture, organizational structures and project decision-making processes.

Link to read more, http://dx.doi.org/10.1386/dbs.1.1.95_1

Lee, J., & Ryu, H. (2014). I remember/know/guess what I saw: a false belief technique to features selection. CHI 2014 (WIP)

Abstract
In this position paper we address issues with the primary decision problem in the Smart TV UI design – feature selection. While the existing feature selection methods that traditionally make up HCI research were not able to render what features are to be prioritised in the new TV design, we will introduce the ‘False belief technique’ for this advancement. This new experimental technique will greatly enable UI/UX researchers to conduct feature selection evaluations that could effectively examine a users’ schema of the smart TVs, in a rather unconscious way at the expense of extra training time, which are unimaginable before

Keywords
Smart TV; feature selection; schema; false memory; DRM paradigm; new product development

Click here to download the PDF

Kim, J., Kim, J., & Ryu, H. (2014) ‘Heart-to-Feel’: A New Audio Description Coding Scheme for the Visually Impaired on Affective Cinematography and Emotive Vibration. TVX 2014

Abstract
The visually impaired increasingly demand more informative audio description (AD), but not an easy coding scheme for the describers and a lack of technological advances have limited this success. Our proof-of-design concept – affective cinematography and emotive vibration, is quick to complete the AD process and deliver the emotional information of the scene through the affective haptic device. A promising way to contribute to social TV viewing experience for the visually impaired is also discussed.

Keywords
Affective haptic, affective cinematography, audio description, visually impaired, social entertainment

Click here to download the PDF

Lee, A., Kim, J., & Ryu, H. (2014) Challenges of Designing a Life-log Sharing System – The Pensieve. AHFE 2014

Abstract
Although the privacy issue is still debated, an incessant interest and attention on the life-log could be explained by its potential applications. Though there are many challenges to realize its potentials, life-log as one’s memory aiding tool has been paid much attention. A series of images captured by the wearable camera are too voluminous and useless to keep the whole data to the collection owner. What the users really want to get from the device is not to review their own whole past life, but retrieve their meaningful moments, further share their meaningful experiences with others. In this regard, the author limited the role of life-log, as resourceful life hints that could give opportunities to reminisce one’s meaningful experience and exchange their experience with others. In this position paper we discuss what is necessary to consider to build a life-log data sharing system for our future research direction, (i) what one’s meaningful experience is; (ii) How the meaningful experience could be defined with life-log data; (iii) How the meaningful data conveys to the 3rd person to exchange and share one’s experience effectively.

Keywords
Wearable camera, visual life-log, experience-transferor, experience-transferee, meaningful experience model, experience-sharing system, medium, storytelling

Click here to download the PDF

Lee, H., Ryu, H., & Parsons, D. (2014) Using Game-based Collaborative Learning to Enhance Critical Thinking Skills. AHFE 2014

Abstract
Critical thinking is an essential quality in complex problem solving. Many researchers have proposed that collaborative peer learning could enhance critical thinking, because learners could understand more diverse views by sharing different perspectives on a given situation. The work presented in this paper aims to explore a possible relationship between game-based collaborative learning and critical thinking. A serious mixed reality business game was developed and employed in the present study. Twenty-five undergraduate students were recruited and divided into three groups: an individual learning group, and two collaborative learning groups with distinct learning conditions. After completing the experimental learning tasks, the participants were asked to create presentations, either alone (for the single players) or with their partners, to outline their approaches to solving the problems presented in the game. The results indicate that the three groups showed only slight differences in learning performance (e.g., memorization); however, the difference in their levels of critical thinking was more significant. There appeared to be a strong relationship between learning in pairs and an array of positive intellectual outcomes. Further, the different learning conditions between the two collaborative learning groups also led to significant differences in the level of critical thinking.

Keywords
Critical thinking, collaborative learning, problem solving, serious games

Click here to download the PDF

Noh, S., Seo, K., & Ryu, H. (2014) Serious Game Based Working Memory Measuring for User Accessibility and Immersion. HCI Korea 2014

Abstract
Working memory holds and processes information the central executive (CE) in which is critical to control information. However, the way to measure CE has not been clearly demonstrated. A serious game to do so is suggested in this article.

Keywords
Working memory, central executive, serious game

Click here to download the PDF

Song, K., Lee, A., Kim, J., & Ryu, H. (2014) A Study on Cognitive Intervention for Older Adults Using ‘Draw Something’ Game. HCI Korea 2014

Abstract
Studies on the elderly’s cognitive structure have been advanced to cognitive interventions for people with MCI (mild cognitive impairment). MCI is a cognitive status staying between the normal aging and Alzheimer’s disease (AD). MCI is considered as an important stage for not going ahead to AD in the medical clinic field, but a lot of researches only suggested a simple and repetitive cognitive training approach. In this article, we aimed at developing a cognitive intervening serious game for MCI, based on Baddeley’s working memory model (i.e., phonological store – articulatory loop). An empirical study was carried out to suggest design principles of a cognitive intervening serious game.

Keywords
Draw Something, MCI, serious game, working memory, design principles

Click here to download the PDF

Lee, J., & Ryu, H. (2014) Development of Features Selection Technique Using Users’ False Belief. HCI Korea 2014

Abstract
Selecting appropriate features that products/services should provide for users is a typical decision making problem for designers, but the existing feature selection methods has crucial limitations when figuring out users’ needs. For example, selecting features considering just users’ preference without analyzing users’ mental model might lead to ‘feature creep’. In this study, we suggest ‘False belief technique’ which detects users’ mental model for the products/services that is formed after being provided with new features. This technique will be utilized as an experimental manner for feature selection decision by designers

Keywords
Feature selection, Schema, False memory, DRM paradigm, False belief technique

Click here to download the PDF

Kim, J., & Ryu, H. (2014) An Empirical Study of the Relation between Information Visualization and Decision Making in Medical Equipment: Focusing on DBS Surgery Case. HCI Korea 2014

Abstract
Many hospitals have adopted the latest medical equipment to improve the quality of medical service, because using of medical equipment is very helpful for doctor’s work such as medical and surgical. However definitive information of medical equipment can be connected to a major accident. So medical equipment avoided to provide definitive information, and should be provide information to assist in doctor’s decision making. The present study find changing of decision making to visualize information that can be helpful for decision making as color and bar graph. For this study, program that designed by MATLAB proceed experiments that is assumed DBS surgery environment.

Keywords
Medical equipment, Definitive information, Visualization, DBS, STN, MER

Click here to download the PDF

Lee, H., & Ryu, H. (2014) The Effects of Collaborative Learning in Critical Problem Solving. HCI Korea 2014

Abstract
An importance of critical thinking has been emphasized in various academic areas, in particular, complex problem-solving situations. To identify the effects of collaborative learning, participants were divided into two groups (learning alone and learning with a partner) and learned about a specific corporate situation by using serious mobile game. After playing game, the participants made a presentation file to account for their problem solving approaches and it was measured against the level of critical thinking. The results indicate that learning with a partner makes the participants more critically thinking the problem situations rather than the learning alone group, and we suggest that the way how they are externalizing their knowledge is a key to critical thinking.

Keywords
Critical thinking, collaborative learning, problem-solving

Click here to download the PDF

Lee, A., & Ryu, H. (2014) The Necessity of Considering Two Different Parties, Experiencer and Experience-receiver in Designing Visual Life-logging System Design. HCI Korea 2014.

Abstract
The visual life-logging data from the wearable camera has been only applied in the personal arena (i.e., memory aid, dietary analysis). Recently, there are many systems that are able to give users to share their memory with others.However it only works as a simple photo browser which conveys the raw captured data to others. In this regard, the present study tried to figure out the current problem of such systems by analyzing how an experience-owner would like to share his or her own emotional experience (episodic memory) with experiencereceiver (semantic memory).

Keywords
Wearable camera, visual life-logging data, sharing experience, experience-owner, experience-receiver

Click here to download the PDF

Seo, K., Kim, J., & Ryu, H. (2014) An Effect of Designer’s Empathy Level to the HCI Design during Co-design Practices. HCI Korea 2014.

Abstract
The effect of designer’s empathy level during the co-design process has been paid much attention. In this paper, two groups of designers conducted a ‘Nike vacuum cleaner’ co-design practice with potential users: empathetic and intuitive designers. The co-design process was analyzed with a thematic analysis and a semistructured interview. The result indicates that the intuitive designer group tends to have ‘design fixation’ problems, and empathetic designers might achieve robust and coherent design quality but not efficiently.

Keywords
Empathy, co-design, co-creation, participatory design, designer

Click here to download the PDF

J-H. Shin, H. Ryu & S. Jang. (2014). A task-specific interactive game-based virtual reality rehabilitation system for patients with stroke: a usability test and two clinical experiments, Journal of NeuroEngineering and Rehabilitation,

Abstract
Background: Virtual reality (VR) is not commonly used in clinical rehabilitation, and commercial VR gaming systems may have mixed effects in patients with stroke. Therefore, we developed RehabMaster™, a task-specific interactive game-based VR system for post-stroke rehabilitation of the upper extremities, and assessed its usability and clinical efficacy. Methods: A participatory design and usability tests were carried out for development of RehabMaster with representative user groups. Two clinical trials were then performed. The first was an observational study in which seven patients with chronic stroke received 30 minutes of RehabMaster intervention per day for two weeks. The second was a randomised controlled trial of 16 patients with acute or subacute stroke who received 10 sessions of conventional occupational therapy only (OT-only group) or conventional occupational therapy plus 20 minutes of RehabMaster intervention (RehabMaster + OT group). The Fugl-Meyer Assessment score (FMA), modified Barthel Index (MBI), adverse effects, and drop-out rate were recorded. Results: The requirements of a VR system for stroke rehabilitation were established and incorporated into RehabMaster. The reported advantages from the usability tests were improved attention, the immersive flow experience, and individualised intervention. The first clinical trial showed that the RehabMaster intervention improved the FMA (P = .03) and MBI (P = .04) across evaluation times. The second trial revealed that the addition of RehabMaster intervention tended to enhance the improvement in the FMA (P = .07) but did not affect the improvement in the MBI. One patient with chronic stroke left the trial, and no adverse effects were reported. Conclusions: The RehabMaster is a feasible and safe VR system for enhancing upper extremity function in patients with stroke.

Link to read more http://www.ncbi.nlm.nih.gov/pubmed/24597650

S. Kim, J. Kim, S. Bae, H. Kang, D. Kim, H. Ryu (2014), High visibility or quiet operation?: A statistical inference model of the wiper operation quality by the dual-code theory, 44(4), 482-492

Abstract
The motor industry has assured drivers that it will continually provide quality vehicles without compromising safety and comfort. In this sense, the quality of a product always needs to be determined by what the customer actually perceives, which has not been the case in the domain of wiper operation quality. To maintain a quality prediction model for wiper operation, this article develops a multi-modal inference model considering both the visible quality (VQ) and acoustic and vibration quality (AVQ). Based on the cognitive multimodal information processing theory, the model combines the two kinds of perceived information from various wiper operating conditions, i.e., the level of visibility obtained by wiper operations, and noises or shudders during operation. The regression model with regard to the customer’s perceived quality was validated empirically, and an interesting gender effect was identified in the perception of wiper quality. Also, in place of the conventional quality evaluation method (e.g., checklists or jury charts), we suggest our model of the multimodal information processing theory not only to predict the perceived wiper operation quality (visibility and quietness), but also to apply it to other related fields in product quality measurement.
This study developed a first multisensory integration model by our team, now we are extending this approach to tactile-visual integration for reducing VR sickness.

Link to read more http://dx.doi.org/10.1016/j.ergon.2014.04.001

J. Kim & H. Ryu. (2014). A Design Thinking Rationality Framework: Framing and Solving Design Problems in Early Concept Generation, Human-Computer Interaction, 29(5-6), 516-553

Abstract
The concept of ‘Design Thinking’ opens up debate regarding the prevalent HCI design practice. This paper focuses specifically on the cognitive processes of designers during their early design activities. Two groups of designers – experts and novices – were asked to develop a fictitious vacuum cleaner. We then examined the different ways in which these groups manage their design thinking processes, and how the groups choose design concepts. The empirical study revealed that expert designers are effective at framing design problems. They make quick decisions (through the use of the affect heuristic) but are more wedded to their own previously developed design concepts, which they do not change in subsequent design stages. In contrast, novice designers are less skilled in framing new design problems, but better able to renounce their initial design concepts. These diverse design thinking approaches are linked to potential problems. We then discuss how to address these concerns in conjunction with empathy for the artefact (i.e. artefact empathy via the mediated self) or user (i.e. user empathy via the simulated self), problem framing with second-order semantic connotations, and irrationality when analysing design solutions. Finally, we propose a design thinking rationality framework that can establish a designer’s view of design activities and thereby assist designers educated in both creative and rational design decisions.

This article concludes that the designer needs to oscillate between high involvement with the artefact for innovation design and intentional detachment from the artefact for rationality design. By this we mean that the design thinking rationality framework would enable the designer to step back from his or her projected experience to take note of wider relationships upon which the qualities of the whole design concept will depend. Furthermore, it claims that the HCI community is in need of amalgamating a broader theme of engineering design processes and design-oriented cognitive models. A better understanding of designers’ cognition would result in suggestions for a new direction for the HCI perspective built upon the engineering design process.

Link to read more, http://dx.doi.org/10.1080/07370024.2014.896706

Kim, J., Lee, A., & Ryu, H. (2013) Personaliy and its effects on learning performance : Design guidelines for an adaptive e-learning system based on a user-model. International Journal of Industrial Ergonomics.
Katuck, N., Kim, J., & Ryu, H. (2013) Experience beyond knowledge: Pragmatic e-learning systems design with learning experience. Computers in Human Behavior.
Lee, D.Y., & Ryu, H. (2013) Learner Acceptance of a Multimedia-Based Learning System. International Journal of Human-Computer Interaction.
Shin, D., Ryu, H., Kim, N., & Kim, J. (2013) Modelling of a human decision-making with Prospect theory. HCI International 2013.
Busodi, M., Kim, N., Shin,D., & Ryu, H. (2013) Bayesian-Affordance agent model for wayfinding behaviors in evacuation simulation. HCI International 2013.
Petersen, S., & Ryu, H. (2013) Design Criteria and Their Coupling in Concept Synthesis. DesignEd 2013.
Song, K., Lee, A., & Ryu, H. (2013) The Fingerstroke-Level Model strikes back: A modified Keystroke-Level Model in developing a gaming UI for 4G networks. ACM Computer-Human Interaction 2013.

Abstract
With the 4G mobile technology, LG U+ established a new business model, inter-network mirroring game service that allows PC and mobile game users to play against each other. However, due to an unsolicited input command design for touch-sensitive UIs, it is hard to adjust competitive levels between them. The traditional Keystroke-Level Model (KLM) was not applicable to predict the task performance in the touchsensitive user interface. This case study thus proposed Fingerstroke Level Model (FLM), and analyzed the internetwork mirroring game – ‘Freestyle II™’ with FLM. The empirical study confirmed the effectiveness and efficiency of FLM, and suggested how HCI methods can improve the design of mobile gaming user interface.

Keywords
Tactile & Haptic UIs; Input and Interaction; Technologies; Entertainment; Empirical Methods; Quantitative Prototyping

Click here to download the PDF

Lee, A., & Ryu, H. (2013) Handbook of Mobile Learning – Social Versus Individual Flow in Mobile Learning. Routledge.
Lee, A., & Ryu, H. (2013) A rehabiliation program design for the stroke patients and caregivers with Activity Theory. HCI Korea 2013.

Abstract
The development of Information and Communication Technologies (ICTs) incesantly extends the scope of medical services which have ben caried out in the limited space and time domain. As to stroke patients, for instance, both caregivers and patients could not obtain suficient suports from their medical staf, thereby joining other online communites from which they can get other practical advices. In this regard, the present study tried to analyze the curent rehabiltation activity systems for stroke patients with Activity Theory and found some design implications, which were adresed by a new artifact caled RehabMaster ‘Mobile’. This alowed us to sugest a design method that is able to develop a socio-technical system through design tensions and contradictions.

Keywords
Stroke patients, socio-technical systems, activity theory, caregivers, physiatrist, therapists. Positve fedback lop.

Click here to download the PDF

Seo, K., Kim, J., Lee, J., Jang, S., & Ryu, H. (2013) Serious Games for Stroke Patients: Attending to Clinical Staff’s Voices. IASDR 2013.

Abstract
A motion-based rehabilitation game is gaining a solid ground in that it can provide the stroke patients with the repetitive training effect for a functional improvement of upper extremity (UE). Many proved potential utilities of the motion-based rehabilitation games for the first stakeholders, i.e., patients, but the organizational benefits for other key stakeholders (e.g., clinical staff such as physiatrists and occupational therapists) have been less attended. This paper reports on our design experience of a natural interaction based rehabilitation program – „RehabMaster‟, addressing the rather different requirements from the clinical staffs. A practical rehabilitation session assessed a two-week clinical trial with sixteen stroke patients, seven physiatrists and three occupational therapists, and essential requirements of the clinical staffs were observed. We learnt that a key to the clinical-level serious game design is to mimic actual social settings of what the stakeholders are indeed interacting with.

Keywords
clinical staff, stroke, motion-based rehabilitation game, design process

Click here to download the PDF

Seo, K., & Ryu, H. (2013) Case study of developing medical serious game ‘RehabMaster’ by participatory design. HCI Korea 2013.

Abstract
Various researches have done about serious game in order to rehabilitate stroke patients. These serious games mainly focused on a stroke patient’s perspective, so they merely consider about other rehabilitation stakeholders like physiatrists and occupational therapists. In this paper, by using participatory design method, we propose design factors which not only fulfilling stroke patients but also physiatrists and therapists. Based on these design factors, we made a rehabilitation serious game RehabMaster. An empirical study was carried out with this serious game to validate both clinical advantage and design factors, and how these design factors would help HCI practitioners was also discussed.

Keywords
Serious game, Stroke, Rehabilitation, Participatory design, Meaningful play, Challenge

Click here to download the PDF

D. Lee & H. Ryu. (2013). Learner Acceptance of a Multimedia-Based Learning System, International Journal of Human-Computer Interaction, 29(6), 419-437

Abstract
The present study applied the technology acceptance model to examine the determinants leading to learners’ behavioral intention to use a multimedia-based learning system. Four exogenous constructs—multimedia self-efficacy, perceived richness of multimedia presentation, perceived learner control, and perceived system responsiveness—were externally added to the framework to improve its predictive power for the specific behavioral context. In addition, the second-order construct of cognitive engagement was created based upon the dimensions of curiosity, attention focus, and interest and was subsequently incorporated into the framework. The hypothesized conceptual framework was validated using sample data collected from 286 respondents who completed an online survey instrument. Results from structural equation analysis revealed that (a) behavioral intention was jointly determined by attitude and perceived usefulness; (b) attitude was jointly determined by perceived usefulness and cognitive engagement; (c) multimedia self-efficacy, perceived richness of multimedia presentation, and cognitive engagement were immediate antecedents to perceived usefulness; and (d) cognitive engagement was a key intervening variable linking the four exogenous constructs with perceived usefulness.

Our theoretical framework was based on the modified version of the TAM, with three proximal constructs of behavioral intention, attitude, and perceived usefulness. Four additional constructs—multimedia self-efficacy, perceived richness of multimedia presentation, perceived learner control, and perceived system responsiveness—were added external to the framework to improve its predictive value for the context.

Link to read more, http://dx.doi.org/10.1080/10447318.2012.715278

Al-Dujaily., Kim, J.., & Ryu, H. (2012) Am I Extravert or Introvert? Considering the Personality Effect Toward e-Learning System. Educational Technology & Society.
Kim, J., Bouchard, C., & Ryu, H. (2012) Emotion find a way to users from designers: Assessing product images to convey designer’s emotion. International Journal of Design Research.
Ryu, H., & Parsons, D. (2012) Risky business or sharing the load? Social flow in collaborative learning. Computers & Education.

Abstract
Mobile learning has been built upon the premise that we can transform traditional classroom or computer-based learning activities into a more ubiquitous and connected form of learning. Tentative outcomes from this assertion have been witnessed in many collaborative learning activities, but few analytic observations on what triggers this collaboration have so far been made. However Social Flow, a concept framework that extends Csikszentmihalyi’s flow theory, may help us to partially explain the triggering mechanism of collaborative mobile learning. Our case study in this article, where learners together explore a built environment as part of a simulated security guard training programme, describes how the concept of social flow in a collaborative learning space might sketch out what triggers an optimal learning experience in collaboration and what can be additionally achieved in a collaborative learning experience. In this learning context, collaborative mobile learning might be seen to prompt more knowledge generation and extra learning tasks by fostering greater motivation than other learning environments.

Link to read more, click here

Petrova, K., Parsons, D., & Ryu, H. (2012) Mobile Gaming: A Serious Business!. IEEE WMUTE 2012.
Lee, J., Seo, K., Yang, H., Kim, J., & Ryu, H. (2012) A cognitive model for predicting human errors: What to choose in two competing task knowledge. Conference on Korean Industrial Engineering 2012.
Kim, H., Lee, A., Kim, J., & Ryu, H. (2012) A model for predicting responses in an emergency task : An empirical account of memory decay. Conference on Korean Industrial Engineering 2012.

Research aims
Predicting human decision-making behaviors in a formal prescriptive model, upon which a ‘computational agent model’ can be built for different human operators in security-critical systems.

Click here to download the PDF

Choi, Y., Cha, J., & Ryu, H. (2012) SqueeView: An evaluation of a serious game for improving spatial cognition. HCI Korea 2012.
Choi, Y., Song, K., & Ryu, H. (2012) Mobile ‘SqueeView’: Caring for the elderly with Mobile Learning. mLearn 2012.
Lee, A., Song, C., & Ryu, H. (2012) Game design for modulating aggression : A new evaluation for gaming effects. HCI Korea 2012.

Abstract
Prior evaluation frameworks for games are mostly grounded on their easeof-use and engagement levels, and how these two factors are able to present flow experience or pleasant-to-use. However, some critics on this approach claimed this traditional usability approach would not uncover the core part of gaming experience, such as player’s psychological and behavioural attitudes, arising from its socialtechnical environment of playing games. The present study empirically demonstrated a new evaluation framework for gaming experience in conjunction with two extra dimensions, i.e., physical aggression and arousal aggression

Keywords
game, evaluation framework, aggression, arousal

Click here to download the PDF

Seo, K., Lee, J., & Ryu, H. (2012) A four-legged learning experience design framework for serious games. HCI Korea 2012.

Abstract
A recent development of e-learning systems design has been made on gaming effects that enhance natural engagement with learning activities arising from the gaming experience. In so doing, at issue is how the learning objectives should be counterbalanced with the gaming effects that might move away people from the learning outcomes expected. The present study proposed a four-legged learning experience design framework that embraces these two goals (i.e., achieving learning objectives as well as better learning experience via playing games). An empirical study was carried out to validate this framework and how this framework would help HCI practitioners was also discussed.

Keywords
serious game, flow experience, design framework, experimentation

Click here to download the PDF

Parsons, D., Petrova, K., & Ryu, H. (2011) Designing mobile games for engagement and learning. ICITA 2011.

Abstract
Game based mobile learning is becoming increasingly popular, now that mobile devices provide support for multimedia content, location awareness, augmented reality and connectivity. However just having technical features does not make a game either engaging or pedagogical. The challenge for designers of games for mobile learning is to embed both effective gaming experiences and worthwhile learning outcomes into the same application. The game described and discussed in this paper was designed as an augmented reality game for two players. The narrative action follows a classic linear fiction model, whereby the game’s phases move through teaser, elaboration, conflict escalation, climax and resolution. This narrative path is reflected by a physical path as players navigate the location and investigate the problem they have to solve.

Keywords
Mobile learning; Mobile games; Augmented reality; Serious games; Business education; Problem solving; Critical thinking

2011_Designing mobile games for engagement and learning

Katuk, N., Wang,R., & Ryu, H. (2011) Enhancement of learning experience using Skill-Challenging Balancing approach. Australasian Joint Conference on Artificial Intelligence 2011.
Katuk, N., & Ryu, H. (2011) Does a logner usage mean flow experience. IEEE-DELTA 2011.
H. Ryu & A. Monk. (2011). Interaction Unit Analysis: A New Interaction Design Framework, Human-Computer Interaction, 24(4), 367-407

Abstract
A pragmatic approach to interaction modeling is presented by which a designer can describe how the user gets tasks done with a newly developing system. The notation proposed allows an interaction designer to make explicit both how user actions cause visible or noticeable changes in the state of the machine and how the user is expected to use this feedback to generate the next action. Interaction Unit (IU) scenarios are constructed where each IU specifies one step in the cycle of interaction. Each IU specifies the visible system state that leads the user to take some action. In addition, the IU makes explicit the state of the goal stack at the start and end of the unit and the mental processes (recall, recognition, or affordance) required. In this way one can describe the intimate connection between goal, action, and the environment in user–machine interaction. To demonstrate the completeness of IU scenario analysis, IU models are presented for some well-known problems in interaction design: hidden and partially hidden modes leading to unexpected system effects, insufficient cues for subgoal construction, insufficient cues for subgoal elimination, and inappropriate affordances for action. These scenarios are accompanied by procedures that designers
can use to detect similar problems in putative interaction designs. To demonstrate the feasibility of using IU scenario analysis in design, 4 graduate students were taught to use IU scenario analysis in a 3-hr session. They then worked as a group to evaluate a prototype handheld warehouse application. A comparable group was taught and then applied Cognitive Walkthrough. Both groups successfully completed the task and detected several problems rated as being of high severity by the designers of the prototype. Analysis of the problems detected by each group suggests that the two techniques are complimentary. IU scenario analysis may be most cost-effective for devices using new interaction paradigms, whereas Cognitive Walkthrough may be most cost-effective for designs using established interaction paradigms.

This article describes how interaction unit scenarios analysis combines aspects of several existing modeling approaches to provide a pragmatic approach to interaction modeling. By interaction modeling we mean a notation that describes cognitive and environmental features at the lowest level of description. Each IU specifies one step in the cycle of interaction, that is, the visible system state that leads the user to take some action, the state of the goal stack at the start and end of the unit, and the mental processes (recall, recognition or affordance) required. This makes it possible to evaluate fragments of dialogue as they are designed by making explicit the assumptions of the designer about how the users get a task done.

Link to read more, http://dx.doi.org/10.1080/07370020903038086

H. Ryu. (2009). Mobile User Interface Analysis and Design

A free book for your reading!!

Link to read Click Here to download PDF