M.Sc. Thesis

My Role: UX Researcher

About the Thesis

It is essential that the products we use in our daily lives, as well as Virtual Reality (VR) devices, are user friendly, those fields of usage have increased in recent years. Although it has a development history dating back to the 1960s, there are obscurities for both designers and developers. The reasons for the obscurities are the interaction components, and the software languages have been developed and changed in VR devices in parallel with the development of technology. Oculus Rift released the developer version in 2016 and still improving the controllers to make them more ergonomic and natural, including other brands. In addition to physical controllers, questions are not completely answered about how experience, interaction, and user interface design methods will be applied to the field.

Overview

Product: Oculus Rift (version 2017)

University: Mimar Sinan University of Fine Arts

Department: Informatics, Computer-Aided Art and Design

Duration: 1 year courses, 1 year thesis

Publish Date: June 2019

Oculus Rift, developer version (2017) by oculus.com

Section #1

Virtual Reality (VR) and the study overview

Photo by vive.com.

Virtual Reality (VR) is an environment that exists independently from the world we are physically located in. The VR environment and objects surrounding us are created by the software systems, which require various electronic sensors/devices to sense it with our sensorial organs. Still, it is usually be perceived as being in the physical (real) world.

What is Virtual Reality (VR)?

Virtual Reality (VR) applications usually provide rich interactions, such as manipulating objects in the environment, as it often combines a 360° environment and 3D objects rather than a flat-screen. This complicates how the Graphical User Interface (GUI) design process for VR environments should be and how the user experience research methodologies should be applied and evaluated. The design process usually includes a wide range of design fields: visual, interaction, and experience design. You can see some topics below:

Challenge

2D User Interface (GUI)
Body Gestures
3D Object Design
360° Environment
Interaction Design
3D User Interface
Controller
360° Sound Design

• How should the GUI design be for Virtual Reality Environments (VREs)?
• Are there design guidelines or methods for VREs?
• Can User Experience(UX) research methodologies be applied in a VREs?
• Do users prefer 3D interfaces in the VREs because it offers kind of a real-world interaction?
• Does it affect the usability if the GUI is designed 2D(flat) or 3D(with 3D object interactions) in the VREs?

Questions

Regarding the literature review in 2018, designers and researchers have been developing new methods for the design process and adjust the UX research methods into VREs. Noller F. (2016)'s study inspired me to evaluate 2D GUI and 3D interaction by conducting usability lab study in VR. Some of the researches are below:
• Tanriverdi V. and R.J.K. Jacob (2001) suggest a design model and methodology (VRID) for the interface design process in VR environments.
• Murtza, R. and others (2017) suggest Virtual Reality Heuristics
• Noller F. (2016) suggests a set of interaction design principles that can be used in the GUI and interaction design process in VR.

Some answers

H1: The usability score of a VR environment that consists of a 3D interface design is higher than a 2D interface design.
‍
H2: The users prefer to accomplish the given task in the VR environment that consists of a 3D interface design against a 2D interface design.

Hypothesis

Section #2

Applying the Usability Lab Study into VR

A participant is completing the task at Virtual Reality Berlin Meetup.

What is Usability Lab Study?

Usability lab studies, one of the user research methods commonly used in the UX design process, are also known as usability testing in the literature. In usability tests; The UX designer gives participants who are representing the target group some tasks and scenarios. The tasks and scenarios are prepared by considering the feature needs to be improved/developed. The participant is asked to complete these tasks through the existing product or the prototype. The purpose of the UX designer is to collect data by measuring the interactions of the participants with the system and observing the user's attitudes in a controlled way to measure the effectiveness of the interface/product. The UX designer asks participants to think-aloud during the study; in that way, he/she can collect the user insights. UX designers can apply surveys or scales to the users before/after the study if it's necessary. The UX designer doesn't give any tips or guidance to the participants, not to affect the results.

The VR apps

For the study, two free applications were selected from the Oculus Rift market. All of the participants completed one task by using the chosen applications: Avatar Editor (2D) and Thread Studio (3D).

2D GUI

Oculus Rift, avatar creation

3D interaction and GUI

Thread Studio by Shopify, customize printable materials

The screenshots are taken in the Oculus Rift avatar editor (2D) and the Thread Studio (3D) apps.

Limitations

Considering the time constraints, I selected two apps that are freely available from the Oculus Rift market, instead of designing and coding new VR applications. Usability lab studies require time and cost as the tested task is performed one to one with each participant. Considering the analysis time of the study results, I limited the number of participants with 20 people. Participants participated in the study voluntarily.

Environment

The study took part in Virtual Reality Berlin Meetup. It is an event organized by Mediadesign Hochschule (MD.H). Throughout the event program, VR companies or researchers present their expertise with the participants. After the presentations, the participants move to the demo area. In the demo area, participants can experience some demo versions developed by the companies, academicians, or volunteer developers. They give feedback and exchange knowledge. The test environment was established in the demo area, as seen in the picture above.

Participants

I identified the representative universe as people who are interested in VR. So, most of the participants were selected from people who attended to the Virtual Reality Berlin Meetup. In total, I conducted the study with 20 participants. I also questioned whether the participants' experience status and gender would affect the results. Therefore, ten of the participants have experienced at least more than two VR apps, and the other ten either hasn't experienced VR or experienced only once. The gender distribution was also equal: I had ten women and ten men participants.

Analysis tools

I collected the qualitative data by observing the participants while they were performing the task and applied the think-aloud technique. To collect quantitative data, I used the meCUE 2.0 and Affect Grid scales and a short questionnaire to collect the participant's personal information, including product preferences.

Research details

Adaptation task

I gave an adaptation task to the participants to make them feel comfortable with the environment and become familiar with the controllers.

2D GUI

Please explore the environment you are in and change the type of glasses worn by the avatar.

3D Interaction and GUI

Please explore the environment you are in and take the glasses on the shelf and drop them on another shelf.

The screenshots are taken in the Oculus Rift avatar editor (2D) and the Thread Studio (3D) apps.

Main task

I paid attention to choosing the two applications with similar visual quality and the same perspective to get an unbiased evaluation from participants. I kept the task very basic and minimized the differences between the two apps. So, the participants could focus on the task and evaluate only the 2D and 3D GUI.

2D GUI

Please change the ambient color of the avatar to white.

3D Interaction and GUI

Please change the T-shirt color of the avatar to white.

The screenshots are taken in the Oculus Rift avatar editor (2D) and the Thread Studio (3D) apps.

Basic think-aloud

In order to collect qualitative data, participants were observed during the time they performed the tasks and asked to think aloud. So, their cognitive processes were examined.

meCUE 2.0

Download meCUE 2.0 scale here..

As a quantitative data collection tool, I used meCUE 2.0: modular evaluation of key Components of User Experience, which is a scientifically proven scale.

Affect Grid

I chose to use the Affect Grid-scale for a detailed analysis of the positive-negative emotions collected with the meCUE questionnaire. The Affect Grid is a quick measure to assess effect along with the theoretical dimensions Pleasure-Displeasure and Arousal-Sleepiness.

Study overview

As tasks are carried out with these two different applications, the study is planned as two steps, as can be seen in the table above. Participants were asked to complete the adaptation task in Avatar Editor (2D) and then the main task in the first step. After he/she completed the task, the meCUE 2.0 scale and Affect Grid scale were applied. In the second step, the same operations were repeated for Thread Studio (3D) app. So, the participants completed the adaptation task and then the main task in Thread Studio (3D), then fill out the meCUE 2.0 and Affect Grid scales. Therefore, all participants completed the tasks in the same order in both applications. Lastly, questionnaires were applied to question personal preferences and gather personal information.

Section #3

Conclusion and Result

meCUE 2.0 results.

Intuitive design, high usability score

It is seen that the product designed in accordance with the interaction design principles positively affects the usability score. Thanks to the consistent interaction and visual design of navigation and selection in the AvatarEditor (2D), the participants quickly completed the task. The observation insights were verified by the highness of usability and overall assessment scores obtained on the meCUE 2.0 scale (module I).

Affect Grid-scale results.

High usability score, high positive feelings

According to the results obtained with the meCUE 2.0 scale, a directly proportional relationship was found between the usability scores of the assessed applications and the positive emotion scores. The positive emotions score of Avatar Editor (2D), which has a high usability score, is higher than Thread Studio (3D).

According to the results of the Affect Grid-scale of Thread Studio (3D), a total of 7 participants marked the region covering the areas of stress, unpleasant feeling, and depression (The reason explained in the further paragraphs). In the scale results of Avatar Editor (2D), it was seen that only 2 participants marked this region. These data also confirmed that there is a directly proportional relationship between usability scores obtained with the meCUE 2.0 scale (module II) and positive emotion scores (Affect Grid-scale).

The effect of 2D or 3D GUI on usability score

The impact of the graphical user interface design, 2D or 3D, on the usability score has not been fully confirmed. However, it was found that the usability score of the 3D GUI was negatively affected by the poor design. Since the three-dimensional object interaction includes complex varieties of interactions and the Thread Studio (3D) had some design mistakes that was discovered during the study. That's why the hypothesis should be examined again. It should be repeated through testing with another 3D application, which includes no interaction design mistakes. The details are discussed in the further paragraph.

H1: The usability score of a VR environment that consists of a 3D interface design is higher than a 2D interface design.

Hypothesis 1 refuted

Thread Studio, flat 2D GUI.

Thread Studio, 3D object interaction.

The screenshots are taken in the Oculus Rift, Thread Studio (3D) app.

Importance of the interaction and object design in 3D GUIs

The design mistake, which can be seen in the picture above (right), affected the usability score of the Thread Studio (3D). The issue was discovered during usability testing. The problem was the color card on the menu was designed as a flat object (2D GUI), although the user was expected to interact as a 3D object with it. So, the participants touched the 2D color card as a flat touch screen and expected it to activate or open. However, they were not able to get feedback on this touch. They had to think for a while, and after several attempts, they tried to grab the card in their hands, inspired by the task of exercising. In order for the interface design to be consistent, the color card should be designed in 3D like other objects in the environment. However, the design and software of Thread Studio (3D) were not made by the author. Due to time constraints, a selection was made among the existing applications. In the absence of software development and design constraints, these design errors can be corrected (by designing the color card as a 3D object), and a similar study can be repeated. So, hypothesis one can be examined again.

The participant liking and preference results.

‍H2: The users prefer to accomplish the given task in the VR environment that consists of a 3D interface design against a 2D interface design.

Hypothesis 2 confirmed

Suggestions

3D object design, interaction design and XR

It is suggested to consider the basic principles of interaction design along with the GUI design principles for the 3D GUI design process. Don Norman's book, Design of Everyday Things, is also thought to help the three-dimensional object and 3D GUI design process. In addition, it is suggested to use Murtza, R. et al. (2017)'s Virtual Reality Heuristics as a checklist based on Heuristic Evaluation, one of the UX research methods.

The Scales that can be applied in VR

Within the scope of this research, observational data and meCUE scale gave consistent results with each other showed that meCUE scale can be used in VR environments. However, it was determined that the scale was not sufficient alone, and observation and Think-Aloud techniques should definitely be used.

Download the Turkish version here..

← PREVIOUS Project HOMEPAGE