Emily Navarro

Nick Mangano, Alex Baker, Mitch Dempsey, Emily Navarro, and André van der Hoek

Abstract: In this position paper, we present Calico, a sketching tool supporting early software design activities. We first provide background information about early design, including the types of models designers use and the behaviors that they typically exhibit. We then describe Calico's main features, and how they were designed to support these models and behaviors. We conclude with our experiences to date and a look at our future work.

In Proceedings of the The VL/HCC Workshop on Sketch Tools for Diagramming, Herrsching, Germany, September 2009

CalicoWorkshop.pdf (182KB)

Emily Navarro and André van der Hoek

Abstract: In this paper, we describe a multi-site evaluation of SimSE, an educational software engineering simulation game. This study was designed to build on our previous experience of evaluating SimSE in courses and controlled lab settings at UC Irvine, in order to validate our findings and discover any factors that come into play when SimSE is used in other institutions. The study consisted of three different universities using SimSE in their respective courses and reporting the results to us. The results confirmed several of our previous findings, as well as highlighted a number of critical considerations that must be taken into account when using SimSE in a course.

In Proceedings of the The 40th ACM Technical Symposium on Computer Science Education, Chattanooga, TN, March 2009

SIGCSE2009.pdf (387KB)

Emily Oh Navarro and André van der Hoek

Abstract: Learning theories describe how people learn. There is a large body of work concerning learning theories on which to draw, a valuable resource of which the domain of software engineering educational research has thus far not taken full advantage. In this chapter, we explore what role learning theories could play in software engineering education. We propose that learning theories can move the field of software engineering education forward by helping us to categorize, design, evaluate, and communicate about software engineering educational approaches. We demonstrate this by: (1) surveying a set of relevant learning theories, (2) presenting a categorization of common software engineering educational approaches in terms of learning theories, and (3) using one such approach (SimSE) as a case study to explore how learning theories can be used to improve existing approaches, design new approaches, and structure and guide the evaluation of an approach.

In H.J.C. Ellis, S.A. Demurjian, and J.F. Naveda (Eds), Software Engineering: Effective Teaching and Learning Approaches and Practices, IGI Global, 2008.

BookChapter-1.pdf (552KB)

Emily Oh Navarro and André van der Hoek

Abstract: Software engineering educational approaches are often evaluated only anecdotally, or in informal pilot studies. We describe a more comprehensive approach to evaluating a software engineering educational technique (SimSE, a graphical, interactive, customizable, game-based software engineering simulation environment). Our method for evaluating SimSE went above and beyond anecdotal experience and approached evaluation from a number of different angles through a family of studies designed to assess SimSE’s effectiveness and guide its development. In this paper, we demonstrate the insights and lessons that can be gained when using such a multi-angled evaluation approach. Our hope is that, from this paper, educators will: (1) learn ideas about how to more comprehensively evaluate their own approaches, and (2) be provided with evidence about the educational effectiveness of SimSE.

In Proceedings of the Twentieth Conference on Software Engineering Education and Training, Dublin, Ireland, July 2007

CSEET2007.pdf (213KB)

Emily Navarro

Abstract: The typical software engineering education lacks a practical treatment of the processes of software engineering—students are presented with relevant process theory in lectures, but have only limited opportunity to put these concepts into practice in an associated class project. Simulation is a powerful educational tool that is commonly used to teach processes that are infeasible to practice in the real world. The work described in this dissertation is based on the hypothesis that simulation can bring to software engineering education the same kinds of benefits that it has brought to other domains. In particular, we believe that software process education can be improved by allowing students to practice, through a simulator, the activity of managing different kinds of quasi-realistic software engineering processes. To investigate this hypothesis, we used a three-part approach: (1) design and build SimSE, a graphical, interactive, educational, customizable, game-based simulation environment for software processes, (2) develop a set of simulation models to be used in seeding the environment, (3) evaluate the usage of the environment, both in actual software engineering courses, and in a series of formal, out-of-class experiments to gain an understanding of its various educational aspects. Some of the educational aspects explored in these experiments included how SimSE compares to traditional teaching techniques, and which learning theories are employed by students who play SimSE. Our evaluations strongly suggest that SimSE is a useful and educationally effective approach to teaching software process concepts. Students who play SimSE tend to learn the intended concepts, and find it a relatively enjoyable experience. These statements apply to students of different genders, academic performance levels, and industrial experience backgrounds. However, in order for SimSE to be used in the most effective way possible, our experience has demonstrated that it is crucial that it be used complementary to other educational techniques and accompanied by an adequate amount of direction and guidance given to the student. Our evaluations also suggested a number of promising directions for future research that can potentially increase the effectiveness of SimSE and be applied to educational simulation environments in general.

Doctoral Dissertation, Donald Bren School of Information and Computer Sciences, University of California, Irvine, 2006

Dissertation.pdf (7.6MB)

Emily Oh Navarro

Abstract: Software engineering education has acquired a notorious reputation for producing students that are ill-prepared for being productive in real-world software engineering settings. Although much attention has been devoted to improving the state of affairs in recent years, it still remains a difficult problem with no obvious solutions. In this paper, I attempt to discover some of the roots of the problem, and provide suggestions for addressing these difficulties. A survey of software engineering educational approaches is first presented. A categorization of these approaches in terms of the learning theories they leverage then reveals a number of deficiencies and potential areas for improvement. Specifically, there are a number of underutilized learning theories (Learning through Failure, Keller’s ARCS, Discovery Learning, Aptitude-Treatment Interaction, Lateral Thinking, and Anchored Instruction), and the majority of existing approaches do not maximize their full educational potential. Furthermore, the approaches that engage the widest range of learning theories (practice-driven curricula, open-ended approaches, and simulation) are also the most infrequently used. Based on these observations, the following recommendations are proposed: Modify existing approaches to maximize their educational potential, design new approaches to address under-utilized learning theories, enhance the most promising approaches to make them more useful and effective, perform more formal and frequent evaluations of software engineering educational approaches, and frame software engineering education research in the context of learning theories.

UCI, ISR Technical Report, UCI-ISR-05-5, April 2005

Survey.pdf (1MB)

Thomas Birkhoelzer, Emily Oh Navarro, and André van der Hoek

Abstract: Teaching and training is one of the important applications of software engineering process simulation. Up until this point, however, it has only been used in the context of students running simulations of process models that were built by someone else. In this paper, we suggest a different approach: to use the modeling activity for teaching as well, rather than the simulation activity only. In particular, we pro-pose to assign students the task of building a new soft-ware process simulation model using an existing educational software process simulation environment, SimSE. First experiences from a feasibility project are reported.

In Proceedings of the 6th International Workshop on Software Process Simulation and Modeling, St. Louis, MO, May 2005

ProSim2005.pdf (42KB)

Emily Oh Navarro and André van der Hoek

Abstract: SimSE is an educational software engineering simulation game that uses a unique software process modeling approach. This approach combines both predictive and prescriptive aspects to support the creation of dynamic, interactive, graphical models for software engineering process education. This paper describes the different constructs in a SimSE process model, introduces the associated model builder tool, describes how we built an initial model of a waterfall process, and discusses the underlying tradeoffs and issues involved in our approach.

In Software Process Improvement and Practice: 10 (3), pp. 311-325. 2004.

SPIP2004.pdf (284KB)

Emily Oh Navarro and André van der Hoek

Abstract: Simulation is an educational tool that is commonly used to teach processes that are infeasible to practice in the real world. Software process education is a domain that has not yet taken full advantage of the benefits of simulation. To address this, we have developed SimSE, an educational, interactive, graphical environment for building and simulating software engineering processes in a game-like setting. We detail the design of SimSE, present an initial simulation model of a waterfall process that we developed, and describe an experiment that we conducted to evaluate the educational potential of SimSE and its initial model.

In Proceedings of the Eighteenth Conference on Software Engineering Education and Training, Ottawa, Canada , April, 2005

CSEET2005-2.pdf (238KB)

Emily Oh Navarro and André van der Hoek

Abstract: Virtually all software engineering courses employ class projects in which students practice their newly-learned skills. By necessity, these projects tend to be of a small scale. In efforts to better educate students in the many aspects and pitfalls of the software process, different alternatives have been tried over time. In this paper, we describe one such experience in which we put all thirty-two students in the course on a single, large project and gave them the open-ended task of building a prototype of “a better software design environment.” Thi s l ead to a completely new set of dynamics and interesting opportunities to teach topics that normally would not be covered or illustrated by students’ experiences in a regular software project. We introduce our course design, present its progression over the quarter, illustrate its strengths and weaknesses, and discuss critical factors for its repeatability.

In Proceedings of the Eighteenth Conference on Software Engineering Education and Training, Ottawa, Canada , April, 2005

CSEET2005-1.pdf (63KB)

Emily Oh Navarro and André van der Hoek

Abstract: The typical software engineering education lacks a practical experience of the process of software engineering—students are presented with relevant process theory in lectures, but have limited opportunity to put these concepts into practice in an associated class project. SimSE is an educational, interactive, fully graphical computer game that simulates software engineering processes, and is designed specifically to train students in situations that require an understanding and handling of software process issues. In this paper we describe SimSE, including its educational goals, its design, and its implementation.

In Proceedings of the 7th IASTED International Conference on Computers and Advanced Technology in Education, Kauai, Hawaii, August 2004

CATE2004.pdf (132KB)

Emily Oh Navarro and André van der Hoek

Abstract: SimSE is an educational software engineering simulation game that uses a unique software process modeling approach. This approach combines both predictive and prescriptive aspects to support the creation of dynamic, interactive, graphical models for software engineering process education. This paper describes the different constructs in a SimSE process model, the associated model builder tool, and discusses the underlying tradeoffs and issues involved in this approach.

In Proceedings of the 5 th International Workshop on Software Process Simulation and Modeling, Edinburgh, Scotland, UK, May 2004

ProSim2004.pdf (111KB)

Alex Baker, Emily Oh Navarro, and André van der Hoek

Abstract: The typical software engineering course consists of lectures in which concepts and theories are conveyed, along with a small “toy” software engineering project which attempts to give students the opportunity to put this knowledge into practice. Although both of these components are essential, neither one provides students with adequate practical knowledge regarding the process of software engineering. Namely, lectures allow only passive learning and projects are so constrained by the time and scope requirements of the academic environment that they cannot be large enough to exhibit many of the phenomena occurring in real-world software engineering processes. To address this problem, we have developed Problems and Programmers, an educational card game that simulates the software engineering process and is designed to teach those process issues that are not sufficiently highlighted by lectures and projects. We describe how the game is designed, the mechanics of its game play, and the results of an experiment we conducted involving students playing the game.

In Journal of Systems of Software: 75 (1-2), pp. 3-16. 2005

JSS2005.pdf (375KB)

Emily Oh Navarro, Alex Baker, and André van der Hoek

Abstract: A typical software engineering course fails to teach its students many of the skills needed in software development organizations. Because lectures and class projects alone cannot adequately teach about the software process, we have developed a pair of games in which the process is simulated, giving students an opportunity to practice it firsthand. Problems and Programmers is an educational software engineering card game and SimSE is an educational computer simulation of the software process.

In Proceedings of the 2004 International Conference on Simulation in Education, San Diego, California, January 2003

ICSIE2004.pdf (168KB)

Alex Baker, Emily Oh Navarro, and André van der Hoek

Abstract: Problems and Programmers is an educational card game that we have developed to help teach software engineering. It is based on the observation that students, in a typical software engineering course, gain little practical experience in issues regarding the software process. The underlying problem is time: any course faces the practical constraint of only being able to involve students in at most a few small software development projects. Problems and Programmers overcomes this limitation by providing a simulation of the software process. In playing the game, students become aware of not only general lessons, such as the fact that they must continuously make tradeoffs among multiple potential next steps, but also specific issues such as the fact that inspections improve the quality of code but delay its delivery time. We describe game play of Problems and Programmers, discuss its underlying design, and report on the results of a small experiment in which twenty-eight students played the game.

In Proceedings of the Twenty-Fifth International Conference on Software Engineering, Portland, Oregon, May 2003

ICSE2003.pdf (137KB)

Alex Baker, Emily Oh Navarro, and André van der Hoek

Abstract: The typical software engineering course consists of lectures in which concepts and theories are conveyed, along with a small "toy" software engineering project which attempts to give students the opportunity to put this knowledge into practice. Although both of these components are essential, neither one provides students with adequate practical knowledge regarding the process of software engineering. Namely, lectures allow only passive learning and projects are so constrained by the time and scope requirements of the academic environment that they cannot be large enough to exhibit many of the phenomena occurring in real-world software engineering processes. To address this problem, we have developed Problems and Programmers, an educational card game that simulates the software engineering process and is designed to teach those process issues that are not sufficiently highlighted by lectures and projects. We describe how the game is designed, the mechanics of its gameplay, and the results of an experiment we conducted involving students playing the game.

In Proceedings of the Sixteenth Conference on Software Engineering Education and Training, Madrid, Spain, March, 2003

CSEET2003.pdf (156KB)

Emily Oh and André van der Hoek

Abstract: A typical software engineering course consists of a series of lectures along with a small associated class project. Although this may seem like a logical approach, practical, didactic, and timing reasons necessarily lead to a lack of an in-depth treatment of the critical issues involved in the overall process of software engineering. This paper intro-duces and lays out our plans for constructing SimSE, a graphical, interactive, educational software engineering simulation environment that teaches the software process in a practical manner without the constraints of an actual class project. We anticipate that the use of SimSE will enable students to form a concrete understanding of the software process by allowing its users to explore different approaches to managing the software process and giving them insight into the complex cause and effect relationships underlying the process.

In Proceedings of the 2002 Frontiers in Education Conference, Boston, MA, November, 2002

FIE2002.pdf (21KB)

Emily Oh

Abstract: The common software engineering education method of theory presented in lectures along with application of these theories in an associated class project is insufficient, on its own, to effectively communicate the complex, fundamental dynamics underlying real-world software engineering processes. This paper introduces and lays out plans for SimSE, a detailed, graphical, fully interactive educational software engineering simulation environment that teaches the software process in a practical manner without the time and scope constraints of an actual class project. Once completed, this tool will enable students to form a concrete understanding of the software process by allowing its users to explore different approaches to managing the software process and giving them insight into the complex cause and effect relationships underlying the process.

In Proceedings of the 2002 International Conference on Software Engineering Doctoral Symposium, Orlando, Florida, May 2002

ICSEDS02.pdf (38KB)

Emily Oh and André van der Hoek

In Proceedings of the 2001 Workshop on Education and Training (WET), Santa Barbara, CA, July 2001

TOOLS01.pdf (16KB)

Emily Oh and André van der Hoek

Abstract: It is well known that traditional educational techniques can be complemented by simulation to achieve a more effective learning experience. One would expect the same phenomenon to be true in software development. However, the simulation techniques used thus far have not been effective. This paper introduces a novel approach to simulation for software development education that is based on the adaptation of game technology. Specifically, we propose to build a simulation environment that interacts with its users much like games such as SimCity and The Sims. In providing direct, graphical feedback, we hypothesize that this approach allows individuals to develop an understanding of the software processes used in their organization, while organizations as a whole benefit from the ability to explore different approaches to their software development process.

In Proceedings of the 13th International Conference on Software Engineering and Knowledge Engineering, Buenos Aires, Argentina, June 2001

SEKE2001.pdf (66 KB)

Emily Oh and André van der Hoek

Abstract: The common software engineering education method of theory presented in lectures along with application of these theories in an associated class project is insufficient, on its own, to effectively communicate the complex, fundamental dynamics underlying real-world software engineering processes. We are constructing a new approach to software engineering education that is based on the use of an educational software engineering simulation environment. However, a major challenge in developing such an environment lies in creating an accurate model of the real world upon which the simulation is based. In order for the simulation to be a successful educational tool, this model must be based on an appropriate economic model, must consist of the correct "fundamental laws" of software engineering, and must encode them quantitatively into accurate mathematical relationships, thereby correctly embodying and portraying all of the various factors, dynamics, and cause-and-effect relationships present in the real-world software engineering process.

In Proceedings of the 3rd International Workshop on Economics-Driven Software Engineering Research, Toronto, Canada, May 2001 (reprinted in Projects & Profits, 4 (8), 43-50)

EDSER3.pdf (54 KB)