Modeling and Simulating Extreme Collaboration An Agent-Based Approach

Modeling and Simulating Extreme Collaboration: AnAgent-Based ApproachNarjès Bellamine - Ben Saoud*University of Tunis, TunisiaNarjes.Bellamine@ensi.rnu.tn1. IntroductionUnderstanding how technology can aid cooperation has always been a great challenge in the Computer Supported Cooperative Work (CSCW) field. In fact, a large number of studies of collaborative work in a naturalistic setting have been used to understand characteristics, interactions, and activities of actors within their shared environment. Understanding this type of “real-world” cooperation is fundamental for designing innovative solutions to support cooperative work. Such solutions are not limited to groupware tools, but also include new cooperation configurations (such as designing groups and their environments) and communication devices. The deeper our understanding of the collaborative process, the better that technology can be designed to support and improve it. Our ultimate goal in understanding cooperation is to improve the usability, and adoption of groupware systems.It is recognized that computer simulation, especially agent-based simulation, is valuable for studying complex systems. Computer simulation in general, and agent-based simulation in particular, is a primary research tool of complexity theory [Axelrod, 1997]. Several agent-based simulation applications has been developed in many domains (e.g. [Kreft, 1998], [Strader, 1998],[Terano, 2000] ). Despite this growing interest in applying computer simulation to study complex systems, there are few simulation studies that have been done in order to understand and model various configurations of computer supported work (c.f. [Dugdale, 2000], [Pavard, 2000]).In this paper, we report first on fieldwork conducted of a collaborative space mission design team. From these results, aspects of the collaborative process were identified and incorporated into a simulation model. Our main objectives in this paper are to design and develop an agent based simulator in order to: Have a virtual collaboration environment where we can evaluate (run and compare) different cooperation scenarios (real and hypothetical) that apply to the type of engineering practices we have observed from a case study in order to study group activities (mainly sidebar conversations and error recovering).Our computer simulation approach would assist CSCW designers in their design choices involving technologies, actors, and environments. The approach could identify the best scenarios and eliminate, for example, “catastrophic” combinations of parameters and values. It would also be useful for defining strategies for integrating solutions into organizations by providing ideas on what to change first and the probable consequences of such a change. Our findings would contribute to the design of CSCW solutions by providing a “manageable” evaluation of great number of different configurations.This paper is organized as follows. In section 2, we describe results from a case study of collaboration in the domain of space mission design. In section 3, we describe how complexity theory relates to this type of collaboration. In section 4, we describe the simulation model and validation. In section 5, we present results of the model. In section 6, are our conclusions and description of future work.* This work has been done in collaboration with Pr. Gloria Mark, University of California, Irvine2. Overview of the case studyIn this paper we simulate collaboration where group members meet face-to-face, co-located in the same room, and work synchronously. We have chosen a unique face-to-face collaborative setting that we refer to as extreme collaboration, where a physically collocated team uses computer technologies, an innovative design process, and the same room environment to streamline communication and information flow. Extreme collaboration is a unique form of concurrent engineering. The team in this study can design a highly complex space mission, involving thousands of parameters, in about nine hours, a remarkable feat. Why have we chosen to study this group? Recently, there has become an interest in studying the effects of placing project teams to work together on an entire project in the same meeting room. Rather than gather teams together exclusively for status meetings, the goal of extreme collaboration is to have teams working together synchronously in all phases of the project. Some results suggest that such environments can lead to increased productivity [Teasley et al. (2000)]. In this paper we present a summary of the fieldwork observations of a space mission design team, engaged in extreme collaboration. We describe here those aspects of the team that could be modeled to inform collaboration design: interdependencies, and errors. For a more detailed description of the collaboration process, see [Mark, in press], which also described the benefits and limitations for extreme collaboration and warrooms.The setting of the space mission design team is NASA’s Jet Propulsion Laboratory (JPL). The team was formed about five years prior to our study to serve as internal consultants to the JPL and NASA to design space missions proposals, e.g. the Mars shuttle. The design proposal defines all aspects of a mission: how the science requirements will be fulfilled, which telecommunications devices to use, how much power and propulsion is needed, what information will be transmitted back to earth, and so on. Team M1 is composed of sixteen members who are engineers with expertise in a particular subsystem for space mission design, such as power, thermal, or telecommunications systems. There is also a team leader and a customer. The whole team works together in a large technology-mediated room, which we call a warroom. Each team member has a workstation at one of four tables in the room, but they move quite often around the room to converse with others. Collaboration is supported by various technologies in the warroom: three public displays which can be networked to anyone’s workstation, databases from past missions that each engineer can access, an orbit visualization program, and a simple publish-subscribe system of networked spreadsheets to exchange information.The fieldwork observations revealed that during design sessions, the group members solve design problems by changing continually between separate individual subsystem work, small group work, and orchestrated entire team work. The team members use all the information available to them in the room in order to guide them in deciding whether to do individual work, small group work, or collective group work. The room configuration, known interdependencies between subsystems, and the ability to selectively monitor information, aids the entire group in recovering errors and in joining relevant sidebars.2.1. Interdependencies among the team membersOne way that the engineers filter information is by focusing in on sidebars (small group conversations) that might be most relevant for their problem-solving. By understanding the complex interdependent relationships in the team, the engineers can determine whether a sidebar might be likely to be relevant for them. For example, the Telecom Systems engineer needs information from the Control Data Systems engineer, Attitude Control Systems, and Ground Systems engineers to calculate his results. Any change of these subsystems’ results after he has incorporated them into his calculations, will mean that he has to recalculate his findings. Figure 1 is intended to show the complexity of the interdependencies of the team members during a design proposal session. This figure was created based 1 A pseudonymon the interviews, and consistency was checked by comparing the different subsystems’ descriptions of their interdependencies.2.2. Monitoring the environmentThe engineers selectively monitor, access, and filter various sources of information in the room: sidebar conversations, public conversations, public displays, the team spreadsheet, the team leader, the customer, their neighbor’s screen, and one’s own spreadsheet. Monitoring the environment has helped the team recover from calculation errors, and even from software errors in the publish-subscribe system.Figure 1: An overview map of interdependencies that exist in the team.3. Extreme collaboration is a complex systemExtreme collaboration is a very complex process where heterogeneous (different skills and roles) agents (actors (engineers, leader, customer)), who are dynamically organized into subgroups, adapt their behavior (individual work, within sidebars, moving, overhearing), monitor information selectively, give feedback continuously and in an unpredictable way. They perform these actions to solve problems, improve the mission design, and ultimately refine costs associated with the mission.Pavard et al. ([Pavard, 2000], [Pavard, 2001]) identified four specific properties of complex systems, which fit with our case study of extreme collaboration: non-determinism, limited functional decomposability, distributed nature of information and representation, and emergence and self-organization. We will explain each of these properties according to our case.Non-determinism refers to the fact that is impossible to anticipate precisely the behavior of the system even if we completely know the functions of its constituents [Pavard, 2000]. Two important examples of non-deterministic and non-traceable mechanisms in our case are: 1) sidebar formation, and 2) broadcasting information within the working room. By broadcasting, we refer to the information that can be heard by anyone in the room as a result of engineers talking together in a sidebar. Even though we know that usually a sidebar conversation is initiated by one subsystem needing information or help from other subsystems, at each moment of the meeting we cannot predict which subsystems will call for a sidebar and with which partner he will discuss or ask for explanations. We also cannot predict how a sidebar will evolve (i.e. for which engineers it will be relevant and which engineers are ready or even able to join). It is extremely difficult to trace the flow of information associated with the broadcasting of sidebar discussions: neither the actors, nor the observer have the means or the cognitive resources to know who heard the message and even less to know how it was interpreted [Pavard, 2000].Limited functional decomposability. Since our system is dynamic and is in permanent interaction with its environment, it is therefore difficult to study its properties by decomposing it into functional stable parts. In the current study, it has been observed that physical collocation provides multi-sensory awareness of teammate activities. An engineer learns about the state of the space mission design by overhearing conversations, by hearing the team leader’s comments, by watching the public displays, by seeing who is speaking with whom, and by seeing the results that have been published by others and are available for him to use. Also, the fieldwork showed that people manage their monitoring activity by using strategies to selectively focus on information that is relevant for them, especially in a busy setting. They do this by focusing on those with whom they are interdependent.Distributed nature of information and representation. In team M, the common object of cooperation, i.e. the mission proposal, is naturally distributed between subsystems and managed by their representatives. The same information (e.g. values or calculations related to the spacecraft) are often integrated and represented in a specific form by each subsystem. (The same data can be converted in different calculation systems having different physical units; a temperature may be expressed in Celsius or Kelvin scale). Also, this team shares resources (e.g. public displays), which enable knowledge distribution and information sharing among the actors. Also, the broadcast mechanism (in verbal sidebar communication, or public displays) provides information diffusion, which becomes “distributed” in the group. Due to the environmental factors, and to the actors’ availability (in terms of their workload which then affects the attention resources they can allocate to monitor activities in the room), we cannot be sure who is capturing the “diffused” information, and how it is interpreted and represented by the actors. In fact, the information distribution is opportunistic and cannot in any way be predicted. Not only do the team members monitor for information specific to their team role, but also for information that informs them of the team progress, especially important since the team is working under time pressure. For example, the configuration graphics diagram on the public display informs them how far along the spacecraft design is. The customer or team leader’s announcement that they approve of an input parameter makes them aware that this part of the proposal has been accepted. The engineers reported that they never look at the clock. The overall progress of the team signals how much longer they must work; the session ends when the design proposal is finished.Emergence and self-organization: Emergence is one of the most important properties of complex systems. Intuitively, a property is emergent when it cannot be anticipated from knowing how the components of the system function. By its multiple local interactions, the system can behave along some global features (emergent), which allow it to evolve towards more effective modes of organization (self organization) without calling upon exterior or interior structuring operations [Pavard, 2000]. For our case, teamwork shifts between individual, subgroup, and entire group work. It is often unpredictable in real-time collaboration as to how a team should divide itself to solve an emerging problem. Thus Team M is capable of self-organization. Sidebar conversations emerge and actors’ behavior evolve over time so that they group and regroup to solve the problem-at-hand.4. The agent-based simulatorThe main objectives of this computational modeling approach are to understand, answer, and solve different questions and problems concerning extreme collaboration. The first and immediate questions aim at understanding sidebars and modeling error detections: what are sidebar mechanisms and what are their real processes? How do sidebars enable engineers to catch errors? How does the level of noise and the spatial location of the engineers affect catching errors? Given that someone has made an error in a calculation, how long is it before errors are caught? What is the optimal group configuration (spatial arrangement, number of simultaneous sidebars) for a better group performance (in terms of error recovery)?4.1. What is being simulated?To summarize, in the empirical field study, we found that monitoring the environment:• provides awareness of sidebar discussions (a sidebar is a dynamic subgroup. It is a set of two or more participants (a subgroup) joining together to solve an emerging problem.) so that team members know where they need to contribute their expertise, and what information they can use for solving problems, and• helps team members to recover from errors in the space mission design ; From the field observations, during the design process of a space mission proposal, different sources of errors were identified by different members of the team. Table 1 summarizes the sources of errors, the ways they may be caught and the potential detectors.Who detects an error How errors are detected Why errors occurEngineer Leader(over)hearing verbalconversationsbeing in a sidebarreading public displayusing software systems(computing)making wrong assumptionsbreaking a link in a database/ notupdating valuesnot talking to the right personstress and tirednessTable 1: Errors in the team: their sources and catching process.Consequently, we are building the simulator with the purpose of simulating two “cooperation patterns” important in extreme collaboration, and in cooperation in general: sidebar monitoring and error recovery.4.2. Overview of the modelWe are designing a computer simulator, which is a virtual environment: - taking into account the actors, their activities and interdependencies as well as the environment in which they are collaborating; - able to mimic the observed team collaborating; - able to support experimentation of new, “hypothetical” scenarios; - as a generic tool, offering reusable modules, which may re-used by other organizations and collaborative situations.We are using an Object-Oriented approach to design and develop our model and simulator in an iterative way. The main components of our model are the environment, team, sidebar and error; where the team is composed of its members. The main classes of our object class model are member, environment, sidebar, and error.4.2.1. Modeling the environmentFrom the field observations and data analysis, an average of 100 sidebars occur for each 3 hour design session, ranging from 20 seconds to 20 minutes of participation for each member. In this warroom, given these room dimensions, it has been observed that when an engineer is speaking (in a normal conversation tone, and not unusually loud), his conversation can be monitored by all the team members. Also, from interviews and observations, we noticed that sometimes the room becomes noisy; engineers reported this, describing that this hinders them from hearing all what is being said in the room. Consequently, in our environment class design, we consider two constant attributes: (room dimensions: width, height) and a variable attribute (level of noise).Level of noiseFrom the observations, during a design session, the entire team is often split into subgroups. The number of simultaneous sidebar conversations varies from 0 to 5. According to the collected data, we found that from session to session we have a correlated data set in terms of percentage of time where simultaneous sidebars are taking place.We modeled the level of noise in the environment as a function of the number of simultaneous sidebars in this room. We define a three level scale as follows:Level of noise = 1. (low) if 0 or 1 sidebar2. (medium) if 2 or 3 sidebars3. (high) if more or equal to 4 sidebarsThe level of noise is updated at each simulation step (an updateLevelOfNoise method is provided in the environment class, and a repeating schedule is calling this method at each step).4.2.2. Modeling sidebar conversationSidebar conversations are highly present in extreme collaboration design sessions (the average number is about 100 during a session), during 68% of a session there is at least one sidebar occurring. We focused our work on this collaboration pattern and model it as a source of information and also as a source of noise. Each sidebar is characterized by:• its initiator: this is the team member who experiences a problem and needs to solve it, or simply needs some explanation or data conversion from his partners to provide input for his subsystem.So, based on the need to discuss something, the initiator would move to the appropriate person in the warroom and initiate this sidebar.• beginning and end time: each sidebar is considered as an event occurring from beginTime to endTime attributes.• Each sidebar is also characterized by its location in the room. In fact, for our model we assumed that all sidebar members would be in the same area; i.e. each agent joining a sidebar would physically move to the sidebar location to be close to other members. Once a sidebar is created, at beginTime, the sidebar location is the partner’s location. Those two first members begin talking together. Their conversation messages are broadcast to people outside this sidebar.As all engineers are continuously monitoring the environment, and in this model capturing sidebar outputs, according to the content of what they are overhearing, they may decide to join the sidebar or not.4.2.3. Modeling the Team and MembersThe team class refers to the entire teamM group. It includes therefore its size and the list of its members. Thus, in our model, the member class refers to any human actor in the group. It is characterized by the name, the role (e.g. leader, customer, power engineer, thermal engineer), the seating position (where his/her terminal is set and is supposed to be unless (s)he has moved to a sidebar). An actor may be a sidebar initiator or joiner (where he becomes a sidebar member).An engineer may have one of the following types of status: computing (default activity coded as status 0), inSidebar (status = 1), leaving a sidebar (status = 2), resuming (status = 3): By default, all engineers are computing and on the same time monitoring their environment and overhearing what is happening in the room. For each sidebar, on its begin time, if the initiator is not already involved in another sidebar, he initiates it and its status changes from computing to inSidebar. On engaging a discussion with a potential partner, this sidebar becomes a source of information which is broadcast to the whole room. For each actor, according to his ability to hear and according to the relevance of this sidebar, he would decide to join the sidebar or not. Also, on detecting an error, an engineer may join a sidebar. All sidebar joiners remain in it until end time, when they leave it to fall into a resuming state. We allow 20 seconds for the engineers to resume their workstation calculations after leaving a sidebar. Thus, the engineers wait 20 seconds before going back to computing, unless they are interrupted again by joining a new sidebar.Behavior of agents while involved in sidebars: when involved in a sidebar, an engineer overhears all what is being said within his/her own sidebar and can also overhear what is coming from other sidebars. Since in our model, and for purposes of simplification, our agents are kept in each sidebar until it ends. We assume that an “inSidebar” agent would catch and process only the messages of the sidebar to which he belongs. This implies that all his attention resources are fully involved for resolving the sidebar problems and then he has greater probability in finding errors when involved in sidebar.Modeling Communication /messagesAs a first model, we consider the verbal communication first. The representation of the verbal communication message is an abstraction of the content. Messages are rather abstractions of packets of information that are generated and sent by one actor and received and processed by another. When broadcast in the warroom, they may contain errors, and may be considered as relevant or not for those hearing/receiving such messages.Modeling the ability to hearAccording to the distance between actors and the environment factors (mainly the level of noise) some messages can be lost. We model the hearing ability and assume that on sending a message from an actorA to an actor B, the sent message will be added to B’s heardList only ifB can hear; where this ability to hear function is related to distance and the level of noise as fellows: if the level of noise is low or medium, messages can be heard by all team members; if the level of noise is high, only neighbors can hear.Modeling sidebar relevanceOn overhearing one or many sidebars (stored in his heardList), an engineer processes this data and decides whether to join a sidebar or not. A same sidebar output may be relevant for some members and not relevant for others. Many strategies may be applied for deciding about the relevance of what is heard. In our model we experimented with the next two “relevance and then joining strategies”.• Strategy 1: One receiver considers a heard message relevant for him if it comes from a sidebar where at least the initiator or his partner are included in his interdependency map. In other terms, this assumes that the sidebar topic is about one of the two initiating subsystems, which is useful for the engineer to join and get information about those subsystems.• Strategy 2: One receiver considers a heard message relevant for him if it comes from a sidebar where at least one of the sidebar members is interdependent with him in terms of exchanging information. In other words, the listener needs one of the subsystems involved in this sidebar to be an engineer that he has an interdependency with at this stage in the design.These two strategies have been chosen among many others since we think that they would be the closest to the real design work, since the field study emphasizes the great importance of interdependencies and the engineer’s selective behavior guided by their needs.4.2.4 Modeling errorsIn our simulator, we will generate errors randomly while only sending messages. We model an error class as a separate object class. This allows us to keep track of each error separately and to easily associate group members with errors. Each error is characterized by: its beginning time (when it occurs), sidebarIdErr (within which sidebar the error occurs), a boolean variable caught (set to true when the error is detected), end time (when the error is caught), and errorDetectorsList (storing the name and status of agents detecting this error ).Modeling error-catchingSince errors are modeled as being a sidebar attribute which is set to false by default and set to true if the sidebar outgoing message is erroneous, error catching may be performed by all those members who are in the sidebar (source of this message) and those who are not involved in any other sidebar. For each of these category of potential detectors, an appropriate catching error probability is assigned: • Members involved in sidebars have high probabilities of catching, since they are doing nothing else but focusing on the topic of their sidebar. This probability is referred to as catchErrorProbability.• Members outside sidebars catch errors with lower probability as only a part of their attention resources are allocated to monitoring the whole environment and eventually to catching errors.Consequently, in our model we consider a catchErrorProbability and anattentionResourcefactor (e.g. 0.2 or 0.4), and compute the inSidebarCatchErrorProb as the product of the two previous ones. InSidebarCatchErrorProb is therefore diminished according to reduced attention resources.4.3. Developed simulatorOur simulator is developed using the SWARM 2 platform (developed by the Santa Fe Institute) and is written in java language.According to our main objectives, we have developed a first model and implemented an agent based simulator focusing on the sidebar conversations and error recovery.Table 2 summarizes the set of simulator parameters, input as well as collected output on running simulation:Simulator parametersInput Output – Probabilities for errors:generating, catching bysidebar members, catchingby overhearing– resuming delay ,- broadcasting frequency – for each member: name, role, seat location, interdependency map – sidebar events: initiator, begin and end time – continuous overview of the sidebar formation and error recovering – sidebars over time, performed or not, size, members) – errors over time, caught or not, duration– level of noise over timeTable 2. Simulator parameters, input output.Figure 6: Overview of the simulator interfaceFigure 6 shows our simulator interface; where various output representations are provided:• Control panel: enabling the user to run a simulation step by step or continuously, to stop it,• Display of the working room, showing the spatial arrangements of subsystems as well as the location and the status of cooperating agents: magenta rectangles with the subsystem names show the assigned positions of agents where they have their terminals; the ovals represent the2 。