Extended requirements traceability Results of an industrial case study

Extended Requirements Traceabil ity:Resul ts of an Industrial Case StudyOrlena Gotel & Anthony Fink elsteinDepartment of Computer Sci enceCi ty Uni versi tyNorthampton SquareLondon EC1V 0HB[olly|acwf]@AbstractContri buti on structures offer a way to model the network ofpeople who have participated in the requi rements engi neeri ngprocess. They further provide the opportunity to extendconventi onal forms of artifact-based requi rements traceabi li tywith the traceability of contri buti ng personnel. In this paper, wedescribe a case study that investigated the modelli ng and use ofcontribution structures in an i ndustri al project. In particular, wedemonstrate how they made it possible to answer previouslyunanswerable questions about the human source(s) ofrequi rements. In so doing, we argue that this i nformati onaddresses important problems currently attributed to inadequaterequi rements traceabi li ty.1.IntroductionThe inability to answer questions regarding the human source(s) of requi rements i nformati on has been found to result i n claims of requirements traceability problems [5]. An approach to address this problem was presented in [6]. Thi s paper describes a case study designed to evaluate, through demonstrati on, whether use of the approach helps answer practitioner’s outstandi ng questions and, in so doing, allevi ates an i mportant class of requi rements traceability problems. The case study is based on materi al gathered from a real industrial project over a period of three years.In Secti on 2, we explain what requi rements traceability is and describe the underlyi ng reason for long-term requi rements traceability problems. W e provide examples of the kind of questions that are problematic to answer as a consequence. We then outli ne an approach to address this fundamental problem and summari se how it is anticipated to provide answers that satisfy practitioners’ needs. In Secti on 3, we describe the case study materi al used to validate our claim. Since the approach was not in existence at the onset of the project, its requirements did not drive the data gathering. From the extensi ve records that were mai ntai ned, we only summari se that data perti nent to the approach. In Secti on 4, we demonstrate how the approach was applied in a post-hoc manner to this data, thereby reveali ng i nformati on about the project’s evoluti on. In Secti on 5, we show how this i nformati on makes i t possible to answer questions about the project regarding involvement, responsibility, ramifications, change notification and work ing relationships. Due to the manner in whi ch the approach was applied, we are only in a posi ti on to validate the feasibility of the approach and the usefulness of the i nformati on it provides in a hi stori cal and subjecti ve manner. Another repercussi on of not being on the project’s critical path is that we can only suggest how the i nformati on could be used, say to assist the mai ntenance process. Based on our experiences and practitioners’ comments, we highlight some outstandi ng issues in Secti on 6, then make recommendati ons for uptake.2.Contribution structures for requirements traceabilityIn this section, we describe what requi rements traceability is, why it is important and what the problems with it are. We then outline an approach to address a fundamental problem that currently makes it difficult to recover i nformati on about the human source(s) of requi rements i nformati on.2.1.Requirements traceabilityRequirements traceability refers to the ability to describe and follow the life of a requi rement in both a forwards and backwards direction (i.e., from i ts origins, through its development and specification, to its subsequent deployment and use, and through periods of on-going refi nement and i terati on in any of these phases). It is considered the primary techni que to help with many project-related acti vi ti es, like ensuri ng that systems andsoftware conform to their changing requi rements, but is commonly cited as a problem area by practitioners.Although the number of tools that clai m to support requi rements traceabi li ty is growing, some more recent ones being described in [10, 11, 14, 15], the schemes that need to be established prior to their use have received rather less attenti on. Wi th few exceptions, examples being the requi rements traceabi li ty models of the U.S. DoD [8, 9] and the requi rements traceability meta models arising from the NATURE project [12], endeavours to i mprove the potential for requi rements traceabi li ty have mostly i nvolved uncoveri ng and recordi ng as much information as possible about the requi rements engi neeri ng process, then li nki ng it in i nteresti ng ways for trace retri eval. Thi s can lead to an unwieldy mass of unstructured and unusable data wi thout some a priori di scri mi nati on concerni ng the type of requi rements i nformati on that practitioners are likely to need access to.Followi ng an empi ri cal study reported in [5], we argued that the most fundamental i nformati on to record for reli evi ng long-term requirements traceability problems was that whi ch identified the human source(s)of requi rements i nformati on. We found that, what are perceived to be requirements traceability problems tend to arise when practitioners are unable to answer questions about the personnel who had been i nvolved in the producti on and refi nement of requi rements. Thi s is because people are frequently the ulti mate baseline whenever requi rements need to be re-exami ned or re-worked. Examples of such questions are gi ven in Figure 1. However, we also reported that information concerni ng the human source(s) is often neglected in the stri ve to replace i nformal social contact with exhausti ve documentary records.“Who has been involved in the production of this requirement and how?”“Who was originally responsible for this requirement, who is currently responsible for it and at what points in its life has this responsibility changed hands?”“At what points in this requirement's life have the working arrangements of all those involved been changed?”“Who needs to be involved in or informed of changes proposed to this requirement?”“What are the ramifications, regarding the loss of requirements-related knowledge, if aspecific individual or group leaves a project?”Figure 1: Personnel-related requirements traceability questions.2.2.Contribution structuresWe described an approach to address this more focal problem underlyi ng long-term requi rements traceability problems in [6]. In that paper, we also described a prototype tool to support the approach and provided scenarios of use. Formali sati on of the approach, plus the i nferences it supports, can be found in [4].We summari se the mai n steps of the approach in Figure 2. The approach is based on modelli ng the contribution structure underlyi ng requi rements. Thi s reflects the network of people who have contributed to the artifacts produced in the requi rements engi neeri ng process. The i nformati on in this model makes it possible to extend conventional forms of artifact-based requi rements traceability with a form of personnel-based requi rements traceability. W e claim that this new dimension can help elicit answers to the above questions and so alleviate an important class of requirements traceability problem.contributed to an artifact’s producti on is also mai ntai ned in its contribution format. Based on Goffman’s work on the nature of participation in soci al encounters [3], this structure delineates the principal, author and documentor of an artifact. As described in [6], these categories have been moti vated by soci oli ngui sti c and textual li ngui sti c theori es, and chosen for thei r analyti c potenti al. Together, the i nformati on they provide can be used to i nfer detai ls about soci al roles, role relati ons and commi tments.3.Case studyIn this section, we give details of the company, project and participants of the case study. We describe the data we gathered and our method for so doing.3.1.ProjectThe project came from a small communi cati ons company employi ng about twenty-fi ve people. It runs many projects concurrently, provi di ng software and procedural soluti ons to communi cati ons-related problems. The ori gi nal objecti ve of this project was to supply a dedicated communi cati on service to complement a customer’s disaster recovery programme. The project was initiated in February 1992 and went live at the end of March 1992.In August 1992, the idea of developi ng a generic service was discussed. Six versi ons of a requi rements and design specification were drawn up throughout September 1992. These were then abandoned until new staff were employed at the end of October 1992 to develop and market the service. Followi ng much staff turn-over, the generic service did not go li ve unti l February 1994. Between October 1992 and February 1994, the specification evolved i nto an operati onal service, an operati ons manual and a high-level manager’s guide. Si nce February 1994, the generic service and accompanying documentati on has undergone conti nuous modi fi cati on to account for the requi rements of new customers.Most of the artifacts produced during the project were i nformal and paper-based. All that remains within the company today is an early specification, an up-to-date operati ons manual, an up-to-date manager's guide, contracts with customers and mi scellaneous correspondence. Requi rements traceability has not been mai ntai ned. Those sti ll i nvolved i n the project no longer remember from where or from whom the vari ous aspects of the current service have been derived. Some problems have resulted from this loss of information but,because the project is restricted in scope, and because the team is small and exhibits some staff continuity, these have not been critical to maintainability.3.2.Data gatheredThe work that occurred from the i ni ti al discussion about provi di ng a generic service, through to the sixth versi on of the requi rements and design specification, was followed closely. We observed all the meeti ngs that took place, made notes and audio recordings, and collected photocopies of any tangible artifacts produced. We also participated in some aspects of the process. During this ti me, a detailed picture of what had happened when developing the i ni ti al customer-specific service was reconstructed with those who had been i nvolved.From the end of October 1992, we mai ntai ned a record of the mai n arti facts produced due to the specification, many of whi ch were in producti on for months. For the purposes of the case study, our definition of “artifact” applied to single physical documents. Not only does this promote identification, but i t enables us to exami ne the viability of the approach at a coarse level of granularity prior to i ntroduci ng further complexity. Some of the traceability i mpli cati ons that arise from decomposing a single physical artifact i nto a number of components are menti oned in Secti on 5. During this period, we also mai ntai ned a record of the people i nvolved in the producti on and di stri buti on of these artifacts. However, we were unable to mai ntai n a full record of the peri pheral artifacts, like the notes made during meeti ngs wi th customers and so forth.Fifty-eight people contributed directly to the project. These i ncluded individuals and groups from wi thi n the company and from outside. We use alphabetic identifiers when we refer to these people below. One hundred and sixty-six mai n artifacts were produced in the project. We use numeric identifiers when we refer to these artifacts below.3.3.Project phases(1)The artifacts to do with the i ni ti al customer-specific service. Twenty-three artifacts were produced between February and March 1992.(2)The artifacts to do with developi ng the baseline specification for thegeneric service. Sixty-five artifacts were produced from August to September 1992.(3)The artifacts to do with developi ng the i ni ti al generic service. Thi rty-nine artifacts were produced from October 1992 to July 1993.(4)The artifacts to do with extending the generic service to address newcustomer requi rements. Thi rty-ni ne artifacts were produced from September 1993 to June 1995.4.Application of approachIn this section, we outli ne how the approach was applied. Based on the data we had gathered, key project participants were tasked to reconstruct the mai n artifact-based relati ons and to give them mi ni mal semanti cs. Under si mi lar condi ti ons, they were also tasked to reconstruct the contri buti on format for each artifact. We then examined what could be inferred about social roles, role relati ons and commi tments as a consequence.4.1.Artifact-based relationsFor each project phase, its artifacts were numbered according to production order, then temporal relati ons were established between them to reflect the order in whi ch they had been produced. Thi s ordering is suggested in Fi gure 3. The nature of these relati ons was further clarified, based on [1], as: before(1,2); during(4,3); equal(11,12); meets(16,17); etc. The coarse flow-down of information amongst these artifacts was also established. This is also shown in Figure 3. Note that, no attempt was made to determi ne all the possible relations, like the many transitive or internal relations for instance.An example of the mi ni mal semanti cs gi ven to these artifact-based relations is shown in Table 1. In the first column of this table, we list the original reason gi ven for the relati on by participants. In the second column, we categorise the nature of this relati on according to standard classifications of cohesi on and coherence relati ons [2]. Based on this, we list the broad communi cati ve functi on between the two related artifacts in the third column. This function encapsulates the finer purpose of the relati on, helps to eliminate individual differences in classification, so provides what we refer to as i ts minimal semantics.Figure 3: The order in which artifacts were produced and the predominant artifact-based relations between them. Note that, elongated artifacts, like 102 and 128, represent all the correspondence between the company and potential customers; correspondence was difficult to keep track of. Note that, the slightly enlarged artifacts, like 115 and 147, indicate that other artifacts were produced whilst in production.(1) Informal description of relation given by practitioner (2) Nature of relation interms of cohesion andcoherence relations(3) Broad communicativefunction of relation givesminimal semantics2 qualifies 1 2 adds to 1 2 adopts 12 is the reason for3 2 frames 3 3 references 24 defines 2 4 adds to 2 4 adopts 22 is the reason for 6 2 frames 6 6 references 23 assists with4 3 substantiates 4 4 references 35 is compared with 4 5 matched with 4 5 references 46 refines 4 6 alters 4 6 adopts 45 assists with6 5 substantiates 6 6 references 57 responds to 4 4 causes 7 4 references 76 is the reason for 8 6 frames 88 references 67 is background for 87 frames 88 references 79 is a result of 88 causes 99 references 89 assists with 109 substantiates 1010 references 910 elaborates 610 adds to 610 adopts 68 is background for 108 frames 1010 references 810 is the reason for 1110 frames 1111 references 1012 replies to 1010 causes 1212 references 1013 replies to 1010 causes 1313 references 1015 extends 1015 adds to 1015 adopts 10Table 1: The nature of some of the main artifact-based relations of phase 1.For phase one, Figure 4 (a) i llustrates those relati ons likely to be represented in conventi onal forms of artifact-based requi rements traceability. Thi s i s because they tend to capture parent-child relati ons, or predecessor-successor relations, so provide requirements history and flow-down [13, 16]. Figure 4 (b) i llustrates those additional relati ons the approach is concerned with, namely the references relati ons of Table 1. It indicates the wealth of contextual i nformati on often not integrated and used actively for requirements traceability purposes.Figure 4: (a) Adopts - the artifact-based relations typically maintained for requirements traceability. The arrows here suggest the flow-down of artifact content. (b) References - the artifact-based relations that further provide context. The arrows here suggest the direction of influence between artifacts.4.2.Contribution formatFor each project phase, the contri buti on format was established for each artifact. Using our example scheme, this indicated the i ndi vi duals or groups who contributed in the capacities of principal, author and documentor. The contribution formats allocated for some of the artifacts produced in phase one are shown in Table 2. The people to whom an artifact was ei ther passed or copied is also shown in this table for completeness.Artifact Principal Author Documentor Distributed to 1BH BI AW AT2AT BB={AW/AV/AT/AR/AX/AU}BB={AW/AV/AT/AR/AX/AU}AA3AA AA/AE AA None4AA AA/AE AU BI/BB={AW/AV/AT/AR/AX/AU}5AA AA/AQ/AP/BB={AW/AV/AT/AR/AX/AU}AA None6AA AA/AT AA AE7BH BI BL AA8AA AA/AE AA None9AA AA/BB={AW/AV/AT/AR/AX/AU}AA None10AA AA AA AE/BI/BB={AW/AV/AT/AR/AX/AU}Table 2: The contribution formats for artifacts 1 to 10. Note that, AA/AE means joint contributors. Note that, group descriptors are decomposed into their members.4.3.QualificationEach contri buti on format was qualified to provide further details about contributions and contributors. As an example, we explain how the authori al status was qualified and highlight the use of this information. Artifact 106 was the first versi on of the operati ons manual for the generic service. It was authored by AI. From artifact 106, the artifact-based relati ons can be used to trace all the paths back to the original author(s). They can also be used to trace forwards to locate the author(s) of all the artifacts arising from it. Details about how each progressive author made use of the previ ous author's contri buti on can thereby be uncovered. Part of this authorial trail is illustrated in Figure 5. From such authori al trails, we can see: whi ch people produce the most ori gi nal artifacts; whi ch use thei r own or another’s contri buti ons the most often; whose contri buti ons get referenced with the greatest frequency; and so on. We can also begin to assess the i nfluence of a person's authored contri buti ons on the surroundi ng body of artifacts and on the project as a whole. Such details can help identify those to notify followi ng different types of change or those to contact regarding different queries.11111212(author of artifact 106)(author of artifact 103)Adopts (adds to)Adopts (alters)(authors of artifact 73)(authors of artifact 85)Adopts (removes from)Adopts(alters)Adopts(adds to)Adopts(removesfrom)Adopts (removes from)(authors of artifact 86)(authors of artifact 88)(authors of artifact 99)(authors of artifact 100)Adopts (alters)(authors of artifact 84)Adopts (adds to)Adopts (alters)Adopts (adds to)(authors of artifact 77) (authors of artifact 81)(authors of artifact 83)(authors of artifact 73)(authors of artifact 74)Adopts (removes from)Adopts (adds to)Adopts (alters)(authors of artifact 65)(author of artifact 80)(authors ofartifact 81)Adopts(removes from)Adopts (alters)(authors of artifact 76)(authors of artifact 77)Adopts (alters)Adopts (alters)Adopts (alters)See Box X(author of artifact 55)(authors of artifact 61)Adopts(removes from)Adopts (adds to)(authors of artifact 58)(author of artifact 60)Adopts (alters)Adopts (alters)See Box Y(authors of artifact 52)(authors of artifact 53)Adopts (alters)Adopts (alters)(authors of artifact 49)Adopts (adds to)Adopts (adds to)(author of artifact 46)Adopts (alters)(author of artifact 21)Adopts (alters)(authors of artifact 45)Adopts (adds to)(author of artifact 36)(author of artifact 30)Adopts (alters)Adopts (adds to) (author of artifact 26)(author of artifact 28)Adopts (adds to)Adopts (alters)Box YAP/AQ AC/AT/AU AI AC/AQ AE/AA/AC/AG AA/AE/AK/AG BB/AP/AQ AE/AA/AC/AG AA/AC/AE/AG AC/AEAC AA/AC/AE/AG AA/AC/AE/AG AE/BWBox XAC/AE AC/AT/AU AE/AA/AC/AGAA/AC/AE/AG AC/AE AA AE/AA/AC AE AA/AC/AE/AG AA/AC/AE/AG AA/AE AE AA AE/BW AC AC AC AA AI [Reads: 106 adopts 103 (106 adds to 103)]See Box X(author of artifact 18)Adopts (alters)Adopts (alters)(author of artifact 15)Adopts (copies)Adopts (alters) (authors of artifact 16)Adopts (adds to)Adopts (copies) (authors of artifact 14)Adopts (alters) (authors of artifact 13) (author of artifact 10)Adopts (adds to)(authors of artifact 6)Adopts (alters) (authors of artifact 4)Adopts (adds to)(author of artifact 2)Adopts (adds to)(author of artifact 1)AA/AT AA/AE BBBI BI/AA BI/AA AA/AEBK/AA/AE AA AA AA Figure 5: Authorial trail following backwards requirements traceability from artifact 106. For clarity, it is filtered to highlight those whose contributions were adopted in getting to106.4.4.Social roles and role relationsThe social roles that people assume when contri buti ng to artifacts can be inferred from the i nformati on we have gathered so far. For instance, if a person is both the principal and author of an artifact, they can be said to be its devisor. If they are solely the documentor, they can be said to be its relayer. The ensui ng role relati ons between people when they joi ntly contribute to artifacts, say as a devisor/relayer pair, reveals more about the underlying contri buti on structure. Not only can we see whom has collaborated with whom, but we can see how they have collaborated and whether these role relations have varied or been sustained.To explain the use of such information, we compare the social roles of two of the project leaders. AI was the project leader when artifacts 99 to 127 were produced and a contri butor to twenty-two of these. AJ was the project leader when artifacts 128 to 162 were produced and a contri butor to twenty-six of these. Thei r social roles when contri buti ng to these artifacts, as well as their role relations to collaborators, are shown in Tables 3 and 4 respectively.Social role of AI On howmanyartifacts?How many onown?Social roles of other contributors whocollaborate with AITrue author (i.e., contributes as P, A and D)1613True author = BM (x1)Ghost author = AA/AE/AD (x1)Ghost author = AA/AE/AD/AT (x1)Nominal author (i.e., PD)20Ghost author = AA/AE/AD/AG (x1)Ghost author = BB/AP/AQ (x1)Representative(i.e., AD)20Sponsor = AD (x2)Ghost author(i.e., A)20Sponsor = AD and Relayer = BO (x2)Table 3: The social roles and role relations for AI.From these tables, we can see that AI worked on his own on over half of the artifacts he contributed to, else he worked with small groups of people. As he worked largely as a true author, he was evi dently a self-sufficient documentor. It is noteworthy that AD tended to collaborate with AI as a sponsor when dealing with customer-related artifacts. In contrast, we can see that AJ worked rarely on his own and mai nly collaborated with one or two others. He had a strong dependency on AE as his ghost author when worki ng together and on many others as relayers, the latter hi nti ng at the need for secretarial support. It is noteworthy that AD was ulti mately responsible for about a third of the artifacts that AJ had contributed to.1313Social role of AJ On howmanyartifacts?How many onown?Social roles of other contributors whocollaborate with AJTrue author (i.e., PAD)72True author = BM (x1)Ghost author = AE (x 4)Ghost author (i.e., A)90Sponsor = AD and Relayer = AP/AS (x1)Sponsor = AD and Relayer = AP (x5)Sponsor = AD and Relayer = AR (x2)Sponsor = AD, Ghost author = AF and Relayer =AO (x1)Devisor (i.e., PA)90Ghost author = AE and Relayer = AW (x2)Ghost author = AE and Relayer = BQ (x1)Relayer = AL (x4)Relayer = AM (x2)Sponsor(i.e., P)10True author = AE (x1)Table 4: The social roles and role relations for AJ.There could be many reasons for the subtle differences in how these two people with the same job description worked in the project. AI did not close any sales and focused on developi ng a marketable service. In contrast, AJ focused on selling what AI had developed and only made subsequent additions to it to account for new customer requirements. Notably, it was with such additions that AE collaborated with AJ as ghost author. Thi s served to mai ntai n some conti nui ty, since AE had also collaborated with AI as ghost author.mitmentBased on the data gi ven in Table 5, we give an example of the kind of i nformati on that can be inferred about the commi tments of project contributors. We can see that AP is mai nly responsible for the physical appearance of artifacts, only responsible for thei r content when collaborati ng with others, though never responsible for thei r ulti mate effect. We can also see which other people AP is commi tted to through thei r joi nt contri buti ons to shared artifacts. Here, we can identify AD and AJ as those with whom AP has collaborated the most often, as well as the number and type of artifacts o n whi ch they collaborated. By extensi on, we can exami ne those people that AP is commi tted to due to the artifact-based relati ons that situate her contri buti ons, though not shown i n the table.The i ntersecti on and difference between commi tments can uncover much i nteresti ng i nformati on. For example, we can identify: whi ch people have collaborated with specified others the most or least often; whi ch people are commi tted to the same set of other people; whi ch people have collaborated1414with customers; which people are committed to the same artifacts and for the same aspects; and which people have contributed to those artifacts that are the i ni ti al sources of requi rements.Contrib to (artifact)Aspect of artifact committed to Contrib with(person)No. artifactscollab on5Content (as one of many contributors)AD622Physical appearance (on own)AJ631Physical appearance (on own)AQ441Content (as one of two contributors)Physical appearance (on own)AT4 85Content (as one of two contributors)AU496Content (as one of many contributors)Physical appearance (as one of two contributors)AW3 100Content (as one of many contributors)AV3111Content (as one of many contributors)AR3139Physical appearance (as one of two contributors)AX3140Physical appearance (on own)AC2148Physical appearance (on own)AD2151Physical appearance (on own)AJ2154Physical appearance (on own)AQ1155Physical appearance (on own)AT1Table 5: AP's artifact and collaborator commitment store.5.Results and discussionIn this section, we select some of the questions gi ven in Figure 1 to demonstrate how they can be addressed. We also menti on other forms of analysis the approach makes possible. The reader is referred to [4] for a more detailed description and a thorough evaluation.5.1.Invol vement“Who has been involved in the production of this requirement and how?”One of the requi rements in versi on two of the requi rements and design specification, artifact 49, was to do with security. It was pursued throughout phase two of the project, cited in all six versi ons of the specification, then dropped in phase three. It led to much i nvesti gati on and many artifacts that became redundant. Once removed, its impact only surfaced over ti me. The resulting problems could have been allevi ated with knowledge of its original source and of those who had pushed for its concern.1515。