Integrating a hypertext interface into a syntax-directed programming environment

Integrating a Hypertext Interfaceinto a Syntax-Directed Programming EnvironmentMichael A.Bell and Mark G.RiversDepartment of Computer Science,University of Liverpool,P.O.Box 147,L693BX,U.K.AbstractThis paper describes an integration of hypertext browsing and authoring facilities into a syntax-directed programming environment.The editor’s recognition of the programming language building blocks has been exploited not only to guide program entry,but to break programs into units which can be connected into a ments and documentation are separated from the actual program and integrated into this hyperdocument to provide an explanatory hypertext view through which programs can be navigated.The environment also incorporates a flexible history record mechanism deployed at appropriate points in the program structure,rather than only at the file level as with many current programming environments.1IntroductionThe conventional software development environment is centred around two main tools,the program editor and the program compiler,both of which exist independently of each other.General-purpose text editors provide a flat page-like view of the subject text:they make no inferences about any other aspect of the text’s structure,since they have no knowledge of programming language syntax or semantics—and indeed cannot even recognise that the edited text is a program.This independence between the writing and interpreting environments means that typing and syntactic errors are left for the compiler to report rather than being detected as they are made.Many compilers produce misleading error reports because precise diagnosis of syntactic errors requires understanding of bad syntax as well as good syntax.Often,compiler error recovery mechanisms are not complete enough to prevent one error producing several subsequent side-effect errors.This frustrates program development,and increases development time.This suggests that programming would be easier and more efficient if the editor constrained the user to entering correct program syntax;such editors are called syntax-directed [9,11,7].Fig.1(a)shows an example view of a program fragment a > 0 b <= 100<STUB>IF THEN WHILELOOPEND LOOP;END IF;stubs are placeholders (b)(a)nodes in bold typerepresent templatesblock>L R a 0<=100b block cond cond LR stub IF WHILE for future code Figure 1:Program code is represented internally by an abstract syntax treewithin a syntax-directed editor,using templates to constrain entry and editing of each program construct (such as the IF clause or the WHILE clause).The unfilled stub can eventually be filled with an appropriate piece of program code,perhaps containing further templates.The use of modular pretty-printed templates could be extended to present other information associated with the program,as well as the program code itself.Program comments could be made into pretty-printed units too,connectedinto the appropriate parts of the code,and consequently(providing the environment runs on a display with adequate capabilities)this new graphical nature could be exploited to allow pictorial information to be incorporated into comments.Syntax-directed editors’modular but strongly structured presentation of program information to the user would seem to invite the use of hypertext-like mechanisms for browsing between interconnected modules.With the incorporation of documentation into the program code this possibility becomes even more attractive.Program sections can provide links to related parts and to associated comments;references to variables and functions can behave as links to their respective declarations;comments attached to particular pieces of code can be linked to other such comments and also to more general pieces of documentation,introducing a more manageable and navigable framework into each programming project.This paper describes a design for a system which attempts to exploit the potential of hypertext within a syntax-directed programming environment,by providing the user with a hypertext environment for program development.The next section describes the design of our syntax-directed editor,and its built in history mechanism.The following section describes the integration of program documentation,and how hypertext techniques can be employed to facilitate user interaction with the resulting integrated program structure—allowing powerful browsing over the program,documentation,and version information.Finally,we draw our conclusions and identify directions for further research.2The Syntax-Directed Editor DesignThe design described uses a hybrid syntax-directed editor for programming in a subset of ADA.It employs templates for editing high-level language constructs such as IF statements and WHILE loops,and for statement sequencing;it uses basic textual editing for expressions and assignment statements.This not only avoids the inconvenience of having to enter simple expressions through a template interface,but interacts well with the history and hypertext mechanisms to be described later.There is always an edit focus which is located within the program element currently being edited.This is the syntax-directed editor’s equivalent of a cursor.Possible shifts of this edit focus involve moving up or down the hierarchy of nested templates.At the lowest level,during textual editing,the edit focus takes the form of a simple textual cursor.Program code is internally represented by,and manipulated as,an abstract syntax tree(AST),as shown in Fig.1(b). This is a truncated parse tree,with matched tokens(such as IF,THEN and END IF)combined together.Each node represents a program construction(e.g.a WHILE loop),and each template corresponds to one node.The children of a node represent the separate variable program elements contained within that construction(e.g.the condition and statement-block parts of a WHILE loop),and placeholders called stubs reside in those child-slots where code still needs to be entered (corresponding to template stubs).History records enable past states of an item to be ing a traditional text editor on programfiles,history records are only possible at the level of edit phrases—for example,phrase insertions or block deletes could be“undone”. The editor cannot identify any super-structure to the text onto which a more sophisticated history mechanism could be hooked.Partial compensation for this lack of recognition of structure is achieved by splitting a program into smaller units each in a separatefile;histories offiles can be maintained using version-control systems such as SCCS[10]or RCS[12]. The project framework which results is often ad-hoc and can become difficult to negotiate and manage.In the design described,the editor’s knowledge of the program structure is exploited to provide a history mechanism that works between these two extreme levels,at points where a history mechanism is more useful to a programmer—for example,on case statements or procedures.Each template node carries its own history record;these take the form of version trees,which can be visualised as residing in a plane orthogonal to that of the AST(Fig.2).Each version in the tree represents a different historical state of the associated template.The state represented by each version comprises the children that branch from the node at a particular point in its history.Whenever an attempt is made to change a state,a new state is created as a successor to the old one.The old statetoo simple to need a version treeversion tree <STUB>WHILE IF b100L <=0aL >cond of WHILE templateversion treeR R cond block block of IF template Figure 2:Integration of a history mechanism into the ASTis left unchanged:versions are thus immutable .This history mechanism is based on one previously developed within an object-oriented CAD data management system [4].A new version of an AST node is created when the edit focus shifts upward to the enclosing node,providing changes have been made.Fig.3shows a program representation in which the edit focus moves from the initial version of theof WHILE template(A)<STUB>WHILE (B)<STUB>IF b100L <=condcond cond version treeversion tree R <STUB>block blockcond blockblock of IF template Figure 3:How the history mechanism supports program changeIF template into the initial version of the WHILE template,where the programmer replaces the conditional stub with an expression,b<=100.When the edit focus returns from the expression editor to the WHILE template,a new version of that WHILE is created (transition A)containing the new expression.Note that at this point,the enclosing version of the IF still references the original WHILE ,with the stub in its condition part.When edit focus returns from the WHILE to the IF a new version of the IF is created,containing the new version of the WHILE (transition B).This one new version contains all the changes made whilst the edit focus was below it;this minimises version proliferation.V ersion trees can be traversed in order to select a different version state to reintroduce at a particular node.This is done by popping up a window that displays the tree,and selecting the desired state using the mouse,or by using a “backtrack”key to recall the predecessor of the current version.3The Hypertext LayerProbably the most well-known example of integration of hypertext into a programming environment is DynamicDesign [2,1].This is a CASE environment for the C language which enables source code,documentation,and system requirements to be connected and browsed as a hyperdocument.Programs are input to DynamicDesign as text files,and a treerepresentation is built using the GraphBuild utility;although the source browser can be used to work on this tree representation,the principal method of program entry is via a conventional,external,text editor.Therefore,whilst DynamicDesign successfully integrates software lifecycle information,using hypertext,it lacks adequate support for the program development phase by failing to integrate editing facilities.Other attempts at exploiting the potential of hypertext within software engineering environments include the Hy-perCASE project[6]and the Integrated Software Engineering Applications(ISEA)system[3].Both of these systems successfully employ hypertext to provide a navigable view of software,but neither adequately address source code creation—only the browsing of the combined source code and documentation.What we are concerned with in this project is making programs more accessible to programmers.Unlike that described above,our system is oriented towards supporting program editing and documentation during the development and maintenance phases.We have chosen hypertext as a paradigm for construction of an extensible,configurable and dynamic view onto a structured programming environment.Through this view programmers can more easily navigate a large project,as it evolves,and co-operatively work on interconnected sections.3.1Integrating Program DocumentationIn most programming situations,the program code and annotations co-exist in the samefiles and are both manipulated from within the same text editor.This is usually achieved by the use of certain lexical tokens to request the compiler not to compile those parts of the text meant as comments.Some systems extend this simple scheme—for example,Knuth’s WEB system[8]integrates Pascal code and program documentation in the samefile,whilst allowing each to be extracted separately.One problem with combining program and documentation is that of visual contention.Often complex procedures or abstract data types are designed which require large chunks of comment to explain:this comment breaks up theflow of the program itself,making it more difficult for the user to see the overall structure of the section.It also reduces the amount of program code that can be displayed by the editor at one time.In the described system,the user is presented with a split-window view,where the left-hand pane is the program view and the right-hand pane the documentation view.This not only prevents comments from cluttering the program code,but gives a greater degree of independence between the two views;it is possible for different pieces of documentation to be displayed alongside the same piece of program code,and vice-versa,which is not possible if the two views are combined in the samefile.The hypertext layer takes the form of an interconnected net of documentation nodes,which are bound to(but separate from)the program itself.The system provides two kinds of documentation node.A template comment is bound to a particular program template,and is laid out so that it looks like the program template but with the blocks of lower level code replaced by blocks of descriptive text.The documentation window will usually display a template comment whenever its associated program template contains the edit focus;alternatively a template comment can be“popped up”by clicking the mouse on the associated program template(see Fig.4).Also,a template comment can be used to obscure lower level code indefinitely,in the program window,by double clicking on the program template.This allows“code folding”,where large areas of code can be compactly represented by a smaller,descriptive comment.The second form of documentation node is the general comment,which is not explicitly associated with any particular part of the program;these can only be viewed from within the documentation browser.These may be used for information that is relevant more widely than to just one template—for example,information onfile formats or general program architecture.Within programs constructed using pure text editors,comments are restricted to consisting only of text.Since the system is graphical,diagrammatic annotations are possible.Diagrams can often communicate the details of constructionsIF n = 0THEN f := 0;ELSE f := 1;WHILE LOOP n is more than 1multiply running product, f,EN by n and continue with the next lower value of n END LOOP;n > 1IF n = 0THENf := 0;ELSEf := 1;WHILELOOPf := f * n;n := n - 1;END LOOP;END IF;Figure 4:A pop-up template commentsuch as data structures in a way that textual description never could.Most programmers use diagrams when designing their programs,so it would be beneficial to include these in the online documentation.3.2Linking Comments to Make HyperdocumentsAll comments can contain links to other comments,so that they become the nodes of a hyperdocument.Browsing between these comments amounts to browsing through the program itself,since the appropriate program template is always brought to the program window and edit focus when its associated documentation template is arrived at during navigation.Fig.5shows the AST representation of a simple program extract,and a descriptive hyperdocument with the template documentation nodes mapped onto the program templates.Two kinds of links are automatically available within the hyperdocument.Implicit links exist between template comments for parent/child templates,indicated in Fig.5by bidirectional dotted arrows;these are not links proper,but these routes are easily traversed by moving the edit focus between parent/child program templates.The other kind of automatic links are created from variable and program identifiers in the program template itself,to the templates within which they are declared.Neither of these types of link can be destroyed.Links can also be created by programmers;each of these jumps from a button in one comment to some other comment.If the destination node is a template comment,traversal of the link raises not only this comment but also the appropriate area of program code,with the edit focus in the associated program template.If the destination node is a general comment,this is raised in the documentation window without moving the edit focus from its current position;moving the edit focus between templates whilst general documentation is displayed will not result in its replacement by template comments until a “dismiss”button is clicked.This allows general information to be displayed indefinitely whilst the programmer works through a whole area of code.Programmer-created links may be public or private.Public links are visible to all programmers working on the project,whereas private links are visible only to the programmer who created them.Private links can be used to attach private working notes to part of a program,without cluttering up other programmers’views.Traversing a link to a template comment may warp the edit focus from its current position to a new position,perhaps far away in the program.It must be ensured that in leaving the previous position,updates are not lost.Thus warping the edit focus must create appropriate new versions of program templates,as if the edit focus had moved from source to destination one step at a time via the nearest shared ancestor node in the AST.The hyperdocument enables the programmer to navigate through the code and documentation in more useful waystemplate-specific general commentscomments<STUB><STUB>123<STUB><STUB><STUB>>=aimplicit linksexplicit linksIF-ELSEWHILEWHILEIF cond ifTrue ifFalse R L cond blockcondblock cond block Figure 5:How the hypertext layer sits over program codethan are possible in a conventional editor.Most text editors provide aids to navigation,such as marking text positions and searching on simple string terms,but such capabilities are not sufficient to permit quick and easy movement between different parts of a file that are logically linked.Separate tools such as cross-referencers help alleviate this problem,for example by listing line numbers on which declarations and uses of particular variables occur,but such data is statically derived and quickly becomes out of date.The hypertext layer described here provides a flexible and dynamic framework in which concrete links can be easily established between logically linked pieces of information,allowing free and easy traversal along useful routes.3.3The Dynamics of the HyperdocumentThe hyperdocument only provides a view onto the current revision of the program.The reason for this is that traversing a link to an old version of a template could not result in its display,since it may have lost the context within which it can be displayed;for example,there may have been many possible program states within which that template had been used:which one should be displayed around the template?Instead,old versions of templates are inspected by perusing the appropriate history tree.The hypertext nodes do not support a history record for their own contents.However,old versions of template comments are retained in case an old version of the associated program template is reinstated,in which case a newer comment may not be valid.Rather than being attached to the program template itself,a template comment is attached to a particular version of that template (Fig.6).If a change is made to the template comment whilst the current version is more recent than this,a copy of the comment is made and attached to the current version,and the change is made to this copy.This way,the template comment for an out-dated version is never modified.The template comment displayed for a particular program template is the one attached to its current version,or to its closest commented predecessor;in Fig.6,the current template comment is B,since that is the most recent comment inherited by the current version.versions of templateb100<=<STUB>LRcondblockWHILE(A)(B)(C)of WHILE templateversion treetemplatecomments on WHILE Figure6:How template comments are versionedOnly general comment nodes can be added or deleted explicitly,since template comments are bound to the program representation.When a general comment is deleted,all links from it and to it are deleted as well.When the current state of the program changes,template comments present as nodes in the hyperdocument may be affected in several ways.Whereas the introduction of a template can cause no problems,removing a template from the current program state(but keeping it as an outdated version)must result in the links to/from its comment becoming “dormant”;they are retained but become invisible and hence impossible to traverse.If the template is reintroduced by backtracking through the version history,the links are reinstated.If a template is changed to a different version,links to/from the old version’s comment become dormant,and links to/from the new version are reintroduced.There are two kinds of link to a template comment:version-specific links to the old version become dormant,whereas version-general links are simply redirected to the new version.The type of a link is specified by the programmer on creation;the default is version-general.3.4Disorientation and Cognitive OverheadThere are two major problems with hypertext systems[5].Disorientation arises in a conventional hypertext system when a user traverses several links in a large and complex hyperdocument,and becomes unaware of how to return to any point with which they are familiar.Cognitive overheads arise in both creating and browsing hyperdocuments,due to the mental effort required to construct a complex net or to remember the locations of those items of interest.Much research effort is currently being devoted to devising ways to help users navigate without suffering these problems. Many of these aids rely on maps and guided tours.The described system provides this kind of help automatically since the underlying program itself acts as a map.Disorientation is unlikely,since there is always some part of the program visible by which the user can re-orientate themselves.Becoming lost within the program itself is unlikely;the name of the procedure currently being edited is always displayed in a title bar.If a user traverses a link which takes them to a completely different part of the program, they can instantly identify the area that they are in.The cognitive overhead involved in creation of the hyperdocument is reduced by the automatic creation of many links,for example between variable references and declarations.Whilst browsing,the general layout of the immediate surrounding hyperdocument is summarised by the program view,except for general comments.To avoid cognitive overhead in locating general documentation of interest,link buttons can be attached to entire code hierarchies,such as procedures.When the edit focus is within a hierarchy,any buttons attached to the hierarchy are displayed beneath the documentation window.This feature can be used to make important general comments easily accessible from within a whole area of program code.4Conclusions and Future WorkThis paper has presented a syntax-directed programming environment design,enhanced by a hypertext interface.The syntax direction ensures that programs are syntactically correct,reducing the incidence of programming errors.Because of the modular internal representation,code and documentation can be interconnected without being combined as in a simple editor.The modularity also allows connection of program and document nodes into a hypertext graph,providing a more flexible,navigable and informative interface to programs.Much of the hypertext graph can be automatically generated from program structure;this structure acts as a map which reduces the chance of users suffering disorientation within the hypertext environment.Overall,we believe the environment could promote greater manageability of software projects and encourage pro-grammer efficiency,especially during software maintenance.The main limitations with this system are due to the higher learning overheads associated with syntax-directed editors,and the sometimes restrictive editing environment.These problems may be considered worth suffering for the manageability and efficiency benefits that could be gained.As future work,it would be useful to build in some semantic checking,such as a type-checking mechanism to ensure that the uses of variables and procedures match their declared types:for example,to ensure that the number and type of parameters passed to a procedure are correct.AcknowledgementsThe authors would like to thank Colin Charlton,Keith Halewood,and Dave Jackson for comments on previous versions of this paper.Michael Bell is supported by the Science and Engineering Research Council(UK).References[1]James Bigelow.Hypertext and CASE.IEEE Software,5(2):23–27,March1988.[2]James Bigelow and Victor Reiley.Manipulating Source Code in DynamicDesign.In Hypertext’87Papers,pages397–408,November1987.[3]P.Brown.Integrated hypertext and program understanding.IBM Systems Journal,30(3):363–392,1991.[4]Colin Charlton,Paul Leng,and Mark Rivers.An Object-Oriented Model of Design Evolution.Submitted forpublication,1993.[5]Jeff Conklin.Hypertext:An Introduction and Survey.IEEE Computer,pages17–41,September1987.[6]Jacob L.Cybulski and Karl Reed.A Hypertext Based Software-Engineering Environment.IEEE Software,pages62–68,March1992.[7]Keith Halewood.Towards a Uniform Graphical Representation of the Program Construction,Debugging and TestingProcess.PhD thesis,Dept.of Computer Science,University of Liverpool,1991.[8]Donald Knuth.Literate Programming.The Computer Journal,27(2):97–111,May1984.[9]T.F.Lunney and R.H.Perrott.Syntax-directed Editing.Software Engineering,3(2):37–46,March1988.[10]Sun Microsystems.SCCS—Source Code Control System,March1990.[11]Tim Teitelbaum and Thomas Reps.The Cornell Program Synthesizer:A Syntax-Directed Programming Environment.Communications of the ACM,24(9):563–573,September1981.[12]W.F.Tichy.Design,Implementation and Evaluation of a Revision Control System.In Proceedings of the6International Conference on Software Engineering,Tokyo,Japan,September1982.IEEE Publishing Inc.。