The Boisdale algorithm - an induction method for a subclass of unification grammar from pos

The Boisdale Algorithm – an Induction Method for a Subclass of Unification Grammar from Positive DataBradford Starkie1,2, and Henning Fernau2,31 Telstra Research Laboratories, 770 Blackburn Rd Clayton, MelbourneVictoria, 3127, AustraliaBrad.Starkie@.au/~bstarkie/2 University of Newcastle, School of Electrical Engineering and Computer Science, Uni-versity Drive, NSW 2308 Callaghan, Australia3 Theoretische Informatik, Wilhelm-Schickard-Institut für Informatik,Universität Tübingen, Sand 13, D-72076 Tübingen, Germanyfernau@informatik.uni-tuebingen.dermatik.uni-tuebingen.de/~fernau/Abstract. This paper introduces a new grammatical inference algorithmcalled the Boisdale algorithm. This algorithm can identify a class of context-free unification grammar in the limit from positive data only. The Boisdalealgorithm infers both the syntax and the semantics of the language, where thesemantics of the language can be described using arbitrarily complex datastructures represented as key value pairs. The Boisdale algorithm is an align-ment based learning algorithm that executes in polynomial time with respectto the length of the training data and can infer a grammar when presentedwith any set of sentences tagged with any data structure. This paper includes adescription of the algorithm, a description of a class of language that it canidentify in the limit and some experimental results.1. Introdu tionIf an algorithm can identify a class of language in the limit from positive data only, then it is guaranteed to learn a grammar of that class exactly at some finite time when presented with an infinite stream of example sentences generated from that language. In this respect, the ability to identify a class of language in the limit is a measure of the quality of the algorithm if the objective of the learning task is to learn a language exactly. The papers of Gold [5] and Angluin [3] are the quintessen-tial texts on identification in the limit of languages from positive data only. A brief but up-to-date introduction to the state of the art of grammatical inference of con-text-free grammars can be found in de la Higuera and Oncina [4].In Starkie [9] a new grammatical inference algorithm for inferring context-free grammars was introduced called the Left-Alignment algorithm. This algorithm has the property that it can identify a class of context-free grammar in the limit from positive data only. The Boisdale algorithm described in this paper is an extension of the left alignment algorithm that enables the semantics of the language to be learntin addition to the syntax of the language. To this end ,we use unification grammars that are similar to Definite Clause Grammars (DCGs) Pereira and Warren [7]; both can be viewed as attributed context-free grammars. Unification grammars can be used to both convert natural language sentences into data structures (represented as key-value pairs) and to convert data structures into natural language sentences. The algorithm can infer a grammar when presented with an arbitrary set of tagged sen-tences, and can do so in polynomial (update) time. We also add a very brief descrip-tion of a class of grammar that can be identified in the limit using the Boisdale algo-rithm (so-called Boisdale grammars); Starkie and Fernau [10] contains a proof showing that all Boisdale grammars can be identified in the limit from positive data only using the Boisdale algorithm. Some experimental results will be presented including an empirical confirmation that the algorithm can infer any Boisdale grammar in the limit from positive data only. Although the Boisdale algorithm can infer any Boisdale grammar in the limit from positive data, there exist some sets of training examples for which no Boisdale grammar exists that can generate those sentences amongst others. In this instance, the Boisdale algorithm still terminates and returns a unification grammar that can generate at least the training examples. The exact characterization of the class of grammars that can be identified in the limit using the Boisdale algorithm is currently an open problem.2. Background2.1 NotationA unification grammar is given by a 5-tuple G =(N,Σ,Π,S,A) where N is the al-phabet of non-terminal symbols; Σ is the alphabet of terminal symbols with N∩Σ= { }; S is a special non-terminal called the start symbol; A is the signature definition which is an ordered list of key-type pairs (k,t) and Π is a finite set of rewrite rules ofthe form r = “Ni (x1..x|A|) →Ω1(x1..x|A|) … Ω|r|(x1..x|A|)” where Ni∈N,Ω i∈(Σ∪N).The rewrite rules of G define transformations of sequences of terms to other se-quences of terms. A term is comprised of a root and a signature. The root is a sym-bol and the signature is an ordered list of symbols. In our definition of unification grammars, all terms in the language have the same number of elements in their signature as there are elements in the signature definition. For example, the term “City(- ?fm)” has the root “City” and the signature “(- ?fm)”. In this paper we will use the symbol ‘-’ to denote an undefined value within a signature. A signature that contains only one or more instances of the symbol “-” within parentheses is referred to as the empty signature. For instance, the term “sydney(- -)” has the root “sydney" and the empty signature “(- -)”. The notation root(X) denotes the root of X.If a term does not explicitly show a signature, then it contains the empty signa-ture, i.e., we can write “sydney” in place of “sydney(- -)”.A term Ω is either a terminal term in which case root(Ω) ∈Σ and begins with a lower case letter, or a non-terminal term in which case root(Ω) ∈N and begins with an upper case letter. A terminal term always has the empty signature.In this paper an uppercase letter is used to represent any non-terminal symbol (e.g., A), a lowercase symbol to represent any terminal symbol (e.g., a) and a Greekletter is used to represent a symbol that could be either terminal or non-terminal (e.g.,Ω or Ψ). The notation A(x) represents a term with an unknown number of symbols in its signature and |x| denotes the length of the signature. An italic upper-case letter is used to represent a sequence of zero or more terminals or non-terminal terms (e.g., A) and an italic bold uppercase letter represents a sequence of one or more terms, either terminal or non-terminal (e.g., A). The notation |A| denotes the number of terms in A. Lowercase italic letters represent a sequence of zero or more terminal terms (e.g., a) and bold italic lowercase letters represent a sequence of one or more terminal terms (e.g., a).2.2 Unification GrammarsAn example unification grammar G that is described using the notation employed in this paper is given below. Relating to the formal definition introduced above, for this grammar the non-terminal alphabet is {S,City}, the terminal alphabet is {syd-ney,perth,perth scotland}and the signature definition is {(fm.fm),(to,fm)}.%slots {fm fm, to fm}%start SS(?fm ?to)→S(?fm -)S(- ?to)S(?fm -)→ from City(?fm -)S(- ?to)→ to City(?to -)City(SYD -)→sydneyCity(PTH -)→perthCity(SCT -)→perth scotlandFigure 1 An example unification grammarThe first part in this denotation is the signature definition. This states that all terms in the grammar have two parameters namely a “to” attribute and an “fm” attribute. Both of these attributes are of the same type, specifically the type “fm”. As well as defining the length of the signature (two) the signature definition enables a term signature to be converted into a key-value form. For instance, the signature (PTH SYD) can be seen as the attributes {fm=PTH, to=SYD}. The start symbol of G is the non-terminal ‘S’ and all sentences that are described by this grammar (denoted L(G)) can be generated by expanding the non-terminal S until no non-terminal terms exist in the term sequence. Non-terminals can be expanded using the rewrite rules. Symbols other than ‘-’ that appear in the signature on the left-hand side of the rule and in the signature of a symbol on the right hand side are variables. (Prefixed by ‘?’ in this paper). If a rule is well formed, then for every variable in the signature of the left hand side of a rewrite rule, there is exactly one instance of that variable contained in the signature of exactly one non-terminal on the right hand side of the rewrite rule. Similarly, for every variable in the signature of a non-terminal on the right hand side of rule, there is exactly one instance of that variable in the signature on the left hand side of the rewrite rule. For instance, the rule “S(?fm ?to)→S(?fm -)S(- ?to)” contains one instance of the variable “?fm” on the left hand side and ex-actly one instance of “?fm” on the right hand side.Before each non-terminal is expanded, all variables in the signature on the left-hand side of the rule need to be instantiated to a constant value via a process re-ferred to as unification . Two signatures are said to unify if there is a set of mappings σ: t → u for the variables in the signatures such that if you replace the variables with their mappings, the two signatures are identical. The notation L σ denotes the result of applying the substitution σ to L .For instance, in the above example the signature (“melbourne” “sydney”) unifies with (?fm ?to) using the mapping σ =(?fm →“melbourne”, ?to →“sydney”). That is,(?fm ?to)σ = (“melbourne” “sydney”). In contrast, the signature (“melbourne” “syd-ney”) does not unify with the signature (- ?to).An important method of expanding the start symbol into a terminal sequence is the rightmos t expans ion (denoted ⇒rm,G ) in which non-terminals are expanded right to left. For example:S(“melbourne”-)S(-“sydney”) ⇒rm,G S(“melbourne”-) to City(-“sydney”).The notation A ⇒*B denotes that A can be expanded using zero or more single step expansions to become B . A constituent is defined as an ordered pair (A , B ) where A ⇒* B . Similarly, a string can be converted to a data structure by a process referred to as reduction (denoted ⇐), where rewrite rules are applied in reverse. One impor-tant method of reduction is the leftmos t reduction (denoted L,G ⇐) in which reduc-tions are applied left to right. For example:to melbourne L,G ⇐ to City(“melbourne”-) L,G ⇐ S(-“melbourne”)Similarly, the notation B ⇐* A denotes that B can be transformed to become A via zero or more single step reductions.To formally describe a leftmost reduction we need to introduce a definition of uniquely inverted that can be applied to unification grammar. A unification gram-mar is uniquely inverted if there are no two rules of the form A(x)→B and C(y)→D such that root(B ) = root(D ). Here the function root is extended to a sequence of terms as follows: root(Ω1 Ω2 .. Ωx ) = root(Ω1) root(Ω2) .. root(Ωx ). Formally a leftmost reduction can be described as follows: if G is uniquely invertedand A B C L,G ⇐ A D(x) C then there exists a substitution ζ and a rule D(y) → Q such that Q ζ = B and D(y) ζ = D(x) and there does not exist any rule of the form E(x)→Fsuch that for some substitution ρ, A B C = H F ρ J ⇐ H E(x)ρ J and H F ρ is a proper prefix of A B . For any uniquely inverted unification grammar, the normal form achieved from a leftmost reduction is deterministic Otto [6]. It can be seen that if B L ⇐* N(x) then N(x) ⇒*rm,G B . To describe the Boisdale algorithm, some additional notation is required. If N(y) is a term, then val(y) denotes the unordered list of values (either constants or variables) in y other than ‘-’. Let pattern(y) be a sequence of 0’s and 1’s that denotes whether each element of the signature is either ‘-’ or not ‘-‘ respectively. Let const(y) denote the unordered list of constants in y; e.g., if y = (“melbourne” - ?action 6), then val(y) ={ “melbourne”, ?action, 6 }, pattern(y)=(1011) and const(y)={ “melbourne”, 6 }. The function val can be extended to terms and term sequences as follows: val(A(x))=val(x) and val(A(x) B ) = val(A(x)) ∪ val(B ). The functions const can be similarly expanded to reference terms and term sequences.2.3 The Typed Leftmost ReductionWe will now introduce the concept of the typed leftmost reduction . Given a set of rewrite rules, a starting term sequence and a set of constants, a typed leftmost reduc-tion defines a deterministic set of reductions. A typed leftmost reduction is similar to a leftmost reduction in that rewrite rules are applied in a left to right sequence with the following important distinction: the definition of a typed leftmost reduction includes a set of constants c such that a rule R can only be used if const(R) ⊆ c. This is reflected by the notation L(c),G ⇐.The function typed_leftmost(I ,G,c) shown inFigure 2 calculates the typed leftmost reduction B such that I L(c),G ⇐*B .Function typed_leftmost(I ,G,c)//I a sequence of terms,G a set of rewrite rules//c a set of constants{i=0;while(i < |I |){shift I [i] onto stack;i++;while(∃ A(x)→ B , ∃σ such that B σ = top |B | of stack,const (x)⊆ c){ pop |B | symbols off stack;push A(x)σ onto stack;}}Figure 2 An algorithm for calculating a typed leftmost reductionFormally, given a target set of constants c, a uniquely inverted set of rewrite rulesG and a starting sequence I a typed leftmost reduction denoted I L(c),G ⇐* B is a se-quence of reduction steps such that if I L(c)⇐* A D C L(c) ⇐ A F(x) C L(c) ⇐* B then there exists a substitution ζ and a rule F(x)ζ → D ζ such that const(x) ⊆ c and theredoes not exist any rule of the form E(y)→F and a substitution σ such that A D C = HF σ J ⇐ H E(y)σ J L ⇐* B and H F σ is a proper prefix of A D , const(y) ⊆ c. Although the definition of a typed leftmost reduction is stated in terms of auniquely inverted grammar, we will extend the definition to enable it to be used as part of the inference process as follows: If two rules R 1 and R 2 can be applied at any point in a typed leftmost reduction sequence then if |const(R 1)| > |const(R 2)|, the sequence is reduced by R 1; otherwise, the sequence is reduced by R 2.2.4 The Working Principle of the Boisdale AlgorithmIt can be seen that for all unification grammars, if B L(c),G ⇐*A(c) then A(c)⇒*rm,G B . The Boisdale algorithm has been designed to infer grammars where for each rule there exists at least one terminal sequence, whose typed leftmost reduction recreates a rightmost derivation in reverse, i.e., for each rule of the form A(d)→E there existsat least one terminal string b such that A(c) ⇒rm,G E σ ⇒*rm,G b and b L(c),G ⇐*E σL(c),G ⇐A(c). The Boisdale algorithm takes as its input a set of positive training examples each tagged with key value pairs representing the meaning of those sentences. The con-stituents of these sentences are then guessed by aligning sentences with common prefixes from the left. In Starkie and Fernau [10] it is proven that a class of grammar exists such that aligning sentences from the left either identifies the correct constitu-ents of the sentence, or if it incorrectly identifies constituents then those constituents will be deleted by the time the algorithm has completed.The Boisdale algorithm is believed to be one example of a class of alignmentbased learning algorithms that can identify classes of unification grammar in the limit from positive data only. A related algorithm could be constructed to infer grammars for which all sentences can be reconstructed using a typed leftmost reduc-tion, i.e.,∀A(c) ⇒rm,G E⇒*rm,G b, b L(c),G⇐* E L(c),G⇐A(c).Similarly the concept of a typed rightmost reduction can be introduced that is identical to a typed leftmost reduction with the exception that reductions occur in a right to left manner. This variant of course gives rise to another class of learnable languages.3. The Boisdale AlgorithmThe Boisdale algorithm begins with a set T of positive examples of the language L(G) where each s∈ T can include a set of key value pairs (denoted attributes(s)) that describe the semantics of s. Although this algorithm uses unstructured key value pairs to describe the semantics of sentences, arbitrarily complex data structures can be mapped into assignment functions and therefore simple key-value pairs as de-scribed in Starkie [8]. (eg date.hours= “4” date.minutes=“30”).The algorithm creates a set of constituents C and hypothesis grammar H with rule set Π. The algorithm is comprised of the following 7 steps.Step 1. (Incorporation Phase)For each sentence s ∈ T, attributes(s) =x a rewrite rule of the form S(x) →s is added to H.Step 2. (Alignment Phase)Rule 1. If there exists two sentences c x1and c x2with a common prefix c and withattributes y1 and y2, respectively, for which the same attribute keys are defined, i.e.,pattern(y1)=pattern(y2), then a new non-terminal X1is introduced and two rules ofthe form X1(y5) →x1and X1(y6) →x2are created. The signatures of these rules (y5and y6) are constructed such that val(y5) = val(y1) – val(y2) and val(y6) = val(y2) –val(y1).Example: When presented with the sentences “from sydney"{fm=SYD} and "from perth" {fm=PTH}, the non-terminal X48is constructed and the rulesX48(SYD -)→sydney and X48(PTH -)→perthare added to the hypothesis grammar.Rule 2. If there exists a sentence c x7 that is a prefix of another sentence c x7x8withattributes y7 and y8, respectively, for which the same attribute keys are defined, i.e.,pattern(y7)=pattern(y8) such that there exists at least one key value pair in y7that isnot in y8 then a new non-terminal X7is created and two rules of the form X7(y10) →x 7 and X7(y11) →x7x8are constructed.Example: When presented with the sentences "from perth" {fm=PTH} and "fromperth scotland" {fm=SCT} the non-terminal X48 is formed and the rules “X48(PTH-)→perth” and “X48(SCT -)→perth scotland” are constructed.At this point the right hand side all of the rewrite rules of the hypothesis grammar contain only terminals. The set of constituents C is then created by copying the hypothesis grammar H, i.e., C ={ (A(x),B ) | “A(x)→B ”∈H }. Continue to step 3. Step 3. (Substitution Phase)The substitution phase consists of two sub phases: normalisation and completion. In both subphases, merging of non-terminals may occur. Merging non-terminals in the Boisdale algorithm involves the relabelling of some non-terminals as well as the reordering of the signatures of non-terminals. A reordering r is an array of length l where l is the number of attributes per term in the grammar. Each element of r (denoted r i ) is set to either ‘-’ or an integer k such that k < l . When a signature s isreordered using a reordering r, a signature u is created such that u i = ‘-’ if r i = ‘-’, otherwise u i = s k where k= r i . After u has been constructed s is replaced by u. We will use the notation y |-|ς z to denote that y is reordered using the reordering ς to be-come z.Example: The signature (- SYD) can be reordered using the reordering (1 -) to be-come (SYD –).When the non-terminals A and C are merged using the reordering r, all signatures attached to A are reordered using r. A well-founded linear ordering <r (not defined here) is used to determine the name of the newly merged non-terminal. Specifically, if A<r C then all instances of C are renamed as A, otherwise all instances of A are renamed as C. For any non-terminal N i other than ‘S’ ‘S’<r N i . This ensures that H always contains at least one rule with the start symbol on its left hand side. Example: If a grammar contains the rulesX 1(- SYD)→sydney and X 2(PTH -)→perthand the non-terminal X 2 is merged with X 1 using the reordering (- 1) then aftermerging these rules become; X 1(- SYD)→sydney and X 1(- PTH)→perth. Normalisation.During normalization for each rule of the form N i (y)→E in H the right hand side E is reduced using a typed leftmost reduction and y (i.e., using the function of Figure2.) That is the value K is calculated where E L(y) ⇐* K.a) I f N i ≠ ‘S’ and a term sequence is encountered while reducing E that beginswith N i , then the rule N i (y)→E is deleted.b) O therwise if K ≠ N i (y) and |K |≠1 and each constant in const(K ) appears the samenumber of times in const(y), then the rule N i (y)→E is replaced by the ruleN i (y)→K.c) I f K ≠ N i (y) and |K |=1 and each constant in const(K ) appears the same number of times in const(y), then the non-terminal K is merged with N i in both H and C. This process is repeated until ∀ r = (A(x)→B ) ∈ Π, B L(x) ⇐* A(x). Example: If H contains only the following rulesS(SYD -)→from sydney and X 48(SYD -)→sydney then after normalisation (part c)) these rules will becomeS(?fm -)→from X 48(?fm -) and X 48(SYD -)→sydney. Completion.During completion the right hand side of each constituent in C is reduced using a typed leftmost reduction. Specifically for all c=(A(x),B ) ∈ C the value D is calcu-lated such that B L(x) ⇐* D .a) I f A ≠ ‘S’ and a term sequence is encountered while reducing B that begins withA the constituent (A(x),B ) is deleted.b) O therwise if D ≠ A(x) and each constant in const(D ) appears the same number of times in const(x) then a rule of the form A(x)→ D is added to H.c) I f D ≠ A(x) and |D |=1 and each constant in const(D ) appears the same number of times in const(x) then the non-terminal D is merged with A in both H and C. This process is repeated until no more rules are added to H.Example: If H contains only the following ruleX 48(SYD -)→sydney and C contains the following constituents(S(SYD -),from sydney) and (X 48(SYD -),sydney) then after completion the rules of H will beS(?fm -)→from X 48(?fm -) and X 48(SYD -)→sydney. In the substitution phase, normalisation is first performed followed by completion. If the grammar has changed, the process is repeated, beginning with normalization; otherwise, the algorithm continues to the merging phase.Step 4. (Merging Phase)During the merging phase, non-terminals are merged by applying the following rules:• If there exists two rules of the form A(x) → B C(y) and A(q) → E D(z) that can be unified using the mapping σ so that the right hand sides are identical (i.e., B =E σ) with the exception of the last non-terminal on the right hand side of each rule, and the signatures of the last non-terminal can be reordered using the reordering ς so that they unify, i.e., y |-|ς z σ then C and D are merged using ς.Example: If there are two rules of the formX 1(?fm ?to) → from City(?fm -) to X 2(- ?to) andX 1(?fm ?to) → from City(?fm -) to City(?to -), then the non-terminals X 2 and City are merged. • If there exists two rules of the form A(x) → B and C(y) → E where the same right hand side of each rule can be unified using some mapping σ (i.e., B =E σ) and the signatures of the left hand sides can be reordered using some reordering ς, i.e., x |-|ς y σ then A and C are merged using ς.Example: If there are two rules of the formX 2(sydney-) → sydney andCity(sydney -) → sydney,then the non-terminals X 2 and City are merged. • If there exists two rules of the form A(x) → Ω1Ω2 …Ωn and A(y) → Ψ1Ψ2…Ψn and ∀i root(Ωi )= root(Ψi ) and there exists a σ1 such that A(y)σ1 = A(x), (Ψ1Ψ2…Ψn ) σ1 = (Ω1Ω2 …Ωn ) but there does not exist a σ2 such that A(x)σ2 = A(y), (Ω1Ω2 …Ωn ) σ2 = (Ψ1Ψ2…Ψn ), then the rule “A(x) ⇒* Ω1Ω2 …Ωn ” is deleted. Example: If a grammar contains two rules of the formS(?fm ?to) → from City(?fm –) to City(- ?to) andS(?fm ?fm) → from City(?fm –) to City(- ?fm),then the rule “S(?fm ?fm) → from City(?fm –) to City(- ?fm)” is deleted. This is because every sentence that can be generated by this rule can be generated from the other rule, although the converse is not true.The merging phase continues until no more non-terminals can be merged. Once the merging phase has completed, if the number of rules or the number of non-terminals has changed, then the substitution phase is repeated; otherwise, the algorithm may continue to the unchunking phase.Step 5. (Unchunking phase)If there exists a non-terminal A in the grammar where A is not the start symbol such that there is only one rule of the form A(x) →B, then for all instances of A(y)on the right hand side of any other rule R1,i.e., R1= C(z) →D A(y) E a substitutionσ is found such that A(x)σ = A(y) and R1 is transformed to become C(z) →D BσE . The rule A(x) →B is then deleted, as well as all constituents of the form (A(y),F). Example: If there are two rules of the formX48(?fm -)→perth X49(?fm -) andX49(SCT -)→scotlandand there are no other rules with X49 on their left hand side then these rules are re-placed by the following rule:X48(SCT -)→perth scotland.The unchunking phase is continued until no-more changes can be made. If the grammar has changed during the unchunking phase then the algorithm returns to the substitution phase otherwise it continues to the catch all stage.Step 6. (Catch all stage)In this stage of the algorithm, rules are added to ensure that every sentence in the training set can be generated using the hypothesis grammar. Specifically, an emptyset of rewrite rules H2 is first created. Then if sL(x)⇐*D and D ≠ S(x), then s isparsed using a chart parser Allen [2]. If S(x)⇒*s then no action is taken otherwisethe rule S(x)→D is added to H2. At the end of the catch all stage, all rules in H2areadded to H.Step 7. (Final stage)In the final stage, rules that are unreachable from the start symbol are deleted. An algorithm for doing this can be found in Aho and Ullman [1].4. ExampleConsider the case where the Boisdale grammar is presented with the following set of training examples:{"from sydney"{ fm=SYD},"from perth"{fm=PTH},"to sydney"{to=SYD},"from perth scotland" {fm=SCT},"from perth to sydney"{fm=PTH to=SYD}}.At the completion of the alignment phase the hypothesis grammar is as follows: S(SYD -)→from sydneyS(PTH -)→from perthS(- SYD)→to sydneyS(SCT -)→from perth scotlandS(PTH SYD)→from perth to sydneyX48(SYD -)→sydneyX48(SCT -)→perth scotlandX48(PTH -)→perthX49(SCT -)→scotlandX50(PTH SYD)→perth to sydneyX51(PTH SYD)→to sydneyAt the beginning of the substitution phase, the rule “S(SYD -)→from sydney” is examined. The right hand side of this rule is reduced using a typed leftmost reduc-tion as follows from sydney L({SYD})⇐* from X48(SYD-). Because the resulting term sequence is not S(SYD-) the rule “S(SYD-)→from X48(SYD-)” is constructed. The right hand side of a rewrite rule cannot contain a constant in a signature therefore the value SYD is replaced by the variable ?fm resulting in the rule “S(?fm-)→from X48(?fm-)”. This rule is well-formed therefore it is added to the hypothesis grammar and the rule“S(SYD-) →from sydney” is deleted. The remainder of the rules are similarly examined. At the end of normalisation sub phase all constituents can be parsed using a typed leftmost reduction therefore no rules are added during the completion sub phase. At the completion of the substitution phase, the grammar is as follows: S(?fm -)→from X48(?fm -)S(?fm ?to)→S(?fm -)S(- ?to)S(- ?to)→ to X48(?to -)X48(SYD -)→sydneyX48(PTH -)→perthX48(?fm -)→perth X49(?fm -)X49(SCT -)→scotlandX50(?fm ?to)→X48(?fm -)S(- ?to).No non-terminals are merged during the merging phase. During the unchunkingphase the non-terminal X49 is unchunked. The algorithm then re-enters the substitu-tion phase, because the hypothesis grammar has changed. The algorithm then transi-tions through the merging, unchunking and catch all stage, without modifying thehypothesis grammar. During the final stage, the non-terminal X50 is deleted, and thealgorithm returns the grammar shown in Figure 1.5. Identification in the LimitA description of a class of unification grammar called Boisdale grammars can be found in Starkie and Fernau [10] along with a proof sketch that the Boisdale algo-rithm can infer the class of Boisdale grammars in the limit from positive data only. The report Starkie and Fernau [10] also contains a proof that the algorithm can exe-cute in polynomial update time with respect to the size of the training data. A basic ingredient of the identification proof is the following concept.Definition.Let C be a set of constituents. A unification grammar G minimally parses C if • the right hand side B of each constituent c i =(A(x),B) ∈ C can be reduced tothe left hand side A(x) of c using a typed leftmost reduction, i.e.,∀(A(x),B) BL(x),G⇐*A(x) and。