Next: , Previous: , Up: Recognizer methods   [Contents]


12.4 Life cycle mutators

Function: int marpa_r_start_input (Marpa_Recognizer r)

Makes r ready to accept input. The first Earley set, the one at earleme 0, will be completed during this call.

Because the call to marpa_r_start_input() completes an Earley set, it may generate events. For details about the events that may be generated during Earley set completion, see the description of the marpa_r_earleme_complete() method.

Return value: On success, a non-negative value. On failure, -2.

Function: int marpa_r_alternative (Marpa_Recognizer r, Marpa_Symbol_ID token_id, int value, int length)

Reads a token into r. The token will start at the current earleme. Libmarpa allows tokens to be ambiguous, to be of variable length and to overlap. token_id is the symbol ID of the token, which must be a terminal. length is the length of the token.

value is an integer that represents the value of the token. In applications where the token’s actual value is not an integer, it is expected that the application will use this value as a “virtual” value, perhaps finding the actual value by using value to index an array. value is not used inside Libmarpa — it is simply stored to be returned by the valuator as a convenience for the application. Some applications may prefer to track token values on their own, perhaps based on the earleme location and token_id, instead of using Libmarpa’s token values.

A value of 0 is reserved for a now-deprecated feature. Do not use it. For more details on that feature, see the section Valued and unvalued symbols.

When marpa_r_alternative() is successful, the value of the furthest earleme is set to the greater of its value before the call, and current+length, where current is the value of the current earleme. The values of the current and latest earlemes are unchanged by calls to marpa_r_alternative().

Several error codes leave the recognizer in a fully recoverable state, allowing the application to retry the marpa_r_alternative() method. Retry is efficient, and quite useable as a parsing technique. The error code of primary interest from this point of view is MARPA_ERR_UNEXPECTED_TOKEN_ID, which indicates that the token was not accepted because of its token ID. Retry after this condition is used in several applications, and is called “the Ruby Slippers technique”.

The error codes MARPA_ERR_DUPLICATE_TOKEN, MARPA_ERR_NO_TOKEN_EXPECTED_HERE and MARPA_ERR_INACCESSIBLE_TOKEN also leave the recognizer in a fully recoverable state, and may also be useable for the Ruby Slippers or similar techniques. At this writing, the author knows of no applications which attempt to recover from these errors.

Return value: On success, MARPA_ERR_NONE. On failure, some other error code.

Function: int marpa_r_earleme_complete (Marpa_Recognizer r)

This method does the final processing for the current earleme. It then advances the current earleme by one. Note that marpa_r_earleme_complete() may be called even when no tokens have been read at the current earleme — in the character-per-earleme input model, for example, tokens can span many characters and, if the input is unambiguous over that span, there will be no other tokens that start inside it.

As mentioned, marpa_r_earleme_complete() always advances the current earleme, incrementing its value by 1. This means that value of the current earleme after the call will be the one plus the value of the earleme processed by the call to marpa_r_earleme_complete(). If any token was accepted at the earleme being processed, marpa_r_earleme_complete() creates a new Earley set which will be the latest Earley set and, after the call, the latest earleme will be equal to the new current earleme. If no token was accepted at the earleme being processed, no Earley set is created, and the value of the latest earleme remains unchanged. The value of the furthest earleme is never changed by a call to marpa_r_earleme_complete().

During this method, one or more events may occur. On success, this function returns the number of events generated, but it is important to note that events may be created whether earleme completion fails or succeeds. When this method fails, the application must call marpa_g_event() if it wants to determine if any events occurred. Since the reason for failure to complete an earleme is often detailed in the events, applications that fail will often be at least as interested in the events as those that succeed.

The MARPA_EVENT_EARLEY_ITEM_THRESHOLD event indicates that an application-settable threshold on the number of Earley items has been reached or exceeded. What this means depends on the application, but when the default threshold is exceeded, it means that it is very likely that the time and space resources consumed by the parse will prove excessive.

A parse is “exhausted” when it can accept no more input. This can happen both on success and on failure. Note that the failure due to parse exhaustion only means failure at the current earleme. There may be successful parses at earlier earlemes.

If a parse is exhausted, but successful, an event with the event code MARPA_EVENT_EXHAUSTED occurs. Because the parse is exhausted, no input will be accepted at later earlemes. It is quite common for a parse to become exhausted when it succeeds. Many practical grammars are designed so that a successful parse cannot be extended.

An exhausted parse may cause a failure, in which case marpa_r_earleme_complete() returns an error whose error code is MARPA_ERR_PARSE_EXHAUSTED. For a parse to fail at an earleme due to exhaustion, it must be the case that no alternatives were accepted at that earleme. In fact, in the standard input model, a failure due to parse exhaustion occurs if and only if no alternatives were accepted at the current earleme.

The circumstances under which failure due to parse exhaustion occurs are slightly more complicated when variable length tokens are in use. Informally, a parse will never fail due to exhaustion as long as it is possible that a token ending at some future earleme will continue the parse. More precisely, a call to marpa_r_earleme_complete() fails due to parse exhaustion if and only if, first, no alternatives were added at the current earleme and, second, that call left the current earleme equal to the furthest earleme.

Return value: On success, the number of events generated. On failure, -2.


Next: , Previous: , Up: Recognizer methods   [Contents]