Makes r ready to accept input. The first Earley set, the one at earleme 0, will be completed during this call.
Because the call to
completes an Earley set,
it may generate events.
For details about
the events that may be generated during
Earley set completion,
see the description of the
Return value: On success, a non-negative value. On failure, -2.
Reads a token into r. The token will start at the current earleme. Libmarpa allows tokens to be ambiguous, to be of variable length and to overlap. token_id is the symbol ID of the token, which must be a terminal. length is the length of the token.
value is an integer that represents the value of the token. In applications where the token’s actual value is not an integer, it is expected that the application will use this value as a “virtual” value, perhaps finding the actual value by using value to index an array. value is not used inside Libmarpa — it is simply stored to be returned by the valuator as a convenience for the application. Some applications may prefer to track token values on their own, perhaps based on the earleme location and token_id, instead of using Libmarpa’s token values.
A value of 0 is reserved for a now-deprecated feature. Do not use it. For more details on that feature, see the section Valued and unvalued symbols.
the value of the furthest earleme is set to
the greater of its value before the call,
where current is the value of the current earleme.
The values of the current and latest earlemes
are unchanged by
Several error codes leave the recognizer in a fully
recoverable state, allowing the application to
Retry is efficient, and quite useable as a parsing
The error code
of primary interest from this point of view
which indicates that the token was not accepted
because of its token ID.
Retry after this condition is used in several
and is called “the Ruby Slippers technique”.
The error codes
also leave the recognizer in a fully recoverable
state, and may also be useable for the
Ruby Slippers or similar techniques.
At this writing,
the author knows of no applications which
attempt to recover from these errors.
Return value: On success,
On failure, some other error code.
This method does the final processing for the current earleme.
It then advances the current earleme by one.
marpa_r_earleme_complete() may be called
even when no tokens have been read at the current earleme —
in the character-per-earleme input model, for example, tokens
can span many characters and, if the input is unambiguous over that
span, there will be no other tokens that start inside it.
marpa_r_earleme_complete() always advances the current earleme,
incrementing its value by 1.
This means that value of the current earleme after the call
will be the one plus the value of the earleme processed by the call
If any token was accepted at the earleme being processed,
marpa_r_earleme_complete() creates a new Earley set
which will be the latest Earley set
and, after the call, the latest
earleme will be equal to the new current earleme.
If no token was accepted at the
earleme being processed,
no Earley set is created,
and the value of the latest earleme remains unchanged.
The value of the furthest earleme is never changed by
a call to
During this method, one or more events may occur.
On success, this function returns the number of events
but it is important to note that events may be
created whether earleme completion fails or succeeds.
When this method fails,
the application must call
if it wants to determine if any events occurred.
Since the reason for failure to complete an earleme is often
detailed in the events, applications that fail will often
be at least as interested in the events as those
indicates that an application-settable threshold
on the number of Earley items has been reached or exceeded.
What this means depends on the application,
but when the default threshold is exceeded,
it means that it is very likely
that the time and space resources consumed by
the parse will prove excessive.
A parse is “exhausted” when it can accept no more input. This can happen both on success and on failure. Note that the failure due to parse exhaustion only means failure at the current earleme. There may be successful parses at earlier earlemes.
If a parse is exhausted, but successful,
an event with the event code
Because the parse is exhausted,
no input will be accepted at later earlemes.
It is quite common for a parse to become exhausted
when it succeeds.
Many practical grammars are designed so that a
successful parse cannot be extended.
An exhausted parse may cause a failure,
in which case
marpa_r_earleme_complete() returns an
whose error code is
For a parse to fail at an earleme due to exhaustion,
it must be the case that no alternatives were
accepted at that earleme.
in the standard input model,
a failure due to parse exhaustion
occurs if and only if
were accepted at the current earleme.
The circumstances under which
failure due to parse exhaustion occurs
are slightly more complicated
when variable length tokens are in use.
a parse will never fail due to exhaustion as long
as it is possible that a token
ending at some future
earleme will continue the parse.
a call to
due to parse exhaustion
if and only if, first,
no alternatives were added at the current earleme
that call left the current
earleme equal to the furthest earleme.
Return value: On success, the number of events generated. On failure, -2.