Token-oriented representation of program code with support for textual editing thereof

摘要

An editor, software engineering tool or collection of such tools may be configured to encode (or employ an encoding of) an insertion point representation that identifies both a particular token of a token-oriented representation and a character offset thereinto. Efficient implementations of insert, remove and replace operations that employ such a representation are described herein. Computational costs of such operations typically scale at worst with the size of fragments inserted into and/or removed from such a token-oriented representation, rather than with buffer size. Accordingly, such implementations are particularly well-suited to providing efficient support for programming tool environments in which a token stream is updated incrementally in correspondence with user edits.