Skip to main content

Module parse2

Module parse2 

Source
Expand description

New netdoc parsing arrangements, with derive

§Parsing principles

A parseable network document is a type implementing NetdocParseable. usually via the NetdocParseable derive=deftly macro`.

A document type is responsible for recognising its own heading item. Its parser will also be told other of structural items that it should not consume. The structural lines can then be used to pass control to the appropriate parser.

A “structural item” is a netdoc item that is defines the structure of the document. This includes the intro items for whole documents, the items that introduce document sections (which we model by treating the section as a sub-document) and signature items (which introduce the signatures at the end of the document, and after which no non-signature items may appear).

§Ordering

We don’t always parse things into a sorted order. Sorting will be done when assembling documents, before outputting.

§Types, and signature handling

Most top-level network documents are signed somehow. In this case there are three types:

  • FooUnverified: a signed Foo, with its signatures, not yet verified. Implements NetdocParseableUnverified, typically by invoking the NetdocUParseablenverified derive macro on Foo.

    Type-specific methods are provided for verification, to obtain a Foo.

  • Foo: the body data for the document. This doesn’t contain any signatures. Having one of these to play with means signatures have already been validated. Can be parsed as part of the signed document, via the NetdocParseable implementation on FooUnverified, and then obtained via .verify_... method(s) on FooUnverified,

  • FooSignatures: the signatures for a Foo. Implements NetdocParseableSignatures, via derive, with #[deftly(netdoc(signatures))].

§Relationship to tor_netdoc::parse

This is a completely new parsing approach, based on different principles. The key principle is the recognition of “structural keywords”, recursively within a parsing stack, via the pNetdocParseable] trait.

This allows the parser to be derived. We have type-driven parsing of whole Documents, Items, and their Arguments and Objects, including of their multiplicity.

The different keyword handling means we can’t use most of the existing lexer, and need new item parsing API:

The different error handling means we have our own error types. (The crate’s existing parse errors have information that we don’t track, and is also a portmanteau error for parsing, writing, and other functions.)

Document signing is handled in a more abstract way.

Some old netdoc constructs are not supported. For example, the obsolete opt prefix on safe-to-ignore Items. The parser may make different decisions about netdocs with anomalous item ordering.

Re-exports§

pub use error::ArgumentError;
pub use error::ErrorProblem;
pub use error::ParseError;
pub use error::UnexpectedArgument;
pub use error::VerifyFailed;
pub use impls::times::NdaSystemTimeDeprecatedSyntax;
pub use keyword::KeywordRef;
pub use lex::ArgumentStream;
pub use lex::ItemStream;
pub use lex::NoFurtherArguments;
pub use lex::UnparsedItem;
pub use lex::UnparsedObject;
pub use lines::Lines;
pub use lines::Peeked;
pub use lines::StrExt;
pub use signatures::HasUnverifiedParsedBody;
pub use signatures::NetdocParseableSignatures;
pub use signatures::NetdocParseableUnverified;
pub use signatures::SignatureHashInputs;
pub use signatures::SignatureHashesAccumulator;
pub use signatures::SignatureItemParseable;
pub use signatures::SignaturesData;
pub use signatures::sig_hashes;
pub use signatures::check_validity_time;Deprecated
pub use signatures::check_validity_time_tolerance;Deprecated
pub use structural::StopAt;
pub use structural::StopPredicate;
pub use traits::IsStructural;
pub use traits::ItemArgumentParseable;
pub use traits::ItemObjectParseable;
pub use traits::ItemValueParseable;
pub use traits::NetdocParseable;
pub use traits::NetdocParseableFields;
pub(crate) use internal_prelude::EP; 🔒

Modules§

derive 🔒
Deriving NetdocParseable
error 🔒
Parsing errors
impls 🔒
Implementations of our useful traits, on external and parsing-mode types
keyword
Keywords in netdocs
lex 🔒
Lexing of netdoc elements
lines 🔒
Version of std::str::Lines that tracks line numbers and has remainder()
multiplicity
Multiplicity of fields (Items and Arguments)
signatures 🔒
Handling of netdoc signatures
structural 🔒
Structural keyword recognition helpers
traits 🔒
Core model for netdoc parsing

Structs§

OutOfBoundsBug 🔒
Error from multi_push_doc
ParseInput
Input to a network document top-level parsing operation
ParseOptions
Options for parsing

Functions§

error_handler 🔒
Return a function for converting ErrorProblem to ParseError
multi_push_doc 🔒
Add (doc, start_pos, end_pos) to docs, checking bounds
parse_internal 🔒
Common code for parse_netdoc and parse_netdoc_multiple
parse_netdoc
Parse a network document - toplevel entrypoint
parse_netdoc_multiple
Parse multiple concatenated network documents - toplevel entrypoint
parse_netdoc_multiple_sophisticated
Parse multiple network documents, with error recovery - toplevel entrypoint
parse_netdoc_multiple_with_offsets
Parse multiple network documents, also returning their offsets - toplevel entrypoint