Add more expressions to parser, add EOL and newline tokens

* Lexer recognizes semicolons as EOL tokens, and newline tokens
* Parser can be configured to ignore newlines (which is also used
  internally)
* Newlines are allowed in lists, tuples, and parenthesized expressions
* Add a bunch of tests for the new stuff

Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
This commit is contained in:
2020-05-05 16:38:07 -04:00
parent d9edf21d16
commit a4f289fb53
6 changed files with 274 additions and 22 deletions

View File

@@ -120,6 +120,7 @@ impl Spanned for Span {
}
}
/*
pub struct Sourced<'t, T: Spanned> {
text: &'t str,
inner: T,
@@ -136,6 +137,7 @@ impl<T: Spanned> Spanned for Sourced<'_, T> {
self.inner.span()
}
}
*/
#[cfg(test)]
mod test {