2024-09-20 16:04:30 -07:00
|
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
|
|
|
|
|
pub enum TokenKind {
|
|
|
|
|
// Keywords
|
|
|
|
|
Return,
|
|
|
|
|
If,
|
|
|
|
|
Else,
|
|
|
|
|
True,
|
|
|
|
|
False,
|
|
|
|
|
Nil,
|
WIP: Add imports and modules
This is a big change because it touches a lot of stuff, but here is the
overview:
* Import syntax:
```
import foo
import bar from foo
import bar from "foo.npp"
import bar, baz from foo
import * from foo
import "foo.npp"
```
* These are all valid imports. They should be pretty
straightforward, maybe with exception of the last item. If you are
importing a path directly, but not importing any members from it,
it does not insert anything into the current namespace, and just
executes the file. This is probably going to be unused but I want
to include it for completeness. We can always remove it later
before a hypothetical 1.0 release.
* The "from" keyword is only ever used as a keyword here, and I am
allowing it to be used as an identifier elsewhere. Don't export
it, because that's weird and wrong and won't work.
* Modules:
* Doing an `import foo` will look for "foo.npp" at compile-time,
relative to the importer's directory, parse it, and compile it.
The importer will then attempt to execute the module with the new
`EnterModule` op. This instruction will execute the module kind of
like a function, assigning the module's global namespace to an
object that you can pass around.
* `import bar from foo` and `import bar from "foo.npp"` et al syntax
is not currently implemented in the compiler.
* There is a new "Module" object that represents a potentially
un-initialized module. This can't be referred to directly in code.
* VM:
* The VM operates around Module objects now. If you want to "call" a
new module, you should call `enter_module`. This is how the main
chunk is invoked.
* TODOs:
* `exit_module` function in the VM
* Finish up module implementation in compiler
* Built-in modules
* Sub-modules - e.g. `import foo.bar` - how does naming work for
this?
* Module directories. In Python you have `foo/__init__.py` and in
Rust you have `foo/mod.rs`.
* Probably a "Namespace" object that explicitly denotes "this is an
imported module that you're dealing with"
* Tests, tests, tests
Signed-off-by: Alek Ratzloff <alekratz@gmail.com>
2024-10-04 10:11:49 -07:00
|
|
|
Import,
|
|
|
|
|
From,
|
2024-09-20 16:04:30 -07:00
|
|
|
|
|
|
|
|
// Expressions
|
|
|
|
|
Name,
|
|
|
|
|
Number,
|
|
|
|
|
String,
|
|
|
|
|
|
|
|
|
|
// Binary operators
|
|
|
|
|
Plus,
|
|
|
|
|
Minus,
|
|
|
|
|
Star,
|
|
|
|
|
Slash,
|
|
|
|
|
|
|
|
|
|
// Unary operators (not already covered)
|
|
|
|
|
Bang,
|
|
|
|
|
|
|
|
|
|
// Boolean operators
|
|
|
|
|
And,
|
|
|
|
|
Or,
|
|
|
|
|
|
|
|
|
|
// Comparison
|
|
|
|
|
BangEq,
|
|
|
|
|
EqEq,
|
|
|
|
|
Greater,
|
|
|
|
|
GreaterEq,
|
|
|
|
|
Less,
|
|
|
|
|
LessEq,
|
|
|
|
|
|
|
|
|
|
// Braces, parens, etc
|
|
|
|
|
LParen,
|
|
|
|
|
RParen,
|
|
|
|
|
LBrace,
|
|
|
|
|
RBrace,
|
|
|
|
|
LBracket,
|
|
|
|
|
RBracket,
|
|
|
|
|
|
|
|
|
|
// Assignment
|
|
|
|
|
Eq,
|
|
|
|
|
|
|
|
|
|
// Dot, comma
|
|
|
|
|
Dot,
|
|
|
|
|
Comma,
|
|
|
|
|
Arrow,
|
|
|
|
|
Colon,
|
|
|
|
|
|
|
|
|
|
// Line end
|
|
|
|
|
Eol,
|
|
|
|
|
|
|
|
|
|
// File end
|
|
|
|
|
Eof,
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#[derive(Debug, Clone)]
|
|
|
|
|
pub struct Token {
|
|
|
|
|
pub line: usize,
|
|
|
|
|
//pub index: usize,
|
|
|
|
|
pub text: String,
|
|
|
|
|
pub kind: TokenKind,
|
|
|
|
|
}
|