summaryrefslogtreecommitdiff
path: root/src/lexer.mll
AgeCommit message (Collapse)Author
2020-09-29Refactor: Change AST type from a union to a structAlasdair
2020-09-21Strip leading *'s from saildocJessica Clarke
We now parse /*! * Paragraph */ and /*! *Paragraph */ the same as /*! Paragraph */ since the first form is prettier, and similar to what Doxygen, Javadoc and other such tools allow. This can cause mild confusion, as if the start of a line in the final form happens to have a * then it will unexpectedly remove it, but this is a problem shared by those tools too and the intent is that everyone just use the first form and never need to worry about it.
2020-08-13Preserve file structure through initial checkAlasdair
Insert $file_start and $file_end pragmas in the AST, as well as $include_start and $include_end pragmas so we can reconstruct the original file structure later if needed, provided nothing like topological sorting has been done. Have the Lexer produce a list of comments whenever it parses a file, which can the be attached to the nearest nodes in the abstract syntax tree.
2020-05-15Add coverage tracking toolAlasdair
See sailcov/README.md for a short description Fix many location info bugs discovered by eyeballing output
2019-05-14Add feature that allows functions to require type variables are constantAlasdair Armstrong
can now write e.g. forall (constant 'n : Int) rather than forall ('n: Int) which requires 'n to be a constant integer value whenever the function is called. I added this to the 'addrsize variable on memory reads/writes to absolutely guarantee in the SMT generation that we don't have to worry about the address being a variable length bitvector.
2019-02-08Allow internal AST nodes in input when -dmagic_hash is onBrian Campbell
2019-01-14Merge remote-tracking branch 'origin/sail2' into asl_flow2Alasdair
2019-01-08Improvements for v85Alasdair Armstrong
2018-12-29Add separate termination_measure declarationsBrian Campbell
2018-12-26More cleanupAlasdair Armstrong
Remove unused name schemes and DEF_kind
2018-12-07Working on better flow typing for ASLAlasdair Armstrong
On a new branch because it's completely broken everything for now
2018-12-06Desugar constraints from atyp rather than n_constraintAlasdair Armstrong
Previously the valid constraints had to be carefully restricted to avoid parser ambiguities between n_constraint and atyp. With the initial check refactored, we can now parse constraints into atyp using ATyp_app for the operators, and changing ATyp_constant into a more general ATyp_lit for true and false. Logically this new structure is more uniform, as atyp is now the parse representation for all Bool-kinded things (constraints), Type-kinded things (regular types), and Int-kinded things (n-expressions), and initial_check.ml now splits all three into n_constraint, typ, and nexp respectively, rather than how it was before with initial_check splitting types and nexps, but constraints already being separate in the parser.
2018-11-30Parser tweaks and fixesAlasdair Armstrong
- Completely remove the nexp = nexp syntax in favour of nexp == nexp. All our existing specs have already switched over. As part of this fix every test that used the old syntax, and update the generated aarch64 specs - Remove the `type when constraint` syntax. It just makes changing the parser in any way really awkward. - Change the syntax for declaring new types with multiple type parameters from: type foo('a : Type) ('n : Int), constraint = ... to type foo('a: Type, 'n: Int), constraint = ... This makes type declarations mimic function declarations, and makes the syntax for declaring types match the syntax for using types, as foo is used as foo(type, nexp). None of our specifications use types with multiple type parameters so this change doesn't actually break anything, other than some tests. The brackets around the type parameters are now mandatory. - Experiment with splitting Type/Order type parameters from Int type parameters in the parser. Currently in a type bar(x, y, z) all of x, y, and z could be either numeric expressions, orders, or types. This means that in the parser we are severely restricted in what we can parse in numeric expressions because everything has to be parseable as a type (atyp) - it also means we can't introduce boolean type variables/expressions or other minisail features (like removing ticks from type variables!) because we are heavily constrained by what we can parse unambigiously due to how these different type parameters can be mixed and interleaved. There is now experimental syntax: vector::<'o, 'a>('n) <--> vector('n, 'o, 'a) which splits the type argument list into two between Type/Order-polymorphic arguments and Int-polymorphic arguments. The exact choice of delimiters isn't set in stone - ::< and > match generics in Rust. The obvious choices of < and > / [ and ] are ambigious in various ways. Using this syntax right now triggers a warning. - Fix undefined behaviour in C compilation when concatenating a 0-length vector with a 64-length vector.
2018-11-30Improvements for ASL parserAlasdair Armstrong
- Fix pretty printing nested constraints - Add flow typing for if condition then { throw exn }; ... blocks - Add optimisations for bitvector concatenation in C
2018-11-07Move inline forall in function definitionsAlasdair Armstrong
* Previously we allowed the following bizarre syntax for a forall quantifier on a function: val foo(arg1: int('n), arg2: typ2) -> forall 'n, 'n >= 0. unit this commit changes this to the more sane: val foo forall 'n, 'n >= 2. (arg1: int('n), arg2: typ2) -> unit Having talked about it today, we could consider adding the syntax val foo where 'n >= 2. (arg1: int('n), arg2: typ2) -> unit which would avoid the forall (by implicitly quantifying variables in the constraint), and be slightly more friendly especially for documentation purposes. Only RISC-V used this syntax, so all uses of it there have been switched to the new style. * Second, there is a new (somewhat experimental) syntax for existentials, that is hopefully more readable and closer to minisail: val foo(x: int, y: int) -> int('m) with 'm >= 2 "type('n) with constraint" is equivalent to minisail: {'n: type | constraint} the type variables in typ are implicitly quantified, so this is equivalent to {'n, constraint. typ('n)} In order to make this syntax non-ambiguous we have to use == in constraints rather than =, but this is a good thing anyway because the previous situation where = was type level equality and == term level equality was confusing. Now all the type type-level and term-level operators can be consistent. However, to avoid breaking anything = is still allowed in non-with constraints, and produces a deprecated warning when parsed.
2018-10-31Rename Reporting_basic to ReportingAlasdair Armstrong
There is no Reporting_complex, so it's not clear what the basic is intended to signify anyway. Add a GitHub issue link to any err_unreachable errors (as they are all bugs)
2018-10-24Add constraint synonymsAlasdair Armstrong
Currently not enabled by default, the flag -Xconstraint_synonyms enables them For generating constraints in ASL parser, we want to be able to give names to the constraints that we attach to certain variables. It's slightly awkward right now when constraints get long complicated because the entire constraint always has to be typed out in full whenever it appears, and there's no way to abstract away from that. This adds constraint synonyms, which work much like type synonyms except for constraints, e.g. constraint Size('n) = 'n in {1, 2, 4, 8} | 128 <= 'n <= 256 these constraints can then be used instead of the full constraint, e.g. val f : forall 'n, where Size('n). int('n) -> unit Unfortunatly we need to have a keyword to 'call' the constraint synonym otherwise the grammer stops being LR(1). This could be resolved by parsing all constraints into Parse_ast.atyp and then de-sugaring them into constraints, which is what happens for n-expressions already, but that would require quite a bit of work on the parser. To avoid this forcing changes to any other parts of Sail, the intended invariant is that all constraints appearing anywhere in a type-checked AST have no constraint synonyms, so they don't have to worry about matching on NC_app, or calling Env.expand_typquant_synonyms (which isn't even exported for this reason).
2018-08-31mappings: Support for unidirectional mapping clausesJon French
2018-06-26Add configuration registers so __SetConfig ASL can be translatedAlasdair Armstrong
Registers can now be marked as configuration registers, for example: register configuration CFG_RVBAR = 0x1300000 They work like ordinary registers except they can only be set by functions with the 'configuration' effect and have no effect when read. They also have an initialiser, like a let-binding. Internally there is a new reg_dec constructor DEC_config. They are intended to represent configuration parameters for the model, which can change between runs, but don't change during execution. Currently they'll only work when compiled to C. Internally registers can now have custom effects for reads and writes rather than just rreg and wreg, so the type signatures of Env.add_register and Env.get_register have changed, as well as the Register lvar, so in the type checker we now write: Env.add_register id read_effect write_effect typ rather than Env.add_register id typ For the corresponding change to ASL parser there's a function is_config in asl_to_sail.ml which controls what becomes a configuration register for ARM. Some things we have to keep as let-bindings because Sail can't handle them changing at runtime - e.g. the length of vectors in other top-level definitions. Luckily __SetConfig doesn't (yet) try to change those options. Together these changes allow us to translate the ASL __SetConfig function, which means we should get command-line option compatibility with ArchEx for running the ARM conformance tests.
2018-06-11change double-caret for string-append-pattern to single caret, since that ↵Jon French
wouldn't be legal in a pattern anyway
2018-05-01add to parserJon French
2018-05-01starting to also do integer supportJon French
2018-05-01start of string pattern matching: currently only literalsJon French
2018-05-01add { ~~fieldname } sugar to record patterns, expanding to { fieldname = ↵Jon French
fieldname }\n\nCan't use ~ for this to be exactly like OCaml, as is used for 'not' and explicitly allowed as an identifier
2018-03-14WIP Latex formattingAlasdair Armstrong
Added option -latex that outputs input to a latex document. Added doc comments that can be attached to certain AST nodes - right now just valspecs and function clauses, e.g. /*! Documentation for main */ val main : unit -> unit These comments are kept by the sail pretty printer, and used when generating latex
2018-02-06Fix lexer so operators cannot start with /* or //Alasdair Armstrong
2018-01-30Fix failing Lem testsAlasdair Armstrong
2018-01-25Add simple conditional processing and file includeAlasdair Armstrong
Can now use C-style include declarations to include files within other sail files. This is done in such a way that all the location information is preserved in error messages. As an example: $include "aarch64/prelude.sail" $define SYM $ifndef SYM $include <../util.sail> $endif would include the file aarch64/prelude.sail relative to the file where the include is contained. It then defines a symbol SYM and includes another file if it is not defined. The <../util.sail> include will be accessed relative to $SAIL_DIR/lib, so $SAIL_DIR/lib/../util.sail in this case. This can be used with the standard C trick of $ifndef ONCE $define ONCE val f : unit -> unit $endif so no matter how many sail files include the above file, the valspec for f will only appear once. Currently we just have $include, $define, $ifdef and $ifndef (with $else and $endif). We're using $ rather than # because # is already used in internal identifiers, although this could be switched.
2018-01-22Add regression test for type synonym with constraints inside existential ↵Alasdair Armstrong
typechecking bug
2018-01-19Added RISCV test case to test suiteAlasdair Armstrong
2018-01-19Added C-style single line commentsAlasdair Armstrong
// is a comment as well as /* is a comment */
2018-01-16Created version of typecheck test suite for sail2 branchAlasdair Armstrong
Currently doesn't try to compile to lem or use the MIPS spec All the failing tests have been removed because I intend to handle them differently - they were very fragile before because there was no indication of why they failed, so as sail evolved they tended to start failing for the wrong reasons and not testing what they were supposed to.
2018-01-15Refactored and improved ocaml interpreterAlasdair Armstrong
2018-01-05Moved parser, lexer and pretty printer to correct locations.Alasdair Armstrong
2018-01-05Removed legacy parser/lexer and pretty printerAlasdair Armstrong
2017-12-13Use big_nums from LemAlasdair Armstrong
2017-12-05Update license headers for Sail sourceAlasdair Armstrong
2017-12-04Fix warnings in test suiteAlasdair Armstrong
2017-11-24Use unbound precision big_ints throughout sail.Alasdair Armstrong
Alastair's test cases revealed that using regular ints causes issues throughout sail, where all kinds of things can internally overflow in edge cases. This either causes crashes (e.g. int_of_string fails for big ints) or bizarre inexplicable behaviour. This patch switches the sail AST to use big_int rather than int, and updates everything accordingly. This touches everything and there may be bugs where I mistranslated things, and also n = m will still typecheck with big_ints but fail at runtime (ocaml seems to have decided that static typing is unnecessary for equality...), as it needs to be changed to eq_big_int. I also got rid of the old unused ocaml backend while I was updating things, so as to not have to fix it.
2017-09-26Added while-do and repeat-until loops to sail for translating ASLAlasdair Armstrong
2017-08-15Added exceptions and try/catch blocks to AST and typechecker in orderAlasdair Armstrong
to translate exceptions in ASL. See test/typecheck/pass/trycatch.sail.
2017-07-26Added syntax for existential typesAlasdair Armstrong
2017-07-21Improvements to sail n_constraintsAlasdair Armstrong
1) Added a new construct to the expression level: constraint. This is the essentially the boolean form of sizeof. Whereas sizeof takes a nexp and has type [:'n:], constraint takes a n_constraint and returns a boolean. The hope is this will allow for flow typing to be represented more explicitly in the generatated sail from ASL. For example we could have something like: default Order dec val bit[64] -> unit effect pure test64 val forall 'n, ('n = 32 | 'n = 64 | 'n = 10) & 'n != 43. bit['n] -> unit effect pure test function forall 'n. unit test addr = { if constraint('n = 32) then { () } else { assert(constraint('n = 64), "64-bit mode"); test64(addr) } } 2) The other thing this example demonstrates is that flow constraints now work with assert and not just if. Even though flow typing will only guarantee us that 'n != 32 in the else branch, the assert gives us 'n = 64. This is very useful as it's a common idiom in the ARM spec to guarantee such things with an assert. 3) Added != to the n_constraint language 4) Changed the n_constraint language to add or and and as constructs in constraints. Previously one could have a list of conjuncts each of which were simple inequalites or set constraints, now one can do for example: val forall 'n, ('n = 32 | 'n = 64) & 'n in {32, 64}. bit['n] -> unit effect pure test This has the very nice upside that every n_constraint can now be negatated when flow-typing if statements. Note also that 'in' has been introduced as a synonym for 'IN' in the constraint 'n in {32,64}. The use of a block capital keyword was a bit odd there because all the other keywords are lowercase.
2017-07-19Better pretty printing for sail functions with no inline type annotationsAlasdair Armstrong
Also added some additional helper functions in type_check_new.mli and changed real literals slightly
2017-07-18Added pretty-printing support for real literalsAlasdair Armstrong
2017-07-18Added real number literals to sail, to better support full ASL translationAlasdair Armstrong
2017-07-17Added pattern guards to sailAlasdair Armstrong
Introduces a when keyword for case statements, as the Pat_when constructor for pexp's in the AST. This allows us to write things like: typedef T = const union { int C1; int C2 } function int test ((int) x, (T) y) = switch y { case (C1(z)) when z == 0 -> 0 case (C1(z)) when z != 0 -> x quot z case (C2(z)) -> z } this should make translation from ASL's patterns much more straightforward
2017-06-28User defined overloaded operatorsAlasdair Armstrong
New typechecker has no builtin overloaded operators - instead can now write something in SAIL like: overload (deinfix +) [id1; id2; id3] to set up functions id1, id2, and id3 as overloadings for the + operator. Any identifier can be overloaded, not just infix ones. This is done in a backwards compatible way, so the old typechecker removes the DEF_overload nodes from the ast so the various backends never see it.
2017-06-28Improvements to implicit type castingAlasdair Armstrong
Added a new feature for implicit casts - now allowable implicit casts can be specified by the user via a valspec such as val cast forall Type 'a, Type 'b. 'a -> 'b effect pure cast_anything with a new AST constructor to represent this as VS_cast_spec. This constructor is removed and replaced with the standard val spec by the old typechecker for backwards compatability, so it's only used by the new typechecker, and won't appear in the ast once it reaches the backends. Also added Num as a synonym for the Nat kind in the parser, via the confusingly named NatNum token (Num by itself was already taken for a numeric constant).
2017-05-24Merge branch 'master' of bitbucket.org:Peter_Sewell/sailShaked Flur
# Conflicts: # src/lem_interp/interp.lem # src/lem_interp/interp_inter_imp.lem # src/lem_interp/interp_interface.lem # src/parser.mly # src/pretty_print_lem.ml