| Age | Commit message (Collapse) | Author |
|---|
|
(#2423)
Co-authored-by: Jack Koenig <koenig@sifive.com>
|
|
Uses virtual method .getBytesBuffered: Iterable[Array[Byte]] to optimize
file emission.
|
|
|
|
These options are generally specific to a stage and thus should not be
propagating across serialization
|
|
|
|
|
|
* Transform, not run in LegalizeAndReduction test
Switch from using FirrtlStage.transform to FirrtlStage.run in one
test. The latter is problematic as it doesn't include wrappers or
pre/post phases which are how things will work in the future for doing
file writing (via HowToSerialize ideas).
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Use execute in FIRRTL testing infra (not run)
Changes the FirrtlStage method in FIRRTL testing infrastructure from
"run" (which does not include Stage-global Phases) to "execute" (which
does).
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Add HowToSerialize Annotation mix-in
This adds an Annotation mix-in, HowToSerialize, that allows an
annotation to declare how it should be serialized to a file. The
mix-in is abstract in a baseFileName and a suffix (used to generate a
filename), a howToSerialize method (defining the string contents of
the file), and a howToResume method (that defines a replacement for
the file-serialized annotation that allows this to be resumed) [^1].
A default implementation for generating a filename (called filename)
is defined that will put the baseFileName+suffix file in the target
directory. This can be overridden by the annotation if desired.
[^1]: When an annotation is serialized to a file, it should be removed
from the emitted JSON-serialized annotations. The howToResume method
defines a way of adding replacement annotations to the JSON-serialized
annotations that tell a downstream tool how to find the serialized
file. E.g., if a FIRRTL circuit is written to a file, this could be
used to add a FirrtlFileAnnotation defining the location of the new
file.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Handle HowToSerialize in WriteOutputAnnotations
This extends firrtl.options.phase.WriteOutputAnnotations to serialize
HowToSerialize annotations to files.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Test HowToSerialize in WriteOutputAnnotationsSpec
This adds tests of the HowToSerialize mix-in inside the
WriteOutputAnnotationsSpec.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* [skip chisel tests] Migrate to HowToSerialize
This migrates EmittedAnnotations (and its children) to mixin the
HowToSerialize trait. This enables this annotations to be
automatically written to files via WriteOutputAnnotations
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Deprecated firrtl.stage.phases.WriteEmitted
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Use streams in HowToSerialize
This converts the HowToSerialize trait to use a Stream[Char] when
defining how an annotation should be serialized.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Switch from Stream[Char] to Stream[Byte]
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Change howToSerialize method to Iterable
Change the type of the HowToSerialize.howToSerialize method from a
stream to an iterable. Using the latter (the superset of both lazy
streams and non-lazy things like String) avoids problems with users
having to choose laziness when they already have an eager object.
In effect, this makes the API more general.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Add Scaladoc to HowToSerialize trait
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Change HowToSerialize to CustomFileEmission
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Add default implementation of replacements
Add a default implementation of CustomFileEmission.replacements.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Avoid unnecessary 2x monad in CustomFileEmission
Change the type of CustomFileEmission.replacements from
Option[AnnotationSeq] to AnnotationSeq. The latter has all the
properties of the former that I'm trying to express here: (1) can
emptiness and (2) monadicity (if the AnnotationSeq is converted to a
sequence first). The latter property is exploited in the
WriteOutputAnnotations phase to concisely flatMap over the annotations
and doing the double-monad is unnecessary.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Restrict CustomFileEmission filename API
Change the API of CustomFileEmission to use a final def for the actual
filename. The baseFileName is then made a method with an AnnotationSeq
parameter to allow the filename to change as a function of other
annotations, e.g., by an output circuit annotation.
By restricting this API, we have more control over the default
behavior of where things are written using the fixed behavior of the
filename method---files will always be written using the behavior that
StageOptions define. Previously, if users want customized behavior,
they would need to duplicate this StageOptions functionality (and
likely subtly deviate from the standard behavior and introduce
problems with their build).
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Add file conflict behavior for CustomFileEmission
Set behavior of file conflicts in CustomFileEmission to be the
following: No two annotations in the same annotation sequence can
serialize to the same file during the WriteOutputAnnotations phase.
However, if the output annotation file already exists, it will be
overwritten.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Return relative path from getBuildFileName
Change FirrtlOptions.getBuildFileName to simply serialize the
underlying Java File instead of converting this to its canonical path.
This should improve the relocatability of files produced by the
CustomFileEmission API.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Normalize paths in StageOptions.getBuildFile
Normalize paths inside the getBuildFileName utility of StageOptions.
Add a check to prevent a null pointer dereference.
Co-authored-by: Jack Koenig <koenig@sifive.com>
Co-authored-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Refer to CustomFIleEmission in deprecation message
Co-authored-by: Jack Koenig <koenig@sifive.com>
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Simplify CustomFileEmission toBytes implementation
Co-authored-by: Jack Koenig <koenig@sifive.com>
Co-authored-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
* Use toBytes, not getBytes, in CustomFileEmission
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
Co-authored-by: Jack Koenig <koenig@sifive.com>
|
|
|
|
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
* Update scalatest to 3.1.1
* Update scalatest to 3.1.1
* Update scalatest to 3.1.1
Co-authored-by: Jim Lawson <ucbjrl@berkeley.edu>
Co-authored-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
This mixes in the new DependencyAPIMigration trait into all Transforms
and Passes. This enables in-tree transforms/passes to build without
deprecation warnings associated with the deprecated CircuitForm.
As a consequence of this, every Transform now has UnknownForm as both
its inputForm and outputForm. This PR modifies legacy Compiler and
testing infrastructure to schedule transforms NOT using
mergeTransforms/getLoweringTransforms (which rely on inputForm and
outputForm not being UnknownForm), but instead using the Dependency
API.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
* Removed unused imports in src/test/
* Update ScalaTest deprecations.
* Update scalatest from 3.0.8 to 3.1.0; apply auto fix for deprecations
Co-authored-by: Jack Koenig <koenig@sifive.com>
|
|
* Pull out common test utilities into a separate package
* Project a fat jar for test utilities
Co-authored-by: Albert Magyar <albert.magyar@gmail.com>
|
|
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
This adds a prettyPrint method to the DependencyManager to enable
textual visualization of the TransformLikes that a DependencyManager
determines need to be run.
This also cleans up the GraphViz visualization with better edge
coloring and now uses the `name` method when labeling graphviz nodes.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
This makes a change to the Dependency API that breaks chisel3. This
needs to [skip chisel tests], but is fixed with
https://github.com/freechipsproject/chisel3/pull/1270.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
|
|
|
|
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
This changes the Dependency API to specify dependencies in terms of
classes subtyping the DependencyAPI trait. Previously, this was
invariant which caused a bunch of ugly, unneeded .asInstanceOf jank.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
Adds tests for the PhaseManager. This includes outputs in
"test_run_dir" of Graphviz output. This will automatically generate a
PNG if "dot" is on the path. Otherwise, it will just generate the
Graphviz output. This includes tests that check for determinism.
|
|
* Add Test for AddDefaults phase
* Refactor AddDefaultsSpec
|
|
Abstracts away option writing such that users no longer have to
understand scopt semantics. This adds a ShellOption class and a
HasShellOptions trait for something which provides one or more
ShellOptions. This refactors the FIRRTL codebase to use this style of
option specification.
Adds and uses DeletedWrapper to automatically generate
DeletedAnnotations.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
- Add tests for DriverCompatibility.AddImplicitEmitter
- Add tests for DriverCompatibility.AddImplicitOutputFile
- Use a different top name in DriverSpec emit circuit tests for better
coverage
- Add tests for DriverCompatibility.WriteEmitted
- Add catchWrites firrtlTests utility to intercept file writes
- Add tests for WriteOutputAnnotations
- Add tests for --custom-transforms reversing
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
This adds FirrtlStage, a reimplementation of the original FIRRTL
Driver as a Stage. This updates the original firrtl.options package to
implement FirrtlStage (namely, TransformLike is added) along with
FirrtlMain. Finally, the original FIRRTL Driver is converted to a
compatibility wrapper around FirrtlStage.
For background, Stage and Phase form the basis of the Chisel/FIRRTL
Hardware Compiler Framework (HCF). A Phase is a class that performs a
mathematical transformation on an AnnotationSeq (in effect, a
generalization of a FIRRTL transform). Curtly, a Stage is a Phase that
also provides a user interface for generating annotations. By their
construction, Phases are designed to be composed sequentially into a
transformation pipeline.
This modifies the existing options package (which provides
Stage/Phase) to build out a type hierarchy around Stage/Phase. This
adds TransformLike[A] which implements a mathematical transformation
over some type A. Additionally, and as an interface between different
TransformLikes, this adds Translator[A, B] which extends
TransformLike[A], but does an internal transformation over type B.
This is used to interface Phases with the existing FIRRTL compiler.
This adds a runTransform method to Phase that, like
Transform.runTransform, will automatically detect deleted Annotations
and generate DeletedAnnotations.
The new FirrtlStage, a reimplementation of FIRRTL's Driver, is added
as a Stage composed of the following Phases:
1. AddDefaults - add default annotations
2. AddImplicitEmitter - adds an implicit emitter derived from the
compiler
3. Checks - sanity check the AnnotationSeq
4. AddCircuit - convert FIRRTL input files/sources to circuits
5. AddImplicitOutputFile - add a default output file
6. Compiler - run the FIRRTL compiler
7. WriteEmitted - write any emitted modules/circuits to files
The Driver is converted to a compatibility layer that replicates old
Driver behavior. This is implemented by first using new toAnnotation
methods for CommonOptions and FirrtlExecutionOptions that enable
AnnotationSeq generation. Second, the generated AnnotationSeq is
preprocessed and sent to FirrtlStage. The resulting Phase order is
then:
1. AddImplicitAnnotationFile - adds a default annotation file
2. AddImplicitFirrtlFile - adds a default FIRRTL file using top name
3. AddImplicitOutputFile - adds an output file from top name
4. AddImplicitEmitter - adds a default emitter derived from a
compiler and any split modules command line option
5. FirrtlStage - the aforementioned new FirrtlStage
Finally, the output AnnotationSeq is then viewed as a
FirrtlExecutionResult. This compatibility layer enables uninterrupted
usage of old Driver infrastructure, e.g., FirrtlExecutionOptions and
CommonOptions can still be mutated directly and used to run the
Driver.
This results in differing behavior between the new FirrtlStage and the
old Driver, specifically:
- FirrtlStage makes a clear delineation between a "compiler" and an
"emitter". These are defined using separate options. A compiler is
"-X/--compiler", while an emitter is one of "-E/--emit-circuit" or
"-e/--emit-modules".
- Related to the above, the "-fsm/--split-modules" has been removed
from the FirrtlStage. This option is confusing once an implicit
emitter is removed. It is also unclear how this should be handled
once the user can specify multiple emitters, e.g., which emitter
should "--split-modules" apply to?
- WriteOutputAnnotations will, by default, not write
DeletedAnnotations to the output file.
- The old top name ("-tn/--top-name") option has been removed from
FirrtlStage. This option is really a means to communicate what
input and output files are as opposed to anything associated with
the circuit name. This option is preserved for the Driver
compatibility layer.
Additionally, this changes existing transform scheduling to work for
emitters (which subclass Transform). Previously, one emitter was
explicitly scheduled at the end of all transforms for a given
compiler. Additional emitters could be added, but they would be
scheduled as transforms. This fixes this to rely on transform
scheduling for all emitters. In slightly more detail:
1. The explicit emitter is removed from Compiler.compile
2. An explicit emitter is added to Compiler.compileAndEmit
3. Compiler.mergeTransforms will schedule emitters as late as
possible, i.e., all emitters will occur after transforms that
output their input form.
4. All AddImplicitEmitter phases (DriverCompatibility and normal)
will add RunFirrtlTransformAnnotations to add implicit emitters
The FIRRTL fat jar utilities are changed to point at FirrtlStage and not
at the Driver. This has backwards incompatibility issues for users
that are using the utilities directly, e.g., Rocket Chip.
The Logger has been updated with methods for setting options based on
an AnnotationSeq. This migrates the Logger to use AnnotationSeq as
input parameters, e.g., for makeScope. Old-style methods are left in
place and deprecated. However, the Logger is not itself a Stage.
The options of Logger Annotations are included in the base Shell and
Stage is updated to wrap its Phases in a Logger scope.
Additionally, this changes any code that does option parsing to always
prepend an annotation as opposed to appending an annotation. This is
faster, but standardizing on this has implications for dealing with
the parallel compilation annotation ordering.
A Shell will now put the initial annotations first (in the order the
user specified) and then place all annotations generating from parsing
after that. This adds a test case to verify this behavior.
Discovered custom transforms (via `RunFirrtlTransformAnnotation`s) are
discovered by the compiler phase in a user-specified order, but are
stored in reverse order to more efficiently prepend (as opposed to
append) to a list. This now reverses the transform order before
execution to preserve backwards compatibility of custom transform
ordering.
The Compiler phase also generates one deleted annotation for each
`RunFirrtlTransformAnnotation`. These are also reversed.
Miscellaneous small changes:
- Split main method of Stage into StageMain class
- Only mix in HasScoptOptions into Annotation companion objects (h/t
@jackkoenig)
- Store Compiler in CompilerAnnotation
- CompilerNameAnnotation -> CompilerAnnotation
- Make Emitter abstract in outputSuffix (move out of FirrtlOptions)
- Add DriverCompatibility.AddImplicitOutputFile that will add an
output file annotation based on the presence of a
TopNameAnnotation. This is important for compatibility with the
old Driver.
- Cleanup Scaladoc
- Refactor CircuitOption to be abstract in "toCircuit" that converts
the option to a FirrtlCircuitAnnotation. This allows more of the
conversion steps to be moved out of AddCircuit and into the actual
annotation.
- Add WriteDeletedAnnotation to module WriteOutputAnnotations
- A method for accessing a FirrtlExecutionResultView is exposed in
FIRRTL's DriverCompatibilityLayer
- Using "--top-name/-tn" or "--split-modules/-fsm" with FirrtlStage
generates an error indicating that this option is no longer
supported
- Using FirrtlStage without at least one emitter will generate a
warning
- Use vals for emitter in Compiler subclasses (these are used to
build RunFirrtlTransformAnnotations and the object should be
stable for comparisons)
- Fixes to tests that use LowTransformSpec instead of
MiddleTransformSpec. (SimpleTransformSpec is dumb and won't
schedule transforms correctly. If you rely on an emitter, you need
to use the right transform spec to test your transform if you're
relying on an emitter.)
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
This breaks firrtl.options.Stage into a small type hierarchy:
* Phase: something that transforms an AnnotationSeq
* Stage extends Phase: a Phase with a Command Line Interface
Some of the old "common options" (input annotation file and target
directory) are moved into firrtl.options and provided as part of the Stage
class. Stage will automatically preprocess an input annotation sequence to
resolve all input annotation files and add a default target directory.
Minor changes:
* Adds ViewException
* Stops mixing in the DoNotTerminateOnExit trait into the default Shell
parser
* Add StageOptionsView
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
|
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@ibm.com>
|
