From b2dd0eb845081609d0aec4a873587ab3f22fe3f7 Mon Sep 17 00:00:00 2001 From: Schuyler Eldridge Date: Tue, 20 Nov 2018 21:07:52 -0500 Subject: Add FirrtlStage, make Driver compatibility layer This adds FirrtlStage, a reimplementation of the original FIRRTL Driver as a Stage. This updates the original firrtl.options package to implement FirrtlStage (namely, TransformLike is added) along with FirrtlMain. Finally, the original FIRRTL Driver is converted to a compatibility wrapper around FirrtlStage. For background, Stage and Phase form the basis of the Chisel/FIRRTL Hardware Compiler Framework (HCF). A Phase is a class that performs a mathematical transformation on an AnnotationSeq (in effect, a generalization of a FIRRTL transform). Curtly, a Stage is a Phase that also provides a user interface for generating annotations. By their construction, Phases are designed to be composed sequentially into a transformation pipeline. This modifies the existing options package (which provides Stage/Phase) to build out a type hierarchy around Stage/Phase. This adds TransformLike[A] which implements a mathematical transformation over some type A. Additionally, and as an interface between different TransformLikes, this adds Translator[A, B] which extends TransformLike[A], but does an internal transformation over type B. This is used to interface Phases with the existing FIRRTL compiler. This adds a runTransform method to Phase that, like Transform.runTransform, will automatically detect deleted Annotations and generate DeletedAnnotations. The new FirrtlStage, a reimplementation of FIRRTL's Driver, is added as a Stage composed of the following Phases: 1. AddDefaults - add default annotations 2. AddImplicitEmitter - adds an implicit emitter derived from the compiler 3. Checks - sanity check the AnnotationSeq 4. AddCircuit - convert FIRRTL input files/sources to circuits 5. AddImplicitOutputFile - add a default output file 6. Compiler - run the FIRRTL compiler 7. WriteEmitted - write any emitted modules/circuits to files The Driver is converted to a compatibility layer that replicates old Driver behavior. This is implemented by first using new toAnnotation methods for CommonOptions and FirrtlExecutionOptions that enable AnnotationSeq generation. Second, the generated AnnotationSeq is preprocessed and sent to FirrtlStage. The resulting Phase order is then: 1. AddImplicitAnnotationFile - adds a default annotation file 2. AddImplicitFirrtlFile - adds a default FIRRTL file using top name 3. AddImplicitOutputFile - adds an output file from top name 4. AddImplicitEmitter - adds a default emitter derived from a compiler and any split modules command line option 5. FirrtlStage - the aforementioned new FirrtlStage Finally, the output AnnotationSeq is then viewed as a FirrtlExecutionResult. This compatibility layer enables uninterrupted usage of old Driver infrastructure, e.g., FirrtlExecutionOptions and CommonOptions can still be mutated directly and used to run the Driver. This results in differing behavior between the new FirrtlStage and the old Driver, specifically: - FirrtlStage makes a clear delineation between a "compiler" and an "emitter". These are defined using separate options. A compiler is "-X/--compiler", while an emitter is one of "-E/--emit-circuit" or "-e/--emit-modules". - Related to the above, the "-fsm/--split-modules" has been removed from the FirrtlStage. This option is confusing once an implicit emitter is removed. It is also unclear how this should be handled once the user can specify multiple emitters, e.g., which emitter should "--split-modules" apply to? - WriteOutputAnnotations will, by default, not write DeletedAnnotations to the output file. - The old top name ("-tn/--top-name") option has been removed from FirrtlStage. This option is really a means to communicate what input and output files are as opposed to anything associated with the circuit name. This option is preserved for the Driver compatibility layer. Additionally, this changes existing transform scheduling to work for emitters (which subclass Transform). Previously, one emitter was explicitly scheduled at the end of all transforms for a given compiler. Additional emitters could be added, but they would be scheduled as transforms. This fixes this to rely on transform scheduling for all emitters. In slightly more detail: 1. The explicit emitter is removed from Compiler.compile 2. An explicit emitter is added to Compiler.compileAndEmit 3. Compiler.mergeTransforms will schedule emitters as late as possible, i.e., all emitters will occur after transforms that output their input form. 4. All AddImplicitEmitter phases (DriverCompatibility and normal) will add RunFirrtlTransformAnnotations to add implicit emitters The FIRRTL fat jar utilities are changed to point at FirrtlStage and not at the Driver. This has backwards incompatibility issues for users that are using the utilities directly, e.g., Rocket Chip. The Logger has been updated with methods for setting options based on an AnnotationSeq. This migrates the Logger to use AnnotationSeq as input parameters, e.g., for makeScope. Old-style methods are left in place and deprecated. However, the Logger is not itself a Stage. The options of Logger Annotations are included in the base Shell and Stage is updated to wrap its Phases in a Logger scope. Additionally, this changes any code that does option parsing to always prepend an annotation as opposed to appending an annotation. This is faster, but standardizing on this has implications for dealing with the parallel compilation annotation ordering. A Shell will now put the initial annotations first (in the order the user specified) and then place all annotations generating from parsing after that. This adds a test case to verify this behavior. Discovered custom transforms (via `RunFirrtlTransformAnnotation`s) are discovered by the compiler phase in a user-specified order, but are stored in reverse order to more efficiently prepend (as opposed to append) to a list. This now reverses the transform order before execution to preserve backwards compatibility of custom transform ordering. The Compiler phase also generates one deleted annotation for each `RunFirrtlTransformAnnotation`. These are also reversed. Miscellaneous small changes: - Split main method of Stage into StageMain class - Only mix in HasScoptOptions into Annotation companion objects (h/t @jackkoenig) - Store Compiler in CompilerAnnotation - CompilerNameAnnotation -> CompilerAnnotation - Make Emitter abstract in outputSuffix (move out of FirrtlOptions) - Add DriverCompatibility.AddImplicitOutputFile that will add an output file annotation based on the presence of a TopNameAnnotation. This is important for compatibility with the old Driver. - Cleanup Scaladoc - Refactor CircuitOption to be abstract in "toCircuit" that converts the option to a FirrtlCircuitAnnotation. This allows more of the conversion steps to be moved out of AddCircuit and into the actual annotation. - Add WriteDeletedAnnotation to module WriteOutputAnnotations - A method for accessing a FirrtlExecutionResultView is exposed in FIRRTL's DriverCompatibilityLayer - Using "--top-name/-tn" or "--split-modules/-fsm" with FirrtlStage generates an error indicating that this option is no longer supported - Using FirrtlStage without at least one emitter will generate a warning - Use vals for emitter in Compiler subclasses (these are used to build RunFirrtlTransformAnnotations and the object should be stable for comparisons) - Fixes to tests that use LowTransformSpec instead of MiddleTransformSpec. (SimpleTransformSpec is dumb and won't schedule transforms correctly. If you rely on an emitter, you need to use the right transform spec to test your transform if you're relying on an emitter.) Signed-off-by: Schuyler Eldridge --- src/main/scala/firrtl/Compiler.scala | 53 +++- src/main/scala/firrtl/Driver.scala | 130 +++------ src/main/scala/firrtl/Emitter.scala | 117 ++++++-- .../scala/firrtl/ExecutionOptionsManager.scala | 49 +++- src/main/scala/firrtl/LoweringCompilers.scala | 16 +- src/main/scala/firrtl/annotations/Annotation.scala | 5 +- src/main/scala/firrtl/options/Exceptions.scala | 20 ++ src/main/scala/firrtl/options/OptionParser.scala | 18 +- src/main/scala/firrtl/options/OptionsView.scala | 10 +- src/main/scala/firrtl/options/Phase.scala | 88 +++++- src/main/scala/firrtl/options/Registration.scala | 9 +- src/main/scala/firrtl/options/Shell.scala | 48 +++- src/main/scala/firrtl/options/Stage.scala | 45 ++- .../scala/firrtl/options/StageAnnotations.scala | 83 ++++-- src/main/scala/firrtl/options/StageOptions.scala | 28 +- src/main/scala/firrtl/options/StageUtils.scala | 8 +- src/main/scala/firrtl/options/package.scala | 9 +- .../scala/firrtl/options/phases/AddDefaults.scala | 8 +- src/main/scala/firrtl/options/phases/Checks.scala | 43 +++ .../options/phases/ConvertLegacyAnnotations.scala | 2 +- .../scala/firrtl/options/phases/GetIncludes.scala | 11 +- .../options/phases/WriteOutputAnnotations.scala | 36 +++ src/main/scala/firrtl/package.scala | 8 +- .../firrtl/passes/memlib/InferReadWrite.scala | 2 +- .../firrtl/passes/memlib/ReplaceMemTransform.scala | 4 +- .../scala/firrtl/stage/FirrtlAnnotations.scala | 320 ++++++++------------- src/main/scala/firrtl/stage/FirrtlCli.scala | 25 ++ src/main/scala/firrtl/stage/FirrtlOptions.scala | 32 +++ src/main/scala/firrtl/stage/FirrtlStage.scala | 39 +++ src/main/scala/firrtl/stage/FirrtlStageUtils.scala | 17 ++ src/main/scala/firrtl/stage/package.scala | 64 +++++ .../scala/firrtl/stage/phases/AddCircuit.scala | 59 ++++ .../scala/firrtl/stage/phases/AddDefaults.scala | 36 +++ .../firrtl/stage/phases/AddImplicitEmitter.scala | 30 ++ .../stage/phases/AddImplicitOutputFile.scala | 35 +++ src/main/scala/firrtl/stage/phases/Checks.scala | 93 ++++++ src/main/scala/firrtl/stage/phases/Compiler.scala | 99 +++++++ .../firrtl/stage/phases/DriverCompatibility.scala | 223 ++++++++++++++ .../scala/firrtl/stage/phases/WriteEmitted.scala | 50 ++++ .../firrtl/util/BackendCompilationUtilities.scala | 2 +- src/main/scala/logger/Logger.scala | 85 ++++-- src/main/scala/logger/LoggerAnnotations.scala | 77 +++++ src/main/scala/logger/LoggerOptions.scala | 38 +++ src/main/scala/logger/package.scala | 21 ++ src/main/scala/logger/phases/AddDefaults.scala | 27 ++ src/main/scala/logger/phases/Checks.scala | 42 +++ src/test/scala/firrtlTests/DriverSpec.scala | 31 +- .../annotationTests/TargetDirAnnotationSpec.scala | 1 - src/test/scala/firrtlTests/options/ShellSpec.scala | 20 +- .../transforms/GroupComponentsSpec.scala | 15 +- .../firrtlTests/transforms/TopWiringTest.scala | 68 +++-- 51 files changed, 1883 insertions(+), 516 deletions(-) create mode 100644 src/main/scala/firrtl/options/Exceptions.scala create mode 100644 src/main/scala/firrtl/options/phases/Checks.scala create mode 100644 src/main/scala/firrtl/options/phases/WriteOutputAnnotations.scala create mode 100644 src/main/scala/firrtl/stage/FirrtlCli.scala create mode 100644 src/main/scala/firrtl/stage/FirrtlOptions.scala create mode 100644 src/main/scala/firrtl/stage/FirrtlStage.scala create mode 100644 src/main/scala/firrtl/stage/FirrtlStageUtils.scala create mode 100644 src/main/scala/firrtl/stage/package.scala create mode 100644 src/main/scala/firrtl/stage/phases/AddCircuit.scala create mode 100644 src/main/scala/firrtl/stage/phases/AddDefaults.scala create mode 100644 src/main/scala/firrtl/stage/phases/AddImplicitEmitter.scala create mode 100644 src/main/scala/firrtl/stage/phases/AddImplicitOutputFile.scala create mode 100644 src/main/scala/firrtl/stage/phases/Checks.scala create mode 100644 src/main/scala/firrtl/stage/phases/Compiler.scala create mode 100644 src/main/scala/firrtl/stage/phases/DriverCompatibility.scala create mode 100644 src/main/scala/firrtl/stage/phases/WriteEmitted.scala create mode 100644 src/main/scala/logger/LoggerAnnotations.scala create mode 100644 src/main/scala/logger/LoggerOptions.scala create mode 100644 src/main/scala/logger/package.scala create mode 100644 src/main/scala/logger/phases/AddDefaults.scala create mode 100644 src/main/scala/logger/phases/Checks.scala (limited to 'src') diff --git a/src/main/scala/firrtl/Compiler.scala b/src/main/scala/firrtl/Compiler.scala index 1ef35891..a15ca6cb 100644 --- a/src/main/scala/firrtl/Compiler.scala +++ b/src/main/scala/firrtl/Compiler.scala @@ -14,6 +14,7 @@ import firrtl.Utils.{error, throwInternalError} import firrtl.annotations.TargetToken import firrtl.annotations.TargetToken.{Field, Index} import firrtl.annotations.transforms.{EliminateTargetPaths, ResolvePaths} +import firrtl.options.StageUtils /** Container of all annotations for a Firrtl compiler */ class AnnotationSeq private (private[firrtl] val underlying: List[Annotation]) { @@ -101,6 +102,9 @@ object CircuitState { sealed abstract class CircuitForm(private val value: Int) extends Ordered[CircuitForm] { // Note that value is used only to allow comparisons def compare(that: CircuitForm): Int = this.value - that.value + + /** Defines a suffix to use if this form is written to a file */ + def outputSuffix: String } // scalastyle:off magic.number @@ -113,7 +117,10 @@ sealed abstract class CircuitForm(private val value: Int) extends Ordered[Circui * * See [[CDefMemory]] and [[CDefMPort]] */ -final case object ChirrtlForm extends CircuitForm(value = 3) +final case object ChirrtlForm extends CircuitForm(value = 3) { + val outputSuffix: String = ".fir" +} + /** High Form * * As detailed in the Firrtl specification @@ -121,7 +128,10 @@ final case object ChirrtlForm extends CircuitForm(value = 3) * * Also see [[firrtl.ir]] */ -final case object HighForm extends CircuitForm(2) +final case object HighForm extends CircuitForm(2) { + val outputSuffix: String = ".hi.fir" +} + /** Middle Form * * A "lower" form than [[HighForm]] with the following restrictions: @@ -129,14 +139,20 @@ final case object HighForm extends CircuitForm(2) * - All whens must be removed * - There can only be a single connection to any element */ -final case object MidForm extends CircuitForm(1) +final case object MidForm extends CircuitForm(1) { + val outputSuffix: String = ".mid.fir" +} + /** Low Form * * The "lowest" form. In addition to the restrictions in [[MidForm]]: * - All aggregate types (vector/bundle) must have been removed * - All implicit truncations must be made explicit */ -final case object LowForm extends CircuitForm(0) +final case object LowForm extends CircuitForm(0) { + val outputSuffix: String = ".lo.fir" +} + /** Unknown Form * * Often passes may modify a circuit (e.g. InferTypes), but return @@ -150,6 +166,8 @@ final case object LowForm extends CircuitForm(0) */ final case object UnknownForm extends CircuitForm(-1) { override def compare(that: CircuitForm): Int = { sys.error("Illegal to compare UnknownForm"); 0 } + + val outputSuffix: String = ".unknown.fir" } // scalastyle:on magic.number @@ -178,7 +196,7 @@ abstract class Transform extends LazyLogging { @deprecated("Just collect the actual Annotation types the transform wants", "1.1") final def getMyAnnotations(state: CircuitState): Seq[Annotation] = { val msg = "getMyAnnotations is deprecated, use collect and match on concrete types" - Driver.dramaticWarning(msg) + StageUtils.dramaticWarning(msg) state.annotations.collect { case a: LegacyAnnotation if a.transform == this.getClass => a } } @@ -293,6 +311,9 @@ trait ResolvedAnnotationPaths { trait Emitter extends Transform { @deprecated("Use emission annotations instead", "firrtl 1.0") def emit(state: CircuitState, writer: Writer): Unit + + /** An output suffix to use if the output of this [[Emitter]] was written to a file */ + def outputSuffix: String } /** Wraps exceptions from CustomTransforms so they can be reported appropriately */ @@ -325,11 +346,10 @@ object CompilerUtils extends LazyLogging { /** Merge a Seq of lowering transforms with custom transforms * - * Custom Transforms are inserted based on their [[Transform.inputForm]] and - * [[Transform.outputForm]]. Custom transforms are inserted in order at the - * last location in the Seq of transforms where previous.outputForm == - * customTransform.inputForm. If a customTransform outputs a higher form - * than input, [[getLoweringTransforms]] is used to relower the circuit. + * Custom Transforms are inserted based on their [[Transform.inputForm]] and [[Transform.outputForm]] with any + * [[Emitter]]s being scheduled last. Custom transforms are inserted in order at the last location in the Seq of + * transforms where previous.outputForm == customTransform.inputForm. If a customTransform outputs a higher form than + * input, [[getLoweringTransforms]] is used to relower the circuit. * * @example * {{{ @@ -356,7 +376,13 @@ object CompilerUtils extends LazyLogging { * of the previous transform. */ def mergeTransforms(lowering: Seq[Transform], custom: Seq[Transform]): Seq[Transform] = { - custom.foldLeft(lowering) { case (transforms, xform) => + custom + .sortWith{ + case (a, b) => (a, b) match { + case (_: Emitter, _: Emitter) => false + case (_, _: Emitter) => true + case _ => false }} + .foldLeft(lowering) { case (transforms, xform) => val index = transforms lastIndexWhere (_.outputForm == xform.inputForm) assert(index >= 0 || xform.inputForm == ChirrtlForm, // If ChirrtlForm just put at front s"No transform in $lowering has outputForm ${xform.inputForm} as required by $xform") @@ -427,7 +453,7 @@ trait Compiler extends LazyLogging { def compileAndEmit(state: CircuitState, customTransforms: Seq[Transform] = Seq.empty): CircuitState = { val emitAnno = EmitCircuitAnnotation(emitter.getClass) - compile(state.copy(annotations = emitAnno +: state.annotations), customTransforms) + compile(state.copy(annotations = emitAnno +: state.annotations), emitter +: customTransforms) } private def isCustomTransform(xform: Transform): Boolean = { @@ -449,7 +475,7 @@ trait Compiler extends LazyLogging { * @return result of compilation */ def compile(state: CircuitState, customTransforms: Seq[Transform]): CircuitState = { - val allTransforms = CompilerUtils.mergeTransforms(transforms, customTransforms) :+ emitter + val allTransforms = CompilerUtils.mergeTransforms(transforms, customTransforms) val (timeMillis, finalState) = Utils.time { allTransforms.foldLeft(state) { (in, xform) => @@ -468,4 +494,3 @@ trait Compiler extends LazyLogging { } } - diff --git a/src/main/scala/firrtl/Driver.scala b/src/main/scala/firrtl/Driver.scala index e96c3c5b..f23be6f5 100644 --- a/src/main/scala/firrtl/Driver.scala +++ b/src/main/scala/firrtl/Driver.scala @@ -4,11 +4,10 @@ package firrtl import scala.collection._ import scala.io.Source -import scala.sys.process.{BasicIO, ProcessLogger, stringSeqToProcess} import scala.util.{Failure, Success, Try} import scala.util.control.ControlThrowable import java.io.{File, FileNotFoundException} - +import scala.sys.process.{BasicIO, ProcessLogger, stringSeqToProcess} import net.jcazevedo.moultingyaml._ import logger.Logger import Parser.{IgnoreInfo, InfoMode} @@ -17,6 +16,10 @@ import firrtl.annotations.AnnotationYamlProtocol._ import firrtl.passes.{PassException, PassExceptions} import firrtl.transforms._ import firrtl.Utils.throwInternalError +import firrtl.stage.{FirrtlExecutionResultView, FirrtlStage} +import firrtl.stage.phases.DriverCompatibility +import firrtl.options.{StageUtils, Phase} +import firrtl.options.Viewer.view /** @@ -40,30 +43,22 @@ import firrtl.Utils.throwInternalError * @see firrtlTests/DriverSpec.scala in the test directory for a lot more examples * @see [[CompilerUtils.mergeTransforms]] to see how customTransformations are inserted */ - +@deprecated("Use firrtl.stage.FirrtlStage", "1.2") object Driver { /** Print a warning message * * @param message error message */ - //scalastyle:off regex - def dramaticWarning(message: String): Unit = { - println(Console.YELLOW + "-"*78) - println(s"Warning: $message") - println("-"*78 + Console.RESET) - } + @deprecated("Use firrtl.options.StageUtils.dramaticWarning", "1.2") + def dramaticWarning(message: String): Unit = StageUtils.dramaticWarning(message) /** * print the message in red * * @param message error message */ - //scalastyle:off regex - def dramaticError(message: String): Unit = { - println(Console.RED + "-"*78) - println(s"Error: $message") - println("-"*78 + Console.RESET) - } + @deprecated("Use firrtl.options.StageUtils.dramaticWarning", "1.2") + def dramaticError(message: String): Unit = StageUtils.dramaticError(message) /** Load annotation file based on options * @param optionsManager use optionsManager config to load annotation file if it exists @@ -107,7 +102,7 @@ object Driver { if (firrtlConfig.annotationFileNameOverride.nonEmpty) { val msg = "annotationFileNameOverride is deprecated! " + "Use annotationFileNames" - Driver.dramaticWarning(msg) + dramaticWarning(msg) } else if (usingImplicitAnnoFile) { val msg = "Implicit .anno file from top-name is deprecated!\n" + (" "*9) + "Use explicit -faf option or annotationFileNames" @@ -159,7 +154,7 @@ object Driver { } // Useful for handling erros in the options - case class OptionsException(msg: String) extends Exception(msg) + case class OptionsException(message: String) extends Exception(message) /** Get the Circuit from the compile options * @@ -216,77 +211,25 @@ object Driver { * @return a FirrtlExecutionResult indicating success or failure, provide access to emitted data on success * for downstream tools as desired */ - //scalastyle:off cyclomatic.complexity method.length def execute(optionsManager: ExecutionOptionsManager with HasFirrtlOptions): FirrtlExecutionResult = { - def firrtlConfig = optionsManager.firrtlOptions - - Logger.makeScope(optionsManager) { - // Wrap compilation in a try/catch to present Scala MatchErrors in a more user-friendly format. - val finalState = try { - val circuit = getCircuit(optionsManager) match { - case Success(c) => c - case Failure(OptionsException(msg)) => - dramaticError(msg) - return FirrtlExecutionFailure(msg) - case Failure(e) => throw e - } + StageUtils.dramaticWarning("firrtl.Driver is deprecated since 1.2!\nPlease switch to firrtl.stage.FirrtlStage") - val annos = getAnnotations(optionsManager) + val annos = optionsManager.firrtlOptions.toAnnotations ++ optionsManager.commonOptions.toAnnotations - // Does this need to be before calling compiler? - optionsManager.makeTargetDir() + val phases: Seq[Phase] = Seq( + new DriverCompatibility.AddImplicitAnnotationFile, + new DriverCompatibility.AddImplicitFirrtlFile, + new DriverCompatibility.AddImplicitOutputFile, + new DriverCompatibility.AddImplicitEmitter, + new FirrtlStage ) - firrtlConfig.compiler.compile( - CircuitState(circuit, ChirrtlForm, annos), - firrtlConfig.customTransforms - ) - } - catch { - // Rethrow the exceptions which are expected or due to the runtime environment (out of memory, stack overflow) - case p: ControlThrowable => throw p - case p: PassException => throw p - case p: PassExceptions => throw p - case p: FIRRTLException => throw p - // Propagate exceptions from custom transforms - case CustomTransformException(cause) => throw cause - // Treat remaining exceptions as internal errors. - case e: Exception => throwInternalError(exception = Some(e)) - } - - // Do emission - // Note: Single emission target assumption is baked in here - // Note: FirrtlExecutionSuccess emitted is only used if we're emitting the whole Circuit - val emittedRes = firrtlConfig.getOutputConfig(optionsManager) match { - case SingleFile(filename) => - val emitted = finalState.getEmittedCircuit - val outputFile = new java.io.PrintWriter(filename) - outputFile.write(emitted.value) - outputFile.close() - emitted.value - case OneFilePerModule(dirName) => - val emittedModules = finalState.emittedComponents collect { case x: EmittedModule => x } - if (emittedModules.isEmpty) throwInternalError() // There should be something - emittedModules.foreach { module => - val filename = optionsManager.getBuildFileName(firrtlConfig.outputSuffix, s"$dirName/${module.name}") - val outputFile = new java.io.PrintWriter(filename) - outputFile.write(module.value) - outputFile.close() - } - "" // Should we return something different here? - } - - // If set, emit final annotations to a file - optionsManager.firrtlOptions.outputAnnotationFileName match { - case "" => - case file => - val filename = optionsManager.getBuildFileName("anno.json", file) - val outputFile = new java.io.PrintWriter(filename) - outputFile.write(JsonProtocol.serialize(finalState.annotations)) - outputFile.close() - } - - FirrtlExecutionSuccess(firrtlConfig.compilerName, emittedRes, finalState) + val annosx = try { + phases.foldLeft(annos)( (a, p) => p.runTransform(a) ) + } catch { + case e: firrtl.options.OptionsException => return FirrtlExecutionFailure(e.message) } + + view[FirrtlExecutionResult](annosx) } /** @@ -321,23 +264,16 @@ object Driver { } object FileUtils { - /** - * recursive create directory and all parents - * + + /** Create a directory if it doesn't exist * @param directoryName a directory string with one or more levels - * @return + * @return true if the directory exists or if it was successfully created */ def makeDirectory(directoryName: String): Boolean = { - val dirFile = new java.io.File(directoryName) + val dirFile = new File(directoryName) if(dirFile.exists()) { - if(dirFile.isDirectory) { - true - } - else { - false - } - } - else { + dirFile.isDirectory + } else { dirFile.mkdirs() } } @@ -361,7 +297,7 @@ object FileUtils { if(file.getPath.split("/").last.isEmpty || file.getAbsolutePath == "/" || file.getPath.startsWith("/")) { - Driver.dramaticError(s"delete directory ${file.getPath} will not delete absolute paths") + StageUtils.dramaticError(s"delete directory ${file.getPath} will not delete absolute paths") false } else { diff --git a/src/main/scala/firrtl/Emitter.scala b/src/main/scala/firrtl/Emitter.scala index 7204eea6..44190b39 100644 --- a/src/main/scala/firrtl/Emitter.scala +++ b/src/main/scala/firrtl/Emitter.scala @@ -19,6 +19,9 @@ import firrtl.PrimOps._ import firrtl.WrappedExpression._ import Utils._ import MemPortUtils.{memPortField, memType} +import firrtl.options.{HasScoptOptions, StageUtils, PhaseException} +import firrtl.stage.RunFirrtlTransformAnnotation +import scopt.OptionParser // Datastructures import scala.collection.mutable.{ArrayBuffer, LinkedHashMap, HashSet} @@ -31,17 +34,70 @@ sealed trait EmitAnnotation extends NoTargetAnnotation { case class EmitCircuitAnnotation(emitter: Class[_ <: Emitter]) extends EmitAnnotation case class EmitAllModulesAnnotation(emitter: Class[_ <: Emitter]) extends EmitAnnotation +object EmitCircuitAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p + .opt[String]("emit-circuit") + .abbr("E") + .valueName("") + .unbounded() + .action{ (x, c) => + val xx = x match { + case "chirrtl" => Seq(RunFirrtlTransformAnnotation(new ChirrtlEmitter), + EmitCircuitAnnotation(classOf[ChirrtlEmitter])) + case "high" => Seq(RunFirrtlTransformAnnotation(new HighFirrtlEmitter), + EmitCircuitAnnotation(classOf[HighFirrtlEmitter])) + case "middle" => Seq(RunFirrtlTransformAnnotation(new MiddleFirrtlEmitter), + EmitCircuitAnnotation(classOf[MiddleFirrtlEmitter])) + case "low" => Seq(RunFirrtlTransformAnnotation(new LowFirrtlEmitter), + EmitCircuitAnnotation(classOf[LowFirrtlEmitter])) + case "verilog" | "mverilog" => Seq(RunFirrtlTransformAnnotation(new VerilogEmitter), + EmitCircuitAnnotation(classOf[VerilogEmitter])) + case "sverilog" => Seq(RunFirrtlTransformAnnotation(new SystemVerilogEmitter), + EmitCircuitAnnotation(classOf[SystemVerilogEmitter])) + case _ => throw new PhaseException(s"Unknown emitter '$x'! (Did you misspell it?)") + } + xx ++ c } + .text("Run the specified circuit emitter (all modules in one file)") +} + +object EmitAllModulesAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p + .opt[String]("emit-modules") + .abbr("e") + .valueName("") + .unbounded() + .action{ (x, c) => + val xx = x match { + case "chirrtl" => Seq(RunFirrtlTransformAnnotation(new ChirrtlEmitter), + EmitAllModulesAnnotation(classOf[ChirrtlEmitter])) + case "high" => Seq(RunFirrtlTransformAnnotation(new HighFirrtlEmitter), + EmitAllModulesAnnotation(classOf[HighFirrtlEmitter])) + case "middle" => Seq(RunFirrtlTransformAnnotation(new MiddleFirrtlEmitter), + EmitAllModulesAnnotation(classOf[MiddleFirrtlEmitter])) + case "low" => Seq(RunFirrtlTransformAnnotation(new LowFirrtlEmitter), + EmitAllModulesAnnotation(classOf[LowFirrtlEmitter])) + case "verilog" | "mverilog" => Seq(RunFirrtlTransformAnnotation(new VerilogEmitter), + EmitAllModulesAnnotation(classOf[VerilogEmitter])) + case "sverilog" => Seq(RunFirrtlTransformAnnotation(new SystemVerilogEmitter), + EmitAllModulesAnnotation(classOf[SystemVerilogEmitter])) + case _ => throw new PhaseException(s"Unknown emitter '$x'! (Did you misspell it?)") + } + xx ++ c } + .text("Run the specified module emitter (one file per module)") +} + // ***** Annotations for results of emission ***** sealed abstract class EmittedComponent { def name: String def value: String + def outputSuffix: String } sealed abstract class EmittedCircuit extends EmittedComponent -final case class EmittedFirrtlCircuit(name: String, value: String) extends EmittedCircuit -final case class EmittedVerilogCircuit(name: String, value: String) extends EmittedCircuit +final case class EmittedFirrtlCircuit(name: String, value: String, outputSuffix: String) extends EmittedCircuit +final case class EmittedVerilogCircuit(name: String, value: String, outputSuffix: String) extends EmittedCircuit sealed abstract class EmittedModule extends EmittedComponent -final case class EmittedFirrtlModule(name: String, value: String) extends EmittedModule -final case class EmittedVerilogModule(name: String, value: String) extends EmittedModule +final case class EmittedFirrtlModule(name: String, value: String, outputSuffix: String) extends EmittedModule +final case class EmittedVerilogModule(name: String, value: String, outputSuffix: String) extends EmittedModule /** Traits for Annotations containing emitted components */ sealed trait EmittedAnnotation[T <: EmittedComponent] extends NoTargetAnnotation { @@ -51,19 +107,21 @@ sealed trait EmittedCircuitAnnotation[T <: EmittedCircuit] extends EmittedAnnota sealed trait EmittedModuleAnnotation[T <: EmittedModule] extends EmittedAnnotation[T] case class EmittedFirrtlCircuitAnnotation(value: EmittedFirrtlCircuit) - extends EmittedCircuitAnnotation[EmittedFirrtlCircuit] + extends EmittedCircuitAnnotation[EmittedFirrtlCircuit] case class EmittedVerilogCircuitAnnotation(value: EmittedVerilogCircuit) - extends EmittedCircuitAnnotation[EmittedVerilogCircuit] + extends EmittedCircuitAnnotation[EmittedVerilogCircuit] case class EmittedFirrtlModuleAnnotation(value: EmittedFirrtlModule) - extends EmittedModuleAnnotation[EmittedFirrtlModule] + extends EmittedModuleAnnotation[EmittedFirrtlModule] case class EmittedVerilogModuleAnnotation(value: EmittedVerilogModule) - extends EmittedModuleAnnotation[EmittedVerilogModule] + extends EmittedModuleAnnotation[EmittedVerilogModule] sealed abstract class FirrtlEmitter(form: CircuitForm) extends Transform with Emitter { def inputForm = form def outputForm = form + val outputSuffix: String = form.outputSuffix + private def emitAllModules(circuit: Circuit): Seq[EmittedFirrtlModule] = { // For a given module, returns a Seq of all modules instantited inside of it def collectInstantiatedModules(mod: Module, map: Map[String, DefModule]): Seq[DefModule] = { @@ -87,15 +145,15 @@ sealed abstract class FirrtlEmitter(form: CircuitForm) extends Transform with Em case ext: ExtModule => ext } val newCircuit = Circuit(m.info, extModules :+ m, m.name) - EmittedFirrtlModule(m.name, newCircuit.serialize) + EmittedFirrtlModule(m.name, newCircuit.serialize, outputSuffix) } } override def execute(state: CircuitState): CircuitState = { val newAnnos = state.annotations.flatMap { case EmitCircuitAnnotation(_) => - Seq(EmittedFirrtlCircuitAnnotation.apply( - EmittedFirrtlCircuit(state.circuit.main, state.circuit.serialize))) + Seq(EmittedFirrtlCircuitAnnotation( + EmittedFirrtlCircuit(state.circuit.main, state.circuit.serialize, outputSuffix))) case EmitAllModulesAnnotation(_) => emitAllModules(state.circuit) map (EmittedFirrtlModuleAnnotation(_)) case _ => Seq() @@ -129,6 +187,7 @@ case class VRandom(width: BigInt) extends Expression { class VerilogEmitter extends SeqTransform with Emitter { def inputForm = LowForm def outputForm = LowForm + val outputSuffix = ".v" val tab = " " def AND(e1: WrappedExpression, e2: WrappedExpression): Expression = { if (e1 == e2) e1.e1 @@ -363,8 +422,8 @@ class VerilogEmitter extends SeqTransform with Emitter { * @return the render reference */ def getRenderer(descriptions: Seq[DescriptionAnnotation], - m: Module, - moduleMap: Map[String, DefModule])(implicit writer: Writer): VerilogRender = { + m: Module, + moduleMap: Map[String, DefModule])(implicit writer: Writer): VerilogRender = { val newMod = new AddDescriptionNodes().executeModule(m, descriptions) newMod match { @@ -384,9 +443,9 @@ class VerilogEmitter extends SeqTransform with Emitter { * @param writer where rendered information is placed. */ class VerilogRender(description: Description, - portDescriptions: Map[String, Description], - m: Module, - moduleMap: Map[String, DefModule])(implicit writer: Writer) { + portDescriptions: Map[String, Description], + m: Module, + moduleMap: Map[String, DefModule])(implicit writer: Writer) { def this(m: Module, moduleMap: Map[String, DefModule])(implicit writer: Writer) { this(EmptyDescription, Map.empty, m, moduleMap)(writer) @@ -458,7 +517,7 @@ class VerilogEmitter extends SeqTransform with Emitter { assigns += Seq("assign ", e, " = ", syn, ";", info) assigns += Seq("`else") assigns += Seq("assign ", e, " = ", garbageCond, " ? ", rand_string(syn.tpe), " : ", syn, - ";", info) + ";", info) assigns += Seq("`endif // RANDOMIZE_GARBAGE_ASSIGN") } @@ -550,7 +609,7 @@ class VerilogEmitter extends SeqTransform with Emitter { initials += Seq("`ifdef RANDOMIZE_MEM_INIT") initials += Seq("for (initvar = 0; initvar < ", bigIntToVLit(s.depth), "; initvar = initvar+1)") initials += Seq(tab, WSubAccess(wref(s.name, s.dataType), index, s.dataType, FEMALE), - " = ", rstring, ";") + " = ", rstring, ";") initials += Seq("`endif // RANDOMIZE_MEM_INIT") } @@ -696,14 +755,14 @@ class VerilogEmitter extends SeqTransform with Emitter { instdeclares += Seq(");") case sx: DefMemory => val fullSize = sx.depth * (sx.dataType match { - case GroundType(IntWidth(width)) => width - }) + case GroundType(IntWidth(width)) => width + }) val decl = if (fullSize > (1 << 29)) "reg /* sparse */" else "reg" declareVectorType(decl, sx.name, sx.dataType, sx.depth, sx.info) initialize_mem(sx) if (sx.readLatency != 0 || sx.writeLatency != 1) throw EmitterException("All memories should be transformed into " + - "blackboxes or combinational by previous passses") + "blackboxes or combinational by previous passses") for (r <- sx.readers) { val data = memPortField(sx, r, "data") val addr = memPortField(sx, r, "addr") @@ -717,7 +776,7 @@ class VerilogEmitter extends SeqTransform with Emitter { //; Read port assign(addr, netlist(addr), NoInfo) // Info should come from addr connection - // assign(en, netlist(en)) //;Connects value to m.r.en + // assign(en, netlist(en)) //;Connects value to m.r.en val mem = WRef(sx.name, memType(sx), MemKind, UNKNOWNGENDER) val memPort = WSubAccess(mem, addr, sx.dataType, UNKNOWNGENDER) val depthValue = UIntLiteral(sx.depth, IntWidth(sx.depth.bitLength)) @@ -756,7 +815,7 @@ class VerilogEmitter extends SeqTransform with Emitter { if (sx.readwriters.nonEmpty) throw EmitterException("All readwrite ports should be transformed into " + - "read & write ports by previous passes") + "read & write ports by previous passes") case _ => } } @@ -918,7 +977,7 @@ class VerilogEmitter extends SeqTransform with Emitter { case EmitCircuitAnnotation(_) => val writer = new java.io.StringWriter emit(state, writer) - Seq(EmittedVerilogCircuitAnnotation(EmittedVerilogCircuit(state.circuit.main, writer.toString))) + Seq(EmittedVerilogCircuitAnnotation(EmittedVerilogCircuit(state.circuit.main, writer.toString, outputSuffix))) case EmitAllModulesAnnotation(_) => val circuit = runTransforms(state).circuit @@ -929,12 +988,12 @@ class VerilogEmitter extends SeqTransform with Emitter { val writer = new java.io.StringWriter val renderer = new VerilogRender(d, pds, module, moduleMap)(writer) renderer.emit_verilog() - Some(EmittedVerilogModuleAnnotation(EmittedVerilogModule(module.name, writer.toString))) + Some(EmittedVerilogModuleAnnotation(EmittedVerilogModule(module.name, writer.toString, outputSuffix))) case module: Module => val writer = new java.io.StringWriter val renderer = new VerilogRender(module, moduleMap)(writer) renderer.emit_verilog() - Some(EmittedVerilogModuleAnnotation(EmittedVerilogModule(module.name, writer.toString))) + Some(EmittedVerilogModuleAnnotation(EmittedVerilogModule(module.name, writer.toString, outputSuffix))) case _ => None } case _ => Seq() @@ -952,3 +1011,9 @@ class MinimumVerilogEmitter extends VerilogEmitter with Emitter { } } + +class SystemVerilogEmitter extends VerilogEmitter { + StageUtils.dramaticWarning("SystemVerilog Emitter is the same as the Verilog Emitter!") + + override val outputSuffix: String = ".sv" +} diff --git a/src/main/scala/firrtl/ExecutionOptionsManager.scala b/src/main/scala/firrtl/ExecutionOptionsManager.scala index 189701dc..3783db28 100644 --- a/src/main/scala/firrtl/ExecutionOptionsManager.scala +++ b/src/main/scala/firrtl/ExecutionOptionsManager.scala @@ -9,7 +9,13 @@ import firrtl.passes.memlib.{InferReadWriteAnnotation, ReplSeqMemAnnotation} import firrtl.passes.clocklist.ClockListAnnotation import firrtl.transforms.NoCircuitDedupAnnotation import logger.LogLevel +import logger.{ClassLogLevelAnnotation, LogClassNamesAnnotation, LogFileAnnotation, LogLevelAnnotation} import scopt.OptionParser +import firrtl.stage.{CompilerAnnotation, FirrtlCircuitAnnotation, FirrtlFileAnnotation, FirrtlSourceAnnotation, + InfoModeAnnotation, OutputFileAnnotation, RunFirrtlTransformAnnotation} +import firrtl.stage.phases.DriverCompatibility.{TopNameAnnotation, EmitOneFilePerModuleAnnotation} +import firrtl.options.{InputAnnotationFileAnnotation, OutputAnnotationFileAnnotation, ProgramArgsAnnotation, StageUtils} +import firrtl.transforms.{DontCheckCombLoopsAnnotation, NoDCEAnnotation} import scala.collection.Seq @@ -19,8 +25,10 @@ import scala.collection.Seq * '''NOTE''' In all derived trait/classes, if you intend on maintaining backwards compatibility, * be sure to add new options at the end of the current ones and don't remove any existing ones. */ +@deprecated("Use firrtl.options.HasScoptOptions and/or library/transform registration", "1.2") trait ComposableOptions +@deprecated("Use firrtl.options.{ExecutionOptionsManager, TerminateOnExit, DuplicateHandling}", "1.2") abstract class HasParser(applicationName: String) { final val parser = new OptionParser[Unit](applicationName) { var terminateOnExit = true @@ -49,6 +57,7 @@ abstract class HasParser(applicationName: String) { * For example, in chisel, by deferring this it is possible for the execute there to first elaborate the * circuit and then set the topName from that if it has not already been set. */ +@deprecated("Use a FirrtlOptionsView, LoggerOptionsView, or construct your own view of an AnnotationSeq", "1.2") case class CommonOptions( topName: String = "", targetDirName: String = ".", @@ -67,8 +76,17 @@ case class CommonOptions( optionsManager.getBuildFileName("log") } } + + def toAnnotations: AnnotationSeq = (if (topName.nonEmpty) Seq(TopNameAnnotation(topName)) else Seq()) ++ + (if (targetDirName != ".") Some(TargetDirAnnotation(targetDirName)) else None) ++ + Some(LogLevelAnnotation(globalLogLevel)) ++ + (if (logToFile) { Some(LogFileAnnotation(None)) } else { None }) ++ + (if (logClassNames) { Some(LogClassNamesAnnotation) } else { None }) ++ + classLogLevels.map{ case (c, v) => ClassLogLevelAnnotation(c, v) } ++ + programArgs.map( a => ProgramArgsAnnotation(a) ) } +@deprecated("Specify command line arguments in an Annotation mixing in HasScoptOptions", "1.2") trait HasCommonOptions { self: ExecutionOptionsManager => var commonOptions = CommonOptions() @@ -169,6 +187,7 @@ final case class OneFilePerModule(targetDir: String) extends OutputConfig * @param compilerName which compiler to use * @param annotations annotations to pass to compiler */ +@deprecated("Use a FirrtlOptionsView or construct your own view of an AnnotationSeq", "1.2") case class FirrtlExecutionOptions( inputFileNameOverride: String = "", outputFileNameOverride: String = "", @@ -287,8 +306,31 @@ extends ComposableOptions { def getAnnotationFileName(optionsManager: ExecutionOptionsManager): String = { optionsManager.getBuildFileName("anno", annotationFileNameOverride) } + + def toAnnotations: AnnotationSeq = { + if (inferRW.nonEmpty) { + StageUtils.dramaticWarning("User set FirrtlExecutionOptions.inferRW, but inferRW has no effect!") + } + + (if (inputFileNameOverride.nonEmpty) Seq(FirrtlFileAnnotation(inputFileNameOverride)) else Seq()) ++ + (if (outputFileNameOverride.nonEmpty) { Some(OutputFileAnnotation(outputFileNameOverride)) } else { None }) ++ + Some(CompilerAnnotation(compilerName)) ++ + Some(InfoModeAnnotation(infoModeName)) ++ + firrtlSource.map(FirrtlSourceAnnotation(_)) ++ + customTransforms.map(t => RunFirrtlTransformAnnotation(t)) ++ + annotations ++ + (if (annotationFileNameOverride.nonEmpty) { Some(InputAnnotationFileAnnotation(annotationFileNameOverride)) } else { None }) ++ + (if (outputAnnotationFileName.nonEmpty) { Some(OutputAnnotationFileAnnotation(outputAnnotationFileName)) } else { None }) ++ + (if (emitOneFilePerModule) { Some(EmitOneFilePerModuleAnnotation) } else { None }) ++ + (if (dontCheckCombLoops) { Some(DontCheckCombLoopsAnnotation) } else { None }) ++ + (if (noDCE) { Some(NoDCEAnnotation) } else { None }) ++ + annotationFileNames.map(InputAnnotationFileAnnotation(_)) ++ + firrtlCircuit.map(FirrtlCircuitAnnotation(_)) + } } + +@deprecated("Specify command line arguments in an Annotation mixing in HasScoptOptions", "1.2") trait HasFirrtlOptions { self: ExecutionOptionsManager => var firrtlOptions = FirrtlExecutionOptions() @@ -333,7 +375,7 @@ trait HasFirrtlOptions { .foreach { _ => val msg = "force-append-anno-file is deprecated and will soon be removed\n" + (" "*9) + "(It does not do anything anymore)" - Driver.dramaticWarning(msg) + StageUtils.dramaticWarning(msg) } parser.opt[String]("output-annotation-file") @@ -489,8 +531,10 @@ trait HasFirrtlOptions { parser.note("") } +@deprecated("Use FirrtlStage and examine the output AnnotationSeq directly", "1.2") sealed trait FirrtlExecutionResult +@deprecated("Use FirrtlStage and examine the output AnnotationSeq directly", "1.2") object FirrtlExecutionSuccess { def apply( emitType : String, @@ -511,6 +555,7 @@ object FirrtlExecutionSuccess { * "sverilog" * @param emitted The emitted result of compilation */ +@deprecated("Use FirrtlStage and examine the output AnnotationSeq directly", "1.2") class FirrtlExecutionSuccess( val emitType: String, val emitted : String, @@ -522,12 +567,14 @@ class FirrtlExecutionSuccess( * * @param message Some kind of hint as to what went wrong. */ +@deprecated("Use FirrtlStage and examine the output AnnotationSeq directly", "1.2") case class FirrtlExecutionFailure(message: String) extends FirrtlExecutionResult /** * * @param applicationName The name shown in the usage */ +@deprecated("Use new FirrtlStage infrastructure", "1.2") class ExecutionOptionsManager(val applicationName: String) extends HasParser(applicationName) with HasCommonOptions { def parse(args: Array[String]): Boolean = { diff --git a/src/main/scala/firrtl/LoweringCompilers.scala b/src/main/scala/firrtl/LoweringCompilers.scala index 262caeea..5bece9fe 100644 --- a/src/main/scala/firrtl/LoweringCompilers.scala +++ b/src/main/scala/firrtl/LoweringCompilers.scala @@ -3,6 +3,7 @@ package firrtl import firrtl.transforms.IdentityTransform +import firrtl.options.StageUtils sealed abstract class CoreTransform extends SeqTransform @@ -134,7 +135,7 @@ import firrtl.transforms.BlackBoxSourceHelper * Primarily useful for changing between .fir and .pb serialized formats */ class NoneCompiler extends Compiler { - def emitter = new ChirrtlEmitter + val emitter = new ChirrtlEmitter def transforms: Seq[Transform] = Seq(new IdentityTransform(ChirrtlForm)) } @@ -142,37 +143,38 @@ class NoneCompiler extends Compiler { * Will replace Chirrtl constructs with Firrtl */ class HighFirrtlCompiler extends Compiler { - def emitter = new HighFirrtlEmitter + val emitter = new HighFirrtlEmitter def transforms: Seq[Transform] = getLoweringTransforms(ChirrtlForm, HighForm) } /** Emits middle Firrtl input circuit */ class MiddleFirrtlCompiler extends Compiler { - def emitter = new MiddleFirrtlEmitter + val emitter = new MiddleFirrtlEmitter def transforms: Seq[Transform] = getLoweringTransforms(ChirrtlForm, MidForm) } /** Emits lowered input circuit */ class LowFirrtlCompiler extends Compiler { - def emitter = new LowFirrtlEmitter + val emitter = new LowFirrtlEmitter def transforms: Seq[Transform] = getLoweringTransforms(ChirrtlForm, LowForm) } /** Emits Verilog */ class VerilogCompiler extends Compiler { - def emitter = new VerilogEmitter + val emitter = new VerilogEmitter def transforms: Seq[Transform] = getLoweringTransforms(ChirrtlForm, LowForm) ++ Seq(new LowFirrtlOptimization) } /** Emits Verilog without optimizations */ class MinimumVerilogCompiler extends Compiler { - def emitter = new MinimumVerilogEmitter + val emitter = new MinimumVerilogEmitter def transforms: Seq[Transform] = getLoweringTransforms(ChirrtlForm, LowForm) ++ Seq(new MinimumLowFirrtlOptimization) } /** Currently just an alias for the [[VerilogCompiler]] */ class SystemVerilogCompiler extends VerilogCompiler { - Driver.dramaticWarning("SystemVerilog Compiler behaves the same as the Verilog Compiler!") + override val emitter = new SystemVerilogEmitter + StageUtils.dramaticWarning("SystemVerilog Compiler behaves the same as the Verilog Compiler!") } diff --git a/src/main/scala/firrtl/annotations/Annotation.scala b/src/main/scala/firrtl/annotations/Annotation.scala index 25e361f3..873bd8fe 100644 --- a/src/main/scala/firrtl/annotations/Annotation.scala +++ b/src/main/scala/firrtl/annotations/Annotation.scala @@ -6,6 +6,7 @@ package annotations import net.jcazevedo.moultingyaml._ import firrtl.annotations.AnnotationYamlProtocol._ import firrtl.Utils.throwInternalError +import firrtl.options.StageUtils import scala.collection.mutable @@ -190,7 +191,7 @@ private[firrtl] object LegacyAnnotation { case other => other } // scalastyle:on - def convertLegacyAnnos(annos: Seq[Annotation]): Seq[Annotation] = { + def convertLegacyAnnos(annos: AnnotationSeq): AnnotationSeq = { var warned: Boolean = false annos.map { case legacy: LegacyAnnotation => @@ -198,7 +199,7 @@ private[firrtl] object LegacyAnnotation { if (!warned && (annox ne legacy)) { val msg = s"A LegacyAnnotation was automatically converted.\n" + (" "*9) + "This functionality will soon be removed. Please migrate to new annotations." - Driver.dramaticWarning(msg) + StageUtils.dramaticWarning(msg) warned = true } annox diff --git a/src/main/scala/firrtl/options/Exceptions.scala b/src/main/scala/firrtl/options/Exceptions.scala new file mode 100644 index 00000000..100ff464 --- /dev/null +++ b/src/main/scala/firrtl/options/Exceptions.scala @@ -0,0 +1,20 @@ +// See LICENSE for license details. + +package firrtl.options + +/** Indicate a generic error in a [[Phase]] + * @param message exception message + * @param cause an underlying Exception that this wraps + */ +class PhaseException(val message: String, cause: Throwable = null) extends RuntimeException(message, cause) + +/** Indicate an error related to a bad [[firrtl.annotations.Annotation Annotation]] or it's command line option + * equivalent. This exception is always caught and converted to an error message by a [[Stage]]. Do not use this for + * communicating generic exception information. + */ +class OptionsException(val message: String, cause: Throwable = null) extends IllegalArgumentException(message, cause) + +/** Indicates that a [[Phase]] is missing some mandatory information. This likely occurs either if a user ran something + * out of order or if the compiler did not run things in the correct order. + */ +class PhasePrerequisiteException(message: String, cause: Throwable = null) extends PhaseException(message, cause) diff --git a/src/main/scala/firrtl/options/OptionParser.scala b/src/main/scala/firrtl/options/OptionParser.scala index 6d8095c0..986c5a8a 100644 --- a/src/main/scala/firrtl/options/OptionParser.scala +++ b/src/main/scala/firrtl/options/OptionParser.scala @@ -2,16 +2,28 @@ package firrtl.options -import firrtl.{FIRRTLException, AnnotationSeq} +import firrtl.AnnotationSeq import scopt.OptionParser -/** Causes an OptionParser to not call exit (call `sys.exit`) if the `--help` option is passed - */ +case object OptionsHelpException extends Exception("Usage help invoked") + +/** OptionParser mixin that causes the OptionParser to not call exit (call `sys.exit`) if the `--help` option is + * passed */ trait DoNotTerminateOnExit { this: OptionParser[_] => override def terminate(exitState: Either[String, Unit]): Unit = Unit } +/** OptionParser mixin that converts to [[OptionsException]] + * + * Scopt, by default, will print errors to stderr, e.g., invalid arguments will do this. However, a [[Stage]] uses + * [[StageUtils.dramaticError]]. By converting this to an [[OptionsException]], a [[Stage]] can then catch the error an + * convert it to an [[OptionsException]] that a [[Stage]] can get at. + */ +trait ExceptOnError { this: OptionParser[_] => + override def reportError(msg: String): Unit = throw new OptionsException(msg) +} + /** A modified OptionParser with mutable termination and additional checks */ trait DuplicateHandling extends OptionParser[AnnotationSeq] { diff --git a/src/main/scala/firrtl/options/OptionsView.scala b/src/main/scala/firrtl/options/OptionsView.scala index aacd997e..49417ded 100644 --- a/src/main/scala/firrtl/options/OptionsView.scala +++ b/src/main/scala/firrtl/options/OptionsView.scala @@ -4,25 +4,25 @@ package firrtl.options import firrtl.AnnotationSeq -/** Type class defining a "view" of an [[AnnotationSeq]] - * @tparam T the type to which this viewer converts an [[AnnotationSeq]] to +/** Type class defining a "view" of an [[firrtl.AnnotationSeq AnnotationSeq]] + * @tparam T the type to which this viewer converts an [[firrtl.AnnotationSeq AnnotationSeq]] to */ trait OptionsView[T] { - /** Convert an [[AnnotationSeq]] to some other type + /** Convert an [[firrtl.AnnotationSeq AnnotationSeq]] to some other type * @param options some annotations */ def view(options: AnnotationSeq): T } -/** A shim to manage multiple "views" of an [[AnnotationSeq]] */ +/** A shim to manage multiple "views" of an [[firrtl.AnnotationSeq AnnotationSeq]] */ object Viewer { /** Convert annotations to options using an implicitly provided [[OptionsView]] * @param options some annotations * @param optionsView a converter of options to the requested type - * @tparam T the type to which the input [[AnnotationSeq]] should be viewed as + * @tparam T the type to which the input [[firrtl.AnnotationSeq AnnotationSeq]] should be viewed as */ def view[T](options: AnnotationSeq)(implicit optionsView: OptionsView[T]): T = optionsView.view(options) diff --git a/src/main/scala/firrtl/options/Phase.scala b/src/main/scala/firrtl/options/Phase.scala index 22715bc2..a660d08a 100644 --- a/src/main/scala/firrtl/options/Phase.scala +++ b/src/main/scala/firrtl/options/Phase.scala @@ -3,18 +3,90 @@ package firrtl.options import firrtl.AnnotationSeq +import firrtl.annotations.DeletedAnnotation -/** A transformation of an [[AnnotationSeq]] +import logger.LazyLogging + +import scala.collection.mutable + +/** A polymorphic mathematical transform + * @tparam A the transformed type + */ +trait TransformLike[A] extends LazyLogging { + + /** An identifier of this [[TransformLike]] that can be used for logging and informational printing */ + def name: String + + /** A mathematical transform on some type + * @param a an input object + * @return an output object of the same type + */ + def transform(a: A): A + +} + +/** A mathematical transformation of an [[AnnotationSeq]]. + * + * A [[Phase]] forms one unit in the Chisel/FIRRTL Hardware Compiler Framework (HCF). The HCF is built from a sequence + * of [[Phase]]s applied to an [[AnnotationSeq]]. Note that a [[Phase]] may consist of multiple phases internally. + */ +abstract class Phase extends TransformLike[AnnotationSeq] { + + /** The name of this [[Phase]]. This will be used to generate debug/error messages or when deleting annotations. This + * will default to the `simpleName` of the class. + * @return this phase's name + * @note Override this with your own implementation for different naming behavior. + */ + lazy val name: String = this.getClass.getName + + /** Perform the transform of [[transform]] on an [[firrtl.AnnotationSeq AnnotationSeq]] and add + * [[firrtl.annotations.DeletedAnnotation DeletedAnnotation]]s for any deleted [[firrtl.annotations.Annotation + * Annotation]]s. + * @param a + */ + final def runTransform(annotations: AnnotationSeq): AnnotationSeq = { + val ax = transform(annotations) + + val (in, out) = (mutable.LinkedHashSet() ++ annotations, mutable.LinkedHashSet() ++ ax) + + (in -- out).map { + case DeletedAnnotation(n, a) => DeletedAnnotation(s"$n+$name", a) + case a => DeletedAnnotation(name, a) + }.toSeq ++ ax + } + +} + +/** A [[TransformLike]] that internally ''translates'' the input type to some other type, transforms the internal type, + * and converts back to the original type. * - * A [[Phase]] forms one block in the Chisel/FIRRTL Hardware Compiler Framework (HCF). Note that a [[Phase]] may - * consist of multiple phases internally. + * This is intended to be used to insert a [[TransformLike]] parameterized by type `B` into a sequence of + * [[TransformLike]]s parameterized by type `A`. + * @tparam A the type of the [[TransformLike]] + * @tparam B the internal type */ -abstract class Phase { +trait Translator[A, B] { this: TransformLike[A] => + + /** A method converting type `A` into type `B` + * @param an object of type `A` + * @return an object of type `B` + */ + protected implicit def aToB(a: A): B + + /** A method converting type `B` back into type `A` + * @param an object of type `B` + * @return an object of type `A` + */ + protected implicit def bToA(b: B): A + + /** A transform on an internal type + * @param b an object of type `B` + * @return an object of type `B` + */ + protected def internalTransform(b: B): B - /** A transformation of an [[AnnotationSeq]] - * @param annotations some annotations - * @return transformed annotations + /** Convert the input object to the internal type, transform the internal type, and convert back to the original type */ - def transform(annotations: AnnotationSeq): AnnotationSeq + final def transform(a: A): A = internalTransform(a) } diff --git a/src/main/scala/firrtl/options/Registration.scala b/src/main/scala/firrtl/options/Registration.scala index 481c095b..a826ec50 100644 --- a/src/main/scala/firrtl/options/Registration.scala +++ b/src/main/scala/firrtl/options/Registration.scala @@ -10,7 +10,14 @@ import scopt.OptionParser */ trait HasScoptOptions { - /** This method will be called to add options to an OptionParser + /** This method will be called to add options to an OptionParser ('''OPTIONS SHOULD BE PREPENDED''') + * + * + * '''The ordering of [[firrtl.annotations.Annotation Annotation]] is important and has meaning for parallel + * compilations. For deterministic behavior, you should always prepend any annotations to the [[firrtl.AnnotationSeq + * AnnotationSeq]]. The [[firrtl.AnnotationSeq AnnotationSeq]] will be automatically reversed after a [[Stage]] + * parses it.''' + * * @param p an option parser */ def addOptions(p: OptionParser[AnnotationSeq]): Unit diff --git a/src/main/scala/firrtl/options/Shell.scala b/src/main/scala/firrtl/options/Shell.scala index 9f0cb8bd..dbc403a5 100644 --- a/src/main/scala/firrtl/options/Shell.scala +++ b/src/main/scala/firrtl/options/Shell.scala @@ -4,22 +4,19 @@ package firrtl.options import firrtl.AnnotationSeq +import logger.{LogLevelAnnotation, ClassLogLevelAnnotation, LogFileAnnotation, LogClassNamesAnnotation} + import scopt.OptionParser import java.util.ServiceLoader -/** Indicate an error in [[firrtl.options]] - * @param msg a message to print - */ -case class OptionsException(msg: String, cause: Throwable = null) extends Exception(msg, cause) - /** A utility for working with command line options * @param applicationName the application associated with these command line options */ class Shell(val applicationName: String) { /** Command line argument parser (OptionParser) with modifications */ - final val parser = new OptionParser[AnnotationSeq](applicationName) with DuplicateHandling + final val parser = new OptionParser[AnnotationSeq](applicationName) with DuplicateHandling with ExceptOnError /** Contains all discovered [[RegisteredLibrary]] */ lazy val registeredLibraries: Seq[RegisteredLibrary] = { @@ -31,6 +28,7 @@ class Shell(val applicationName: String) { parser.note(lib.name) lib.addOptions(parser) } + libraries } @@ -44,6 +42,7 @@ class Shell(val applicationName: String) { transforms += tx tx.addOptions(parser) } + transforms } @@ -53,19 +52,38 @@ class Shell(val applicationName: String) { * line options via methods of [[Shell.parser]] */ def parse(args: Array[String], initAnnos: AnnotationSeq = Seq.empty): AnnotationSeq = { - val rtString = registeredTransforms.map(r => s"\n - ${r.getClass.getName}").mkString - val rlString = registeredLibraries.map(l => s"\n - ${l.getClass.getName}").mkString - parser.note(s"""| - |The following FIRRTL transforms registered command line options:$rtString - |The following libraries registered command line options:$rlString""".stripMargin) - + registeredTransforms + registeredLibraries parser - .parse(args, initAnnos) + .parse(args, initAnnos.reverse) .getOrElse(throw new OptionsException("Failed to parse command line options", new IllegalArgumentException)) + .reverse } parser.note("Shell Options") - Seq( InputAnnotationFileAnnotation(), - TargetDirAnnotation() ) + Seq( TargetDirAnnotation, + ProgramArgsAnnotation, + InputAnnotationFileAnnotation, + OutputAnnotationFileAnnotation ) + .map(_.addOptions(parser)) + + parser.opt[Unit]("show-registrations") + .action{ (_, c) => + val rtString = registeredTransforms.map(r => s"\n - ${r.getClass.getName}").mkString + val rlString = registeredLibraries.map(l => s"\n - ${l.getClass.getName}").mkString + + println(s"""|The following FIRRTL transforms registered command line options:$rtString + |The following libraries registered command line options:$rlString""".stripMargin) + c } + .unbounded() + .text("print discovered registered libraries and transforms") + + parser.help("help").text("prints this usage text") + + parser.note("Logging Options") + Seq( LogLevelAnnotation, + ClassLogLevelAnnotation, + LogFileAnnotation, + LogClassNamesAnnotation ) .map(_.addOptions(parser)) } diff --git a/src/main/scala/firrtl/options/Stage.scala b/src/main/scala/firrtl/options/Stage.scala index 4e74652f..38cc9d42 100644 --- a/src/main/scala/firrtl/options/Stage.scala +++ b/src/main/scala/firrtl/options/Stage.scala @@ -4,7 +4,7 @@ package firrtl.options import firrtl.AnnotationSeq -case class StageException(val str: String, cause: Throwable = null) extends RuntimeException(str, cause) +import logger.Logger /** A [[Stage]] represents one stage in the FIRRTL hardware compiler framework. A [[Stage]] is, conceptually, a * [[Phase]] that includes a command line interface. @@ -19,37 +19,56 @@ abstract class Stage extends Phase { /** A utility that helps convert command line options to annotations */ val shell: Shell - /** Run this [[Stage]] on some input annotations + /** Run this stage on some input annotations * @param annotations input annotations * @return output annotations */ def run(annotations: AnnotationSeq): AnnotationSeq - /** Execute this [[Stage]] on some input annotations. Annotations will be read from any input annotation files. + /** Execute this stage on some input annotations. Annotations will be read from any input annotation files. * @param annotations input annotations * @return output annotations + * @throws OptionsException if command line or annotation validation fails */ final def transform(annotations: AnnotationSeq): AnnotationSeq = { - val preprocessing: Seq[Phase] = Seq( - phases.GetIncludes, - phases.ConvertLegacyAnnotations, - phases.AddDefaults ) + val annotationsx = Seq( new phases.GetIncludes, + new phases.ConvertLegacyAnnotations ) + .foldLeft(annotations)((a, p) => p.runTransform(a)) - val a = preprocessing.foldLeft(annotations)((a, p) => p.transform(a)) - - run(a) + Logger.makeScope(annotationsx) { + Seq( new phases.AddDefaults, + new phases.Checks, + new Phase { def transform(a: AnnotationSeq) = run(a) }, + new phases.WriteOutputAnnotations ) + .foldLeft(annotationsx)((a, p) => p.runTransform(a)) + } } - /** Run this [[Stage]] on on a mix of arguments and annotations + /** Run this stage on on a mix of arguments and annotations * @param args command line arguments * @param initialAnnotations annotation * @return output annotations + * @throws OptionsException if command line or annotation validation fails */ final def execute(args: Array[String], annotations: AnnotationSeq): AnnotationSeq = transform(shell.parse(args, annotations)) - /** The main function that serves as this [[Stage]]'s command line interface +} + +/** Provides a main method for a [[Stage]] + * @param stage the stage to run + */ +class StageMain(val stage: Stage) { + + /** The main function that serves as this stage's command line interface. * @param args command line arguments */ - final def main(args: Array[String]): Unit = execute(args, Seq.empty) + final def main(args: Array[String]): Unit = try { + stage.execute(args, Seq.empty) + } catch { + case a: OptionsException => + StageUtils.dramaticUsageError(a.message) + System.exit(1) + } + } diff --git a/src/main/scala/firrtl/options/StageAnnotations.scala b/src/main/scala/firrtl/options/StageAnnotations.scala index fdad07c3..e8a1a288 100644 --- a/src/main/scala/firrtl/options/StageAnnotations.scala +++ b/src/main/scala/firrtl/options/StageAnnotations.scala @@ -3,40 +3,89 @@ package firrtl.options import firrtl.AnnotationSeq -import firrtl.annotations.NoTargetAnnotation +import firrtl.annotations.{Annotation, NoTargetAnnotation} import scopt.OptionParser -sealed trait StageOption extends HasScoptOptions +sealed trait StageOption { this: Annotation => } + +/** An annotation that should not be serialized automatically [[phases.WriteOutputAnnotations WriteOutputAnnotations]]. + * This usually means that this is an annotation that is used only internally to a [[Stage]]. + */ +trait Unserializable { this: Annotation => } + +/** Holds the name of the target directory + * - set with `-td/--target-dir` + * - if unset, a [[TargetDirAnnotation]] will be generated with the + * @param value target directory name + */ +case class TargetDirAnnotation(directory: String = ".") extends NoTargetAnnotation with StageOption + +object TargetDirAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("target-dir") + .abbr("td") + .valueName("") + .action( (x, c) => TargetDirAnnotation(x) +: c ) + .unbounded() // See [Note 1] + .text(s"Work directory for intermediate files/blackboxes, default is '.' (current directory)") +} + +/** Additional arguments + * - set with any trailing option on the command line + * @param value one [[scala.Predef.String String]] argument + */ +case class ProgramArgsAnnotation(arg: String) extends NoTargetAnnotation with StageOption + +object ProgramArgsAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.arg[String]("...") + .unbounded() + .optional() + .action( (x, c) => ProgramArgsAnnotation(x) +: c ) + .text("optional unbounded args") +} /** Holds a filename containing one or more [[annotations.Annotation]] to be read * - this is not stored in [[FirrtlExecutionOptions]] * - set with `-faf/--annotation-file` * @param value input annotation filename */ -case class InputAnnotationFileAnnotation(value: String) extends NoTargetAnnotation with StageOption { +case class InputAnnotationFileAnnotation(file: String) extends NoTargetAnnotation with StageOption + +object InputAnnotationFileAnnotation extends HasScoptOptions { def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("annotation-file") .abbr("faf") .unbounded() .valueName("") - .action( (x, c) => c :+ InputAnnotationFileAnnotation(x) ) + .action( (x, c) => InputAnnotationFileAnnotation(x) +: c ) .text("Used to specify annotation file") } -object InputAnnotationFileAnnotation { - private [firrtl] def apply(): InputAnnotationFileAnnotation = InputAnnotationFileAnnotation("") +/** An explicit output _annotation_ file to write to + * - set with `-foaf/--output-annotation-file` + * @param value output annotation filename + */ +case class OutputAnnotationFileAnnotation(file: String) extends NoTargetAnnotation with StageOption + +object OutputAnnotationFileAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("output-annotation-file") + .abbr("foaf") + .valueName ("") + .action( (x, c) => OutputAnnotationFileAnnotation(x) +: c ) + .unbounded() + .text("use this to set the annotation output file") } -/** Holds the name of the target directory - * - set with `-td/--target-dir` - * - if unset, a [[TargetDirAnnotation]] will be generated with the - * @param value target directory name +/** If this [[firrtl.annotations.Annotation Annotation]] exists in an [[firrtl.AnnotationSeq AnnotationSeq]], then a + * [[firrtl.options.phase.WriteOutputAnnotations WriteOutputAnnotations]] will include + * [[firrtl.annotations.DeletedAnnotation DeletedAnnotation]]s when it writes to a file. + * - set with '--write-deleted' */ -case class TargetDirAnnotation(dir: String = ".") extends NoTargetAnnotation with StageOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("target-dir") - .abbr("td") - .valueName("") - .action( (x, c) => c ++ Seq(TargetDirAnnotation(x)) ) - .unbounded() // See [Note 1] - .text(s"Work directory for intermediate files/blackboxes, default is '.' (current directory)") +case object WriteDeletedAnnotation extends NoTargetAnnotation with StageOption with HasScoptOptions { + + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p + .opt[Unit]("write-deleted") + .unbounded() + .action( (_, c) => WriteDeletedAnnotation +: c ) + .text("Include deleted annotations in the output annotation file") + } diff --git a/src/main/scala/firrtl/options/StageOptions.scala b/src/main/scala/firrtl/options/StageOptions.scala index af5ea17b..7905799c 100644 --- a/src/main/scala/firrtl/options/StageOptions.scala +++ b/src/main/scala/firrtl/options/StageOptions.scala @@ -7,10 +7,31 @@ import java.io.File /** Options that every stage shares * @param targetDirName a target (build) directory * @param an input annotation file + * @param programArgs explicit program arguments + * @param outputAnnotationFileName an output annotation filename */ -final case class StageOptions( - targetDir: String = TargetDirAnnotation().dir, - annotationFiles: Seq[String] = Seq.empty ) { +class StageOptions private [firrtl] ( + val targetDir: String = TargetDirAnnotation().directory, + val annotationFilesIn: Seq[String] = Seq.empty, + val annotationFileOut: Option[String] = None, + val programArgs: Seq[String] = Seq.empty, + val writeDeleted: Boolean = false ) { + + private [options] def copy( + targetDir: String = targetDir, + annotationFilesIn: Seq[String] = annotationFilesIn, + annotationFileOut: Option[String] = annotationFileOut, + programArgs: Seq[String] = programArgs, + writeDeleted: Boolean = writeDeleted ): StageOptions = { + + new StageOptions( + targetDir = targetDir, + annotationFilesIn = annotationFilesIn, + annotationFileOut = annotationFileOut, + programArgs = programArgs, + writeDeleted = writeDeleted ) + + } /** Generate a filename (with an optional suffix) and create any parent directories. Suffix is only added if it is not * already there. @@ -18,7 +39,6 @@ final case class StageOptions( * @param suffix an optional suffix that the file must end in * @return the name of the file * @note the filename may include a path - * @throws IllegalArgumentException if the filename is empty or if the suffix doesn't start with a '.' */ def getBuildFileName(filename: String, suffix: Option[String] = None): String = { require(filename.nonEmpty, "requested filename must not be empty") diff --git a/src/main/scala/firrtl/options/StageUtils.scala b/src/main/scala/firrtl/options/StageUtils.scala index ebbbdd03..cf7cc767 100644 --- a/src/main/scala/firrtl/options/StageUtils.scala +++ b/src/main/scala/firrtl/options/StageUtils.scala @@ -27,7 +27,11 @@ object StageUtils { println("-"*78 + Console.RESET) } - // def canonicalFileName(suffix: String, directory: String = TargetDirAnnotation().targetDirName) { - // } + /** Generate a message suggesting that the user look at the usage text. + * @param message the error message + */ + def dramaticUsageError(message: String): Unit = + dramaticError(s"""|$message + |Try --help for more information.""".stripMargin) } diff --git a/src/main/scala/firrtl/options/package.scala b/src/main/scala/firrtl/options/package.scala index a0dcc194..8cf2875b 100644 --- a/src/main/scala/firrtl/options/package.scala +++ b/src/main/scala/firrtl/options/package.scala @@ -7,10 +7,15 @@ package object options { implicit object StageOptionsView extends OptionsView[StageOptions] { def view(options: AnnotationSeq): StageOptions = options .collect { case a: StageOption => a } - .foldLeft(StageOptions())((c, x) => + .foldLeft(new StageOptions())((c, x) => x match { case TargetDirAnnotation(a) => c.copy(targetDir = a) - case InputAnnotationFileAnnotation(a) => c.copy(annotationFiles = a +: c.annotationFiles) + /* Insert input files at the head of the Seq for speed and because order shouldn't matter */ + case InputAnnotationFileAnnotation(a) => c.copy(annotationFilesIn = a +: c.annotationFilesIn) + case OutputAnnotationFileAnnotation(a) => c.copy(annotationFileOut = Some(a)) + /* Do NOT reorder program args. The order may matter. */ + case ProgramArgsAnnotation(a) => c.copy(programArgs = c.programArgs :+ a) + case WriteDeletedAnnotation => c.copy(writeDeleted = true) } ) } diff --git a/src/main/scala/firrtl/options/phases/AddDefaults.scala b/src/main/scala/firrtl/options/phases/AddDefaults.scala index f0749b22..2d4e4e40 100644 --- a/src/main/scala/firrtl/options/phases/AddDefaults.scala +++ b/src/main/scala/firrtl/options/phases/AddDefaults.scala @@ -10,14 +10,10 @@ import firrtl.options.{Phase, StageOption, TargetDirAnnotation} * This currently only adds a [[TargetDirAnnotation]]. This isn't necessary for a [[StageOptionsView]], but downstream * tools may expect a [[TargetDirAnnotation]] to exist. */ -object AddDefaults extends Phase { +class AddDefaults extends Phase { def transform(annotations: AnnotationSeq): AnnotationSeq = { - var td = true - annotations.collect { case a: StageOption => a }.map { - case _: TargetDirAnnotation => td = false - case _ => - } + val td = annotations.collectFirst{ case a: TargetDirAnnotation => a}.isEmpty (if (td) Seq(TargetDirAnnotation()) else Seq()) ++ annotations diff --git a/src/main/scala/firrtl/options/phases/Checks.scala b/src/main/scala/firrtl/options/phases/Checks.scala new file mode 100644 index 00000000..0691e9b0 --- /dev/null +++ b/src/main/scala/firrtl/options/phases/Checks.scala @@ -0,0 +1,43 @@ +// See LICENSE for license details. + +package firrtl.options.phases + +import firrtl.AnnotationSeq +import firrtl.annotations.Annotation +import firrtl.options.{OptionsException, OutputAnnotationFileAnnotation, Phase, TargetDirAnnotation} + +/** [[firrtl.options.Phase Phase]] that validates an [[AnnotationSeq]]. If successful, views of this [[AnnotationSeq]] + * as [[StageOptions]] are guaranteed to succeed. + */ +class Checks extends Phase { + + /** Validate an [[AnnotationSeq]] for [[StageOptions]] + * @throws OptionsException if annotations are invalid + */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + + val td, outA = collection.mutable.ListBuffer[Annotation]() + annotations.foreach { + case a: TargetDirAnnotation => td += a + case a: OutputAnnotationFileAnnotation => outA += a + case _ => + } + + if (td.size != 1) { + val d = td.map{ case TargetDirAnnotation(x) => x } + throw new OptionsException( + s"""|Exactly one target directory must be specified, but found `${d.mkString(", ")}` specified via: + | - explicit target directory: -td, --target-dir, TargetDirAnnotation + | - fallback default value""".stripMargin )} + + if (outA.size > 1) { + val x = outA.map{ case OutputAnnotationFileAnnotation(x) => x } + throw new OptionsException( + s"""|At most one output annotation file can be specified, but found '${x.mkString(", ")}' specified via: + | - an option or annotation: -foaf, --output-annotation-file, OutputAnnotationFileAnnotation""" + .stripMargin )} + + annotations + } + +} diff --git a/src/main/scala/firrtl/options/phases/ConvertLegacyAnnotations.scala b/src/main/scala/firrtl/options/phases/ConvertLegacyAnnotations.scala index 7ff05370..39f59572 100644 --- a/src/main/scala/firrtl/options/phases/ConvertLegacyAnnotations.scala +++ b/src/main/scala/firrtl/options/phases/ConvertLegacyAnnotations.scala @@ -7,7 +7,7 @@ import firrtl.annotations.LegacyAnnotation import firrtl.options.Phase /** Convert any [[firrtl.annotations.LegacyAnnotation LegacyAnnotation]]s to non-legacy variants */ -object ConvertLegacyAnnotations extends Phase { +class ConvertLegacyAnnotations extends Phase { def transform(annotations: AnnotationSeq): AnnotationSeq = LegacyAnnotation.convertLegacyAnnos(annotations) diff --git a/src/main/scala/firrtl/options/phases/GetIncludes.scala b/src/main/scala/firrtl/options/phases/GetIncludes.scala index 8156dbbf..9e198c61 100644 --- a/src/main/scala/firrtl/options/phases/GetIncludes.scala +++ b/src/main/scala/firrtl/options/phases/GetIncludes.scala @@ -5,7 +5,7 @@ package firrtl.options.phases import net.jcazevedo.moultingyaml._ import firrtl.AnnotationSeq -import firrtl.annotations.{AnnotationFileNotFoundException, DeletedAnnotation, JsonProtocol, LegacyAnnotation} +import firrtl.annotations.{AnnotationFileNotFoundException, JsonProtocol, LegacyAnnotation} import firrtl.annotations.AnnotationYamlProtocol._ import firrtl.options.{InputAnnotationFileAnnotation, Phase, StageUtils} @@ -15,7 +15,7 @@ import scala.collection.mutable import scala.util.{Try, Failure} /** Recursively expand all [[InputAnnotationFileAnnotation]]s in an [[AnnotationSeq]] */ -object GetIncludes extends Phase { +class GetIncludes extends Phase { /** Read all [[annotations.Annotation]] from a file in JSON or YAML format * @param filename a JSON or YAML file of [[annotations.Annotation]] @@ -46,15 +46,14 @@ object GetIncludes extends Phase { */ private def getIncludes(includeGuard: mutable.Set[String] = mutable.Set()) (annos: AnnotationSeq): AnnotationSeq = { - val phaseName = this.getClass.getName annos.flatMap { case a @ InputAnnotationFileAnnotation(value) => if (includeGuard.contains(value)) { - StageUtils.dramaticWarning("Tried to import the same annotation file twice! (Did you include it twice?)") - Seq(DeletedAnnotation(phaseName, a)) + StageUtils.dramaticWarning(s"Annotation file ($value) already included! (Did you include it more than once?)") + None } else { includeGuard += value - DeletedAnnotation(phaseName, a) +: getIncludes(includeGuard)(readAnnotationsFromFile(value)) + getIncludes(includeGuard)(readAnnotationsFromFile(value)) } case x => Seq(x) } diff --git a/src/main/scala/firrtl/options/phases/WriteOutputAnnotations.scala b/src/main/scala/firrtl/options/phases/WriteOutputAnnotations.scala new file mode 100644 index 00000000..66f40d3c --- /dev/null +++ b/src/main/scala/firrtl/options/phases/WriteOutputAnnotations.scala @@ -0,0 +1,36 @@ +// See LICENSE for license details. + +package firrtl.options.phases + +import firrtl.AnnotationSeq +import firrtl.annotations.{DeletedAnnotation, JsonProtocol} +import firrtl.options.{Phase, StageOptions, Unserializable, Viewer} + +import java.io.PrintWriter + +/** [[firrtl.options.Phase Phase]] that writes an [[AnnotationSeq]] to a file. A file is written if and only if a + * [[StageOptions]] view has a non-empty [[StageOptions.annotationFileOut annotationFileOut]]. + */ +class WriteOutputAnnotations extends Phase { + + /** Write the input [[AnnotationSeq]] to a fie. */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + val sopts = Viewer.view[StageOptions](annotations) + val serializable = annotations.filter{ + case _: Unserializable => false + case _: DeletedAnnotation => sopts.writeDeleted + case _ => true + } + + sopts.annotationFileOut match { + case None => + case Some(file) => + val pw = new PrintWriter(sopts.getBuildFileName(file, Some(".anno.json"))) + pw.write(JsonProtocol.serialize(serializable)) + pw.close() + } + + annotations + } + +} diff --git a/src/main/scala/firrtl/package.scala b/src/main/scala/firrtl/package.scala index bc23049c..fc07e05d 100644 --- a/src/main/scala/firrtl/package.scala +++ b/src/main/scala/firrtl/package.scala @@ -8,9 +8,9 @@ package object firrtl { implicit def annoSeqToSeq(as: AnnotationSeq): Seq[Annotation] = as.underlying /* Options as annotations compatibility items */ - @deprecated("Use firrtl.stage.TargetDirAnnotation", "3.2") - type TargetDirAnnotation = firrtl.stage.TargetDirAnnotation + @deprecated("Use firrtl.stage.TargetDirAnnotation", "1.2") + type TargetDirAnnotation = firrtl.options.TargetDirAnnotation - @deprecated("Use firrtl.stage.TargetDirAnnotation", "3.2") - val TargetDirAnnotation = firrtl.stage.TargetDirAnnotation + @deprecated("Use firrtl.stage.TargetDirAnnotation", "1.2") + val TargetDirAnnotation = firrtl.options.TargetDirAnnotation } diff --git a/src/main/scala/firrtl/passes/memlib/InferReadWrite.scala b/src/main/scala/firrtl/passes/memlib/InferReadWrite.scala index 3494de45..f524d60b 100644 --- a/src/main/scala/firrtl/passes/memlib/InferReadWrite.scala +++ b/src/main/scala/firrtl/passes/memlib/InferReadWrite.scala @@ -155,7 +155,7 @@ class InferReadWrite extends Transform with SeqTransformBased with HasScoptOptio .opt[Unit]("infer-rw") .abbr("firw") .valueName ("") - .action( (_, c) => c ++ Seq(InferReadWriteAnnotation, RunFirrtlTransformAnnotation(new InferReadWrite)) ) + .action( (_, c) => Seq(InferReadWriteAnnotation, RunFirrtlTransformAnnotation(new InferReadWrite)) ++ c ) .maxOccurs(1) .text("Enable readwrite port inference for the target circuit") diff --git a/src/main/scala/firrtl/passes/memlib/ReplaceMemTransform.scala b/src/main/scala/firrtl/passes/memlib/ReplaceMemTransform.scala index 1f8e89be..32d83181 100644 --- a/src/main/scala/firrtl/passes/memlib/ReplaceMemTransform.scala +++ b/src/main/scala/firrtl/passes/memlib/ReplaceMemTransform.scala @@ -110,8 +110,8 @@ class ReplSeqMem extends Transform with HasScoptOptions { .opt[String]("repl-seq-mem") .abbr("frsq") .valueName ("-c::-i::-o:") - .action( (x, c) => c ++ Seq(passes.memlib.ReplSeqMemAnnotation.parse(x), - RunFirrtlTransformAnnotation(new ReplSeqMem)) ) + .action( (x, c) => Seq(passes.memlib.ReplSeqMemAnnotation.parse(x), + RunFirrtlTransformAnnotation(new ReplSeqMem)) ++ c ) .maxOccurs(1) .text("Replace sequential memories with blackboxes + configuration file") diff --git a/src/main/scala/firrtl/stage/FirrtlAnnotations.scala b/src/main/scala/firrtl/stage/FirrtlAnnotations.scala index c88591fb..600e825b 100644 --- a/src/main/scala/firrtl/stage/FirrtlAnnotations.scala +++ b/src/main/scala/firrtl/stage/FirrtlAnnotations.scala @@ -4,260 +4,165 @@ package firrtl.stage import firrtl._ import firrtl.ir.Circuit -import firrtl.annotations.NoTargetAnnotation -import firrtl.transforms.BlackBoxTargetDirAnno -import firrtl.options.HasScoptOptions - -import logger.LogLevel +import firrtl.annotations.{Annotation, NoTargetAnnotation} +import firrtl.options.{HasScoptOptions, OptionsException} import scopt.OptionParser -/** Indicates that a subclass is an [[firrtl.annotations Annotation]] that includes a command line option - * - * This must be mixed into a subclass of [[annotations.Annotation]] - */ -sealed trait FirrtlOption extends HasScoptOptions +import java.io.FileNotFoundException +import java.nio.file.NoSuchFileException -/** Holds the name of the top module - * - set on the command line with `-tn/--top-name` - * @param value top module name +/** Indicates that this is an [[firrtl.annotations.Annotation Annotation]] directly used in the construction of a + * [[FirrtlOptions]] view. */ -case class TopNameAnnotation(topName: String) extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("top-name") - .abbr("tn") - .valueName("") - .action( (x, c) => c :+ TopNameAnnotation(x) ) - .unbounded() // See [Note 1] - .text("This options defines the top level circuit, defaults to dut when possible") -} - -object TopNameAnnotation { - private [firrtl] def apply(): TopNameAnnotation = TopNameAnnotation(topName = "") -} +sealed trait FirrtlOption { this: Annotation => } -/** Holds the name of the target directory - * - set with `-td/--target-dir` - * - if unset, a [[TargetDirAnnotation]] will be generated with the - * @param value target directory name +/** Indicates that this [[firrtl.annotations.Annotation Annotation]] contains information that is directly convertable + * to a FIRRTL [[firrtl.ir.Circuit Circuit]]. */ -case class TargetDirAnnotation(targetDirName: String = ".") extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("target-dir") - .abbr("td") - .valueName("") - .action( (x, c) => c ++ Seq(TargetDirAnnotation(x)) ) - .unbounded() // See [Note 1] - .text(s"Work directory for intermediate files/blackboxes, default is ${CommonOptions().targetDirName}") -} +sealed trait CircuitOption extends { this: Annotation => -/** Describes the verbosity of information to log - * - set with `-ll/--log-level` - * - if unset, a [[LogLevelAnnotation]] with the default log level will be emitted - * @param level the level of logging - */ -case class LogLevelAnnotation(globalLogLevel: LogLevel.Value = LogLevel.None) extends NoTargetAnnotation with FirrtlOption { - val value = globalLogLevel.toString + /** Convert this [[firrtl.annotations.Annotation Annotation]] to a [[FirrtlCircuitAnnotation]] + */ + def toCircuit(info: Parser.InfoMode): FirrtlCircuitAnnotation - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("log-level") - .abbr("ll") - .valueName("") - .action( (x, c) => c :+ LogLevelAnnotation(LogLevel(x)) ) - .validate{ x => - lazy val msg = s"$x bad value must be one of error|warn|info|debug|trace" - if (Array("error", "warn", "info", "debug", "trace").contains(x.toLowerCase)) { p.success } - else { p.failure(msg) }} - .unbounded() // See [Note 1] - .text(s"Sets the verbosity level of logging, default is ${CommonOptions().globalLogLevel}") } -/** Describes a mapping of a class to a specific log level - * - set with `-cll/--class-log-level` - * @param name the class name to log - * @param level the verbosity level - */ -case class ClassLogLevelAnnotation(className: String, level: LogLevel.Value) extends NoTargetAnnotation - with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[Seq[String]]("class-log-level") - .abbr("cll") - .valueName("[,...]") - .action( (x, c) => c ++ (x.map { y => - val className :: levelName :: _ = y.split(":").toList - val level = LogLevel(levelName) - ClassLogLevelAnnotation(className, level) }) ) - .unbounded() // This can actually occur any number of times safely - .text(s"This defines per-class verbosity of logging") -} - -object ClassLogLevelAnnotation { - private [firrtl] def apply(): ClassLogLevelAnnotation = ClassLogLevelAnnotation("", LogLevel.None) -} - -/** Enables logging to a file (as opposed to STDOUT) - * - enabled with `-ltf/--log-to-file` - */ -case object LogToFileAnnotation extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[Unit]("log-to-file") - .abbr("ltf") - .action( (x, c) => c :+ LogToFileAnnotation ) - .unbounded() - .text(s"default logs to stdout, this flags writes to topName.log or firrtl.log if no topName") -} - -/** Enables class names in log output - * - enabled with `-lcn/--log-class-names` +/** An explicit input FIRRTL file to read + * - set with `-i/--input-file` + * - If unset, an [[FirrtlFileAnnotation]] with the default input file __will not be generated__ + * @param file input filename */ -case object LogClassNamesAnnotation extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[Unit]("log-class-names") - .abbr("lcn") - .action( (x, c) => c :+ LogClassNamesAnnotation ) - .unbounded() - .text(s"shows class names and log level in logging output, useful for target --class-log-level") -} +case class FirrtlFileAnnotation(file: String) extends NoTargetAnnotation with CircuitOption { -/** Additional arguments - * - set with any trailing option on the command line - * @param value one [[scala.Predef.String String]] argument - */ -case class ProgramArgsAnnotation(value: String) extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.arg[String]("...") - .unbounded() - .optional() - .action( (x, c) => c :+ ProgramArgsAnnotation(x) ) - .text("optional unbounded args") -} + def toCircuit(info: Parser.InfoMode): FirrtlCircuitAnnotation = { + val circuit = try { + FirrtlStageUtils.getFileExtension(file) match { + case ProtoBufFile => proto.FromProto.fromFile(file) + case FirrtlFile => Parser.parseFile(file, info) } + } catch { + case a @ (_: FileNotFoundException | _: NoSuchFileException) => + throw new OptionsException(s"Input file '$file' not found! (Did you misspell it?)", a) + } + FirrtlCircuitAnnotation(circuit) + } -object ProgramArgsAnnotation { - private [firrtl] def apply(): ProgramArgsAnnotation = ProgramArgsAnnotation("") } -/** An explicit input FIRRTL file to read - * - set with `-i/--input-file` - * - If unset, an [[InputFileAnnotation]] with the default input file __will not be generated__ - * @param value input filename - */ -case class InputFileAnnotation(value: String) extends NoTargetAnnotation with FirrtlOption { +object FirrtlFileAnnotation extends HasScoptOptions { def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("input-file") .abbr("i") .valueName ("") - .action( (x, c) => c :+ InputFileAnnotation(x) ) - .unbounded() // See [Note 1] + .action( (x, c) => FirrtlFileAnnotation(x) +: c ) + .unbounded() .text("use this to override the default input file name, default is empty") } -object InputFileAnnotation { - private [firrtl] def apply(): InputFileAnnotation = InputFileAnnotation("") -} - /** An explicit output file the emitter will write to * - set with `-o/--output-file` - * @param value output filename + * @param file output filename */ -case class OutputFileAnnotation(value: String) extends NoTargetAnnotation with FirrtlOption { +case class OutputFileAnnotation(file: String) extends NoTargetAnnotation with FirrtlOption + +object OutputFileAnnotation extends HasScoptOptions { def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("output-file") .abbr("o") .valueName("") - .action( (x, c) => c :+ OutputFileAnnotation(x) ) + .action( (x, c) => OutputFileAnnotation(x) +: c ) .unbounded() .text("use this to override the default output file name, default is empty") } -object OutputFileAnnotation { - private [firrtl] def apply(): OutputFileAnnotation = OutputFileAnnotation("") -} - -/** An explicit output _annotation_ file to write to - * - set with `-foaf/--output-annotation-file` - * @param value output annotation filename - */ -case class OutputAnnotationFileAnnotation(value: String) extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("output-annotation-file") - .abbr("foaf") - .valueName ("") - .action( (x, c) => c :+ OutputAnnotationFileAnnotation(x) ) - .unbounded() // See [Note 1] - .text("use this to set the annotation output file") -} - -object OutputAnnotationFileAnnotation { - private [firrtl] def apply(): OutputAnnotationFileAnnotation = OutputAnnotationFileAnnotation("") -} - /** Sets the info mode style * - set with `--info-mode` - * @param value info mode name - */ -case class InfoModeAnnotation(value: String = "append") extends NoTargetAnnotation with FirrtlOption { + * @param mode info mode name + * @note This cannote be directly converted to [[Parser.InfoMode]] as that depends on an optional [[FirrtlFileAnnotation]] + */ +case class InfoModeAnnotation(modeName: String = "use") extends NoTargetAnnotation with FirrtlOption { + require(modeName match { case "use" | "ignore" | "gen" | "append" => true; case _ => false }, + s"Unknown info mode '$modeName'! (Did you misspell it?)") + + /** Return the [[Parser.InfoMode]] equivalent for this [[firrtl.annotations.Annotation Annotation]] + * @param infoSource the name of a file to use for "gen" or "append" info modes + */ + def toInfoMode(infoSource: Option[String] = None): Parser.InfoMode = modeName match { + case "use" => Parser.UseInfo + case "ignore" => Parser.IgnoreInfo + case _ => + val a = infoSource.getOrElse("unknown source") + modeName match { + case "gen" => Parser.GenInfo(a) + case "append" => Parser.AppendInfo(a) + } + } +} + +object InfoModeAnnotation extends HasScoptOptions { def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("info-mode") .valueName ("") - .action( (x, c) => c :+ InfoModeAnnotation(x.toLowerCase) ) - .unbounded() // See [Note 1] - .text(s"specifies the source info handling, default is ${FirrtlExecutionOptions().infoModeName}") + .action( (x, c) => InfoModeAnnotation(x) +: c ) + .unbounded() + .text(s"specifies the source info handling, default is ${apply().modeName}") } /** Holds a [[scala.Predef.String String]] containing FIRRTL source to read as input * - set with `--firrtl-source` * @param value FIRRTL source as a [[scala.Predef.String String]] */ -case class FirrtlSourceAnnotation(value: String) extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("firrtl-source") - .valueName ("A FIRRTL string") - .action( (x, c) => c :+ FirrtlSourceAnnotation(x) ) - .unbounded() // See [Note 1] - .text(s"A FIRRTL circuit as a string") -} +case class FirrtlSourceAnnotation(source: String) extends NoTargetAnnotation with CircuitOption { + + def toCircuit(info: Parser.InfoMode): FirrtlCircuitAnnotation = + FirrtlCircuitAnnotation(Parser.parseString(source, info)) -object FirrtlSourceAnnotation { - private [firrtl] def apply(): FirrtlSourceAnnotation = FirrtlSourceAnnotation("") } -/** Indicates that an emitted circuit (FIRRTL, Verilog, etc.) will be one file per module - * - set with `--split-modules` - */ -case object EmitOneFilePerModuleAnnotation extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[Unit]("split-modules") - .abbr("fsm") - .action( (x, c) => c :+ EmitOneFilePerModuleAnnotation ) +object FirrtlSourceAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("firrtl-source") + .valueName ("A FIRRTL string") + .action( (x, c) => FirrtlSourceAnnotation(x) +: c ) .unbounded() - .text ("Emit each module to its own file in the target directory.") + .text(s"A FIRRTL circuit as a string") } -/** Holds a filename containing one or more [[annotations.Annotation]] to be read - * - this is not stored in [[FirrtlExecutionOptions]] - * - set with `-faf/--annotation-file` - * @param value input annotation filename +/** Holds a [[Compiler]] that should be run + * - set stringly with `-X/--compiler` + * - If unset, a [[CompilerAnnotation]] with the default [[VerilogCompiler]] + * @param compiler compiler name */ -case class InputAnnotationFileAnnotation(value: String) extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("annotation-file") - .abbr("faf") - .unbounded() - .valueName("") - .action( (x, c) => c :+ InputAnnotationFileAnnotation(x) ) - .text("Used to specify annotation file") -} +case class CompilerAnnotation(compiler: Compiler = new VerilogCompiler()) extends NoTargetAnnotation with FirrtlOption -object InputAnnotationFileAnnotation { - private [firrtl] def apply(): InputAnnotationFileAnnotation = InputAnnotationFileAnnotation("") -} +object CompilerAnnotation extends HasScoptOptions { + + private [firrtl] def apply(compilerName: String): CompilerAnnotation = { + val c = compilerName match { + case "none" => new NoneCompiler() + case "high" => new HighFirrtlCompiler() + case "low" => new LowFirrtlCompiler() + case "middle" => new MiddleFirrtlCompiler() + case "verilog" => new VerilogCompiler() + case "mverilog" => new MinimumVerilogCompiler() + case "sverilog" => new SystemVerilogCompiler() + case _ => throw new OptionsException(s"Unknown compiler name '$compilerName'! (Did you misspell it?)") + } + CompilerAnnotation(c) + } -/** Holds the name of the compiler to run - * - set with `-X/--compiler` - * - If unset, a [[CompilerNameAnnotation]] with the default compiler ("verilog") __will be generated__ - * @param value compiler name - */ -case class CompilerNameAnnotation(value: String = "verilog") extends NoTargetAnnotation with FirrtlOption { def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("compiler") .abbr("X") - .valueName ("") - .action( (x, c) => c :+ CompilerNameAnnotation(x) ) - .unbounded() // See [Note 1] + .valueName ("") + .action{ (x, c) => CompilerAnnotation(x) +: c } + .unbounded() .text(s"compiler to use, default is 'verilog'") } /** Holds the unambiguous class name of a [[Transform]] to run * - will be append to [[FirrtlExecutionOptions.customTransforms]] * - set with `-fct/--custom-transforms` - * @param value the full class name of the transform + * @param transform the full class name of the transform */ -case class RunFirrtlTransformAnnotation(transform: Transform) extends NoTargetAnnotation with FirrtlOption { +case class RunFirrtlTransformAnnotation(transform: Transform) extends NoTargetAnnotation + +object RunFirrtlTransformAnnotation extends HasScoptOptions { def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[Seq[String]]("custom-transforms") .abbr("fct") .valueName (".") @@ -265,26 +170,29 @@ case class RunFirrtlTransformAnnotation(transform: Transform) extends NoTargetAn x.map(txName => try { Class.forName(txName).asInstanceOf[Class[_ <: Transform]].newInstance() } catch { - case e: ClassNotFoundException => throw new FIRRTLException( + case e: ClassNotFoundException => throw new OptionsException( s"Unable to locate custom transform $txName (did you misspell it?)", e) - case e: InstantiationException => throw new FIRRTLException( + case e: InstantiationException => throw new OptionsException( s"Unable to create instance of Transform $txName (is this an anonymous class?)", e) - case e: Throwable => throw new FIRRTLException( + case e: Throwable => throw new OptionsException( s"Unknown error when instantiating class $txName", e) } ) p.success } ) - .action( (x, c) => c ++ x.map(txName => - RunFirrtlTransformAnnotation(Class.forName(txName).asInstanceOf[Class[_ <: Transform]].newInstance())) ) + .action( (x, c) => + x.map(txName => + RunFirrtlTransformAnnotation(Class.forName(txName).asInstanceOf[Class[_ <: Transform]].newInstance())) + .reverse ++ c ) .unbounded() .text("runs these custom transforms during compilation.") } -object RunFirrtlTransformAnnotation { - private [firrtl] def apply(): RunFirrtlTransformAnnotation = RunFirrtlTransformAnnotation(new firrtl.transforms.VerilogRename) -} - /** Holds a FIRRTL [[firrtl.ir.Circuit Circuit]] - * @param value a circuit + * @param circuit a circuit */ -case class FirrtlCircuitAnnotation(value: Circuit) extends NoTargetAnnotation with FirrtlOption { - def addOptions(p: OptionParser[AnnotationSeq]): Unit = Unit +case class FirrtlCircuitAnnotation(circuit: Circuit) extends NoTargetAnnotation with FirrtlOption { + /* Caching the hashCode for a large circuit is necessary due to repeated queries, e.g., in + * [[Compiler.propagateAnnotations]]. Not caching the hashCode will cause severe performance degredations for large + * [[Annotations]]. + */ + override lazy val hashCode: Int = circuit.hashCode + } diff --git a/src/main/scala/firrtl/stage/FirrtlCli.scala b/src/main/scala/firrtl/stage/FirrtlCli.scala new file mode 100644 index 00000000..15c9069e --- /dev/null +++ b/src/main/scala/firrtl/stage/FirrtlCli.scala @@ -0,0 +1,25 @@ +// See LICENSE for license details. + +package firrtl.stage + +import firrtl.options.Shell + +/** [[firrtl.options.Shell Shell]] mixin that provides command line options for FIRRTL. This does not include any + * [[firrtl.options.RegisteredLibrary RegisteredLibrary]] or [[firrtl.options.RegisteredTransform RegisteredTransform]] + * as those are automatically loaded by the [[firrtl.options.Stage Stage]] using this [[firrtl.options.Shell Shell]]. + */ +trait FirrtlCli { this: Shell => + parser.note("FIRRTL Compiler Options") + Seq( FirrtlFileAnnotation, + OutputFileAnnotation, + InfoModeAnnotation, + FirrtlSourceAnnotation, + CompilerAnnotation, + RunFirrtlTransformAnnotation, + firrtl.EmitCircuitAnnotation, + firrtl.EmitAllModulesAnnotation ) + .map(_.addOptions(parser)) + + phases.DriverCompatibility.TopNameAnnotation.addOptions(parser) + phases.DriverCompatibility.EmitOneFilePerModuleAnnotation.addOptions(parser) +} diff --git a/src/main/scala/firrtl/stage/FirrtlOptions.scala b/src/main/scala/firrtl/stage/FirrtlOptions.scala new file mode 100644 index 00000000..235f82c0 --- /dev/null +++ b/src/main/scala/firrtl/stage/FirrtlOptions.scala @@ -0,0 +1,32 @@ +// See LICENSE for license details. + +package firrtl.stage + +import firrtl.{Compiler, Transform} +import firrtl.ir.Circuit + +/** Internal options used to control the FIRRTL compiler stage. + * @param outputFileName output file, default: `targetDir/topName.SUFFIX` with `SUFFIX` as determined by the compiler + * @param compiler which compiler to use (default: [[VerilogCompiler]]) + * @param infoModeName the policy for generating [[firrtl.ir Info]] when processing FIRRTL (default: "append") + * @param firrtlCircuit a [[firrtl.ir Circuit]] + */ +class FirrtlOptions private [stage] ( + val outputFileName: Option[String] = None, + val compiler: Compiler = CompilerAnnotation().compiler, + val infoModeName: String = InfoModeAnnotation().modeName, + val firrtlCircuit: Option[Circuit] = None) { + + private [stage] def copy( + outputFileName: Option[String] = outputFileName, + compiler: Compiler = compiler, + infoModeName: String = infoModeName, + firrtlCircuit: Option[Circuit] = firrtlCircuit ): FirrtlOptions = { + + new FirrtlOptions( + outputFileName = outputFileName, + compiler = compiler, + infoModeName = infoModeName, + firrtlCircuit = firrtlCircuit ) + } +} diff --git a/src/main/scala/firrtl/stage/FirrtlStage.scala b/src/main/scala/firrtl/stage/FirrtlStage.scala new file mode 100644 index 00000000..e4620913 --- /dev/null +++ b/src/main/scala/firrtl/stage/FirrtlStage.scala @@ -0,0 +1,39 @@ +// See LICENSE for license details. + +package firrtl.stage + +import firrtl.{AnnotationSeq, CustomTransformException, FIRRTLException, Utils} +import firrtl.options.{Phase, PhaseException, Shell, Stage, StageMain, OptionsException} +import firrtl.passes.{PassException, PassExceptions} + +import scala.util.control.ControlThrowable + +import java.io.PrintWriter + +class FirrtlStage extends Stage { + val shell: Shell = new Shell("firrtl") with FirrtlCli + + private val phases: Seq[Phase] = Seq( + new firrtl.stage.phases.AddDefaults, + new firrtl.stage.phases.AddImplicitEmitter, + new firrtl.stage.phases.Checks, + new firrtl.stage.phases.AddCircuit, + new firrtl.stage.phases.AddImplicitOutputFile, + new firrtl.stage.phases.Compiler, + new firrtl.stage.phases.WriteEmitted + ) + + def run(annotations: AnnotationSeq): AnnotationSeq = try { + phases.foldLeft(annotations)((a, f) => f.runTransform(a)) + } catch { + /* Rethrow the exceptions which are expected or due to the runtime environment (out of memory, stack overflow, etc.). + * Any UNEXPECTED exceptions should be treated as internal errors. */ + case p @ (_: ControlThrowable | _: PassException | _: PassExceptions | _: FIRRTLException | _: OptionsException + | _: PhaseException) => throw p + case CustomTransformException(cause) => throw cause + case e: Exception => Utils.throwInternalError(exception = Some(e)) + } + +} + +object FirrtlMain extends StageMain(new FirrtlStage) diff --git a/src/main/scala/firrtl/stage/FirrtlStageUtils.scala b/src/main/scala/firrtl/stage/FirrtlStageUtils.scala new file mode 100644 index 00000000..e2304a92 --- /dev/null +++ b/src/main/scala/firrtl/stage/FirrtlStageUtils.scala @@ -0,0 +1,17 @@ +// See LICENSE for license details. + +package firrtl.stage + +private [stage] sealed trait FileExtension +private [stage] case object FirrtlFile extends FileExtension +private [stage] case object ProtoBufFile extends FileExtension + +/** Utilities that help with processing FIRRTL options */ +object FirrtlStageUtils { + + private [stage] def getFileExtension(file: String): FileExtension = file.drop(file.lastIndexOf('.')) match { + case ".pb" => ProtoBufFile + case _ => FirrtlFile + } + +} diff --git a/src/main/scala/firrtl/stage/package.scala b/src/main/scala/firrtl/stage/package.scala new file mode 100644 index 00000000..b8d49208 --- /dev/null +++ b/src/main/scala/firrtl/stage/package.scala @@ -0,0 +1,64 @@ +// See LICENSE for license details. + +package firrtl + +import firrtl.annotations.DeletedAnnotation +import firrtl.options.{OptionsView, PhasePrerequisiteException, Viewer} +import firrtl.stage.phases.WriteEmitted + +/** The [[stage]] package provides an implementation of the FIRRTL compiler using the [[firrtl.options]] package. This + * primarily consists of: + * - [[FirrtlStage]], the internal and external (command line) interface to the FIRRTL compiler + * - A number of [[options.Phase Phase]]s that support and compartmentalize the individual operations of + * [[FirrtlStage]] + * - [[FirrtlOptions]], a class representing options that are necessary to drive the [[FirrtlStage]] and its + * [[firrtl.options.Phase Phase]]s + * - [[FirrtlOptionsView]], a utility that constructs an [[options.OptionsView OptionsView]] of [[FirrtlOptions]] + * from an [[AnnotationSeq]] + * - [[FirrtlCli]], the command line options that the [[FirrtlStage]] supports + * - [[FirrtlStageUtils]] containing miscellaneous utilities for [[stage]] + */ +package object stage { + implicit object FirrtlOptionsView extends OptionsView[FirrtlOptions] { + + /** + * @todo custom transforms are appended as discovered, can this be prepended safely? + */ + def view(options: AnnotationSeq): FirrtlOptions = options + .collect { case a: FirrtlOption => a } + .foldLeft(new FirrtlOptions()){ (c, x) => + x match { + case OutputFileAnnotation(f) => c.copy(outputFileName = Some(f)) + case InfoModeAnnotation(i) => c.copy(infoModeName = i) + case CompilerAnnotation(cx) => c.copy(compiler = cx) + case FirrtlCircuitAnnotation(cir) => c.copy(firrtlCircuit = Some(cir)) + } + } + } + + private [firrtl] implicit object FirrtlExecutionResultView extends OptionsView[FirrtlExecutionResult] { + + private lazy val dummyWriteEmitted = new WriteEmitted + + def view(options: AnnotationSeq): FirrtlExecutionResult = { + val fopts = Viewer.view[FirrtlOptions](options) + val emittedRes = options + .collect{ case DeletedAnnotation(dummyWriteEmitted.name, a: EmittedAnnotation[_]) => a.value.value } + .mkString("\n") + + options.collectFirst{ case a: FirrtlCircuitAnnotation => a.circuit } match { + case None => FirrtlExecutionFailure("No circuit found in AnnotationSeq!") + case Some(a) => FirrtlExecutionSuccess( + emitType = fopts.compiler.getClass.getSimpleName, + emitted = emittedRes, + circuitState = CircuitState( + circuit = a, + form = fopts.compiler.outputForm, + annotations = options, + renames = None + )) + } + } + } + +} diff --git a/src/main/scala/firrtl/stage/phases/AddCircuit.scala b/src/main/scala/firrtl/stage/phases/AddCircuit.scala new file mode 100644 index 00000000..f6ae6370 --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/AddCircuit.scala @@ -0,0 +1,59 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.stage._ + +import firrtl.{AnnotationSeq, Parser, proto} +import firrtl.options.{OptionsException, Phase, PhasePrerequisiteException} + +/** [[firrtl.options.Phase Phase]] that expands [[FirrtlFileAnnotation]]/[[FirrtlSourceAnnotation]] into + * [[FirrtlCircuitAnnotation]]s and deletes the originals. This is part of the preprocessing done on an input + * [[AnnotationSeq]] by [[FirrtlStage]]. + * + * The types of possible annotations are handled in the following ways: + * - [[FirrtlFileAnnotation]]s are read as Protocol Buffers if the file extension ends in `.pb`. Otherwise, these are + * assumed to be raw FIRRTL text and is sent to the [[Parser]]. The original [[FirrtlFileAnnotation]] is deleted. + * - [[FirrtlSourceAnnotation]]s are run through the [[Parser]]. The original [[FirrtlSourceAnnotation]] is deleted. + * - [[FirrtlCircuitAnnotation]]s are left untouched (along with all other annotations). + * + * If a [[Parser]] is used, its [[Parser.InfoMode InfoMode]] is read from a ''mandatory'' [[InfoModeAnnotation]]. If + * using an [[Parser.InfoMode InfoMode]] that expects a filename, the filename is used for [[FirrtlFileAnnotation]]s + * and `[anonymous source]` is used for [[FirrtlSourceAnnotation]]s. + * + * @note '''This must be run after [[AddDefaults]] as this [[firrtl.options.Phase Phase]] depends on the existence of + * an [[InfoModeAnnotation]].'''. + * @define infoModeException firrtl.options.PhasePrerequisiteException if no [[InfoModeAnnotation]] is present + */ +class AddCircuit extends Phase { + + /** Extract the info mode from an [[AnnotationSeq]] or use the default info mode if no annotation exists + * @param annotations some annotations + * @return the info mode + * @throws $infoModeException + */ + private def infoMode(annotations: AnnotationSeq): Parser.InfoMode = { + val infoModeAnnotation = annotations + .collectFirst{ case a: InfoModeAnnotation => a } + .getOrElse { throw new PhasePrerequisiteException( + "An InfoModeAnnotation must be present (did you forget to run AddDefaults?)") } + val infoSource = annotations.collectFirst{ + case FirrtlFileAnnotation(f) => f + case _: FirrtlSourceAnnotation => "anonymous source" + }.getOrElse("not defined") + + infoModeAnnotation.toInfoMode(Some(infoSource)) + } + + /** Convert [[FirrtlFileAnnotation]]/[[FirrtlSourceAnnotation]] into [[FirrtlCircuitAnnotation]] and delete originals + * @throws $infoModeException + */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + lazy val info = infoMode(annotations) + annotations.map { + case a: CircuitOption => a.toCircuit(info) + case a => a + } + } + +} diff --git a/src/main/scala/firrtl/stage/phases/AddDefaults.scala b/src/main/scala/firrtl/stage/phases/AddDefaults.scala new file mode 100644 index 00000000..dc8dc025 --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/AddDefaults.scala @@ -0,0 +1,36 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.AnnotationSeq +import firrtl.options.{Phase, TargetDirAnnotation} +import firrtl.transforms.BlackBoxTargetDirAnno +import firrtl.stage.{CompilerAnnotation, InfoModeAnnotation, FirrtlOptions} + +/** [[firrtl.options.Phase Phase]] that adds default [[FirrtlOption]] [[firrtl.annotations.Annotation Annotation]]s. + * This is a part of the preprocessing done by [[FirrtlStage]]. + */ +class AddDefaults extends Phase { + + /** Append any missing default annotations to an annotation sequence */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + var bb, c, im = true + annotations.foreach { + case _: BlackBoxTargetDirAnno => bb = false + case _: CompilerAnnotation => c = false + case _: InfoModeAnnotation => im = false + case a => + } + + val default = new FirrtlOptions() + val targetDir = annotations + .collectFirst { case d: TargetDirAnnotation => d } + .getOrElse(TargetDirAnnotation()).directory + + (if (bb) Seq(BlackBoxTargetDirAnno(targetDir)) else Seq() ) ++ + (if (c) Seq(CompilerAnnotation(default.compiler)) else Seq() ) ++ + (if (im) Seq(InfoModeAnnotation()) else Seq() ) ++ + annotations + } + +} diff --git a/src/main/scala/firrtl/stage/phases/AddImplicitEmitter.scala b/src/main/scala/firrtl/stage/phases/AddImplicitEmitter.scala new file mode 100644 index 00000000..84f98cdb --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/AddImplicitEmitter.scala @@ -0,0 +1,30 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.{AnnotationSeq, EmitAnnotation, EmitCircuitAnnotation} +import firrtl.stage.{CompilerAnnotation, RunFirrtlTransformAnnotation} +import firrtl.options.Phase + +/** [[firrtl.options.Phase Phase]] that adds a [[firrtl.EmitCircuitAnnotation EmitCircuitAnnotation]] derived from a + * [[firrtl.stage.CompilerAnnotation CompilerAnnotation]] if one does not already exist. + */ +class AddImplicitEmitter extends Phase { + + def transform(annos: AnnotationSeq): AnnotationSeq = { + val emitter = annos.collectFirst{ case a: EmitAnnotation => a } + val compiler = annos.collectFirst{ case CompilerAnnotation(a) => a } + + if (emitter.isEmpty && compiler.nonEmpty) { + annos.flatMap{ + case a: CompilerAnnotation => Seq(a, + RunFirrtlTransformAnnotation(compiler.get.emitter), + EmitCircuitAnnotation(compiler.get.emitter.getClass)) + case a => Some(a) + } + } else { + annos + } + } + +} diff --git a/src/main/scala/firrtl/stage/phases/AddImplicitOutputFile.scala b/src/main/scala/firrtl/stage/phases/AddImplicitOutputFile.scala new file mode 100644 index 00000000..a328f2da --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/AddImplicitOutputFile.scala @@ -0,0 +1,35 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.{AnnotationSeq, EmitAllModulesAnnotation} +import firrtl.options.{Phase, Viewer} +import firrtl.stage.{FirrtlCircuitAnnotation, FirrtlOptions, OutputFileAnnotation} + +/** [[firrtl.options.Phase Phase]] that adds an [[OutputFileAnnotation]] if one does not already exist. + * + * To determine the [[OutputFileAnnotation]], the following precedence is used. Whichever happens first succeeds: + * - Do nothing if an [[OutputFileAnnotation]] or [[EmitAllModulesAnnotation]] exist + * - Use the main in the first discovered [[FirrtlCircuitAnnotation]] (see note below) + * - Use "a" + * + * The file suffix may or may not be specified, but this may be arbitrarily changed by the [[Emitter]]. + * + * @note This [[firrtl.options.Phase Phase]] has a dependency on [[AddCircuit]]. Only a [[FirrtlCircuitAnnotation]] + * will be used to implicitly set the [[OutputFileAnnotation]] (not other [[CircuitOption]] subclasses). + */ +class AddImplicitOutputFile extends Phase { + + /** Add an [[OutputFileAnnotation]] to an [[AnnotationSeq]] */ + def transform(annotations: AnnotationSeq): AnnotationSeq = + annotations + .collectFirst { case _: OutputFileAnnotation | _: EmitAllModulesAnnotation => annotations } + .getOrElse { + val topName = Viewer + .view[FirrtlOptions](annotations) + .firrtlCircuit + .map(_.main) + .getOrElse("a") + OutputFileAnnotation(topName) +: annotations + } +} diff --git a/src/main/scala/firrtl/stage/phases/Checks.scala b/src/main/scala/firrtl/stage/phases/Checks.scala new file mode 100644 index 00000000..19109dab --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/Checks.scala @@ -0,0 +1,93 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.stage._ + +import firrtl.{AnnotationSeq, EmitAllModulesAnnotation, EmitCircuitAnnotation} +import firrtl.annotations.Annotation +import firrtl.options.{OptionsException, Phase, StageUtils} + +/** [[firrtl.options.Phase Phase]] that strictly validates an [[AnnotationSeq]]. The checks applied are intended to be + * extremeley strict. Nothing is inferred or assumed to take a default value (for default value resolution see + * [[AddDefaults]]). + * + * The intent of this approach is that after running this [[firrtl.options.Phase Phase]], a user can be absolutely + * certain that other [[firrtl.options.Phase Phase]]s or views will succeed. See [[FirrtlStage]] for a list of + * [[firrtl.options.Phase Phase]] that commonly run before this. + */ +class Checks extends Phase { + + /** Determine if annotations are sane + * + * @param annos a sequence of [[annotation.Annotation]] + * @return true if all checks pass + * @throws firrtl.options.OptionsException if any checks fail + */ + def transform(annos: AnnotationSeq): AnnotationSeq = { + val inF, inS, eam, ec, outF, td, i, foaf, comp, im, inC = collection.mutable.ListBuffer[Annotation]() + annos.foreach( + _ match { + case a: FirrtlFileAnnotation => a +=: inF + case a: FirrtlSourceAnnotation => a +=: inS + case a: EmitAllModulesAnnotation => a +=: eam + case a: EmitCircuitAnnotation => a +=: ec + case a: OutputFileAnnotation => a +=: outF + case a: CompilerAnnotation => a +=: comp + case a: InfoModeAnnotation => a +=: im + case a: FirrtlCircuitAnnotation => a +=: inC + case _ => }) + + /* At this point, only a FIRRTL Circuit should exist */ + if (inF.isEmpty && inS.isEmpty && inC.isEmpty) { + throw new OptionsException( + s"""|Unable to determine FIRRTL source to read. None of the following were found: + | - an input file: -i, --input-file, FirrtlFileAnnotation + | - FIRRTL source: --firrtl-source, FirrtlSourceAnnotation + | - FIRRTL circuit: FirrtlCircuitAnnotation""".stripMargin )} + + /* Only one FIRRTL input can exist */ + if (inF.size + inS.size + inC.size > 1) { + throw new OptionsException( + s"""|Multiply defined input FIRRTL sources. More than one of the following was found: + | - an input file (${inF.size} times): -i, --input-file, FirrtlFileAnnotation + | - FIRRTL source (${inS.size} times): --firrtl-source, FirrtlSourceAnnotation + | - FIRRTL circuit (${inC.size} times): FirrtlCircuitAnnotation""".stripMargin )} + + /* Specifying an output file and one-file-per module conflict */ + if (eam.nonEmpty && outF.nonEmpty) { + throw new OptionsException( + s"""|Output file is incompatible with emit all modules annotation, but multiples were found: + | - explicit output file (${outF.size} times): -o, --output-file, OutputFileAnnotation + | - one file per module (${eam.size} times): -e, --emit-modules, EmitAllModulesAnnotation""" + .stripMargin )} + + /* Only one output file can be specified */ + if (outF.size > 1) { + val x = outF.map{ case OutputFileAnnotation(x) => x } + throw new OptionsException( + s"""|No more than one output file can be specified, but found '${x.mkString(", ")}' specified via: + | - option or annotation: -o, --output-file, OutputFileAnnotation""".stripMargin) } + + /* One mandatory compiler must be specified */ + if (comp.size != 1) { + val x = comp.map{ case CompilerAnnotation(x) => x } + val (msg, suggest) = if (comp.size == 0) { ("none found", "forget one of") } + else { (s"""found '${x.mkString(", ")}'""", "use multiple of") } + throw new OptionsException( + s"""|Exactly one compiler must be specified, but $msg. Did you $suggest the following? + | - an option or annotation: -X, --compiler, CompilerAnnotation""".stripMargin )} + + /* One mandatory info mode must be specified */ + if (im.size != 1) { + val x = im.map{ case InfoModeAnnotation(x) => x } + val (msg, suggest) = if (im.size == 0) { ("none found", "forget one of") } + else { (s"""found '${x.mkString(", ")}'""", "use multiple of") } + throw new OptionsException( + s"""|Exactly one info mode must be specified, but $msg. Did you $suggest the following? + | - an option or annotation: --info-mode, InfoModeAnnotation""".stripMargin )} + + annos + } + +} diff --git a/src/main/scala/firrtl/stage/phases/Compiler.scala b/src/main/scala/firrtl/stage/phases/Compiler.scala new file mode 100644 index 00000000..4d46f8a0 --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/Compiler.scala @@ -0,0 +1,99 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.{AnnotationSeq, ChirrtlForm, CircuitState, Compiler => FirrtlCompiler, Transform, seqToAnnoSeq} +import firrtl.options.{Phase, PhasePrerequisiteException, Translator} +import firrtl.stage.{CircuitOption, CompilerAnnotation, FirrtlOptions, FirrtlCircuitAnnotation, + RunFirrtlTransformAnnotation} + +import scala.collection.mutable + +/** An encoding of the information necessary to run the FIRRTL compiler once */ +private [stage] case class CompilerRun( + stateIn: CircuitState, + stateOut: Option[CircuitState], + transforms: Seq[Transform], + compiler: Option[FirrtlCompiler] ) + +/** An encoding of possible defaults for a [[CompilerRun]] */ +private [stage] case class Defaults( + annotations: AnnotationSeq = Seq.empty, + transforms: Seq[Transform] = Seq.empty, + compiler: Option[FirrtlCompiler] = None) + +/** Runs the FIRRTL compilers on an [[AnnotationSeq]]. If the input [[AnnotationSeq]] contains more than one circuit + * (i.e., more than one [[FirrtlCircuitAnnotation]]), then annotations will be broken up and each run will be executed + * in parallel. + * + * The [[AnnotationSeq]] will be chunked up into compiler runs using the following algorithm. All annotations that + * occur before the first [[FirrtlCircuitAnnotation]] are treated as global annotations that apply to all circuits. + * Annotations after a circuit are only associated with their closest preceeding circuit. E.g., for the following + * annotations (where A, B, and C are some annotations): + * + * A(a), FirrtlCircuitAnnotation(x), B, FirrtlCircuitAnnotation(y), A(b), C, FirrtlCircuitAnnotation(z) + * + * Then this will result in three compiler runs: + * - FirrtlCircuitAnnotation(x): A(a), B + * - FirrtlCircuitAnnotation(y): A(a), A(b), C + * - FirrtlCircuitAnnotation(z): A(a) + * + * A(a) is a default, global annotation. B binds to FirrtlCircuitAnnotation(x). A(a), A(b), and C bind to + * FirrtlCircuitAnnotation(y). Note: A(b) ''may'' overwrite A(a) if this is a CompilerAnnotation. + * FirrtlCircuitAnnotation(z) has no annotations, so it only gets the default A(a). + */ +class Compiler extends Phase with Translator[AnnotationSeq, Seq[CompilerRun]] { + + /** Convert an [[AnnotationSeq]] into a sequence of compiler runs. */ + protected def aToB(a: AnnotationSeq): Seq[CompilerRun] = { + var foundFirstCircuit = false + val c = mutable.ArrayBuffer.empty[CompilerRun] + a.foldLeft(Defaults()){ + case (d, FirrtlCircuitAnnotation(circuit)) => + foundFirstCircuit = true + CompilerRun(CircuitState(circuit, ChirrtlForm, d.annotations, None), None, d.transforms, d.compiler) +=: c + d + case (d, a) if foundFirstCircuit => a match { + case RunFirrtlTransformAnnotation(transform) => + c(0) = c(0).copy(transforms = transform +: c(0).transforms) + d + case CompilerAnnotation(compiler) => + c(0) = c(0).copy(compiler = Some(compiler)) + d + case annotation => + val state = c(0).stateIn + c(0) = c(0).copy(stateIn = state.copy(annotations = annotation +: state.annotations)) + d + } + case (d, a) if !foundFirstCircuit => a match { + case RunFirrtlTransformAnnotation(transform) => d.copy(transforms = transform +: d.transforms) + case CompilerAnnotation(compiler) => d.copy(compiler = Some(compiler)) + case annotation => d.copy(annotations = annotation +: d.annotations) + } + } + c + } + + /** Expand compiler output back into an [[AnnotationSeq]]. Annotations used in the construction of the compiler run are + * removed ([[CompilerAnnotation]]s and [[RunFirrtlTransformAnnotation]]s). + */ + protected def bToA(b: Seq[CompilerRun]): AnnotationSeq = + b.flatMap( bb => FirrtlCircuitAnnotation(bb.stateOut.get.circuit) +: bb.stateOut.get.annotations ) + + /** Run the FIRRTL compiler some number of times. If more than one run is specified, a parallel collection will be + * used. + */ + protected def internalTransform(b: Seq[CompilerRun]): Seq[CompilerRun] = { + def f(c: CompilerRun): CompilerRun = { + val statex = c + .compiler + .getOrElse { throw new PhasePrerequisiteException("No compiler specified!") } + .compile(c.stateIn, c.transforms.reverse) + c.copy(stateOut = Some(statex)) + } + + if (b.size <= 1) { b.map(f) } + else { b.par.map(f).seq } + } + +} diff --git a/src/main/scala/firrtl/stage/phases/DriverCompatibility.scala b/src/main/scala/firrtl/stage/phases/DriverCompatibility.scala new file mode 100644 index 00000000..e116dac3 --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/DriverCompatibility.scala @@ -0,0 +1,223 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.stage._ + +import firrtl.{AnnotationSeq, EmitAllModulesAnnotation, EmitCircuitAnnotation, FirrtlExecutionResult, Parser} +import firrtl.annotations.NoTargetAnnotation +import firrtl.proto.FromProto +import firrtl.options.{HasScoptOptions, InputAnnotationFileAnnotation, OptionsException, Phase, StageOptions, + StageUtils} +import firrtl.options.Viewer + +import scopt.OptionParser + +import java.io.File + +/** Provides compatibility methods to replicate deprecated [[Driver]] semantics. + * + * At a high level, the [[Driver]] tries extremely hard to figure out what the user meant and to enable them to not be + * explicit with command line options. As an example, the `--top-name` option is not used for any FIRRTL top module + * determination, but to find a FIRRTL file by that name and/or an annotation file by that name. This mode of file + * discovery is only used if no explicit FIRRTL file/source/circuit and/or annotation file is given. Going further, the + * `--top-name` argument is implicitly specified by the `main` of an input circuit if not explicit and can be used to + * derive an annotation file. Summarily, the [[firrtl.options.Phase Phase]]s provided by this enable this type of + * resolution. + * + * '''Only use these methods if you are intending to replicate old [[Driver]] semantics for a good reason.''' + * Otherwise, opt for more explicit specification by the user. + */ +object DriverCompatibility { + + /** Shorthand object for throwing an exception due to an option that was removed */ + private def optionRemoved(a: String): String = + s"""|Option '$a' was removed as part of the FIRRTL Stage refactor. Use an explicit input/output options instead. + |This error will be removed in 1.3.""".stripMargin + + /** Convert an [[firrtl.AnnotationSeq AnnotationSeq]] to a ''deprecated'' [[firrtl.FirrtlExecutionResult + * FirrtlExecutionResult]]. + * @param annotations a sequence of [[firrtl.annotations.Annotation Annotation]] + */ + @deprecated("FirrtlExecutionResult is deprecated as part of the Stage/Phase refactor. Migrate to FirrtlStage.", "1.2") + def firrtlResultView(annotations: AnnotationSeq): FirrtlExecutionResult = + Viewer.view[FirrtlExecutionResult](annotations) + + /** Holds the name of the top (main) module in an input circuit + * @param value top module name + */ + @deprecated(""""top-name" is deprecated as part of the Stage/Phase refactor. Use explicit input/output files.""", "1.2") + case class TopNameAnnotation(topName: String) extends NoTargetAnnotation + + object TopNameAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p + .opt[Unit]("top-name") + .abbr("tn") + .hidden + .unbounded + .action( (_, _) => throw new OptionsException(optionRemoved("--top-name/-tn")) ) + } + + /** Indicates that the implicit emitter, derived from a [[CompilerAnnotation]] should be an [[EmitAllModulesAnnotation]] + * as opposed to an [[EmitCircuitAnnotation]]. + */ + private [firrtl] case object EmitOneFilePerModuleAnnotation extends NoTargetAnnotation { + + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p + .opt[Unit]("split-modules") + .abbr("fsm") + .hidden + .unbounded + .action( (_, _) => throw new OptionsException(optionRemoved("--split-modules/-fsm")) ) + + } + + /** Determine the top name using the following precedence (highest to lowest): + * - Explicitly from a [[TopNameAnnotation]] + * - Implicitly from the top module ([[firrtl.ir.Circuit.main main]]) of a [[FirrtlCircuitAnnotation]] + * - Implicitly from the top module ([[firrtl.ir.Circuit.main main]]) of a [[FirrtlSourceAnnotation]] + * - Implicitly from the top module ([[firrtl.ir.Circuit.main main]]) of a [[FirrtlFileAnnotation]] + * + * @param annotations annotations to extract topName from + * @return the top module ''if it can be determined'' + */ + private def topName(annotations: AnnotationSeq): Option[String] = + annotations.collectFirst{ case TopNameAnnotation(n) => n }.orElse( + annotations.collectFirst{ case FirrtlCircuitAnnotation(c) => c.main }.orElse( + annotations.collectFirst{ case FirrtlSourceAnnotation(s) => Parser.parse(s).main }.orElse( + annotations.collectFirst{ case FirrtlFileAnnotation(f) => + FirrtlStageUtils.getFileExtension(f) match { + case ProtoBufFile => FromProto.fromFile(f).main + case FirrtlFile => Parser.parse(io.Source.fromFile(f).getLines().mkString("\n")).main } } ))) + + /** Determine the target directory with the following precedence (highest to lowest): + * - Explicitly from the user-specified [[firrtl.options.TargetDirAnnotation TargetDirAnnotation]] + * - Implicitly from the default of [[firrtl.options.StageOptions.targetDir StageOptions.targetDir]] + * + * @param annotations input annotations to extract targetDir from + * @return the target directory + */ + private def targetDir(annotations: AnnotationSeq): String = Viewer.view[StageOptions](annotations).targetDir + + /** Add an implicit annotation file derived from the determined top name of the circuit if no + * [[firrtl.options.InputAnnotationFileAnnotation InputAnnotationFileAnnotation]] is present. + * + * The implicit annotation file is determined through the following complicated semantics: + * - If an [[InputAnnotationFileAnnotation]] already exists, then nothing is modified + * - If the derived topName (the `main` in a [[firrtl.ir.Circuit Circuit]]) is ''discernable'' (see below) and a + * file called `topName.anno` (exactly, not `topName.anno.json`) exists, then this will add an + * [[InputAnnotationFileAnnotation]] using that `topName.anno` + * - If any of this doesn't work, then the the [[AnnotationSeq]] is unmodified + * + * The precedence for determining the `topName` is the following (first one wins): + * - The `topName` in a [[TopNameAnnotation]] + * - The `main` [[FirrtlCircuitAnnotation]] + * - The `main` in a parsed [[FirrtlSourceAnnotation]] + * - The `main` in the first [[FirrtlFileAnnotation]] using either ProtoBuf or parsing as determined by file + * extension + * + * @param annos input annotations + * @return output annotations + */ + class AddImplicitAnnotationFile extends Phase { + + /** Try to add an [[InputAnnotationFileAnnotation]] implicitly specified by an [[AnnotationSeq]]. */ + def transform(annotations: AnnotationSeq): AnnotationSeq = annotations + .collectFirst{ case a: InputAnnotationFileAnnotation => a } match { + case Some(_) => annotations + case None => topName(annotations) match { + case Some(n) => + val filename = targetDir(annotations) + "/" + n + ".anno" + if (new File(filename).exists) { + StageUtils.dramaticWarning( + s"Implicit reading of the annotation file is deprecated! Use an explict --annotation-file argument.") + annotations :+ InputAnnotationFileAnnotation(filename) + } else { + annotations + } + case None => annotations + } } + + } + + /** Add a [[FirrtlFileAnnotation]] if no annotation that explictly defines a circuit exists. + * + * This takes the option with the following precedence: + * - If an annotation subclassing [[CircuitOption]] exists, do nothing + * - If a [[TopNameAnnotation]] exists, use that to derive a [[FirrtlFileAnnotation]] and append it + * - Do nothing + * + * In the case of (3) above, this [[AnnotationSeq]] is likely insufficient for FIRRTL to work with (no circuit was + * passed). However, instead of catching this here, we rely on [[Checks]] to validate the annotations. + * + * @param annotations input annotations + * @return + */ + class AddImplicitFirrtlFile extends Phase { + + /** Try to add a [[FirrtlFileAnnotation]] implicitly specified by an [[AnnotationSeq]]. */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + val circuit = annotations.collectFirst { case a @ (_: CircuitOption | _: FirrtlCircuitAnnotation) => a } + val main = annotations.collectFirst { case a: TopNameAnnotation => a.topName } + + if (circuit.nonEmpty) { + annotations + } else if (main.nonEmpty) { + StageUtils.dramaticWarning( + s"Implicit reading of the input file is deprecated! Use an explict --input-file argument.") + FirrtlFileAnnotation(Viewer.view[StageOptions](annotations).getBuildFileName(s"${main.get}.fir")) +: annotations + } else { + annotations + } + } + } + + /** Adds an [[EmitAnnotation]] for each [[CompilerAnnotation]]. + * + * If an [[EmitOneFilePerModuleAnnotation]] exists, then this will add an [[EmitAllModulesAnnotation]]. Otherwise, + * this adds an [[EmitCircuitAnnotation]]. This replicates old behavior where specifying a compiler automatically + * meant that an emitter would also run. + */ + @deprecated("""AddImplicitEmitter should only be used to build Driver compatibility wrappers. Switch to Stage.""", + "1.2") + class AddImplicitEmitter extends Phase { + + /** Add one [[EmitAnnotation]] foreach [[CompilerAnnotation]]. */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + val splitModules = annotations.collectFirst{ case a: EmitOneFilePerModuleAnnotation.type => a }.isDefined + + annotations.flatMap { + case a @ CompilerAnnotation(c) => + val b = RunFirrtlTransformAnnotation(a.compiler.emitter) + if (splitModules) { Seq(a, b, EmitAllModulesAnnotation(c.emitter.getClass)) } + else { Seq(a, b, EmitCircuitAnnotation (c.emitter.getClass)) } + case a => Seq(a) + } + } + + } + + /** Adds an [[OutputFileAnnotation]] derived from a [[TopNameAnnotation]] if no [[OutputFileAnnotation]] already + * exists. If no [[TopNameAnnotation]] exists, then no [[OutputFileAnnotation]] is added. + */ + @deprecated("""AddImplicitOutputFile should only be used to build Driver compatibility wrappers. Switch to Stage.""", + "1.2") + class AddImplicitOutputFile extends Phase { + + /** Add an [[OutputFileAnnotation]] derived from a [[TopNameAnnotation]] if needed. */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + val hasOutputFile = annotations + .collectFirst{ case a @(_: EmitOneFilePerModuleAnnotation.type | _: OutputFileAnnotation) => a } + .isDefined + val top = topName(annotations) + + if (!hasOutputFile && top.isDefined) { + OutputFileAnnotation(top.get) +: annotations + } else { + annotations + } + } + + } + +} diff --git a/src/main/scala/firrtl/stage/phases/WriteEmitted.scala b/src/main/scala/firrtl/stage/phases/WriteEmitted.scala new file mode 100644 index 00000000..b6d95d68 --- /dev/null +++ b/src/main/scala/firrtl/stage/phases/WriteEmitted.scala @@ -0,0 +1,50 @@ +// See LICENSE for license details. + +package firrtl.stage.phases + +import firrtl.{AnnotationSeq, EmittedModuleAnnotation, EmittedCircuitAnnotation} +import firrtl.options.{Phase, StageOptions, Viewer} +import firrtl.stage.FirrtlOptions + +import java.io.PrintWriter + +/** [[firrtl.options.Phase Phase]] that writes any [[EmittedAnnotation]]s in an input [[AnnotationSeq]] to one or more + * files. The input [[AnnotationSeq]] is viewed as both [[FirrtlOptions]] and [[firrtl.options.StageOptions + * StageOptions]] to determine the output filenames in the following way: + * - [[EmittedModuleAnnotation]]s are written to a file in [[firrtl.options.StageOptions.targetDir + * StageOptions.targetDir]] with the same name as the module and the [[EmittedComponent.outputSuffix outputSuffix]] + * that the [[EmittedComponent]] specified + * - [[EmittedCircuitAnnotation]]s are written to a file in [[firrtl.options.StageOptions.targetDir + * StageOptions.targetDir]] using the [[FirrtlOptions.outputFileName]] viewed from the [[AnnotationSeq]]. If no + * [[FirrtlOptions.outputFileName]] exists, then the top module/main name will be used. The + * [[EmittedComponent.outputSuffix outputSuffix]] will be appended as needed. + * + * This does no sanity checking of the input [[AnnotationSeq]]. This simply writes any modules or circuits it sees to + * files. If you need additional checking, then you should stack an appropriate checking phase before this. + * + * Any annotations written to files will be deleted. + */ +class WriteEmitted extends Phase { + + /** Write any [[EmittedAnnotation]]s in an [[AnnotationSeq]] to files. Written [[EmittedAnnotation]]s are deleted. */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + val fopts = Viewer.view[FirrtlOptions](annotations) + val sopts = Viewer.view[StageOptions](annotations) + + annotations.flatMap { + case a: EmittedModuleAnnotation[_] => + val pw = new PrintWriter(sopts.getBuildFileName(a.value.name, Some(a.value.outputSuffix))) + pw.write(a.value.value) + pw.close() + None + case a: EmittedCircuitAnnotation[_] => + val pw = new PrintWriter( + sopts.getBuildFileName(fopts.outputFileName.getOrElse(a.value.name), Some(a.value.outputSuffix))) + pw.write(a.value.value) + pw.close() + None + case a => Some(a) + } + + } +} diff --git a/src/main/scala/firrtl/util/BackendCompilationUtilities.scala b/src/main/scala/firrtl/util/BackendCompilationUtilities.scala index bd5d2a9a..34be1d1e 100644 --- a/src/main/scala/firrtl/util/BackendCompilationUtilities.scala +++ b/src/main/scala/firrtl/util/BackendCompilationUtilities.scala @@ -10,7 +10,7 @@ import java.util.Calendar import firrtl.FirrtlExecutionOptions import scala.sys.process.{ProcessBuilder, ProcessLogger, _} - + trait BackendCompilationUtilities { /** Parent directory for tests */ lazy val TestDirectory = new File("test_run_dir") diff --git a/src/main/scala/logger/Logger.scala b/src/main/scala/logger/Logger.scala index 6e8dbeb1..1cf7d7ee 100644 --- a/src/main/scala/logger/Logger.scala +++ b/src/main/scala/logger/Logger.scala @@ -4,7 +4,12 @@ package logger import java.io.{ByteArrayOutputStream, File, FileOutputStream, PrintStream} -import firrtl.ExecutionOptionsManager +import firrtl.{ExecutionOptionsManager, HasFirrtlOptions, AnnotationSeq} +import firrtl.stage.FirrtlOptions +import firrtl.options.StageOptions +import firrtl.options.Viewer.view +import firrtl.stage.FirrtlOptionsView +import logger.phases.{AddDefaults, Checks} import scala.util.DynamicVariable @@ -120,24 +125,9 @@ object Logger { * @tparam A The return type of codeBlock * @return Whatever block returns */ - def makeScope[A](manager: ExecutionOptionsManager)(codeBlock: => A): A = { - val runState: LoggerState = { - val newRunState = updatableLoggerState.value.getOrElse(new LoggerState) - if(newRunState.fromInvoke) { - newRunState - } - else { - val forcedNewRunState = new LoggerState - forcedNewRunState.fromInvoke = true - forcedNewRunState - } - } - - updatableLoggerState.withValue(Some(runState)) { - setOptions(manager) - codeBlock - } - } + @deprecated("Use makeScope(opts: FirrtlOptions)", "1.2") + def makeScope[A](manager: ExecutionOptionsManager)(codeBlock: => A): A = + makeScope(manager.commonOptions.toAnnotations)(codeBlock) /** * See makeScope using manager. This creates a manager from a command line arguments style @@ -147,6 +137,7 @@ object Logger { * @tparam A return type of codeBlock * @return */ + @deprecated("Use makescope(opts: FirrtlOptions)", "1.2") def makeScope[A](args: Array[String] = Array.empty)(codeBlock: => A): A = { val executionOptionsManager = new ExecutionOptionsManager("logger") if(executionOptionsManager.parse(args)) { @@ -157,6 +148,29 @@ object Logger { } } + /** Set a scope for this logger based on available annotations + * @param options a sequence annotations + * @param codeBlock some Scala code over which to define this scope + * @tparam A return type of the code block + * @return the original return of the code block + */ + def makeScope[A](options: AnnotationSeq)(codeBlock: => A): A = { + val runState: LoggerState = { + val newRunState = updatableLoggerState.value.getOrElse(new LoggerState) + if(newRunState.fromInvoke) { + newRunState + } + else { + val forcedNewRunState = new LoggerState + forcedNewRunState.fromInvoke = true + forcedNewRunState + } + } + updatableLoggerState.withValue(Some(runState)) { + setOptions(options) + codeBlock + } + } /** * Used to test whether a given log statement should generate some logging output. @@ -341,20 +355,33 @@ object Logger { * from the command line via an options manager * @param optionsManager manager */ - def setOptions(optionsManager: ExecutionOptionsManager): Unit = { - val commonOptions = optionsManager.commonOptions - state.globalLevel = (state.globalLevel, commonOptions.globalLogLevel) match { + @deprecated("Use setOptions(annotations: AnnotationSeq)", "1.2") + def setOptions(optionsManager: ExecutionOptionsManager): Unit = + setOptions(optionsManager.commonOptions.toAnnotations) + + /** Set logger options based on the content of an [[firrtl.AnnotationSeq AnnotationSeq]] + * @param inputAnnotations annotation sequence containing logger options + */ + def setOptions(inputAnnotations: AnnotationSeq): Unit = { + val annotations = Seq( AddDefaults, + Checks ) + .foldLeft(inputAnnotations)((a, p) => p.runTransform(a)) + + val lopts = view[LoggerOptions](annotations) + state.globalLevel = (state.globalLevel, lopts.globalLogLevel) match { case (LogLevel.None, LogLevel.None) => LogLevel.None case (x, LogLevel.None) => x case (LogLevel.None, x) => x case (_, x) => x case _ => LogLevel.Error } - setClassLogLevels(commonOptions.classLogLevels) - if(commonOptions.logToFile) { - setOutput(commonOptions.getLogFileName(optionsManager)) + setClassLogLevels(lopts.classLogLevels) + + if (lopts.logFileName.nonEmpty) { + setOutput(lopts.logFileName.get) } - state.logClassNames = commonOptions.logClassNames + + state.logClassNames = lopts.logClassNames } } @@ -399,3 +426,9 @@ class Logger(containerClass: String) { Logger.showMessage(LogLevel.Trace, containerClass, message) } } + +/** An exception originating from the Logger + * @param str an exception message + * @param cause a reason for the exception + */ +class LoggerException(val str: String, cause: Throwable = null) extends RuntimeException(str, cause) diff --git a/src/main/scala/logger/LoggerAnnotations.scala b/src/main/scala/logger/LoggerAnnotations.scala new file mode 100644 index 00000000..2811cc6c --- /dev/null +++ b/src/main/scala/logger/LoggerAnnotations.scala @@ -0,0 +1,77 @@ +// See LICENSE for license details. + +package logger + +import firrtl.AnnotationSeq +import firrtl.annotations.{Annotation, NoTargetAnnotation} +import firrtl.options.{HasScoptOptions, StageUtils} + +import scopt.OptionParser + +/** An annotation associated with a Logger command line option */ +sealed trait LoggerOption { this: Annotation => } + +/** Describes the verbosity of information to log + * - set with `-ll/--log-level` + * - if unset, a [[LogLevelAnnotation]] with the default log level will be emitted + * @param level the level of logging + */ +case class LogLevelAnnotation(globalLogLevel: LogLevel.Value = LogLevel.None) extends NoTargetAnnotation with LoggerOption + +object LogLevelAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[String]("log-level") + .abbr("ll") + .valueName("") + .action( (x, c) => LogLevelAnnotation(LogLevel(x)) +: c ) + .validate{ x => + lazy val msg = s"$x bad value must be one of error|warn|info|debug|trace" + if (Array("error", "warn", "info", "debug", "trace").contains(x.toLowerCase)) { p.success } + else { p.failure(msg) }} + .unbounded() + .text(s"Sets the verbosity level of logging, default is ${new LoggerOptions().globalLogLevel}") +} + +/** Describes a mapping of a class to a specific log level + * - set with `-cll/--class-log-level` + * @param name the class name to log + * @param level the verbosity level + */ +case class ClassLogLevelAnnotation(className: String, level: LogLevel.Value) extends NoTargetAnnotation with LoggerOption + +object ClassLogLevelAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[Seq[String]]("class-log-level") + .abbr("cll") + .valueName("[,...]") + .action( (x, c) => (x.map { y => + val className :: levelName :: _ = y.split(":").toList + val level = LogLevel(levelName) + ClassLogLevelAnnotation(className, level) }) ++ c ) + .unbounded() + .text(s"This defines per-class verbosity of logging") +} + +/** Enables logging to a file (as opposed to STDOUT) + * - maps to [[LoggerOptions.logFileName]] + * - enabled with `--log-file` + */ +case class LogFileAnnotation(file: Option[String]) extends NoTargetAnnotation with LoggerOption + +object LogFileAnnotation extends HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = { + p.opt[String]("log-file") + .action( (x, c) => LogFileAnnotation(Some(x)) +: c ) + .unbounded() + .text(s"log to the specified file") + } +} + +/** Enables class names in log output + * - enabled with `-lcn/--log-class-names` + */ +case object LogClassNamesAnnotation extends NoTargetAnnotation with LoggerOption with HasScoptOptions { + def addOptions(p: OptionParser[AnnotationSeq]): Unit = p.opt[Unit]("log-class-names") + .abbr("lcn") + .action( (x, c) => LogClassNamesAnnotation +: c ) + .unbounded() + .text(s"shows class names and log level in logging output, useful for target --class-log-level") +} diff --git a/src/main/scala/logger/LoggerOptions.scala b/src/main/scala/logger/LoggerOptions.scala new file mode 100644 index 00000000..299382f0 --- /dev/null +++ b/src/main/scala/logger/LoggerOptions.scala @@ -0,0 +1,38 @@ +// See LICENSE for license details. + +package logger + +/** Internal options used to control the logging in programs that are part of the Chisel stack + * + * @param globalLogLevel the verbosity of logging (default: [[logger.LogLevel.None]]) + * @param classLogLevels the individual verbosity of logging for specific classes + * @param logToFile if true, log to a file + * @param logClassNames indicates logging verbosity on a class-by-class basis + */ +class LoggerOptions private [logger] ( + val globalLogLevel: LogLevel.Value = LogLevelAnnotation().globalLogLevel, + val classLogLevels: Map[String, LogLevel.Value] = Map.empty, + val logClassNames: Boolean = false, + val logFileName: Option[String] = None) { + + private [logger] def copy( + globalLogLevel: LogLevel.Value = globalLogLevel, + classLogLevels: Map[String, LogLevel.Value] = classLogLevels, + logClassNames: Boolean = logClassNames, + logFileName: Option[String] = logFileName): LoggerOptions = { + + new LoggerOptions( + globalLogLevel = globalLogLevel, + classLogLevels = classLogLevels, + logClassNames = logClassNames, + logFileName = logFileName) + + } + + /** Return the name of the log file, defaults to `a.log` if unspecified */ + def getLogFileName(): Option[String] = if (!logToFile) None else logFileName.orElse(Some("a.log")) + + /** True if a [[Logger]] should be writing to a file */ + @deprecated("logToFile was removed, use logFileName.nonEmpty", "1.2") + def logToFile(): Boolean = logFileName.nonEmpty +} diff --git a/src/main/scala/logger/package.scala b/src/main/scala/logger/package.scala new file mode 100644 index 00000000..52a3331a --- /dev/null +++ b/src/main/scala/logger/package.scala @@ -0,0 +1,21 @@ +// See LICENSE for license details. + +import firrtl.AnnotationSeq +import firrtl.options.OptionsView + +package object logger { + + implicit object LoggerOptionsView extends OptionsView[LoggerOptions] { + def view(options: AnnotationSeq): LoggerOptions = options + .foldLeft(new LoggerOptions()) { (c, x) => + x match { + case LogLevelAnnotation(logLevel) => c.copy(globalLogLevel = logLevel) + case ClassLogLevelAnnotation(name, level) => c.copy(classLogLevels = c.classLogLevels + (name -> level)) + case LogFileAnnotation(f) => c.copy(logFileName = f) + case LogClassNamesAnnotation => c.copy(logClassNames = true) + case _ => c + } + } + } + +} diff --git a/src/main/scala/logger/phases/AddDefaults.scala b/src/main/scala/logger/phases/AddDefaults.scala new file mode 100644 index 00000000..f6daa811 --- /dev/null +++ b/src/main/scala/logger/phases/AddDefaults.scala @@ -0,0 +1,27 @@ +// See LICENSE for license details. + +package logger.phases + +import firrtl.AnnotationSeq +import firrtl.options.Phase + +import logger.{LoggerOption, LogLevelAnnotation} + +/** Add default logger [[Annotation]]s */ +private [logger] object AddDefaults extends Phase { + + /** Add missing default [[Logger]] [[Annotation]]s to an [[AnnotationSeq]] + * @param annotations input annotations + * @return output annotations with defaults + */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + var ll = true + annotations.collect{ case a: LoggerOption => a }.map{ + case _: LogLevelAnnotation => ll = false + case _ => + } + annotations ++ + (if (ll) Seq(LogLevelAnnotation()) else Seq() ) + } + +} diff --git a/src/main/scala/logger/phases/Checks.scala b/src/main/scala/logger/phases/Checks.scala new file mode 100644 index 00000000..c706948c --- /dev/null +++ b/src/main/scala/logger/phases/Checks.scala @@ -0,0 +1,42 @@ +// See LICENSE for license details. + +package logger.phases + +import firrtl.AnnotationSeq +import firrtl.annotations.Annotation +import firrtl.options.Phase + +import logger.{LogLevelAnnotation, LogFileAnnotation, LoggerException} + +import scala.collection.mutable + +/** Check that an [[firrtl.AnnotationSeq AnnotationSeq]] has all necessary [[firrtl.annotations.Annotation Annotation]]s + * for a [[Logger]] */ +object Checks extends Phase { + + /** Ensure that an [[firrtl.AnnotationSeq AnnotationSeq]] has necessary [[Logger]] [[firrtl.annotations.Annotation + * Annotation]]s + * @param annotations input annotations + * @return input annotations unmodified + * @throws logger.LoggerException + */ + def transform(annotations: AnnotationSeq): AnnotationSeq = { + val ll, lf = mutable.ListBuffer[Annotation]() + annotations.foreach( + _ match { + case a: LogLevelAnnotation => ll += a + case a: LogFileAnnotation => lf += a + case _ => }) + if (ll.size > 1) { + val l = ll.map{ case LogLevelAnnotation(x) => x } + throw new LoggerException( + s"""|At most one log level can be specified, but found '${l.mkString(", ")}' specified via: + | - an option or annotation: -ll, --log-level, LogLevelAnnotation""".stripMargin )} + if (lf.size > 1) { + throw new LoggerException( + s"""|At most one log file can be specified, but found ${lf.size} combinations of: + | - an options or annotation: -ltf, --log-to-file, --log-file, LogFileAnnotation""".stripMargin )} + annotations + } + +} diff --git a/src/test/scala/firrtlTests/DriverSpec.scala b/src/test/scala/firrtlTests/DriverSpec.scala index ae1e08e7..d26aadf2 100644 --- a/src/test/scala/firrtlTests/DriverSpec.scala +++ b/src/test/scala/firrtlTests/DriverSpec.scala @@ -206,18 +206,35 @@ class DriverSpec extends FreeSpec with Matchers with BackendCompilationUtilities } // Deprecated - "Annotations can be read implicitly from the name of the circuit" in { + "Annotations can be read implicitly from the name of the circuit" - { + val input = """|circuit foo : + | module foo : + | input x : UInt<8> + | output y : UInt<8> + | y <= x""".stripMargin val top = "foo" val optionsManager = new ExecutionOptionsManager("test") with HasFirrtlOptions { commonOptions = commonOptions.copy(topName = top) + firrtlOptions = firrtlOptions.copy(firrtlSource = Some(input)) } val annoFile = new File(optionsManager.commonOptions.targetDirName, top + ".anno") - copyResourceToFile("/annotations/SampleAnnotations.anno", annoFile) - optionsManager.firrtlOptions.annotations.length should be(0) - val annos = Driver.getAnnotations(optionsManager) - annos.length should be(12) // 9 from circuit plus 3 general purpose - annos.count(_.isInstanceOf[InlineAnnotation]) should be(9) - annoFile.delete() + "Using Driver.getAnnotations" in { + copyResourceToFile("/annotations/SampleAnnotations.anno", annoFile) + optionsManager.firrtlOptions.annotations.length should be(0) + val annos = Driver.getAnnotations(optionsManager) + annos.length should be(12) // 9 from circuit plus 3 general purpose + annos.count(_.isInstanceOf[InlineAnnotation]) should be(9) + annoFile.delete() + } + "Using Driver.execute" in { + copyResourceToFile("/annotations/SampleAnnotations.anno", annoFile) + Driver.execute(optionsManager) match { + case r: FirrtlExecutionSuccess => + val annos = r.circuitState.annotations + annos.count(_.isInstanceOf[InlineAnnotation]) should be(9) + } + annoFile.delete() + } } // Deprecated diff --git a/src/test/scala/firrtlTests/annotationTests/TargetDirAnnotationSpec.scala b/src/test/scala/firrtlTests/annotationTests/TargetDirAnnotationSpec.scala index 8f157131..eb061d8f 100644 --- a/src/test/scala/firrtlTests/annotationTests/TargetDirAnnotationSpec.scala +++ b/src/test/scala/firrtlTests/annotationTests/TargetDirAnnotationSpec.scala @@ -5,7 +5,6 @@ package annotationTests import firrtlTests._ import firrtl._ -import firrtl.stage.TargetDirAnnotation /** Looks for [[TargetDirAnnotation]] */ class FindTargetDirTransform(expected: String) extends Transform { diff --git a/src/test/scala/firrtlTests/options/ShellSpec.scala b/src/test/scala/firrtlTests/options/ShellSpec.scala index d87a9a30..50000550 100644 --- a/src/test/scala/firrtlTests/options/ShellSpec.scala +++ b/src/test/scala/firrtlTests/options/ShellSpec.scala @@ -2,12 +2,24 @@ package firrtlTests.options -import org.scalatest._ +import org.scalatest.{FlatSpec, Matchers} +import firrtl.annotations.NoTargetAnnotation import firrtl.options.Shell class ShellSpec extends FlatSpec with Matchers { + case object A extends NoTargetAnnotation + case object B extends NoTargetAnnotation + case object C extends NoTargetAnnotation + case object D extends NoTargetAnnotation + case object E extends NoTargetAnnotation + + trait AlphabeticalCli { this: Shell => + parser.opt[Unit]('c', "c-option").unbounded().action( (x, c) => C +: c ) + parser.opt[Unit]('d', "d-option").unbounded().action( (x, c) => D +: c ) + parser.opt[Unit]('e', "e-option").unbounded().action( (x, c) => E +: c ) } + behavior of "Shell" it should "detect all registered libraries and transforms" in { @@ -19,4 +31,10 @@ class ShellSpec extends FlatSpec with Matchers { info("Found BarLibrary") shell.registeredLibraries.map(_.getClass.getName) should contain ("firrtlTests.options.BarLibrary") } + + it should "correctly order annotations and options" in { + val shell = new Shell("foo") with AlphabeticalCli + + shell.parse(Array("-c", "-d", "-e"), Seq(A, B)).toSeq should be (Seq(A, B, C, D, E)) + } } diff --git a/src/test/scala/firrtlTests/transforms/GroupComponentsSpec.scala b/src/test/scala/firrtlTests/transforms/GroupComponentsSpec.scala index c54e02e3..b4c27875 100644 --- a/src/test/scala/firrtlTests/transforms/GroupComponentsSpec.scala +++ b/src/test/scala/firrtlTests/transforms/GroupComponentsSpec.scala @@ -8,7 +8,7 @@ import firrtl.ir._ import FirrtlCheckers._ -class GroupComponentsSpec extends LowTransformSpec { +class GroupComponentsSpec extends MiddleTransformSpec { def transform = new GroupComponents() val top = "Top" def topComp(name: String): ComponentName = ComponentName(name, ModuleName(top, CircuitName(top))) @@ -71,15 +71,18 @@ class GroupComponentsSpec extends LowTransformSpec { | output out: UInt<16> | inst inst of Child | node n = UInt<16>("h0") - | inst.in_IN <= in - | node a = UInt<16>("h0") - | node b = a + | wire a : UInt<16> + | wire b : UInt<16> | out <= inst.w_OUT + | inst.in_IN <= in + | a <= UInt<16>("h0") + | b <= a | module Child : - | input in_IN : UInt<16> | output w_OUT : UInt<16> - | node w = in_IN + | input in_IN : UInt<16> + | wire w : UInt<16> | w_OUT <= w + | w <= in_IN """.stripMargin execute(input, check, groups) } diff --git a/src/test/scala/firrtlTests/transforms/TopWiringTest.scala b/src/test/scala/firrtlTests/transforms/TopWiringTest.scala index 5a6b3420..d5b1aa6d 100644 --- a/src/test/scala/firrtlTests/transforms/TopWiringTest.scala +++ b/src/test/scala/firrtlTests/transforms/TopWiringTest.scala @@ -55,7 +55,7 @@ trait TopWiringTestsCommon extends FirrtlRunners { /** * Tests TopWiring transformation */ -class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { +class TopWiringTests extends MiddleTransformSpec with TopWiringTestsCommon { "The signal x in module C" should s"be connected to Top port with topwiring prefix and outputfile in $testDirName" in { val input = @@ -78,8 +78,8 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | output x: UInt<1> | x <= UInt(0) """.stripMargin - val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", - ModuleName(s"C", CircuitName(s"Top"))), + val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", + ModuleName(s"C", CircuitName(s"Top"))), s"topwiring_"), TopWiringOutputFilesAnnotation(testDirName, topWiringTestOutputFilesFunction)) val check = @@ -113,7 +113,7 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { execute(input, check, topwiringannos) } - "The signal x in module C inst c1 and c2" should + "The signal x in module C inst c1 and c2" should s"be connected to Top port with topwiring prefix and outfile in $testDirName" in { val input = """circuit Top : @@ -177,7 +177,7 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { execute(input, check, topwiringannos) } - "The signal x in module C" should + "The signal x in module C" should s"be connected to Top port with topwiring prefix and outputfile in $testDirName, after name colission" in { val input = """circuit Top : @@ -203,8 +203,8 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | output x: UInt<1> | x <= UInt(0) """.stripMargin - val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", - ModuleName(s"C", CircuitName(s"Top"))), + val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", + ModuleName(s"C", CircuitName(s"Top"))), s"topwiring_"), TopWiringOutputFilesAnnotation(testDirName, topWiringTestOutputFilesFunction)) val check = @@ -213,14 +213,16 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | output topwiring_a1_b1_c1_x_0: UInt<1> | inst a1 of A | inst a2 of A_ - | node topwiring_a1_b1_c1_x = UInt<1>("h0") + | wire topwiring_a1_b1_c1_x : UInt<1> + | topwiring_a1_b1_c1_x <= UInt<1>("h0") | topwiring_a1_b1_c1_x_0 <= a1.topwiring_b1_c1_x_0 | module A : | output x: UInt<1> | output topwiring_b1_c1_x_0: UInt<1> | inst b1 of B - | node topwiring_b1_c1_x = UInt<1>("h0") + | wire topwiring_b1_c1_x : UInt<1> | x <= UInt(1) + | topwiring_b1_c1_x <= UInt<1>("h0") | topwiring_b1_c1_x_0 <= b1.topwiring_c1_x | module A_ : | output x: UInt<1> @@ -240,7 +242,7 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { execute(input, check, topwiringannos) } - "The signal x in module C" should + "The signal x in module C" should "be connected to Top port with topwiring prefix and no output function" in { val input = """circuit Top : @@ -262,8 +264,8 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | output x: UInt<1> | x <= UInt(0) """.stripMargin - val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", - ModuleName(s"C", CircuitName(s"Top"))), + val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", + ModuleName(s"C", CircuitName(s"Top"))), s"topwiring_")) val check = """circuit Top : @@ -296,7 +298,7 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { execute(input, check, topwiringannos) } - "The signal x in module C inst c1 and c2 and signal y in module A_" should + "The signal x in module C inst c1 and c2 and signal y in module A_" should s"be connected to Top port with topwiring prefix and outfile in $testDirName" in { val input = """circuit Top : @@ -321,11 +323,11 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | output x: UInt<1> | x <= UInt(0) """.stripMargin - val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", - ModuleName(s"C", CircuitName(s"Top"))), + val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", + ModuleName(s"C", CircuitName(s"Top"))), s"topwiring_"), - TopWiringAnnotation(ComponentName(s"y", - ModuleName(s"A_", CircuitName(s"Top"))), + TopWiringAnnotation(ComponentName(s"y", + ModuleName(s"A_", CircuitName(s"Top"))), s"topwiring_"), TopWiringOutputFilesAnnotation(testDirName, topWiringTestOutputFilesFunction)) val check = @@ -350,8 +352,9 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | module A_ : | output x: UInt<1> | output topwiring_y: UInt<1> - | node y = UInt<1>("h1") + | wire y : UInt<1> | x <= UInt(1) + | y <= UInt<1>("h1") | topwiring_y <= y | module B : | output x: UInt<1> @@ -371,7 +374,7 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { execute(input, check, topwiringannos) } - "The signal x in module C inst c1 and c2 and signal y in module A_" should + "The signal x in module C inst c1 and c2 and signal y in module A_" should s"be connected to Top port with topwiring and top2wiring prefix and outfile in $testDirName" in { val input = """circuit Top : @@ -396,11 +399,11 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | output x: UInt<1> | x <= UInt(0) """.stripMargin - val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", - ModuleName(s"C", CircuitName(s"Top"))), + val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"x", + ModuleName(s"C", CircuitName(s"Top"))), s"topwiring_"), - TopWiringAnnotation(ComponentName(s"y", - ModuleName(s"A_", CircuitName(s"Top"))), + TopWiringAnnotation(ComponentName(s"y", + ModuleName(s"A_", CircuitName(s"Top"))), s"top2wiring_"), TopWiringOutputFilesAnnotation(testDirName, topWiringTestOutputFilesFunction)) val check = @@ -425,8 +428,9 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | module A_ : | output x: UInt<1> | output top2wiring_y: UInt<1> - | node y = UInt<1>("h1") + | wire y : UInt<1> | x <= UInt(1) + | y <= UInt<1>("h1") | top2wiring_y <= y | module B : | output x: UInt<1> @@ -446,7 +450,7 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { execute(input, check, topwiringannos) } - "The signal fullword in module C inst c1 and c2 and signal y in module A_" should + "The signal fullword in module C inst c1 and c2 and signal y in module A_" should s"be connected to Top port with topwiring and top2wiring prefix and outfile in $testDirName" in { val input = """circuit Top : @@ -471,11 +475,11 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | output fullword: UInt<1> | fullword <= UInt(0) """.stripMargin - val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"fullword", - ModuleName(s"C", CircuitName(s"Top"))), + val topwiringannos = Seq(TopWiringAnnotation(ComponentName(s"fullword", + ModuleName(s"C", CircuitName(s"Top"))), s"topwiring_"), - TopWiringAnnotation(ComponentName(s"y", - ModuleName(s"A_", CircuitName(s"Top"))), + TopWiringAnnotation(ComponentName(s"y", + ModuleName(s"A_", CircuitName(s"Top"))), s"top2wiring_"), TopWiringOutputFilesAnnotation(testDirName, topWiringTestOutputFilesFunction)) val check = @@ -500,8 +504,9 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | module A_ : | output fullword: UInt<1> | output top2wiring_y: UInt<1> - | node y = UInt<1>("h1") + | wire y : UInt<1> | fullword <= UInt(1) + | y <= UInt<1>("h1") | top2wiring_y <= y | module B : | output fullword: UInt<1> @@ -576,8 +581,9 @@ class TopWiringTests extends LowTransformSpec with TopWiringTestsCommon { | topwiring_b1_c2_fullword <= b1.topwiring_c2_fullword | module A_ : | output fullword: UInt<1> - | node y = UInt<1>("h1") + | wire y : UInt<1> | fullword <= UInt(1) + | y <= UInt<1>("h1") | module B : | output fullword: UInt<1> | output topwiring_fullword: UInt<1> -- cgit v1.2.3