Support for verilog memory loading. (#840)

* Ability to load memories at simulation startup
* first pass
* create annotation
* create skeleton Transform

* Work in progress
Building out transform and pass now

* Support for LoadMemory annotation
* Creates chisel and firrtl LoadMemory annotations
* LoadMemoryTransform converts annotation into BlackBox InLine
* Simple test that verilog bound modules get created.

* Support for LoadMemory annotation
* Supports Bundled/multi-field memories
* more tests
* support for `$readmemh` and `$readmemb`
* warns if suffix used in file specification.

* Support for LoadMemory annotation
* Use standard chisel annotation idiom

* Support for LoadMemory annotation
* Fixes for @seldridge nits and super-nits

* Support for LoadMemory annotation
- transform now only runs if emitter is an instance of VerilogEmitter
- suffixes on memory text files are now respected
  - if suffix exists and memory is aggregate, aggregate sub-fields will now be inserted before suffix
- every bind module created gets a unique number
  - this is required when multiple loaded memories appear in a module
  - this should be generalized for other uses of binding modules

* Support for LoadMemory annotation
- remove un-needed suffix test

* Support for LoadMemory annotation
- remove instance walk, now just processes each module

* Support for LoadMemory annotation
- Move LoadMemoryTransformation into Firrtl for treadle to access it.

* Support for LoadMemory annotation
- One more bug in suffix handling has been eliminated

* Support for LoadMemory annotation
- remove unused findModule per jackkoenig
- fixed complex test, bad filename edge case

* Support for LoadMemory annotation
- changed to not use intellij style column alignment for : declarations

* Load memory from file
Fixes based on @jkoenig review
- remove unused BindPrefixFactory
- Moved code from CreateBindableMemoryLoaders into to LoadMemoryTransfrom
- Made map to find relevant memory annotations faster
- Made map to find modules referenced by annotations faster
- Made things private that should be private
- DefAnnotatedMemorys are no longer referenced, shouldn't be found here.
- println of error changed to failed

* Loading memories from files
- Many changes based on review
- move stuff into experimental
- clean up annotation manipulation
- manage tests better
- use more standard practices for transform

* Loading memories from files
- More review changes
- Move doc from annotation to the object apply method that generates the annotation
- Make scalastyle directives more specific
- Use more efficient collect to generate name to module map
- Made lines obey style length limit
- a couple of cleanups of imports in tests
- removed some commented out code
- optimized checking for lines using .exists
- use _ for unused variable in match

1 files changed, 188 insertions, 0 deletions

diff --git a/src/test/scala/chiselTests/LoadMemoryFromFileSpec.scala b/src/test/scala/chiselTests/LoadMemoryFromFileSpec.scala
new file mode 100644
index 00000000..330cdd3e
--- /dev/null
+++ b/src/test/scala/chiselTests/LoadMemoryFromFileSpec.scala
@@ -0,0 +1,188 @@
+// See LICENSE for license details.
+
+package chiselTests
+
+import java.io.File
+
+import chisel3._
+import chisel3.util.experimental.loadMemoryFromFile
+import chisel3.util.log2Ceil
+import firrtl.FirrtlExecutionSuccess
+import firrtl.annotations.MemoryLoadFileType
+import org.scalatest.{FreeSpec, Matchers}
+
+class UsesThreeMems(memoryDepth: Int, memoryType: Data) extends Module {
+  val io = IO(new Bundle {
+    val address = Input(UInt(memoryType.getWidth.W))
+    val value1  = Output(memoryType)
+    val value2  = Output(memoryType)
+    val value3  = Output(memoryType)
+  })
+
+  val memory1 = Mem(memoryDepth, memoryType)
+  val memory2 = Mem(memoryDepth, memoryType)
+  val memory3 = Mem(memoryDepth, memoryType)
+  loadMemoryFromFile(memory1, "./mem1")
+  loadMemoryFromFile(memory2, "./mem1")
+  loadMemoryFromFile(memory3, "./mem1")
+
+  io.value1 := memory1(io.address)
+  io.value2 := memory2(io.address)
+  io.value3 := memory3(io.address)
+}
+
+class UsesMem(memoryDepth: Int, memoryType: Data) extends Module {
+  val io = IO(new Bundle {
+    val address = Input(UInt(memoryType.getWidth.W))
+    val value   = Output(memoryType)
+    val value1  = Output(memoryType)
+    val value2  = Output(memoryType)
+  })
+
+  val memory = Mem(memoryDepth, memoryType)
+  loadMemoryFromFile(memory, "./mem1")
+
+  io.value := memory(io.address)
+
+  val low1 = Module(new UsesMemLow(memoryDepth, memoryType))
+  val low2 = Module(new UsesMemLow(memoryDepth, memoryType))
+
+  low2.io.address := io.address
+  low1.io.address := io.address
+  io.value1 := low1.io.value
+  io.value2 := low2.io.value
+}
+
+class UsesMemLow(memoryDepth: Int, memoryType: Data) extends Module {
+  val io = IO(new Bundle {
+    val address = Input(UInt(memoryType.getWidth.W))
+    val value   = Output(memoryType)
+  })
+
+  val memory = Mem(memoryDepth, memoryType)
+
+  loadMemoryFromFile(memory, "./mem2")
+
+  io.value := memory(io.address)
+}
+
+class FileHasSuffix(memoryDepth: Int, memoryType: Data) extends Module {
+  val io = IO(new Bundle {
+    val address = Input(UInt(memoryType.getWidth.W))
+    val value   = Output(memoryType)
+    val value2  = Output(memoryType)
+  })
+
+  val memory = Mem(memoryDepth, memoryType)
+
+  loadMemoryFromFile(memory, "./mem1.txt")
+
+  io.value := memory(io.address)
+
+  val low = Module(new UsesMemLow(memoryDepth, memoryType))
+
+  low.io.address := io.address
+  io.value2 := low.io.value
+}
+
+class MemoryShape extends Bundle {
+  val a = UInt(8.W)
+  val b = SInt(8.W)
+  val c = Bool()
+}
+
+class HasComplexMemory(memoryDepth: Int) extends Module {
+  val io = IO(new Bundle {
+    val address = Input(UInt(log2Ceil(memoryDepth).W))
+    val value   = Output(new MemoryShape)
+  })
+
+  val memory = Mem(memoryDepth, new MemoryShape)
+
+  loadMemoryFromFile(memory, "./mem", MemoryLoadFileType.Hex)
+
+  io.value := memory(io.address)
+}
+
+
+/**
+  * The following tests are a bit incomplete and check that the output verilog is properly constructed
+  * For more complete working examples
+  * @see <a href="https://github.com/freechipsproject/chisel-testers">Chisel Testers</a> LoadMemoryFromFileSpec.scala
+  */
+class LoadMemoryFromFileSpec extends FreeSpec with Matchers {
+  def fileExistsWithMem(file: File, mem: Option[String] = None): Unit = {
+    info(s"$file exists")
+    file.exists() should be (true)
+    mem.foreach( m => {
+      info(s"Memory $m is referenced in $file")
+      val found = io.Source.fromFile(file).getLines.exists { _.contains(s"""readmemh("$m"""") }
+      found should be (true)
+    } )
+    file.delete()
+  }
+
+  "Users can specify a source file to load memory from" in {
+    val testDirName = "test_run_dir/load_memory_spec"
+
+    val result = Driver.execute(
+      args = Array("-X", "verilog", "--target-dir", testDirName),
+      dut = () => new UsesMem(memoryDepth = 8, memoryType = UInt(16.W))    )
+
+    result match {
+      case ChiselExecutionSuccess(_, _, Some(FirrtlExecutionSuccess(_, _))) =>
+        val dir = new File(testDirName)
+        fileExistsWithMem(new File(dir, "UsesMem.UsesMem.memory.v"), Some("./mem1"))
+        fileExistsWithMem(new File(dir, "UsesMem.UsesMemLow.memory.v"), Some("./mem2"))
+        fileExistsWithMem(new File(dir, "firrtl_black_box_resource_files.f"))
+      case _=>
+        throw new Exception("Failed compile")
+    }
+  }
+
+  "Calling a module that loads memories from a file more than once should work" in {
+    val testDirName = "test_run_dir/load_three_memory_spec"
+
+    val result = Driver.execute(
+      args = Array("-X", "verilog", "--target-dir", testDirName),
+      dut = () => new UsesThreeMems(memoryDepth = 8, memoryType = UInt(16.W))
+    )
+
+    result match {
+      case ChiselExecutionSuccess(_, _, Some(FirrtlExecutionSuccess(_, _))) =>
+        val dir = new File(testDirName)
+        fileExistsWithMem( new File(dir, "UsesThreeMems.UsesThreeMems.memory1.v"), Some("./mem1"))
+        fileExistsWithMem( new File(dir, "UsesThreeMems.UsesThreeMems.memory2.v"), Some("./mem1"))
+        fileExistsWithMem( new File(dir, "UsesThreeMems.UsesThreeMems.memory3.v"), Some("./mem1"))
+        fileExistsWithMem( new File(dir, "firrtl_black_box_resource_files.f"))
+      case _=>
+        throw new Exception("Failed compile")
+    }  }
+
+  "In this example the memory has a complex memory type containing a bundle" in {
+    val complexTestDirName = "test_run_dir/complex_memory_load"
+
+    val result = Driver.execute(
+      args = Array("-X", "verilog", "--target-dir", complexTestDirName),
+      dut = () => new HasComplexMemory(memoryDepth = 8)
+    )
+
+    result match {
+      case ChiselExecutionSuccess(_, _, Some(FirrtlExecutionSuccess(emitType, firrtlEmitted))) =>
+        val dir = new File(complexTestDirName)
+        val memoryElements = Seq("a", "b", "c")
+
+        memoryElements.foreach { element =>
+          val file = new File(dir, s"HasComplexMemory.HasComplexMemory.memory_$element.v")
+          file.exists() should be (true)
+          val fileText = io.Source.fromFile(file).getLines().mkString("\n")
+          fileText should include (s"""$$readmemh("./mem_$element", HasComplexMemory.memory_$element);""")
+          file.delete()
+        }
+
+      case _=>
+        fail(s"Failed compile")
+    }
+  }
+
+}