Scala and Cake Patterns and the Problem

Scala and Cake Patterns and the Problem

Standard design patterns in scala recommend the cake pattern to help compose larger programs from smaller ones. Generally, for simple cake layers, this works okay. Boner's article suggests using it to compose repository and service layers and his focus is on DI-type composition. As you abstract more of your IO layers however, you realize that you the cake pattern as described does not abstract easily and usage becomes challenging. As the dependencies mount, you create mixin traits that express those dependence and perhaps they use self-types to ensure they are mixed in correctly.
Then at the end of the world, you have to mix in many different traits to get all the components. In addition, perhaps you have used existential types and now you must have a val/object somewhere (i.e. a well defined path) in order to import the types within the service so you can write your program. Existential types within the cake layer, say within the service definition itself, require path dependent types to access the types for use in the methods of that layer. This is standard scala. You don't have to use path dependent types but if you want to avoid mixing objects together then path-dependent types can help you.
Generally, I do not think the value proposition for scala and the cake patterns/compositional patterns are the key value proposition that scala brings to the table. In may ways, the container-less approach with scala gives you some more options and capabilities, but I'm not sure they are 100x type improvements over not having containers. Where scala does provide tremendous value is allowing you to refine, extend and also integrate these components together. For me, this was the important part of the value proposition in this blog versus a specific compositional approach. I have used spring containers extensively and found them easy to use. When software gets complex, whether its spring or scala-oriented techniques, there are always issues to be solved. The option of using scala in the ways described in this blog and that do not involve proxies or byte-code generation is of value to the software I am concerned about.
Scala provides a variety of ways to handle object composition and the "constructor/member" "setting/injection" dilemma with implicits, constructor args, objects, vals and cake patterns but mixing them altogether can be tough at times.
Let's say you define a ''notes'' service that saves notes in a specialized data store (note that this is also a cake layer by itself). The rest of the database support needed in the application is handled by other database techniques. Let's just focus on the NotesService.
trait NotesService {

  type NoteId
  type FindNoteInfo <: class="kt" color="#445588" font="" style="font-weight: bold;">FindNoteInfoLike
  type Note <: class="kt" color="#445588" font="" style="font-weight: bold;">NoteLike
  type NoteSource <: class="kt" color="#445588" font="" style="font-weight: bold;">NoteSourceLike[_]

  /**
   * Find a note based on search criteria.
   */
  def find(info: FindNoteInfo): Option[Note]

  /**
   * Obtain the actual note based on the id.
   */
  def get(id: NoteId): Option[Note]

  /**
   * Save or update the note. Use the returned Note. The
   * input Note is no longer valid.
   */
  def saveOrUpdate(note: Note): Note

  /**
   * Delete a note. This may or may not result in a physical
   * deletion.
   */
  def delete(id: NoteId)

  /**
   * The source of a note when a note needs to be read into the
   * service or out of the service.
   */
  trait NoteSourceLike[T] {
    def source: T
  }

  /**
   * Search criteria info for finder methods.
   */
  trait FindNoteInfoLike

  /**
   * The actual note object for use by the application.
   * The content can be obtained in a lazy fashion.
   */
  trait NoteLike {
    def id: NoteId
    def content: Option[NoteSource]
  }

  /**
   * Used to indicate that a note has not been saved yet.
   */
  val UnassignedNoteId: NoteId

}
You realize that to use this service, you'll need to use path dependent types in your code and there's alot of refinement that will go on for both the TraitService "container" as well as those existential types. In addition, as you subclass the trait to specialize it for specific backing stores, you realize that you want to keep this NotesService trait separate from a specific cake pattern e.g. NotesServiceComponent because it is either owned by someone else er you do not want to constrain how this service is used.
But lets say you want to use this service in a cake layer and embed it directly in the cake. So you write the standard definition recognizing that subclasses must also mixin the specific backend (e.g. a RDBMS backend configuration)required by that subclass.
For example, doing a standard "DI component" cake layer:
trait NotesServiceComponent {

  val notesService: NotesService

  trait NotesService { ...see text above...}
}
This is rather restricting in that you are mixing in the NotesService with a specification of how you want that object to be declared. As you go through the layers and subclasses, you start using self-types and suddenly you are forcing your clients to use a very specific, and perhaps tangled, way of using your NotesService. I've found that once you start using the DI-component cake layer model, everything has to start staying in the cake layer because it becomes difficult to evolve both the "component" cake layer and the "service" component over time. So using the cake pattern directly for the trait limits how it is consumed. We would like to keep it separate but still allow us to access those internal, path-dependent types easily to write our code. This means we have to statically declare the value somewhere so that it has a well-formed path.

Evolution of the Service

Let's say you are handed the NotesService class along with a subclass:
trait RdbmsNotesService extends NotesService {

  /**
   * Subclasses must defined the actual id structure.
   */
  type RdbmsId

  /**
   * Subclasses must define what an unassigned id looks like.
   */
  val UnassignedNoteId: NoteId

  /**
   * To find note, enter some string criteria.
   */
  case class FindNoteInfo(criteria: String) extends FindNoteInfoLike

  /**
   * To put or get the actual note, input streams are used.
   * This means notes can be binary objects as well or a 
   * picture or ... alright, just use a string for now.
   */
  case class NoteSource(source: String) extends NoteSourceLike[String]

  val EmptyNoteSource = NoteSource(null)

  /**
   * Represent a node id by a parameterized type.
   * 
   * You cannot use a value class here due to a variety of
   * language restrictions.
   */
  case class NoteId(val id: RdbmsId)

  /**
   * A note has a MIME type as well as the content.
   */
  trait Note extends NoteLike {
    def mimeType: String
  }
}
You may need to make a specific implementation of this service. The standard approach is to define a subclass of the service. Let's assume that you want a slick backend:
trait ProfileAware {
  import slick.driver.JdbcProfile
  implicit protected val profile: JdbcProfile
}

trait DatabaseAware extends ProfileAware {
  implicit def database: profile.simple.Database
}

trait NotesComponents { self: ProfileAware =>
  import profile.simple._

  val html = "text/html"
  val plain = "text/plain"

  case class Mime(name: String, description: Option[String])

  class Mimes(tag: Tag) extends Table[Mime](tag, "Mimes") {
    def mime = column[String]("mime", O.PrimaryKey)
    def description = column[Option[String]]("description")
    def * = (mime, description) <> (Mime.tupled, Mime.unapply)
  }

  class Notes(tag: Tag) extends Table[(Long, String, Option[String])](tag, "Notes") {
    def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
    def mime = column[String]("mime")
    def content = column[Option[String]]("content")
    def * = (id, mime, content)
    def mimeFk = foreignKey("mimeFK", mime, Mimes)(_.mime)
  }

  lazy val Mimes = TableQuery[Mimes]
  lazy val Notes = TableQuery[Notes]

}

/**
 * Notes service using slick, and hence JDBC, for the backend.
 * Since the {{{NotesService}}} has path-dependent types, you can
 * only access the {{{Note, NoteId, ...}}} types through the
 * instance.
 */
trait SlickNotesService extends RdbmsNotesService {
  self: DatabaseAware with NotesComponents =>

  type RdbmsId = Long

  override val UnassignedNoteId: NoteId = NoteId(-1)

  import profile.simple._

  case class Note(id: NoteId = UnassignedNoteId, mimeType: String,
    content: NoteSource) extends super.Note

  /**
   * Import this to obtain some helper functions.
   */
  object Helpers {

    implicit def long2NoteId(id: Long): NoteId = new NoteId(id)

    implicit def string2NoteSource(source: Option[String]): NoteSource =
      source match {
        case None => EmptyNoteSource
        case Some(v) => NoteSource(v)
      }

    def noteSource2String(ns: NoteSource): Option[String] =
      ns match {
        case EmptyNoteSource => None
        case ns: NoteSource => Option(ns.source)
        case _ => None
      }
  }

  import Helpers._

  def find(info: FindNoteInfo): Option[Note] = {
    database.withTransaction {
      implicit session =>
        None
    }
  }

  def delete(id: NoteId): Unit = {
    database.withTransaction {
      implicit session =>
        getNoteById(id.id).delete
    }
  }

  private def getNoteById(id: Long) =
    for (n <- span=""> Notes if n.id === id) yield n

  def get(id: NoteId): Option[Note] = {
    database.withTransaction { implicit session =>
      getNoteById(id.id).firstOption match {
        case Some(n) =>
          Some(Note(id = NoteId(n._1), mimeType = n._2,
            content = Helpers.noteSource2String(n._3)))
        case _ => None
      }
    }
  }

  /**
   * @todo perform versioning and save previous version
   */
  def saveOrUpdate(note: Note): Note = {
    database.withTransaction { implicit session =>
      val id = note.id.id
      // Find existing note, if it exists
      getNoteById(id).firstOption match {
        case Some(n) =>
          // update it
          Notes.update((id, note.mimeType, noteSource2String(note.content)))
          note
        case _ =>
          // create a new one
          val newId = (Notes returning Notes.map(_.id)) insert
            (id, note.mimeType, noteSource2String(note.content))
          new Note(NoteId(newId), note.mimeType, note.content)
      }
    }
  }

}
Now we have a slick backend, but it has a dependency. It expects to be mixed in with a DatabaseAware trait that holds the actual database. Its rather prescriptive here in how it used self-types and force the mixing. NoteService was fairly open in its usage but now its a bit more restricted in how it can be used. If the SlickNotesService had been dependent on types inside the actual backend driver, thedef databatase may need to be a val, but a def database means that the database can be created when the application is ready to create it and it does not need to be created statically. That's the right model for an application that configures itself at the start.
Now we can use our driver very easily like below:
object TestNotesService {

  import slick.driver.H2Driver

  object Service extends SlickNotesService
    with DatabaseAware with NotesComponents {
    val profile = slick.driver.H2Driver.profile
    val database = profile.simple.Database.forURL(url = "jdbc:h2:mem:test;DB_CLOSE_DELAY=-1",
      driver = "org.h2.Driver")
  }

  import Service._
  import profile.simple._
  import Service.Helpers._

  def main(args: Array[String]): Unit = {
    database.withSession { implicit session =>
      val ddl = Notes.ddl ++ Mimes.ddl
      println("ddl: " + ddl.createStatements.mkString(";\n"))
      ddl.create
      Mimes ++= Seq(Mime(html, Some("html notes")),
        Mime(plain, Some("text notes")))
      require(Mimes.filter(_.mime === plain).firstOption.isDefined)
    }

    var note = new Note(mimeType = plain, content = EmptyNoteSource)
    note = saveOrUpdate(note)
    println("Initial Note: " + note)
    var newNote = note.copy(content = NoteSource("Updated note!"))
    newNote = saveOrUpdate(newNote)
    println("Updated Note: " + newNote)
    val aNoteId = newNote.id
    println("Note id: " + aNoteId)
    val foundNote = get(aNoteId)
    println("Found updated Note: " + foundNote)
  }
}
We may also realize that it would be nice to add some helper methods so we created the Helpers object inside the service.
But where should these go? How do you keep these helper methods together? Do we always have to create them inside the class? What if we want to add more helpers? I created a nice anchored object in the above test program but do I always have to specify everything at program start? What's the best way to combine this service with 10 other services I need so its easy to figure out all the dependent objects you need for your application?
Again, if everything is statically known, including the actual database connection parameters at the start of the program, then this is a mute point. I mostly encounter designs where the database link is configured at runtime in some way and not from static config files only. And its clear that the incoming data components may have very different design that force a specific way of managing your dependencies and objects. If everything is statically known, we could change the def database into a val and define it directly.
Some applications components are simple and are provided to you in a simple DI-dependency cake like boner describer, some look like NotesService and some may come in with specific implementation designs such as the self-type in SlickNotesDriver. You'll need to be able to handle all the different types to fuse them together.
To use the SlickNotesService is not hard. But let's look at what happens with some variations of application configuration. We'll assume that we need to initialize the database in some way and that the NotesService subclasses like SlickNotesService are not dependent on the actual backend types that require path-dependent type resolution. Slick is a stateless backend that lifts queries into its driver environment. So it is fairly flexible in its design.

Using NotesService

Configure Using Inheritance

object ObjectRegistry extends SlickNotesService with DatabaseAware with TableComponents with TableQueryComponents {
 import H2Driver.profile._
 val profile: profile
 // create a singleton database object for the app, but not statically defined since we need to use slick to create the Database
 def database: profile.simple.Database = _database
 private var _database: profile.simple.Database

 def init(url: String): Unit = { _database = Database.forUrl(url) }
}

...in another program area...
import ObjectRegistry._
var newNote = Note(UnAssignedNoteId, plain, Option(EmptyNoteSource))
newNote = save(newNote)
There are multiple issues here including making ObjectRegistry inherit from many different components which is distracting as well as that the ObjectRegistry is actually now a NotesService object itself! That may not be bad, but it looks goofy. Also, we are importing the contents of the entire ObjectRegistry in to get easier access to the path-dependent types which is not good.

Configure Using Composition

Now we want to use composition a bit more:
object ObjectRegistry {
 object notesService extends SlicksNotesService with DatabaseAware with TableComponents with TableQueryComponents {
   import H2Driver.profile._
   val profile: profile
   def database: profile.simple.Database = _database
   private var _database: profile.simple.Database
   def init(url: String): Unit = { _database = Database.forUrl(url) }
 }
}

....in another program area...
import ObjectRegistry._
notesService.init(yourUrl)
...
import notesService._
var newNote = Note(UnAssignedNoteId, plain, Option(EmptyNoteSource))
newNote = save(newNote)
...
Its around this point, you think it is easier to just add a Database constructor parameter for the database, or perhaps an implicit constructor value. You can make the giant object extension signature go away by using an intermediate definition so its much shorter but the real issue is that you have to have very specific knowledge about how to use the provided NotesService.

Use a Standard Cake Layer to Mix in the NotesService

Let's still keep the NotesService separate but define a small cake layer to handle how the mixin occurs.
trait NotesServiceComponent { 
  val notesService: NotesService
}

object ObjectRegistry with NotesServiceComponent {
  val notesService: SlickNotesService = new SlickNotesService with ... { ... }
}
...or...
object ObjectRegistry with NotesServiceComponent {

 object notesService extends SlickNotesService with ... { ... }
}
So with this approach, again keeping NotesService separate, we are now in the same place we were before except we did indicate that the ObjectRegistry is indeed a provider of a NotesService object. So this did not buy us too much.

Creating a Separate Object

We can also create a separate, configured object with everything we need and then set members into this object that are needed for configuration (or if the configuration object is not shared, created directly in the separate object). This essentially creates the notes service object externally and is not much different than the other options above.
...some other file...
object MyNotesService extends SlickNotesService with TableComponents with TableQueryComponents with DatabaseAware
{
   import H2Driver.profile._
   val profile: profile
   def database: profile.simple.Database = _database
   private var _database: profile.simple.Database
   def init(url: String): Unit = { _database = Database.forUrl(url) }
}

....in your main app...
object ObjectRegistery {

  val notesService = MyNotesService

}
The nice thing about this approach is that it is all kept separate. However, if you have a few services that have the same dependencies, you'll have to grab those dependencies wherever they are created and pull them back into ObjectRegistry--that's not a scalabale way to do this if you have several services with several dependencies. What is nice about this approach is MyNotesService plays the role of a "driver" in the sense that it pulls together the service and the dependencies all in one place.

Creating a Separate Config Object

Another approach is to create a separate shared configuration object and use more existential types to provide a centralized configuration point within a central, well known location like the ObjectRegistry. You have a couple of ways to do this but the example below shows this approach with two services one that is externally given to you that you cannot change how it has expressed its dependencies (NotesService) and one that you have control over the code (AnotherService). ConfigNotesService could, if it wanted to, inherit from SlickNotesService as well. There are a few variations you can make on the below that also make sense.
// This looks alot like the NotesService! It's the same idea.
trait ConfigComponent {
  trait Config
  val config: Config
}

trait ConfigNotesService extends ConfigComponent {
  type Config <: span=""> ConfigDef

  trait ConfigDef extends super.Config {
   val profile: JdbcProfile
   def database: profile.simple.Database = _database
  }

  val notesService: SlickNotesService
}

trait ConfigAnotherService extends AnotherService with ConfigComponent ... {
   type Config <: span=""> ConfigDef

   trait ConfigDef extends super.config {
     val moreConfig: String
   }

  val anotherService: AnotherService

  trait AnotherService extends super.AnotherService {
     ...something that is designed to use ConfigDef directly...
  }
}

object ObjectRegistry extends ConfigNotesService with ConfigAnotherService {

  type Config = config.type

  object config extends super[ConfigNotesService].ConfigDef with super[ConfigAnotherService].ConfigDef {
     val moreConfig = "aValue"
     import H2Driver.profile._
     val profile: profile
     val database = Database.fromDataSource("jdbc2:h2:mem:app") // or use a def and private val as needed
  }

  val notesService = new SlickNotesService with TableQueriesComponent with TableComponents with DatabaseAware {
    // Some boilerplate...
    val profile = config.profile
    def database = config.database
  }
  val anotherService: new AnotherService { } // service defined within the cake layer ConfigAnotherService

  // more registry objects here
}
This is actually an interesting approach in that we are using a separate "configuration" object that creates well-known paths for the application to work from. But depending on how your service is defined, you may have to have some supporting declarations (boilerplate) as we needed for the NotesService. AnotherService had already been adapted to work with the config object so there was no boilerplate for that service. The use of a config object forced, at least, the dependencies to be created even if the NotesService required some boilerplate to make sure the scope of the dependencies was available to the NotesService instance.
We can also once again see that if you have control over your code, then like AnotherService, you can bake in your dependency approach quite easily. But if the code is given to you and you cannot change it, as in NotesService, you have to adapt around it.
The key lesson from this option is that you can define the service instance, you can define some type of separate dependency object (in this case ConfigComponent) and you can have some behavior that initializes (creates or configs) the dependencies. So that's three separate parts that are needed to form a general solution to managing dependencies regardless of whether you control the service codebase or not.

Using Something Else to Handle Composition and Abstracting

So its pretty clear that to use the NotesService in traditional ways, is a bit tough. Not overly hard or impossible by any stretch and you have very fine-grained choices on how to mix it in. It's important to realize that by keeping NotesServices separate, you preserve those choices for the client. That's good.
But we have also seen where if we have different requirements and needs, it gets difficult to know how to create the mix. It is highly dependent on what you have coming in the door and their specific formulation. So if we look at the different scenarios we may encounter:
  • Services may be statically defined completely at the start or must be "configured" during runtime.
  • Services may use complex types that require path-dependent types to be used, hence, there must be a well-formed path.
  • Services have different configuration requirements (e.g. constructor, implicits, self-types)
  • Services may have dependencies (and many of them) that are statically known at the start or must be "configured" during runtime
  • Services may have helper methods that are provided or not provided to you and they have to be easy to find and use.
  • Services may be created by you or someone else and you have to adapt them to your application in many different ways.
  • Services may need to be composed into other objects or inherited in
That's alot of variety. It an be said however, that all problems can be solved through enough layers of abstraction...we need to
  • Find a way to force an object, like ObjectRegistry, to declare a service
  • Find a way to declare a service that sometimes requires well-formed paths to help with path-dependent types
  • Find a way to declare a set of dependencies for that service
  • Find a way to allow dependencies to be shared, when needed, across services
  • Find a way to initialize/change those dependencies to be ready for use
  • Find a way to allow helper objects and definitions be available
  • Find a way to bundle all the above together in a coherent, consistent approach that also balances DRY principles
More on this in the next blog. Stay tuned!

Read the rest on gisthub: here

Comments

Post a Comment

Popular posts from this blog

quick note on scala.js, react hooks, monix, auth

quick note on scala.js, react hooks, monix, auth Reactjs hooks finally came out of preview in a recent version of React. scalajs-reaction has had support for hooks since they were in preview. React hooks seem like the hot new thing but for me the question is, do hooks help improve your code by improving maintainability, time to market, TCO, etc.? Let’s take a look using authentication as an example and employ monix , a reactive observable framework that also builds on cats . We will assume we are building a SPA. Does it improve your code and make your application & development experience better? Auth Auth processing can be complex. Many react auth examples you find in blogs like to show how to use “react-routing” and “auth” as a switch that determines which routes are protected or unprotected. Protected routes show application specific data. Unprotected routes show login content . Based on storing a “User” object in the browser session/local storage, th...
Read more

attributes with react and typescript.md

If you use typescript with react you may wonder what the right signature is for your “properties” interface. Obviously, your component will have few to many specific properties that reflect your domain. However, you may wish to also provide for the ability to add other attributes, such as those valid for a div property. In other words, you want to extend your props interface with that of the properties from a div element. How do you do that? There are really two issues: How do you extend your props interfaces How do you filter the props once you receive them to just those that apply. Extend Your Prop Interface or Use Sum Types It’s not uncommon to see interface Props { className ? : string yourProp ? : number } with the idea of using it in class MyComponent extends React.Component<Props, any> somewhere. If your component will have an outer div, you may also pass in a react element so that its configurable. But let’s say your outer element is a div and you wan...
Read more

zio environment and modules pattern: zio, scala.js, react, query management

zio environment and modules pattern: zio, scala.js, react, query management This blog covers a zio-based fetch design for scala.js, react web applications. It heavily uses the zio “environment” feature to help motivate a point of view on the modules pattern in effect systems. At the end, I summarize some thoughts on ergonomics and complexity. This blog assumes a basic knowledge of zio and react. scala.js is just like scala so you mostly already know scala.js. This blog is a follow-on to previous blogs: https://appddeevvmeanderings.blogspot.com/2019/10/scalajs-react-fetcher-hook_48.html and https://appddeevvmeanderings.blogspot.com/2019/11/scalajs-and-react-suspense.html . I have been keeping an eye on and culling ideas from typscript/javascript fetching libraries that address query management concerns. The formulation below incorporates some of those concerns but does not try to comprehensively duplicate well established javascript libraries. I only care about ...
Read more