Scala and Cake Patterns and the Problem
Scala and Cake Patterns and the Problem
Standard design patterns in scala recommend the cake pattern to help compose larger programs from smaller ones. Generally, for simple cake layers, this works okay. Boner's article suggests using it to compose repository and service layers and his focus is on DI-type composition. As you abstract more of your IO layers however, you realize that you the cake pattern as described does not abstract easily and usage becomes challenging. As the dependencies mount, you create mixin traits that express those dependence and perhaps they use self-types to ensure they are mixed in correctly.
Then at the end of the world, you have to mix in many different traits to get all the components. In addition, perhaps you have used existential types and now you must have a val/object somewhere (i.e. a well defined path) in order to import the types within the service so you can write your program. Existential types within the cake layer, say within the service definition itself, require path dependent types to access the types for use in the methods of that layer. This is standard scala. You don't have to use path dependent types but if you want to avoid mixing objects together then path-dependent types can help you.
Generally, I do not think the value proposition for scala and the cake patterns/compositional patterns are the key value proposition that scala brings to the table. In may ways, the container-less approach with scala gives you some more options and capabilities, but I'm not sure they are 100x type improvements over not having containers. Where scala does provide tremendous value is allowing you to refine, extend and also integrate these components together. For me, this was the important part of the value proposition in this blog versus a specific compositional approach. I have used spring containers extensively and found them easy to use. When software gets complex, whether its spring or scala-oriented techniques, there are always issues to be solved. The option of using scala in the ways described in this blog and that do not involve proxies or byte-code generation is of value to the software I am concerned about.
Scala provides a variety of ways to handle object composition and the "constructor/member" "setting/injection" dilemma with implicits, constructor args, objects, vals and cake patterns but mixing them altogether can be tough at times.
Let's say you define a ''notes'' service that saves notes in a specialized data store (note that this is also a cake layer by itself). The rest of the database support needed in the application is handled by other database techniques. Let's just focus on the NotesService.
You realize that to use this service, you'll need to use path dependent types in your code and there's alot of refinement that will go on for both the TraitService "container" as well as those existential types. In addition, as you subclass the trait to specialize it for specific backing stores, you realize that you want to keep this NotesService trait separate from a specific cake pattern e.g. NotesServiceComponent because it is either owned by someone else er you do not want to constrain how this service is used.
But lets say you want to use this service in a cake layer and embed it directly in the cake. So you write the standard definition recognizing that subclasses must also mixin the specific backend (e.g. a RDBMS backend configuration)required by that subclass.
For example, doing a standard "DI component" cake layer:
This is rather restricting in that you are mixing in the NotesService with a specification of how you want that object to be declared. As you go through the layers and subclasses, you start using self-types and suddenly you are forcing your clients to use a very specific, and perhaps tangled, way of using your NotesService. I've found that once you start using the DI-component cake layer model, everything has to start staying in the cake layer because it becomes difficult to evolve both the "component" cake layer and the "service" component over time. So using the cake pattern directly for the trait limits how it is consumed. We would like to keep it separate but still allow us to access those internal, path-dependent types easily to write our code. This means we have to statically declare the value somewhere so that it has a well-formed path.
Let's say you are handed the NotesService class along with a subclass:
You may need to make a specific implementation of this service. The standard approach is to define a subclass of the service. Let's assume that you want a slick backend:
Now we have a slick backend, but it has a dependency. It expects to be mixed in with a DatabaseAware trait that holds the actual database. Its rather prescriptive here in how it used self-types and force the mixing. NoteService was fairly open in its usage but now its a bit more restricted in how it can be used. If the SlickNotesService had been dependent on types inside the actual backend driver, the
def databatasemay need to be a val, but a
def databasemeans that the database can be created when the application is ready to create it and it does not need to be created statically. That's the right model for an application that configures itself at the start.
Now we can use our driver very easily like below:
We may also realize that it would be nice to add some helper methods so we created the Helpers object inside the service.
But where should these go? How do you keep these helper methods together? Do we always have to create them inside the class? What if we want to add more helpers? I created a nice anchored object in the above test program but do I always have to specify everything at program start? What's the best way to combine this service with 10 other services I need so its easy to figure out all the dependent objects you need for your application?
Again, if everything is statically known, including the actual database connection parameters at the start of the program, then this is a mute point. I mostly encounter designs where the database link is configured at runtime in some way and not from static config files only. And its clear that the incoming data components may have very different design that force a specific way of managing your dependencies and objects. If everything is statically known, we could change the
def databaseinto a val and define it directly.
Some applications components are simple and are provided to you in a simple DI-dependency cake like boner describer, some look like NotesService and some may come in with specific implementation designs such as the self-type in SlickNotesDriver. You'll need to be able to handle all the different types to fuse them together.
To use the SlickNotesService is not hard. But let's look at what happens with some variations of application configuration. We'll assume that we need to initialize the database in some way and that the NotesService subclasses like SlickNotesService are not dependent on the actual backend types that require path-dependent type resolution. Slick is a stateless backend that lifts queries into its driver environment. So it is fairly flexible in its design.
There are multiple issues here including making ObjectRegistry inherit from many different components which is distracting as well as that the ObjectRegistry is actually now a NotesService object itself! That may not be bad, but it looks goofy. Also, we are importing the contents of the entire ObjectRegistry in to get easier access to the path-dependent types which is not good.
Now we want to use composition a bit more:
Its around this point, you think it is easier to just add a Database constructor parameter for the database, or perhaps an implicit constructor value. You can make the giant object extension signature go away by using an intermediate definition so its much shorter but the real issue is that you have to have very specific knowledge about how to use the provided NotesService.
Let's still keep the NotesService separate but define a small cake layer to handle how the mixin occurs.
So with this approach, again keeping NotesService separate, we are now in the same place we were before except we did indicate that the ObjectRegistry is indeed a provider of a NotesService object. So this did not buy us too much.
We can also create a separate, configured object with everything we need and then set members into this object that are needed for configuration (or if the configuration object is not shared, created directly in the separate object). This essentially creates the notes service object externally and is not much different than the other options above.
The nice thing about this approach is that it is all kept separate. However, if you have a few services that have the same dependencies, you'll have to grab those dependencies wherever they are created and pull them back into ObjectRegistry--that's not a scalabale way to do this if you have several services with several dependencies. What is nice about this approach is MyNotesService plays the role of a "driver" in the sense that it pulls together the service and the dependencies all in one place.
Another approach is to create a separate shared configuration object and use more existential types to provide a centralized configuration point within a central, well known location like the ObjectRegistry. You have a couple of ways to do this but the example below shows this approach with two services one that is externally given to you that you cannot change how it has expressed its dependencies (NotesService) and one that you have control over the code (AnotherService). ConfigNotesService could, if it wanted to, inherit from SlickNotesService as well. There are a few variations you can make on the below that also make sense.
This is actually an interesting approach in that we are using a separate "configuration" object that creates well-known paths for the application to work from. But depending on how your service is defined, you may have to have some supporting declarations (boilerplate) as we needed for the NotesService. AnotherService had already been adapted to work with the config object so there was no boilerplate for that service. The use of a config object forced, at least, the dependencies to be created even if the NotesService required some boilerplate to make sure the scope of the dependencies was available to the NotesService instance.
We can also once again see that if you have control over your code, then like AnotherService, you can bake in your dependency approach quite easily. But if the code is given to you and you cannot change it, as in NotesService, you have to adapt around it.
The key lesson from this option is that you can define the service instance, you can define some type of separate dependency object (in this case ConfigComponent) and you can have some behavior that initializes (creates or configs) the dependencies. So that's three separate parts that are needed to form a general solution to managing dependencies regardless of whether you control the service codebase or not.
So its pretty clear that to use the NotesService in traditional ways, is a bit tough. Not overly hard or impossible by any stretch and you have very fine-grained choices on how to mix it in. It's important to realize that by keeping NotesServices separate, you preserve those choices for the client. That's good.
But we have also seen where if we have different requirements and needs, it gets difficult to know how to create the mix. It is highly dependent on what you have coming in the door and their specific formulation. So if we look at the different scenarios we may encounter:
- Services may be statically defined completely at the start or must be "configured" during runtime.
- Services may use complex types that require path-dependent types to be used, hence, there must be a well-formed path.
- Services have different configuration requirements (e.g. constructor, implicits, self-types)
- Services may have dependencies (and many of them) that are statically known at the start or must be "configured" during runtime
- Services may have helper methods that are provided or not provided to you and they have to be easy to find and use.
- Services may be created by you or someone else and you have to adapt them to your application in many different ways.
- Services may need to be composed into other objects or inherited in
That's alot of variety. It an be said however, that all problems can be solved through enough layers of abstraction...we need to
- Find a way to force an object, like ObjectRegistry, to declare a service
- Find a way to declare a service that sometimes requires well-formed paths to help with path-dependent types
- Find a way to declare a set of dependencies for that service
- Find a way to allow dependencies to be shared, when needed, across services
- Find a way to initialize/change those dependencies to be ready for use
- Find a way to allow helper objects and definitions be available
- Find a way to bundle all the above together in a coherent, consistent approach that also balances DRY principles
More on this in the next blog. Stay tuned!
Read the rest on gisthub: here