scala, slick and NotesService

I forgot to add a full version of the NotesService that was discussed in a previous blog.

Let's revisit the NotesService interface:

/**
 * Primarily for larger notes/docs storage.
 * The API allows the note content to be retrieved
 * separately. Sometimes, we want to separate
 * out the larger, less changing content from
 * our core database into a separate component. A note
 * can be a picture, a spreadsheet, HTML, XML or
 * even just a text document. While times,
 * these types of documents are just stored in a
 * standard RDBMS database, there are other options
 * such as a document management systems or
 * file-system.
 *
 * This trait does not prescribe how to
 * create {{{Note}}} objects. Also, the {{{Note}}},
 * {{{NoteId}}} and other types will probably escape
 * whatever object instantiates this object
 * so the types will need to be accessible to the outer
 * world for working with the service. That means
 * we will have to watch-out for path-dependent
 * type accesss issues as we employ this service.
 *
 */
trait NotesService {

  type NoteId
  type FindNoteInfo <: FindNoteInfoLike
  type Note <: NoteLike
  type NoteSource <: NoteSourceLike[_]

  /**
   * Find a note based on search criteria.
   */
  def find(info: FindNoteInfo): Option[Note]

  /**
   * Obtain the actual note based on the id.
   */
  def get(id: NoteId): Option[Note]

  /**
   * Save or update the note. Use the returned Note. The
   * input Note is no longer valid. Error messages are
   * return on the left.
   */
  def saveOrUpdate(note: Note): Either[String, Note]

  /**
   * Delete a note. This may or may not result in a physical
   * deletion.
   */
  def delete(id: NoteId)

  /**
   * The source of a note when a note needs to be read into the
   * service or out of the service.
   */
  trait NoteSourceLike[T] {
    def source: T
  }

  /**
   * Search criteria info for finder methods.
   */
  trait FindNoteInfoLike

  /**
   * The actual note object for use by the application.
   * The content can be obtained in a lazy fashion.
   */
  trait NoteLike {
    def id: NoteId
    def content: NoteSource
  }

  /**
   * Used to indicate that a note has not been saved yet.
   */
  val UnassignedNoteId: NoteId

  /**
   * An empty note value.
   */
  val EmptyNoteSource: NoteSource
}

We want to provide 2 specalizations. One for using a RDBMS and as such the notes content should be stored as a byte array so it can easily be moved into a RDBMS using a blob and the other for handling revisions to notes:

/**
 * A notes service that deals with mime types, uses byte arrays to read
 * the data in and out of the service. A specific {{{NoteId}}} structure
 * is not specified. The byte arrays are interpreted using {{{MIME}}}
 * specifications.
 *
 */
trait RdbmsNotesService extends NotesService {

  /**
   * To find note, enter some string criteria.
   */
  case class FindNoteInfo(criteria: String) extends FindNoteInfoLike

  /**
   * To put or get the actual note, input streams are used. This
   * allows you to construct any type of note-like object out of the
   * bytes stored in the repository. If the byte array represents
   * character data, {{{charset}}} specifies the character set.
   */
  case class NoteSource(source: Array[Byte], charset: String = Charset.defaultCharset.name) extends NoteSourceLike[Array[Byte]] {
    def asString = new String(source, Charset.forName(charset))
  }

  /**
   * Subclasses or external clients can instantiate for helper methods.
   */
  trait Helpers {

    /**
     * Convert a string to an {{{NoteSource}}}
     */
    implicit def string2NoteSource(source: Option[String]): NoteSource =
      source match {
        case None => EmptyNoteSource
        case Some(v) =>
          NoteSource(v.getBytes)
      }

    /**
     * Convert a {{{NoteSource}}} to an optional string. This
     * function cannot check whether the NoteSource holds
     * a string unless you use {{{EmptyNoteSource}}}.
     */
    implicit def noteSource2String(ns: NoteSource): Option[String] =
      ns match {
        case EmptyNoteSource => None
        case ns: NoteSource => Some(new String(ns.source, ns.charset))
        case _ => None
      }
  }

  /**
   * The empty NoteSource.
   */
  val EmptyNoteSource = NoteSource(null)

  /**
   * A note has a MIME type as well as the content.
   */
  trait NoteLike extends super.NoteLike {
    def mimeType: String
  }
}

/**
 * Support revisions on the notes. In order to not be too specific about how revisions
 * are stored, e.g. an {{{id}}} of some sort, we just provide a way
 * to see how many revisions are available and retrieve the previous
 * or next revision given a specific {{{Note}}. Subclasses can add more customized
 * access methods. This also implies that the {{{Note}}} instance
 * must carry some type of information that allows the previous or
 * next note to be found.
 */
trait NotesServiceWithRevisions extends NotesService {

  /**
   * Given a note, return the total number of revisions
   * including forward and backward ones.
   */
  def getRevisionCount(id: NoteId): Int

  /**
   * Obtain the previous revision, if any.
   */
  def getPrevious(id: Note): Option[Note]

  /**
   * Obtain the next revision, if any.
   */
  def getNext(id: Note): Option[Note]

}

We keep the revision enhancement rather simple. Normally, people define some type of revision id structure, but that's not needed if you think of revisions as a linked list. This makes more easily implemented in other databases, such as a graph database.

The implementation is now fairly straight-forward. We'll choose to keep everything in a single table. We are assuming that is not an intensive document management database, but something to store notes expressed as text content e.g. HTML, plain text or other markup formats.

trait NotesTableComponents2 { self: ProfileAware =>
  import profile.simple._

  val html = "text/html"
  val plain = "text/plain"

  case class Mime(name: String, description: Option[String])

  class Mimes(tag: Tag) extends Table[Mime](tag, "Mimes") {
    def mime = column[String]("mime", O.PrimaryKey)
    def description = column[Option[String]]("description")
    def * = (mime, description) <> (Mime.tupled, Mime.unapply)
  }

  case class NotesDTO(id: Long, docId: String, mime: String,
    previousId: Option[Long], createdOn: java.sql.Timestamp,
    latest: Boolean, content: java.sql.Blob, deleted: Option[Boolean])

  /**
   * Implement versions using a singly linked list in the same table.
   */
  class Notes(tag: Tag) extends Table[NotesDTO](tag, "Notes") {
    def id = column[Long]("id", O.AutoInc)
    def docId = column[String]("docId") // UUID
    def mime = column[String]("mime")
    def previousId = column[Option[Long]]("previousId")
    def createdOn = column[java.sql.Timestamp]("createdOn")
    def latest = column[Boolean]("latest")
    def content = column[java.sql.Blob]("content")
    def deleted = column[Option[Boolean]]("deleted")

    def * = (id, docId, mime, previousId, createdOn, latest, content, deleted) <>
      (NotesDTO.tupled, NotesDTO.unapply)

    def mimeFk = foreignKey("mimeFK", mime, Mimes)(_.mime)
    def index1 = index("docId_index", docId)
    def index2 = index("createdOn_index", createdOn)
  }

  /**
   * Create a UUID for the docId
   */
  def uuid() = {
    val value = UUID.randomUUID()
    val sb = new StringBuilder()
    sb.append(java.lang.Long.toHexString(value.getMostSignificantBits())).append(java.lang.Long.toHexString(value.getLeastSignificantBits()))
    sb.toString()
  }

  lazy val Mimes = TableQuery[Mimes]
  lazy val Notes = TableQuery[Notes]

}

/**
 * Slick backend supporting revisions. The trick party in this service is that
 * the NoteId is really the docId in the physical database table. Deletes
 * are logical deletes only. Some other administrative layer will need to clean out
 * logically deleted notes.
 */
trait SlickNotesService2 extends RdbmsNotesService with NotesServiceWithRevisions {
  self: DatabaseAware with NotesTableComponents2 =>

  case class NoteId(val id: String)

  override val UnassignedNoteId: NoteId = NoteId("")

  import profile.simple._
  object Helpers extends Helpers
  import Helpers._

  private implicit def blob2Bytes(blob: java.sql.Blob): Array[Byte] = {
    if (blob.length == 0)
      return Array[Byte]()
    val r = blob.getBytes(1, blob.length.toInt)
    blob.free
    r
  }

  private implicit def bytes2Blob(bytes: Array[Byte]) = {
    new SerialBlob(bytes)
  }

  private implicit def blob2NoteSource(blob: java.sql.Blob) = NoteSource(blob)
  private implicit def noteSource2Blob(ns: NoteSource): java.sql.Blob = { ns.source }
  private implicit def timestamp2Instant(ts: java.sql.Timestamp): Instant = ts.toInstant
  private implicit def instant2Timestamp(i: Instant): java.sql.Timestamp = java.sql.Timestamp.from(i)
  private implicit def notesDTO2Note(dto: NotesDTO): Note =
    Note(NoteId(dto.docId), dto.mime, dto.content, dto.createdOn)

  case class Note(id: NoteId, mimeType: String,
    content: NoteSource, createdOn: Instant = Instant.MIN) extends super[NotesServiceWithRevisions].NoteLike
    with super[RdbmsNotesService].NoteLike

  /**
   * Get a note selecting that with the maximum vesion.
   */
  private def getNoteByDocIdAndTimestamp(docId: String, timestamp: java.sql.Timestamp) =
    for {
      n <- Notes
      if n.docId === docId;
      if n.createdOn === timestamp
    } yield n

  /**
   * Find the maximium most recent timestamp based on the createdOn field.
   */
  private def getMaxRevision(docId: String) = {
    val maxQuery = Notes
      .groupBy { _.docId }
      .map {
        case (docId, note) => note.map(_.createdOn).max
      }
    maxQuery
  }

  /**
   * Find the latest note using the latest flag.
   */
  private def latestNote(docId: String) =
    for {
      n <- Notes
      if n.docId === docId;
      if n.latest === true;
      if n.deleted === false
    } yield n

  /**
   * Find a NotesDTO by the database id, not the doc id.
   */
  private def getByDbId(id: Long) =
    for {
      n <- Notes
      if n.id === id
    } yield n

  def getRevisionCount(id: NoteId): Int = {
    val countQuery = Notes.groupBy(_.docId).map { case (docId, note) => note.length }
    database.withSession { implicit session =>
      countQuery.first
    }
  }

  def getPrevious(note: Note): Option[Note] = {
    database.withSession { implicit session =>
      val x = for {
        dto <- getNoteByDocIdAndTimestamp(note.id.id, note.createdOn).firstOption
        previousId <- dto.previousId
        previous <- getByDbId(previousId).firstOption
      } yield previous
      x match {
        case Some(dto) => Some(dto)
        case _ => None
      }
    }
  }

  /**
   * @todo Implement me!
   */
  def getNext(note: Note): Option[Note] = None

  def find(info: FindNoteInfo): Option[Note] = {
    database.withSession {
      implicit session =>
        None
    }
  }

  def delete(id: NoteId): Unit = {
    database.withSession { implicit session =>
      Notes.filter(_.docId === id.id).map(_.deleted).update(Some(true))
    }
  }

  /**
   * Returns the latest note for the given id.
   */
  def get(id: NoteId): Option[Note] = {
    database.withSession { implicit session =>
      latestNote(id.id).firstOption match {
        case Some(n) => Some(n)
        case _ => None
      }
    }
  }

  private def now = java.sql.Timestamp.from(Instant.now)

  def saveOrUpdate(note: Note): Either[String, Note] = {
    database.withSession { implicit session =>
      note.id match {
        case UnassignedNoteId =>
          // Its an insert
          val docId = uuid
          val dbId = (Notes returning Notes.map(_.id)) insert
            NotesDTO(-1, docId, note.mimeType, None, now,
              true, note.content, Some(false))
          Right(note.copy(id = NoteId(docId)))
        case _ =>
          latestNote(note.id.id).firstOption match {
            case Some(cnote) =>
              // Its an update, first mark all previous notes as not current
              Notes.filter(_.id === cnote.id).map(_.latest).update(false)
              // Then insert the new one as current
              val newNote = NotesDTO(-1, cnote.docId, cnote.mime, Some(cnote.id),
                now, true, note.content, Some(false))
              // we don't need this for the return value, but this shows how to get it :-)
              val dbId = (Notes returning Notes.map(_.id)) insert newNote
              Right(newNote)
            case _ =>
              Left("Could not update note " + note.id)
          }
      }
    }
  }

  def addMime(mime: String, description: Option[String]) = {
    database.withSession { implicit session =>
      Mimes += Mime(mime, description)
    }
  }

  def findMime(mime: String): Option[Mime] = {
    database.withSession { implicit session =>
      (for { m <- Mimes if m.mime === mime } yield m).firstOption
    }
  }

  def findAllMime(): Seq[Mime] = {
    database.withSession { implicit session =>
      Mimes.list
    }
  }

  def deleteMime(mime: String): Either[String, Boolean] = {
    database.withSession { implicit session =>
      // If the mime is in use in the Notes table, cannot delete
      val countQuery = (Notes filter (_.mime === mime) map (_.mime)).countDistinct
      val count = Query(countQuery).first
      if (count > 0)
        Left("Mime " + mime + " is in use")
      else {
        (Mimes filter (_.mime === mime)).delete
        Right(true)
      }
    }
  }

}

That's about it!