lucidchart xtract library and composing readers

If you use "readers" similar to play's json reader framework, you know that its a nice framework to compose your json converters. There is a similar framework for reading XML from lucidchart called xtract.
xtract only provides XML reading. Its very similar to play json. Many of the examples that demonstrate the play json or xtract library are fairly simple and did not help me understand how to compose readers that need to have alternatives. For example, an XML fragment may have an element called Fault or an element called Data and you want to compose a reader that automatically handles both in one reader instance. How do you do that?
Here's some scalatest examples that show one way to do that. You have some choices about how to navigate and how to compose so pay special attention to the details in the tests. I prefer to compose by using readers at the top element and use and/or on the builders, but you can choose to do it anyway you wish.

The code may be easier to read as a gist: github gist
import org.scalatest._

class readerspecs extends FlatSpec with Matchers {

  import scala.xml._
  import com.lucidchart.open.xtract._
  import com.lucidchart.open.xtract.{ XmlReader, __ }
  import com.lucidchart.open.xtract.XmlReader._
  import play.api.libs.functional.syntax._

  import responseReaders._

  val faultXml =
    <Fault>
      <ErrorCode>1</ErrorCode>
      <Message>Error</Message>
    </Fault>

  val faultInXml =
    <Response>
      <Body>
        <MultipleResponse>
          { faultXml }
        </MultipleResponse>
      </Body>
    </Response>

  val noFaultInXml =
    <Response>
      <Body>
        <MultipleResponse>
          <Data><key>k</key><value>1</value></Data>
        </MultipleResponse>
      </Body>
    </Response>

  val responseFaultXml =
    <Envelope>
      { faultInXml }
    </Envelope>

  val responseDataXml =
    <Envelope>
      { noFaultInXml }
    </Envelope>

  val bodyXml =
    <Envelope>
      <Flag>false</Flag>
      <Response>
        <Body>
          <MultipleResponse>
            <Data><key>k</key><value>1</value></Data>
          </MultipleResponse>
        </Body>
      </Response>
    </Envelope>

  sealed trait ResponseBody
  case class IFault(i: Int, e: String) extends ResponseBody
  case class IData(k: String, v: Int) extends ResponseBody

  case class IBody(flag: Boolean, r: ResponseBody)

  implicit val dataReader = ((__ \ "key").read[String] and (__ \ "value").read[Int])(IData.apply _)
  implicit val faultReader = ((__ \ "ErrorCode").read[Int] and (__ \ "Message").read[String])(IFault.apply _)

  val responsePathReader: XmlReader[NodeSeq] = (__ \ "Response").read
  val bodyPathReader: XmlReader[NodeSeq] = (__ \ "Body").read
  val faultPathReader = (__ \ "Fault").read[NodeSeq]
  val multipleResponsePathReader: XmlReader[NodeSeq] = (__ \ "MultipleResponse").read
  val multipleResponsePathReaderJump: XmlReader[NodeSeq] = (__ \\ "MultipleResponse").read // find anywhere in children

  "responseReaders" should "read a Fault" in {
    val r = faultReader.read(faultXml)
    r.toOption shouldBe Some(IFault(1, "Error"))
  }

  it should "find a fault or None inside a response body using a piece by piece XPath traversal" in {
    val bpath = (__ \\ "Body")
    val rpath = (__ \\ "MultipleResponse")
    val fpath = (__ \\ "Fault")
    val r: XmlReader[NodeSeq] = (bpath ++ rpath ++ fpath).read
    withClue("has fault:") { (r andThen faultReader).read(faultInXml).toOption shouldBe Some(IFault(1, "Error")) }
    withClue("no fault:") { (r andThen faultReader).read(noFaultInXml).toOption shouldBe None }
  }

  it should "read a constant using pure" in {
    val r = (XmlReader.pure(-1) and XmlReader.pure("Not an error"))(IFault.apply _)
    r.read(faultXml).toOption shouldBe Option(IFault(-1, "Not an error"))
  }

  it should "read something with a direct path" in {
    val r = (__ \\ "Data").read[IData]
    r.read(noFaultInXml).toOption shouldBe Option(IData("k", 1))
  }

  it should "find a fault inside a response body using a composition of readers traversal" in {
    val r = (responsePathReader andThen bodyPathReader andThen multipleResponsePathReader andThen faultPathReader andThen faultReader)
    r.read(responseFaultXml).toOption shouldBe Option(IFault(1, "Error"))
  }

  it should "not find a fault inside a response body using a composition of readers traversal that are not correct" in {
    val r = (responsePathReader andThen bodyPathReader andThen /*multipleResponsePathReader andThen*/ faultPathReader andThen faultReader)
    r.read(responseFaultXml).toOption shouldBe None
  }

  it should "follow a path using composition starting with XmlReaders" in {
    // We use map because we are starting with readers that read NodeSeq, see the example below
    // Shows off all combinators or embedding an XPath in the middle as well but as a reader of nodeseq.
    val tmp: XmlReader[ResponseBody] = (responsePathReader andThen bodyPathReader andThen multipleResponsePathReader andThen
      ((faultPathReader andThen faultReader) or ((__ \ "Data").read[NodeSeq] andThen dataReader)))
    val r = tmp.map(IBody(true, _))

    withClue("fault:") { r.read(responseFaultXml).toOption shouldBe Option(IBody(true, IFault(1, "Error"))) }
    withClue("data:") { r.read(responseDataXml).toOption shouldBe Option(IBody(true, IData("k", 1))) }
  }

  it should "follow a path using composition with builders" in {
    // Since the expression starts with a Builder 'and' Builder, we can use apply instead of map like above
    val tmp = ((__ \ "Flag").read[Boolean] and
      (responsePathReader andThen bodyPathReader andThen multipleResponsePathReader andThen
        ((faultPathReader andThen faultReader) or ((__ \ "Data").read[NodeSeq] andThen dataReader))))
    val r = tmp(IBody.apply _)

    withClue("data:") { r.read(bodyXml).toOption shouldBe Option(IBody(false, IData("k", 1))) }
  }

  it should "like the previous test but jump to the location using an XPath then compose" in {
    // Since the expression starts with a Builder 'and' Builder, we can use apply instead of map like above
    val r = ((__ \ "Flag").read[Boolean] and (multipleResponsePathReaderJump andThen
      ((faultPathReader andThen faultReader)
        or
        ((__ \ "Data").read[NodeSeq] andThen dataReader))))(IBody.apply _)

    withClue("data:") { r.read(bodyXml).toOption shouldBe Option(IBody(false, IData("k", 1))) }
  }

    val prefixattr =
    <kvs xmlns:i="mynamespace">
      <el i:type="string">
        theel
      </el>
    </kvs>

  "attribute reader" should "read a prefixed attribute" in {
    val areader = XmlReader.attribute[String]("{mynamespace}type")
    val r = (__ \ "el")(prefixattr)
    val prefixResult = areader.read(r)
    assert(prefixResult.isSuccessful)
    prefixResult.toOption shouldBe Some("string")

    // combine this together for a real read

    val elAndAttrReader = (
      (__ \ "el").read[String].map(_.trim) and
      (__ \ "el").read(areader))((_, _))
    elAndAttrReader.read(prefixattr).toOption shouldBe Some(("theel", "string"))
  }

  val prefixattrcomplex =
    <kvs xmlns:i="mynamespace">
      <el i:type="string">stringvalue</el>
      <el i:type="int">30</el>
      <el i:type="long">30</el>
    </kvs>

  it should "allow failing fast in order to get the right reader" in {
    implicit val elTypeReader = XmlReader.attribute[String]("{mynamespace}type")
    sealed trait ElType
    case class StringEl(v: String) extends ElType
    case class IntEl(v: Int) extends ElType
    case class LongEl(v: Long) extends ElType

    val kvReader = (__ \\ "el").read(seq(elTypeReader))
    val result = kvReader.read(prefixattrcomplex)
    withClue("reading just the attributes:") {
      result.foreach(r => r should contain inOrderOnly ("string", "int", "long"))
    }

    case class WrongTypeError(expected: String) extends ValidationError

    // Fail if the namespaced attribute is not a match and return a nodeseq so we can compose it
    def filteritype(t: String) = nodeReader.filter(WrongTypeError(t))(n => elTypeReader.read(n).getOrElse("") == t)

    val stringElReader: XmlReader[StringEl] = filteritype("string").map(n => StringEl(n.text))
    val intElReader = filteritype("int").map(n => IntEl(n.text.toInt))
    val longElReader = filteritype("long") andThen nodeReader.map(n => LongEl(n.text.toInt)) // slight variation

    val eltypereader =
      (__.read(stringElReader) orElse
        __.read(intElReader) orElse
        __.read(longElReader))
    val readEls = (__ \\ "el").read(seq(eltypereader))
    withClue("reading full objects:") {
      readEls.read(prefixattrcomplex).foreach(r => r should contain inOrderOnly (StringEl("stringvalue"), IntEl(30), LongEl(30)))
    }
  }

  val f =
    <FormattedValues>
      <b:KeyValuePairOfstringstring>
        <c:key>creditonhold</c:key>
        <c:value>No</c:value>
      </b:KeyValuePairOfstringstring>
      <b:KeyValuePairOfstringstring>
        <c:key>donotcontact</c:key>
        <c:value>Allow</c:value>
      </b:KeyValuePairOfstringstring>
      <b:KeyValuePairOfstringstring>
        <c:key>bstate</c:key>
        <c:value>1</c:value>
      </b:KeyValuePairOfstringstring>
    </FormattedValues>

  val kvpreader = (
    (__ \ "key").read[String] and
    (__ \ "value").read[String])((_, _))

  "reading kv pairs" should "read a sequency correctly" in {
    val r = (__ \\ "KeyValuePairOfstringstring").read(seq(kvpreader))
    val tmp = r.read(f)
    r.map(v => v should contain inOrderOnly (("creditonhold", "No"), ("donotcontact", "Allow"), ("bstate", "1")))
  }


}

Comments

Post a Comment

Popular posts from this blog

quick note on scala.js, react hooks, monix, auth

quick note on scala.js, react hooks, monix, auth Reactjs hooks finally came out of preview in a recent version of React. scalajs-reaction has had support for hooks since they were in preview. React hooks seem like the hot new thing but for me the question is, do hooks help improve your code by improving maintainability, time to market, TCO, etc.? Let’s take a look using authentication as an example and employ monix , a reactive observable framework that also builds on cats . We will assume we are building a SPA. Does it improve your code and make your application & development experience better? Auth Auth processing can be complex. Many react auth examples you find in blogs like to show how to use “react-routing” and “auth” as a switch that determines which routes are protected or unprotected. Protected routes show application specific data. Unprotected routes show login content . Based on storing a “User” object in the browser session/local storage, th
Read more

zio environment and modules pattern: zio, scala.js, react, query management

zio environment and modules pattern: zio, scala.js, react, query management This blog covers a zio-based fetch design for scala.js, react web applications. It heavily uses the zio “environment” feature to help motivate a point of view on the modules pattern in effect systems. At the end, I summarize some thoughts on ergonomics and complexity. This blog assumes a basic knowledge of zio and react. scala.js is just like scala so you mostly already know scala.js. This blog is a follow-on to previous blogs: https://appddeevvmeanderings.blogspot.com/2019/10/scalajs-react-fetcher-hook_48.html and https://appddeevvmeanderings.blogspot.com/2019/11/scalajs-and-react-suspense.html . I have been keeping an eye on and culling ideas from typscript/javascript fetching libraries that address query management concerns. The formulation below incorporates some of those concerns but does not try to comprehensively duplicate well established javascript libraries. I only care about
Read more

user experience, scala.js, cats-effect, IO

user experience, scala.js, cats-effect, IO This is not a complicated snippet but I wanted to ensure I remembered it. When creating a UI, say react for browsers, you will run into the “fetch” data problem. When you fetch data, you typically have a component that performs the fetch and acts as a cache for the result. The fetch component renders a child and passes along the data that was fetched (or the fetch state, etc.). Even with the upcoming react suspense mechanism that will make this a smoother user experience, you may have a flash of a “loading” indicator that shows for a moment before the content is fetched from the server. If the flash is too fast, it is visually disruptive. A shimmer mechanism for the UI my also not be the answer. Similarly, if the fetch request takes too long, you will want it to time-out and show an error message that the data could not be fetched or whatever is appropriate for feedback in your application. Generally, I use cats-effect
Read more