## Advent of Code - Solving the Third Problem

It's that time of the year again, and the 2022 Advent of Code is up and running! If you're unfamiliar with it, the Advent of Code (AoC) is a series of programming puzzles that runs from December 1 - 25; one question a day, with two parts each. You can claim points on a leader board by being the fastest to solve a problem, but I generally just do them for fun.

I don't think I've ever finished all 25 days before, due to inevitably falling too far behind over the course of the month, but I have a fondness for it, because it pushed me to use a very valuable skill, that I try to use every day: solve the next question before it's asked. Or, you know, try to.

When I first started participating, as a casual player, I would usually try to solve the problem with one-liners, and add operating logic while mapping through steps that was as close to a series of one-liners as I could make it. Then, for the second part, I'd find myself doing mostly the same, from scratch, to solve the twist on the original question that the second part asked for. It was a real slog. Eventually, I stopped trying to solve the first question for what was asked, and started to solve it for what I thought the second question was going to be.

That's what I consider the third, and most challenging problem; solving the first one in such a way that the code can be (mostly) re-used in anticipation of the second part with minimal re-write, but also not over-engineering a bloated solution that would be a pain to maintain. Also, it's the most fun problem to solve! It's always exciting to get it right, and worse case you've invested yourself deeper in the problem-space you're trying to solve.

Here's an example of from Day 4. The premise of the question is that some elves are assigned rooms to clean, but it's noticed that, given a pair of elves, some of their assigned rooms overlap. The (literal) question to part one:

In how many assignment pairs does one range fully contain the other?

The straight-forward thing to do, as we iterate over pairs of elves, is to check does one completely overlap, or not?

``def redundant: Boolean =    (workers._1.assignedTo.toSet, workers._2.assignedTo.toSet) match {        case (a, b) if a.subsetOf(b) | b.subsetOf(a) => true        case _                                       => false    }``

And that's the right solution! It will get you the answer. Ship it! But, perhaps ask yourself, "What's a next logical question I might be asked once I know if there has been overlapping rooms assigned?". My solution for this part of the question answers if there is a redundancy, and if so, which elf is redundant.

``def redundant: Option[Int] =    (workers._1.assignedTo.toSet, workers._2.assignedTo.toSet) match {        case (a, b) if a.subsetOf(b) => Some(1)        case (a, b) if b.subsetOf(a) => Some(2)        case _                       => None    }``

That solution serves the same logic as the first - I can check if the `Option` is defined or not for a true/false solution, but now I have the added power of peaking into the result if I need to know which one to re-assign. Spoiler alert, the follow up question didn't ask about it this time.

I've found these little pieces of logic can start to add up in a larger code base encompassing some domain, and when you're asked how long it would take to roll out some new feature, instead of saying "2 weeks", you can say, "We'll, it's funny you should ask that..."

If you're interested in my full solution for day 4, you can find it here.

## Advent of Code - Day 3

``import zio.*import zio.stream.*object Day3 extends ZIOAppDefault {  val source: String => ZStream[Any, Throwable, String] =    fileName =>      ZStream        .fromFileName(fileName)        .via(ZPipeline.utfDecode >>> ZPipeline.splitLines)        .filter(_.nonEmpty)  // Indexes start at 0. Avoid an extra map by prepending an element, and taking the tail.  val priorities: Map[Char, Int] =    (('a' +: ('a' to 'z')) ++ ('A' to 'Z')).zipWithIndex.tail.toMap  val data = "day-3-1.data"  override def run: ZIO[Any, Any, Any] = for {    _ <- source(data)      .map(str => str.splitAt(str.length / 2)) // Split in two      .map { case (a, b) => a.intersect(b).head } // Elfs promised there would be only one      .map(k => priorities.getOrElse(k, throw new RuntimeException("hau ruck")))      .run(ZSink.sum)      .debug("Answer pt.1")    _ <- source(data)      .grouped(3) // Get it together, Elves!!!      .map(_.reduceLeft((a, b) => a.intersect(b)).head) // Elfs promised there would be only one      .map(k => priorities.getOrElse(k, throw new RuntimeException("hau ruck")))      .run(ZSink.sum)      .debug("Answer pt.2")  } yield Exit.Success}``

## Advent of Code - Day 2

``import zio.*import zio.stream.*object Day2 extends ZIOAppDefault {  sealed trait RPS  object RPS {    case object Rock extends RPS    case object Paper extends RPS    case object Scissors extends RPS    def apply(s: String): RPS = s match {      case "A" | "X" => Rock      case "B" | "Y" => Paper      case "C" | "Z" => Scissors      case _         => throw new RuntimeException("That's cheating!")    }    extension(rps: RPS) {      def losesTo: RPS = rps match {        case Rock     => Paper        case Paper    => Scissors        case Scissors => Rock      }      def winsAgainst = rps match {        case Rock     => Scissors        case Paper    => Rock        case Scissors => Paper      }    }    def score(a: RPS, b: RPS): Int = {      val choice = b match {        case Rock     => 1        case Paper    => 2        case Scissors => 3      }      val outcome = (a, b) match {        case (x, y) if x == y            => 3 // Draw        case (x, Rock) if x == Scissors  => 6 // Winner        case (x, Paper) if x == Rock     => 6 // Winner        case (x, Scissors) if x == Paper => 6 // Chicken substitute dinner        case _                           => 0 // If ya ain't first, yer last      }      // A refactor of outcome, after extension methods were added for part 2      val betterOutcome = (a, b) match {        case (x, y) if x == y         => 3 // Draw        case (x, y) if x.losesTo == y => 6 // Winner        case _                        => 0 // If ya ain't first, yer last      }      choice + betterOutcome    }    def throwTheGame(a: RPS, b: RPS): Int = (a, b) match {      case (x, Rock)     => score(x, x.winsAgainst) // Need to lose      case (x, Paper)    => score(x, x) // Need to tie      case (x, Scissors) => score(x, x.losesTo) // Need to win    }  }  val source: String => ZStream[Any, Throwable, String] =    fileName => ZStream.fromFileName(fileName).via(ZPipeline.utfDecode)  val pt1Pipeline: ZPipeline[Any, Nothing, String, Int] =    ZPipeline.splitLines >>>      ZPipeline.map[String, Int] { str =>        val arr = str.split(" ").map(RPS.apply)        RPS.score(arr.head, arr.last)      }  val pt2Pipeline: ZPipeline[Any, Nothing, String, Int] =    ZPipeline.splitLines >>>      ZPipeline.map[String, Int] { str =>        val arr = str.split(" ").map(RPS.apply)        RPS.throwTheGame(arr.head, arr.last)      }  val scoreSink: ZSink[Any, Nothing, Int, Nothing, Int] = ZSink.sum[Int]  val data = "day-2-1.data"  override def run: ZIO[Any, Any, Any] = for {    _ <- source(data)      .via(pt1Pipeline)      .run(scoreSink)      .debug("Answer pt.1")    _ <- source(data)      .via(pt2Pipeline)      .run(scoreSink)      .debug("Answer pt.2")  } yield ExitCode.success}``

## Advent of Code - Day 1

``import zio.*import zio.stream.*object Day1 extends ZIOAppDefault {  // Add a couple new lines so we don't have to collectLeftover  val source: String => ZStream[Any, Throwable, String] =    fileName =>      ZStream.fromFileName(fileName).via(ZPipeline.utfDecode) ++ ZStream("\n\n")  val pt1Pipeline: ZPipeline[Any, Nothing, String, Int] =    // Split by lines    ZPipeline.splitLines >>>      // group into Chunk[Chunk[_]] based on empty lines      ZPipeline        .mapAccum[String, Chunk[String], Chunk[String]] {          Chunk.empty        } { (state, elem) =>          if (elem.nonEmpty) (state :+ elem, Chunk.empty)          else (Chunk.empty, state)        }        //Get rid of empty Chunks        .filter(_.nonEmpty) >>>      // Add up the inner Chunk      ZPipeline.map[Chunk[String], Int](_.map(_.toInt).sum)  // Find the max  val pt1Sink: ZSink[Any, Nothing, Int, Nothing, Int] =    ZSink.collectAll[Int].map(_.max)  // Sum of biggest 3  val pt2Sink: ZSink[Any, Nothing, Int, Nothing, Int] =    ZSink.collectAll[Int].map(_.sortBy(-_).take(3).sum)    override def run: ZIO[Any, Any, Any] = for {    _ <- source("day-1-1.data")      .via(pt1Pipeline)      .run(pt1Sink)      .debug("Answer pt.1")    _ <- source("day-1-1.data")      .via(pt1Pipeline)      .run(pt2Sink)      .debug("Answer pt.2")  } yield ExitCode.success}``

## ZIO HTTP Tutorial: The REST of the Owl

I contributed an article about ZIO-HTTP a bit ago - You can check it out over at RockTheJVM:

https://blog.rockthejvm.com/zio-http/

## ZIO Streams: An Introduction

I've been working on contributing a piece to Rock the JVM, and it just went up today! 🎉

https://blog.rockthejvm.com/zio-streams/

## Publishing Artifacts to GitHub

Hello, class. Today we're going to use sbt to publish artifacts to GitHub packages via GitHub Actions when we tag/release our codebase, and we're not going to use any sbt plugins to do it!

## It's not that scary​

If you check the official SBT Documentation, you can see that the main things you need to do are specify where you are going to publish

``publishTo := Some("Sonatype Snapshots Nexus" at "https://oss.sonatype.org/content/repositories/snapshots")``

and how to authenticate with that repository

``credentials += Credentials("Sonatype Nexus Repository Manager", "my.artifact.repo.net", "admin", "admin123")``

that's it!™️

I'm working on a (very new) project that's a slim framework to build ZIO based CLI apps called ursula, so I will use this as an example, and talk through the `build.sbt` file, and what the important "gotchas" are. The general plan is:

1. Show the full build.sbt file
2. Discuss parsing tags to artifact versions using default environment variables
3. Configure SBT to publish to our repositories package endpoint
4. Cover some SBT gotchas

### 1 The full build.dbt​

The general structure of this project is that the main library lives in a project/folder named `ursual`, and there is an `example` project that depends on it. We'll cover this in the "gotchas", but there is not a `root` project.

``val tagWithQualifier: String => String => String =  qualifier =>    tagVersion => s"%s.%s.%s-\${qualifier}%s".format(tagVersion.split("\\."): _*)val tagAlpha: String => String = tagWithQualifier("a")val tagBeta: String => String = tagWithQualifier("b")val tagMilestone: String => String = tagWithQualifier("m")val tagRC: String => String = tagWithQualifier("rc")val defaultVersion: String = "0.0.0-a0"val versionFromTag: String = sys.env  .get("GITHUB_REF_TYPE")  .filter(_ == "tag")  .flatMap(_ => sys.env.get("GITHUB_REF_NAME"))  .flatMap { t =>    t.headOption.map {      case 'a' => tagAlpha(t.tail) // Alpha build, a1.2.3.4      case 'b' => tagBeta(t.tail) // Beta build, b1.2.3.4      case 'm' => tagMilestone(t.tail) // Milestone build, m1.2.3.4      case 'r' => tagRC(t.tail) // RC build, r1.2.3.4      case 'v' => t.tail // Production build, should be v1.2.3      case _ => defaultVersion    }  }  .getOrElse(defaultVersion)ThisBuild / organization := "com.alterationx10"ThisBuild / version := versionFromTagThisBuild / scalaVersion := "2.13.8"ThisBuild / publish / skip := trueThisBuild / publishMavenStyle := trueThisBuild / versionScheme := Some("early-semver")ThisBuild / publishTo := Some(  "GitHub Package Registry " at "https://maven.pkg.github.com/alterationx10/ursula")ThisBuild / credentials += Credentials(  "GitHub Package Registry", // realm  "maven.pkg.github.com", // host  "alterationx10", // user  sys.env.getOrElse("GITHUB_TOKEN", "abc123") // password)lazy val ursula = project  .in(file("ursula"))  .settings(    name := "ursula",    libraryDependencies ++= Seq(      "dev.zio" %% "zio" % "2.0.0-RC6"    ),    fork := true,    publish / skip := false  )lazy val example = project  .in(file("example"))  .settings(    publishArtifact := false,    fork := true  )  .dependsOn(ursula)``

### 2 Setting the package version​

Note that this section is more about how I am deploying versions for packages. You likely already have a versioning scheme, and are handling that mapping, but here you go anyway 😆

Maven as a version ordering specification that we'll use for non-numeric qualifiers, which has this ordering:

"alpha" < "beta" < "milestone" < "rc" = "cr" < "snapshot" < "" = "final" = "ga" < "sp"

In all honesty, for simple projects this many qualifiers is probably overkill! I've mapped out `alpha`, `beta` , `milestone`, `rc` and `""` (which is no qualifier, or "final"/"ga").

A note about GitHub packages that was true the last time I tried publishing `SNAPSHOTS` (not sure if this is still the case), but they do not allow you to overwrite a package - so to publish over top of an existing SNAPSHOT - you'd need to delete it first, and upload the new one. That's more work than it's worth, so I've designated `alpha`s as my "snapshots"

With that in mind, I want to use git tags to map to these, so, for example, I've designated that tags `a.1.2.3.4` should build with version `1.2.3-a4`. So by providing a different initial character (`a/b/m/r`), I can control what qualifier it's release as.

With that outlined, I can achieve this with the `tagWithQualifier` function below (and it's helpers).

``val tagWithQualifier: String => String => String =  qualifier =>    tagVersion => s"%s.%s.%s-\${qualifier}%s".format(tagVersion.split("\\."): _*)val tagAlpha: String => String = tagWithQualifier("a")val tagBeta: String => String = tagWithQualifier("b")val tagMilestone: String => String = tagWithQualifier("m")val tagRC: String => String = tagWithQualifier("rc")``

And when I want to do a "production release", I just use the common `v1.2.3` tag.

We will use default environment variables to read the git tags, so we can parse them.

We will check + filter for `GITHUB_REF_TYPE`; this can be `branch` or `tag` (we want `tag`). If we made it this far, we will then check `GITHUB_REF_NAME` - which at this point, should be the value of out git tag.

``val defaultVersion: String = "0.0.0-a0"val versionFromTag: String = sys.env  .get("GITHUB_REF_TYPE")  .filter(_ == "tag")  .flatMap(_ => sys.env.get("GITHUB_REF_NAME"))  .flatMap { t =>    t.headOption.map {      case 'a' => tagAlpha(t.tail) // Alpha build, a1.2.3.4      case 'b' => tagBeta(t.tail) // Beta build, b1.2.3.4      case 'm' => tagMilestone(t.tail) // Milestone build, m1.2.3.4      case 'r' => tagRC(t.tail) // RC build, r1.2.3.4      case 'v' => t.tail // Production build, should be v1.2.3      case _ => defaultVersion    }  }  .getOrElse(defaultVersion)``

Now we have a way to dynamically set the version published based on git tagging!

``ThisBuild / version := versionFromTag``

### 3 Where to publish​

We need to set our `publishTo` and `credentials`. For the publishTo, GitHub has the structure `"https://maven.pkg.github.com/USER/REP"`, so just update with your information. This pattern should hold for orgs as well. An important thing to note is the realm `"GitHub Package Registry"`. This is handled automatically, but when publishing hits the repository, it'll give back a 401 and tell you how you should authenticate and what the realm is. The significant thing to note, is that the value here for the realm is fixed, and determined by the hosting server. sbt will use this realm to find the matching set of `credentials`.

``ThisBuild / publishTo := Some(  "GitHub Package Registry" at "https://maven.pkg.github.com/alterationx10/ursula")ThisBuild / credentials += Credentials(  "GitHub Package Registry", // realm  "maven.pkg.github.com", // host  "alterationx10", // user  sys.env.getOrElse("GITHUB_TOKEN", "abc123") // password)``

We will use an environment variable `GITHUB_TOKEN` to provide our password. Note, that you could do the same thing for the user value.

### 4 SBT gotchas​

This isn't an all-inclusive list, but just a couple of things to keep in mind.

GitHub packages only supports Maven structure, so we need to set `publishMavenStyle` to true. We will set out version schema to "early-semver", which keeps binary compatibility across patch updates within `0.Y.z` until you hit `1.0.0`.

The most important "gotcha" here is `ThisBuild / publish / skip := true`. Since I do not have a `root` project here, sbt will make a `default` one, and aggregate the projects into. This means that it will also try to publish a package named `default`! We can either define a `root` project as a placeholder, and configure it accordingly - or globally set the default to skip publishing, but then re-enabling it in the project we're looking to deploy. The latter is shown here.

``ThisBuild / publish / skip := trueThisBuild / publishMavenStyle := trueThisBuild / versionScheme := Some("early-semver")lazy val ursula = project  .in(file("ursula"))  .settings(    name := "ursula",    libraryDependencies ++= Seq(      "dev.zio" %% "zio" % "2.0.0-RC6"    ),    fork := true,    publish / skip := false  )``

## Lights! Camera! GitHub Action!​

Now that sbt has been included in the environment loaded into the setup-java action, this is easier than it's ever been. For any action, you can use that and just `sbt <your task>`.

For out case, we only want this to run when we create a release (which is a git tag action), so note the `on:` block.

We've set up our `build.sbt` file to use ENV variables that are automatically provided, but we also use the auto-generated ci token: `GITHUB_TOKEN` which is available automatically - that should be set in the `env:` block. If you wanted to use a personal access token, you could store and access the secret in the same way!

``name: Publish Artifact on Releaseon:  release:    types: [ created ]env:  GITHUB_TOKEN: \${{ secrets.GITHUB_TOKEN }}jobs:  publish:    runs-on: ubuntu-latest    steps:      - name: Checkout        uses: actions/checkout@v3      - name: Set up JDK        uses: actions/setup-java@v3        with:          distribution: 'temurin'          java-version: '17'          cache: 'sbt'      - name: Publish        run: sbt publish``

To kick it off, you just need to create a release with a structured git tag. the hardest part is not mistyping for tag 🤣 The packages wills tart to show up on you repositories page, right below the "release" section.

## Wrapping up​

Now, you too can publish your scala artifacts to GitHub packages without relying on a pre-made sbt plugin! How exciting.

## Contextual Abstractions: Extensions

Scala 2 => 3 Series

This is a part in an ongoing series dealing with migrating old ways of doing things from Scala 2 to Scala 3. It will cover the What's New in Scala 3 from the official site.

Check the `Scala 2 => 3` tag for others in the series! For the repo containing all the code, visit GitHub. There are code samples for both Scala 2 and Scala 3 together, that are easy to run via `scala-cli`.

This post is centered around retroactively extending classes.

In Scala 2, extension methods had to be encoded using implicit conversions or implicit classes. In contrast, in Scala 3 extension methods are now directly built into the language, leading to better error messages and improved type inference.

Extensions are one of my favorite things to use in Scala. Personally, I like the ability to add functionality to "upstream" resources implicitly, but call that functionality explicitly. To me, it makes it less likely to break things during a refactor when you don't have to un-ravel a mysterious series of implicit def methods / conversions that you might not realize are being called.

## The preface​

For this example, let's say that we have some upstream domain model from a service we use but don't control.

``case class UpstreamUser(id: Long, created: Instant, lastSeen: Instant)``

In our service, we have a concept of when a user goes "stale" based on usage - but other services also have this notion, and differing beliefs about what conditions make a user stale - so we can't ask the upstream service to implement this for us on our model. Perhaps our model of what a stale user is changes over time as well.

Our conditions for a user going stale are:

• A user was created over a year ago
• A user hasn't been seen in the last week.

With that in mind, we could write some logic such as

``import java.time.Instantimport java.time.temporal.ChronoUnit._def isStale(created: Instant, lastSeen: Instant):Boolean = {     lastSeen.plus(7, DAYS).isBefore(Instant.now) &&      created.plus(365, DAYS).isBefore(Instant.now)  }``

but calling that everywhere becomes a bit cumbersome, and it would be great if we could attach that functionality directly on `UpstreamUser`.

## Scala 2​

In scala 2, we can use an `implicit class` to achieve our goal. An implicit class should have only one constructor argument, of the Type that is being extended. It also needs to be housed in something, typically an outer object. This can make setting up implicit classes feel a bit "boilerplate-y".

``object UpstreamUserExtensions {  implicit class ExtendedUpstreamUser(u: UpstreamUser) {    def isStale: Boolean = {      u.lastSeen.plus(7, DAYS).isBefore(Instant.now) &&      u.created.plus(365, DAYS).isBefore(Instant.now)    }  }}``

Now, with `ExtendedUpstreamUser` in scope to implicitly add our new functionality, we can (explicitly) call `upstreamUserInstance.isStale` as if it were on the model directly.

## Scala 3​

In Scala 3, it works much the same, but with less boilerplate. Instead of declaring an implicit class, you declare an extension: `extension (u: UpstreamUser)` where the argument matches the `Type` you're adding functionality to. This doesn't need to be housed in an object either!

The corresponding Scala 3 code would look like:

``extension (u: UpstreamUser) {  def isStale: Boolean = {    u.lastSeen.plus(7, DAYS).isBefore(Instant.now) &&    u.created.plus(365, DAYS).isBefore(Instant.now)  }}``

and then we'll get the same `upstreamUserInstance.isStale` functionality as before.

## Final Thoughts​

Although the looks of the code have changed, if you're used to Scala 2 implicit classes, Scala 3 extensions will probably be a welcomed ergonomics change, with a familiar feel for usage.

## Contextual Abstractions: Using, Given

Scala 2 => 3 Series

This is a part in an ongoing series dealing with migrating old ways of doing things from Scala 2 to Scala 3. It will cover the What's New in Scala 3 from the official site.

Check the `Scala 2 => 3` tag for others in the series! For the repo containing all the code, visit GitHub. There are code samples for both Scala 2 and Scala 3 together, that are easy to run via `scala-cli`.

This post is centered around the new way of passing implicit arguments to methods via using-clauses.

Abstracting over contextual information. Using clauses allow programmers to abstract over information that is available in the calling context and should be passed implicitly. As an improvement over Scala 2 implicits, using clauses can be specified by type, freeing function signatures from term variable names that are never explicitly referred to.

## The preface​

For this example, let's say that we have some interface that we're going to be passing around a lot, and that it could have multiple implementations.

``trait BaseLogger {    def log[T](t: T): Unit}case class PrintLogger() extends BaseLogger {  def log[T](t: T): Unit = println(s"Logger result: \${t.toString}")}case class FancyLogger() extends BaseLogger {  def log[T](t: T): Unit = println(s"Ye Olde Logger result: \${t.toString}")}``

## Scala 2​

In Scala 2, we could write a method, and have our trait's implementation passed in as a separate implicit argument.

``  def loggingOp[A,B](a: A, b: B)(implicit logger: BaseLogger): Int = {      val result = a.toString.map(_.toInt).sum + b.toString.map(_.toInt).sum      logger.log(result)      result  }``

At this point, we could call our method by still passing the argument in explicitly

``object Using_2 extends App {    val printLogger: PrintLogger = PrintLogger()  val fancyLogger: FancyLogger = FancyLogger()    loggingOp(40, 2)(printLogger)  loggingOp(40, 2)(fancyLogger)  }``

However, if we define an instance of type `BaseLogger` in scope implicitly, then we don't need to pass it in as an argument every time! Of course, we still have the option to pass something in explicitly, if we don't want to use the instance that is in scope implicitly.

``object Using_2 extends App {    val printLogger: PrintLogger = PrintLogger()  val fancyLogger: FancyLogger = FancyLogger()    loggingOp(40, 2)(printLogger)  loggingOp(40, 2)(fancyLogger)    // With an implicit of type BaseLogger in scope...   implicit val defaultLogger = printLogger    // ... I no longer need to pass it as an argument  loggingOp(true, false)  loggingOp(17, "purple")  // ... but I can still call implicit arguments explicitly!  loggingOp("car", printLogger)(fancyLogger)}``

## Scala 3​

In Scala 3, we don't use the implicit key word when defining a method - we now use `using`. A faithful port of the Scala 2 code above would look something like:

``  // You can specify the name logger, but don't have to  def loggingOp_withParamName[A, B](a: A, b: B)(using logger: BaseLogger): Int = {    val result = a.toString.map(_.toInt).sum + b.toString.map(_.toInt).sum    logger.log(result)    result  }``

The awesomeness of Scala 3 doesn't stop there, though, because you can define your methods by just declaring the type! In this case, we just `summon` an instance internally, and use reference to that.

There are only two hard things in Computer Science: cache invalidation and naming things.

Guess it's just invalidating caches now!

``  def loggingOp[A, B](a: A, b: B)(using BaseLogger): Int = {    val logger = summon[BaseLogger]    val result = a.toString.map(_.toInt).sum + b.toString.map(_.toInt).sum    logger.log(result)    result  }``

From here, our code works mostly the same - one caveat being that when explicitly passing arguments, you need to use the `using` keyword - where previously you didn't need to declare the values you were passing in were `implicit`. We're also declaring our `BaseLogger` in scope using alias givens

``object Using_3 {    val printLogger: PrintLogger = PrintLogger()  val fancyLogger: FancyLogger = FancyLogger()  @main  def main = {    // We can still call things explicitly...    loggingOp(40, 2)(using printLogger)    loggingOp(40, 2)(using fancyLogger)    // .. but we have a new way of defining what type is in scope implicitly    // implicit val defaultLogger = printLogger // <- this would still work    given defaultLogger: BaseLogger = printLogger // <- but probably use this    loggingOp(true, false)    loggingOp(true, false)    loggingOp(17, "purple")    loggingOp("car", printLogger)(using fancyLogger)  }}``

## Final Thoughts​

Using clauses can be a bit more complex, but with the simple example outlined above - we have one less scary new thing, that we can mentally map back to our years of Scala 2 use!

## Premise​

The example in this post is about using a kubernetes `CustomResourceDefinition` and `Operator` implemented with `ZIO` to simplify our lives as someone who made need to run a lot of infrastructure set up (dare I even say Dev/Ops).

The example is complete/functioning, but isn't the most robust solution for what it does. It is meant to be enough to work, and illustrate the concept with a solution to a made-up problem - but not exactly a model code base 👼

Let's dig in!

### Hey, can you set me up a database?​

Perhaps you're the one with the password/access to the database, or the only person nearby on the team that "knows SQL", but it's part of your daily life to set up databases for people. In between your coding work, you run a lot of the following type of code for people who need to access their own database from a kubernetes cluster:

``CREATE DATABASE stuff;CREATE USER stuff PASSWORD 'abc123';GRANT ALL ON DATABASE stuff TO stuff;``

Your hard work is then rewarded by remembering to set up a `Secret` for each database as well, so the user can easily mount it to their pods for access.

But, wait a minute - you've just picked up a nifty framework called `ZIO`, and have decided to automate a bit of you daily todos.

## Enter ZIO​

Let's create a `SQLService` that will set up a matching database and user:

``trait SQLService {  def createDatabaseWithRole(db: String): Task[String]}// We're going to be lazy, and not use a Loggercase class SQLServiceLive(cnsl: Console.Service) extends SQLService { override def createDatabaseWithRole(db: String): Task[String] = ???}``

We aren't running this so often that we need a dedicated connection pool, so let's just grab a connection from the driver, and use this neat new thing we've learned about called `Zmanaged`.

``private val acquireConnection =  ZIO.effect {    val url = {      sys.env.getOrElse(        "PG_CONN_URL", // If this environment variable isn't set...        "jdbc:postgresql://localhost:5432/?user=postgres&password=password" // ... use this default one.      )    }    DriverManager.getConnection(url)  }private val managedConnection: ZManaged[Any, Throwable, Connection] =  ZManaged.fromAutoCloseable(acquireConnection)// We'll use a ZManaged for Statements too!private def acquireStatement(conn: Connection): Task[Statement] =  Task.effect {    conn.createStatement  }def managedStatement(conn: Connection): ZManaged[Any, Throwable, Statement] =  ZManaged.fromAutoCloseable(acquireStatement(conn))``

What's a `ZManaged`?

ZManaged is a data structure that encapsulates the acquisition and the release of a resource, which may be used by invoking the use method of the resource. The resource will be automatically acquired before the resource is used and automatically released after the resource is used.

So a `ZManged` is like a `try/catch/finally` that handles your resources - but you don't have to set up a lot of boilerplate. A common pattern I've used in the past would be to use a `thunk` to do something similar. The (very unsafe, with no error handling) example below handles the acquisition and release of the connection + statement, and you just need to pass in a function that takes a statement, and produces a result.

``def sqlAction[T](thunk: Statement => T): T = {  val url: String = "jdbc:postgresql://localhost:5432/?user=postgres&password=password"  val connection = DriverManager.getConnection(url)  val statement: Statement = connection.createStatement()  val result: T = thunk(statement)  statement.close()  connection.close()  result}def someSql = sqlAction { statement =>  // do something with statement  ???}``

In the spirit of our thunk, we'll write a ZIO function that takes a `Statement`, a `String` (some SQL), and will execute it. We'll print the SQL we run, or log the error that falls out.

``val executeSql: Statement => String => ZIO[Any, Throwable, Unit] =  st =>    sql =>      ZIO        .effect(st.execute(sql))        .unit        .tapBoth(          err => cnsl.putStrLnErr(err.getMessage),          _ => cnsl.putStrLn(sql)        )``

Now with all of our pieces in place, we can implement our `createDatabaseWithRole` that will safely grab a `Connection` + `Statement`, run our SQL, and then automatically close those resources when done. It'll even hand back the random password generated:

``override def createDatabaseWithRole(db: String): Task[String] = {    managedConnection.use { conn =>      managedStatement(conn).use { st =>        for {          pw <- ZIO.effect(scala.util.Random.alphanumeric.take(6).mkString)          _  <- executeSql(st)(s"CREATE DATABASE \$db")          _  <- executeSql(st)(s"CREATE USER \$db PASSWORD '\$pw'")          _  <- executeSql(st)(s"GRANT ALL ON DATABASE \$db TO \$db")        } yield pw      }    }  }``

😍 A thing ouf beauty! Now we can just make a simple ZIO program to call our new service, and call it a day!

``val simpleProgram: ZIO[Has[SQLService], Nothing, Unit] =    SQLService(_.createDatabaseWithRole("someUser"))      .unit      .catchAll(_ => ZIO.unit)``

## Automate the Automation​

j/k you still have to stop what you're doing to run this for people, and you still need to make the `Secret`! Wouldn't it be neat if we could have some sort of Kubernetes resource that allowed anyone to just update a straightforward file? What if we had something like:

``apiVersion: alterationx10.com/v1kind: Databasemetadata:  name: databasesspec:  databases:    - mark    - joanie    - oliver``

Well, it turns out we can have nice things! We can create a `CustomResourceDefinition` that will use that exact file as shown above! The following yaml sets up our own `Kind` called `Database` that has a spec of databases, which is just an array of String.

``apiVersion: apiextensions.k8s.io/v1kind: CustomResourceDefinitionmetadata:  # name must match the spec fields below, and be in the form: <plural>.<group>  name: databases.alterationx10.comspec:  # group name to use for REST API: /apis/<group>/<version>  group: alterationx10.com  # list of versions supported by this CustomResourceDefinition  versions:    - name: v1      # Each version can be enabled/disabled by Served flag.      served: true      # One and only one version must be marked as the storage version.      storage: true      schema:        openAPIV3Schema:          type: object          properties:            spec:              type: object              properties:                databases:                  type: array                  items:                    type: string  # either Namespaced or Cluster  scope: Namespaced  names:    # plural name to be used in the URL: /apis/<group>/<version>/<plural>    plural: databases    # singular name to be used as an alias on the CLI and for display    singular: database    # kind is normally the CamelCased singular type. Your resource manifests use this.    kind: Database    # shortNames allow shorter string to match your resource on the CLI    shortNames:      - db``

Since we don't want to run jobs manually, we can create an `Operator` that will watch for our `CustomResourceDefinition` , and take action automatically! With the zio-k8s library, these can be fairly straightforward to implement.

``  val eventProcessor: EventProcessor[Clock, Throwable, Database] =    (ctx, event) =>      event match {        case Reseted() =>          cnsl.putStrLn(s"Reseted - will (re) add any existing").ignore        case Added(item) =>          processItem(item)        case Modified(item) =>          processItem(item)        case Deleted(item) =>          cnsl.putStrLn(s"Deleted - but not performing action").ignore      }``

For our example program, we will always try and create the databases listed in the resources, and log/ignore the error if a database already exists on `Added` and `Modified`. We will also take the auto-generated password, and create a secret for it as well! We won't tear anything down on `Deleted`.

``def processItem(item: Database): URIO[Clock, Unit] =    (for {      // Get all of our databases      dbs <- ZIO.fromOption(item.spec.flatMap(_.databases).toOption)      // For each database      _ <- ZIO.foreach(dbs) { db =>        (for {          _ <- cnsl.putStrLn(s"Processing \$db...")          // Create things          pw <- sqlService.createDatabaseWithRole(db)          _  <- cnsl.putStrLn(s"... \$db created ...")          // Put the generated PW in a k8s secret          _ <- upsertSecret(            Secret(              metadata = Some(                ObjectMeta(                  name = Option(db),                  namespace = item.metadata                    .map(_.namespace)                    .getOrElse(Option("default"))                )              ),              data = Map(                "POSTGRES_PASSWORD" -> Chunk.fromArray(                  pw.getBytes()                )              )            )          ).tapError(e => cnsl.putStrLnErr(s"Couldn't make secret:\n \$e"))          _ <- cnsl.putStrLn(s"... Secret created for \$db")        } yield ()).ignore      }    } yield ()).ignoredef upsertSecret(      secret: Secret  ): ZIO[Clock, K8sFailure, Secret] = {    for {      nm       <- secret.getName      ns       <- secret.getMetadata.flatMap(_.getNamespace)      existing <- secrets.get(nm, K8sNamespace(ns)).option      sec <- existing match {        case Some(_) => secrets.replace(nm, secret, K8sNamespace(ns))        case None    => secrets.create(secret, K8sNamespace(ns))      }    } yield sec  }``

That's about it! We now have the code we need to automate our daily drudgery!

## Deploying​

This example is targeted at deploying to the instance of Kubernetes provided by Docker, mainly so we can use our locally published docker image.

### Auto generation of our CRD client​

We will need the `zio-k8s-crd` SBT plugin to auto generate the client needed to work with our CRD. Once added, we can update our `build.sbt` file with the following, which points to the new CRD. With this in place, a compile step will generate the code for us.

``externalCustomResourceDefinitions := Seq(  file("crds/databases.yaml"))enablePlugins(K8sCustomResourceCodegenPlugin)``

### Building a Docker image of our service​

We'll use the `sbt-native-packager` SBT plugin to build the docker image for us. We'll need a more recent version of Java than what is default, so well set `dockerBaseImage := "openjdk:17.0.2-slim-buster"` and set our project to `.enablePlugins(JavaServerAppPackaging)`. Now, when we run `sbt docker:publishLocal`, it will build and tag an image with the version specified in our `build.sbt` file that we can use in our kubernetes deployment yaml.

``REPOSITORY      TAG            IMAGE ID       CREATED         SIZEsmooth-operator 0.1.0-SNAPSHOT a4e2c2025cba   2 days ago      447MB``

### Who doesn't love more YAML?​

This section will go over the kubernetes yaml needed to deploy everything we need for our app.

We will create a standard `Deployment` of postgres, configured to have the super secure password of `password` 🤫. We will also create a `Service` to route traffic to it.

``apiVersion: apps/v1kind: Deploymentmetadata:  name: postgres  labels:    app: postgresspec:  replicas: 1  selector:    matchLabels:      app: postgres  template:    metadata:      labels:        app: postgres    spec:      containers:        - name: postgres          image: postgres          env:            - name: POSTGRES_PASSWORD              value: "password"---apiVersion: v1kind: Servicemetadata:  name: postgresspec:  selector:    app: postgres  ports:    - port: 5432      targetPort: 5432      protocol: TCP``

For deploying our `Operator`, we ultimately are going to set up a `Deployment` for it, but we're going to need a few more bells and whistles first. Our app will need the right permissions to be able to watch our `CustomResourceDefinition`s, as well as accessing `Secrets` - these actions are done by the `ServiceAccount` our pod runs under. We will create a `ClusterRole` that has the required permissions, and use a `ClusterRoleBinding` to assign the `ClusterRole` to our `ServiceAccount`.

A very useful `kubectl` command to check and make sure your permissions are correct is `kubectl auth can-i ...` command.

``kubectl auth can-i create secrets --as=system:serviceaccount:default:db-operator-service-account -n defaultkubectl auth can-i watch databases --as=system:serviceaccount:default:db-operator-service-account -n default``

With all that in mind, we can use the following yaml to get our app up and running.

``apiVersion: v1kind: ServiceAccountmetadata:  name: db-operator-service-accountautomountServiceAccountToken: true---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata:  name: db-operator-cluster-rolerules:  - apiGroups: [ "alterationx10.com" ]    resources: [ "databases" ]    verbs: [ "get", "watch", "list" ]  - apiGroups: [ "" ]    resources: [ "secrets" ]    verbs: ["*"]---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata:  name: db-operator-cluster-role-bindingsubjects:  - kind: ServiceAccount    name: db-operator-service-account    namespace: defaultroleRef:  kind: ClusterRole  name: db-operator-cluster-role  apiGroup: rbac.authorization.k8s.io---apiVersion: apps/v1kind: Deploymentmetadata:  name: db-operator  labels:    app: db-operatorspec:  selector:    matchLabels:      app: db-operator  template:    metadata:      labels:        app: db-operator    spec:      serviceAccountName: db-operator-service-account      containers:        - name: db-operator          image: smooth-operator:0.1.0-SNAPSHOT          env:            - name: PG_CONN_URL              value: "jdbc:postgresql://postgres:5432/?user=postgres&password=password"---``

Note: When deploying an operator "for real", you want to take care that only one instance is running/working at a time. This is not covered here, but you should look into Leader Election

## Running the Example​

You can view the source code on GitHub, tagged at `v0.0.3` at the time of this blog post.

Assuming you have Docker/Kubernetes et up, you should be able to run the following commands to get an example up and running:

``# Build/publish our App to the local Docker reposbt docker:publishLocal# Deploy our CustomResourceDefinitionkubectl apply -f crds/databases.yaml# Deploy postgreskubectl apply -f yaml/postgres.yaml# Deploy our appkubectl apply -f yaml/db_operator.yaml# Create Database Resourcekubectl apply -f yaml/databases.yaml``

If you check the logs of the running pod, you should hopefully see the SQL successfully ran, and can also use `kubectl` to check for new `Secrets`!

``➜ smooth-operator (main) ✗ kubectl logs db-operator-74f756c89c-x5f5b SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".SLF4J: Defaulting to no-operation (NOP) logger implementationSLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.Reseted - will (re) add any existingProcessing mark...CREATE DATABASE markCREATE USER mark PASSWORD 'VCaHar'GRANT ALL ON DATABASE mark TO mark... mark created ...... Secret created for markProcessing joanie...CREATE DATABASE joanieCREATE USER joanie PASSWORD 'mdlQKB'GRANT ALL ON DATABASE joanie TO joanie... joanie created ...... Secret created for joanieProcessing oliver...CREATE DATABASE oliverCREATE USER oliver PASSWORD 'vYODSt'GRANT ALL ON DATABASE oliver TO oliver... oliver created ...... Secret created for oliver``

Nice.

There you have it! After a day or two of set up, now you too can save tens of minutes every day!

Tags: