Ditching your favorite Cassandra client.

Last year, I wrote a post titled Using Clerk notebooks as documentation for your open-source Clojure libraries – in which I had shown a few examples of using my open source library – SqlForCql (which exposes query / update capabilities that plain CQL does not allow). At that time, I was not aware of how to properly leverage the full capabilities of Clerk (which is touted as – Moldable Live Programming for Clojure). Even today, I’ve barely scratched the surface of this excellent tool, but I’ve figured out the next step in my journey.

So coming back to the sensational claim – why would anyone want to ditch their favorite Cassandra client? For example, I use the intuitive GUI client for interacting with Cassandra called TablePlus. If you haven’t checked it out yet, please try it out once. You can use it to connect not just to Cassandra but a lot of the RDBMSes, MongoDB etc. So what’s wrong with it? Well nothing per se, but it is limited to what CQL can do, and that’s where the combination of SqlForCql and Clerk lets us kill two birds with a single (combination) stone 😉.

Let’s see how to achieve that lofty goal. Below is some source code which Clerk notebook will display –

^{:nextjournal.clerk/visibility :hide-ns}
(ns sqlforcqlex
  (:require [sqlforcql.cql :as cql]
            [sqlforcql.analyze :as analyze]
            [nextjournal.clerk :as clerk]))

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "# Query capabilities")

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "### The following queries are obvious (though nothing which CQL can't do) -")

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "*select * from players;*")

{:nextjournal.clerk/visibility {:code :show :result :show}}
(clerk/table (cql/get-all "players"))

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "_select count(*) from players;_")

{:nextjournal.clerk/visibility {:code :show :result :show}}
(cql/get-count "players")

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "*select * from players where city = 'Ajmer' allow filtering;*")

{:nextjournal.clerk/visibility {:code :show :result :show}}
(clerk/table (cql/get-by-non-pk-col "players" {:city "Ajmer"}))

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "*select * from players where nickname = 'fedex';*")

{:nextjournal.clerk/visibility {:code :show :result :show}}
(clerk/table (cql/get-by-pk-col "players" {:nickname "fedex"}))

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "## Now this is where (the CQL betrays us) and the fun starts -")

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "*select * from players where city like 'Dhabi';*")

{:nextjournal.clerk/visibility {:code :show :result :show}}
(clerk/table (cql/get-by-non-pk-col-like "players" {:city "Dhabi"}))

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "# Other capabilities")

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "### Getting counts of the number of rows of a few (base, and the supporting query) tables -")

{:nextjournal.clerk/visibility {:code :show :result :show}}
(analyze/get-counts ["players" "players_by_city"])

{:nextjournal.clerk/visibility {:code :hide :result :show}}
(clerk/md "### Getting difference in the rows of a base table, and it's supporting query table -")

{:nextjournal.clerk/visibility {:code :show :result :show}}
(analyze/get-diff "players" "players_by_city")

And this is the output –

The above 4 screenshots show the entire output of the queries shown in the preceding code. And with Clerk’s hot-reload, you get a simple but much more capable editor to interact with your Cassandra data. Because, not only can you run (plain but utterly limiting) CQL queries, but also those queries which plain CQL does not support (using the SqlForCql Clojure library) and view the output in a nicely formatted table.

I’ll have to check how to sort the table for every displayed column, but till then, enjoy the enhanced CQL editor.

The state of Scala (in Dec, 2022).

Note: This is an experience report. I’ve been using Scala for last ~ 4 years, and I’ve shipped Scala code in production. If you don’t want to read the long post, then you may directly jump to the conclusion section.

Background

Let’s take a look at the Scala / Java scene circa 2018. I had come to the JVM land after a decade of .Net / C# experience, and I always hated the duality which a .Net programmer had to endure if he / she was not using the default MS libraries (like Ado.Net, Enterprise library etc.). I always liked the open source libraries (like NHibernate, NUnit, Spring.Net) and mostly got frustrated that the prevalent literature / books / blogs were abundant for the original Java frameworks and very few for their .Net counterparts. Also, for the most part of my career, all I was doing in .Net land was writing a UI (if it was a thick client, then WinForms or WPF and if it was a thin client then Asp.Net or Angular) for some data in a Relational DB.

Hence, when I got an opportunity to actually switch to and use Java, I was ecstatic. I started with Java 7, and it frustrated me endlessly that there were no map, filter etc. methods on the standard Java collections (huge kudos to Linq in C#). Fortunately, I moved to Java 8 pretty soon and thankfully, similar methods on Java 8 streams were some relief (although I could never remember the .collect(Collectors.toList()) syntax for a long, long time).

The Honeymoon period

After spending some time with Java 7 & 8, I got the opportunity to use Scala (for the backend) and I picked up Play framework for writing APIs. Now, the Play framework and Apache Spark are the poster boys in the Scala world. Akka is also very famous, but I (personally) am not aware of many companies using it (except for the success story of Flipkart and a few other companies, though I could be wrong or less informed here). Similarly, I am not sure how many companies use Spark through Scala, because many of them might be using Spark through the Python bindings.

Writing Scala code (as compared to Java) was a joy –

  1. case classes (with value equality, auto-implemented toString method) are cool (a must needed feature in every programming language)
  2. map, filter, fold, mkString methods on collections are what one would expect from a modern language
  3. instantiating collections (with Seq(), Map()) is a breeze (without requiring extra third party libraries like Guava or something)

On top of that, Play framework is very intuitive for those who have used any kind of MVC frameworks before (I had used Asp.Net MVC, which itself is a Ruby on Rails clone), and the hot reload feature of Play during development (i.e. modify backend code and refresh the browser) makes it extremely powerful (specially when you are working with a type-safe language like Scala). I would even argue that maybe Play is the true poster child for Scala as Rails was for Ruby.

The settling in

After tasting success with writing REST APIs using the Play framework, I decided to experiment with micro-services (because that was the hotness), and after some R&D, I picked up THE ONLY micro-services framework available in Scala at that time, which was Lagom (I think to this date, Lagom still is the only Scala specific micro-services framework available). Now Lagom shines in a lot of places and it makes some things super simple – like event sourcing and CQRS, and guaranteeing once only, delivery of messages to Kafka etc. However, the downsides can’t be ignored –

  1. Because of events being stored together with the data, any database migration activity is non-trivial – you just can’t manipulate the data directly in DB tables (using CQL scripts, or code in some high level language)
  2. Similarly, if you are migrating from one Lagom version to another, you have to take care of migrating events (along with the data) and the process could be different for different Lagom versions
  3. The DAOs (or repositories) are so Lagom specific that you just can’t extract that code out into a separate jar or DLL. So if you need to access data from some other project, you don’t have any data access layer at your disposal and unfortunately, you’ll have to write that code again
  4. Lagom tries to map HTTP related concepts to Scala concepts, so for ex, to make an HTTP call, you don’t have to construct a URL or something, you make a method call. The problem is that this mapping is not foolproof, because you can’t use Try, Option or Either as return types of the methods which represent HTTP calls
  5. Last but not the least, there are just 1 or 2 books written on Lagom (unlike Play, which enjoys far greater acceptance)

The pure FP nirvana

We (the developers) realize the importance of pure functions and the Nirvana that pure FP promises, however, the fact remains that most of the non-trivial code which a developer has to write, is not pure, because it is littered with logging statements and until we grok libraries like Cats Effect (or ZIO, the new entrant), we are far from that goal. And believe me, grokking and finally using these libraries is not easy – at least that has been my experience so far – and if you don’t trust me, then listen to Daniel Ciocirlan (the Rock the JVM fame guy) – he too claims in one of his videos that it’s a challenge to learn any of these libraries. So if you don’t use any one of these libraries, then you would’ve mountains of impure code (in some time) – and this will be the case with most teams who are moving to Scala from Java. Initially, they’ll all start writing Java-ish code in Scala, then they’ll get familiar with Scala idioms and finally they have to break the pure FP barrier and it is by no means a small learning curve.

The Future (data type) conundrum

I don’t know if it is just me but if there are a lot of functions returning Future(s), then it becomes non-trivial to compose code that uses those functions. Here, I would like to specifically compare Future(s) with the async-await (called, the direct style) code permitted in many other languages like F#, C#, Python or JS. As per my experience, it is fairly trivial to compose async code in these latter languages, so much so that the flow of the code where all the functions are non-async vs. the flow of the code where all the functions are async is not much different. The only change you’ll see in those two pieces of code would be the ! (or some similar) operator. But in Scala, if every function you wrote previously which returned a T, starts returning a Future[T], then the code which is composing / using all these functions goes for a toss – not only that all the code has to be re-written, but the new code would not look even remotely similar to the old code. What’s worse is that there cannot be a common strategy using which one can rewrite the code. It once took me a week to write some code which was calling a lot of functions which all returned Future(s) – conceptually, I knew which calls were to be made, in which order, but to compose them together was a real PITA (had async await was in Scala, I could’ve finished the task in half a day or a day in worst case scenario).

However, the emergence of zio-direct, is a testimony to the fact that others have felt the pain too. And zio-direct seems like the much needed library for all my Future (related) problems. However, I am not sure if that library can be used alone without committing to the whole ZIO stack. If you are a Scala developer, please do yourself a favor by reading the README of this library.

The (jarring) jolts

Now comes the sad part for which, we the Scala users (developers), are not responsible.

Foremost in the list is Scala 3 which makes breaking changes and introduces new syntax 😞. Now, it’s not that difficult to learn a new syntax, but it’s an unnecessary inconvenience – OK to accept for a single developer but with a team of N developers, it’s a sizable number of wasted hours / productivity. Not to mention that all the open source libraries have also to be ported and you can imagine the magnitude of the problem. A Clojure stalwart BG (Baishampayan Ghose) pointed this out to me last to last year during IN/Clojure 2020 that Scala will go the Python way, and I wasn’t so sure about it, but the problem becomes clearer with every passing day. Even John De Goes tweeted the exact same sentiment a few days ago, that “the whole Scala ecosystem (libraries, tools, apps) needs a prolonged period of language stability to turn early traction into sustained momentum”.

Then come the two announcements by Lightbend about the Play and Lagom frameworks – that they are moving the future development of these frameworks to a community led organization.

  1. On the future of Play framework
  2. The future of Lagom

And the final nail in the coffin is the licensing of Akka (previously, it was an open source framework). This announcement has been the irony of my life. I wanted to move to Java because I wanted to use open source stuff, and after moving to the JVM land, the open source framework was licensed 🤷‍♂️(thankfully I didn’t learn or use Akka, otherwise it would’ve been one more wasted effort). And worse still is the fact that both Play and Lagom depend on Akka so I doubt anyone would pick any of these in the foreseeable future.

The conclusion

For me personally, Scala did not give me the best returns on the time I invested learning and mastering it. After close to 4 years of using it, I should have been in a position to use it for as many tasks across the stack (frontend, backend, thick clients, browser based UIs, mobile development, Notebooks and what not) but unfortunately I am far far away from it. I understand that Scala never aimed for it in certain areas e.g. mobile development, but unfortunately, Scala JS also did not gain as much traction as I would’ve liked. And worse, the libraries I invested in (Play, Lagom) have either been relegated to be maintained by the community (definitely, not a negative thing), or have been affected by the closed sourcing of Akka.

Worse still, is the fact that I have to learn a new syntax for Scala 3 (which is the present and definitely is the future). Most of the libraries which have a Scala 3 specific version are somehow associated with (or use) the pure FP ecosystem libraries in the Scala world i.e. Cats Effect or ZIO. So for example, if someone asks me to write a REST API in Scala, I am in a fix – because I want to use Scala 3 (because Scala 2 will be legacy soon) but I am not aware of the Play framework roadmap for supporting Scala 3, not to mention that Play uses Akka, so I would not like to touch it. And I don’t know http4s yet, because it’s built on top of Cats Effect, and I am yet to learn Cats Effect itself.

The recommendation – whether to choose Scala or not?

  1. Newer versions of Java have adopted a lot of (Scala) features which were missing back then when I started with Scala, like records (for case classes), pattern matching etc. It has Quarkus, Micronaut and some other frameworks for writing microservices. So a huge team may not see the productivity boosts which they might expect by adopting a whole new language (because of not just the initial learning curve, but the fact that most of the promising libraries are pure FP based). It’s still OK for smaller teams or teams with people having enough experience to learn pure FP concepts / libraries, to think of migrating to Scala.
  2. Scala is a very good language – with a lot of useful features and is much succinct than Java. However, if you plan to adopt it, directly go for Scala 3 because that’s the present and the future. And very importantly, do think of adopting either of the pure FP (advocating) libraries i.e. Cats Effect or ZIO. If you (or your team) can’t commit to learning any of these, then you’ll be in for a very hard time, because a lot of other libraries (like http4s) are built on top of one of these.
  3. 2023 is almost here and I don’t want to use different languages for different parts of the stack and Scala seems to be dominant only on the backend / server side code.
  4. F# which already compiles to JS (through Fable and Fable 4 (whose stable version is about to debut) is already targeting Rust, Python & Dart 🤯). Same goes for Clojure, which has ClojureScript (that targets JS), clj-python and Clojure-Dart will be a reality soon.
  5. Another advantage with both F# and Clojure is their lineage (ML and Lisp respectively). So if you know any one of them, you’ll be able to understand code from other languages which share that lineage (like ReScript or Racket).
  6. So my dear developer friends, if you want more bang for your buck (RoI), then look elsewhere (Clojure, F#). Scala (without learning the pure FP libraries) is not my recommended choice.

Functional Conf (Online) 2022 – an experience report.

This year was my very first attendance at the Functional Conf and wow, what an experience it was, worth every minute! I learnt something from almost all of the talks I attended, and I am pretty sure that the other talks which I could not attend would’ve been full of learnings too. So here is my summary of the talks I attended.

Day 1

First was a talk from Richard Feldman titled “The Essence of Functional Programming” during which he explained the difference between lexical (closer to the definition site) vs. dynamic (closer to the usage site) binding. If you don’t know the difference between these, then check the following code between JS vs. Scala and you’ll understand the difference 🤓. The point – most of the modern languages go with lexical closures these days.

A JS example –

var suffix = "!";
let exclaim = (str) => {
  return str + suffix;
};
exclaim("hello");
var suffix = "???";
exclaim("hello");

The same program in Scala

var suffix = "!"
def exclaim = (s: String) => s"$s$suffix"
exclaim("hello")
var suffix = "???"
exclaim("hello")

And his final advice – avoid mutations and avoid side effects.

Then came Bartosz Milewski’s “Teaching Optics through Conspiracy Theories” another amazing talk which talks about Lenses and/or Optics – this is a little advanced topic in itself so if you are interested then please watch this talk.

Then was John Azariah’s “Nature-Inspired Optimization Algorithms with F#” and not just the algorithms he showed were cool, but the kind of code which you can write in F# is cooler 😉 Again a must watch talk.

Then came the most difficult decision making moment for me, where I had to choose between Debasish Ghosh’s “Functional Programming patterns for designing modular abstractions” vs. Alfonso Garcia-Caro’s “Beyond: Crossing the Platform Boundaries with F# and Fable” and although I use Scala at work I chose the latter because frankly F# attracts me more than Scala (these days) because –

(1) It looks like, eventually F# will enable me (a programmer) to use the language in many more contexts (UI, both web and mobile, data-science(y) tasks and others)
(2) Learning languages which are members of a family e.g. ML (OCaml, ReScript) or Lisp (Clojure, Racket) etc. is much more beneficial than learning a language with a completely new syntax (like Scala). I guess that’s the reason why Scala is gravitating towards indentation sensitive syntax to appeal more to Python programmers

And, if you don’t trust me on (1) above, then watch Alfonso’s talk and be prepared to be blown away 🤯. The spoiler – Fable already allows you to compile F# to JS, but work is ongoing to compile it to PHP, Python, Rust and Dart !!! Thank you F#🤩and Alfonso mentions in his talk that learning a new ecosystem, new tools is much harder than it seems, so thank you Alfonso and others for trying to make our lives easier. BTW, there is another exciting ongoing effort to compile Clojure to Dart 🤩

I’ll definitely watch Debasish’s talk later and you must too (specially if you are a Scala programmer).

Then was Luca Mugnaini’s “Familiarity or Guarantees? Functional Programming for the front-end” talk which clearly makes the case about using Elm for the front-end and how it benefited them immensely.

The following talk from Adam Granicz, “Re-targeting F# for the web: from JS to WebAssemblymakes another persuasive point about how you, as a developer, and how various companies can’t go wrong with adopting F#.

Ayush Mittal’s “Type classes in Scala 3” was another practical talk about the advantages of Scala 3 over the older versions. Definitely, Scala 3 will enable greater productivity, but sadly, one has to spend time in learning new syntax, new concepts.

Mey Beisaron’s “Multiplayer Online Game in Clojure: Attack of the Clones” was a fun talk to attend which underlines the simplicity of solutions written in Clojure, which, in turn enables higher productivity.

As per my opinion, one can use F# or Clojure for more wide ranging tasks than Scala (at least, that's the situation right now). There is Scala.js but it's adoption seems less widespread than ClojureScript or the F# counterparts (Fable, SAFE stack, WebSharper and others combined).

Dean Wampler’s keynote, “Lessons Learned from 15 Years of Scala in the Wild” was another excellent one with a poignant question about whether FP adoption will continue to grow? And his answer sadly, is No and he thinks that either it will decline over time or at least the growth will mostly stall, because (in Dean’s words) –

(1) For most of the world’s developers, FP is either too hard or they lack the motivation to learn it (I can attest the point about it being hard – I haven’t been able to fully grok advanced Scala libraries like Cats, Cats Effect or ZIO, even after having 20+ years programming experience and 3+ years of Scala experience – I definitely understand bits and pieces of these frameworks but haven’t been able to use these broadly in my day-to-day work)
(2) In contrast, OOP is “naively” intuitive and therefore seductive
(3) Data science people who are domain experts in Statistics etc. but not so much in programming (or FP) and they mostly write small and focused code (scripts in Python & R)

And finally, Dave Yarwood’s “Clojure through the lens of music” was the perfect talk to end day one with some soothing music as the output of the examples he presented using his super-fun, super-cool alda-clj library. Hats off Dave for this ultra enjoyable library – I’ll have to produce some music now 🎹. One must watch this talk to understand how REPL-driven development in Clojure differs from other languages which support a REPL.

Day 2

Day 2 started with a bang 💥with Adam Rosien’s “Concurrent State Machines with Cats Effect” talk. Now, I’ve been struggling to grok these advanced typed FP libraries in the Scala ecosystem (as mentioned above) for some time but I was itching to try out the code ⌨️he was showing in his slides! Judging by his talk, I can safely say that his book Essential Effects would be awesome too.

Then came another informative talk by Nikhil Barthwal “Implementing Event-Driven Microservices architecture in Functional language” with examples in F#. Since we are writing micro-services using Lagom at work, this talk reinforced a lot of the concepts I knew by improving my understanding about those, and at the same time imparting fresh knowledge about other concepts.

Then I watched Tony Morris’ “Type-hole development” and although I don’t use or haven’t looked much into Haskell, the examples he was solving and dishing out techniques to solve those problems as a skillful master was a joy to look at. A must watch talk if you are into Haskell.

Then came another highlight of the day for me with Joydeep Banik Roy’s “Free Based DSLs For Distributed Compute Engines” talk – mind totally blown away🤯by learning how they’ve used Free Monads to create Zeoflow, a unified data processing DSL and how they’ve managed to abstract away compute engines like Apache Spark and Beam 🎯. I think I finally understand Free Monads now and I might be able to create my own (little) DSL 🤞. BTW, if you happen to use Spark through Scala, then you must check out Zeotap’s spark-property-tests library.

Grahame Dixon’s “Extending Railway Oriented Programming in Elm to Make Complex User Flows Simple” talk was another informative one. I think I first read this term Railway Oriented Programming in Scott Wlaschin’s F# for fun and profit blog posts but it was long ago and had totally forgotten what it is, so it was a good refresher.

Then came another wow talk by Allister Beharry on “The Z3 SMT solver and functional programming” – I had no idea of SMT solvers and the examples he showed using F# quotations were mind blowing 🤓. I must learn some meta-programming using either Clojure macros or F# quotations or both. The code looked very similar to its Python counterpart largely because of the use of quotations. Again, a must watch talk for F# lovers.

Then came another learning heavy talk by Michael Pilquist on “fs2.Chunk” – a data structure that powers the fs2 (Functional Streams for Scala) streaming library. I thought I’ll get to see some examples on how to use fs2 but Michael instead talked about the design of Chunk and in particular, how design constraints guided its evolution. There were so many things to learn about Scala and data structures in general, from this talk.

And finally, the icing on the cake was the final keynote by Bruce Tate & Francesco Cesarini “Navigating the loop in water, on land and in programming models” 🛥️- I couldn’t have imagined before that such a talk is even possible. A fun and practical mix of real life issues and situations mapped to the world of programming; sort of like weaving two stories together with a common pitch.

All in all, a conference with a potpourri of talks using various programming languages, ranging in depth, breadth and fun. If you haven’t attended any Functional Conference before then make sure to not miss the next edition. You’ll be glad that you took my advice. And don’t just take my word on this, but better yet, watch the videos on the Fn Conf YouTube channel.

Thanks to Naresh Jain @nashjain and the entire team of organizers for the overall meaningful experience spread over 2 days and thanks to the speakers not just for sharing their knowledge but keeping us excited about the journey ahead.

Using Clerk notebooks as documentation for your open-source Clojure libraries.

If you’ve used / created Jupyter notebooks then you know how useful they are. You can easily run the code in the shared notebook and collaborate easily. However, there are a couple of small problems / annoyances –

  • either you have to know Julia, Python or R
  • or, at the very least, you need to install Python (if you have to run the notebooks locally, even for other languages supported by Jupyter Notebooks)

But what if you are not a Python programmer? And like me, most of your work is on either of the two Enterprise(y) platforms JVM and / or .Net (i.e. any non data-science work)? Well, .Net has you covered with .Net Interactive Notebooks in VS Code – where you can create Notebooks with C# or F# and the best part is that you don’t have to install Python on your machine.

So what about Clojure then, which arguably is the best language on the JVM. Well, we now have Clerk – which advertises itself as Local First Notebooks for Clojure. And if you are not clear what that means, then please check the rationale for it and you’ll understand.

It’s obvious that it can be used for the same purpose as other Notebooks i.e. conveying code, ideas in code with markdown (for documentation / explanation / comments) and code interspersed. So I tried to use a simple open-source library – SqlForCql, which I wrote some time ago, to help me in querying Cassandra which is not supported using plain CQL. Unfortunately, the code was somehow not working in an older version of Clerk, but with the current version it worked seamlessly. So let’s see what local first notebooks look like in action.

Below is a simple code which uses my library

(ns clerkex.sforcex
  (:require [sqlforcql.core :as core]
            [sqlforcql.cql :as cql]
            [sqlforcql.analyze :as analyze]
            [nextjournal.clerk :as clerk]))

(defn connect []
  ;; sqlforcql below is the cassandra keyspace
  (core/connect-to-default-db "localhost" 9042 "username" "password" "sqlforcql"))

(defn disconnect []
  (core/disconnect-from-default-db))

(connect)

(clerk/table (cql/get-all "players"))
(clerk/html [:h3 (str "Count: ") (cql/get-count "players")])
(clerk/table (cql/get-by-non-pk-col "players" {:city "Ajmer"}))
(clerk/table (cql/get-by-pk-col "players" {:nickname "fedex"}))
(clerk/table (cql/get-by-pk-col "players_by_city" {:city "Jodhpur" :country "India"}))
(clerk/table (cql/get-by-non-pk-col-like "players" {:city "Shabi"}))
(clerk/table (analyze/get-counts ["players" "players_by_city"]))
(clerk/table (analyze/get-diff "players" "players_by_city"))

(disconnect)

And now let’s see how Clerk renders it in the browser (by default at localhost:7777) –

And a few more examples –

Notice, Clerk has just run the code from the code I pasted above the screenshots. Surely, the cells are not interactive at this point, but I am sure that’s what the authors are aiming for. Huge thanks to Martin Kavalar, Michiel Borkent and the other contributors for this excellent tool (in the making). Till then, add some bit of a visual appeal to the documentation of your open source efforts – happy hacking!

Essential Scala tricks.

If you have some experience with Scala but are new to advanced functional programming concepts, used / propagated by libraries like ScalaZ or Cats, then a few of the tricks mentioned here are good to have, and a few others are must have.

First & foremost, you must understand the concepts as mentioned in the Cats jump start guide. A lot of the concepts here are taken from this same document, so let’s start the adventure.

1. Converting a value to an optional value

Suppose, you have to convert an Int to an Option[Int]. The usual way is:

Option(5)
Some(5)

Using cats:

import cats.syntax.option._

5.some

2. Converting a value to an Either

The usual way:

Right(7)
Left(“some error occurred”)

Using cats:

import cats.syntax.either._

7.asRight
“some error occurred”.asLeft

3. Converting a value to a Future value

The usual way:

Future.successful(5)

We can use the following implicit class:

import scala.concurrent.Future

implicit class ObjOps[T](obj: T) {
  def asFuture: Future[T] = Future.successful(obj)
}

And then we may use:

5.asFuture

Now you may say that we haven’t gained much apart from the convenience of a few less keystrokes, but these are huge time savers when you have to change a lot of existing values to a different type of value. e.g. suppose there was a function which takes an Int parameter, and for some reason, we modified that function to take an Option[Int] parameter, and suppose this function is getting called multiple times, then to wrap an int value with Option() or Some() is a lot more tedious than just appending .some to that value 😀

4. Converting a List[Right[_]] or List[Left[_]] to Right[List[_]] or Left[List[_]]

val xs: List[Either[String, Int]] = List(Right(1), Right(2), Right(3))
val seqXs = xs.sequence // Right(List(1, 2, 3))

val ys: List[Either[String, Int]] = List(Right(1), Left("error"), Right(3))
val seqYs = ys.sequence // Left(error)

5. Getting a product of two Either(s)

Suppose you have two Either values, and if you know that both of them are Right values, then you can get a tuple (product) of both the Right values as follows:

import cats.syntax._

type EitherString[T] = Either[String, T]

val r1: Either[String, Int] = 5.asRight
val r2: Either[String, Int] = 6.asRight
val mergedRs: EitherString[(Int, Int)] = Semigroupal[EitherString].product(r1, r2)
println(s"merged rights: $mergedRs") // merged rights: Right((5,6))

Be careful, that if both of them are Left values, then you do not get the product of the left values but only the first left value (error), as is evident by the type signature.

val e1: Either[String, Int] = "Can't divide by zero".asLeft
val e2: Either[String, Int] = "Can't divide by negative number".asLeft
val mergedLs: EitherString[(Int, Int)] = Semigroupal[EitherString].product(e1, e2)
println(s"merged lefts: $mergedLs") // merged lefts: Left(Can't divide by zero)

6. Getting results from unrelated Futures

Suppose you have a few functions which return Future(s), and these functions are not related to each other, meaning that the return value of any of the functions is not being used in any other function, then instead of calling those functions using a for-comprehension (because then, they will all execute sequentially, one after the other), you can use cats mapN function (because then, these futures will execute parallelly).

package example

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import java.time.LocalDateTime
import java.time.Duration
import scala.util.Failure
import scala.util.Success
import java.time.ZoneOffset

import cats.instances.future._
import cats.syntax.apply._

object FutureEx extends App {

  val before = LocalDateTime.now()

  val sum = for {
    a <- getA
    b <- getB
    c <- getC
  } yield (a + b + c)

  sum onComplete {
    case Failure(ex) => println(s"Error: ${ex.getMessage()}")

    case Success(value) =>
      val after = LocalDateTime.now()
      println(s"before: $before")
      println(s"after: $after")

      val diff = getDiff(before, after)
      println(s"diff: $diff")
      println(s"sum: $value")
  }

  // let the above finish
  Thread.sleep(20000)

  val beforeN = LocalDateTime.now()
  val usingMapN = (getA, getB, getC).mapN(add)

  usingMapN onComplete {
    case Failure(ex) => println(s"Error: ${ex.getMessage()}")

    case Success(value) =>
      val afterN = LocalDateTime.now()
      println(s"beforeN: $beforeN")
      println(s"afterN: $afterN")

      val diff = getDiff(beforeN, afterN)
      println(s"diffN: $diff")
      println(s"sum: $value")
  }

  // let the above finish
  Thread.sleep(20000)

  def getA: Future[Int] = Future {
    println("inside A")
    Thread.sleep(3000)
    2
  }

  def getB: Future[Int] = Future {
    println("inside B")
    Thread.sleep(3000)
    3
  }

  def getC: Future[Int] = Future {
    println("inside C")
    Thread.sleep(3000)
    4
  }

  def add(a: Int, b: Int, c: Int) = a + b + c

  def getDiff(before: LocalDateTime, after: LocalDateTime): Long = {
    Duration.between(before.toInstant(ZoneOffset.UTC), after.toInstant(ZoneOffset.UTC)).toMillis()
  }
}

/** Using for comprehension
  *
  * inside A
  * inside B
  * inside C
  * before: 2021-02-01T19:07:44.148
  * after: 2021-02-01T19:07:53.343
  * diff: 9195
  * sum: 9
  */

/** Using mapN
  *
  * inside A
  * inside C
  * inside B
  * beforeN: 2021-04-29T11:14:51.508
  * afterN: 2021-04-29T11:14:54.530
  * diffN: 3022
  * sum: 9
  */

As you can see in the comments above, the first version (using for comprehension) took more than 9 secs. (because we’ve used Thread.sleep(3000) in each of the getA, getB and getC functions). However, the second version (using mapN) took a little over 3 secs. because all 3 functions started executing together and in fact, in this particular run, the function getC started executing before the function getB.

Har Har Modi

I think for the first time in my life, and so many other youths and old Indians, people from rural or urban areas, people across caste, creed, religion have been so excited in any Lok Sabha elections, so much so that the voting percentage rose in most constituencies.

The sweet results of the Lok Sabha elections 2014 in the largest democracy in the world have been spectacular and historic for so many reasons –

1) For the first time post independence, first non-Congress party got a full majority
2) Most regional parties (including the major culprits SP, BSP, JDU) except, TMC and ADMK fared badly – which means people have voted less on the basis of caste,
creed, communal basis, language and have voted more for development, good governance, for India as a whole
3) People have voted against dynasty, family
4) People have voted for a stable government (since it’s a clear majority for BJP)
5) For the first time, the opposition has to form a coalition, and not the ruling party

There have been some notable, well-deserved losers – specially either those ministers who were in the earlier government, or their kith-and-kin, namely –

Ajit Singh – Baghpat – RLD
Chhagan Bhujbal – Nashik – NCP
Rabri Devi, Misa Bharti – Bihar – RJD

Pawan Bansal – Chandigarh – Congress
Kapil Sibal, Ajay Maken, Kapil Sibal – Delhi – Congress
Sachin Pilot – Ajmer – Congress
Meira Kumar – Sasaram – Congress

Ghulam Nabi Azad – J&K – Congress
Farukh Abdulla – J&K – National Conference
Salman Khurshid – Farrukhabad, UP – Congress
Priya Dutt, Milind Deora, Sanjay Nirupam – all from Mumbai – Congress

Karti Chidambaram (P Chidambaram’s son) – Sivaganga, Chennai – Congress

The winning BJP has a clean sweep in 6 states – Gujarat, Rajasthan, Delhi, Goa, Uttarakhand and Himachal Pradesh and more than a couple of Union Territories. Many regional parties have been utterly destroyed – Mayawati’s BSP got 0, Mulayam Singh’s SP got 5 (only those seats where he or his family members fought) – so he can become the PM of his own family ;-), Nitish Kumar’s JDU got only 2 seats, Lalu’s RJD got only 4.

Ultimately, Indians have chosen the person (Modi) who talked only about development, and have defeated parties which talked of caste, pseudo-secularism and vilified Modi (and the BJP).

This is my small tribute to the person and the party who did the seemingly impossible task of garnering a majority in India –

‘हर हर मोदी’

 

कांग्रेस सरकार ने किये बड़े ही झोल
पर बेचारे ‘मौन-मोहन’ कभी ना पाये बोल

माया, मुलायम सेंकते थे जात-धर्म की रोटी
पर ‘भारतीयों’ ने कर दी उन सबकी किस्मत खोटी

लालू, शरद पवार भी बेहतर हाल में होते
पर भाजपा के कारण हैं अपने घर में रोते

सिबल, खुर्शीद और बेनी की नहीं देखनी शक्ल
पाँच साल घर बैठ के सीखो थोड़ी अक्ल

ना चली नितीश की ‘टोपी’, ना चला ‘आप’ का झाडू
अब क्या करें ‘अच्छे दिन जो अाने वाले हैं’ भीड़ू

बड़े बड़े तीसमार खाँओं ने अपनी इज्जत खो दी
अब तो ‘हर हर मोदी घर घर मोदी, हर हर मोदी घर घर मोदी’

                                                                   — मनोज वाईकर

Progress of nations.

I came back from a tour of Thailand, Malaysia and Singapore. The more I see different countries, the more I realize that India is tops in the bottom 10% of the countries. In all these three countries, the roads are excellent, people follow traffic rules, the buildings are made not just for housing or business but also for viewing pleasure, the lamp posts are not just vertical structures from which a street light pokes out but beautifully shaped ones, trees are cut, shapely, obviously to enhance the beauty.

So –

Christian countries are advanced (US, UK etc.)
Muslim countries are advanced (Malaysia (which is a multi racial country, but has majority Muslims), Dubai etc.)
Communist countries are advanced (like China)

Shall I then infer that India is not prospering because it is divided by various religions, languages, castes etc.?

Sigh!!! Will India ever prosper?

Elitism

I used to think that most (not all) people in the society should have access to something which the government is building. But I was naive 🙂 I read this beautiful article about Macau (in the Marathi newspaper Saamna), and how it is China’s ATM. The key things to take away from the article are –

1) Even though the local currency is pataca, the currencies which are allowed in the casinos are Euro, Dollars etc. (the stronger currencies)
2) It is (along with Hong Kong) two special administrative regions of China – so one country two systems. [Even India has special provisions for J&K ;-)]
3) Even Chinese nationals need a visa to visit Macau or Hong Kong (to avoid crowding, WOW).

There is a time and place for everything and the five fingers are not all alike.

The beginning of end for OO?

Is it the beginning of end for OO – OO programming out of CMU CS introductory curriculum.

Well, I don’t know but after learning some lisp programming, I think other languages have far too incidental complexities – the cancer of semi-colon 😉 specifying types etc. that makes the syntax ugly and is of no help in coding. On the other hand, Lisp is beautiful, it is symmetry and simplicity combined and it would’ve done me a ton of good, if I had learnt Lisp after learning C. So at least for me it is a step in right direction and I hope other Indian schools / universities take notice.