This is the third in a series of posts introducing Functional Programming concepts. The previous posts are:
Today we're looking at the 'big scary thing' - otherwise known as Monads.
Introduction
Every programming language / style has its 'big scary thing'. The thing that you've been warned is impossible to comprehend but (allegedly) opens the gates to undreamed of coding splendour. Once you have mastered the BST you gain the respect of your peers and are officially welcomed into the cognoscenti. Back when I was at college learning 'C' the official BST was pointers, but for those raised in OO it was polymorphism.
Monads are no more or less than the official BST of functional programming. But rest easy, if you were capable of making sense of pointers and/or polymorphism there's nothing (well little) to fear.
A comparison with Interfaces
Indulge me for a moment and consider interfaces in Java. Back when you first learnt OO, and had just got your head around inheritance, the notion of an interface probably made little sense. After all, interfaces are redundant in that they can be represented as abstract classes containing only abstract methods. This makes them unnecessary in small codebases intended only for a single environment. So exhortations by your betters to 'only use List' and 'program to the highest type possible' probably made little sense.
However, once you made the move up to JEE and started writing vendor neutral code the advantages of interfaces suddenly became apparent. At some point you realised that 90% of the references in your JEE components were of interface types and that this ensured independence from containers, databases, providers etc. You started using them in your own work to express contracts and perhaps became an 'interface bore' during code reviews.
Monads in FP are a lot like interfaces in OO. Technically they are an optional feature, and in small codebases they don't necessarily make a lot of sense. However, the more you play with FP the more you will end up either faking them yourself (perhaps unknowingly) or moving to languages that support them.
An initial example
Heres some Angular 1 code that uses its built in $http module to access RESTful services. Angular allows promises to be chained so that the output from one asynchronous request becomes the input of another. If all the promises complete then we will execute the onComplete handler. But if at any point a response fails we jump to onError.
$http.get(serviceUrl)
.then(worker1)
.then(worker2)
.then(worker3)
.then(onComplete)
.catch(onError);A practical example of this might be a price comparison scenario, where we acquire an initial cost for a service and then want to see if it can be beaten by a number of other suppliers. So as we call successive suppliers we always pass in the lowest quote seen so far. If you've ever used this feature, or anything like it, then you've already implemented whats known as monadic composition.
Let's back away from this for the moment and work up to it with some simpler use cases.
Iteration One - Optional Extras
Let's say our intern Josh, who at time of writing has less than a week to go, decides to play a prank and hide the network password on me. He assures me that if I go to the training room PC and get the the value of the property, whose name is the value of the property, whose name is the value of the property, whose name is the value of the property 'foo' then all will be revealed. Here's how I could write code to solve the problem in Scala:
import scala.util.Properties._
object Program {
def setup(): Unit = {
setProp("foo","bar")
setProp("bar","zed")
setProp("zed","wn1hgb")
}
def main(args: Array[String]): Unit = {
setup()
demo1("foo")
demo2("foo")
}
private def demo1(firstProperty: String) = {
val option = for (
step1 <- propOrNone(firstProperty);
step2 <- propOrNone(step1);
step3 <- propOrNone(step2)
) yield step3
println(option.getOrElse("Damn you Josh!!!"))
}
private def demo2(firstProperty: String) = {
val option = for (
step1 <- propOrNone(firstProperty);
step2 <- propOrNone(step1);
step3 <- propOrNone(step2)
) yield step3
option match {
case Some(passwd) => println(passwd)
case None => println("Damn you Josh!!!")
}
}
}I wrote a detailed intro to Option back in 2014. But for the present discussion all you need to know is that an Option is a set of either zero or one members. A Some is a set of size one and a None is a set of size zero. Whenever we call the method propOrNone it returns a Some[String] if the property exists and a None[String] otherwise. I can use Scala's handy for syntax (which looks like a loop but isn't) to make the chain of method calls for me and yield the final result. I've written the solution two ways, once using getOrElse to handle the final check and once using a match statement to make the two options more obvious.
Assuming this makes sense let's add a little complexity...
Iteration Two - Exceptional Times
Let's say my colleague and (until now) friend Eamonn decides to get in on this pranking jape and re-hide the network password. He places it in a file, whose name is held as the content of a file, whose name is held as the content of a file, whose name is held as the content of a file that he emails me.
Once again we solve the problem in Scala:
import java.nio.file.Files.readAllLines
import java.io.File
import scala.io.StdIn._
import scala.util.{Try,Success,Failure}
object TryDemo {
def firstLineOrFail(origin: String): Try[String] = {
val path = new File(origin).toPath()
Try(readAllLines(path).get(0))
}
def main(args: Array[String]): Unit = {
val initialPath = readLine("Enter the path to the first file\n")
demo1(initialPath)
demo2(initialPath)
}
private def demo1(initialPath: String) = {
val attempt = for (
step1 <- firstLineOrFail(initialPath);
step2 <- firstLineOrFail(step1);
step3 <- firstLineOrFail(step2)
) yield step3
println(attempt.getOrElse("Damn you Eamonn!!!"))
}
private def demo2(initialPath: String) = {
val attempt = for (
step1 <- firstLineOrFail(initialPath);
step2 <- firstLineOrFail(step1);
step3 <- firstLineOrFail(step2)
) yield step3
attempt match {
case Success(passwd) => println(passwd)
case Failure(ex) => println("Damn you Eamonn!!!")
}
}
}Note that what I've done in firstLineOrFail is to take the Java 8 Files.readAllLines method and wrap it up in a Try. Just like Option[T] a Try[T] is a data structure with two possibilities - the first is a Success holding an instance of T whilst the second is a Failure holding some type of exception. Once again I can use the handy for syntax to chain together all the method calls and resolve the final result implicitly (with getOrElse) or explicitly (with a match).
Hopefully you can see a certain structure beginning to repeat itself...
Iteration Three - Promises, Promises...
Let's say that my boss Tara is so enamoured by all this hilarity that he decides to join in and secrete the network password behind a RESTful service. In this case, sending the right sequence of GET requests will retrieve the password, or a 500 if an incorrect value is used.
@RestController
@RequestMapping("/prank")
public class PrankGarthService {
private Map<String,String> values;
public PrankGarthService() {
values = new HashMap<String,String>();
values.put("foo", "bar");
values.put("bar", "zed");
values.put("zed", "wn1hgb");
}
@RequestMapping(value = "/{input}", method = RequestMethod.GET, produces = "application/json")
public ResponseEntity<String> ping(@PathVariable("input") String input) throws Exception {
System.out.println("Prank service called with input of " + input);
if (!values.containsKey(input)) {
return new ResponseEntity<String>("Incorrect input", HttpStatus.INTERNAL_SERVER_ERROR);
}
Thread.sleep(5000);
return new ResponseEntity<String>(values.get(input), HttpStatus.OK);
}
}
Once again Scala rides to the rescue, in the form of the Play WS API which is now available as a standalone library.
This time we have some setup to do since:
- The WS API relies on an Akka materializer object
- We need to create our own
StandaloneAhcWSClient - Once our work is done the above needs to be closed
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import play.api.libs.ws._
import play.api.libs.ws.ahc._
import scala.concurrent.Future
import scala.util.{Success, Failure, Try}
object ProgramOne {
import scala.concurrent.ExecutionContext.Implicits._
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem()
system.registerOnTermination { System.exit(0) }
implicit val materializer = ActorMaterializer()
val wsClient = StandaloneAhcWSClient()
def fetchOrFailClient = fetchOrFail(wsClient)_
val attempt = for (
step1 <- fetchOrFailClient("foo");
step2 <- fetchOrFailClient(step1);
step3 <- fetchOrFailClient(step2)
) yield step3
attempt.onComplete(result => {
process(result)
shutdown(system,wsClient)
})
}
def process(result: Try[String]) = result match {
case Success(value) => println(s"Retrieved $value from server")
case Failure(ex) => println("Damn you Tara!!!")
}
def fetchOrFail(client: StandaloneWSClient)(input: String): Future[String] = {
val rootUrl = "http://localhost:8080/prank/"
return client.url(rootUrl + input).get().map(_.body)
}
private def shutdown(system: ActorSystem, wsClient: StandaloneAhcWSClient) = {
wsClient.close()
system.terminate()
}
}
As you can see I've written a fetchOrFail method which takes the WS Client and URL in separate parameter lists. If you remember from the previous blog post this will allow us to partially invoke the function, which gives us fetchOrFailClient. Yet again the snazzy for syntax enables us to chain together the method calls and yield the final result. But what types are we working with?
If you examine the return value of fetchOrFail you will see that it's sending back a Future. Futures are also known as Promises or IOUs and represent a value which has yet to be produced. Code running in one thread passes a promise to another such that once that final value has been 'cooked' the client thread can extract it without fear of heisenbugs or blocking. Note that a Future has a very similar structure to Option and Try in that it is a container which comes in two separate varieties.
Monads explained in OO terms
If you're still with me then you should have a good idea of what a Monad is. A Monad is a type of container that enables a value to be passed through a chain of operations where (to quote Mythbusters) "Failure Is Always An Option". When I was trying to grok Monads I found this series of articles to be extremely helpful, and especially the idea of an 'Amplified Type'.
As I understand it an Amplified Type is any type that contains one or more instances of another in order to add extra functionality. Hence any kind of container is an Amplified Type, as is an Option, a Try, a Promise, a java.io.InputStream or java.io.OutputStream and anything that would be termed a Decorator or Adapter (in Design Patterns terminology).
We use Monadic Composition when we are using an Amplified Type to pass values through a chain of operations, just like in the examples above. The only difference is that typically the types of the values will change, instead of always being strings. So for example an Option[Order] might be used to create an Option[Shipment] and then in turn an Option[Invoice].
If we want to use Monads in our code then it's very helpful if the language itself has a built in operator for performing the composition, and that of course is what the for syntax does in Scala. As discussed in this excellent presentation by Dick Wall most coders start out assuming that for is only a loop but incrementally discover it's actually something much more powerful.
Returning to our original example
Returning to our original example from the Angular 1 $http library we can now see that they came up with their own implementation of the Promise Monad for their own needs. All well and good, but if the language was a little bit more functional they wouldn't have needed to.
$http.get(serviceUrl)
.then(worker1)
.then(worker2)
.then(worker3)
.then(onComplete)
.catch(onError);Iteration Four - Try and Try Again...
If you look back at our third iteration above you will see that the value referenced by attempt was of type Future[Try[String]] instead of Future[String]. This is because in any kind of network IO errors are possible. What we didn't take account of was the fact that the RESTful service might return something other that a 200 response to a valid request.
As the Play documentation says this idiom "is a good way to chain WS calls in a trusted environment". However, I don't trust Tara that much :-)
Lets modify fetchOrFail so that we check the status of the response that comes back from the server using the Try Monad. The return value from the method therefore becomes Future[Try[String]] and hence the final result is of type Try[Try[String]]. This illustrates one of the nicest points about Monads which is that, in the words of Oleg Kiselyov, they "turn control flow into data flow, where it can be constrained by the type system".
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import play.api.libs.ws._
import play.api.libs.ws.ahc._
import scala.concurrent.Future
import scala.util.{Success, Failure, Try}
object ProgramTwo {
import scala.concurrent.ExecutionContext.Implicits._
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem()
system.registerOnTermination { System.exit(0) }
implicit val materializer = ActorMaterializer()
val wsClient = StandaloneAhcWSClient()
def fetchOrFailClient = fetchOrFail(wsClient)_
val attempt = for (
step1 <- fetchOrFailClient(Success("foo"));
step2 <- fetchOrFailClient(step1);
step3 <- fetchOrFailClient(step2)
) yield step3
attempt.onComplete(result => {
process(result)
shutdown(system,wsClient)
})
}
def process(result: Try[Try[String]]) = result match {
case Success(value) => {
value match {
case Success(result) => println(s"Retrieved $result from server")
case Failure(ex) => println("Damn you Tara!!!")
}
}
case Failure(ex) => println("Damn you Tara!!!")
}
def fetchOrFail(client: StandaloneWSClient)(input: Try[String]): Future[Try[String]] = {
val rootUrl = "http://localhost:8080/prank/"
return client.url(rootUrl + input.get).get().map(resp => resp.status match {
case 200 => Success(resp.body)
case _ => Failure(new RuntimeException())
})
}
private def shutdown(system: ActorSystem, wsClient: StandaloneAhcWSClient) = {
wsClient.close()
system.terminate()
}
}
Iteration Five - Introducing Either
Looking back at that last iteration we did a slightly silly thing. We wanted to consider 4** and 5** status codes as errors but the Play WS API didn't, so we forced the issue by saying Failure(new RuntimeException()). A more harmonious solution would be to use the Either monad. Either represents a value from one of two types, represented by the classes Left and Right. In our case both types will be String but we will use the left type to represent a 2** response from the server and the right for all other cases.
The code is otherwise the same as before, but we have better expressed our intent.
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import play.api.libs.ws._
import play.api.libs.ws.ahc._
import scala.concurrent.Future
import scala.util.{Failure, Success, Try}
object ProgramThree {
import scala.concurrent.ExecutionContext.Implicits._
def main(args: Array[String]): Unit = {
implicit val system = ActorSystem()
system.registerOnTermination { System.exit(0) }
implicit val materializer = ActorMaterializer()
val wsClient = StandaloneAhcWSClient()
def fetchOrFailClient = fetchOrFail(wsClient)_
val attempt = for (
step1 <- fetchOrFailClient(Left("foo"));
step2 <- fetchOrFailClient(step1);
step3 <- fetchOrFailClient(step2)
) yield step3
attempt.onComplete(result => {
process(result)
shutdown(system,wsClient)
})
}
def process(result: Try[Either[String,String]]) = result match {
case Success(value) => {
value match {
case Left(result) => println(s"Retrieved $result from server")
case Right(error) => println(s"Error: $error")
}
}
case Failure(ex) => println("Damn you Network!!!")
}
def fetchOrFail(client: StandaloneWSClient)(input: Either[String,String]): Future[Either[String,String]] = {
val rootUrl = "http://localhost:8080/prank/"
return client.url(rootUrl + input.left).get().map(resp => resp.status match {
case 200 => Left(resp.body)
case _ => Right("Damn you Tara!!!")
})
}
private def shutdown(system: ActorSystem, wsClient: StandaloneAhcWSClient) = {
wsClient.close()
system.terminate()
}
}Peeking under the hood
If you read more technical introductions to Monads they often start with the assertion that this is all a Monad is:
def unit[A](x: A): TheMonad[A]
def flatMap[B](x: A, y: A => TheMonad[B]): TheMonad[B]This is true, but amazingly unhelpful. Its like going into a OO Design Patterns class, being presented with some UML diagrams, and being told "thats all Visitor is" or "thats all Composite is". What is meant is that in order to use something as a Monad two operations must be available.
The first is normally called unit and simply takes an instance of the input type A and returns it within the Monad. The second is more complex, it takes an instance of A and a function that is capable of transforming the instance of A into a Monad of B and applies the one to the other. Once you have these operations then your languages syntax for composing monads should work for your type.
Conclusions
I hope this post has provided some clarity as to what a Monad is and why you would wish to use one. If you have any special requests for the next topic in this series feel free to send suggestions via the Facebook page or Twitter feed.
