highaltitudedev: Netty Conflict with TwitterServer and Play WS

TL;DR when an app runs from the IDE but not when invoked by another means, such as command line, it could be a dependency conflict where classpath ordering matters. AKA, seemingly inexplicable errors are often explicable, but only after some headachy troubleshooting

I might be the first person to run TwitterServer with akka and Play WS. There's really no reason for this combination of frameworks. I first started working with Play WS (built on AsyncHttpClient) because it's super simple (much lass convoluted than akka-http). See, super easy:

val updateIp = client
.url(s"https://app.herokuapp.com/ip-address/${ip}")
.withMethod("POST")
.execute()

updateIp.onComplete {
case Success(response) =>
   response.status match {
     case 200 =>
       log.info(s"Heroku ip address update returned ${response.body}")
     case _ =>
       log.warning(s"Heroku update ip address failed with status ${response.status}")
   }
case Failure(e) =>
   log.warning(s"Heroku update ip address failed with exception ${e}")
}

Then I added akka and later tossed TwitterServer into the mix for the sweet stats. Here's a screenshot of the admin server UI that comes with TwitterServer

Admittedly this chart isn't too exciting but the stats provided by TwitterServer (actually from Finagle) and the included utilities (profiling etc) are exceedingly useful.

After I added TwitterServer, the app was running just fine in Intellij but I could not run from the command line. Every time I saw:

java.net.BindException: Failed to bind to 0.0.0.0/0.0.0.0:9999: Unable to create Channel from class class io.netty.channel.socket.nio.NioServerSocketChanne\

When I see this my first reaction is I left an instance running

netstat -lnp | grep 9990

but nope, so this made absolutely no sense.

Then, if I removed TwitterServer from the code but kept the dependency I got past the bind exception:

WARN c.r.app.AppActor - Failed to get ip from heroku java.net.ConnectException: Unable to create Channel from class class io.netty.channel.socket.nio.NioSocketChannel

In this case there were conflicting dependencies with Netty. Specifically the Netty version required by TwitterServer and Play WS were not in agreement. When I was running from Intellij the Netty version that loaded first happened to work for both (Play WS and TwitterServer). But, the SBT JavaServerAppPackaging loaded the problematic/conflicting Netty classes first, which resulted in the weird and misleading Netty errors. I was able to solve the problem by simply shuffling the order of the dependencies in build.sbt.

Works:

libraryDependencies += "com.twitter" %% "twitter-server" % "1.25.0"

libraryDependencies += "com.twitter" % "finagle-stats_2.11" % "6.40.0"

libraryDependencies += "com.typesafe.play" %% "play-ws" % "2.5.4"

libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.4.16"

libraryDependencies += "com.typesafe.akka" %% "akka-http-core" % "10.0.0"

libraryDependencies += "com.typesafe.akka" %% "akka-http" % "10.0.0"

libraryDependencies += "com.typesafe.akka" %% "akka-http-testkit" % "10.0.0"

libraryDependencies += "com.typesafe.akka" %% "akka-http-spray-json" % "10.0.0"

libraryDependencies += "com.typesafe.akka" %% "akka-http-jackson" % "10.0.0"

libraryDependencies += "com.typesafe.akka" %% "akka-http-xml" % "10.0.0"

libraryDependencies += "com.typesafe.akka" %% "akka-slf4j" % "2.4.16"

libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.1.3"

libraryDependencies += "com.typesafe.scala-logging" %% "scala-logging" % "3.1.0"

Fails: