Skip to content

Instantly share code, notes, and snippets.

@fanf
Last active November 17, 2021 06:59
Show Gist options
  • Save fanf/e99fe83f716ae37e94ae264b38634bbb to your computer and use it in GitHub Desktop.
Save fanf/e99fe83f716ae37e94ae264b38634bbb to your computer and use it in GitHub Desktop.
Jan Nasiadka interview questions about ZIO

Jan Nasiadka from scalac contacted me to get some feedback about our usage of ZIO. My answers below, since they can benefit the community more broadly.

Context: we use ZIO in the context of https://rudder.io, a scala application that is 11 years old - far from a greenfield application with ZIO as base architectural choice. We used ZIO as a better framework to cleanly manage errors, porting piece of existing code to it. We are part of ZIO community since 2018 (since ZIO inception, when it was not yet ZIO but a part of scalaz, and when there was only a bifunctor, no context in it). Given our usage, we use ZIO in a specific way: we have tons of entry points and evaluation of ZIO effects, not one main entry point in the "main" method of the app. For correctness, that forced us to call a lot of blocking effects (you never know what a java lib is doing), and so we stressed the thread pool ergonomics, and helped make that part better (and some part of the changes in ZIO 2 regarding pool management are linked to that, and issue like: zio/zio#1275)

With that in mind, let's process questions:

1/ What do you think about ZIO’s typed errors? How do you think they help to write better applications?

This is heart of why we switched to ZIO. Actually, I even gave a talk at Scala.IO 2019 about that(in french, slides here) and a revised, more entry level and broader version here for devoxxfr 2021(in French, too - slides here).

And here, you have the main issue in our bugtracker to explain the why: #14870 Use ZIO for effect management in Rudder.

To sum up: ZIO allows to manage errors systematically in a principled, composable, extensible way; it gives us the tool to cleanly reports them to the 3 kinds of users of an app (final users, ops and dev) and react to them appropriatly. It's a joice as a developer to correctly take care of errors with ZIO.

2/ Do you think ZIO helps you to make code more testable? Why?

As I said, we don't use a full-fledge ZIO app, so we don't take benefits of the full power of ZIO and ZIO tests for that. Specifically, we are not using ZLayer and the whole IoC framework given by ZIO, which is a big part of making integrations (or "big unit tests") easy. Nonetheless, composition (in the FP meaning, ie being able to split a big problem into smaller ones, solve (and prove they are solved because of properties of the solution) each small problem, compose back small solutions into a bigger one that KEEPS the properties you checked) is a key part of testability. By helping us to split big problems as we need (not only sequentially, or by object responsiblity, but as really we need it, even if it implies concurrency, distributed states, complex error handling on asynchrones things, etc), ZIO helps us make smaller parts from our big programs, and so make them easier to unit tests. Testing a function with just input/output is much more easy than testing an app with a complicated context to set up.

3/ What’s your experience with ZManaged? What problems do you think it helps with?

Our experience is great :) We use it broadly to cleanly manage closing of disposable resources without even having to thing about it, which is a relief, because that problem is damn hard and I really don't want to have my team spend countless hours on things like "we are leaking file descriptors and it kills the app/db connection/whatever", which adds exactly 0 value to our product and is extremelly complicated to get right in concurrent/async env. And rudder is doing a lot of that concurrent/async work, with file systems and DB and etc.

The case for ZManaged is actually the same for all other concurrent primitive given by ZIO in the context of effects. These problems are hard and likely need to be handled by your more experienced dev who should spend time elsewhere; they rarely had value to your product directly (ie your product business is not "managing concurrency", it's more "build a MMO game", or whatever); but they create massively impacting bugs, even showstopper ones (so you can't just ignore them). I'm very glad I don't have to spend more time about thinking about if my file input stream is cleanly closed in all possible cases of concurrent exceptions.

4/ What’s your experience with ZLayers?

We barely have. We assessed that ZLayers in V1.0 of ZIO were too imature and best practices not understood enought to pay for the massive task that porting Rudder to them would be - again, that is specific to our use case of a hundred of thousand of line of existing code prior to ZIO.

But the ZLayer concept is absolutly wonderfull, and I dreamed about a principled and composable IoC framework for the scala ecosystem since 2006. I only had to wait 15 years to get it, especially since ZIO 2.0 is cleaning a lot of the complicated part of using Zlayer, and best practices start to come. Still, the price for us is extremelly hight, and I'm not sure we will be able to pay for it in Rudder. We will need to introduce ZLayer by bit, which is not simple.

For a greenfield app, it's the best thing that happened to JVM application modularity since springframework (which had good ideas and TERRIBLY bad implementation, with lots of shared state and impossible to understand weawing logic, plus all the boilerplate of the XML years of java...)

5/ An important characteristic of ZIO is compositionality, how do you think that helps when writing real-world applications?

I think I already answered to that, but to sum up: being able to split big problems into smaller ones, solve the small ones proving that some properties hold, then compose back the small solutions to solve the big problem BEING ASSURED that the properties are kept - even if the composition implies effects, concurrency, asynchronicity, etc.

6/ Do you have experience with ZStreams? What problems do you think it helps with?

No experience with them. Streams were not a thing in 2009 when we started rudder, and so they are not part of our major arch choice (not our core business problem iether). We would use them today, in greenfield app, because streams are such an elegant solution to lots of problems. Typically, in rudder we compute configuration compliance for servers based on requirement set by users and feedback about the real state of things in servers. All that could be seen as a stream of effects (configuration changes, compliance reports) that continually updates the state of our world view ("that server has that configuration requirements and reports these assessments, its compliance it now that"). Again, a big change that perhaps we won't choose to pay.

7/ What do you think about ZIO’s interruption model and how it helps with achieving global application efficiency?

A big question. The fiber model is very nice, it allows to split problems as best fit for the problem solution, not as it is technically possible given the reality of resources (threads) on the OS. Typically, it's generally much simpler to thing about a process on 10 000 servers as 10 000 time the process on one server - but thread pool are not meant to do that, especially if the process has steps like "do these 100 things (semantically) in parallel, go to next step when one these conditions arise". Plus, ZIO debuggability (fiber dump) is exceptionnal, making actually easy to understand what went wrong in an async/concurrent environment, which is such a relief and an upgrade compared to whatever I know elsewhere.

8/ How did ZIO benefit your projects?

Already answered, but sum-up with: greatly. Like the jvm, it handles big, hard, low business value things efficiently (memory management and portability for the jvm for ex, effect composition for ZIO) so that we don't have to deal with them, which is a relief.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment