Functional Programming and Composing Actors

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
 50
 
  With the world being non-deterministic, with failure being abundant, and with communication latency being very real—how do we design systems that are capable of dealing with these conditions and how can we expose abstractions that are feasible to reason about?
Related documents
Share
Transcript
  • 1. &√
 Deputy CTO
 Functional Programming Composing Actors
  • 2. & Functional Programming without sacrificing Communication Resilience √
 Deputy CTO

  • 3. 4 Connectedness «Software is becoming increasingly interconnected»
  • 4. 5 Klang’s Conjecture «If you cannot solve a problem without programming; you cannot solve a problem with programming.»
  • 5. Reality Imaginary >
  • 6. My office door 7 Stunning views
  • 7. Reality • separation in space & time gives us • communication for coordination • variable delays • partial failures • only partial/local/stale knowledge • systems 8
  • 8. 9 system «a set of things working together as parts of a mechanism or an interconnecting network;
 a complex whole» noun
  • 9. Systems • Purpose is typically simple • Complex inner workings • Consist of collaborating components • Components can be systems themselves • Components are as simple as feasible, but not simpler 10
  • 10. 11 Resilience «Any sufficiently complex system is always running in degraded mode» — Richard I. Cook, MD “How complex systems fail” (paraphrased)
  • 11. Communication • The production and consumption of messages • Messaging implies that we need to be able to address recipients • Addresses/Identities are important • They are knowledge that can be shared • Messages can become delayed, lost 
 or misunderstood 12
  • 12. Think: Reliability • Are we ever guaranteed delivery? • at-most-once • at-least-once • exactly-once • It’s not about guarantees,
 it’s about reliability 13
  • 13. Two-Phased Commit 14
  • 14. Burstiness • Most communication is bursty • some is predictable • some is unpredictable • load shedding can cause burstiness 15
  • 15. Flow control / Back pressure 16 • Buffers are only grease between cogs • Does not solve overload problems • Load shedding does not inform sender • Reasons • When shedding will end www.reactive-­‐streams.org
  • 16. Resilience • Never assume that other entities are immortal • Treat expectation violations as failures • Always have a Plan B • Clients are not responsible to fix a faulty provider • Fail fast & predictably 17
  • 17. 18 Supervision • Responsibility to deal with the failure/corruption of other • Does not place the burden of fixing it on the collaborators «Quis custodiet ipsos custodes?» — Decimus Iunius Iuvenalis
  • 18. 19 actors • Akka's unit of computation is called an actor • Akka actors are purely reactive components: • Current behavior • Address • Local storage • Mailbox • Scheduled to execute when sent a message • An actor has a parent actor, handling its failures • An actor may have 0..N child actors
  • 19. 20 « 2500 nodes × millions of actors per GB RAM = a lot» actors • An actor processes a message at a time • Multiple-producers & Single-consumer • The overhead per actor is about ~450bytes • Run millions of actors on commodity hardware • Akka Cluster currently handles ~2500 nodes actors
  • 20. 21 actors • Great for • Communication • Location transparency • Elasticity • Resilience • Main challenge • Composition
  • 21. Functional Composition a → b → c
  • 22. 23 Functional Programming • Great for • Composition • Main challenge • Communication • Resilience
  • 23. 24 typed: On the horizon • Project Gålbma • distill an Actor to its essence: the Behavior • everything is a message—for real this time • remove the danger to close over Actor environment • behavior composition • completely pure Actor implementation,
 through a process algebra (inspired by JoinCalculus)
  • 24. Behavior is King 25 abstract class Behavior[T] { def management(ctx: ActorContext[T], msg: Signal): Behavior[T] def message(ctx: ActorContext[T], msg: T): Behavior[T] def narrow[U <: T]: Behavior[U] = this.asInstanceOf[Behavior[U]] }
  • 25. Behavior example 26 object Server { sealed trait Command case class Get(id: Int)(val replyTo: ActorRef[Got]) extends Command case class Put(name: String, ref: ActorRef[OtherCommand]) extends Command case class Got(id: Int, contents: Map[String, ActorRef[OtherCommand]]) val initial: Behavior[Command] = withMap(Map.empty) private def withMap(map: Map[String, ActorRef[OtherCommand]]) = Total[Command] { case g @ Get(id) => g.replyTo ! Got(id, Map.empty) Same case Put(name, ref) => withMap(map.updated(name, ref)) } }
  • 26. Most common dev task? • Receive inputs • Transform data • Produce outputs • Drink coffee 27
  • 27. 28 #undef STREAMS #define STREAMS • ephemeral, time-dependent, sequences of elements • possibly unbounded in length • in essence: transformation & transportation of data «No man ever steps in the same river twice, for it's not the same river and he's not the same man.» — Heraclitus
  • 28. streams
  • 29. i m m u t a b l e REUSABLE c o m p o s a b l e c o o r d i n a t e d
 asynchronous
 transformations
  • 30. Getting data across an asynchronous b o u n d a r y
  • 31. Getting data across an asynchronous b o u n d a r y
 with non-blocking
 back pressure
  • 32. Reactive Streams Initiative T H E
  • 33. 38 Requirements Push Pull support potentially unbounded sequences 😃 😃 sender runs separately from receiver 😃 😃 rate of reception may vary from rate of sending 😃 😃 dropping elements should be a choice and not a necessity 💩 😃 minimal (if any) overhead in terms of latency and throughput 😃 💩 Comparing Push vs Pull
  • 34. & 39 Supply Demand
  • 35. 40 Publisher Subscriber data demand
  • 36. • “push” when subscriber is faster • “pull” when publisher is faster • switches automatically between both • batching demand allows batching ops 41 Dynamic Push–Pull Reactive Streams
  • 37. 42 Requirements Push Pull Both support potentially unbounded sequences 😃 😃 😃 sender runs separately from receiver 😃 😃 😃 rate of reception may vary from rate of sending 😃 😃 😃 dropping elements should be a choice and not a necessity 💩 😃 😃 minimal (if any) overhead in terms of latency and throughput 😃 💩 😃 Comparing Push vs Pull vs Both 🌟
  • 38. Stream splitting 43 demand data splitting the data means merging the demand
  • 39. Stream merging 44 merging the data means splitting the demand
  • 40. Source Flow Sink RunnableGraph
  • 41. 46 streams: Linear transformations val fives = Source.repeat(5) val timesTwo = Flow[Int].map(_ * 2) val intToString = Flow[Int].map(_.toString) val transform = timesTwo via intToString val sysOut = Sink.foreach(println) val r = fives via transform.take(10) to sysOut 
 r.run() // a Materializer needs to be in scope
  • 42. BidiFlows
  • 43. 48 streams: Selected BidiFlows Examples val codec: 
 BidiFlow[Foo, ByteString, ByteString, Foo, Unit] = …
 
 val crypto:
 BidiFlow[ByteString, ByteString, ByteString, ByteString, Unit] = …
 
 val framing: 
 BidiFlow[ByteString, ByteString, ByteString, ByteString, Unit] = …
 
 val protocol = codec atop crypto atop framing
  • 44. DAG DCG WAT
  • 45. Fan-In Fan-Out &
  • 46. Fan-tastic!
  • 47. 52 streams: Cthulhu-Merge-Route Explained
  • 48. OI
  • 49. Materialization
  • 50. 55 streams: Materialization • Akka Streams separates the what from the how & when • declarative Source/Flow/Sink DSL to create a blueprint • ActorMaterializer turns this into running Actors • enables customizable materialization strategies • optimization • verification / validation • distributed deployment
  • 51. 56 streams: MaterializedValues • At Compositiontime • MaterializedValues of respective sides are composed • At Materializationtime • A value of the final type of the MaterializedValue
 of the graph is returned
  • 52. 57 streams: Composition
  • 53. 58 streams: Composition val runnableGraph = FlowGraph.closed() { implicit builder => import FlowGraph.Implicits._ val A: Outlet[Int] = builder.add(Source.single(0)) val B: UniformFanOutShape[Int, Int] = builder.add(Broadcast[Int](2)) val C: UniformFanInShape[Int, Int] = builder.add(Merge[Int](2)) val D: FlowShape[Int, Int] = builder.add(Flow[Int].map(_ + 1)) val E: UniformFanOutShape[Int, Int] = builder.add(Balance[Int](2)) val F: UniformFanInShape[Int, Int] = builder.add(Merge[Int](2)) val G: Inlet[Any] = builder.add(Sink.foreach(println)) C <~ F A ~> B ~> C ~> F B ~> D ~> E ~> F E ~> G }
  • 54. 59 streams: Composition Functional Programming Akka Streams Akka Typed Functional Programming
  • 55. 60 streams: Future? Functional Programming Akka Streams Akka Typed Functional Programming
  • 56. 61 Summary Akka Streams and Akka Typed promises both the compositionalstrengths of FP, paired with the communicationalstrengths and resilience of the Actor Model.
  • 57. √
  • 58. Opportunity: Self-tuning back pressure 63 • Each processing stage can know • Latency between requesting more and getting more • Latency for internal processing • Behavior of downstream demand • Latency between satisfying and receiving more • Trends in requested demand (patterns) • Lock-step • N-buffered • N + X-buffered • “chaotic”
  • 59. Opportunity: Operation elision 64 • Compile-time, using Scala Macros • fold ++ take(n where n > 0) == fold • drop(0) == identity • <any> ++ identity == <any> • Run-time, using intra-stage simplification • map ++ dropUntil(cond) ++ take(N) • map ++ identity ++ take(N) • map ++ take(N)
  • 60. Opportunity: Operation fusion 65 • Compile-time, using Scala Macros • filter ++ map == collect • Run-time, using intra-stage simplification • Rule: <any> ++ identity == <any>
 Rule: identity ++ <any> == <any> • filter ++ dropUntil(cond) ++ map • filter ++ identity ++ map == collect
  • 61. Opportunity: Execution optimization 66 • synchronous intra-stage execution N steps then trampoline and/or give control to other Thread / Flow
  • 62. 67 public interface Publisher<T> { public void subscribe(Subscriber<T> s); } public void Subscription { public void request(long n); public void cancel(); } public interface Subscriber<T> { public void onSubscribe(Subscription s); public void onNext(T t); public void onError(Throwable t); public void onComplete(); } public interface Processor<T, R> extends Subscriber<T>, Publisher<R> { }
  • 63. How does it connect? 68 SubscriberPublisher subscribe(subscriber) onSubscribe(subscription)
  • 64. How does data flow? 69 SubscriberPublisher subscription.request(1) onNext(element) subscription.request(3) onNext(element) subscription.request(1)
  • 65. How does data flow? 70 SubscriberPublisher onNext(element) onNext(element) subscription.request(2) onNext(element) onNext(element)
  • 66. How does it complete? 71 SubscriberPublisher subscription.request(1) onNext(element) onComplete() onNext(element)
  • 67. What if it fails? 72 SubscriberPublisher subscription.request(1) onNext(element) subscription.request(5) onError(exception) ☠
  • 68. 73 Try Akka Streams: (1.0)
 https://github.com/typesafehub/activator-akka-stream-scala References Reactive Streams for JVM
 https://github.com/reactive-streams/reactive-streams-jvm
  • Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks