Distributed Systems in One Lesson By Tim Berglund

Distributed Systems in One Lesson By Tim Berglund

1. what is Distributed System?
A collection of independent computers that appear to its users as one computer. ---Andrew tannebaum
Three characteristics:
Operate concurrently/fail independently/Do not share a global clock.

Three Topics: Storage/Computation/Messaging

Storage:
Single - Master storage
        Read/Write
Replication.
    Sharding
        Consistent hashing

Consistency
R + W > N

CAP Theorem. hacker news
        Consistency
        Availability
        Partition Tolerance
Shared Writing project

Distributed Computation
MapReduce
        map: count words, word: number
        shuffle: move similar word together
        Reduce: add up the numbers
        
Hadoop:
        MapReduce API
        MapReduce job management
        Distributed FileSystem(HDFS)
        Enormous ecosystem
Spark:
        Scatter/gather paradigm(similar to MapReduce)
        More general data model(RDDs, Datasets)
        More general programming model(transform/action)
        Storage agnostic
Kakfa:
        Focuses on real-time analysis, not batch jobs
        Streams and streams only
        Except streams are also tables(sometimes)
        No cluster required

Messaging
        *Means of loosely coupling subsystems
        *Messages consumed by subscribers
        *Created by one or more producers
        *Organized into topics
        *Processed by brokers
        *Usually persistent over the short term
Messaging Problems
        *What if a topic gets too big for one computer?
        *not reliable enough
        *guarantee delivery?
Apache kafka

Definitions:
        Message: an immutable array of bytes
        Topic: a feed of messages
        Producer: a process that publishes messages to a topic
        Consumer: A single-threaded process that subscribe to a topic
        Broker: one of the servers that comprises a cluster.
Kafka(Trivial version)
        Topic partitioning.
Kafka(Interesting version)
  * Message bus
  * Turned events into rows-in-place
  * compute events
Without streams.

Lammda Architecture
        Long-term storage
        Batch processing
        Slow, complete
      
        Tempoarary queuing
    Stream processing
    Fast summaries
With streams



听课时候做的笔记，很有收获，和大家共享。 谢谢加米。        

        


        

safari book 有这个教材的进阶版。
Distributed Systems in One Lesson， 但是每个component讲的都比较浅， 需要自己再做额外的学习。

hotinherre 发表于 2019-5-26 13:48
safari book 有这个教材的进阶版。
Distributed Systems in One Lesson， 但是每个component讲的都比较浅 ...

多谢，多谢。搜了下。在这里：https://player.oreilly.com/videos/9781491924914

谢谢楼主分享这个视频，很有用！

Megan1129 发表于 2019-5-26 07:21
谢谢楼主分享这个视频，很有用！

哈哈，不客气。有用就好！

非常有用！

Thanks for sharing

多谢楼主，入门好资料

Thank you so much

不错啊 mark一下