Martin Kleppmann Profile Banner
Martin Kleppmann Profile
Martin Kleppmann

@martinkl

50,187
Followers
993
Following
413
Media
10,988
Statuses

Find me at @martin .kleppmann.com on Bluesky, @martin @nondeterministic .computer on Mastodon. Author of @intensivedata , Associate Professor @Cambridge_CL . he/him

Cambridge, England
Joined October 2008
Don't wanna be here? Send us removal request.
Pinned Tweet
@martinkl
Martin Kleppmann
8 months
I’m no longer active on Twitter. Find me at on Bluesky, or on Mastodon. If I follow you here or know you personally, you can DM me for a Bluesky invitation code. (Sorry, don’t have enough invitation codes for people I don't know.)
3
2
18
@martinkl
Martin Kleppmann
4 years
In 1665, the University of Cambridge temporarily closed due to the bubonic plague. Isaac Newton had to work from home, and he used this time to develop calculus and the theory of gravity.
310
8K
16K
@martinkl
Martin Kleppmann
3 years
The G7 summit should be followed by a C major summit to resolve everything.
64
930
5K
@martinkl
Martin Kleppmann
4 years
✨New educational materials!✨ Today I'm announcing two new, free resources I've written: an 8-lecture course on fundamentals of distributed systems, and a 30-page tutorial on elliptic curve cryptography.
29
475
2K
@martinkl
Martin Kleppmann
1 year
My partner and I had a baby! I’m on parental leave for the next 3 months (plus planning to work part-time for a while later). Therefore I probably won’t be tweeting much. Feeling fortunate to be in a place with good maternity care and generous parental leave ❤️
Tweet media one
91
3
1K
@martinkl
Martin Kleppmann
6 years
Pro tip: instead of paying $399/year for @safari books online (frankly outrageous), you can get ACM professional membership for $99/year, which includes Safari access.
42
317
1K
@martinkl
Martin Kleppmann
6 years
I passed my PhD viva (aka thesis defence) yesterday. Yay! 👨‍🎓😎🎉
79
8
1K
@martinkl
Martin Kleppmann
1 year
This looks like a great resource – a series of distributed systems programming challenges designed by Kyle Kingsbury aka aphyr
3
195
986
@martinkl
Martin Kleppmann
3 years
My partner is learning programming. When she struggles to figure something out, but I can solve it in a few seconds, I remind her: she can sit at the piano and easily sight-read a piece she’s never seen, while I have to struggle through note by note. Skills require practice.
13
66
967
@martinkl
Martin Kleppmann
4 years
Happy new year 🥂 Ever wondered where the western musical scales come from, and why there are 12 notes in an octave? I wrote up a quick explainer.
Tweet media one
Tweet media two
Tweet media three
Tweet media four
19
166
956
@martinkl
Martin Kleppmann
6 years
Estimated number of clock cycles it takes to do various things on a modern CPU, e.g. atomic CAS or context switch: — useful for a concurrent systems course I'm currently teaching
9
402
943
@martinkl
Martin Kleppmann
4 years
I am concerned that a lot of computer systems research is solving problems that only huge tech companies have (eg. fast datacenter networking, big data analytics systems), rather than working on technologies that empower individual users and the underprivileged.
@random_walker
Arvind Narayanan
4 years
By the same token, it should be a sobering moment for computer science academia. With few exceptions, work that tries to bring accountability to big tech companies is relegated to the fringes of our discipline. CS these days cozies up to power far more than speaking truth to it.
5
125
477
17
243
936
@martinkl
Martin Kleppmann
2 years
Very excited to announce that I've been awarded a €1.3m grant to start a research group on local-first software at TU Munich @TU_Muenchen ! This will enable us to build the foundation for collaboration software of the future.
56
83
926
@martinkl
Martin Kleppmann
2 years
Hello Munich! I have moved here from Cambridge and will be starting a new position at @TU_Muenchen in October. Excited!
Tweet media one
Tweet media two
42
16
904
@martinkl
Martin Kleppmann
4 years
Over 100,000 copies of my book @intensivedata have now been sold. In this blog post I examine some of the economics of book-writing
59
134
901
@martinkl
Martin Kleppmann
6 years
Such an amazing slide deck by @deniseyu21 on why distributed systems are so hard
Tweet media one
6
407
852
@martinkl
Martin Kleppmann
3 years
I give away lots of work for free: conference talks, educational videos, research papers, open source code, blog posts. If you want me to continue doing this, please consider supporting me on Patreon! Thanks ✨ ➡️ Blog post:
24
174
680
@martinkl
Martin Kleppmann
6 years
Philosophiae Doctor
Tweet media one
55
7
665
@martinkl
Martin Kleppmann
5 years
Netflix have made their own Change Data Capture (CDC) tool for MySQL and PostgreSQL… though maybe it would have been better to contribute to @debezium ?
19
189
646
@martinkl
Martin Kleppmann
4 years
lol, thank you hacker news for the thoughtful critique 🙄
Tweet media one
61
26
638
@martinkl
Martin Kleppmann
5 years
My book @intensivedata is apparently the second-best-selling book across @OReillyMedia 's entire book catalog in 2019. Delighted that so many people are finding it useful!
28
45
631
@martinkl
Martin Kleppmann
2 years
How amazing is it that GPS actually works? Satellites are at least 20,000 km away; measuring the distance to ~2m accuracy means 0.00001% error. Also the satellite is moving at 3 km/sec and the measurement equipment fits in your phone. What other consumer device has such accuracy?
15
36
609
@martinkl
Martin Kleppmann
3 years
I was asked to give a keynote at the ACM Conference on Distributed and Event-based Systems ( @ACM_DEBS ). So I tried to summarise everything I know about events in 50 minutes! 🎥 📄 Hope you enjoy it ✨
12
138
592
@martinkl
Martin Kleppmann
1 year
Google's Service Weaver claims that it can hide the difference between RPC and local method calls. Just like CORBA claimed in 1991. Is it different this time? The docs say barely a word about handing failures, such as RPCs timing out
31
85
582
@martinkl
Martin Kleppmann
7 years
Forgive my self-indulgence for a second… I walked into Heffers in Cambridge, and guess what I found there on the shelf? 😎
Tweet media one
Tweet media two
25
41
569
@martinkl
Martin Kleppmann
4 years
Apparently I’ve now crossed 30k followers... hello everybody and welcome! 👋 What are you hoping to hear about? Is miscellaneous distributed systems geekery good? 😊
53
4
558
@martinkl
Martin Kleppmann
7 years
Never before have I been sooo excited to see a pile of paper!! 😆📚😄🐗 @intensivedata
Tweet media one
Tweet media two
41
78
548
@martinkl
Martin Kleppmann
4 years
I finally got round to reading @ifesdjeen 's Database Internals. It's a nice book! If you enjoyed @intensivedata and want more detail on certain topics (especially storage engines and consensus), it's a good follow-on read.
Tweet media one
6
103
546
@martinkl
Martin Kleppmann
2 years
I have officially started at @TU_Muenchen as of this week! Over the coming months and years I will be building up a research group to focus on local-first software ✨🧑‍🔬
Tweet media one
Tweet media two
19
23
543
@martinkl
Martin Kleppmann
4 years
Today in “distributed systems are hard”: I wrote down a simple CRDT algorithm that I thought was “obviously correct” for a course I’m teaching. Only 10 lines or so long. Found a fatal bug only after spending hours trying to prove the algorithm correct. 😭
9
38
528
@martinkl
Martin Kleppmann
4 years
New video! “CRDTs: The Hard Parts” 🎥 This talk explains several new research results in CRDTs, including problems in some published algorithms, weird edge cases, and work we’re doing to improve performance.
3
148
507
@martinkl
Martin Kleppmann
5 years
Our article “Online Event Processing: Achieving consistency where distributed transactions have failed”, with @arberesford and @bsvingen : Now in print in the latest edition of @CACMmag : (paywall) and online in @ACMQueue : (free)
Tweet media one
7
170
483
@martinkl
Martin Kleppmann
4 years
The distributed systems course covers many of the fundamental algorithms behind today's apps. It consists of 87 pages of detailed notes (including exercises) and 7 hours of lecture videos
9
121
466
@martinkl
Martin Kleppmann
4 years
To all interested in CRDTs: Marc Shapiro, @anne_biene and I have set up a little CRDT community website. Lots of links to papers, blog posts, talks, and implementations. Contributions welcome!
6
195
459
@martinkl
Martin Kleppmann
3 years
✨ New article: Peritext, a CRDT for rich-text collaboration 📄 This one was hard work. We went through at least half a dozen algorithm designs until we found one that worked! Joint work with @geoffreylitt @sliminality and @pvh at @inkandswitch
7
115
429
@martinkl
Martin Kleppmann
6 years
Over 50,000 copies of my book @intensivedata have been sold. I can’t believe it! Thank you everyone who has helped spread the word about it.
24
36
388
@martinkl
Martin Kleppmann
2 years
This book on PostgreSQL internals looks like a great resource on isolation levels, concurrency control, and various storage engine topics
5
72
386
@martinkl
Martin Kleppmann
6 years
Such a powerful idea; keeps getting reinvented under different names (event sourcing; lambda/kappa architecture; database inside-out/unbundled; state machine replication; etc).
@neil_conway
Neil Conway
6 years
Reflecting on the fact that I've built some flavor of the "immutable, append-only log + materialized views" pattern into every non-trivial software project I've built since 2006.
8
34
207
9
140
382
@martinkl
Martin Kleppmann
3 years
Was Base64 intentionally designed so that encoding all zeroes looks like screaming into the void? AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
4
33
384
@martinkl
Martin Kleppmann
3 years
This is mind-blowingly clever work by @justinetunney . A single binary that runs natively on Windows/Linux/Mac/*BSD without any VM, implementing a web server that serves the content of a zip archive stored in the binary itself! 🤯🤯👏👏
8
109
375
@martinkl
Martin Kleppmann
5 years
Kafka is inching ever closer to being a database, and I think this is pretty cool. Subscribe to a query, and get notified when its results change.
@jaykreps
Jay Kreps
5 years
We think this is a big step in making stream processing easier. I'm really excited about it.
9
189
510
8
107
377
@martinkl
Martin Kleppmann
3 years
I see parallels between musical instrument skills and programming skills. Many people learn them in their youth, but both are totally learnable as adults if you’re willing to put in the many hours of practice they require. It’s not natural talent, it’s persistence that matters.
12
37
371
@martinkl
Martin Kleppmann
2 years
Folks moving to Mastodon are swapping a service run by a capricious egomaniac for one where the admin of your home instance controls everything about your account. And you probably don't know what your server admin is like when you sign up. Is this really much better?
30
53
365
@martinkl
Martin Kleppmann
2 years
The picture is from "General Zoology" by George Shaw, vol. ii, part 2, page 458, published in 1801. It's even on the Internet Archive!
@teej_m
» teej
2 years
I thought these were drawn exclusively for O’Reilly. My whole life is a lie.
Tweet media one
375
4K
36K
6
47
366
@martinkl
Martin Kleppmann
5 years
My student's experimental new database beats MySQL by approximately 700% on the TPC-C benchmark in initial tests. This research is going somewhere interesting!
10
36
343
@martinkl
Martin Kleppmann
2 years
I took a look at a GCSE Computing exam paper (an exam that 16-year-olds sit in the UK). It made me sad how dull, arbitrary, and pointless many of the questions are. Not appropriate for beginners. Surely this sort of material will put many kids off going into computing.
Tweet media one
Tweet media two
Tweet media three
Tweet media four
21
57
334
@martinkl
Martin Kleppmann
4 years
Nice blog post by @davex_tech summarising the discussion around “Is Kafka a database?”, and when to use streams versus when to use a relational DB.
7
82
323
@martinkl
Martin Kleppmann
2 years
When users enter sensitive data into an app (eg. period trackers) and it’s stored unencrypted in the cloud, the app developer looks increasingly irresponsible. Local-only or local-first software with p2p data sync and/or end-to-end encryption needs to become the default.
@mattblaze
matt blaze
2 years
The central server design, on the other hand, puts EVERYONE's data in one place. And someone might obtained it hacking into the app's servers, by buying it from them, or by serving them with a legal order to turn it over. Any of these could happen without users' knowledge. 7/
2
7
49
8
78
315
@martinkl
Martin Kleppmann
4 years
I set myself a goal of not working weekends (which I used to do regularly). Looking at my GitHub activity, it seems like I've been doing okay. Hooray for avoiding burnout!
Tweet media one
3
7
322
@martinkl
Martin Kleppmann
6 years
This thoughtful article by @martinfowler on the pros and cons of remote working is made even better by featuring a tortoise on a rocket-powered skateboard 🐢🔥
Tweet media one
2
148
312
@martinkl
Martin Kleppmann
3 years
Every technology contributes to determining who has what power. Therefore, every technology is political. If a technology seems apolitical, that probably just means that it supports the incumbents and doesn’t challenge the status quo. Likewise, every organisation is political.
11
67
314
@martinkl
Martin Kleppmann
6 years
The energy cost of a single Bitcoin transaction is about 300kWh. The same amount of energy will let you drive 1,000 miles in a car!
26
205
303
@martinkl
Martin Kleppmann
4 years
I have never waited more than 2 minutes to vote in an election in UK/Germany. With queues like this, voter suppression campaigns, gerrymandering, a two-party duopoly, extreme partisanship, etc. the US doesn’t really look like the shining model democracy that it was once seen as.
10
44
305
@martinkl
Martin Kleppmann
3 years
Did you know that the breadth-first-search algorithm in most textbooks is needlessly inefficient? A simple tweak can sometimes turn an O(n^2) runtime into O(n)
2
53
311
@martinkl
Martin Kleppmann
6 years
$ npm install -g create-react-app $ create-react-app demo $ du -hs demo 158M OMG, JavaScript people, what are you doing?
18
92
301
@martinkl
Martin Kleppmann
5 years
I just shared a taxi with Leslie Lamport, but unfortunately I was too tired to have any sensible sort of conversation
10
6
295
@martinkl
Martin Kleppmann
1 year
For discussion I refer you to Waldo et al.’s classic “A Note on Distributed Computing”, which is still as true today as it was 30 years ago
Tweet media one
6
44
297
@martinkl
Martin Kleppmann
2 years
New paper alert! 📄 Most CRDT algorithms assume that all participants correctly follow the protocol. I show that we can make CRDTs robust against Byzantine behaviour, and it's not even terribly difficult.
2
57
293
@martinkl
Martin Kleppmann
4 years
Our group’s work on CRDTs and collaboration software is built explicitly upon a philosophy of trying to shift power away from cloud providers, and back towards end users. More on this in our paper-cum-manifesto on “local-first software”
7
54
292
@martinkl
Martin Kleppmann
4 years
Modern business jargon uses the word “talent” to refer to employees. Is it just me, or is this almost as offensive as calling people “resources”? I find “talent” dehumanising, and it focuses the attention on innate characteristics rather than developing learnable skills.
20
33
285
@martinkl
Martin Kleppmann
8 years
New paper by @arberesford and me: “A Conflict-Free Replicated JSON Datatype” — we figured out a JSON CRDT
15
167
283
@martinkl
Martin Kleppmann
4 years
Proof-of-work is an abomination. Totally irresponsible.
@bascule
Tony "Abolish ICE" Arcieri 🦀🌹
4 years
Bitcoin energy consumption hits a new all time high of nearly 9GW, comparable to Chile, a country with 18M people. Carbon footprint is ~37 Mt CO2 annually, about that of New Zealand. And yet it still does ~4 transactions per second...
21
201
318
6
86
277
@martinkl
Martin Kleppmann
6 years
It’s so heartwarming when strangers come up to me and tell me how my talks/book have helped them (in designing their systems, getting a job, …) ❤️ Thank you!
20
11
282
@martinkl
Martin Kleppmann
7 years
A CRDT-based collaborative editing feature for @AtomEditor ! Exciting, CRDTs are catching on 😄
5
141
278
@martinkl
Martin Kleppmann
3 years
I just re-recorded my lecture on the Raft consensus algorithm, since the previous version had a couple of mistakes. New version here: Accompanying lecture notes:
1
55
276
@martinkl
Martin Kleppmann
4 years
I am so delighted that “The Art of Doing Science and Engineering” by Richard Hamming (inventor of error-correcting codes and many other things) is now available in print again. Thank you @stripepress @_TamaraWinter for sending me a copy!
Tweet media one
7
19
269
@martinkl
Martin Kleppmann
10 years
I'm writing a book called “Designing Data-Intensive Applications” for O'Reilly! @intensivedata http://t.co/4Ywa4j7SZb http://t.co/jTr7jyBc47
Tweet media one
23
103
263
@martinkl
Martin Kleppmann
5 years
Wow. Certain SSDs have a firmware bug causing them to irrecoverably fail after exactly 32,768 hours of operation. SSDs that were put into service at the same time will fail simultaneously, so RAID won't help. 😱
4
132
269
@martinkl
Martin Kleppmann
8 years
New blog post: “How to do distributed locking”. A critique of Redis’ Redlock algorithm, and general guide to locks:
6
204
261
@martinkl
Martin Kleppmann
3 years
adding " @NotionHQ going offline, apparently due to problems renewing their Somalian (.so) domain name" to my list of reasons why local-first software is a good idea
8
20
257
@martinkl
Martin Kleppmann
4 years
Tried using AWS Lambda/API Gateway for a little side project. The getting started wizard first asks you to choose between a “REST API” and a “HTTP API” (???). Then it pre-populates the new function with broken example code that crashes when you try to run it.
20
17
257
@martinkl
Martin Kleppmann
3 years
Our paper got accepted to ACM CCS (a leading security conference) — yay! 😄😄 Going to open a nice bottle of Riesling tonight 🥂
8
5
252
@martinkl
Martin Kleppmann
7 years
Very pleased that my research grant got extended, so I'll be at Cambridge working on CRDTs for another two years 😄
@trvedata
TRVE DATA
7 years
TRVE Data update for September 2017
0
2
5
10
15
250
@martinkl
Martin Kleppmann
6 years
“Is Kafka a Database?” That’s the (slightly provocative) title of my keynote at #kafkaSummit today. It will be live-streamed, starting at 9am Pacific Time / 5pm UK time
7
89
243
@martinkl
Martin Kleppmann
2 years
Excited to announce my latest fun project: I wrote an illustrated children's book on cryptography together with Mitch Seymour @_round_robin ! We're launching it this Thursday, and to celebrate we're doing an AMA on Reddit (4pm UK / 8am Pacific). Come join us!
Tweet media one
5
8
241
@martinkl
Martin Kleppmann
3 years
Happy that our paper on a move operation for CRDT trees has finally been accepted by IEEE TPDS! We've spent over two years trying to get this thing published, eventually successful on the sixth submission…
11
30
238
@martinkl
Martin Kleppmann
5 years
The talk I gave today is already online! See a live demo of using theorem-proving software to prove the correctness of a simple consensus algorithm.
@strangeloop_stl
Strange Loop Conf
5 years
Strange Loop 2019: "Correctness proofs of distributed systems with Isabelle" by Martin Kleppmann @martinkl
0
49
165
8
55
237
@martinkl
Martin Kleppmann
5 years
New article! “Local-first software: You own your data, in spite of the cloud” — a result of my long-standing collaboration with @inkandswitch .
6
103
237
@martinkl
Martin Kleppmann
4 years
New paper, which @pvh and I wrote together: “PushPin: Towards Production-Quality Peer-to-Peer Collaboration”. It's about our hard-won experience trying to build peer-to-peer local-first software using CRDTs. Very practical, lots of insights. Take a look!
2
55
227
@martinkl
Martin Kleppmann
3 years
My partner complained that she has to report her covid test result twice a week on a government website, requiring ~20 clicks plus login. So we wrote a Selenium script together to automate the whole thing, simulating clicks in a web browser. Much faster, much happier 😆
4
13
230
@martinkl
Martin Kleppmann
6 years
OMG, this is the highest rank I've seen for @intensivedata so far —  #559 across all books on 🎉
Tweet media one
20
13
223
@martinkl
Martin Kleppmann
6 years
What’s current best practice for GDPR compliance (in particular, right to deletion) in systems with append-only logs/event sourcing/bl*ckchains, which are supposed to keep history forever?
30
73
219
@martinkl
Martin Kleppmann
3 years
Fascinating Google paper on CPU cores that occasionally (and nondeterministically) compute the wrong result. Detecting and mitigating this is going to be an interesting challenge.
4
46
223
@martinkl
Martin Kleppmann
7 years
If not, fuck it, you have my permission to steal my book over BitTorrent. I care about sharing knowledge, not royalties.
6
77
221
@martinkl
Martin Kleppmann
5 years
I love this artwork so much (“Paradoxymoron“ by Patrick Hughes, found in the basement of the British Library in London)
3
38
218
@martinkl
Martin Kleppmann
5 years
Happy 50th birthday to the Internet today! On 29 October 1969, the first message was sent over the ARPANET
2
93
217
@martinkl
Martin Kleppmann
3 years
I think about this a lot. Lockdown is middle-class people staying at home, and poor people bringing them things.
@Firecrazee
Pol Wiseman
4 years
Tweet media one
9
111
440
6
33
208
@martinkl
Martin Kleppmann
4 years
ACM Europe calls for an EU-wide carbon tax, with no exemptions for computing, and strongly recommends against “any proof-of-work based distributed ledger technology” due to its “exorbitant power consumption”. Hooray! 🇪🇺🌍
4
73
205
@martinkl
Martin Kleppmann
4 years
Seph Gentle @josephgentle , author of ShareDB, explains how he came to realise that CRDTs are the future. And hardly anybody knows knows collaboration algorithms better than Seph!
4
58
204
@martinkl
Martin Kleppmann
6 years
Basically, I think owning Bitcoin is incompatible with having any sort of environmental conscience
48
49
203
@martinkl
Martin Kleppmann
2 years
I used to think the NATO target of spending 2% of GDP on defence was excessive, and that the US’s 3.7% was far too militaristic. But in the last week, any pacifist tendencies I once had have been shattered. Bring on those defence budgets.
21
20
201
@martinkl
Martin Kleppmann
3 years
Decentralised web technologies will need to figure out when, why, and how to ban abusive users. Without central control, how do systems determine what behaviour is acceptable? Some thoughts in my latest blog post
15
59
202
@martinkl
Martin Kleppmann
4 years
Great detailed exploration by @dave_universetf of how NAT traversal works — a.k.a. the joys of trying to make peer-to-peer software work (attn @pvh )
3
49
198
@martinkl
Martin Kleppmann
3 years
I have been informed that I can now have a second career as lookalike of Daði Freyr, Iceland's entry at Eurovision
Tweet media one
8
1
197