Yao Yue 岳峣 @thinkingfish Twitter profile

Pinned Tweet

Yao Yue 岳峣

2 years

My old friends, new friends, and strangers, I urge you to move on to the free Internet, and protect your emotional well-being from the capricious child king. Find my new home in my bio. Be brave and see you on the other side.

14

23

485

Last Seen Profiles

@gusano197

@JadSaalb

@FortuneTuneTuna

@55qpo_

@LisaSewell8354

@stats_feed

@Micheal2076

@ara_ara_arasic2

@hospivian

@prakopcyk

@bj2turntt

@7s8__

@BDollbrittney

@Europe1

@shioemon777

@SvarreAgger

@philip_iconn

@ouimoioipi

@KarenCarter1493

@stw_pdg

@qunweex

@xstelair

@FTwagiramungu

@esenyurtadeva

@verysanskarii

@OCFCBR

@DaveChappelle

@jandakembangstw

@rollingstanss

@Luci_ferruwu

@mybmc

@StMoguesGAA

@Elec_All

@PrimeTime_Kellz

@Dictator_Band

@TorioGK

Yao Yue 岳峣

@thinkingfish

2 years

After 12 amazing years and 3 weeks of chaos, I’m officially fired by Twitter. Never expected I would have stayed this long, and never expected I would be this relieved to be gone. I have a lot of stories to tell. But to my fellow (ex-)tweeps- #LoveWhereYouWorked 🫡

866

2K

26K

Yao Yue 岳峣

@thinkingfish

2 years

I’m so fucking proud of my coworkers. Like we said, “Once a tweep, always a tweep.”

36

90

2K

Yao Yue 岳峣

@thinkingfish

2 years

Unless you have a percentile, it’s just made up. Also note the mention of plural “core services” but singular “latency”… So tell me, which service is it? Have you managed to remember their names yet?

Elon Musk

@elonmusk

2 years

Twitter core services latency reduced by ~400ms. Should feel noticeably faster.

17K

18K

300K

14

62

1K

Yao Yue 岳峣

@thinkingfish

2 years

Imagine being so desperate for attention that you publicize your internal comms on the regular, including falsely claiming “all time higher” multiple times in a week, and dangling features tested years ago as your own with no context.

Elon Musk

@elonmusk

2 years

Slides from my Twitter company talk

17K

21K

232K

22

92

987

Yao Yue 岳峣

@thinkingfish

2 years

Don't resign, let him fire you. You gain literally nothing out of a resignation.

Zoë Schiffer

@ZoeSchiffer

2 years

Elon Musk just told Twitter staff: “If you can physically make it to an office and you don’t show up, resignation accepted."

699

3K

24K

19

88

952

Yao Yue 岳峣

@thinkingfish

2 years

@elonmusk The slow part (last hop) is a single API request plus CDN, and the backend in general is not the slow part regardless of batching. This pattern is well established with other similar services such as Facebook. DNS, time-to-first-byte, and payload size are the most important.

56

39

837

Yao Yue 岳峣

@thinkingfish

2 years

My most proud achievement as software engineer is open-sourcing Pelikan ( @pelikan_cache ), a modular distributed caching framework. See for more info, and DM me with any comment/questions.

17

69

794

Yao Yue 岳峣

@thinkingfish

2 years

I joined Twitter in 2010 when it was a legitimate startup of ~300 people instead of a pretend one. I was oncall for cache and messaging that went down constantly. But never did I fail to take at least 5 weeks PTO off per year, including one that was ~3wk, and still got shit done.

Zoë Schiffer

@ZoeSchiffer

2 years

NEW: According to Twitter’s Slack, the company is changing its PTO policy. “Twitter management has decided that two weeks PTO leave is not acceptable,” a manager said. “It is expected that most of us will work through December.” Noted there will be downtown around Xmas & NYE

219

490

4K

6

31

497

Yao Yue 岳峣

@thinkingfish

2 years

Within groups, selfish individuals win against altruists, but groups of altruists beat groups of selfish individuals. - Edward O. Wilson, Genesis, page 87

5

76

449

Yao Yue 岳峣

@thinkingfish

2 years

@kdspaul I have lots of time now, you name it.

1

447

Yao Yue 岳峣

@thinkingfish

2 years

@nabokov7 I’m so sorry. You are not alone and please organize with your local tweeps to seek Justice.

5

9

438

Yao Yue 岳峣

@thinkingfish

2 years

I came so far for beauty I left so much behind

8

18

425

Yao Yue 岳峣

@thinkingfish

4 years

@Susan_Hennessey Though dashed with bold lines, the outcome is balanced by the restraints shown in the choice of color(s), which stayed within the gray scale palette.

1

3

338

Yao Yue 岳峣

@thinkingfish

2 years

@nickrw Back at you, sir. I am proud to graduate in the same class.

0

1

361

Yao Yue 岳峣

@thinkingfish

2 years

@elonmusk @schrep “Server Control Team”? I’m only gone for a few hours and they’ve already spun up a team I have never heard of… 🤔

17

22

365

Yao Yue 岳峣

@thinkingfish

4 years

Went for the citizenship test today. And because it’s 2020, the ceremony has been reduced to a socially distanced, subjects-only, minimalist affair that happened the same day. So just like that, I’m now a US citizen.

42

0

352

Yao Yue 岳峣

@thinkingfish

3 years

Loaded.

13

2

322

Yao Yue 岳峣

@thinkingfish

2 years

“Being deprecated”, or nobody is left to maintain it? 🤔

Elon Musk

@elonmusk

2 years

Just leaving Twitter HQ code review

29K

33K

429K

10

17

295

Yao Yue 岳峣

@thinkingfish

1 year

@elonmusk I think you are mixing up “user not in timeline cache because they are not active” with “tweet not delivered”. For fanout service, the miss ratio has historically been 95% or higher due to the former. This means the fanout action has no effect, doesn’t mean service malfunction.

6

16

258

Yao Yue 岳峣

@thinkingfish

1 year

Cutting off Slack, Google chat at Twitter reminds me of what people observe about Russian army- when you have no trust and no respect for your folks in the trenches, communication beyond direct orders is seen as a threat. The consequence will be similar too- paralysis in crises.

3

8

259

Yao Yue 岳峣

@thinkingfish

4 years

@cbrewsayso Sorry to learn about you this way, my coworker and Mission neighbor. Rest In Peace. “Born to amuse, to inspire, to delight Here one day Gone one night Like a sunset Dying with the rising of the moon Gone too soon Gone too soon”

0

222

Yao Yue 岳峣

@thinkingfish

2 years

@1a1a11a @elonmusk Interesting. Unfortunately I think the team that was in charge of monitoring client side performance was entirely laid off.

8

12

231

Yao Yue 岳峣

@thinkingfish

2 years

It's a cruel world We've so much to lose And what we have to learn, we rarely choose

4

13

235

Yao Yue 岳峣

@thinkingfish

2 years

@TwitterSafety Elon, is that you? Stop borrowing twitter team accounts- your own account can have more than one poll per day, trust me.

2

7

226

Yao Yue 岳峣

@thinkingfish

2 years

@taylorleese A badge of honor at this point.

42

2

216

Yao Yue 岳峣

@thinkingfish

8 years

Meet Simon, born on Aug 7. He refuses to let his arms stay swaddled.

44

0

164

Yao Yue 岳峣

@thinkingfish

9 years

This tweet is served out of the systems I work on. #ILookLikeAnEngineer http://t.co/vPb1wRsBVo

3

42

146

Yao Yue 岳峣

@thinkingfish

2 years

@skr Lone Palm?

3

1

154

Yao Yue 岳峣

@thinkingfish

2 years

When will Twitter announce the only links allowed are other tweets? Close the loop, Elon, you know the control won't be total otherwise.

5

3

154

Yao Yue 岳峣

@thinkingfish

2 years

This is what working for Twitter really is like.

Mike Cvet

@mikecvet

2 years

I will happily do peer reference calls for anyone from Twitter who needs one, just DM me

13

6

380

4

2

152

Yao Yue 岳峣

@thinkingfish

7 years

A note from A Note on Distributed Computing

6

73

145

Yao Yue 岳峣

@thinkingfish

2 years

Must resist the temptation of calling out this “O” dude for a cache duel 🤪

Dan Luu

@danluu

2 years

It's a bit ironic that, with how Elon's run meetings, the people saying Twitter is easy because would've been fired if they were in a meeting and claimed, e.g., Twitter is easier than LessWrong and scaling it is a small matter of building one of the simplest caches in the world.

15

7

251

10

2

149

Yao Yue 岳峣

@thinkingfish

5 years

Martin, who was born on Feb 22, is about the chilliest baby I know. Thanks for making the 2nd time easy.

16

0

133

Yao Yue 岳峣

@thinkingfish

2 years

@elonmusk UAM slide uses data dated from 2020. I happen to remember a thing or two about that era, aka COVID-19 year zero.

2

5

120

Yao Yue 岳峣

@thinkingfish

4 years

Man next to me was a Belgian who has been in the US since 1980. Asked him “why now”? He said he wanted to vote in November. And that’s my plan, too.

1

2

118

Yao Yue 岳峣

@thinkingfish

8 years

A happy engineer has three projects: one that's (going to be) useful, one that improves things, one that requires imagination.

2

62

100

Yao Yue 岳峣

@thinkingfish

2 years

Some pointed out that there has been no promoted tweets in timelines and this is still the case for me.

11

2

105

Yao Yue 岳峣

@thinkingfish

6 years

I thought product teams have cool swags but it turns out I’m wrong. This is hands down the coolest thing I’ve ever received at work. What else can I do for data center engineering, @jenniferfraser 😁?

13

2

96

Yao Yue 岳峣

@thinkingfish

2 years

@elonmusk Don't lose heart, just keep turning them off one by one, you will find the one that paralyze the site / workforce eventually.

4

3

91

Yao Yue 岳峣

@thinkingfish

2 years

Oh look! Dan and I wrote a post (well, mostly Dan, I am too lazy) that summarizes half of my work and learning for the past decade.

Dan Luu

@danluu

2 years

A history of major cache incidents at Twitter

7

49

247

6

7

93

Yao Yue 岳峣

@thinkingfish

2 years

This is the reality. RSU conversion price puts employee comp above market. Having been on infra efficiency projects at Twitter for years, I want to add that no amount of infra spending cuts will cover the current net cash outflow. So FTE & contractor cost cutting is inevitable.

nickrw

@nickrw

2 years

Theory of what's going to happen next at [CHIRPBIRDICON] 1. Musk lays off very significant proportion of perm employees. We're too expensive to keep on, having RSUs that converted to vesting cash at $54.20 when the deal closed.

39

143

2K

2

4

84

Yao Yue 岳峣

@thinkingfish

2 years

Maybe I should just send them to read papers, such as this one about the hundreds of clusters we run for just one of the cache offerings at Twitter:

2

6

90

Yao Yue 岳峣

@thinkingfish

2 years

@svetlanathon Same

4

1

85

Yao Yue 岳峣

@thinkingfish

3 years

A few years ago an excellent programmer with a 4+ decade programming career gave a presentation at Twitter. One of us asked the question of how to become a better programmer. Her answer? “Be a better writer.”

Adam Grant

@AdamMGrant

3 years

Coding is more about communicating than computing. New data: the best predictor of how quickly people learned to code wasn't math or cognitive ability, but language aptitude. Math skill was almost irrelevant. Coding is mastering a language, not numbers.

139

1K

4K

0

20

82

Yao Yue 岳峣

@thinkingfish

8 years

At last, I don't have to worry about getting kicked out of the country after losing my job.

19

0

88

Yao Yue 岳峣

@thinkingfish

2 years

@amdev @igb @pelikan_cache It depends: - if you run it at home, it’s bare metal; - if you run it on someone else’s computer, it’s a microservice; - if I run it for you, it’s serverless.

2

3

83

Yao Yue 岳峣

@thinkingfish

2 years

@RedParchel Yup!

1

0

82

Yao Yue 岳峣

@thinkingfish

2 years

@alexstamos I’ll provide commentary on anything related to caching.

3

79

Yao Yue 岳峣

@thinkingfish

7 years

OMG this 8yo C++ programmer, Clara, just captured the entire room and won some hearts at #YOW17 with her live programming and demo on @jewelbots .

0

14

65

Yao Yue 岳峣

@thinkingfish

8 years

I've been working at Twitter for 6 whole years. Unbelievable.

1

0

67

Yao Yue 岳峣

@thinkingfish

2 years

@MishaVanMollusq Because he never ran a service in production?

1

65

Yao Yue 岳峣

@thinkingfish

2 years

@GergelyOrosz More than 10

2

3

69

Yao Yue 岳峣

@thinkingfish

4 years

Perf tip: first make it predictable, then make it fast.

4

3

63

Yao Yue 岳峣

@thinkingfish

4 years

I have a joke about cache but it’s a miss.

2

63

Yao Yue 岳峣

@thinkingfish

4 years

#IVoted for the first time today.

3

0

55

Yao Yue 岳峣

@thinkingfish

2 years

@colmmacc No on pubsub but that’s what I want to add to cache… I guess not at Twitter at this point 😛

3

0

56

Yao Yue 岳峣

@thinkingfish

4 years

What makes production software work well has something in common with public health. It does not ask for dramatic rescues or wickedly clever tricks, it demands the boring stuff- sticking to a few sound principles or policies, but follow them consistently, everywhere.

1

7

56

Yao Yue 岳峣

@thinkingfish

6 months

Algorithms + Data Structures = Programs Pascal was what made me really fall in love with programming. RIP Mr. Wirth.

Bertrand Meyer

@Bertrand_Meyer

6 months

We lost a titan of programming languages, programming methodology, software engineering and hardware design. Niklaus Wirth passed away on the first of January. We mourn a pioneer, colleague, mentor and friend.

152

2K

5K

5

16

60

Yao Yue 岳峣

@thinkingfish

9 months

Another typical day on Valencia St: within 30 seconds I first got chased out of the bike lane by a SFFD truck, then a car tried to make a mid-block U-turn right in front of @SFPDMission , almost driving into me, and I had to yell. Enough, @SFMTA_Muni @jeffreytumlin make it safe.

3

5

57

Yao Yue 岳峣

@thinkingfish

9 years

My Strangeloop talk on production-ready cache is now online.

Strange Loop Conf

@strangeloop_stl

9 years

"Cache à la carte: a framework for in-memory caching" by Yao Yue: http://t.co/bgJtdY8Zyu via @YouTube

0

7

31

3

13

49

Yao Yue 岳峣

@thinkingfish

7 years

Amazing math art pieces at @BridgesMathart 2017.

2

23

55

Yao Yue 岳峣

@thinkingfish

1 year

@ZoeSchiffer This is a strictly worse excuse than “88188 is perfect by being a palindrome number and we should preserve it.”

1

0

56

Yao Yue 岳峣

@thinkingfish

8 years

More than 5 years later I'm still @Twitter . Because I: - do the kind of work I'm proud of; - work with people I like and respect. #oneteam

1

52

Yao Yue 岳峣

@thinkingfish

2 years

Elon clearly wants to have fun at Twitter’s expense, the least I could do is not paying for his entertainment with my mental well-being.

0

1

50

Yao Yue 岳峣

@thinkingfish

2 years

@Hornet_blockIII Some core services have such high fanout per typical external API request (eg home timeline) that you most certainly care about latencies at ~p999 or further into the tail instead of p90.

2

0

48

Yao Yue 岳峣

@thinkingfish

2 years

@BeijingPalmer A society dominated by propaganda (and often as a side effect of that, hearsay) means most people simply assume the media or any public statement is a performance- insincere and untrustworthy, regardless of whether they are for or against the message. This habit sticks with them.

3

8

46

Yao Yue 岳峣

@thinkingfish

7 years

An avalanche lily. Summer at Mount Rainier.

5

2

49

Yao Yue 岳峣

@thinkingfish

4 years

So burned out by management that fixing CMake file feels "fun".

2

1

47

Yao Yue 岳峣

@thinkingfish

2 years

Today, Tuesday 22-2-22, is Martin’s birthday (he’s turning 3). This is the birthday card he picked for himself.

3

0

47

Yao Yue 岳峣

@thinkingfish

6 years

Simon hasn’t learned to pronounce g or k so he liberally replaces them with other consonants. “Simon, how do you say Google and Facebook?” “Doodoo and Facebutt.”

2

0

45

Yao Yue 岳峣

@thinkingfish

2 years

Thank you, Ernie, for teaching me so much through life and beyond. You were one of my more important life decisions. I will miss you. 11/11

12

0

43

Yao Yue 岳峣

@thinkingfish

2 years

Yesterday @ernierubs was put to sleep forever. 1/n

Ernie the dog

@ernierubs

2 years

Goodbye.

1

0

5

8

0

42

Yao Yue 岳峣

@thinkingfish

3 years

5yo: “When I’m a grownup I can work for you.” Me: “Oh?” 5yo: “Yea. I can work for you and dad. You’ll be so old, I can help with groceries…” Me: “Awww, that’s very nice.” 5yo: “If I will help you then, what can you do for me *now*? Can I watch <anime show name>?” Me: 🤣😂😆

1

0

45

Yao Yue 岳峣

@thinkingfish

2 years

@vkostyukov @pelikan_cache Batching is too 1998. In the new world, pipelining > batching.

2

39

Yao Yue 岳峣

@thinkingfish

2 years

@marius Sure but is shitposting still included in benefits?

2

1

43

Yao Yue 岳峣

@thinkingfish

3 years

@danluu Even 10 years ago I routinely went to kernel engineers to understand why the hell I was seeing what I saw on distributed cache, and without them I would have spent 10x more time debugging some issues or failed to root cause about half of them at all.

1

43

Yao Yue 岳峣

@thinkingfish

7 years

Simon, eating his speaker badge to show how proud he is of his mommy.

2

0

39

Yao Yue 岳峣

@thinkingfish

7 years

This little monkey turns one today.

6

0

43

Yao Yue 岳峣

@thinkingfish

2 years

@safestreetrebel I biked right through this evening. Thanks, you are awesome!

0

1

35

Yao Yue 岳峣

@thinkingfish

2 years

Saturday going swimmingly- the 6 yo dumped a Costco sized box of Cheerios on the kitchen floor, and he instructed his 3.5 yo brother to eat cheerios off the floor as quickly as possible to lighten the cleanup burden. Needless to say, I’m staying the hell out of the kitchen.

3

1

39

Yao Yue 岳峣

@thinkingfish

7 years

OH: “We need to hire more ‘adults’, but the problem is they are expensive.”

0

7

40

Yao Yue 岳峣

@thinkingfish

5 years

After spending many weeks and relied on tedious tricks to uncover sub-secondly performance woes, to see them in unmistakable charts in realtime with Rezolus is incredibly empowering & satisfying. Very proud of supporting @brayniac ’s cutting edge work. (Also, eBPF! @rustlang !)

3

11

38

Yao Yue 岳峣

@thinkingfish

8 years

It took longer than I expected, but from here we voyage toward open water!

Pelikan Cache

@pelikan_cache

8 years

Twitter Cache team just open-sourced Pelikan, a cache framework. More details on project website:

0

100

148

2

7

39

Yao Yue 岳峣

@thinkingfish

2 years

Strategy comes and goes, plumbing is forever.

2

6

39

Yao Yue 岳峣

@thinkingfish

7 years

Chocolate stout cake. Almost too perfect for glaze, almost.

3

0

39

Yao Yue 岳峣

@thinkingfish

8 months

Yesterday a car drove across the bike lane mid block between 15 & 16, blocking my way. At some point it even started backing into me, so I shouted at the driver, “what are you doing?” She relied, “but this is convenient for ME!” I guess this video will change her mind🙄

SFMTA

@SFMTA_Muni

8 months

There's a new video to help people navigate the pilot design on Valencia Street: the city's 1st center-running protected bikeway, a new curb management plan & pedestrian safety upgrades. Get project updates at & find the video here:

56

10

5

3

1

39

Yao Yue 岳峣

@thinkingfish

3 years

18 months after introducing my now 5 yo to Legos, my role has been officially reduced to “mom put this sticker on” in assembling the 4K+ piece technic set.

3

0

37

Yao Yue 岳峣

@thinkingfish

7 years

Parental leave at its best.

1

0

36

Yao Yue 岳峣

@thinkingfish

11 months

@jpd Congratulations! You were always the one who kept the site together, took care of the team, while treating everybody coming to TCC (which was like half the company!) with respect. I do hope the next place that’s lucky enough to have you won’t ask as much from you as Twitter did.

0

37

Yao Yue 岳峣

@thinkingfish

2 years

@lmanul You missed the part where great engineers who started out as Elon fans retreating from that section of the Venn diagram after applying some reasoning against the recent events.

0

1

35

Yao Yue 岳峣

@thinkingfish

2 years

@GergelyOrosz One comment: I would assume Twitter 2.0 will offer no PBP, which was ~25% for Staff and 30% for Sr Staff assuming revenue was on target. Considering this, the cash portion of the comp is not really higher than before.

0

1

34

Yao Yue 岳峣

@thinkingfish

4 years

Redis stands as a visionary product, more than a decade later. It must be bittersweet to let go of the control of such an important project. Cheers to life’s new possibilities.

antirez

@antirez

4 years

Today I'm stepping down as Redis leader and maintainer. I published a blog post about this: Thanks Redis community, you have my love <3 Thanks Redis Labs for the joint adventure so far.

251

763

3K

1

2

35

Yao Yue 岳峣

@thinkingfish

2 years

@alexstamos “It looks like they just restarted all shards in quick succession resulting in a completely cold cache- this is going to get interesting fast… Wait, is that thundering herd that I am hearing? Yep, there it is.”

0

2

36

Yao Yue 岳峣

@thinkingfish

2 years

@kdspaul @elonmusk This is what happens when you let the people you meet decide the content you focus on, instead of deciding what the focus should be and then seeking the people you need for that.

1

0

37

Yao Yue 岳峣

@thinkingfish

2 years

I helped an elderly lady on a wheelchair use the bathroom today. It took us an hour and I almost broke a sweat, zero chance she could’ve made that trip herself. The infrastructure in SF, especially for disabled people on cheap wheelchairs, is incredibly bad.

1

0

36

Yao Yue 岳峣

@thinkingfish

2 years

Ritual Coffee on Valencia St is done with renovation and open for indoor dining again after two years. I’m happy to have little nice things anchor me in life.

1

0

36

Yao Yue 岳峣

@thinkingfish

7 years

Growing up in swampland taught me how to handle heat: 1. nap in the hottest hours; 2. chilled watermelon 🍉 when you wake up

0

34

Yao Yue 岳峣

@thinkingfish

3 years

@PoliticusSarah @AdamKinzinger You cannot shake a man who knows he has done the right thing and has inner peace. He sleeps better than those who are bound to each other by fear.

0

2

28

Yao Yue 岳峣

@thinkingfish

6 years

Look what I found- it's old Larry!

0

34