A very interesting article of how someone fixed GTA V loading times by fixing their JSON deserializer. He improved speeds by 70%, and got awarded 10k from Rockstar, and Rockstar actually implemented his fix in an update! Cool story :)
Finished version 1.0 of my Low latency UDP client made in DPDK. got nice improvements in the ping pong messages :) around 50 micro lower median & means. orange line is python udp client, blue is dpdk udp client
Hey everyone! I gained a lot of followers in the past days. If you're new, im a guy trying to write his own networking protocol with 'DPDK' in c++ to lower the latencies! I hope to share more about this in the coming weeks, and keep you updated on the journey. Thank you all❤️
Hi everyone sorry for inactivity will be fully back next friday due to finals week! Learning for finals really bored me so much these past days. Today I thought lets upgrade the ol' deserializer with Bybit Orderbook stream:
Okay got some free time and decided to test out
@levbeta
's deserialization bench. I did not expect my deser algorithm to be this fast. Try to beat it! let's start a deserialization arms race!🔥
Wow. It took sony 2 years to bring the full functionalities of Discord to PS5, and I almost finished my DPDK websocket after only 3 months.
@Sony
pay me 10 mill and an astrobot figure and u can use my udp client for lower latency discord voice chat.
Go follow this dude! He is really amazing. He's honest about the bad times, and good times he has trading, and I always really enjoy this guy's content.
Here's my updated P&L since last year, trading a low-frequency, long only trend model in crypto.
I've been 82 days in drawdown so far. Pretty much ever since the top on the 14th of March.
Just the name of the game, carry on.
(apologies for the y axis, net p&l is 139.58%)
If it's one thing that helped me in my ongoing journey in HFT, it's communication. A lot of these tasks in crypto may very well be infeasible to achieve by yourself. Find out your strengths and weaknesses, and find someone that complements you in the areas where you lack
Working on cutting down deserialization times, had a vague idea, turns out it worked pretty well :) If this is a bottleneck for you take some time to really think about it, you'll probably cut down your time atmost in a days time by 30%
How to reduce the latency of x? Just go through the code of x and make it do less things! So don't shy away from trying to peek under the hood of the functions and etc that you are using. See how they work, and see what can be skipped.
Many have asked me to share my alpha. Today I deliver:
@ekrii3
's resources were pivotal in cutting down my deserialization time.
You can never beat serde without having some knowledge of circle packing. Here are the links to his amazing resources.
Feeling super generous so, somewhat reluctantly, here are 5 industry secret holy grail books that have immensely assisted me in my quantitative trading endeavours (as an ex manual HFT mm) that I highly recommend new and experienced traders pick up if they can. No specific order.
Yaaay! Fixed my code im about 2 times faster using DPDK and look at the jitter. 🥰 Next step will be tuning the kernel using optimal settings I found using the repo my bro
@Dub0x3A
made with a little help by my side ()
Hey guys could use some advice for a new pc build. My laptop is on its last legs. Now my externally connected mouse is also barely working, and the system has trouble shutting down now. I don't have a lot of money so I tried to compose this build for hft. Please share ur opinions
Do you know how broken my setup is? I'm working with a laptop that has a malfunctioning keyboard—some keys, like 'E,' simply don't respond. (Guess which letter is the most frequent in the English Language). (1/7)
Thanks everyone for your help! I found an easier and faster method. I pasted some websocket code in the dev console of google chrome, and followed this tutorial: and now I can see all the packets decrypted in wireshark :)
How can I decrypt the TLS packets that are received by e.g. a rust websocket? I just want to see the data that is received using wireshark but cant figure out how to let wireshark decrypt it
Would you guys be interested in a substack where I could actually go much more into detail about DPDK, deserialization and etc? These subjects need more than 140 characters :)
Do you know how broken my setup is? I'm working with a laptop that has a malfunctioning keyboard—some keys, like 'E,' simply don't respond. (Guess which letter is the most frequent in the English Language). (1/7)
Working on cutting down deserialization times, had a vague idea, turns out it worked pretty well :) If this is a bottleneck for you take some time to really think about it, you'll probably cut down your time atmost in a days time by 30%
If I had access to the AWS datacenter I would purposely heat the cables of my competitors so that the latency would overwhelm them. Maybe its a good thing AWS keeps those locations a secret 😂
This is for getting top 5 bids and asks and parsing everything to f32. Damn amazing! To think that serde takes 100 micros, and this specialized deserializer 200 ns!
You take longer to complete your studies because you need to retake a course. I take longer to complete my studies to extend my loan period. We are not the same.
This week, I had to learn 4 weeks of coursework in one day. Whenever I had trouble understanding one of the slides. I screenshotted it, sent to chatgpt and it explained e.g that the pressure force is always directed inwards to the surfaces of a control volume. It saved my life
Next steps will be to finish TCP handshake and implement TLS so that I can finally make a low latency binance socket and couple that with my deserialization algorithm to obtain extremely fast t2t :)
@BeatzXBT
@volatilitynoob
They do still work, but the infra/profit ratio for it are insane. Wouldn't recommend, you can get more profit with one tenth of the energy by looking at other strats
If you like math dont search up Fast Matrix Multiplication Tensors. Its an incredible problem. It is extremely easy to understand, solving it would reap big rewards (increase the speed of matrix multiplication), but solving it is horrendous. The solution space is enormous.
What theory do they think board game developers use? Its group theory. Try to solve a rubics cube without knowing how to define the group homomorpishm for it; You can't! Market making is simply finding a bijection between bids,asks and the tiles of a rubics cube. Ask
@BeatzXBT
I think after finishing my low latency websockets Im gonna finally return to running strats again and making p&l, since the next improvements to lower latency cost considerable money (e.g. renting metal instances)
@circlepacker
@BeatzXBT
@volatilitynoob
I tried to tri arb some shitcoin with a market cap of 70k. Even with kernel tuning, and tuning the network stack I only won 40% of the tri arb oppertunities. And each winning trade netted a profit of 1 cent💀 so yeah a lot of work for nothing
Thanks for the 150 followers. It's a month since I made this account and wouldn't have predicted this much people following me already. Thank you all for the support :)
@nayshins
@shakoistsLog
yeah man same was true for me aswell. I also noticed that when I tutored kids their learning attitude would improve when i told them of the practical applications of whatever theyre learning. it makes the learning feel important rather than a useless chore
How can I decrypt the TLS packets that are received by e.g. a rust websocket? I just want to see the data that is received using wireshark but cant figure out how to let wireshark decrypt it
It takes 50% more time than my binance implementation, but I believe I'm able to get the median and p99,p99.9 down. I made this adaptation of my deserializer in an hour, and was frankly surprised it worked this efficient without giving further thoughts about optimization
“Let your eyes look straight ahead; fix your gaze directly before you. Give careful thought to the paths for your feet and be steadfast in all your ways” Proverbs 4:25-26
@Dub0x3A
@OBS_orderflow
@BeatzXBT
This. Take a much needed break if possible, or find another thing u can change about ur daily life. Go to a different grocery store, take a diff route home, tell a stranger you like their shoes whatever gets you out of the endless routine even for a split second
There's always something holding you back, right? Do I enjoy working under these conditions? Definitely not. But hey, I'm a student—I don't have the funds to fix these issues, and that's okay! In the grand scheme of things, (4/7)
I've been looking for good quantitative trading engineer accounts to follow.
Mainly interested in design and development in Rust/C++, but Python is fine too.
Any tips?
@soundsonacid
@levbeta
@0xpemulis
Just choose what is best for the situation. People that think no other programming languages than Rust are allowed, have probs never went to uni lol. Im not gonna make my own plotting software just to plot some current measurements.
@MarkBestForex
What?? is it actually about using quantum mechanics for finance? As a physicist i have seen this a couple of times, and each time i lose braincells😭
and trust that your perseverance will lead you to brighter days ahead. Your journey may have its bumps and glitches, but with your determination and resilience, you're sure to emerge stronger and more capable than ever before. Keep believing in yourself, (6/7)
@serzhiio
@Dub0x3A
Yeah and adding 5 levels of orderbook levels also doesnt seem to have a significant effect on means or medians. It does worsen 99.9th percentile, but 99th is fine. The whole experience is extremely humbling regarding my programming intuition. Will share later :)
Yesterday, I've had to resort to digging through old Rust folders and deleting '/target' to free up space. To add to the chaos, my laptop charger sparks every time I plug it into the wall. Oh, and let's not forget to mention that my mousepad on my laptop doesn't work. (3/7)
Thanks for the 150 followers. It's a month since I made this account and wouldn't have predicted this much people following me already. Thank you all for the support :)
It was originally a Windows laptop for my engineering studies (good luck running obscure electrical circuit simulators on Linux or Mac!). I installed Ubuntu alongside it to learn programming . With only 80 GB of storage, my laptop's capacity is now nearly maxed out. (2/7)
@Vertox_DF
Point 3 is very true. I plan to shed some more light about this when the websocket is done, which hopefully will be this week. Nice thread! Was enjoyable to read :)
@quant_arb
Real. I have to be very abusive at the start of each conversation, just so im not ddos'ed with tons of useless info. I asked you for a simple DNS script in python, not a fucking thesis
@ShivamKuma1642
Ah Okay, sometimes My English is a bit bad! Was just checking! Yeah might be because of the different rules in our countries. In my country you can do this without having too much debt added (live in EU). dont think it would be safe to try it out in the US
@MarkBestForex
@levbeta
@Dub0x3A
I've taking a small break for till next week friday for my finals week and i see you guys have gone into madness with your picoseconds💀💀
@laurentzeimes
HTTP 3.0, but the choice of measuring the RTT for UDP packets was a practical one. UDP does not have a handshake protocol, which made it easier to test than TCP
@cryptohound666
😂😂 Yeah, im gonna resume the DPDK HTTP client again. Other project ideas for the coming months are: to make my own websocket application made for low latency (to see how fast we can be without kernel bypass), and to combine my deserializer with DPDK, and see what magic happens