There are rusty ways to write code and if you come from languages like python or even c++ it can be quite different. These are a few things that could help people getting into it and its slanted towards
#hft
/
#hpc
.
@PaulSkallas
This is a big deal if you're into astronomy. Orange light is easy to filter. LEDs are basically impossible. So if you live in a modern city, star gazing sucks.
Recent discussions about tick to trade made me do a more in depth analysis of the practical problems. Data is really bursty and the problem is actually worse than i thought. Thanks
@BruHFT_quant
for the inspiration.
Do you need more rust in your life?
Do you need to shave nano seconds off your hot path logging?
Were you dumb enough to copy paste my old logging code?
In that case this new article maybe this is of interest.
Just finished this. A lot of useful ideas for parameter optimisation of hard to simulate systems (see mm). Author is ex getco. Its a pretty easy read and the stats are not too spicy.
I finally got to deep dive Py03 and use it for something real. I have to say, I am really impressed with it. I can now process billions of ticks without ever needing to leave python. I really wish I'd looked into it sooner.
@__paleologo
A lot of traders I know aren't math people. It's in fact pretty much the opposite. I'd say their main skills are sniffing out sentiment and knowing which markets to be in when.
If you have a trading strategy with a 20% chance of being profitable. If you keep running it for 6 months there's a 120% chance you'll make money. It really is that simple.
@patio11
You miss the game theory element. Not answering the margin calls is a red flag that led to them dumping. That's the genius in this.
Some of the senior bond traders i kmow would just have a sixth sense. They'd just know something was wrong and dump positions at great expense.
@TimMeggs
I heard one of the bond quants at DB did this. Not in the middle of the day but just disappeared and never came to work again. No one knew what happened to him and it was the days before social media.
@Dub0x3A
You can test its ability to predict forward returns.
So what has lower error price-fair or future price- price.
There are also causality models I.e granger causality.
You can also test for the proces mean reversion to your fair price.
When you start using a new library its good to write a bunch of test cases around it. This has two advantages:
1. You can check it works the way you think it does
2. You can make sure this is true over time as version change.
And yes, i just got burned by not doing this.
Remember to not abuse definitions.
This formula is called Jensen's alpha. It's a simple way to test if beta is > 1 and if alpha > 0. People should be quoting their confidence intervals on alpha/beta if they want you to take them seriously.
It is AMAZING the latitude of opinions within finwit about the meaning of alpha and beta. It's mostly a matter of definitions, and i don't aspire to be a beta lawyer, but you people should have a giant rave party, hug each other, and come up with a common definition.
@emil_priver
If you don't think history matters, you've never had to backtrack a release or you're working at such a slow pace and releasing a couple of features at a time. Git is not that complicated and if you can't keep a clean history it only says that you've never worked professionally.
How do you control for idio syncratic risk in a portfolio? Do you use regularisation or do you estimate it and put caps inside the portfolio construction step? There's just always risk in crypto that one coin goes bananas so I'm wondering how you manage that?
@DegenSugarBoo
@BruHFT_quant
@Dub0x3A
@BeatzXBT
@nayshins
Where you need to fix yech debt is
1. you will make the system much better from a business stand point.
2. The tech debt is causing other dev to be done in a suboptimal way.
3. It caused preventable prod issues.
There's a good spolsky blog why rewiting code is bad.
Dialect in europe is crazy. I can meet people from new zealand/india/kenya etc and have no problem talking to them. In Europe, you can travel 50km and have no idea what anyone is saying. Language seems to be a continuous thing with dialects bridging the gap between countries.
@BeatzXBT
@SmallCapSmarts
@Dub0x3A
Its interesting that not a lot of c++ hft stuff gets posted. Probably because it's all in funds and people tend not to share it. That or its absolutely horrendous and its hard to explain a 1000 lines of asm and avx512 riddled code.
I ordered a book from the US and its gone missing while in customs. I rarely order anything from outside the EU because its such a pain. The issues is a lot of stuff isn't made here. I don't know if this is just govt inefficiency or done on purpose to promote buying local.
@macrocephalopod
This is what your pnl would look like if your account exclusively posts click bait. I asked a while ago about something else and was instructed that i just didn't know what LSTM was. 10/10 no notes.
@fengtality
It's one of the few metrics that isn't path dependent. Its not a great proxy to pnl since the time to hedge is also a function of toxicity.
Its one of those "all models are wrong, some models are useful" things.
@experquisite
@ByrneHobart
A huge amount of hft is working out if the conditional return distribution is the same as the signal return distribution. For some signals you can see their good because of the alpha decay as a function of latency.
ChatGtp et Al is really going to change education. I've been using it to learn German. The ability to ask specific questions is super helpful in working out the grammatical nuances. I hope this helps a lot of kids that don't like a subject because their teacher sucks.
@miniapeur
I started to study cosmology and astro physics. I have a math background so figured it'd be not too difficult to grasp the basics. I eventually gave up. I get the feeling many of the theories seems to be more story telling than science. Dark matter is just Texas sharpshooting.
The thing that is missed from this is the detrimental effect on compounding. Starting with a $10k portfolio, over the last 40 years of snp returns it would end with $327k. With 10% unrealised cap gains the final balance is 209k, and 20% unrealised cap gains would make it 133k.
@phoenixstealthy
@BeatzXBT
The only tricky thing with go is the gc. The production of garbage is correlated to the amount of work you're doing. That in turn is correlated to if the gc will run. I'm generally not a fan of gc based languages for hft.
Given previous discussions about sharpe ratio vs sortino, has anyone used the ulcer index?
I was reading and although I've seen the UI before in quantstats etc ive not dug into it. It seems to improve on the Calmar and solve pnl path related issues.
@Dub0x3A
How much data is this per day. I.e how many ticks? It must be insane. I am assuming this is not for eth or btc?
Also how are you making sure the timestamps on recorded data is aligned?
@MyLatinLife
then you go to a coffee shop and no one can understand you at all. Reading Portuguese isn't the hard part, speak it is. The tonics, man, that is just raw pain.
@BeatzXBT
I'd actually make a counter point to this as if you get a burst of messages on the websocket, if your processing time is about 1ms, you can get quiet behind. Cpp/rust ipc is between 100 and 500 ns to put it into comparison. At least then the qs don't get backed up.
@Dub0x3A
And the last issue is if your fair price I'd even tradeable. If you're using certain features you'll get a great prediction but you'll never be able to execute it.
@_kacper____
@BeatzXBT
@SmallCapSmarts
@Dub0x3A
The Art of Writing Efficient Programs: An advanced programmer's guide to efficient hardware utilization and compiler optimizations using C++ examples
This is a good general book that's got a lot about valgrind etc.
@DegenSugarBoo
@BruHFT_quant
@Dub0x3A
@BeatzXBT
@nayshins
So you need to know what is the cost of leaving it there. What is the benefit of changing it. How much work is it to fix and also important is how likely are you to break other things fixing it.
@PtrPomorski
This repo sucks. A good repo would be 70% data prep, 20% about oos robustness and the rest just linear regression. You don't need an CNN if your input data is garbage.
One reason why English people can't handle 26 degrees is no one opens the windows. In Germany it's often in the mid 30s but you'll flush the house in the morning and then keep the shutters down. It's a very effective strategy for managing the heat. They also have better windows.
@PtrPomorski
Not everyone doesn't. I've run stuff that optimised the calmar. It has some practical problems in that it tends to prefer low trade counts which have a high calmar due to luck. So you need to correct for that.
@BeatzXBT
not all LOC are equal. When its green field you can get a lot done, if you are fixing someone else code or even your own it might take 2 days to make a 2 line fix.
LOC ~= Productivity.
@tsoding
A guy in marketing using chatgpt built and fit some survival models without any deep knowledge of stats and econometrics. It was using Cox models and others. He managed to find a few really good insights from the data . This changed my opinion of what possible with LLMs.
@Vertox_DF
I love that no one thought to ask you this.
With amd you get more cores and it really helps for thing like compile times and also runing parrallel simulations. I have 3950x but the current one is ryzen 9 5950x. I would also suggest getting a large slow HDD for market data.
@andrew_lfc18
They are really not great as an online filter fyi. They're useful for calibration of online models by fitting deviation expectations. They are just a center smoother and when used online it has to guess half the data which ruins any abilty to predict.
My friends a Web dev/designer and looking for an internship or a placement. He used to do a ton of 3d modelling and has a really nice eye for design.
If you can share this around/help I'd much appreciate it.
@quantymacro
There is a bit of a chicken and egg problem. You can't get hired without a sharpe of x, it's really hard to learn to trade yourself. The main thing seems to be there are not that many seats at top firms and they can pick and choose.
@DegenSugarBoo
@BruHFT_quant
@Dub0x3A
@BeatzXBT
@nayshins
I'd say it will always be a judgement call. There are some tech debt that really annoy me but I've not fixed. An example is for time we use u64 and not an i64. So we have to be careful to not get issues computing time deltas. Simpler things were fixed as they caused prod issues
@BeatzXBT
Once you start using signals across instruments/exchanges it becomes a lot of data very quickly.
I know your feeling about python. I've written hft systems in python so know the limitations. I'm not sure how much you can offset with numba/ cython.
@RGBCubed
Python scoping is an absolute dumpster fire.
Its super easy to introduce bugs due to it and it also means refactoring tools suck since its difficult to work out variable scope.
@BruHFT_quant
@Dub0x3A
You might want to look more at your latency tracing as it should be able to detect this. Dm if it's actually a concern. I know you're mainly focuses on multi asset stink bidding atm.
@DegenQuant
@levbeta
I was doing a bit more research into this and i think criterion benches give much better numbers than its possible to get live.
I'm assuming it's just because the cache is super hot with using the same message. In reality I get super different numbers than the bench gives.
@Dub0x3A
With all of this you have to be careful. Most stat tests are L2 and use some square loss function. In trading you have an L1 loss. So you can also just run a backtest and compute its ability to predict that way.
@DegenQuant
@levbeta
Its also cope since i dont want to rework my deserialisers at the moment and im upset you guys are getting these numbers! Its interesting to see that you can get a lot more out of serde with a little tuning.
@arnaud710
@0xMagnus_
@ScottPh77711570
At this point it is just a bit sad. This is not a legitimate response to someone asking questions. If you ever want to work as a quant/trader, having a job will be 10x more brutal than this.
@KlondikeFX
@Dub0x3A
The problem with the stat tests is they're not conditional on fills. If you were to run a take strategy then you'd only trade the distribution tails and your loss function would zero everything else. So its best imo to think how to trade and walk backwards.
@PtrPomorski
I didn't really want to write that. It's just that bad. The other repo you recently added I thought was good.
I mainly took issue at this since a lot of courses & books focus too much on models. Then newbies think, "if I just use an LSTM model it's easy money."