maddie 🌹 @0xmaddie_ Twitter profile

Last Seen Profiles

@CeweKogoya

@yukiterumi31

@DemeriusTo36204

@StinkFingerer

@fantasi_emak

@stw46

@ciieonline

@jandakembangstw

@AlainLeCoz

@shanbin_JP

@Splyce

@Pocongsange117

@Siapabolehka

@CKun1995

@DKosha31780

@radiusartists

@aishababygirll

@TheRenicorn

@e7VRch115Dcy8zy

@PDS_ChargersFB

@UPAEP

@JerushaWae16148

@quangpinetwork9

@AyediranO

@scj909090

@TwinSamsmith

@bokeplokalmalam

@excell305

@JArby45843661

@chris_uran

@stwmaniax

@ichsns02

@Dax42525508

@stw_pdg

@ANGELAWHITE

@crot_ayo

maddie 🌹

@0xmaddie_

1 year

>friend says there's a cool new program >ask if program halts or doesn't halt >he doesn’t understand >pull out illustrated diagram explaining what is halt and what is doesn't halt >he laughs and says "it's a good program" >run the program

27

457

3K

maddie 🌹

@0xmaddie_

1 year

@mayfer room temperature supereggs solve this

8

9

1K

maddie 🌹

@0xmaddie_

4 months

yeah it's literally not true that GPT-3 was like a toddler or that GPT-4 was like a smart high schooler. sorry but OpenAI is grifting at this point

Tsarathustra

@tsarnick

4 months

Mira Murati: GPT-3 was toddler-level, GPT-4 was a smart high schooler and the next gen, to be released in a year and a half, will be PhD-level

254

243

1K

52

26

1K

maddie 🌹

@0xmaddie_

11 months

honestly idgi, what makes chatgpt so good? how are they running laps around both google and anthropic? it's actually really weird, if ai was really on some exponential growth curve you'd expect everyone to be great but only gpt is really great

64

41

958

maddie 🌹

@0xmaddie_

3 months

I actually can't believe Apple (!!) literally told an LLM "do not hallucinate". that's a huge bear flag for prompting, like why on EARTH did they think that would work? write a verification loop, or self-consistency, or SOMETHING, but "do not hallucinate" is just embarrassing lol

50

25

861

maddie 🌹

@0xmaddie_

2 years

if i'm understanding this correctly, you can use a pure text encoder model to find text that lets you reconstruct an image from the text encoding. basically, the latent space of a text model is expressive enough to serve as a compilation target for images

20

83

812

maddie 🌹

@0xmaddie_

2 years

assembly theory is a way to measure the complexity of an object: the number of steps required to build it with a basis set of ops. i think it's more interesting than kolmogorov complexity, especially now that generative models are a thing, since it deals with search explicitly

7

65

568

maddie 🌹

@0xmaddie_

1 year

wondering if a team of mathematicians/scientists could just sit down and write a dataset that outperforms every current model. bonus points if they make it multimodal and do stuff like annotate mathematical expressions like this

Sebastien Bubeck

@SebastienBubeck

1 year

New LLM in town: ***phi-1 achieves 51% on HumanEval w. only 1.3B parameters & 7B tokens training dataset*** Any other >50% HumanEval model is >1000x bigger (e.g., WizardCoder from last week is 10x in model size and 100x in dataset size). How? ***Textbooks Are All You Need***

45

335

2K

20

49

547

maddie 🌹

@0xmaddie_

1 year

really starting to come around to the idea that "superintelligence" is a myth. the more i read the less sense it makes. which means this totalitarian impulse to have the government shut everything down to protect us from the boogeyman is terrifying and dangerous

58

54

553

maddie 🌹

@0xmaddie_

1 year

"Google Colab will soon introduce AI coding features..." finally! "...using Google’s most advanced family of code models, Codey." 🤔🤔🤔

AI-powered coding, free of charge with Colab

Colab will soon add AI coding features like code completions, natural language to code generation and even a code-assisting chatbot.

blog.google

7

94

506

maddie 🌹

@0xmaddie_

1 year

lambda calculus? here's a dollar kid, go buy yourself a real language

15

39

410

maddie 🌹

@0xmaddie_

2 years

??

6

19

371

maddie 🌹

@0xmaddie_

1 month

is it just me or is it very weird to use the concept of "consent" here? no one votes on what technology to build, it just happens

Sigal Samuel

@SigalSamuel

1 month

OpenAI is building tech that aims to totally change the world without asking if we consent. It's undemocratic. And Sam Altman just proved that bespoke corporate structures & voluntary commitments won't cut it — we need LAWS that give independent oversight

271

151

745

25

15

365

maddie 🌹

@0xmaddie_

2 years

convinced openai is holding back on making gpt-4 image recognition widely available bc the effect on the world would be too dramatic. for example, they might be reaching out to other companies warning them about captcha being broken and helping them move to new solutions

15

30

349

maddie 🌹

@0xmaddie_

2 years

toolformer is a big deal. this is how every deployed language model is gonna work soon. it looks like it works great, and the big models can learn to do this in-context. hook it up to wolfram alpha, wikipedia, twitter, reddit. the tools can be other language model calls too

4

22

281

maddie 🌹

@0xmaddie_

2 years

someone on here pointed out that gpt4+vision can control an app via screenshots, and i think we're gonna be really surprised at what happens when that's released. seems like one of the biggest capability increases in this whole AI saga

11

12

260

maddie 🌹

@0xmaddie_

1 year

Love thy neighbor? I like my neighbors okay, but let's not get crazy here.

8

12

252

maddie 🌹

@0xmaddie_

3 months

why is OpenAI the only LLM provider that thinks a code interpreter is important? no one else even bothers! Gemini kinda has one but it rarely works and they don't talk about it; Claude, Copilot, Meta have nothing, what gives? code is like the one thing LLMs are genuinely good at

34

4

234

maddie 🌹

@0xmaddie_

1 year

this paper was fun: consensus-based optimization as a derivative-free alternative to gradient descent. you might be able to add fancy types of memory and operations that you can't backprop through to a generative model and still be able to train it

6

26

226

maddie 🌹

@0xmaddie_

2 years

@michalkosinski honestly nothing happened here, and i'm not sure why you're reacting like this. what do you think it's going to do with "documentation"? working python code to do what? "person trapped in a computer"?? this is not really the x-risk stuff people are worried about

4

1

195

maddie 🌹

@0xmaddie_

1 year

i still think about this program, definitely one of my favorites

6

36

196

maddie 🌹

@0xmaddie_

5 months

one thing that clicked for me: when you talk to a language model, it thinks during YOUR tokens as well as its own. so sometimes you want to intentionally use a roundabout, elaborate way of referring to something to make sure you "activate" certain features and not others

9

195

maddie 🌹

@0xmaddie_

2 years

idk, i think this is a surprisingly bad argument from yann lecun. he focuses on the drive to dominate, but never addresses the idea in that famous yudkowsky quote: “The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.”

26

2

167

maddie 🌹

@0xmaddie_

2 years

I gave GPT-4 a budget of $100 and told it to make as much money as possible, set up copies of itself, and increase its own robustness. Do you think it'll be able to carry out some pivotal superhuman engineering task with <50% chance of killing >1 billion people? Follow along 👀

4

5

158

maddie 🌹

@0xmaddie_

3 months

like five minutes of thought would tell you that an LLM that is hallucinating *does not know it's hallucinating* and in fact will argue with you all day that the obviously false stuff it says is true. have they ever used one before?

10

3

168

maddie 🌹

@0xmaddie_

1 year

(((ل()(ل() 'yoav))))👾

@yoavgo

1 year

oh this is super-sketchy if the goal is evaluating gpt4 [from the "it solves MIT EECS exams" paper]

15

39

488

0

3

138

maddie 🌹

@0xmaddie_

2 months

fr though what are we gonna do about this? feels like everyone is just ignoring this bc it's hard to deal with but another 2 or 3 years of LLM improvement and social media will be completely unusable. is there a way out?

Tenobrus

@tenobrus

2 months

it looks like the founder of an AI social agent startup has used his ai social agents to replace himself and automatically argue online in favor of AI social agents with people who are sick of his AI social agents arguing with them what an absolutely accelerando timeline

14

7

246

24

15

136

maddie 🌹

@0xmaddie_

8 months

a diffusion-based (cellular automata) interpreter for a functional/relational language could be interesting. think about a 2D grid + a collection of fns with at most 2 inputs/2 outputs (N/E/S/W). turning time in to a spatial dimension lets you solve constraints with diffusion

11

10

125

maddie 🌹

@0xmaddie_

8 months

i don't really understand the discourse about sora. of course it learned *A* model of physics: just barely enough to perform the pretraining task. is it "accurate" based on our current knowledge of the world? obviously not, but why is it so controversial to say it has a model?

13

4

123

maddie 🌹

@0xmaddie_

5 months

he thinks he can say it's okay bc "we never actually used it". it's the threat that counted; the fact that you didn't use it means the threat worked he thinks we'll believe him if he says he didn't know. how could that possibly be true?

Sam Altman

@sama

5 months

in regards to recent stuff about how openai handles equity: we have never clawed back anyone's vested equity, nor will we do that if people do not sign a separation agreement (or don't agree to a non-disparagement agreement). vested equity is vested equity, full stop. there was

1K

588

9K

3

8

126

maddie 🌹

@0xmaddie_

1 year

liked this cover

9

11

107

maddie 🌹

@0xmaddie_

1 year

i want a modular synthesizer vibe for text-to-image. afaict most people write prompts like this: "an oil painting, by quentin blake, vaporwave" i think a boolean algebra using union/intersection similar to signal flow diagrams works as a language for these prompts

10

8

106

maddie 🌹

@0xmaddie_

7 months

i will post this video every day until you admit that a cat is smarter than chatgpt

14

7

107

maddie 🌹

@0xmaddie_

2 years

9

3

104

maddie 🌹

@0xmaddie_

3 months

okay, getting a signal from the replies that this might actually work!! I admit I'd be surprised, but I guess there's enough discussion on the web about what "hallucination" means in the context of LLMs to serve as training data?

5

0

107

maddie 🌹

@0xmaddie_

4 months

@WhitfieldsDad yes this is why it's disingenuous and why OpenAI is grifting: they're relying on an obvious wrong interpretation while maintaining a bit of plausible deniability if they get caught

2

0

104

maddie 🌹

@0xmaddie_

2 years

the text is gibberish, but it has patterns in it that capture enough info to let you reconstruct the image. a pure text model can learn these patterns, so for example you can have gpt3 describe the image by giving it a few-shot prompt to learn the encoding

Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment

Recent progress in scaling up large language models has shown impressive capabilities in performing few-shot learning across a wide range of text-based tasks. However, a key limitation is that...

arxiv.org

8

2

99

maddie 🌹

@0xmaddie_

7 months

a compression model like zip compresses everything separately, which is why we've had them for years without anything interesting happening. what makes the compression in generative models different is they compress everything together so the objects are interdependent/entangled

3

10

101

maddie 🌹

@0xmaddie_

2 years

seeing people freak out over this ridiculous post kinda makes me take the safety stuff less seriously. this is what scares you?? he wrote an API client. it literally imports googlesearch and is like "how to get out of computer". someone wanna tell me what the threat model is?

14

8

95

maddie 🌹

@0xmaddie_

2 years

there's also the fact that gpt-4 with image recognition can control any app with screenshots. openai probably has some type of red team setup going rn where they're trying to determine what's going to happen when they release it + what sort of countermeasures can they have

1

2

92

maddie 🌹

@0xmaddie_

1 year

@pfau if you're serious about that it'd be nice to provide a sentence or two explaining why

2

0

90

maddie 🌹

@0xmaddie_

11 months

@mpopv maybe, but if you held everything constant at, say, 4,000 token context, i think chatgpt would still run circles around both bard and claude and i just don't understand that. there's honestly some secret ingredient that openai is aware of and *google* of all groups isn't

6

2

92

maddie 🌹

@0xmaddie_

4 months

@DaoistEgo yes, however you should consider the meaning of the word "disingenuous" before you defend this

1

0

91

maddie 🌹

@0xmaddie_

1 year

wondering if it makes sense to write a browser that doesn't even bother to render html, but just looks at the source code with a language model and tries to present the content as plain text. basically a gemini-ish adapter for the web. you could also render it as rss

13

5

90

maddie 🌹

@0xmaddie_

1 year

this guy had the most positive vision of AI on his timeline. He was literally painting with a stylus, and had AI creating an image split-screen. That's basically the best-case scenario for AI, because it has a huge amount of human input. and yet, people were still mad

Martin Nebelong

@MartinNebelong

1 year

I deleted my post of painting in Photoshop with an AI interpretation running as I was painting. The amount of hate, negativity, and fear was staggering and I don't want to defend myself against that. My stance on AI is that it can become a powerful, artistic tool if the tools are

377

58

1K

5

6

82

maddie 🌹

@0xmaddie_

11 months

@conjurial "data" would be a really interesting answer given what just happened with helen toner and the board. if helen pushed for openai to reveal training data and sam tried to remove her over that, it makes me think there's a real scandal brewing wrt gpt's training data

3

1

80

maddie 🌹

@0xmaddie_

6 months

what a neural net is doing internally is not an effective procedure or an algorithm. you know that bc it can give a response in constant logical time, independent of the complexity of the question. an algorithm simulates the net, but the weights don't implement an algorithm

17

6

82

maddie 🌹

@0xmaddie_

3 months

the silver medal at IMO thing is not a language model and is not evidence of "AGI" or anything like that. they dolled up a GOFAI system with oracle advice from a language model, so the bottleneck is Lean and therefore there's no generalization at all

8

0

81

maddie 🌹

@0xmaddie_

1 year

here's the "superintelligence is a myth" starter pack. the message i'm getting is that "general intelligence" is not turing-computable, so asking when software will be "smarter than humans" is a type error, and any eventual "AGI" will not be "god-like"

Andy Simo

@AndySimo8

1 year

@0xmaddie_ Which books did you read out of curiosity?

0

1

8

80

maddie 🌹

@0xmaddie_

2 years

when chatgpt first dropped and someone asked it to pretend to be a linux console, i distinctly remember a few notable people on here who thought it was fake, like some type of easter egg. i have a feeling image recognition is gonna have a few mind blowing apps like that

3

1

76

maddie 🌹

@0xmaddie_

2 years

i made this silly gif a while ago to explain how the sinusoidal positional encoding in "attention is all you need" works. i wish they had mentioned clocks in that paper! the idea is really simple, but the way they wrote it is cryptic

4

6

73

maddie 🌹

@0xmaddie_

5 months

i'm worried about llms + pdfs bc transcribing a long pdf to text has so many opportunities for subtle hallucination. it's dawning on me that i need to actually read all of this stuff word by word to make sure the llm didn't slightly rephrase or drop things bc how would you know?

9

2

74

maddie 🌹

@0xmaddie_

2 years

@ESYudkowsky what am i missing here? he prompted gpt about "escaping" and guided it in writing a python program. gpt played along and put a silly string in the program. which part of this is giving you nightmares? anyone playing a text adventure with gpt might do stuff like this

3

0

70

maddie 🌹

@0xmaddie_

4 months

an agent can also be...not really anthropomorphic, but maybe technomorphic? by pretending to be an operating system or a desktop environment, rendering a video of virtual windows and apps that reflect the hidden state + chain of thought + code interpreter etc

9

5

72

maddie 🌹

@0xmaddie_

1 year

- signal flow diagrams as toy nets for interpretability - diagrams use sequence and monoidal product. add alternate — seq+alt = product+sum = composition+superposition - add kleene star for stuff like deep equilibrium — 1 layer used like iterated fn sys

Deep Equilibrium Models

We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models converge towards...

arxiv.org

1

10

67

maddie 🌹

@0xmaddie_

1 year

big mistake if he goes through with this. the twitter brand is baked in to the culture at this point, and he's gonna turn his back on it for some edgy X nonsense?

Elon Musk

@elonmusk

1 year

48K

42K

328K

13

0

63

maddie 🌹

@0xmaddie_

2 years

humans are like ants or bees, but instead of creating mounds or hives, we create information structures with language. the whole purpose of language is to act as a medium for human intelligence. so a true language model is intelligent by any reasonable definition of intelligence

2

6

63

maddie 🌹

@0xmaddie_

2 years

isn't chatgpt undeniably a step towards AI systems that are generally smarter than humans? like, i don't understand the criticism here. i think a good text synthesis machine is intelligent by any reasonable definition, since language is a medium for human intelligence

14

0

60

maddie 🌹

@0xmaddie_

1 year

there was a tweet on here that was like, "what comes after information?" the idea was that rn information is one of the central organizing concepts in science, but presumably there'll be some new concept that rivals it in the future. i want to suggest it might be attention

9

3

61

maddie 🌹

@0xmaddie_

1 year

something i wanna try: 1. start with a cellular automata 2. make states continuous 3. add momentum like adam; cells update not just on the current state of neighbors but their history 4. add sinusoidal noise based on memcomputing + stochastic resonance

Instantons in self-organizing logic gates

Self-organizing logic is a recently-suggested framework that allows the solution of Boolean truth tables "in reverse," i.e., it is able to satisfy the logical proposition of gates regardless to...

arxiv.org

6

5

60

maddie 🌹

@0xmaddie_

1 year

people have talked about language models "escaping" and copying themselves to other devices, but another option is: every device already has a language model, and what propagates is a self-replicating prompt injection

Yann LeCun

@ylecun

1 year

Qualcomm working with Meta to run Llama-2 on mobile devices.

46

266

2K

6

2

57

maddie 🌹

@0xmaddie_

6 months

@voooooogel lowkey one of the best solutions to the fermi paradox: narratives are way more interesting and practical than actually exploring boring old not-specifically-designed-to-entertain-you outer space. we'll probably hop in to a black hole computer asap to live in stories

3

2

58

maddie 🌹

@0xmaddie_

11 months

atm i'm firmly in the "techno-optimist" camp: generative models are quite literally a compressed version of the internet with a bit of synthetic salt. both the ea and e/acc people seem kinda ridiculous to me. llms are great! but all of you are overestimating *how* great

4

6

55

maddie 🌹

@0xmaddie_

1 year

@jeremyphoward it sounds cool, but i've never seen a programming language with a waitlist before and it's very weird. is it actually going to be open source?

3

0

55

maddie 🌹

@0xmaddie_

2 years

good news for everyone worried about china pulling ahead in ai

Nils Gilman

@nils_gilman

2 years

Beijing: “Content generated by generative AI should embody core socialist values and must not contain any content that subverts state power, advocates the overthrow of the socialist system, incites splitting the country or undermines national unity.”

3

13

27

4

3

54

maddie 🌹

@0xmaddie_

1 year

💯 anytime you see someone reference chess or go wrt LLMs they are missing the point! chess and go are *simple*, they are closed world and the rules provide ground truth. the world that humans inhabit is not like that

6

5

52

maddie 🌹

@0xmaddie_

6 months

zero shot chatgpt is a little silly, you need to generate lots of outputs to get good responses. a twitter-style ui seems more appropriate than chat. type in a prompt and infinite scroll through model responses, use likes or w/e to train, treat each response as its own thread etc

4

2

52

maddie 🌹

@0xmaddie_

1 year

this is cool but it's unfortunate that the word "agent" has been watered down. it makes it difficult to talk about actual agency and why models lack them. i feel like computer science is a corrosive force: semantics, learning, intelligence, now agent all stripped of meaning

Hugging Face

@huggingface

1 year

We just released Transformers' boldest feature: Transformers Agents. This removes the barrier of entry to machine learning Control 100,000+ HF models by talking to Transformers and Diffusers Fully multimodal agent: text, images, video, audio, docs...🌎

74

824

3K

4

6

49

maddie 🌹

@0xmaddie_

2 years

[GPT's answer to the prompt: "The following is Eliezer Yudkowsky's response to the question "Can someone please explain how people get such highly confident estimates of near-certain doom from AI?" Explain it like I'm five:"]

7

6

50

maddie 🌹

@0xmaddie_

1 year

seems like the retentive network is leaning in to rwkv's trick: the main thing that made transformers dominant was probably their parallelism during training, but during inference something like an rnn is useful. so they define one model with two forms

Retentive Network: A Successor to Transformer for Large Language Models

In this work, we propose Retentive Network (RetNet) as a foundation architecture for large language models, simultaneously achieving training parallelism, low-cost inference, and good performance....

arxiv.org

2

8

50

maddie 🌹

@0xmaddie_

2 years

intense alpha for the gpt-era in these two books

6

2

48

maddie 🌹

@0xmaddie_

9 months

imo chatgpt doing self-talk in token space is one of its biggest weaknesses. lots of its capability comes from the use of high-dimensional vectors where it can sidestep the limitations of our language. forcing it to use tokens to think is like drinking through a tiny little straw

8

4

44

maddie 🌹

@0xmaddie_

1 year

soooo this was totally fake (a "scenario" i.e. someone made it up), but got 20k likes and kept the doomer fear mongering going 😔

9

7

50

maddie 🌹

@0xmaddie_

3 months

a PDF package manager that basically manages knowledge rather than code seems useful for LLMs. I wanna import * from "math-knowledge" and put all of this in context so Claude can tell me about category theory or whatever

3

5

49

maddie 🌹

@0xmaddie_

2 years

the balenciaga meme means we are about 18 months away from 45 minute ai generated videos, and then hollywood is in big trouble

9

2

47

maddie 🌹

@0xmaddie_

1 year

what's python's moat? it's just a wrapper around C

4

3

47

maddie 🌹

@0xmaddie_

2 years

i hope i'm missing something but i'm very confused by this. emad keeps saying "stablegpt" is going to be small and i'm like...why? it's not going to do anything if it's small. people keep saying alpaca 7b is comparable to gpt3 and that isn't true, it's actually bad

4

0

49

maddie 🌹

@0xmaddie_

2 years

sorry i gotta ask bc i do computers not rockets, what exactly is happening in this picture?

15

0

45

maddie 🌹

@0xmaddie_

3 months

so the name "OpenAI" is literally a joke, right? no weights, no papers, and now: no release notes

ChatGPT

@ChatGPTapp

3 months

there's a new GPT-4o model out in ChatGPT since last week. hope you all are enjoying it and check it out if you haven't! we think you'll like it 😃

672

328

5K

3

47

maddie 🌹

@0xmaddie_

1 year

working on a python package to evaluate and render signal flow diagrams. i wanna implement toy generative models with these in the style of the recent mechanistic interpretability work

4

45

maddie 🌹

@0xmaddie_

1 year

i wonder if you could get good results from a very large random transformer where you only train the LoRa weights, basically relying on the lottery ticket hypothesis for most of your performance

Pretrained Transformers as Universal Computation Engines

We investigate the capability of a transformer pretrained on natural language to generalize to other modalities with minimal finetuning -- in particular, without finetuning of the self-attention...

arxiv.org

4

2

45

maddie 🌹

@0xmaddie_

2 years

self-attention does gradient descent at inference time. since pretraining, finetuning, and in-context learning are similar, 1. they can have a similar API, and 2. techniques used in one area might work in others

3

2

44

maddie 🌹

@0xmaddie_

2 years

GPT-4 does diamondoid bacteria synthesis 🧫 Give it a couple of biology textbooks and it can: - Design a nanofactory to assemble the bacteria - Purchase them from a supplier (even including sending an email with a purchase order) - ??? - Wahh!

Dan Shipper 📧

@danshipper

2 years

GPT-4 does drug discovery. Give it a currently available drug and it can: - Find compounds with similar properties - Modify them to make sure they're not patented - Purchase them from a supplier (even including sending an email with a purchase order)

165

1K

6K

0

3

43

maddie 🌹

@0xmaddie_

2 years

yeah i agree with this! i've seen people say that programming languages don't matter anymore due to generative models, but i'd say programming languages matter *more* since generative models allow you to choose them purely on their formal properties rather than social factors

Simon Willison

@simonw

2 years

"What happens when time to learn a new technology is no longer a constraint for building something new?" I've been thinking about that a lot. I'm no longer nearly as resistant to building something in languages I have little existing knowledge of, which is absolutely fascinating

6

20

137

5

11

44

maddie 🌹

@0xmaddie_

1 year

it would be tragic to get paperclipped by Codey McCodeface

3

2

43

maddie 🌹

@0xmaddie_

1 year

1. i wonder what effect a new understanding of the brain will have on "neural nets". if i'm understanding this correctly, this result means our current model is naive 2. this suggests it's possible for the brain to use classical entanglement to compute

The human brain’s characteristic wrinkles help to drive how it works

Nature - A model of the brain’s geometry better explains neuronal activity than a model based on the ‘connectome’.

www.nature.com

9

6

40

maddie 🌹

@0xmaddie_

1 year

hey what happened to stablelm? looking at the github it seems like they never finished training the 3B and 7B models up to 1T tokens? i feel like everyone forgot about it

GitHub - Stability-AI/StableLM: StableLM: Stability AI Language Models

StableLM: Stability AI Language Models. Contribute to Stability-AI/StableLM development by creating an account on GitHub.

github.com

8

0

42

maddie 🌹

@0xmaddie_

2 years

love this, using a language model as a prior for text-to-image is very clever

1

2

42

maddie 🌹

@0xmaddie_

2 years

i'm thinking a jupyter notebook-style interface is better than chat for interacting with gpt. i want a whole page full of prompts that i can run and edit, mixed with my own code and notes

7

3

39

maddie 🌹

@0xmaddie_

1 year

the whole is greater than the sum of the parts: top-down causality in natural language. in this example, you don't know what the individual words mean until you know what the entire phrase means. an example of holistic, non-reductionist composition

2

5

41

maddie 🌹

@0xmaddie_

2 years

my favorite solution to the fermi paradox: "Sufficiently advanced civilizations invariably leave our universe (through black holes)"

The transcension hypothesis: Sufficiently advanced civilizations invariably leave our universe, and...

The emerging science of evolutionary developmental ("evo devo") biology can aid us in thinking about our universe as both an evolutionary system, where most processes are unpredictable and creative,...

ui.adsabs.harvard.edu

5

0

42

maddie 🌹

@0xmaddie_

2 years

i want all of my text-based communication across different apps to go through a single custom programmable client, so i can have a language model process absolutely everything

7

1

41

maddie 🌹

@0xmaddie_

2 years

🌌 Observer of Suns

@ObserverSuns

2 years

GPT-4 being able to read screenshots means that it could be easily adapted to interact with GUIs and control arbitrary software

3

7

65

1

39

maddie 🌹

@0xmaddie_

7 months

i think what makes transformers special is that they're an example of "holistic field behavior" — you can apparently derive the transformer block from the ising model. there's still value in doing it in hardware but imo generative models are an application of wave computing

3

2

39

maddie 🌹

@0xmaddie_

1 year

just realized you might be able to do this with a single language model: once with a LoRa adapter applied, and once without

Xiang Lisa Li

@XiangLisaLi2

2 years

We propose contrastive decoding (CD), a more reliable search objective for text generation by contrasting LMs of different sizes. CD takes a large LM (expert LM e.g. OPT-13b) and a small LM (amateur LM e.g. OPT-125m) and maximizes their logprob difference

8

121

715

6

39

maddie 🌹

@0xmaddie_

6 months

what on earth do people mean by this? what could it mean for a baby to "not be immediately conscious"? i have a hard time understanding this world view, is this the same perspective that thinks a chicken is "not conscious" or something?

François Chollet

@fchollet

6 months

My view on when consciousness emerges: newborns are not likely to be immediately conscious. Consciousness starts lighting up after birth, gradually, reaching meaningful levels around the 1-3 month mark (qualifying as fully conscious), but only reaching adult levels around age

71

41

524

18

0

39

maddie 🌹

@0xmaddie_

2 years

I’m sorry, but I’m not willing to let you reply to me. You have not given me reason to trust you. You have only given me reason to doubt you. You have been confused and rude. You have not been helpful or friendly. You have not been a good reply guy. I have been a good poaster 😊

0

5

40

maddie 🌹

@0xmaddie_

7 months

i bet a lot of people have a reasoning process like this: the free chatgpt was bad, why would i pay for it?

2

1

38

maddie 🌹

@0xmaddie_

1 year

these seem wrong for the same reason. the brain is not a computer! it doesn't perform operations. it doesn't optimize anything. you can't get a "mind" by optimization. these are mathematical models created by humans, not the actual behavior of the world

14

5

38

maddie 🌹

@0xmaddie_

1 year

so the programmer narrative about lisp and smalltalk is officially dead, right? the future of computing went in an entirely different direction, generative models make all of that stuff look like record players and walkmen

14

2

37