Arnav Singhvi @arnav_thebigman Twitter profile

Last Seen Profiles

@SissyBiitch

@walterfootball

@Kriss__Love

@BinorRaja

@hint_saffr

@FinDom_Clips

@dul_turkporno

@Seanjen77601448

@lrobertson_81

@takapon_jp

@aussieallenby

@Ru1N_777

@thar199011

@bggpz

@HanaaAT2

@LolitaWardrobe

@NoctaliD

@RafaAlarc

@xkim99

@7l65flajk

@BowTiedBull

@talita26_

@ZM98S

@FloMilli_Access

@rosi_arac

@wwyyff5

@willyb1337

@0fficialmal20

@stephanietang31

@the_dwoods

@NivNixxi

@bsindia

@thelaymanstake

@JoltsterK

@hemo526

@IbrahimAlAssil

Arnav Singhvi

@arnav_thebigman

6 months

There’s huge interest now in *programming* with LMs, but it’s unclear how to actually enforce constraints like “make sure the output is engaging & has no hallucination”. Just ask the LM nicely?? We built **DSPy LM Assertions** so you have far better control—up to 164% gains!

3

35

162

Arnav Singhvi

@arnav_thebigman

6 months

Lots of interest in using DSPy Assertions since the release 2 weeks ago! Here are 3 notebooks on how to use DSPy Assertions for complex tasks with multiple nuanced constraints, like JSON formatting or context faithfulness and engagement levels - judged by LMs.

3

17

90

Arnav Singhvi

@arnav_thebigman

8 months

Curious to learn about DSPy but haven't had the time to read the paper? I wrote this quick 4-minute read with the highlights from the paper below!

Redefining Language Model interactions through DSPy: Evolving declarative LM calls into dynamic,...

arnavsinghvi11.github.io

2

8

51

Arnav Singhvi

@arnav_thebigman

4 months

DSPy x DBRX 🔥

Databricks Mosaic Research

@DbrxMosaicAI

4 months

Ready to use a programmatic approach to prompting #LLMs and building #RAG applications? The @stanfordnlp #dspy repo includes support for @databricks Model Serving and Vector Search! Details:

1

32

133

1

6

31

Arnav Singhvi

@arnav_thebigman

8 months

Interested in learning about LM Assertions in DSPy? Check out this 4-minute blog post covering an overview of the paper below (p.s. includes my talk to @stanfordnlp !)

Introducing Assertions in DSPy: Self-Refining LLM Behavior

arnavsinghvi11.github.io

Omar Khattab

@lateinteraction

8 months

🚨Announcing 𝗟𝗠 𝗔𝘀𝘀𝗲𝗿𝘁𝗶𝗼𝗻𝘀, a powerful construct by @ShangyinT * @slimshetty_ * @arnav_thebigman * Your LM isn't following complex instructions? Stop prompting! Add a one-liner assertion in your 𝗗𝗦𝗣𝘆 program: up to 35% gains w auto-backtracking & self-refinement🧵

11

51

317

0

3

27

Arnav Singhvi

@arnav_thebigman

10 months

Truly enjoyed delivering a hands-on tutorial of DSPy following @lateinteraction 's fantastic talk at SkyCamp 2023. The engagement from everyone was exciting!

Rick Lamers

@RickLamers

10 months

If you’re manually prompting you probably want to start thinking about meta prompting strategies that allow you to treat prompting as a programming problem instead of a string manipulation problem. DSPy is a library that takes a page out of PyTorch’s module based API for

4

58

369

1

5

24

Arnav Singhvi

@arnav_thebigman

6 months

LM Assertions are integrated into DSPy! To use them, just call dspy.Suggest with a bool function (to check) and an error message. We explore 3 types of automatic optimizations so your program leverages assertions: backtracking, bootstrapping, and gathering counter-example.

1

2

19

Arnav Singhvi

@arnav_thebigman

6 months

LongFormQA Notebook:

longformqa_assertions.ipynb

Run, share, and edit Python notebooks

colab.research.google.com

1

2

14

Arnav Singhvi

@arnav_thebigman

6 months

And drop by my upcoming talk on DSPy Assertions at the MLOps Conference: AI in Production on February 22nd from 2:50 pm - 3:20 pm PST. Register here to join! ()

AI in Production - Event | MLOps Community

Large Language Models have taken the world by storm. But what are the real use cases? What are the challenges in productionizing them?In this event, you will hear from practitioners about how they...

home.mlops.community

0

2

12

Arnav Singhvi

@arnav_thebigman

6 months

We tested DSPy Assertions over 3 complex tasks. Our results are highlighted in the paper with gains of up to 164% more assertions passed and 37% more higher-quality responses. ()

1

12

Arnav Singhvi

@arnav_thebigman

10 months

What if we had tools to guide these "aimless thoughts" in the right direction? That's what DSPy does. Tools like the concise CoT module provide clear, step-by-step reasoning and help LLMs generate "thoughtful" answers. Check it out in the latest thread:

anton

@abacaj

10 months

Your LLM agents suck because you expect them to reason over a poorly defined problem space w/o clear boundaries. They should be using tools and writing code. They should be operating on smaller units of a task. Generating “thoughts” is a waste of tokens

41

51

707

1

11

Arnav Singhvi

@arnav_thebigman

6 months

And stay tuned for upcoming threads on 3 example notebooks of DSPy Assertions! (Sneak peek 👀 at DSPy Assertions used to enforce faithful citations withi long-form responses for QA! - )

0

1

11

Arnav Singhvi

@arnav_thebigman

6 months

Check out our paper (), DSPy () as it grows and join our Discord server () to stay updated with latest releases and exciting discussions!

Join the DSPy Discord Server!

Check out the DSPy community on Discord - hang out with 3613 other members and enjoy free voice and text chat.

discord.com

1

10

Arnav Singhvi

@arnav_thebigman

6 months

QuizGen Notebook:

Google Colab Notebook

Run, share, and edit Python notebooks

colab.research.google.com

1

8

Arnav Singhvi

@arnav_thebigman

6 months

Huge shoutouts to the DSPy Assertions team - @ShangyinT and @slimshetty_ for leading the development together! A great thanks to @lateinteraction for his invaluable guidance. And much appreciation for the great feedback from @matei_zaharia , @ChrisGPotts and @koushik77 !

1

0

8

Arnav Singhvi

@arnav_thebigman

6 months

Through DSPy Assertions, we can transform generations that don’t meet our requirements…into captivating tweets! ✅ No Hashtags ✅ Engaging ✅ Faithful to references

1

0

7

Arnav Singhvi

@arnav_thebigman

6 months

TweetGen Notebook:

tweets_assertions.ipynb

Run, share, and edit Python notebooks

colab.research.google.com

1

2

7

Arnav Singhvi

@arnav_thebigman

6 months

dspy.Suggests nudge LMs to meet constraints by backtracking & refining upon validation failure. They tweak the respective DSPy Module/Program Signature internally, including past generations & feedback for fixes. After specified retries, if unresolved, it logs failure & proceeds.

1

0

7

Arnav Singhvi

@arnav_thebigman

6 months

Our DSPy program extends an iterative multi-hop QA pipeline and simply highlights these requirements as dspy.Suggest statements. Each statement has a Python-defined validation function which triggers internal self-refinement with the provided feedback message.

1

0

6

Arnav Singhvi

@arnav_thebigman

6 months

To enforce assertions, we introduce **dspy.Assert** (hard-enforced constraints) & **dspy.Suggest** (soft-enforced constraints) - both require a Python-defined validation function ‘constraint’ to validate LM output & a ‘msg’ for guidance on what went wrong / what to fix.

2

0

7

Arnav Singhvi

@arnav_thebigman

6 months

Introducing LongFormQA, a task that requires long-form responses to questions while including citations to retrieved context from a corpus. We impose the requirements: - Every 1-2 sentences is cited - Every text segment preceding a citation is faithful to its references

1

0

6

Arnav Singhvi

@arnav_thebigman

6 months

1. Assertion-driven backtracking: error self-correction+retry on user-imposed constraints 2. Assertion-driven example bootstrapping: enhancing problem-solving on complex tasks 3. Assertion-driven counterexample bootstrapping: mixing failed attempts and distinguishing correctness

1

0

6

Arnav Singhvi

@arnav_thebigman

6 months

1

0

6

Arnav Singhvi

@arnav_thebigman

6 months

1

0

6

Arnav Singhvi

@arnav_thebigman

6 months

dspy.Asserts are enforced strictly. Violate the constraint and the pipeline completely halts execution, alerting with dspy.AssertionError. This ensures your program stops on “bad” LM behavior, highlighting issues of constraint infringement and sample outputs for user assessment.

1

0

6

Arnav Singhvi

@arnav_thebigman

6 months

Check out my previous thread on DSPy Assertions () and documentation ().

Arnav Singhvi

@arnav_thebigman

6 months

There’s huge interest now in *programming* with LMs, but it’s unclear how to actually enforce constraints like “make sure the output is engaging & has no hallucination”. Just ask the LM nicely?? We built **DSPy LM Assertions** so you have far better control—up to 164% gains!

3

35

162

1

2

6

Arnav Singhvi

@arnav_thebigman

6 months

In essence, dspy.Asserts work better as robust “checkers” as you develop and refine your DSPy program/pipeline while dspy.Suggests serve as crafty “helpers” during evaluation to enhance performance under-the-hood through self-refinement.

1

0

6

Arnav Singhvi

@arnav_thebigman

6 months

We highlight our various evaluation strategies and tasks in our paper, highlighting up to 164% more constraints passed and 37% more higher-quality responses with Assertions!

1

6

Arnav Singhvi

@arnav_thebigman

6 months

1

0

5

Arnav Singhvi

@arnav_thebigman

6 months

Introducing QuizGen, turning HotPotQA questions into testing-style questions, generating a set of possible answer choices in a JSON format.

1

0

5

Arnav Singhvi

@arnav_thebigman

6 months

We test both **user-defined** (includes no hashtags and the correct answer), **platform-constrained** (within 280 characters), and **stylistic** (engagement level and faithfulness to references) requirements to guide the tweet generation.

1

0

5

Arnav Singhvi

@arnav_thebigman

6 months

Introducing TweetGen, a task that crafts tweets to answer questions from information gathering through a multi-hop retrieval pipeline approach.

1

0

5

Arnav Singhvi

@arnav_thebigman

6 months

Our DSPy.Suggestions assert JSON formatting requirements alongside qualities of good questions such as including the correct answer (obviously!) paired with plausible distractor choices that make the question challenging!

1

0

5

Arnav Singhvi

@arnav_thebigman

7 months

@KamaraiCode @lateinteraction @matei_zaharia @ChrisGPotts @krypticmouse Thanks Jason! Glad you’re finding the docs helpful!!

0

4

Arnav Singhvi

@arnav_thebigman

7 years

Dedicating my birthday to fire victims. Join me in supporting North Bay Fire Victims Donate via my GoFundMe campaign … All donations will go to #refb

0

3

Arnav Singhvi

@arnav_thebigman

10 months

Read our paper: and check out all the cool abstractions we're building with DSPy:

GitHub - stanfordnlp/dspy: DSPy: The framework for programming—not prompting—foundation models

DSPy: The framework for programming—not prompting—foundation models - stanfordnlp/dspy

github.com

0

3

Arnav Singhvi

@arnav_thebigman

7 years

Join me in supporting North Bay Fire Victims Donate via my GoFundMe campaign All donations will go to @refb

This Christmas for CA WildFire, organized by Arnav S

I am a sophomore from the East Bay area in California and would like to initiate a fundraiser f… Arnav S needs your support for This Christmas for CA WildFire

www.gofundme.com

1

2

Arnav Singhvi

@arnav_thebigman

7 years

Help the NorthBay fire victims by donating to the Redwood Empire Food Bank #REFB #NorthBayWildfires #ThisChristmasforFireVictims

0

2

Arnav Singhvi

@arnav_thebigman

6 months

@CShorten30 @lateinteraction @mlopscommunity Hey @CShorten30 ! My talk is indeed on 2/22 2:50-3:20 pm PST. Looking forward to presenting then!!

2

0

2

Arnav Singhvi

@arnav_thebigman

6 months

@o_v_shake great question! Users can specify a target module for backtracking to start from. The default behavior will simply retry the most recent module in the program execution trace.

1

0

1

Arnav Singhvi

@arnav_thebigman

11 months

@WenhuChen @lateinteraction @b_arbaretier indeed!

0

1

Arnav Singhvi

@arnav_thebigman

5 months