Rohit Agarwal @jumbld Twitter profile | Pikagi

Pikagi

Rohit Agarwal

@jumbld

2,919

Followers

1,392

Following

312

Media

5,225

Statuses

Building @PortkeyAI - The fast & fun way to ship AI applications to production. Shipped my first open source product this year, passionate about llms in prod.

https://t.co/o38N3Ja6Tg

Joined September 2008

Don't wanna be here? Send us removal request.

Pinned Tweet

@jumbld

Rohit Agarwal

5 months

Ok, here goes. The mega implementation guide for FrugalGPT that can get you upto 98% cost savings on LLMs without compromising on accuracy. I've written this over 6 months of research & iteration with what works and what doesn't - compressed into 3000 words for the web.

Tweet media one

8

39

315

Last Seen Profiles

@MSSUBaseball

@yorozuyarde

@akuncampurann2

@SrEpstein

@akuncampurann2

@mgufc

@saloncinema

@Xdoudou2

@ibubohay2

@Xdoudou2

@HarixVincent

@helencarnateNH

@aichiyouyudetu1

@xx_shin24

@thegraphqlconf

@bapakdananak

@COdagaki75253

@samir123448171

@emperour_eth

@rawarrenjr

@SweetloveAika

@abhi_v5

@bokeplokalmalam

@Backspaceartz

@ibubohay2

@BrokenBoatTony

@KPHYudi

@pygmypearls

@Drscottyoung224

@i_ichijyuu

@BitahRosen59528

@guild_kayoha

@Evermusic210320

@Fatimah_dz

@umalmend

@Whatmakesmemad

@jumbld

Rohit Agarwal

9 months

We're open sourcing our AI gateway today. we built this over 200 days managing over a 100B tokens in production. just put it up on HN -> please check it out. More deets to follow on - what is it? - why should you care? - and why tf did we open source it?

Tweet media one

22

62

559

@jumbld

Rohit Agarwal

1 year

I've been trying to figure out how to get ChatGPT to only spit formatted content. Only code, or only JSON, etc.. Here's a simple "prompt construction" (h/t @hwchase17 for the term) which works almost every time! It goes like - "Starting from the next line, write <what you want

42

38

373

@jumbld

Rohit Agarwal

1 year

Would it be useful if we had virtual keys for LLM providers much like virtual cards? - Manage spends better - Link different applications to different keys - Better security - A key for every developer - Hand out spend locked keys for hackathons - Build bring-your-own-key as a

7

9

272

@jumbld

Rohit Agarwal

2 months

We're launching our open-source AI guardrails framework on our AI gateway today. Been building it with inputs from 600+ teams who use the gateway in production and have collectively made 1.4 billion API requests on our hosted platform itself! trying our luck with an HN launch

Tweet media one

17

33

174

@jumbld

Rohit Agarwal

4 years

Small announcement to my Twitter universe - I've joined @Pepper_Content after an extremely memorable 5 years at @FreshworksInc In these years, Freshworks has grown > 10x on almost every metric while retaining the amazing culture, set by the founders! 👇🏻

Tweet media one

25

2

148

@jumbld

Rohit Agarwal

1 year

Excited to announce ⚡️ ⚡️ - A foundational model ops platform to help companies ship gen AI apps & features with confidence! 💡 Monitor usage, latency & costs 💼 Manage models with ease 🔒 Protect your user data #FMOps #LLMops w/ @ayushgarg_xyz

Tweet media one

Tweet media two

14

21

142

@jumbld

Rohit Agarwal

9 months

Tweet card media

GitHub - Portkey-AI/gateway: A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+...

A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API. - Portkey-AI/gateway

3

9

96

@jumbld

Rohit Agarwal

7 months

If Postman is **just** a GUI wrapper on curl, you should build that OpenAI wrapper.

@ptr_to_joel

Joel 🇦🇺

7 months

If a GUI wrapper around curl has the courage to host a conf, you can make your side projects public

58

437

4K

1

4

91

@jumbld

Rohit Agarwal

7 months

This is awesome! Just converted the notebook to Portkey Prompts and been playing around it multiple large + small model combos. Opus and Haiku are great. GPT-4 and Llama-7B on Anyscale also worked really well! Maybe will open source the notebook to let everyone try any model

@mattshumer_

Matt Shumer

7 months

Introducing `claude-opus-to-haiku` ✍️ Get the quality of Claude 3 Opus, at a fraction of the cost and latency. Give one example of your task, and Claude 3 Opus will teach Haiku (60x cheaper!!) how to do the task perfectly. And it's open-source:

50

257

2K

6

7

80

@jumbld

Rohit Agarwal

11 months

At @PortkeyAI we are the default AI gateway for ~10M LLM Requests everyday across @OpenAI @Azure @anyscalecompute and other LLM providers. We track the API success rates of various providers internally. Sharing a glimpse that OpenAI has been seeing problems in the past 2 days.

Tweet media one

5

11

66

@jumbld

Rohit Agarwal

3 years

Just stress-tested over 280 online and offline Indian brands across almost all genres. Brain is fried with all the marketing copy, but boy.. are we growing up! Dropping something big soon🤐

2

4

54

@jumbld

Rohit Agarwal

3 years

A group of super folks (BITSians, Sydney) is trying to procure and deliver cryogenic oxygen canisters, cylinders and concentrators to districts and non-metros which are not in media attn but need support. Read more: , Donate:

Medical Oxygen for All (MOfA) | Notion

Oxygen Concentrators and BiPaP Ventilators Ambulances

anishsinha.notion.site

1

21

50

@jumbld

Rohit Agarwal

3 years

2019 - Netflix & Chill 2021 - Dolo and Rest?

2

4

49

@jumbld

Rohit Agarwal

1 year

We've built so much at @PortkeyAI along with our early partners, that our docs looked a little lacklustre. Today, we're launching a revamped docs experience complete with 🫡 Guides with @OpenAI @AnthropicAI @cohere @LangChainAI @Azure and more ✨ Detailed sections on every

1

6

46

@jumbld

Rohit Agarwal

4 years

@ShaanVP Plot twist: clubhouse builds a SaaS product for teams to run their stand ups on and do chit chats. The tech world is already on it, like the format. Use it for work?

3

0

46

@jumbld

Rohit Agarwal

5 years

Heard of the 10x engineer? Here’s 10x marketing. #Cricket4Life

Tweet media one

1

4

44

@jumbld

Rohit Agarwal

4 years

@ShaanVP Facebook still acquires it. But it’s called Workplace Voice.

1

0

44

@jumbld

Rohit Agarwal

5 months

The 2 graphs that should matter to any AI engineer. by @karpathy on tactics v/s improvements in LLM accuracy and this one on tactics v/s reduction in LLM spend

Tweet media one

Tweet media two

1

5

43

@jumbld

Rohit Agarwal

4 years

Well well well... Look who just joined the 4 digit follows club! 🥳

1

0

43

@jumbld

Rohit Agarwal

1 year

This is what "Doing it right" looks like.

Tweet media one

2

5

41

@jumbld

Rohit Agarwal

10 months

After a lot of work, announcing all the good stuff we're working on with @anyscalecompute ! Anyscale Endpoints is the simplest, fastest, cheapest inference layer for open source models today. @PortkeyAI enables experimentation of these cutting edge models through a fast AI

Tweet media one

Tweet media two

0

6

39

@jumbld

Rohit Agarwal

1 year

🔖 Super excited to share a blog that I've been working on these last 3 weeks with help from a lot of folks! Accuracy & Hallucinations is one of the MAIN challenge for adopting LLMs in production. Evaluations as part of CI/CD and in real-time are very good counter-measures.

1

7

38

@jumbld

Rohit Agarwal

1 year

Fine-tuning Llama-2 13B on 4.7M tokens took just 19 mins on @anyscalecompute ! The GPT-3.5 fine-tuning took ~50 mins for the same dataset. Time for the interesting bits now! Collaborating with @sahilshah91 and Abhishek on evaluating these fine-tunes with base models.

Tweet media one

@jumbld

Rohit Agarwal

1 year

The OpenAI Fine-tune is ready! We picked the longest 25k character length examples from the `sql-create-context` dataset. Took 50 mins to complete the fine-tune, I probably spent ~$250 on this. (Hope this works!)

Tweet media one

1

0

6

3

7

37

@jumbld

Rohit Agarwal

1 year

Some great panel discussions ongoing at BITSync 2023. Very proud to see so many friendly BITSAA alums here in the Bay Area!

1

2

38

@jumbld

Rohit Agarwal

2 months

Is it normal that I almost start to feel guilty when Claude is doing a lot of stuff for me and all I do is keep giving it feedback and making it do more work?

9

2

38

@jumbld

Rohit Agarwal

6 months

Multimodal LLMs are here today, and they are ready to go to production I’m thrilled to share that Portkey’s whole product suite across Observability, AI Gateway, Prompt Management, and Security - now supports multimodal use cases. Reality is fundamentally multimodal. LLMs that

Tweet media one

1

5

35

@jumbld

Rohit Agarwal

6 months

@mattshumer_ I’ve seen that using a rubric for the model to grade works even better. Give it 1 point if ______ Etc

2

2

35

@jumbld

Rohit Agarwal

8 months

3,500 stars for the Portkey Gateway — WOW! Open-sourcing the project was a leap of faith, and seeing the community respond like this… it's incredibly validating. Huge thanks to everyone who has supported us, contributed code, or built something amazing with the Gateway.

4

4

34

@jumbld

Rohit Agarwal

4 years

I’ve levelled up! My favourite PM comic interviewed me :) Thanks @AparnaChugh @that_tall_chica !!!

@ShippingTmrw

Shipping Tomorrow 🚀

4 years

In our latest post, we interviewed @jumbld , Head of Product at Pepper Content, about what he looks for when he interviews product managers. This post has tips about how to prepare, what to expect and things you must avoid. Subscribe for enlightenment⚡

0

3

17

2

5

32

@jumbld

Rohit Agarwal

2 years

Opened @retool after a while today and a notification said that it's gotten 75% faster. That's a big claim. Built an app on it, and can DEFINITELY feel the performance upgrades. Kudos :D

0

2

33

@jumbld

Rohit Agarwal

1 year

Once you make peace with the fact that LLMs hallucinate (and we're all getting there fast), we realise the bigger problems with these APIs while integrating. 🔴 All LLM APIs are different; 🔴 they're not entirely reliable, 🔴 have high latencies; 🔴 need us to manage

2

6

33

@jumbld

Rohit Agarwal

2 years

If you've spent time writing content on Google Docs, I'm sure you've seen how powerful the revision history can be. It has a ton of use cases as you edit content written by someone else OR just browse through the evolution of a great content piece! It's done very smartly...

Tweet media one

4

9

33

@jumbld

Rohit Agarwal

1 year

Going to be talking about the GenAI scene in India on @DDIndialive today evening along with @Ankitthe1 🏃‍♂️adding “as seen on TV” to my linkedin bio soon

@DDIndialive

DD India

1 year

Watch #IndiaIdeas at 8 pm IST tonight where @grrroy talks 'India's Gen AI Empowering Startups' with @nasscom 's @Ankitthe1 , @PortkeyAI 's @jumbld & 's @arvindnair__ @DGDDNews @PriyaKumar2012 @DDNewslive

0

4

9

4

1

33

@jumbld

Rohit Agarwal

1 year

This was a great discussion! I’m glad that India is taking Gen AI very seriously and the level of discourse is high.

@PortkeyAI

Portkey

1 year

. @jumbld is live on India's national broadcaster, discussing India's efforts towards Gen AI and how Portkey is empowering AI innovation!

Tweet media one

Tweet media two

0

1

12

5

1

32

@jumbld

Rohit Agarwal

1 year

I'm very excited about this communicative agent framework. Instead of using chat completions on OpenAI only as system, user & assistant - what if we could have multiple AI assistants in the same chat instance? A group chat with multiple AI agents in the room who can also..

@hwchase17

Harrison Chase

1 year

🐪CAMEL🐪 Communicative Agents for “Mind” Exploration of LLM Society This paper shows how to put 2 agents in a sandbox with each other and watch them interact. Now implemented in LangChain! (s/o @guohao_li ) Original Paper: Docs:

30

138

692

3

2

31

@jumbld

Rohit Agarwal

1 year

At the Berkeley LLM AI hackathon’s OSS Alley today! Along with awesome folks from @arizeai @weaviate_io @vocodehq @lancedb @LangChainAI @llama_index @chainlit_io and semantic kernel from @Microsoft What energy!!

Tweet media one

Tweet media two

2

3

31

@jumbld

Rohit Agarwal

11 months

It's time to hedge?

Tweet media one

2

1

30

@jumbld

Rohit Agarwal

1 year

🤯 that @PortkeyAI processes ~2B tokens a day! And while doing this it, 🐞 reduces error rates from 5% to 0.02% 💽 serves ~20% of all calls from cache ⚡ at latencies below 20ms (with cache speedups, we actually improve latencies!) Sharing more data soon!

0

6

30

@jumbld

Rohit Agarwal

5 months

We're partnering up with @F5 to bring our fast AI Gateway and comprehensive Observability Suite to more AI teams! <NGINX:APIGateway::Portkey:AIGateway>

Tweet media one

1

3

30

@jumbld

Rohit Agarwal

9 months

🚨 Implementing semantic cache at scale🚨 Semantic caching can be extremely useful and feels very simple to implement. But, when you get to production you'll realise the 100s of corner cases and realise that it's much much harder to implement. I'm very glad to be doing a

3

2

30

@jumbld

Rohit Agarwal

2 months

I have nothing but respect for bootstrapped founders. They're charting their story and building profitable and sustainable businesses no doubt. I look at folks like @kar2905 and @pbteja1998 building amazing companies and I'm regularly inspired by them! I also have heard (and

2

1

28

@jumbld

Rohit Agarwal

1 year

While amazing techniques & libraries exist for this, there's little literature on how to use them in production. Here's a blog that decodes the OpenAI's evals framework and takes you step-by-step through how to use it to your advantage. 🔁

Tweet card media

Decoding OpenAI Evals - what is eval, templates,

Learn how to use the eval framework to evaluate models & prompts to optimise LLM systems for the best outputs.

2

3

29

@jumbld

Rohit Agarwal

11 months

I was at #RaySummit2023 and was blown away by all the stuff @anyscalecompute was rapidly launching. One thing that was particularly interesting was anyscale endpoints that allowed everyone to use SOTA OSS models in an OpenAI-esque way. On the keynote day, we realised that

Tweet media one

2

6

29

@jumbld

Rohit Agarwal

4 years

Wait a minute, is emoji search multilingual?! #ios

Tweet media one

1

0

28

@jumbld

Rohit Agarwal

5 months

DSPy is prompt engineering for MLEs

3

2

28

@jumbld

Rohit Agarwal

3 years

Developer-focused products seem to have just the most amazing websites! ✅Focused on the message, ✅clear value prop, ✅carefully spread call to actions! Looking at @supabase_io @prisma @tailwindcss @tiptap_editor

3

2

28

@jumbld

Rohit Agarwal

8 months

gave in to gen-z lingo and replaced 👍🏻 with 💯 for interactions on slack and linkedin

7

0

27

@jumbld

Rohit Agarwal

4 years

The excitement / anxiety when someone buys your product subscription before you've finished building it. 🤩

3

1

27

@jumbld

Rohit Agarwal

1 year

Ok, recorded my first podcast with @CShorten30 of @weaviate_io We keep it informal and chat about taking LLM apps to production, convince Connor that semantic cache is a possibility, middleware to load balance and fallback LLM api calls, open source v/s closed source and much

@CShorten30

Connor Shorten

1 year

I am SUPER excited to publish the 61st Weaviate Podcast on LLMOps with Rohit Agarwal ( @jumbld ) from Portkey! 🚀 There is a super interesting LLM middleware layer emerging from load balancing LLMs to semantic caching and more, loved this conversation! 🧠

0

7

25

1

4

26

@jumbld

Rohit Agarwal

5 months

1.3 Better utilize a smaller model with a more optimized prompt (h/t @mattshumer_ ) There may be certain tasks that can only be accomplished with bigger models. This is because prompting a bigger model is easier or also because you can write a more general purpose prompt, do

Tweet media one

1

2

26

@jumbld

Rohit Agarwal

1 year

Was a charged up session yesterday at Tinker Together! Insights: - Privacy becomes an important factor as we think of RAG use cases in Gen AI - How do you avoid cache leaks specially for semantic caches - PII protection at multiple levels came up. People even requested it as a

Tweet media one

1

2

26

@jumbld

Rohit Agarwal

3 years

Did you know that "The Hobbit" is the most popular book in the world? It did more than $140mn in sales! It contains 95,346 words. I looked it up since @peppertype_ai generated 22mn words 3 days back. 22 MILLION! Mr. Tolkien with Peppertype...

2

2

25

@jumbld

Rohit Agarwal

3 years

We've graduated from high-quality content through @peppertype_ai to now high-performance content as well! This is something that's been in the works for the past 2 months now. Glad to see it working irl.

Tweet media one

0

3

25

@jumbld

Rohit Agarwal

7 months

was showing how fast and cool @GroqInc is to my wife, who is a ChatGPT power user. After explaining why it's so much better, her only response was "but, what's the point of fast?" #realuserfeedback

7

0

25

@jumbld

Rohit Agarwal

9 months

Been blown away with the support for our open source LLM gateway! thought I'd post it here and leave for the day, but grateful for all the feedback and questions which came our way! gives the TS and OSS community more power

Tweet media one

@jumbld

Rohit Agarwal

9 months

We're open sourcing our AI gateway today. we built this over 200 days managing over a 100B tokens in production. just put it up on HN -> please check it out. More deets to follow on - what is it? - why should you care? - and why tf did we open source it?

Tweet media one

22

62

559

1

3

25

@jumbld

Rohit Agarwal

3 years

I’m so proud of what we’ve been able to build and the team that got us here! The plan for 2022 is even more ambitious, do check out Peppertype if you haven’t. If you have, please check out PH and vote for us!

@ProductHuntHi

Product Hunt Hi

3 years

Congrats to @jumbld as your product ‘’ is a finalist in @ProductHunt ’s #GoldenKittyAwards2021 for the Product of the year category 🎉

0

2

15

2

4

24

@jumbld

Rohit Agarwal

5 months

Super duper excited to be hosting AI enthusiasts in New York for @PortkeyAI ’s LLMs in Prod event along with @flybridge ! The setup looks great at @GunderIPO !

2

1

24

@jumbld

Rohit Agarwal

1 year

At the #warpspeed2023 hackathon and heard this gem from @amasad when asked about GPT wrappers and differentiation “Most software could be called an AWS wrapper” 😅

0

4

24

@jumbld

Rohit Agarwal

6 months

I’ve been sitting on a content idea for a long time. Last year, after reading the FrugalGPT paper, I was so excited about the possibilities that I actually made a neat implementation of all the concepts and derived a practical example of what works. I’ve just thought I could

1

0

24

@jumbld

Rohit Agarwal

1 year

We launched AI Grants Finder on @ProductHunt today. Care to check it out and support us? Please RT. A little bit because you like me, but more because you want AI founders to know about all the grants, credits and investment options available!

Tweet card media

Portkey - Product Information, Latest Updates, and Reviews 2024 | Product Hunt

Portkey makes productionizing your AI app a breeze: - AI Gateway: Ensure app reliability with load balancing and automated fallbacks and manage all AI providers with one unified API. - Observability:...

www.producthunt.com

1

11

22

@jumbld

Rohit Agarwal

3 months

Was woken up by my mom who excitedly called me after listening to this podcast! Wait for it world, she's learning about AI engineering faster than most :D Thanks @HaimantikaM and @hashnode ! Was a lot of fun talking about AI development, the AI Gateway Pattern, @PortkeyAI and

@hashnode

Hashnode

3 months

As software development becomes more complex, @portkey offers an innovative solution in observability and AI. @HaimantikaM and @jumbld discuss recent trends in software development, the role of AI, and the impact of contributing to Portkey. Link in the comments below 👇'

1

2

8

4

1

23

@jumbld

Rohit Agarwal

2 months

For any production service, a 12-hour API outage can spell disaster. Last Thursday, that nightmare became reality for many @AnthropicAI users. But for Portkey users, it was a different story. Our fallback feature ensured a 99.86% SUCCESS rate for Anthropic requests during the

@PortkeyAI

Portkey

2 months

🚨 PSA: Set the Fallback mode ON Last Thursday, the @AnthropicAI API was unstable for 12 hours. Due to Portkey Gateway's fallback feature, 99.86% of the requests made to Anthropic during this time succeeded despite Anthropic being down.

Tweet media one

0

1

7

1

2

23

@jumbld

Rohit Agarwal

8 months

Talking about LLMOps for Production Success today evening! Had a lot of fun talking to companies about their challenges in going to production with LLM Apps and compiled some problem statements & solutions.

Tweet media one

3

1

23

@jumbld

Rohit Agarwal

6 months

Not week zero Not day zero Probably hour zero integration Possible though the time we spent building the core architecture of our AI gateway. Open source, check it out!

@PortkeyAI

Portkey

6 months

Here's a quick Llama 3 cookbook using Portkey Gateway & Together AI. Get started! 🚀

Tweet media one

0

0

6

0

4

22

@jumbld

Rohit Agarwal

6 months

Wait, this works? Caught @ayushgarg_xyz checking out @PortkeyAI control panel for a new (upcoming big) deployment on this big screen!

0

1

22

@jumbld

Rohit Agarwal

1 year

This is definitely the most awaited release for many of us from OpenAI! GPT-3.5 is an extremely fast model and allowing fine-tunes on it enables organisations to take an already awesome model and train it for specific use cases to improve accuracy. Pricing's not too bad either!

Tweet media one

Tweet media two

@shyamalanadkat

shyamal

@shyamalanadkat

1 year

fine-tuning for gpt-3.5 is here!

2

2

30

0

2

22

@jumbld

Rohit Agarwal

3 months

We focus a lot on UX with Portkey. And while nobody really asked us to do this, we wanted even our URLs to be clean. We're making some big updates to Portkey for production use cases and our engg team made these changes alongside! Btw, can you find the easter egg / marvel

@PortkeyAI

Portkey

3 months

🔊 Sound on BIG Update to Portkey — the URL slugs for all pages are now ✨clean✨ but there's more 👇

0

1

11

2

0

22

@jumbld

Rohit Agarwal

1 year

First booth appearance for ⁦ @PortkeyAI ⁩ at the Gen AI summit today. Thanks ⁦ @varshnisree ⁩ for helping manage the booth so well!

Tweet media one

2

2

22

@jumbld

Rohit Agarwal

3 years

As a 90's kid in India - you'd have seen this show - M.A.D on TV. We hosted Rob on Pepper Spotlight yesterday where we talked about Content in 2010 v/s Content in 2021. Fun conversation!

Tweet media one

0

3

22

@jumbld

Rohit Agarwal

2 months

Dear AI Engineers, Lets Ship Fast and Break Stuff. I've been thinking about AI guardrails in production for almost 7 months now, and I'm convinced that using them to "control", "secure", or "block" LLM outputs is not the best use for them ❌. Using them to "guide",

2

4

22

@jumbld

Rohit Agarwal

9 months

Many powerful AI apps ask me to share my @OpenAI API key to start using their features but I've always been hesitant. Would it make sense for me to share a Virtual Key that the platforms can then use instead? I'd have the capability to control where these keys are being used,

8

0

21

@jumbld

Rohit Agarwal

3 years

While we've been dropping product updates since February about Peppertype, we felt ready to launch it to a larger audience on Product Hunt today after 50+ product releases. Would love your review here - Thanks! 🙏

Tweet card media

Peppertype.ai - Product Information, Latest Updates, and Reviews 2024 | Product Hunt

Your virtual content assistant that helps you generate quality content within seconds.

www.producthunt.com

0

5

20

@jumbld

Rohit Agarwal

1 year

I'm beyond thrilled to unveil Portkey's Semantic Cache to the world. Something we've been working on this past month and tuning it be prod-ready. "x-portkey-cache": "semantic" will enable Portkey's semantic cache across your requests.

@PortkeyAI

Portkey

1 year

Introducing 7 Spells of Portkey🪄 Our production-grade features that help your AI app run at scale Day 1: 🧠 Semantic Cache ✅ Serve faster responses ✅ Save API costs Seamless integration with your existing workflows. #Portkey #LLMOps

1

3

9

2

3

20

@jumbld

Rohit Agarwal

4 years

Pepper is building the top 1% creator marketplace AND the platform to enable the huge volumes of content being produced and distributed today. "Every company is going to be a content company." and I see this as a platform that will power the future of content creation.

2

1

20

@jumbld

Rohit Agarwal

8 months

Someone quickly build an AI tool to help me complete my tax proof submissions for the year!

3

1

18

@jumbld

Rohit Agarwal

1 year

Gen AI summit at Santa Clara! Will try sharing interesting tidbits from here.

Tweet media one

2

0

19

@jumbld

Rohit Agarwal

1 year

Have been thinking about interfaces that will evolve around LLMs since @thesephist opened up our minds in a talk a couple days back. This is a beautiful way to use rich text to control prompts for image outputs. Projects like these open up possibilities if not anything else!

@_akhaliq

AK

1 year

Expressive Text-to-Image Generation with Rich Text abs: project page:

24

208

930

0

1

19

@jumbld

Rohit Agarwal

7 years

Love the new @swiggy_in UI on iOS! I'm just gonna order more to get to see the subtle animations and the sweet design!

2

4

19

@jumbld

Rohit Agarwal

1 year

We've setup monitoring across @OpenAI APIs at @PortkeyAI and noticed that similar size prompts for GPT-3.5 and GPT-4 are 30% faster on weekends! Traffic to ChatGPT is probably related to inference time. I've wondered how @Azure ends up offering faster APIs for OpenAI's models.

2

3

19

@jumbld

Rohit Agarwal

1 year

Changed my default search to @perplexity_ai on Chrome because this happened. Google is still awesome for finding stuff fast. But, when researching about anything, finding answers or solutions to problems and even while learning about a topic, Perplexity offers an evolved

2

0

19

@jumbld

Rohit Agarwal

1 year

Would you care if 35% of your OpenAI calls became 20x faster and FREE? We teased Semantic Caching a little bit last month. I'm excited to share a lot more details on how it works in Portkey and why you should care. It needs hand-holding to get started with right now (so reach

@PortkeyAI

Portkey

1 year

💡Did you know that you could be paying more than you need to for your GPT4 calls? If your app deals with repeated queries like those in this image, you're wasting valuable time and resources. What if there was a way to smartly handle these repeat queries? Intrigued? Let's 🧵!

Tweet media one

1

0

2

0

0

18

@jumbld

Rohit Agarwal

1 year

I've spoken to leaders at over 40 companies large & small using LLMs in production over the last 3 weeks. While @chipro 's superb blog covers a lot that companies need to look out for, I heard these 3 as the TOP problem areas companies are looking to solve as they deploy LLMs 👇🏻

2

2

18

@jumbld

Rohit Agarwal

4 years

Looking forward to working with the BITSian founder duo of @SinglaAnirudh and @shekharrishabh8 💯 + Humbled to partner with folks like @dkhare and all the amazing angels who're turning hardcore evangelists. 😄

5

1

17

@jumbld

Rohit Agarwal

5 months

Note to self - this is the first time I'm seeing this. The first of many.

Tweet media one

5

1

18

@jumbld

Rohit Agarwal

8 months

OpenAI Assistant API challenges I've heard from a bunch of users. Care to add +ves and -ves? - No streaming - "Runs" is a cumbersome concept - No control over the RAG journey Compiling a list of thoughts & ideas from companies who've tried it out...

6

1

18

@jumbld

Rohit Agarwal

9 months

The amazing @KevinBlancoZ and I recorded a low-code demo of “chatting with your database” using @theappsmith and @PortkeyAI You can 1. Connect any database 2. Never write SQL again No kidding, build a UI using appsmith on your database and connect it to Portkey prompts!

1

5

17

@jumbld

Rohit Agarwal

11 months

Wrote about OpenAI DevDay's Implications for LLM Apps in Prod. Here's the tl;dr ▪️ GPT-4 Turbo is the new cheap, fast, large context window model. ▪ Function calling has been improved and models talk JSON on demand. ▪ Generate reproducible outputs with

OpenAI DevDay's Implications for LLM Apps in Prod

We hosted a watch party for OpenAI's DevDay on our Discord channel and had a lot of fun discussing everything new and improved that was launched. If you're just catching up, read about all the...

blog.portkey.ai

3

2

17

@jumbld

Rohit Agarwal

8 months

This was probably one of our best "LLMs in Production" webinar. We hosted @RohitChatter from @Walmarttech sometime back on our community webinar to talk about how Walmart achieved some insane stats with semantic search & cache on their latest search experience. (If you

1

2

17

@jumbld

Rohit Agarwal

3 years

Wow, conquest has only got more exciting over time!

@ConquestBITS

Conquest, BITS Pilani

3 years

In an ever-changing ecosystem, innovation & evolution are the only constants. Conquest is back with its 18th edition - a brand new identity, an inspiring vision, & a driven team. Check out & follow @ConquestBITS to gear up & #MakeAMark

10

30

102

1

2

17

@jumbld

Rohit Agarwal

1 year

I had a great time talking about The Confidence Checklist for LLMs in Production at the LLMs in Production Conference II by @mlopscommunity . You can check out the whole presentation here -

Tweet card media

The Confidence Checklist for LLMs in Production - Video | MLOps Community

For 2 whole years of working with a large LLM deployment, I always felt uncomfortable. How is my system performing? Are my users liking the outputs? Who needs help? Probabilistic systems can make...

home.mlops.community

0

1

17

@jumbld

Rohit Agarwal

3 years

What???!!!

Tweet media one

@LoopinHQ

LoopinHQ

3 years

What does your calendar say about your year? 2021Looped is out now. Plus, holiday surprises await🤫 Check it out and let us know what you think.

Tweet media one

0

4

20

3

2

17

@jumbld

Rohit Agarwal

5 months

The paper's (by @ChenLingjiao , @matei_zaharia and @james_y_zou ) summary is still one of the most viewed papers on the @PortkeyAI blog and I finally got the time through last week to compile all our notes on implementing this! Big thanks to Lingjiao, Matei and James for

1

2

17

@jumbld

Rohit Agarwal

5 years

I'm super proud of the @FreshworksInc support team. They're the most empathetic set of people I've worked with. I'm excited that we opened up our CSAT ratings to the world in the spirit of transparency! It's live here - #custserv

Tweet card media

Customer Happiness Report | Freshdesk

We at Freshdesk believe in delivering moments of WOW to our customers every single day. Find the Freshdesk Customer Happiness Report here!

www.freshworks.com

1

5

17

@jumbld

Rohit Agarwal

5 months

The 3rd crappy 16 hour @airindia flight AI180. Even though I just became a gold member of their loyalty program, I don’t think I will fly the airline for a few years as they get upgraded. Across the 3 flights - screens didn’t work - no utilities at all in economy - unclean

10

1

17

@jumbld

Rohit Agarwal

2 months

Our opinion on Gateway + Guardrails - And docs for the curious -

Tweet card media

Guardrails | Portkey Docs

Ship to production more confidently with Portkey Guardrails on your requests & responses

docs.portkey.ai

0

2

17

@jumbld

Rohit Agarwal

9 months

I feel oddly sad about the text-davinci-003 deprecation. I remember, at @peppertype_ai we had such a significant boost when it launched that @prrranavv and I eagerly wrote a blog about how crazy good text-davinci-003 was. That too, on the very day it released. @Kyle_L_Wiggers

Tweet media one

3

2

16

@jumbld

Rohit Agarwal

10 months

Going strong. I'm particularly excited to try JSON mode in Portkey's Prompt Library! Thanks for all the help the push on this @robnsngh

@robertnishihara

Robert Nishihara

@robertnishihara

10 months

Today we're releasing new @anyscalecompute features. ☑️ JSON mode ☑️ function calling ☑️ new models This has been a massive gap in the open source LLM ecosystem. 📜 JSON mode: Outputs valid JSON based on your schema requirements. 📞 Function calling: Lets the LLM choose a

6

19

140

1

3

16

@jumbld

Rohit Agarwal

10 months

Gearing up for announcements is always fun as a founder.

2

0

16

@jumbld

Rohit Agarwal

4 months

Brainchain is building a compelling real-world application of AI - exploiting supply chain disruptions to protect and grow revenue. Will they pick @PortkeyAI ? 🙃

Tweet media one

2

3

16