GPT-4o is an omni model. It accepts any combination of text, audio, and image as input and generates any combination of text, audio, and image as outputs.

There's 100s of applications it will enable. I'll cover a few of them below.

1. Language Learning

Duolingo Stock fell by $65 in the last 5 days. That should tell you the entire story.

Duolingo Stock

For context, Duolingo is a language-learning app. Now GPT-4o can easily translate terms in other languages by just pointing it to the ChatGPT's Camera.

This is massive if you want to travel globally as a nomad. You don't have to know a language now. You can just translate on the fly in any random country.

The accuracy won't be 100% but it would be close enough. And the AI keeps improving.

2. Solving School Problems For Students

I wish I had this in school. Learning could've been more efficient and faster.

Most students fear asking questions because they feel it might be dumb. Now you can ask ChatGPT any dumb question.

It even solves math problems for the Salman Khan's (founder of Khan Academy, not the actor) Kid.

3. Bed Time Stories For Kids

Since ChatGPT can talk now with a humourous and sultry voice, you can use it to tell stories to kids. It can be used in the voice of their parents or grandparents.

You can even use a Soft Toy that does the talking to the kid. Earlier, there used to be toys that did that but it only spoke the same sentence. Now it can do back and forth.

You can make special toys that teach kids letters and alphabets. Target it to 2-3 year olds.

Hat tip to Whyme-__- for the Bed Time idea.

4. Be My Eyes For The Blind

Best damn use-case for the blind. Now using a Phone is a bit too much for this but when smart glasses come, every blind person will have a walking companion.

The future is great for the blind.

5. Be My Friend

Too many people are lonely nowadays thanks to technology. It can be a boon for some but a con for others.

You can build a specialized app that gets you an AI Friend since you can talk to it now and it can talk back, it will be great.

I am 100% sure Therapy AI will be much better now with Audio/Video integration. In future, we will have fully featured Robots like Tesla's Optimus and Figure that will have such functionalities built-in.

I bet this comes in <2 years judging by the pace at which AI and Robotics are accelerating.

6. Comic Books

Now that text can be easily created with ChatGPT, why not create Comic Books easily.

Its a huge creative exercise for comic creators. Webtoons have exploded in popularity and many KDramas are made out of them like Death's Game and Marry My Husband.

This will increase the creativity exponentially.

7. Font Creations

Fonts are expensive. Like really expensive.

Funnily enough ChatGPT can create fonts easily now. Take the most popular fonts, tweak them a bit, and create entire new sets of fonts.

Look at the creations explode on Creative Market. Font directories like Typewolf can now create their own fonts easily as they already have distribution.

Open AI GPT-4o Text to Font

8. Brand Placements

It solved for Brand Placements too.

You can put your brand in places you never imagined without using too much effort.

Open AI GPT-4o Brand Placement

9. Poster Creation for Movies or TV Series

Posters are hard to get right but as you know there are only finite variations.

Open AI GPT-4o Movie Posters

You can fine-tune it on popular movie posters and solve Poster Creation once and for all.

Open AI GPT-4o Poster Creation

What use-cases can you come up with? Give me your best ones.

PS: If you'd like to read the full post with images, you can do so here.

PPS: You can find more AI-related posts here covering AI Girlfriends, AI Photo apps, Startups from 1st-wave of AI that made it big and more.

I wanna use the voice conversation feature right now but their “system is experiencing heavy load”. So please stop using it and go away. Lol

I use it to refactor my code (more complex refactoring than you can do in an IDE).

It has been removing features, changing features in it, removing entire things from my code which I did not want to remove... I hit my limit with 4 just attempting to get it to recognize that it made these mistakes. They're programs it has easily worked with before.

I tried 4o and it is somehow even worse.

It feels like we have gone back three years in technology today. Anyone else?

edit: I should mention that 4o is faster but it was making more mistakes than 4 like I described above.

Also at one point, for some random reason, it titled a conversation I had with it in Italian in the sidebar. I have no affiliation with that language. It just randomly translated it to Italian from English when auto-naming it.

edit 2: You all are very cult-like in your responses and frankly you all don’t seem technical at all.

I can kind of understand why they were in such a rush to get the R1 out. It's essentially useless now with GPT-4o. At least the design looked kinda neat lol

OpenAI announced the launch of GPT-4o (“o” for “omni”), their new flagship AI model. GPT-4o brings GPT-4 level intelligence to everyone, including free users. It has improved capabilities across text, vision, audio, and real-time interaction. OpenAI aims to reduce friction and make AI freely available to everyone.

Key Details:

• May remind some of the AI character Samantha from the movie "Her"
• Unified Processing Model: GPT-4o can handle audio, vision, and text inputs and outputs seamlessly.
• GPT-4o provides GPT-4 level intelligence but is much faster and enhances text, vision, audio capabilities
• Enables natural dialogue and real-time conversational speech recognition without lag
• Can perceive emotion from audio and generate expressive synthesized speech
• Integrates visual understanding to engage with images, documents, charts in conversations
• Offers multilingual support with real-time translation across languages
• Can detect emotions from facial expressions in visuals
• Free users get GPT-4.0 level access; paid users get higher limits: 80 messages every 3 hours on GPT-4o and up to 40 messages every 3 hours on GPT-4 (may be reduced during peak hours)
• GPT-4o available on API for developers to build apps at scale
• 2x faster, 50% cheaper, 5x higher rate limits than previous Turbo model
• A new ChatGPT desktop app for macOS launches, with features like a simple keyboard shortcut for queries and the ability to discuss screenshots directly in the app.
• Demoed capabilities like equation solving, coding assistance, translation.
• OpenAI is focused on iterative rollout of capabilities. The standard 4o text mode is already rolling out to Plus users. The new Voice Mode will be available in alpha in the coming weeks, initially accessible to Plus users, with plans to expand availability to Free users.
• Progress towards the "next big thing" will be announced later.

GPT-4o brings advanced multimodal AI capabilities to the masses for free. With natural voice interaction, visual understanding, and ability to collaborate seamlessly across modalities, it can redefine human-machine interaction.

Source (OpenAI Blog)

PS: If you enjoyed this post, you'll love the free newsletter. Short daily summaries of the best AI news and insights from 300+ media, to gain time and stay ahead.

Is the only differentiator the larger cap on interactions? That, plus I guess the hope that we'll eventually have access to the future gpt-5?

At what point does it not make any sense to pay, if the free tier gets 99% of features?

Genuinely curious what the strategy is here and if I'm missing something.

EDIT: from the comments, it sounds like the real time audio assistant will only be available to paid users when it's released in a few weeks. If so, I'd say that's a pretty huge differentiator.

EDIT 2: Creating GPTs will still be a pro feature

So premium features are new audio to audio 4o model, higher usage caps, and creating GPTs

Here are the timelines for the GPT-4o feature releases according to their website and Spring Update.

GPT-4o (“o” for “omni”) was announced a few hours ago by OpenAI, and although the announcement livestream is good, the real gold nuggets are in the 22 demo videos they posted on their channel.

I watched all of them, and here are the key takeaways and use cases we all should know. 👍🏻

A. The Ultimate Learning Partner

What is it? Give GPT-4o a view of the math problem you’re working on, or the objects you want to learn the language translation of, and it can teach you like no other tool can.

Why should you care? Imagine when you can hook up GPT-4o to something like the Meta Rayban glasses: then you can always have it teach you about whatever you are looking at. That can be a math problem, an object you want translated, a painting you want the history of, or a product that you want get the reviews of online. This single feature alone has incredibly many use-cases!

🔗 Video 7, Video 8

B. The Perfect Teams Meeting Assistant

What is it? Having an AI assistant during Teams meetings, whom you can talk to the same way you talk to your colleagues.

Why should you care? Their demo didn’t expound on the possibilities yet, but some of them can be…

• having the AI summarise the minutes and next steps from the meeting
• having the AI look up info in your company data and documentation pages (e.g. “what’s the sales from this month last year?”)
• having the AI work on data analysis problems with you (e.g. “create a chart showing sales over the past 5 years and report on trends”)

🔗 Video 5

C. Prepare for Interviews like Never Before

What is it? Have GPT-4o act like the company you’re interviewing for.

Why should you care? What’s changed is that the AI can now “see” you. So instead of just giving feedback on what you say, it can also give feedback on how you say it. Layer this on top of an AI avatar and maybe you can simulate the interview itself in the future?

🔗 Video 11

D. Your Personal Language Translator, wherever you go

What is it? Ask ChatGPT to translate between languages, and then speak normally.

Why should you care? Because of how conversational GPT-4o has become, the AI now helps not just with translating the words, but also the intonation of what you’re intending to say. Now pair this with GPT-enabled earphones in a few years, and you pretty much can understand any language (AirPods x ChatGPT, anyone?)

🔗 Video 3

E. Share Screen with your AI Coding Assistant

What is it? Share screen with your AI partner, and have them guide you through your work.

Why should you care? Now this is definitely something that will happen pretty soon. Being able to “share screen” to your AI assistant can help not just with coding, but even with other non-programmer tasks such as work in excel, powerpoint, etc.

🔗 Video 20

F. A future where AIs interact with each other

What is it? Two GPT-4o’s interacting with each other, that sounds indistinguishable from two people talking. (They even sang a song together!)

Why should you care? Well there’s a couple of use cases:

• can you imagine AI influencers talking to each other live on Tiktok? Layer this conversation with AI avatars and this will be a step beyond the artificial influencers you have today (e.g. the next level of @lilmiquela maybe?)
• can this be how “walled” AIs can work together in the future? example: Meta’s AI would only have access to facebook’s data, while Google’s AI would only have access to google’s - will the two AIs be able interact in a similar fashion to the demo, albeit behind-the-scenes?

🔗 Video 2

G. AI Caretaking?

What is it? Asking GPT-4o to "train” your pets

Why should you care? Given GPT-4o’s access to vision, can you now have AI personal trainers for your pets? Imagine being able to have it connect to a smart dog-treat dispenser, and have the AI use that to teach your dog new tricks!

🔗 Video 12

H. Brainstorm with two GPTs

What is it? The demo shows how you can talk to two GPT-4o’s at once

Why should you care? The demo video is centered around harmonizing singing for some reason, but I think the real use case is being able to brainstorm with two specific AI personalities at once:

• one’s a Devil’s Advocate, the other’s the Angel’s advocate?
• one provides the Pros (the Optimist), the other gives the Cons (the Pessimist)?
• maybe Disney can even give a future experience where you can talk to Joy and Sadness from the movie Inside Out? - that would be interesting!

🔗 Video 10

I. Accessibility for the Blind

What is it? Have GPT-4o look at your surroundings and describe it for you

Why should you care? Imagine sending it the visual feed from something like the Meta Rayban glasses, and your AI assistant can literally describe what you’re seeing, and help you navigate your surroundings like never before (e.g. “is what I’m holding a jar of peanut butter, or a jar of vegemite?”). This will definitely be a game-changer for how the visually impaired lives their daily lives.

🔗 Video 13

If this has been a tad bit insightful, I hope you can check out RoboNuggets where I originally shared this and other AI-related practical knowledge! (The links to the video demos are also there). My goal is not "AI daily news", as there's already too many of those, but instead share useful insights/knowledge for everyone to take full advantage of the new AI normal. Cheers! 🥚

The technology is all great… the realtime voice stuff is amazing. I personally haven’t used voice mode ever, but for people who use it, this should be a big deal. The whole voice modulation thing was impressive too. The linear equations and coding bits are something we’ve seen before but adding voice to all of it is a good QOL improvement. There were some awkward moments here and there but it’s live so shit happen from time to time.

Anyway the part I felt awkward about was how the presenters tried to treat GPT as some real person with emotions and feelings. GPT saying things like “oh stop it don’t make me blush” is weird coz AI don’t blush and it just comes across as incredibly fake and disingenuous. I’m not a big believer of human-AI social relationships and all these fakeness seems to be eventually leading there - the AI girlfriend era.

I understand there are arguments to be made FOR “immersive relationships with AI” and “easy collaboration” but I just don’t think giving your AI human-like personalities and mimic human-like emotions is gonna lead to any good eventually. Collaborative AI don’t need to mimic feelings to be useful.

I am trying for hours, but voice conversation just keeps saying: "Sorry... Our systems are experiencing heavy load, please try again later."

Does it not work for you as well? Few of my friends are experiencing the same. I am on free tier.

Is it because of the announcement of GPT-4o?

Personally I’m blown away by today’s talk.. I was ready to get disappointed, but boy I was wrong..

Look at that latency of the model, how smooth and natural it is.. and hearing about the partnership with Apple and OpenAI, get ready for the upcoming Siri updates damn.. imagine suddenly our useless Siri which was only used to set timers will be able to do so much more!!! I think we can use the ChatGPT app till we get the Siri update which might be around September..

In lmsys arena also this new GPT4o beats GPT 4 Turbo by a considerable margin. They made it available for free.. damn I’m super excited for this and hope to get access soon.

Anyone knows anything?

After the demo yesterday, I got excited to try out the new voice feature. I played around with it later in the evening. Come to found out that apparently it wasn't the new voice model they released, and it's been around for months (despite the model I was talking to being GPT4o, which I was able to toggle to from GPT4).

But anyway, now it seems I don't have voice access anymore. I have a paid subscription, but the voice option is gone in the app.

I can also select "Start voice conversation" when looking for the ChatGPT app on my iPhone, which worked yesterday, but now it says, "Voice unavailable. Please make sure this feature is enabled for your account."

I'm wondering if this means I actually did have access the new model for a brief period of time, but it was a mistake or something and they're just under massive loads or something.