.png){kind=small}
The last few weeks have been very interesting in the AI space. We saw the release of a new updated version of ChatGPT from GPT-3.5 to GPT-4. Within a couple of days, Google soft-launched their competitor AI chatbot, Bard (available in the US and UK). Adobe dropped Firefly, Canva dropped their own image AI, and Bing added DALL-E.
We also saw Opera browser incorporate AI prompts using GPT and DuckDuckGo using it to summarise search queries. But by far the biggest development was the release of ChatGPT plugins.
Plugins are big for two obvious reasons. Number one, now that ChatGPT has access to third-party tools it can give more accurate answers from trusted sources and databases. For example, you may ask the GPT to solve a math equation, and now it can use Wolframa to provide an answer. And secondly, plugins are big because now GPT can automate tasks across multiple tools — it’s not just responding, it can now create actions.
It’s going to be awesome when more plugin developers get their hands on the documentation. We’ll see some really cool automation paths in the coming weeks.
I want to discuss a few ways that developers can and have been using these plugins but first let’s talk about the actual plugins so you can get an idea of what’s available at the moment.
Third-party plugins
According to the OpenAI blog, there are only 12 plugins currently available to a small set of paid users. After an Alpha period, more developers will get access and API users will be able to integrate plugins into their products. Here are the available third-party ChatGPT plugins:
- Expedia: Flights, cars, hotels, and travel
- FiscalNote: Real-time data sets for legal, political, and regulatory data
- Instacart: Order groceries from your local grocery stores
- KAYAK: Flights, cars, hotels, and travel
- Klarna Shopping: Search and compare prices from online shops
- Milo Family AI: Co-pilot for parents, powered by GPT-4
- OpenTable: Restaurants and restaurant reservations
- Shop: Search for millions of products from the world’s greatest brands
- Speak: Learn how to say anything in another language
- Wolfram: Access computation, math, curated knowledge & real-time data
- Zapier: Workflow automation across 5000+ apps
The best NEW plugins
To be honest, all of these plugins will be very useful to many people. Some people have been comparing it to an app store. The potential is crazy. But I only want to focus on a few of these plugins I think are the most interesting.
1. Browsing has been added
Up until now, ChatGPT-4 hasn’t had access to live data, for example:
With the browsing plugin, we can get up-to-date information. Your research will be current and you’ll be able to ask more specific questions. The plugin also lets you see exactly the path the chatbot took to arrive at the answer. It’ll display a list of what was searched and the pages it read to arrive at its final answer. This allows you to fact-check the bot and source data.
I’ve also seen it used for solving code issues. It’s reassuring to know that the bot is pulling information from StackOverflow or relevant documentation to solve your queries.
2. Code interpreter
ChatGPT has so far been a very impressive coder. It can write and read in multiple languages. It can QA code, or provide constructive feedback. But if you’ve used it for coding, you’re aware that a lot of the output has to be put into a different environment to test it. With the Code Interpreter (CI) plugin you can now do much more of the development inside ChatGPT.
I would recommend reading ChatGPT + Code Interpreter = Magic by Andrew Mayne. He goes into depth on the CI plugin and shows its various uses. If you don’t have time I’ll give you the TL;DR.
What’s new:
- Downloadable files
- QR codes
- Images
Use cases:
- Solving mathematical problems, both quantitative and qualitative
- Doing data analysis and visualisation
- Converting files between formats
ChatGPT can provide you with a downloadable link tailored to the file type you ask it for. Sound file types and .txt were demonstrated. Not only do you see the code snippet but you’ll also see the result.
In the article I mentioned above, Andrew creates mini-games, sound files, working QR codes, face-finding algorithms, images converted to ASCII images, calendar invites, drawings, and an animated graph.
3. Retrieval
Retrieval is like the admin plugin. The retrieval plugin enables ChatGPT access to personal or organisational information sources. With access to data sources, files, notes, emails, and documentation, you’ll be able to query the bot for any task. For example, a HR person might want to know an employee's tax details. They query GPT and it gives them the number. They could even ask for a list of all employee tax IDs, turn the list into a table and create a presentation all within the same environment.
ChatGPT can be a data management tool. The tedious task of creating a report is going to be almost eliminated. Think about it, you ask for a list of data (maybe it’s linked with Power BI or Google Analytics) without having to create a ticket for the data team. You query the bot about the data and then turn it into a presentation or animated graph. What could have taken a week and crossed teams, takes you five minutes?
Use cases:
- Indexing and searching documents
- Summarising email and draft response
- Data retrieval
- Searching for lost or missing files
- General queries
4. Zapier
This is where it gets fun! Using the browsing or retrieval plugin and the Zapier plugin you can set up automation sequences to trigger actions. So from within the same environment you can search for data, turn that information into a report or image and send it off to one of your email contacts.
That’s just a simple example, I’m sure you can think of a bunch of tasks within your role or within a department that can be completely automated with the integration of ChatGPT.
This was a really cool example of an automated sequence in GPT, not 100% sure if this is Zapier though.
Use cases:
- Draft message and send it to team Slack channel
- Pull latest email lead and generate a response
- Search Notion database for any item and add or update it directly inside Notion
These are the example provided by Zapier, and to be honest I don’t think these are great examples. It’s easy enough to open slack and send a message — it doesn’t really make sense to set up Zapier for that, especially when it’s a paid app. Generating weekly reports or data retrieval tasks that are more or less predictable events is the first thing that comes to my mind.
READ MORE: Try Zapier's ChatGPT
5. Swift Playground
Now this plugin isn’t available in the store it was created by a Beta user. He created a Swift Playground plugin so that he could quickly create iOS applications. In the example below he is creating animal flashcards.
This next user (with no Swift experience) creates a mostly functional iOS app in less than two hours with payment and OpenAI integration. The idea for the app is you tell it what’s in your fridge and your fitness goals and it tells you what you should eat (and can order from Instacart). Very impressive.
More ChatGPT-4 plugins…
There are a bunch of secret plugins not yet accessible. A user was hacking the ChatGPT API and found over 80 hidden plugins. For a few hours, it was possible to use these plugins but OpenAI was quick to fix the big. Thankfully there are some screenshots of the hidden plugins so we know what’s coming.
Some of the secret plugins:
- DAN (change personalities)
- United Nations (documentation)
- Slack
- NBA Standings
- Crypto Prices
- Vogue
Okay, so nothing to hold your breath over, but still cool!
Response to the launch of plugins
Overall, plugins are a game-changer and have just added a lot more value to a product that is already groundbreaking and changing how we work in real-time. Let’s just wait and see what developers out there can do with this technology once it’s opened to the public. To leave you, here’s Brock asking ChatGPT to summarise the public response to the new plugin update.
READ MORE:
