Introduction

That was a pretty awesome talk, by the way, for anyone that wants to look at like actual the hallucination problem.

1This is something that I've been going on about over and over how the biggest part around hallucinations actually is a theory, not a solved problem, but you can definitely get rid of a lot of it by techniques like this, which I think very few people know, which means you get a lot of companies getting Or not rolling out LLMs because they're scared of hallucinations not knowing that you can get rid of a lot of them with these techniques.

The Hallucination Problem in LLMs

I'm gonna do something entirely differently and I actually think it's gonna hallucinate a lot. So I wanted to give you an idea of how far with today's technology you can push some of these automations without any code, which I think everyone in the room here will be able to reproduce what I'm doing. and how far that spread goes.

Addressing Hallucinations with Techniques

So we run this AI Mastery program and this is like the, when I say mastery, it's really competency because it starts from, basically you don't need to know anything about ChatterBT, but the last part that we go through is we create an email bot. So I'm going to live create an email bot that's gonna try and respond to any of your emails that you'll be able to live send to my email to answer questions about the AI Mastery program we run.

And I'm going to try and do that within the next 15 minutes.

Demonstration: Building an Email Bot

So how many of you are familiar with Zapier? Yeah, there are a few.

Familiarizing with Zapier

How many of you have built a bot on Zapier before? one. Okay.

Yes. So now you'll go from knowing about it to actually seeing one.

Um, so I'm just going to, there's a big create button at the top, nothing more to it. You create his app and you have, they rolled out about two months ago, an AI, uh, creation, Tool you can just tell it what you want to create rather than going through the steps It really barely saves you any work because just setting up the trigger and the actions goes there, but it's a nice It allows them to say that they now are in AI enhanced.

Creating an App with AI Tools

So I am gonna use it just to showcase it so I'm gonna say When an email hits my gmail account Send it to chat GPT and and use the reply from chat GPT to send another email from my Gmail account.

And I hit generate. And the only thing it's going to do, it's going to try and figure out automatically how to populate the trigger and how to add more actions. It actually did it right.

So it's saying what it understood is when an email hits my account, send that, step two, start a conversation with ChatGPT, and step three, send an email from my account. Very simple.

Now we have 15 minutes, so I am going to run through this slightly faster than I normally would. But you can see the three steps that have been populated.

And so here I'm going to choose my Google account, which is demos at mindstone.com. I'm going to hit continue.

1Extremely important step is I only want this to trigger when there is a new email in my inbox. First time I did this, I didn't realize it was looking at new emails in my sent items. which means I was doing the demo and every time I was answering, there was a new email in my sent items, which re-triggered the Zap, which then meant that by the end of the demo had sent more than 10,000 emails and I was locked out of my Gmail account. so this is the type of stuff that happens with with sapir which is that you can build these loops that do an infinite amount of actions and before you know it you have hit an extreme amount of actions so with great power comes great responsibility which is test your stuff otherwise you have problems like i have so here i'm only going to trigger this when there's a new email in my inbox

Email Trigger Setup and Testing

And I can test the trigger. I actually did this demo earlier today because we had a level three AI mastery course earlier today. So it's going to pick up one of them.

So here there is an email. So the reason I do tests, by the way, is that it populates your data for the future steps. So it means that you can actually see what you are what you're sending into step two and step three.

So here there's an email that came in interest in participating into the program. Not sure if I can afford it. The whole thing.

So I'm going to select this test record. So what it's done here, it's gone into my inbox and selected one email as a test record so I can use it. And then I hit Continue. It's now moving me on to step two.

So now I'm looking at step two, and it's a ChatGPT step. So I'm going to send this to ChatGPT. OK, interesting.

There's a new version available. I'm not going to use that. Zapier is telling me there's a new version of the ChatGPT app available, but I haven't tested it with that one yet. So I'm going to live demo it with this one.

Integrating ChatGPT into the Workflow

I'm going to select conversation with an assistant. So there were multiple different things I could do with the ChatGPT API. Conversation with an assistant, for those of you that are not familiar with it, the word assistant is the equivalent of the word GPTs that they use in the commercial application. So assistants are pre-configured versions of ChatGPT that you gave a bunch of instructions to, and you can use them through the API. And somehow in the API, they called them assistants. I don't know why they don't call them the same way, but it is what it is.

So I set one up earlier. So I'm going to choose my account. I've got my ChatGPT account here. Continue on that.

And I'm going to say the message I'm going to send to this assistant, this GPT, is this is the email thread. And I'm going to select the body of the email that came in on step one. So I'm literally just passing that email thread to the assistant, and I'm not gonna choose the assistant, which is called the AI and LND Q&A demo, and I will actually quickly show you. So it's very simple to do, but I'm not gonna go into detail on that.

Here, you go to platform.openai.com, you can create an account, you get $20 of free credits. $20 of free credits on the API actually gets you thousands of requests, so it actually goes much, much further than most people think. And then on the left-hand side, you have assistants. And you can create them here.

And there are a bunch of instructions that have been created. So here, I created an assistant that, and those of you that have seen me do a GPT demo, it follows the structure of the template that I use for my GPTs. It inspects the email thread that it's given. It consults the uploaded knowledge documents about the offering that we have for AI mastery. It thinks about two great non-obvious questions that aren't answered in the email thread. which helps the assistant qualify the interest in the program and then respond to the query using the template.

And this is a trick around actually making ChatGPT write emails that make sense, is the template is defined by, hey, first name, give it like an introduction, something to break the ice, answer the actual question that's asked, and then add the questions that I'm asking of it, right? So you're basically constraining it, which makes it feel like a genuine email that would have come from a human, but at the same time, it's entirely automatically generated. So it's like a templated response, which really works.

Okay, so I'm going to do that. I select AI and L&D. All the configurations are the same. I'm going to test this step.

Obviously here it's actually testing the steps, so it's going to OpenAI, placing the request and then coming back, which does take a few seconds. The other thing is that if you do this demo at 10 a.m., it works instantly. If you do it at 5 p.m., 6 p.m., 7 p.m., the US has woken up and the API of OpenAI is slower. There's more people placing requests, but hey, it's come back, so here we go.

and you have to feel through the answers a little bit but you can see okay here it's actually answered something thanks for getting back to us with your concerns i understand that you're finding a program fits the program fits your budget it's basically it's constructed an answer okay i'm going to continue and now i'm going to say send email so it's going to step three step three is sending an email from demos at mindset.com i'm going to send this to the from email in the first step. So whoever sent me that first email, I want to reply back to them.

I am going to say it's coming from demos at MindStone, the name Joshua and the subject. Subject is just re whatever the subject was of the original original email. which is a very standard way of naming a reply.

The body type is plain. You don't really need to know this, but basically you can write a full HTML email. Basically it makes it look fancy. You can add images. You can do everything you want with it. But here I'm just going to do plain text.

And the text of the email is going to be what I got back from ChatGPT. So I'm going to look at where is this assistant response parts is. This is the bit that is actually the most annoying is there's so much stuff coming back from ChatGPT that you have to wait through and get the right value. OK, here you go. Full response. That's the variable that I actually want from from step two. And then I'm also going to add. the email that came back initially. So that I have both, it looks like an actual reply.

And I think that is it. I'm gonna hit continue. I'm not gonna test this step in this case, because if I test this step, it's actually gonna send an email to the test record that I selected, which was at the demo before. In this case, I don't want them to get another email, because they don't know what this is about, because they are not aware that I'm doing this demo at the moment. So I'm skipping the test. And then I'm going to publish this.

Engaging the Audience in a Live Demo

And now I would like you all to send me an email, take out your phones, send an email to demos, D E M E D E M O S at mindstone.com. I'll actually there. That's the email. send an email to demos at mindstone.com about anything about AI mastery, about how you, a question about upskilling on AI in the company, a question about individuals, about teams, whatever the thing is, please send me an email so I can continue the demo. That would be great.

Yes. Yeah. Yeah, it is actually. Yes, that is indeed.

Document Uploads and Query Handling

Yeah, so I've uploaded documents indeed. And actually, it's a very good question because you can see in the instructions that I added, and this is why my question was initially, is in the instructions, I added, you only ever respond to queries on Mindstone's AI Mastery offering and politely refuse to answer anything else. That is to avoid, I know probably one of you in the audience asked a question that has nothing to do with AI mastery and you're hoping it's going to come on screen. Hopefully that doesn't happen because it's going to politely refuse to answer that question.

Okay. So I'm just going to refresh this. Now, these apps, when I get emails, they're not instantly triggered. So it's going to take 30 seconds or so or a minute before this app realizes that there is a new email in my inbox.

And then I'll run you through what happens with the rest. Any questions actually before this will fill the Ah, there we go. Actually, it's there.

Well, no more questions. You can see, okay, here you have this little button on the left-hand side, Zap Runs, which basically shows the Zap running through its cycle. So it's detected there's an email coming in. So I'm going to select one, and you can live look at how it's executing.

So right now... It went to one, it has not yet executed two. Why, how does it, ah, there we go, okay. It's live going through. So you can see how it's executing the steps and where it's stuck.

So this email that came in, I hope I can demo this. Okay, does the Mindstorm program in AI mastery offer scholarships? That is the email that came in.

You can then look at step two. So you can see that in step two, the data in is the email just fed. Does the email, uh, does the Mindstone program in iMastery offer scholarships? The data out is what I get from the step.

What, uh, what the agent has responded. So thanks for reaching out with your question. After checking our resources, I couldn't find specific information regarding scholarships. This might indicate that Mindstone does not have a scholarship offering.

However, programs and offerings can often change, so it would be best to directly contact our team for the most up-to-date information. Interesting. As you're interested in a topic, have you thought about what specific outcomes or objectives you're looking to achieve through the program? And it could help tailor learning experience for your exact needs or how would you how could you?

How could integrating the master program into your team or personal learning journey impact your productivity or professional development goals? Actually, not too bad of a reply if you're thinking about like hey, there is a lead You're not quite sure someone's asking about about scholarships what would be the questions that help you qualify this in a better way and And that email should have gone out.

So have you all received a reply based on this bot now? This took 10 minutes actually to set up. Now, you can go much further with this.

And I wonder if I have the time to go and do the thing. Yeah, we've got Dominic as well. I'll finish here with one small thing.

The way that this becomes even more powerful is that what I normally would run through is an additional step that you insert between step one and step two, which is another assistant that rates the ability of the assistant to answer a particular query. So it looks at the documents, it looks at the question that comes in, it then rates based on the information that I have, how confident am I that I can answer this question in a way that leads to the outcome that I want.

I get the assistant to give me a rating from zero to 100 on how confident it is. If it is under 90, then, I loop in, rather than getting the answer from ChatGPT, instead I create a tree.

So you can actually create basically two different choices. If the response that comes back is under 90, it's like loop in a team member and instead say, hey, sorry, I don't know how to answer this question, but Joshua in CC can answer this question. Instead, if it is more than 90% certain that it can answer the question, it just answers the question instantly.

Advanced Automation Techniques

And so you can start to see how you can build these things for, I mean, I know you're all running a whole bunch of programs internally. There are 50,000 questions being asked as you're rolling them out. 10 minutes it took for me to take a document, put it into an assistant and have an email bot that just answers those questions so long as it's actually certain that it can, or not certain, but very highly likely that it has the context in order to answer.

Conclusion

Yeah, and it is working today. So hopefully that gives you an idea of what's possible. Thank you very much.