Today’s topic is AI. True to my word, I promise to be more positive, but there will be the odd rant.

Can’t help myself with a nice lovely rant.

I explored training algorithms last Sunday.

First I followed Unreal’s “Learn to Drive” tutorial and it was fruitless sadly. I finished everything and clicked play to see my agents learn to drive to find that my graphics card is too new for the python libraries that Unreal 5.7 Learning Agents plugin uses. My card is 2 years old now and this feature is experimental. Anyway, I could not run the learning algorithm. And I did try. I tried manually changing the requirements.txt in the intermediates folder to be the newer version of torch but the requirements are overwritten every time you regenerate the project files. I tried looking at the experimental plugin build.cs and uproject file to change the required libraries there, but the sections on it are hashed and encrypted so I had no idea what I was changing. So, I decided to stop there, knowing from experience it was just going to be a headache. I moved on to Godot and the reinforcement agent’s plugin. And I tested it first this time. Roughly ten minutes to get it working. Then another 2 or 3 hours to make my own drone racing training sim thing.

Look!

It was great fun.

They scatter around like complete idiots. And now I need to learn how to give the right sort of observations to the agent so that it can learn. They currently only know how to accelerate, strafe, turn and some details about distance and relative velocity to the target checkpoint. These parameters are not enough to learn from. I need to give better observation and better rewards and punishments. So that’s what I’m doing next.

Next section! How Large Language Models work.

Prepare yourself for a large amount of text

Large Language Models weigh what word should come next in a sentence using the context from the previous words.

They use tokens which are like parts of words or whole words but for a loose but close reference to the real way these models work, we’ll call tokens words from here. They use weighted guesses where millions of words are checked depending on the sample taken from the collection of words in the model.

There are often billions of words in the model, and usually the top 10 percent get used when generating the next word in a sentence.

It’s a bit like shouting at a billion words in a massive box and shaking the box to see what words come to the surface, and then grabbing a handful of those words, hoping that it’ll be a good bunch.

This is why there was so much hallucination in the beginning. I hope this is making sense.

So, with our handful of random words, we select the nicest, weightiest, looking one and that’s the next word in the sentence being generated. Exact same box for the next token as we shout and shake it, and the next, and the next.

If the algorithm uses a deterministic random seed, then by using the same seed we can get the same sequence of handfuls we had before, but these floating-point numbers can be funny when dealing with such a very large number of things so you might get a slightly different handful each time, like the chance of a coin with two heads faces landing on its edge. We can often generate the same response nearly every single time. So, there’s no real intelligence in this “artificial intelligence”.

The current form of AI can be boiled down to random weighted words.

Also, you’ll rarely have access to set the random seed in case you’re wondering why your responses are always different. I won’t get into how they are weighted against the context. I don’t think I’d know when to stop if I keep going.

So that’s how they work. And this highlights a problem.

We end up at the most frustrating thing about working with large language models. The closer you get to making your context clearer to the model for what you need, the dirtier those handfuls of words get. Your context for the prompt gets larger or clearer to you, but the words in your handful are still the same generic ones that best fit the overall context as they have the most relative weight to the whole context. This makes the weights of each word in your hand very similar in reference to the actual context of your prompt and so even though your intention is clearer, the words in your handful are much more diverse but similarly weighted making it a poorer collection.

It's a bit difficult to have a visual metaphor for this. But imagine you go car shopping and you see a broad selection of 20 cars. Pickup, SUV, Hatchback, Sports. Lots of variation and choice. But then you narrow your scope to just Sports, but instead of there being less cars, there's still 20 only with minor variations. Then you decide you want a red one, and now there are 20 red sports cars. 2 door, now there are 19 red sports cars with 2 doors and a bicycle because the AI needs to keep the sample count and it's been weighted as a stronger choice. That's sort of what happens. Onwards!

The model responses seem to get further away from giving you a nice clear solution.

I’m sure you’ve found this before if you’ve tried making any sort of document or code with AI and iterated a bunch of times. The model obfuscates the response with jargon and nonsense that bloats the response with confident vocabulary.

Like talking to a middle manager who wants to leave you confused so they can escape. Often useless, and a waste of time. It does this because it wants to give a good response no matter what.

Another issue is that LLMs are built using the general public collective of knowledge. From the best to the worst of sources, AI has been trained on it. And by the means of average, that means the responses are likely to be below average ability in a skilled field.

This is why people think AI is great at skills in areas they are not skilled in, and why people skilled in those areas think that AI isn’t that good. I’m among those people that get frustrated because AI just can’t do what I need it to and I’m certain there a lot of people in the same situation. If I could make AI do what I do, I would adopt it and exploit it. But it’s just not good enough and I think the chances of it getting good enough are low with the current design.

I’ve worked with AI on and off a lot these last couple of years. I made tools for it, used existing tools. Created an agent that ran locally to make changes to a code base. Every time I tried to use an AI, it just disappointed. I’ve tried lots of models too, although none of the massive expensive ones. Getting it to point me towards documentation has always been satisfying though, doesn’t seem to matter what it is. Rust crates, Python libraries, the robot framework, The Qwen agent toolkit, OpenGL and GLSL, C++ reference. Although with Rust and Python the number of libraries available make it very difficult to pick the right one.

Going back to the agent that I made. I called it “My boy” as a bit of fun. It just couldn’t do what I asked. I wanted a proof of concept so the task was simple, make an agent that can run up a local database. Literally just typing “pg_ctl start” into a terminal. I used Qwen 3.5 9b locally and the Qwen agent framework. It didn’t do well. It couldn’t modify code to a good standard and didn’t stick to rules. I gave the agent more tools hoping that would help but it just got perplexed by the additional context of the tools themselves and at one point just kept thinking, going over all the tools available and never doing anything. This could simply be my inexperience of how to get an Agent working though as I've only tried once.

But the agent wrote 250 lines of python to launch and close a database with the most unsafe code I have ever seen. It was 45 lines after I corrected the code and simplified it. And that’s the thing; simplicity is a skill. If you ask AI to do anything simple, it never works. This week was the first week I used python in my job of nearly 3 years. My last time professionally was writing a tool to convert a spreadsheet into Json data, so you know my knowledge of python is basic. Code, AI can do in a fraction of the time it takes me, but it’s always a fraction of the quality and so I end up rewriting it anyway because maintaining poor code is more expensive in the long term. Oh god I’m just moaning away about AI. I do try to adopt it. I just wish it was as good as they say.

So how do we use these LLMs?

It helps to rubber duck. It gets you moving if you’re stuck. It helps with knowledge of areas you are unfamiliar with in your discipline, like finding the right library or suggesting a design pattern that you might not have thought of. I ask it to write me some code warmups in the morning. In my discipline, I know when it’s wrong but it’s not entirely useless because it can get me thinking outside of the box. It gets you moving. More importantly, it gets you thinking and trying new things if you use it right. But the key is YOU thinking. You get better every single time you do something. But a model doesn’t, it’s stuck in its version and when it changes, it’s often not smarter, it’s just different.

A nice use of AI I had was when I was considering a poor design for an interface. Mock code here. Not real obviously.

class MyCommandRelayInterface {

public:
    void SendRelayCommand(const String& Name, void* data) {
        if (Name != LastCommandRecieved) {
            ExecuteRelayCommand(Name,data);
            for (auto Neighbour : Neighbours) {
                Neighbour->SendRelayCommand(Name,data);
                SendRelayCommand(Name,data);
            }
        }
    }

    protected:
    void ExecuteRelayCommand(const String& Name, void* data) {
        //Override in implementation.
    }

    private:
    std::vector<MyCommandRelayInterface*> Neighbours;
    String LastCommandRecieved = NoName;
}

This interface is dangerous. The execute relay command takes a void pointer parameter that could be of any type. So implementing this relay interface means that all objects in this relay will try to cast that void pointer if any checks succeed that are unintentional, this will lead to unpredictable outcomes, crashes or some mad issues. I asked AI if I’m mad for considering it. It suggested using the standard library variants. The design it suggested wasn’t great, but variants library did become my solution with polymorphic functions in the interface. You call generic function with a variant, and the interface decides which polymorphic function to execute. It was very elegant. I'll mock that up here but again, won't be real and this will be simplified a bit as the real thing has lots of handling for sent to interfaces and checking if the message is duplicate. I only want to highlight the improvement in the design choice of using variants. This is basically really simple to extend with less to consider when adding variants to what can be sent down the relay of objects.

std::vartiant<float,bool,Object*> RelayVarients;

class MyCommandRelayInterface {

public:
    void SendRelayCommand(const String& Name,RelayVarients Value) {
        if (Name != LastCommandRecieved) {
            ExecuteRelayCommand_Internal(Name,Value);
            for (auto Neighbour : Neighbours) {
                Neighbour->SendRelayCommand(Name,Value);
                SendRelayCommand(Name,Value);
            }
        }
    }

    protected:
    void ExecuteRelayCommand(const String& Name, float val) {
        //Override in implementation for float
    }
    void ExecuteRelayCommand(const String& Name, bool val) {
        //Override in implementation for bool
    }
        void ExecuteRelayCommand(Const String& Name, Object* val) {
        //Override in implementation for Object
    }

    private:

    template<typename Type>
    void ExecuteRelayCommand_Internal(const String& Name, const Type Val) {
        ExecuteRelayCommand(Name,Val);
    }

    std::vector<MyCommandRelayInterface*> Neighbours;
    String LastCommandRecieved = NoName;
}

Forgive the bad code and poor colour. I'm doing this in a HTML editor and I'm lazy. Maybe I'll make something to show code properly.

This implementation is much better to expand on. The template function automatically calls the correct polymorphic function to execute on the interface and all the complexity of implementing the interface on an object that needs to be a relay is simple. And on top of that, it's all type safe as it's impossible to send something that is not a variant to the interface with the public send relay command function. Elegant and simple with the complexity hidden and follows the brilliant SOLID principles too. We can all sigh in relief.

So that’s where LLMs work best, when it works with you, not for you. Anyway. It can be good when used right. If you find you are trying to go faster than you really can, you’re probably becoming reliant on AI and it will let you down, stifle your growth, and likely frustrate you.

We’ve only really covered LLMs here. The drone training thing I’ll cover more when I get a bit further with it. But that’s reinforcement learning which is different to supervised learning used to create LLMs or generative audio or generative visual models. Reinforcement learning is where the model learns directly in the environment with observations and scoring. (I think, I’m no expert)

As always I hope you enjoyed reading and learned something from this waffle. Until next time!