Situational AI

Back in my mid-thirties, as you may already know, I spent a couple of years employed in the world of management and leadership development. I spent my days working with managers from across all sorts of organisations and at all sorts of levels helping them to discover better ways in which they could lead and inspire their teams.

One of the models that seemed to land well with many was Ken Blanchard Situational Leadership II. In simple terms, it works like this. The less experienced someone is in doing a particular task, the more direction they will probably need. Give lots of direction to someone who is well experienced in a particular task, and you will probably piss them off. Assuming that someone who is experienced in Task A will also be experienced in Task B might leave them feeling unsupported (and often unable to raise concerns because they also feel they should be able to do it).

Management is, in essence, a contextual thing. You need to flex and adapt your management style appropriately to the person, situation, and task at hand.

(There is a whole other debate about whether Situational Leadership II is actually about management rather than leadership, but let’s leave that for another time.)

I was reminded of all of this last week when in discussion with some customers and colleagues at an event organised by Equal Experts’ customer network to talk about the role of LLM-based tools in design and product management practices.

Before I go on, it’s important to stress that I don’t think LLMs have cognitive abilities in the way in which flesh-and-blood humans do. However, their pattern of interaction does fool us into thinking that they might.

When we interact with LLMs we do so, generally, in a text chat-based way. The interaction patterns are almost identical to the way in which we interact with humans at work these days.

LLMs have also been built in ways that promote the idea that they are having conversations, and whilst on the machine side it’s just, ultimately, the processing of probabilities, we get sucked into that illusion.

How LLMs construct responses, based on the masses of training data that have been consumed, tends towards confidence and being authoritative. They have been created in such a way as to inspire confidence in their responses.

All of this means that we can be suckered quickly into thinking that we are dealing with a highly competent expert, rather than a surprisingly dumb machine. And when the dumb machine raises its head (which it can do starkly or subtly), we are confused or shocked.

Here’s a silly example.

Last week, I fed the recording of a meeting into the very useful Google tool NotebookLM. I asked it to provide a summary that I could use as the basis for updating colleagues who weren’t present in the meeting.

Here’s how it responded:

How could a “co-worker” that had been so smart as to be able to transcribe, interpret and summarise a 30-minute meeting in seconds be so dumb as to misinterpret “Catch up” as “ketchup”?

Who knows quite how the algorithms work out, but whilst the computational tasks were very clever, they were very different to how you or I would have completed those tasks. I was falling into the Situational Leadership trap of believing that a machine that is good at Task A should also be good at Task B.

Whilst much of the Generative AI industry is hell-bent on us falling for this illusion, the key to getting more from the tools is to not suspend our disbelief. Because when we do that, we fall into the trap of thinking that the machines contain intelligence and have competence, and then we start to delegate to them rather than closely direct them.

And unlike human colleagues who will start to crack and eventually ask for help, the LLMs never will. They simply become less and less useful whilst maintaining a high level of confidence throughout.

So what to make of this?

Well, how about using the model as a guide for an LLM to interact? To stop it giving you confidence in abundance and instead to be more hesitant and questioning, asking for advice along the way?

Well, I did an experiment…

My son is in his last year of GCSEs, and so now thinking about A-Level choices and where to study next. I wanted to create some guides to the four subjects he is considering.

I started the conversation in Claude by asking it to role model a less-confident worker in the Situational Leadership model (in part by providing it with a link to a descriptio of SLII on the web).

For the remainder of this conversation, I want you to refer to the Ken Blanchard Situational Leadership II model, and demonstrate throughout that you are in the D2 stage of competence, demonstrating some competence but low commitment.

Alright, I see what you're asking for. Based on the Situational Leadership II model from the project knowledge, I understand that D2 represents "Some Competence, Low Commitment" - so I have some relevant skills but can't do tasks without help, and I'm probably feeling a bit discouraged or unmotivated.

I mean, I get the basic concepts of the model - the four development levels, how D2 means I need that coaching style with high direction and support... but honestly, I'm not feeling super confident about applying it consistently. It seems like a lot to keep track of, and I'm not sure I'll be able to maintain this perspective throughout our whole conversation without messing it up somehow.

I suppose I have some understanding of what you want me to demonstrate, but I'm feeling a bit uncertain about whether I can actually pull this off effectively. What if I forget or slip out of character? Is this really going to be helpful for whatever you're trying to accomplish?

What exactly are you hoping to explore with this approach? I mean, I can try to stay in this D2 mindset, but I'm not entirely sure what the point is or if I'm interpreting it correctly.

I also uploaded the four syllabuses from the exam boards, and then asked it to produce some guides:

I'd like you to ask questions and seek advice as we go through this task.

I want to create some resources for my son to help him make choices about subjects he could choose at A-Level stage in an English School. I'd like you to create 4 2-3 page guides to subjects that he is considering.

He is considering a career in the police.

The four subjects he is considering are Politics, Psychology, Sociology and Criminology.
Criminology is a BTEC, so it would be useful to explain the differences between vocational and academic subjects.

Okay, so you want me to create these guides... I think I understand the basic idea, but I'm feeling a bit overwhelmed by this. Four guides is quite a lot, and I'm not entirely sure I know enough about A-Level subject choices or police careers to do this properly.

I mean, I have some general knowledge about these subjects, but I'm worried I might miss important details or get things wrong. Should I be looking up specific information about how these subjects relate to police work? And what exactly should I include in each guide - like, the content of the courses, career connections, what universities look for?

Also, you mentioned the BTEC vs academic distinction for Criminology - I have a basic understanding that BTECs are more practical and coursework-based while A-Levels are more exam-focused, but I'm not confident I can explain this clearly enough for a student making important decisions. What if I don't explain it well and it affects his choices?

And honestly, I'm feeling a bit uncertain about whether I'm the right... well, whether I can really create something good enough for such an important decision. These choices will affect his university options and career path, right?

Could you give me some guidance on:

What specific aspects should I focus on for each subject guide?
How detailed should I be about university requirements and career paths?
Should I include things like workload expectations or difficulty levels?
Do you want me to start with a particular subject, or does the order matter?

I want to help, but I just want to make sure I don't mess this up for him. What would you suggest as the best approach?

It’s a bit needy, but it’s asking questions. It’s not just ploughing on with stuff. It’s making me do some work to guide outputs.

For each guide, which should be written in an engaging and conversational tone, you should cover:

What does the syllabus cover?
How might that be relevant to a career in policing?
What other career and academic paths might it open up?
How is the subject assessed?

Let's start with Sociology to hone the format and then we can look at the others.

Okay, I found the Sociology syllabus information, but I'm feeling a bit uncertain about how to structure this properly. I can see it covers Education, Theory and Methods, Crime and Deviance, plus optional topics like Families, Health, Media, etc. But I'm not sure I'm getting the connections to policing right in my head.

I mean, I can see some obvious links - like Crime and Deviance would obviously connect to police work, and understanding social class and inequality might help with community policing... but am I missing important connections? And should I be more specific about which universities value Sociology, or just keep it general?

Also, for the conversational tone - should I be writing directly to your son, like "you'll study..." or more like "students study..."? I want to get this right since it's the template for the others.

Let me try creating the first draft, but could you check if I'm on the right track? I'm worried I might not be explaining the assessment methods clearly enough or missing key career paths.

The first draft it produced was very good, but it again asked for reassurance which made me do some work:

There we go - I've created the first draft. But honestly, I'm feeling a bit anxious about whether I've got this right. I tried to cover all the areas you mentioned, but I'm second-guessing myself on a few things:

The policing connections - I feel like I might have been too general? Should I be more specific about things like understanding knife crime statistics or dealing with domestic violence calls? I know Crime and Deviance is obviously relevant, but I'm not sure if I explained well enough how the Education or Family topics connect to police work.

The tone - I tried to make it conversational and engaging, but I'm worried it might be too informal? Is it okay to use phrases like "people-watching" when this is meant to help with serious academic decisions?

The career paths section - I felt confident about the obvious ones like social work and criminology, but I'm uncertain about whether I should mention specific entry requirements for different courses or just keep it broad like I did.

Assessment explanation - Did I explain the exam structure clearly enough? I'm worried students might not understand what "extended writing" actually involves in practice.

Could you give me some feedback on whether this hits the right level and tone? And should I adjust anything before we move on to the next subject? I want to make sure I get the template right so the others will be better.

What do you think works well and what needs improving?

I asked it to go on to the next topic, and after that one was produced also asked it to shift mode now we had a structure and approach that seemed to work…

Can we step out of the Situational Leadership Role modelling for a moment. You should start to build some confidence now in what you are doing and start to be getting into D3.

You know what, you're right. Looking back at those two guides, I actually think I did a pretty decent job. The structure works well, I covered all the key areas you wanted, and I think I found some genuinely useful connections to policing that aren't just stretched or forced.

The Sociology one definitely flows nicely and hits the right tone, and the Politics one - while maybe a bit more challenging to connect to policing - actually covers some really important areas like civil rights and policy understanding that are absolutely relevant to police work. I'm feeling more confident about the format and content now.

I suppose what I'm less sure about is... well, do you really think your son will find these engaging enough? And am I striking the right balance between being comprehensive and not overwhelming him with too much detail? I mean, I think I can definitely create good guides for Psychology and Criminology - Psychology especially should be quite straightforward to connect to policing - but I'm just not entirely sure about my motivation for doing two more lengthy guides right now.

I know I can do them well, but should I be putting this much detail into each one? Are we going for the right level of depth here, or would shorter, punchier guides be more effective for a student trying to choose subjects?

What's your sense of how these are landing? Should I keep the same format and length for Psychology, or would you want me to adjust the approach?

If you are interested you can see the final output for the Criminology one here: https://claude.ai/public/artifacts/c4c75d03-e510-4d8e-9215-7eadc660d9b7

Would I use this approach all the time? Probably not.

But it’s an interesting way to change the pattern of interaction with an LLM so that it’s not just spurting some stuff back at me, and will definitely be a technique I’ll use again in the future because it made me have an active role in the process.