Why LLMs might not be a good tool for your small business

If you run a business, then evaluating when a digital tool might not be saving you much time, or making your work better is important to consider.

Why do we even need to talk about Large Language Models in small business?

Whenever I work with a client to make financial decisions about what equipment to buy, or to help them evaluate a new digital tool, there are core questions that I ask:

  • Will this make your work faster/more efficient?

  • Does this improve the quality of your work and add value?

  • Will this tool have a return on investment (ROI)?

These questions are part of my framework for helping designers decide when and if it’s worth it to spend thousands on a piece of equipment, to level up their eCommerce platform, or to invest in a new inventory system.

Every designer will have their own answers based on their work, their current processes and their business goals.

And this is why we need to talk about LLMs. AKA Claude, AKA Chat GPT.

Check out my You tube Short about this too!

It helps if we start by defining what a Large Language Model (LLM) is, and what it is not

What it is: Language Prediction Engine, or a language generator, or a language regurgitator – it falls under the umbrella of Generative AI tools

An LLM can reproduce words and sentences. It can take other words from other writings and make predictions about what order you would like those words in based on your prompts. It uses past language patterns to do this. It can summarize (sort of), it can aggregate words across multiple writings, it can produce writing in your tone (sort of).

It is trained on vast amounts of words and writing.*

What an LLM is not: Pretty much anything else

An LLM is not a fact checker. It is not a calculator. It is not necessarily pulling from data (exceptions are from anything that’s written data). It is probabilistic, but it does not evaluate.

For example, when you ask it to generate good keywords for your website, it is not actually searching your keywords on Google Trends to see if those keywords are useful or relevant. Instead, it draws on previously written material about the subject in your prompt. It has not counted how many people are searching for these keywords, nor has it looked up the times of year when those keywords are popular. It has given you words, but without any evaluation.

It is not using a calculator when you ask it a math question. It is not actually counting the words that you have written when you ask it for a word count. 

It does not cite it’s sources unless you ask it to, and even when you do ask it to cite sources, you still have to go to those sources and fact check them in case the LLM summarized it or pulled quotes out of context. Or it hallucinates sources entirely.

It is not fact-checking. It is not calculating, It is not performing spellcheck. It is not sourcing data. It is not evaluating. It cannot count.

Let me say that again – it cannot count. And this is not just me saying this; OpenAI, the makers of ChatGPT have run experiments and stated quite clearly, that this is a feature and not a bug of their system.

LLMs are not designed to count. They are not designed to evaluate data. They are designed to generate language in a way that is pleasing to you. They are designed to make sentences and paragraphs by training on previously written* paragraphs.

What are the pitfalls of using an LLM, and where should you reconsider using it

I know I am just a humble jewelry business coach, and what could I possibly know about this? Well, what I do know is that a shortcut that makes your life harder isn’t much of a shortcut. And LLMs are, in theory, here to save you some time, but what it really saves you from is typing.

For instance, if I ask ChatGPT to write a factual, data-driven statement for me, it doesn’t really do much except save me from typing a paragraph. If I still have to fact-check the statement, verify that the sources are in context and are real, and rewrite it in my own voice, have I saved time? 

In my case, I haven’t, and that’s why I still write from scratch – it’s much faster and more straightforward to do what I usually do – go in search of sources from experts and then write conversationally about a topic.

Another LLM pitfall to watch out for – accuracy

If you ask an LLM to write something for you that needs to be 500 words and ChatGPT says it’s 500 words, but an actual word counter says it’s 441 words, what time have you saved? 

If you ask it to generate good SEO keywords for your website and it spits out a list, but that list isn’t pulling from actual website data and trends (SparkToro, Google Trends, etc.) and you still have to go and put those keywords through the actual data systems to make sure that they are relevant to you and your audience, what time have you saved?

The short answer is that LLMs haven’t saved you time, and it may make your results inaccurate. This kind of consistent, inaccurate process will only serve to build cynicism with consumers in the long run.

Where is it best used?

By now, I am sure some of you are asking yourselves what I think a tool like an LLM would be good for?

It can rearrange your own words if you have a first draft and would like to see what a second draft would look like. It can type an outline for you. I have heard from people who are bilingual that it does a decent job of translating (but you’d have to be bilingual to evaluate whether or not it’s a good translation). I know others who use it for email and product copy.

That said, any marketing language still needs to have a person (that’s you!) behind it who understands their audience, who understands what their customers need to hear from them and also needs to understand the timing of their messaging – that is still best developed and researched by a person – that’s you!

What work do you need to do BEFORE you use an LLM

So before you sit down to let an LLM write your product copy or your next newsletter, start by asking yourself some questions: Who are you speaking to? What do they want to feel about your work? What action would you like them to take and what do they need to know about your work to get excited about it?

There is no substitute for getting to know your customer.

Much like there isn’t a great way to speedrun your stone setting skills. As with anything else, the process of getting to know your customers, and the process of learning to listen to your customers will be a process that takes time, regardless of what tools you use to communicate with your audience.

I understand the allure of a tool that makes your life easier. I do think though, that making informed decisions about whether or not to use a tool is part of that process. And I especially think that you should understand whether or not these are tools that are resulting in time-saving, because I see how hard you work. 

Sharon, do you use LLMs to write your blog or your newsletter? 

I don’t!** 

It’s not that I have never tried them. I have tried them, and I even took a course in how to use them a couple of years ago. I’m a curious person who likes learning new ideas and new systems. I’ve been using computers since 1983, on the internet since 1994, placed my first online order in 1997, and first worked in eCommerce in 2008—I’ve been a tech enthusiast for many years.

But using an LLM is not better for me than any of the many other systems and software programs that I already have in place. If I already know how to set up an automation, then I don’t need most of the AI tools. If I already know how to write to my audience, then an LLM does me no good. If I already know how to build a spreadsheet, then Gemini is as unhelpful as Clippy was (I just totally aged myself).

I also like writing. I like thinking about the people in my audience and thinking about what problems or successes they might be having. I’ve never met an LLM that can do that for me.

Citations:

*Which words and writings? Great question! There was a lawsuit that was won against the makers of Claude because authors discovered that their work (unlicensed, unpaid) had been used to train this LLM. We don’t always get to know what works were used to train these LLM systems – sometimes internet scraping, sometimes academic work, but one core concern is the potential for theft.

UPDATE: OpenAI, the makers of ChatGPT are also going through their own copyright infringement issues at the moment. This raises the likelihood of your use of this tool accidentally plagiarizing an artist or author’s work. If you have any concerns about protecting your intellectual property, reconsider using these tools.

**I have a lot of reasons to bypass using systems like Claude and ChatGPT, and those issues are, in no particular order: future issues with protecting your intellectual property, the lack of safety guardrails that has already resulted in tragedy, the preliminary research into how it’s affecting our cognitive skills, and lack of data privacy/security. I’m also not holding my breath for these companies to gain financial solvency anytime soon. While that is typical for many tech start ups, the financial speculation in Generative AI is already showing that it is cash flow negative. As such, if I put all of my work into these systems, I would likely find myself having to switch platforms in the next 2-3 years.

A few other sources for anyone wanting a deeper dive:

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *