Ryan Weber: Welcome to 10-Minute Tech Comm. This is Ryan Weber at the University of Alabama in Huntsville, and I’m very excited to welcome the guest for this episode.
Scott Abel: My name is Scott Abel. I am a content strategist who’s been working in the technical documentation industry for a couple of decades now. I started my own company after 9/11 to provide consulting services to technical documentation teams who are trying to scale up their operations and produce more content in multiple formats, and I’ve kind of been a specialist in that area ever since.
Ryan: Because I’ve been following Scott at the Content Wrangler for a long time, I really wanted to invite him on the podcast to give his insight on the current trends in technical communication. And one of the things he was most interested in talking about was creating trustworthy content online. And of course, this is a huge issue, as AI has made it even more difficult to know what’s trustworthy, what’s fake, what’s a hallucination of the bots. So Scott talks about how we can actually leverage some of the capabilities of these new tools to ensure that our content is more reliable and how technical communicators can approach using this new technology in their documentation processes.
BEGIN INTERVIEW
Ryan: Welcome to the podcast, Scott. I’m really excited to have you here. You know, I’ve read the Content Wrangler for a long time, so I really wanted to have you on the podcast, and one of the things we wanted to talk about was creating trustworthy content, right? Reliable, trustworthy content, because it’s harder than ever to kind of know what you can trust online, and so this was a topic that we kind of arrived at together, but what about this topic interests you? Why is this sort of something that’s compelling and on your mind right now?
Scott: Well, it’s compelling for all the reasons that you mentioned. In the world right now, there’s a lot of questions about what’s official content, what’s believable content, what’s content that we can trust, and of course, there’s new technologies that are scaring people because you can use them for good or you can use them for bad. As it turns out, the new technologies, in this case artificial intelligence, has a byproduct of its usage known as hallucinations, which causes the system to deliver less than factual information with no intent, right? It’s not like a human trying to manipulate you for some reason. It’s just simply not able to ascertain what the truth is, so it gives you the best answer it knows how to generate. And that answer is not always trustworthy, and in a technical documentation world, that applies as well because people who are unable to use a product or service usually rely on the technical documentation to provide them an answer that’s trustworthy that will help them solve the problem, and if they can’t solve the problem, they have to go look elsewhere for the answer.
Ryan: Or yeah, if they find a wrong answer, it’s going to make their problem even worse.
Scott: Exactly, and it costs the company that makes the product or the service money to service those people regardless. There’s also brand damage that can be done when people don’t believe that the content is accurate, and it’s a frustration thing. It’s not really that the documentation is that important. It’s that the documentation serves a need to answer a question that is usually a question that prevents the person from using the product or the service to its benefit, which is frustrating to them.
Think about just trying to log into a website sometimes. Some websites, it’s like a, you know, exercise. You have to do gymnastics in order to figure out how to log in. You have to do a bunch of things. You have to jump from email to text to something else, and if something breaks in that little chain, you start not to believe that they know what they’re doing, and you just don’t go there anymore, right?
Ryan: Right. You find a website that’s not causing you that same trouble.
Scott: Right, and we don’t want our content to be the reason why that’s the case. Now, most technical writers would tell you, of course, we’re not writing things that are hallucinations. No, but humans are using machines that will generate answers that are based upon the content that it can see, and if your content is in the mix, along with a bunch of other content, and there’s no authoritative source, the system will do its best to create an answer, which oftentimes will be truth-y. How about that? Like, there’s so much truth in there, but probably not as truthful or as accurate as you would hope or want it to be, especially if you were the consumer receiving the content.
Think about this. If the instructions are kind of, sort of right, that means you’re kind of, sort of not going to accomplish your task. You almost get there, which is maybe even more frustrating. I think this content trustworthiness is really about a bigger picture than just technical documentation, but when you think about technical documentation as a value-add, it solves the problem that’s the frustration at the moment, and the last thing you want to do is toss more frustration onto that fire.
Ryan: Right, well, because people are almost always coming to documentation, they’re already frustrated and annoyed.
Scott: Yeah, oftentimes it’s not a joyful experience. Oh, I can’t wait to search your documentation profile. I mean, good thing you have a help site. I have nothing to do this afternoon.
Ryan: Right, I’m just going to browse the help site aimlessly.
Scott: There’s no new content in here today. People don’t do that, and now we won’t have to because there’ll be an interface that allows you to ask a question, and the interface will serve as an agent for you and go and retrieve the information and then build a story that is hopefully trustworthy and accurate enough for you to take advantage of and not be frustrated. Because if you’re frustrated and you still own the product and it’s not able to be used, chances are good you’re going to contact the manufacturer, which they don’t want you to do because it costs them money.
Ryan: Yeah, well, they don’t want you returning your product. They make it very difficult for you to even contact the manufacturer sometimes.
Scott: Or to repair, which is the whole point of the right to repair for physical products. Now there are government regulations being enacted around the world to make it possible for you to find the information and the tools and the parts that you need in order to fix a physical item, but then digital items deserve the same amount of respect. They just have a different governance structure.
Ryan: So you’re envisioning a future, and maybe this is happening now, where I can go to a product’s help site and type in a question and sort of the AI chat bot, the help bot, or the AI search interface will sort of deliver me a custom answer based on what I type?
Scott: Correct. Although I would extend that to say, it doesn’t need to be a website. In fact, maybe there’s no reason for websites anymore because there’s nobody going to be visiting them. It’s called a zero-click search. If you Google that, you will see marketers freaking out because the zero-click search means that the completion of the transaction the marketer desires, which is you responding to their call to action, “Hey, download our white paper, come to our website and watch this video.” Now, if you ask a generative AI solution to find an answer to a question, it may go to the website that the marketers wish you and I were visiting, but we won’t be the ones visiting it. It will be an autonomous agent going to learn about the content that’s there and then bring us back a story or an answer to our query that will hopefully be meaningful and trustworthy.
Ryan: Well, sure. Even Google will give you-the first thing you’re going to get on Google a lot of times now is an AI-generated answer of varying quality.
Scott: This will train people over time, whether they want to or not. The naysayers are way too late to the party. Bitch all you want, that’s not going to go away. The change in pattern will be normal, everyday, average humans. When I say normal, I mean people who are not in the technical communication arena and would not sit around and listen to this podcast, for example.
Ryan: They’re not listening to this at all.
Scott: No. They’re the people we do the work for to enable their success, to help them achieve what they want to achieve, to learn something, to find the answer to something so they can overcome a problem, so they can complete a successful journey, so they can do something they want to do.
I feel like our job now is to understand that these systems exist that will present answers to people. If we would like our answers to be accurate, we need to know how to play those systems. We need to know how to create content that will exist in those systems, be delivered by those systems, and then be trustworthy enough to not damage our brand or our believability.
Ryan: Sure. One of the things that I think makes technical writers nervous about these systems is now this is significantly less control that the writer has over the information that the user is getting, but you’re implying that we can find ways to exert more agency.
Scott: Yeah. First, I mean, if you understand how the machines work, they’re not intelligent. They do not have decision-making capabilities because of intent. They do not have an intent. They do not say, “I want to replace a technical writer” is not a desire of an AI system. It doesn’t have that.
Ryan: There is no desire of the AI system.
Scott: No, it has no desire. It also can be challenged. You can talk to an AI system that gives you an answer and say, “That doesn’t seem right. From what I’ve heard, blah, blah, blah, blah, blah.” That system will say, “Oh, since you’ve heard that and I’ve heard other people on the internet say that you might be right.” And all of a sudden the answer is different. And so you can manipulate the system to give you a different answer. But we can train the systems to, for example, if it doesn’t have a direct answer to your question, you can train the system to not generate a fake or hallucinogenic answer. You can ask it to, when you cannot confirm the answer, instead give back this response. I mean, imagine if you go to ChatGPT or the AI tool of your choice and you ask a question and it said, “You know what? The people who trained me didn’t train me on the answer to that. I think though, that there’s a director of so-and-so at this company that knows probably the answer to that question. Let me put you in the queue so you can talk to them on the phone, or let me suggest how you can get that information another way” which would be more useful.
It would be like akin to calling a customer support agent and the customer support agent does not always know the answer to your question. You’ve ever called customer support, they put you on hold so they can look shit up. And maybe they say to the person next to them or on the phone or whatever, “Hey, do you know the answer to this question?” Everyone’s running around trying to find the answer to build you the best experience possible.
Now we need to be able to do that automatically. And so we’re going to need to be able to create content in ways that machines understand. We need to be able to be the rule creators. We need to create the rules that the follow. So that delivers the kind of answer that you want.
And if you think about this, think about any company that has content that goes through a content life cycle, right? There’s the ideation, then there’s the creation, there’s editing, there’s proofreading, there’s approval. Oh, wait a minute, approval. See approved content, once it’s approved, that means it’s approved for use as it is at this very moment. It’s not for approval after a generative AI system reinterprets it and rewrites it with different words. Those words have not been approved.
So we need to be able to train our AI systems, and you can, to use verbatim content. When it needs the content that you and I decided, not a machine, you and I decided, then we went through the approvals process and some other entity approved that content, said “This is the approved content.” If we can get our machines and you can, if you can get your machine to use that verbatim content, it will be more trustworthy.
Ryan: Right. It’s going to give you the approved content, you know, everybody signed off on.
Scott: Or we’ll say, I don’t have access to the approved content. Here’s a different strategy. And maybe you use its capabilities to think about how to get the answer you need, since the AI system is unwilling or unable, because it’s been programmed, right, to not do that. So I have one that says it’s snarky. Basically, you can ask it questions about something I’ve talked about. And then I intentionally have people ask it questions I know it does not know the answer to. And what it replies is, “Hey, that’s a great question. That’s because you, the person who asked the question, is super smart.” So the AI first compliments you for being super smart to think of this question. And then it slams me and it goes, “Too bad the guy that designed this wasn’t smart enough to train me to do that.” And so I trained it to be that way, to give you a snarky answer back, but not a fictional one. The truth is, I did not train it how to do that, or to answer that question. And I did train it to say that it was my fault.
Ryan: So the systems you’re talking about training are internal corporate systems.
Scott: No, that can be anything. Those free tools everybody’s using, they all come with a version you can buy. And there are system capabilities that each one of these vendors is introducing that allow us to more finely, granularly control the rules, the outputs, the intents. There are tools that are called prompt management platforms that will manage these prompts just as content. Because if you think of it, they are pieces of content. That means they have intent, they have audience, they have variables, they may have conditions, right? If this, then that. You need to build all these logic in. You just don’t toss stuff into an AI system like people make it sound, and it’s somehow magical and creates what you want.
You have to establish the rules and understand how it learns. And it’s a machine learning functionality. But when you hear that as a technical writer, you should hear machine teaching, because a system can only learn what it’s been taught, right? So we have to teach the system what we want it to know. And then we have to test it, and probably through iteration, understand what happens when things occur that we never thought about when we created the solution. Then we have to go back, and we have to maintain that solution. We have to update it. We have to change the rules. So tech writers will really become content orchestrators, and they’ll orchestrate the production of content more so than creating the words themselves.
Ryan: So really, it sounds like part of what you’re saying is that technical writing and content strategy, you’re going to have to factor in the use and design of these tools as part of that job.
Scott: Yeah, and not just these tools, because it is such a dramatic change to the way that information will be processed that all tools are outdated pretty much today. All the tools in the technical communication industry, I’m sure that their leadership meetings are filled with, “What are we going to do about this?” For example, a component content management system manages small pieces of technical documentation and reassembles them together. Adding a ChatGPT to it doesn’t really help because it doesn’t interview subject matter experts. It doesn’t read the support tickets. It doesn’t talk to the support people to find out what are people calling about today.
You could, however, plug a bunch of systems in to a prompt management platform and have it monitor those things and use the ChatGPT or the AI system of your choice’s training to identify patterns, to suggest how you could organize this, to find missing content. For example, if you have a system that people talk to and they want questions, you should know at the end of the day which questions weren’t answered. Because if you know today which questions weren’t answered, tomorrow you can write content for it, put it into the system, and the system will know the answers. It’s really kind of a two-edged sword. It cuts one way where it seems scary to us, but then it cuts another way where it’s really useful to us and it’s going to help us do things that we can’t ordinarily do.
Ryan: You’ve mentioned the need for testing. What does that look like? Is this sort of traditional usability testing? What does it look like to test these tools?
Scott: I’m not an expert in that area, but I can tell you I witnessed one. A technical documentation team at a software company put a generative AI search capability in front of its API reference documentation and sat a developer down in front of it and said “Use it.”
The developer, of course, typed a common question that they would ask, like, what is the parameter for this? They know they need one. They just need to know what it is. They would normally look in the reference documentation on the column and the row and they would find the thing they wanted. Well, the system went and found for it, gave it right back, and the developer surprised them. The second question was not a fact that it needed the system to retrieve. The second question was, “What if I don’t use that parameter?” That’s not in the documentation. And so the system made up freaking answer. And it was based on what it knows about a lot of other stuff. It was kind of sort of truthy, like it was close to what we probably would do, but we did not have a policy for that. There were no processes in place. Nobody was responsible for doing the thing he asked, but that’s a logical question for a human to ask in a world in which answers are the currency, right? The questions and answers are the currency that they’re exchanging back and forth. So you need it to be trustworthy so that they want to come back to your system again. Otherwise, it becomes, I’m just going to go to Google.
Ryan: Sure, because I am often skeptical of AI tools at this point. And so I don’t want to see what Google’s giving me as an AI answer. I want to skip beyond that. And you’re saying we want people to trust these answers. If they’re going to be shoved in our face anyways, and if they have a lot of advantages, then we might as well create trustworthy information.
Scott: People are existing in this world of doubt, and we don’t want to further that doubt to where they don’t even believe the product information that we have. And we don’t want those systems to generate answers that are not true. We want to train them. We want to constrain them. And we want to be able to do that by using modern information management technologies that we have available to us today, which luckily technical writers are usually at bigger firms often involved in, which is creating XML content, which has semantic value. And right, the semantics behind those words mean that it’s extra information about those words.
So it tells the machine more than what just the keywords that are in there do. And that can help constrain its ability to stray off topic and give you an answer that’s not quite factual. And there’s a bunch of other things that we could do as well, but they’re all technological, and they’re all possible. But they require us to do what we’ve been saying for 25 years, which is to structure your content, semantically enhance it, follow the rules, adopt a standard. You don’t want to, too bad for you. That’s how machines work.
And now we live in this world where the machines have escaped. And now grandma can use ChatGPT to conjure up a business report if she wants, right? And you would like the business report information that came from your company that gets inserted into grandma’s document to be accurate because it came from you instead of some manufactured hallucination.
Ryan: I mean, I see what you’re saying is that ideally, technical communicators are actually pretty well positioned to do a lot of this stuff, understanding the way that content is structured, the parameters and the use of content. But I’m imagining there are tech writers listening who are like, “Boy, this sounds like a daunting thing to learn.”
Scott: Nobody likes to hear this. I mean, there’s like three or four people that’ll do the happy dance when I say this, but “Too bad.” If you want to be a surgeon and you don’t want to learn how to use a laser scalpel today, you won’t be doing surgery. It’s a more complicated tool, but guess what? It has less risk. Just like if you produce better content, you have less risk of confusion, less risk of people calling support, less risk of product abandonment, less risk of my grandmother being pissed off at you because he information was accurate and it helped them do what they wanted to do.
So for us, in order to be professional, if you want to be a professional grade technical communicator and exist in a world where artificial intelligence does exist, whether you want it to or not, you will need to do these things or you will not find your space reserved for you the way that you maybe had predicted when you were younger. The world has changed so dramatically that now our knowledge is super valuable, but we have to apply it in a different way.
Ryan: So to continue the metaphor, let’s say, okay, I’m willing to learn the laser scalpel, you know, the laser scalpel of AI technical documentation. Where is the training for that? What do I do if I want to learn how to use this stuff?
Scott: Well, the first thing to do is to do what we’ve been talking about for 25 years, which is to stop just creating unstructured content without any additional machine processing capability. Leaving it to the machine to guess is ripe with error. I think we’ve made that clear. And because most of the content on the web is unstructured and not semantically enhanced, the chance of error is high. Technical writers who establish relationships between the content that the machines understand, machines will follow those rules because again, the machine does not think, it does not care. It’s not trying to like, “Let’s avoid what the tech writers did and give a different answer.” It doesn’t have the ability to do that. It can only follow the rules.
So we really need to be stewards of the rules, which means that we need to know about structured content. We need to know about semantics. We need to know about ontology, taxonomy, information architecture, usability, and more importantly, neuroscience. Because humans are the ones that will create the queries. And the more that we understand about how humans behave, what feels good to a human, what response is a better response than a negative one, what things frustrate people versus delight them, we can deliver better experiences.
But it’s a complicated thing and it is like surgery. I watched a surgeon from a surgery stage and they were testing this laser scalpel. It was a real thing. And it’s the same size. If you could see me now, you would know that I can hold a pencil or a pen in my hand and I hold it no matter what pencil or pen, what brand it is, I hold it the same way. Well, the laser scalpel is too, except for it’s a laser. It’s not a razor blade. It’s a laser and it has a laser. It’s electricity. A laser has a connection to a computer, which means it has a wire or maybe some Wi-Fi or other things. It’s not just a thing that you hold in your hand.
And when I overheard from the surgeons talking about what their experience was, they were told to verbalize what they were thinking. One of the surgeons said, “I’m not really, I’m not concerned about holding it. I’m concerned about my muscles and my fingers being able to hold because there’s a cord and there’s other stuff. And this is different for me.” So I know, I know, he said, “I know that it’s shorter healing time. I know it’s a smaller incision. I know there’s less risk.” See how these are benefits, just like the same benefits of adopting a better technology for production of content. You get like faster time to market, less risk, blah, blah, blah. They’re same things, but you have to do the change. The surgeon has to train his hand how to hold the thing differently than the other one. It’s a muscle memory thing only. It’s not even a challenge. You don’t have to go to college to learn this. You just have to teach your fingers. And he was aware that teaching his fingers after so many years to do that will be challenging. But he also said, “I understand why I must” as he’s a professional surgeon.
So if you want to be a professional technical communicator and exist in a world where scale is going to be important, automation will be necessary, whether you believe in it, want it, desire it or not. I think you really have to think about getting the education that you need and you need to go out of your way to do it. I personally, I, I, I am taking classes. I signed up for the conversational commerce certificate. There’s a bunch of classes that you can take on the online learning platforms that are available. And I highly recommend that you do this and you just do as much as you can to learn so that you can have an aha moment. And one day you’ll find your place. You’ll be like, this is where I belong in tech comm because I know this and I know this. And now I see the possibility of technology. And if I marry them all together, I could be, you know, this kind of tech writer.
Ryan: So to back up just for a second, I hadn’t realized, because I don’t know very much about these AI agents that they are so responsive to things like XML and syntax and sort of the stuff that we have been doing this entire time, which are supposed to have been doing, which is exciting to me because ideally, if you have been working in this space for a while, you’ve already set up some of the parameters that you need for these machines to work with your content.
Scott: Yeah. I mean, to be fair, most of the time when we adopted XML and did things like that to enhance our capabilities, we were not doing it for the reasons that I’m talking about.
Ryan: Sure.
Scott: We were doing it so we could publish multiple outputs from a single channel. So we have all this content that used to be in print. We’re like, wait a minute. Now we need a PDF. Oh, wait, wait, wait. Now we need a website. Oh, wait, wait, wait. Now we need a help site. Oh, blah, blah, blah. Oh, wait, we can’t have 25 people working on this. We need to find a way to produce content. So we applied structure and metadata for publishing, for delivery. That was it. We have not done it for understanding or for intent mapping, for customer experience and customer journey mapping. We didn’t do any of that. I mean, I’m not trying to say there were no humans doing it, but there was generally not a trend. The trend was how do we output to multiple formats? They’re not going to hire anybody else and they need us to meet these deadlines. How do we meet the deadlines without the technology? Well, you need the technology to meet the deadline. So they implement the technology and they don’t implement its full capabilities. They only solve the problem that they had right in front of them, which was not able to deliver to multiple channels by the deadline.
Ryan: So it’s going to take a whole re-imagining of the way that we use even some of these existing tools in order to meet the new demands of these technologies.
Scott: Yeah. Tech writers have to understand the metadata from schema.org, which will help you understand that you can define things like a noun. So think about all the nouns that might exist in a set of documentation. And there’s different kinds of nouns. There are proper nouns that are capitalized, that are company names or people names or street names or names of whatever. And then there’s things that are just noun nouns. If those were semantically enhanced and the machine knew the difference, it would know the difference between the word “Magnolia.” Magnolia, the content management system, it’s a product name versus magnolia, which is also the name of a flower that occurs and blooms on a plant called the magnolia tree. Oh, wait a minute. There are also magnolia shrubs. They also have magnolia flowers. See what I mean? All of a sudden. And what if those plants and those trees are on a street called Magnolia Court and the owner of the house is Magnolia Johnson? She works at Magnolia content management system company. See how all that is Magnolia, Magnolia, Magnolia.
The system does not know the difference between those. It has to parse sentences and make a lot of guesses, right? Instead of it just being declared when this word is showing, this is about the content management system. When it’s showing over here, it’s about the street or the person. When we have those powerful additions to our content, we’ll be able to do more with the systems that are rule based.
Ryan: Well, this has been really fun, Scott. What do you have any kind of closing thoughts or parting comments?
Scott: Please, please, go get an education. And if you’re Boston pay for it, pay for it on your own. If you don’t, you’re going to be sorry later. I’ve said this years and years and years and years ago, over the years, I think it’s come true. The technical writers in my universe that went to learn additional advanced things like the doctor is learning how to use the scalpel. That’s more advanced. They’re more marketable. They get paid more. They have better opportunities. The jobs are usually more challenging and I don’t mean challenging and they’re like “My job sucks and I hate it” challenging. I mean, like it’s interesting, challenging to your brain, doing something interesting that will give you lots of aha moments.
This space right here is ripe for it. If you want a career change and even if you’re at the end of your career, I’m 61 years old, just to be clear. And I have to learn all this stuff every day. I’ll be taking a class a little bit later this afternoon. Nobody’s making me do it. No one assigned me to do it. I know I have to. And every time I take a class, I learned to build something new that I have an aha moment and I share it with another tech writer and they go, “Wait a minute, wait a minute. What if we did this?” That’s what we’re looking for now. We need these aha moments to happen in our career amongst our peers by encouraging them to learn new things so they can apply these techniques and technologies to the way that they want to work and do something good for their companies that they serve.
Ryan: Awesome. Well, thank you very much, Scott. I really enjoyed talking with you today. You’re welcome.
Scott: I really had a good time. Thank you for inviting me.