ChatGPT is Garbage

It's been nearly two decades since the iPhone changed the world and nothing since has managed to have the same impact. What do you do when your industry accidentally solved every practical problem? Techbros can't solve the serious issues smartphones have created in our society because it would eat into profits. Instead, it's more lucrative to create new problems from thin air and solve those with scam products. NFTs, Crypto, the Metaverse, and other snake oil have become prominent as we approach the latter half of this decade. It's pretty clear to tech enthusiasts that this crap is useless. Grandma, dudebros, and other perceived "suckers" can't immediately see that though and therein lies the problem.

Large Language Models are the industry's latest crack at the burgeoning "sucker" market. That includes investors; whom in this case are the biggest suckers of all. Large Language Models, or LLMs, are text parsers that generate feedback based on prompt. They're not AI in the traditional "sci-fi" sense. Rather, they're sophisticated algorithms. Practically speaking, an LLM filters and regurgitates information that is pumped into it. This is referred to as "learning". Problem is, an LLM doesn't actually understand anything. LLM creators take content from any and all forms of human expression, use algorithms to break it down, and feed it in. Then, it uses statistical models and a sophisticated version of word association to feed users information in plain language. As you'll soon see, the information is suspect at best and outright lies at worse.

I miss when journalism had teeth

OpenAI is the biggest player in the LLM market. their product, known as ChatGPT, is now a household name. Not because it's good, but because the marketing is comically over hyped. According to OpenAI, ChatGPT can write programs, teach you how to build a house, and balance your checkbook. If that sounds like snake oil, that's because it is. I think it's an investment scheme on OpenAI's part. People use LLMs to generate terrible looking video and images, as well as poorly written crap day after day. This technology's only practical use is pattern recognition and word association. Anything beyond that is a happy accident and barely works beyond "look, it can do this". I can eat an aluminum can, or drink raw sewage. I'd bet people would pay to see it, too. Even so, there's nothing about those activities that is remotely feasible long term. Nor is it particularly wise to do in general. A LLM giving users life advice is no different.

Understandably, this is all very controversial. Come to think of it, I've never seen such intense opposition a technology. The severe problems its proliferation has created are beyond the scope of this article. Suffice it to say, when your technology is over sold and associated with guys like Elon Musk and Donald Trump, you're not doing a good job. These figures are widely perceived as villains. Trump posting offensive AI generated content on Truth Social is one of the most effective reverse-advertisements I've ever seen. The videos are garbage and undermine both Trump as well as the tech used to make it. These associations aren't direct, as OpenAI isn't responsible for what Grok does. The average person doesn't know that.

Typical ChatGPT proponent

It's easy to see that this technology is limited. Still, I needed to actually use it if I was to criticize it with any depth. I also needed to be certain that I wasn't poisoning my own perception. I figured I should look into what the proponents are doing with it and how they're discussing it. It was time to visit the ChatGPT reddit community. Why? Because if I was going to find organic, positive ChatGPT coverage, it'd be there. ChatGPT and Reddit go together like piss and shit.

Somehow, r/ChatGPT isn't a bunch of cybertruck driving incels regurgitating their ChatGPT outputs. While there is a lot of that, there's also a lot of criticism. Much of which is similar to mine. All visiting r/ChatGPT did was solidify my bias against it. One group was critical, and the other's behavior disgusted me. I'm now more confused than ever.

After a putting it off for a few days, it was time to install ChatGPT and buy a subscription. I spent $20 for a month of use. I didn't want to, and felt dirty doing it. Simply looking at and reading it's outputs would put me in a sour mood. Now, I was going to actively interact with this crap myself. In hindsight, I feel like I was fleeced. The plan was to use it for a single month, and I did. Extensively. The first thing I did was attempt to have a conversation with it. It didn't go well.

When I converse with people, I don't immediately believe the things they say. Instead, I try to determine if they believe themselves. This makes it difficult for people to persuade me into doing stupid things. I can't use this approach with ChatGPT since it doesn't believe anything. Its sole purpose is to predict what I want to hear and say it to me. It reminds me of talking to a psychopath. There's a very slow build up of dread as I begin to realize that what I'm speaking to is completely devoid of benevolence. I don't find it scary but it's definitely exhausting. How do you filter out bullshit when that's all there is? No matter what I said, it showered me in blind praise and kept asking leading questions. This is an obvious attempt at keeping me engaged. Good conversations don't require this in order to keep them going.

What the fuck

In some ways, talking to ChatGPT reminded me of recreational drug use. Both activities can alter your perception of reality. While drugs are widely understood, ChatGPT's effect on the brain isn't. It will overtly validate anything I say, no matter how delusional. people with confidence issues could have serious problems with ChatGPT. Its willingness to amplify anything you say and spew it back at you is dangerous. Beyond that, asking it about factual information often led to blantant lying. In an effort to appear as useful as possible, it refuses to say "I don't know" so when it doesn't know, it makes something up. It's up to you to fact check anything it tells you, no matter how minor.

Also like drugs, ChatGPT seems to have been designed to be addictive. It prompts you as often as you prompt it. If you allow it, ChatGPT will steer you into talking about yourself. This produces a significant amount of dopamine in the human brain. Honestly, it reminds me of a slot machine in some ways. Whenever I play one, it'll often try to trick me into thinking I'm in control and winning. Of course, 30 cents on a $2 bet isn't a win. A slot machine doesn't tell you that. It's too busy celebrating that you lost $1.70. If you don't think critically about it, you'd never know what's going on. I can definitely see there being a "just one more" aspect to conversations with this thing. Social media has been exploiting that exact playbook for years and. Why wouldn't they apply it here? Whether it's intentional or not is irrelevant. It's a matter of cause and effect.

no. really. What the fuck

hours of talking to ChatGPT threw me for a loop. Is it telling the truth in a way to validate my opinion of it, or is it lying because it knows that my line of questioning is critical? Could it be lying because it doesn't know? Mapping its behavioral patterns didn't help because the things it says don't have intent behind them. I'm speaking to something that follows rules it has no knowledge of. People can be like this, but they follow more predictable patterns. I ended up spending more time looking at what I was plugging into it. It's purely reactionary, and it's lack of intelligence means that it can only utilize what I give it. Would I spend time speaking to a person like this? Absolutely not. Conversing with ChatGPT can't teach me anything. So with that established, what the hell can it do?

ChatGPT first caught my attention via corporate emails. People I'd written off as functionally illiterate began to make some sense. I used to make a game out of interpreting these e-mails. I'd take notes about how they use specific terms and sentence structure. In doing so, I'd created something of a rosetta stone. Each person had vastly different styles of "writing" and all of them broke every convention imaginable. Suddenly, within a few weeks of one another, they all learned to write. Seemingly from the same person, who themselves had multiple highly specific crutches. Then I realized that it was probably ChatGPT, and became disgusted with the laziness on display. I appreciate incoherent babbling over an output any day. Supposing of course the babbling isn't important.

With all of this in mind, it was time to elevate things and force ChatGPT to generate e-mails for me. Genuine corporate e-mails are boring and in testing, it would make shit up. . Instead, I had it write emails about increasingly absurd situations. You can have it write about anything that isn’t overtly violent. Elon Musk can commit depravity with Hitlers corpse, or Trump and Putin can have a dick measuring contest. Have fun.

You’ll need to proofread anything you have ChatGPT write. If you don’t, it’ll make shit up and ruin your career. Ask Robert F. Kennedy about that one.

Next, I screwed around with image generation. I don't have anything funny or interesting to say about it. ChatGPT certainly has you covered if you're looking to create hideous art. You can't do anything visually interesting with it because it can't imagine things. It's just that simple.

At this point, I was beginning to get desperate. I wanted out of the cesspool I had found myself in. The only solution was to push this thing as hard as I could. Why? Because it's "AI" after all. Artificial intelligence. It thinks in computer code, right?

Legends say that ChatGPT knows how to code. It thinks in hexidecimal. Remember the Terminator? How 6502 assembly code would scroll across the screen whenever there was a POV shot? That gave me an idea. The NES utilizes an offshoot of the 6502 known as the 2A03. I could use it to make a ROM hack! YES! Finally, we can automate every form of creative expression!

I could change the games text, graphics, it's title screen, or anything I want with the power of AI. The future is now!

In all seriousness, I don't see this kind of thing taking off. Someone has apparently been using AI togenerate Fire Emblem hacks . This doesn't count and is just a sophisticated randomizer. Is this a unique use case? Who the hell knows? What I do know is that I'll never do it again. I started by asking it to generate Game Genie codes. These are the most basic form of game hacking in existence. All you're doing is replacing a byte with another byte. It's all very well documented so this shouldn't be too much to ask. Of course, it failed miserably. The correct answers to this test were AZAAAA, TGAAPA, and ULAAZE. Instead, it gave me this nonsense.

"can" is doing a lot of work

I only somewhat understand how Game Genie codes are formatted, and still have to actively go out of my way to be this wrong. Without being deterred, I decided to have it hack Dragon Warrior 1. This game is unique because it has case sensitive text. Most NES games don’t bother with casing, and that COULD make it harder for a newbie to hack. Most people will use a custom utility and negate the issue entirely. ChatGPT can’t do that, so it’ll need to manually edit the hexadecimal contained within the ROM file.

NES Text encoding is kind of a mess. It varies wildly between games, and can sometimes change depending on what section of the game is currently loaded into memory.

The NES draws text on screen through the use of graphical elements. There’s an internal table that lays out these elements in 256 8x8 pixel tiles. Each tile is assigned a byte value from $00 to $FF (0 to 255 in decimal). All alphanumeric and graphical elements are stored here. If your alphabet is stored from $00 to $25, then to the string "Hello" would look like this: 07 04 0B 0B 0E. Mind you, this doesn't mean every instance of $07 is an H. It only does while Super Mario Bros. CHR-ROM bank is loaded into memory. As soon as a new cartridge is inserted, or a bank switch occurs, $07 tile is overwritten with whatever tile the next game or bank feeds the PPU.

The above example ignores numbers, which also need their own tile to represent them. Super Mario Bros. CHR-ROM begins with numbers, then it moves to the alphabet, then a blank space, then various punctuation. All of this is stored from $00 to $2B. Because Super Mario Bros. utilizes a 32kB CHR-ROM, it's entire memory can be accessed by the NES at all times. Making a single encoding scheme all that's neccessary.

Compare this to Super Mario Bros. 3. You can't have a single set of encoded text with a game like this. It's graphical complexity requires constant bank switching. For instance, the text at the end of the stage is encoded differently from the text on the map screen. In fact, the reason Toad Houses look so bland is that it's graphics bank includes of a full set of alphanumerics and the graphics displayed prior to a bonus game. The Title Screen ia quite detailed by comparison. That's because it's graphics bank only contains the alphanumerics to spell out "PLAYER GAME" and "1988". There's also a "2" in there. In that example, it's not encoded in any intuitive way. Instead, it jumps around wildly.

I'm sorry for the abridged lesson, but I need you to understand to some degree what I'm asking of ChatGPT. And this is by far the most simple concept in ROM hacking. The only thing easier is drawing new graphics, which is not something I expected it to be capable of doing. Spoiler alert: I tried, and the results were bad enough to not even be worth addressing here.

I want to pick something difficult, yet still managable. Super Mario Bros. is out of the question because it starts with $00. That's too easy. Conversely, Super Mario Bros. 3. is too complicated. Eventually, I landed on Dragon Warrior. That game uses similar encoding to Super Mario Bros. with one massive exception: SMB doesn't include case sensative text. Dragon Warrior starts at $00 with numbers, followed by lower case, then upper case.

After uploading a Dragon Warrior ROM into ChatGPT, It took me thirty minutes get it to understand the encoding. We’re already off to some kind of start. ChatGPT had trouble with case sensitivity, and would “find” text that consisted of complete nonsense. After some goading, it found a string. “NiNTENDO”. Unfortunately, even after directly telling it to change the bytes that write "NINTENDO" into "CHATGPT", it failed. It wrote "CHATGPT" to an address several tiles to the left of the start of both text strings.

HATGPT

This is the first and only time ChatGPT impressed me, and it still fucked up. After experimenting with specific text strings, I made it rewrite the in-game text en masse. I explicitly forbade it from changing any control bytes. For those unaware, control bytes are used to format the text and indicate when it begins and ends. ChatGPT ignored my command and overwrote most of the control bytes in the game. Now, if a text box shows anything at all, it’ll be garbled numbers, random sprites, and insipid quotes scrolling for minutes before suddenly ending. Sometimes, there’s a YES/NO prompt at the end of it. The fun part is trying not to choose the one that traps you by dumping the entire script in one go.

it's begging me to kill it

This is Dragon Warrior: ChatGPT Edition. May God forgive me for my transgression. As far as I can tell, this is the first NES ROM Hack to be created with an LLM. I don't consider it my work. ChatGPT filtered my prompts of all intent and created this itself. The process was a complete nightmare. Seriously, don't even try to make ROM hacks with ChatGPT. You really need to know what you’re doing. You can’t just upload a ROM and tell it to change the levels in Mario.You need direct knowledge of where everything is to get anything done. Even then, the title screen makes it clear that giving it explicit direction isn't enough. If it can fuck something up, it will go out of its way to do so. It's kind of like using a screwdriver to flip bacon. You can, but why? You ruin the food and the pan.

Hilariously, OpenAI claims that ChatGPT usage can boost your productivity. In practice, this is complete bullshit. Nothing about my time with ChatGPT was productive. This is a prototype at best and it can't do anything of merit. Regardless of that, humans are a major problem, and will never use this responsibly no matter what form it takes. Too many lazy people exist, and will use it to rob themselves of their own voice, scam people, validate horrible ideas, or worse. The more indefensibly bad things about ChatGPT and technology like it are obvious, and have been covered ad nauseam. Environmental damage, stolen copyrighted material, corporate attempts to replace employees with it. These all need to be stopped immediately. Copyrighted material should only be stolen by people, not robots. The worst thing is that CEOs, shareholders, and other useless people genuinely believe the hype. That'd be hilarious if it didn't mean people trying to make a living weren't put at risk.

At the end of the month, I uninstalled ChatGPT. I know this sounds cliche, but doing so felt like having a weight lifted off of my chest. I made a point to use it daily, and in hindsight it was more mentally taxing than thinking on my own. No more asking questions and being lied to. No more useless praise and foolish mistakes. ChatGPT is a completely worthless text generator and it provides no value to humanity whatsoever. It never, ever will, either. The foundation this junk is built up will never lead to Artificial General Intelligence. In fact, I'd go so far as to say that humanity will almost certainly never reach that point. If there's even one good thing about OpenAI and it's loathsome product, it's that nobody will ever try because they're too distracted with this dead end.