…at least this one thing.

In the old 8-bit Microsoft BASICs, the tokenizer routine scans the input line and converts keywords into one or two byte tokens. This crunches the line down so it takes up less memory, and speeds up execution since the detokenizer just has to grab a token then run the related function. If BASIC were stored as ASCII text, it would have to scan the line text and parse multiple bytes to figure out it is a “PRINT” versus just looking at a one byte token.

In Color BASIC on the Radio Shack Color Computer, the tokenizer needs a space between variable names and keywords so it can tell where the variable stops and the keyword begins. For example:

IFSC=100THEN500

That line works fine, since “SC” (a score variable in this case) is followed by an equal sign. BASIC can find it easily. But if you were comparing against another variable:

IFSC>HSTHEN500

…you would get an ?SN ERROR on that line. I believe this is because Color BASIC “supports” longer variable names, but only honors the first two characters. Thus, these are all the same variable:

LO=1
LONG=1
LONGER=1
LONGEST=1

They are recognized as LO In the “IFSC>HSTHEN500” example, how is BASIC to know that we are using “HS” versus “HST” or even “HSTHEN” as a variable? It must be scanning until it hits something that is clearly not part of a variable name.

To make things even more confusing — variable names begin with a letter and can end with a number (or numbers, that are ignored), you can make variable names like this:

HSTHEN500=10
OK

PRINT HSTHEN500
10

And with that understanding, of course a space is required after variables before the following token. You have to add one between the variable and the next keyword:

IFSC>HS THEN500

BASIC can easily figure out where SC starts since a variable cannot contain a “>”, but it needs the space to know where HS ends. Thus, a FOR/NEXT loop like this requires spaces.

I fired up the “work in progress” Tandy CoCo engine in the cool Clock Signal emulator to test it out.

And then VIC-20 enters the arena…

But my VIC-20 did not need the space. Its parser can figure this out. Once again, I used the Clock Signal emulator which also has VIC-20.

When I was relearning VIC-20 a few years ago, I ran into this difference and wondered why. As I began revisiting it recently, I thought maybe the VIC-20 does not allow longer variable names, and has a better way to tell where a variable ends? I tried to use “START=1” but got an error. “Ah, it must not!” But then I found “ST=1” also did not work, and assumed “ST” must be some kind of reserved keyword.

Oddly, “AAAAA=1” works, and shows the same value as “AABBB=5”. For that test, it works like Color BASIC.

But “LO=1” works while “LON=1” does not work. Is LON some keyword too??? I clearly never learned all the keywords when I had my VIC.

SUPERMAN=1
SUPERGIRL=2

PRINT SU
2

I guess the VIC-20 BASIC does work like Color BASIC, but with some different keywords making the VIC-20 list of forbidden variables different than the CoCo’s list of forbidden variables. (See that link — I’ve now written about this at least three times, before this post.)

When I had my VIC-20, I did not know Microsoft had created the BASIC it uses. It just says “CBM BASIC” so I’d always thought (back then) that Commodore Business Machines wrote their own BASIC. It wasn’t until the modern Internet that I learned Microsoft wrote the 6502 BASIC used by Commodore. (If you have never dug into this, check it out sometime. There is some interesting history between Microsoft and Commodore. Apparently, that’s what led Microsoft into placing hidden “MICROSOFT” easter eggs in their other BASICs to be able to prove it was their code. But I digress…)

I find this interesting. I do not know 6502 assembly, but I am tempted to try to find the VIC-20 equivalent of “Color BASIC Unravelled” and see if I can learn how the Microsoft 6502 BASIC parser works compared to their 6809 parser.

But hopefully one of you knows, and can tell us all in the comments.

Until then…