The title refers to an ongoing series with two videos so far. Part 1 (28 minutes) is about how the tokenizer works and how programs are stored in memory, and Part 2 (19 minutes) is on what the BASIC interpreter does to execute the program as it’s running. These are pretty detailed and a bit technical, but interesting of these kinds of shapes can fit into your brain. More below.
Some extra links from the videos, a disassembly of the C64 BASIC ROM from pagetable.com, and the open-sourced by Microsoft code of their generalized BASIC interpreter. It should be noted that the original version of Microsoft BASIC were written for the Altair 8800 kit computer by Bill Gates and Paul Allen themselves.
I’ll distill the basics of the videos for you. Commodore BASIC is a dialect of Microsoft BASIC. It puts the loaded program at the bottom of memory, starting at 2049, or $0801 in hex. (The custom on the C64 is to denote hex numbers with a dollar sign.) When lines are written, the C64’s screen editor start from the beginning looking for a line number, and if it’s found tries to tokenize the rest of the line. Unrecognized keywords are still stored as PETSCII characters (which will cause a syntax error when run).
Commands are matched against a list of keywords stored in ROM. All the keywords are stored as “tokens,” single bytes, to conserve program space and to help with execution speed. At runtime, the tokens are detected by their high bit being set, then run through a lookup table. It pushes the high and low bytes of the command’s address in memory then does an RTS, which then jumps to the routine by acting like it’s returning from a JSR. Notably, the short routine that gets the next byte of BASIC text to act upon, CHRGET, is actually copied into zero page RAM to run, to take advantage of faster zero page versions of some instructions, but also because it’s self-modifying.
