lol not wizz kids, those would just use a smartphone + google to convert nowadays. rather old fart oldschool computer freaks. the kind of who invented trackers. it is just the convenience of having bytes within 2 symbols, remember how small computer screens used to be, and also for being able to easily seperate bytes of part of them from each other. is a bit closer to how computers work.
it is not such magic, you really just need to half-way know what full/half/quarter/etc range values in hex are. I.e. think of the values as kind of percentages along a scale 0%-100%. And from there how to count a bit along the scale for finder variation. After that you’ll be able to use it in trackers. no need to calculate with them in head etc, even nerds use a computer for this nowadays.
and the 256 vs 255 confusion stems from the fact, that you would sometimes think about a number, sometimes a count of different possibilities. obviously a byte has 256 different numbers it is able to represent, and within this count the zero is counted as a possible state, too. But you can only represent numbers from 0-255 with it, this is the standard way of using bytes for numbers. like in decimals, you would use for one digit 0…9 actually being 10 different numbers representable.