8 is bit 4, 3 is bits 1 and 2, 128 is bit 8
You're off by one in bit number.
Lowest bit is bit number 0, which is 1 or 0x01, which is also 2^0, or 1 << 0.
By starting bit numbers by 0, the power, the shift, and the number of zeroes right of the 1 (ie the zeroes shifted in) all match.
dec = hex = shift = power = bits
1 = 0x01 = 1 << 0 = 2^0 = 0001
2 = 0x02 = 1 << 1 = 2^1 = 0010
4 = 0x04 = 1 << 2 = 2^2 = 0100
8 = 0x08 = 1 << 3 = 2^3 = 1000
If you start counting at 1, you get "1 << (1-1)", and "2^(1-1)" in the first line which is more cumbersome to get right all the time.
Edit:
As others have noted, hex is mostly useful if you are interested in bits than the numeric value.
In fact, I read hex numbers always as bit patterns, and not as hex number. For example, "0xDA" I don't read as hex "D" and "A", I read it as bit patterns "1101" and "1010", I see the bit patterns expressed by the hex digits.