1 Representing Text
Computer at the base level represent everything at bits (0s or 1s). The common unit of a byte is 8 bits. Since each bit can be either 0 or 1, 8 bits can represent 2^8 (256) possible values.
We typically represent text on computers in one of a few formats :
- ASCII
- Unicode
- UTF8
If we used these as our vocabulary for our model, that would mean we need no transformation of the text.