Nov 16, 2020
Nice article !
Actually, CharacterBERT can be very useful in many other domains, as Text to Code issues.
However, I think you miss an interessting comparison with BPE technique, which is also a technique to avoid vocabulary dependencies and use a character embeding for tokens.