v1.1.8 Time Signature tokens for Octuple
Changes
- 3c3c5da
TimeSignature
tokens are implemented for Octuple ! Thanks @ilya16 for this great contribution ! These tokens are optional and can be set with theadditional_tokens
parameter. - df1edd1 Added a fail-check for Bar / Pos based tokenizations, for when a token sequence begins by a
Position
tokens before anyBar
. - 5ab55f4 Bugfix when loading tokenizer params from config file with tempos.
- 08540a2
SOS
andEOS
tokens are not adresses to -1 and -2 anymore as this could led to issues.
Compatibility
SOS
and EOS
tokens saved with v1.1.7 and before will not be compatible anymore.
You can however easily convert them. You just have to convert SOS
(-1) and EOS
(-2) tokens to respectively len(tokenizer.vocab)
and len(tokenizer.vocab) + 1
.