Microsoft.ML.Tokenizers 0.22.0-preview.24162.2

Microsoft.ML.Tokenizers contains the implmentation of the tokenization used in the NLP transforms.

No packages depend on Microsoft.ML.Tokenizers.

https://aka.ms/mlnetreleasenotes

.NET 8.0

.NET Standard 2.0

Version Downloads Last updated
2.0.0-preview.1.25127.4 7 07/10/2025
2.0.0-preview.1.25125.4 6 07/10/2025
1.0.2 6 07/10/2025
1.0.1 6 07/10/2025
1.0.0 6 07/10/2025
0.22.0 6 07/10/2025
0.22.0-preview.24526.1 6 07/10/2025
0.22.0-preview.24522.7 6 07/10/2025
0.22.0-preview.24378.1 6 07/09/2025
0.22.0-preview.24271.1 7 07/10/2025
0.22.0-preview.24179.1 6 07/10/2025
0.22.0-preview.24162.2 6 07/10/2025
0.21.1 6 07/10/2025
0.21.0 6 07/10/2025
0.21.0-preview.23511.1 6 07/10/2025
0.21.0-preview.23266.6 6 07/10/2025
0.21.0-preview.22621.2 6 07/10/2025
0.20.1 7 07/10/2025
0.20.1-preview.22573.9 6 07/10/2025
0.20.0 6 07/10/2025
0.20.0-preview.22551.1 6 07/10/2025