whisper server elixir

whisper_server/deps/tokenizers/native/ex_tokenizers/README.md

@@ -0,0 +1,20 @@
+# NIF for Tokenizers.Native
+
+## To build the NIF module:
+
+- Your NIF will now build along with your project.
+
+## To load the NIF:
+
+```elixir
+defmodule Native do
+    use Rustler, otp_app: :tokenizers, crate: "ex_tokenizers"
+
+    # When your NIF is loaded, it will override this function.
+    def add(_a, _b), do: :erlang.nif_error(:nif_not_loaded)
+end
+```
+
+## Examples
+
+[This](https://github.com/hansihe/NifIo) is a complete example of a NIF written in Rust.