[roastedcabbage]

Recently nvidia has released a new library called optimum-NVIDIA that will boost inference performance up to 28x to the baseline.
You have to replace only one line of code and you are good to go. Thought i would share if anyone missed this.

Check out the repo here - https://github.com/huggingface/optimum-nvidia

[adjective]

Which line of code do you have to change?

[roastedcabbage]

If you are using transformers library from huggingface you have to replace that with optimum.nvidia

for example if you have the following code:

from transformers.pipelines import pipeline

you have to replace the above as below

from optimum.nvidia.pipelines import pipeline

and rest remains the same.