Skip to main content

Transformers

Transformers is a popular AI framework, and we have incorporated native support for Transformers to provide essential Large Language Model (LLM) capabilities. superduperdb allows users to work with arbitrary transformers pipelines, with custom input/ output data-types.

Supported Model types​

TextClassification​

...

LLM​

You can quickly utilize LLM capabilities using the following Python function:

from superduperdb.ext.transformers import LLM
llm = LLM(model_name_or_path="facebook/opt-350m")
llm.predict_one("What are we having for dinner?")

Or use a method similar to transformers’ from_pretrained, just need to supplement the identifier parameter.

from superduperdb.ext.transformers import LLM
llm = LLM.from_pretrained(
    "facebook/opt-350m", 
    load_in_8bit=True, 
    device_map="cuda", 
    identifier="llm",
)

The model can be configured with the following parameters:

  • adapter_id: Add an adapter to the base model for inference.
  • model_kwargs: a dictionary; all the model_kwargs will be passed to transformers.AutoModelForCausalLM.from_pretrained. You can provide parameters such as trust_remote_code=True.
  • tokenizer_kwargs: a dictionary; all the tokenizer_kwargs will be passed to transformers.AutoTokenizer.from_pretrained.

Training​

LLM fine-tuning​

SuperDuperDB provides a convenient fine-tuning method based on the trl framework to help you train data in the database.

You can find training examples here.

Running Training

Supported Features​

Training Methods:

  • Full fine-tuning
  • LoRA fine-tuning

Parallel Training:

Parallel training is supported using Ray, with data parallelism as the default strategy. You can also pass DeepSpeed parameters to configure parallelism through DeepSpeed configuration.

  • Multi-GPUs fine-tuning
  • Multi-nodes fine-tuning

Training on Ray:

We can use Ray to train models. When using Ray as the compute backend, tasks will automatically run in Ray and the program will no longer be blocked.