Augmenter that apply operation (sentence level) to textual input based on abstractive summarization.
AbstSummAug(model_path='t5-base', tokenizer_path='t5-base', min_length=20, max_length=50, batch_size=32, temperature=1.0, top_k=50, top_p=0.9, name='AbstSumm_Aug', device='cpu', force_reload=False, verbose=0, use_custom_api=True)¶
Augmenter that leverage contextual word embeddings to find top n similar word for augmentation.
- model_path (str) – Model name or model path. It used transformers to load the model. Tested ‘facebook/bart-large-cnn’, t5-small’, ‘t5-base’ and ‘t5-large’. For models, you can visit https://huggingface.co/models?filter=summarization
- batch_size (int) – Batch size.
- min_length (int) – The min length of output text.
- max_length (int) – The max length of output text.
- temperature (float) – The value used to module the next token probabilities.
- top_k (int) – The number of highest probability vocabulary tokens to keep for top-k-filtering.
- top_p (float) – If set to float < 1, only the most probable tokens with probabilities that add up to top_p or higher are kept for generation.
- device (str) – Default value is CPU. If value is CPU, it uses CPU for processing. If value is CUDA, it uses GPU for processing. Possible values include ‘cuda’ and ‘cpu’. (May able to use other options)
- force_reload (bool) – Force reload the contextual word embeddings model to memory when initialize the class. Default value is False and suggesting to keep it as False if performance is the consideration.
- name (str) – Name of this augmenter
>>> import nlpaug.augmenter.sentence as nas >>> aug = nas.AbstSummAug()
augment(data, n=1, num_thread=1)¶
- data (object/list) – Data for augmentation. It can be list of data (e.g. list of string or numpy) or single element (e.g. string or numpy). Numpy format only supports audio or spectrogram data. For text data, only support string or list of string.
- n (int) – Default is 1. Number of unique augmented output. Will be force to 1 if input is list of data
- num_thread (int) – Number of thread for data augmentation. Use this option when you are using CPU and n is larger than 1
>>> augmented_data = aug.augment(data)