NemoCurator Domain Classifier Model Overview This is a text classification model to classify documents into one of 26 domain classes: Model Architecture The model architecture is Deberta V3 Base Context length is 512 tokens Training Details Training data: 1 million Common Crawl samples, labeled using Google Cloud’s Natural Language API: https://cloud.google.com/natural language/docs/classifying text 500k Wikepedia articles, curated using Wikipedia API: https://pypi.org/project/Wikipedia API/ Training steps: Model was trained in multiple rounds using Wikipedia and Common Crawl data, labeled by a combination of pseudo labels and Google Cloud API. How To Use This Model Input The model takes one or several paragraphs of text as input. Example input: Output The model outputs one of the 26 domain classes as the predicted domain for each input sample. Example output: How to Use in NVIDIA NeMo Curator The inference code is available on NeMo Curator's GitHub repository. Check out this example notebook to get started. How to Use in Transformers To use the domain classifier, use the following code: Evaluation Benchmarks Evaluation Metric: PR AUC PR AUC score on evaluation set with 105k sample…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy