Prem 1B SQL (HuggingFace) Read the blogpost here PremSQL Library GitHub Prem 1B SQL is one of the very first series of fully local Text to SQL models developed by Prem AI. Being a 1B parameter model it easily fits on low GPU devices (and CPU devices when quantized). We believe that AI assisted data analysis should be a Local first approach. Because exposing Databases to third party closed source models can lead to data security breaches. We will be publishing some of the public benchmark results of this model very soon. We will also be iterating on this model for more better results. Developed by: Prem AI License: [MIT] Results We evaluated our model on two popular benchmark datasets: BirdBench and Spider. BirdBench consists of a public validation dataset (with 1534 data points) and a private test dataset. Spider comes up with only a public validation dataset. Here are the results: Dataset Execution Accuracy BirdBench (validation) 46% BirdBench (private test) 51.54% Spider 85% The BirdBench dataset is distributed across different difficulty levels. Here is a detailed view of the private results across different difficulty levels. Difficulty Count EX Soft F1 Simple 949 60.70 61.48 M…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy