Space Regulatory Glossary AI & Data Science

inference

/ˈɪnfərəns/ noun

1. The process of running a trained AI model to produce outputs from new inputs, as opposed to training the model. Inference cost, latency, and throughput are key factors in choosing between API-hosted models and locally deployed ones.
2. Batch inference processes large volumes of data (thousands of regulatory filings) through a model in sequence, while real-time inference handles individual queries on demand. The economics differ significantly: batch workloads favor rented GPUs, while low-latency queries favor API providers.

Related terms

large language model A neural network trained on large text corpora that generates, summarizes, and reasons about natural language. Models… fine-tuning The process of further training a pre-trained AI model on a domain-specific dataset to improve its performance on… multi-model architecture A system design that routes different tasks to different AI models based on each model's strengths, cost, and latency…

Browse all terms in the Space Regulatory Glossary →