Allow neural networks to independently determine which data they need to study. On the other hand, it is a fundamental software object for managing databases, upon which the entire infrastructure of modern AI solutions is built.
Query Set in the Active Learning Paradigm
The traditional approach to machine learning requires a pre-prepared array of labeled data. However, in real-world tasks, labeling is extremely expensive: it requires the time of specialized professionals, such as radiologists, or the execution of complex laboratory tests. Active learning solves this problem through an iterative process where the algorithm itself dictates exactly what it needs to be shown.
In this system, data is divided into three categories:
Labeled Set (L): verified examples on which the model has already been trained.
Unlabeled Set (U): a mass of «raw,» unlabeled data.
Query Set (Q): a carefully selected batch of data from U, which the algorithm requests from an expert (oracle) at the current step.
The task of the Query Set here is not merely to provide random data, but to highlight the most informative examples for the model, which will yield the maximum accuracy gain at the minimum labeling cost.
Methods for selecting data into the Query Set:
Search for the zone of maximum uncertainty: The algorithm selects objects from U where its confidence in classification is minimal (for example, the probabilities of two classes are 0.501 and 0.499). This is data lying right on the decision boundary.
Committee voting (Query-by-Committee): Several independent models are launched. Examples where their predictions diverged most radically are sent to Q.
Impact assessment on weights: Objects are selected that, when added to training, will cause the greatest restructuring of the neural network’s parameters.
Search by representativeness: The sample is formed to cover different areas of the data distribution, excluding the duplication of overly similar examples.
Practical case study: In Russian projects developing computer vision systems for medicine (for example, for analyzing CT scans), manual annotation of each slice takes hours of a doctor’s work. Instead of asking medics to annotate an array of 50,000 scans, the system trains on the first thousand. It then forms a Query Set of 20 scans where the algorithm sees anomalous but non-obvious opacities. The specialist confirms or refutes the pathology only on these 20 frames. The model is fine-tuned, and in the next cycle, its «doubtful zones» become smaller. This approach allows achieving industrial-grade model quality while saving up to 80–90% of the labeling budget.
Query Set as a Data Management Tool in the AI Stack
No neural network operates in a vacuum — it needs data, which is most often stored in relational databases. In the Python ecosystem (and especially in the Django framework), the primary tool for working with data is the QuerySet object. For a Machine Learning Engineer (MLE), understanding its mechanics is critically important when building preprocessing pipelines.
A QuerySet is not the data array itself, but its declarative description. The object records filtering, sorting, and table joining conditions, but does not access the disk until the data is required for computations.
Specifics of working with QuerySet in AI infrastructure:
Lazy Evaluation principle: You can build a chain of filters of any complexity. The database access will occur only at the moment you start iterating over the object in a loop or save it to a list. This allows dynamically collecting samples for augmentation without placing an excessive load on the server.
Combating the N+1 query problem: Data for AI is often normalized (tables Image, Label, User).
The select_related() method pulls related entities via SQL JOIN (efficient for single relationships).
The prefetch_related() method makes separate queries to related tables and merges them in memory (necessary for multiple relationships, for example, when dozens of objects need to be found in a single frame for detection).
Memory and cache management: After the first full read, the QuerySet saves the result in RAM. However, working with slices (e.g., qs[0]) is not cached — each such call generates a new SQL query with LIMIT 1, which can become a fatal performance error in high-load platforms for crowdsourced labeling.
Direct export for training: The .values() and .values_list() methods allow obtaining clean dictionaries or tuples, bypassing the creation of heavyweight model objects. This is the de facto standard for preparing jsonl or csv files for feeding them into a DataLoader.
From SQL to Gradient Descent
In modern production systems, these two definitions of Query Set merge into a single technological process.
A typical workflow cycle for an MLOps platform looks like this:
The data collection service forms a Query Set in the DB, selecting millions of unlabeled objects based on complex conditions (e.g., uploaded_at__gte=yesterday and device_type=’thermal’).
The data is lazily read from the DBMS, undergoes primary validation, and is fed into an already trained model.
The neural network calculates entropy or an uncertainty metric for each example.
The active learning algorithm forms a Query Set (Q) in the mathematical sense — a list of identifiers of the most useful frames for training.
The backend service accesses the database again, using filter(id__in=q_ids), to retrieve links to these files and send them to RabbitMQ or Kafka for subsequent manual labeling.
Consequently, a deep understanding of how the Query Set is structured at the database level — from SQL compilation to caching nuances — is as mandatory a competency for an AI systems developer as knowledge of Transformer architectures or gradient descent optimization methods.
Contact our experts to select the optimal hardware configuration and calculate rental pricing for your AI workloads.